- The open.amsterdam project is using Zimbra as their mailserver. Manou Chen and Stephan Wildeboer both where in Curacao and talked lovingly about it. I was going to install Scalix, but it was too difficult, lack of good support for an Ubuntu/Debian installer made me loose a lot of time. Zarafa also lost me somewhere during installation. Gawd I wished Canonical came with a full fledged Zimbra/Scalix/Zarafa like server in their standard distribution. But I don't want to wait for that. So I took to the task of converting my mail from Kmail - which has served me very well just like Pegasus Mail in the many years before that... except Pegasus Mail didn't move to Open Source and wasn't available for Linux... and Kmail makes it impossible to reindex folders save from delteing the .index* files...
- Converting from Kmail's weird maildir implementation to one readable by courier-imap.
It seems the best way to move your mail from some MUA to Zimbra is using imapsync (see for documentation here) but very soon I discovered that the Kmail way of storing messages in a 'maildir' is quite different from what you'd expect. I love maildir - qmail-ldap works with it and it's very handy for backups via rsync. And you can grep efficiently in a maildir; it's easy to weed out some spam or locate a specific mail with standard unix tools.
It's one of the reasons I never wanted to use Evolution or Thunderbird. Lack of support for maildir for local storage is a no-go for an MUA.
I always tought Kmail used maildir format, but it's using a slightly different maildir format, apparently. I needed to write a small script that fixes that. Here it is. PLEASE make sure you know what you do and make backups of your mail before you use it. The script is far from perfect. If you named your folder 'somehting.index' for some reason, it's gets ignored and if you named your folder 'something.directory' you're in trouble too, and dots in mailbox names are being handled badly too. But these problems are all easy to avoid and the script is easy to improve with some checks and balances. Works for me as is, your mileage may vary...
#!/bin/bash
# suares & co 2008, you may use this scipt at your own risk.
# converts a kmail mail directory to a maildir structure that
# can be read by an IMAP server
# all this as a preparation to migrate to zimbra.
# set the field seperator to 'newline' so we can handle
# spaces in folder names
IFS='
'
# usage: program kmaildir maildir
# set kmaildir
KMAILDIR=$1
echo "KMAILDIR = $KMAILDIR ..."
if [ ! -d $KMAILDIR ];
then
echo "KMAILDIR DOES NOT EXIST. EXIT"
exit 1
fi
# set maildir, it will be created if needed
# if it already exists... hmm... might overwrite stuff...
MAILDIR=$2
echo "MAILDIR = $MAILDIR ..."
if [ ! -d $MAILDIR ]
then
mkdir -p $MAILDIR
echo "MAILDIR DOES NOT EXIST. CREATED"
fi
# function doIt does either
# - skip a file if it has '.index' in it's name (kmail indexes)
# - dive into a directory (by calling doIt recursively)
# if it has '.directory' in it's name
# - move the folder to the new maildir directory
doIt () {
echo "The Path is $1 ..."
for i in `ls -1A`
do
# check for index files
echo "Checking $i ..."
j=`echo $i | sed "s/\.index//g"`
if [ "$i" == "$j" ]
then
# check for directory
j=`echo $i | sed "s/\.directory//g"`
if [ "$i" == "$j" ]
then
# this is a plain folder and needs to be moved.
# (actually, it might be some totally unrelated file... damn.)
j=`echo "$i" | sed "s/ /_/g"` # remove spaces
j=`echo "$j" | sed "s/\./-/g"` # remove dots
mv "$i" "$MAILDIR/$1.$j"
echo " $i moved to $MAILDIR/$1.$j ..."
else
# this is a directory and we need to enter it.
# (it could be a directory. Let's hope so.)
j=`echo "$j" | sed "s/ /_/g"`
echo " Entering $i ..."
cd "$i"
doIt "$1$j"
echo " Leaving $i ..."
cd ..
fi
else
# echo "Skipping index file $i ..."
# rm "$i"
q=1 # this is here because bash complained about an empty else...
fi
done
}
cd $KMAILDIR
pwd
doIt
You can invoke the script in your homedir by telling your friend to do 'scriptname /home/you/Mail /home/you/Maildir' but your script might not be called 'scriptname' and you mail might not be stored in ~/Mail and your ~/Maildir might already exist for other reasons... please sort these things out before you start doing things that will obliterate your 3 GB of mail since 1992...
- Time to setup courier-imap
sudo apt-get install courier-imap impasync
Since my zimbra install was running on the same machine as courier-imap, I changed the port to 999. I am sure you can do that too, somewhere in /etc/courier/imapd.
There is various ways to test if it worked. I for one, welcome configuring kmail to use an extra IMAP account that points to localhost:999 but there must be more clever ways. Or not.
- Syncing with Zimbra
There is ample documentation about this on the page I mentioned earlier. You need Date::Manip (I installed it via cpan).
What worked for me was this:
imapsync --buffersize 8192000 --nosyncacls --subscribe \
--syncinternaldates --host1 127.0.0.1 --port1 999 --user1 ace \
--password1 Ahl0wGOOGLEiSUXzaseipeicaiThaez \
--host2 10.20.40.80 --user2 ace@mysuares.an.domain \
--password2 shae6ORxDONTaTHEYich8OhphoquodohRi \
--noauthmd5 --ssl2
Of course, you should have both ip addresses be 127.0.0.1 but this was after I moved the whole ZCS to another dorky machine. Zimbra was too heavy on my desktop (E6550 with 4GB) and Sauerbraten suffered from it. The 10.20.40.80 is an E6600 with 1 GB... seems to run well. Oh and if you already have seperate machines the courier-imap can be on port 143... yeah? The --noauthmd5 and --ssl2 where needed for zimbra. The truth must be out there... somewhere. I just briddled till it worked.
It's not funny tough, this imapsync business.
Time : 25573 sec (arr... 7 harrs!)
Messages transferred : 184194
Messages skipped : 0
Total bytes transferred: 3259235882
Total bytes skipped : 0
Total bytes error : 174875341
Detected 13 errors
...and what are these THIRTEEN errors... on the 13th of this month...! Am I missing 174 MB of mail?
- Kaddressbook
The 'normal' way of doing an import of kaddressbook is to export to CSV and then import in Zimbra. Didn't work at all. I had 44 of the 389 contacts and they where all nonames. Removing the addresses is easy, just select all and delete. I exported kaddressbook to one large vcf (3.0) file and following the same documentation as mentioned above, I came up with this:
sudo apt-get install curl
curl -u ace@mysuares.an.domain:shae6ORxDONTaTHEYich8OhphoquodohRi \
--data-binary @/wherever/20081013addressbook.vcf http://10.20.40.80/service/home/ace@mysuares.an.domain/contacts?fmt=vcf
Yes, that 'schemers' in the docs is your account...
- The Calender
This worked by just importing the .ics file from korganizer. You need to find out where it is; I had renamed it several times trying to get it to work reliably - which it didn't - and first imported the wrong one. The docs show you a way to remove all those wrongly imported appointments. Hint: google zmprov emptyfolder... Second link. And read trough to the end.
- Annoyances so far
• The inability to apply filters to existing mail. See this bug might enlighten you. Someone even offered money to fix it.
• Can't sort on 'from'? I click and I click but only 'received' and 'subject' are sortable... Untill I discovered the 'View' dropdown. Sort 'by message' instead of 'by conversation' and now the 'from' field is sortable. But 'size' still is not.
• No feedback on progress of External Mail download... I got a POP3 account configured, and when I check it with 'Get External Mail', my mouseponter becomes a rotating disc and the browser is blocked (can't read mail in that window, or use scroll bars, etc).
Aftger a while I get a greenish popup saying that my POP3 account has been 'loaded'. Yeah right, what's that mean? And that's only when I don't have to much mail to download - because when the download takes to long, a popup is saying that the server doesnt' respond and do I want to cancel the request? Don't cancel, it's all working like it should, but the feedback is missing...
• The preview pane can not be relocated. In Kmail, I had the preview pane to the right of the message list. To the left of the message list was the folder list. This setup works very well for wide screens. Since I have a 3-head behemoth, I stretched Kmail over 2 screens, so the folder and messages list was on one screen and the preview pane took up it's own screen. That was very convenient. But in Zimbra, the preview pane can not be relocated; it sits beneath the messages list and that could work well if you had two screens vertically stacked up - but not horizontally. Another annoyance that I hope gets fixed...
• When Firefox crashes (and it does that too often), you loose the mail you where composing. In Preferences, you can set a regular backup of composed mail, but you can't tell how often. It seems it backups at irregular intervals, and you can't set the interval. I just lost large part of a complicated mail that way...
• The paginator is annoying too. It indicates '1 - 100' but doesn't say 'of 1289' so you can't really see if there's a lot of messages in a mailbox.
• When using filters, mail is automatically tucked away in some subfolders. But the main folder is not highlighted when there are new messages in a subfolder. So if I move postfix-users mail to Mailinglists/Postfix, then the Postfix folder is highlighted after the filter puts some new mail in there, but the Mailinglist folder is not. Kmail did that and it made you aware of new mail in subfolders. My oh my.
- Server status data is not available. To see the server status, logger service must be installed.
If you see this message and you are on Ubuntu, you probably have syslog-ng installed instead of the old rusty syslog...
ps auxwww|grep sys
If ps reports syslog-ng, you might just want to install the old syslog instead, or modify syslog-ng configuration... (how?). If /var/log/zimbra.log is empty, it is almost always a sign that you have the 'wrong' syslog.
- Swatch/Actions.pm, and other CPAN woes, like Zimbra/Util missing and 'logger stopped' and amavisd not working...
Please, if you ever move the entire /opt/zimbra directory to another disk or partition, don't do it like this:
rsync -a /opt/zimbra/* /newopt/zimbra
...because the hidden files in ~zimbra, which is /opt/zimbra won't get copied over. (Think of .bashrc and .ssh and so on). This leads to many ha-haa-funny error messages like 'can't find zmcontrol' and all the missing CPAN modules. Better do it like so:
rsync -a /opt/zimbra /newopt
My, do I feel stupid after 4 hrs debugging and installing cpan modules...
- NO [TOOBIG] request too long with imapsync
If you are using imapsync to copy messages from another IMAP server to Zimbra, sometimes you find errors in the summary. It's not very clear what those errors are, but if you check the output from imapsync, you might encounter:
NO [TOOBIG] request too long
This happens becuase the maximum file upload sizes and maximum mail message size are about 10 MB in a default install. You can fix this with:
sudo - zimbra -c zmprov mcf zimbraFileUploadMaxSize 50000000 #50 MB
sudo - zimbra -c zmprov mcf zimbraMtaMaxMessageSize 60000000 #60 MB
/etc/init.d/zimbra restart # don't forget to restart!
Set the values according to what you think is reasonable.