Projet

Général

Profil

Historique

HowTo Mail Backup - Ikujam » Historique » Version 3

« Précédent - Version 3/19 (diff) - Suivant » - Version actuelle
iku jam, 29/01/2012 21:45

Presentation

several projects with mail servers
request of certain stability, needed documentation
free software user, activist and contributor
idea is to produce a complete test environment with vms on a single machine

CC-NC-SA

Requirements

to follow you need some linux admin skills:

  • basic shell (bash)
  • at least basic knowledge of debian package system (install & setup packages with apt-get, manage services)
  • able to setup ssh public key authentication
  • i don't like nano, feel free to use it - or another editor - instead of vi

Host system

  • debian
  • qemu-kvm
  • bind

This howto uses

  1. cat /etc/debian_version
    wheezy/sid
  2. uname -a
    Linux master 3.1.0-1-amd64 #1 SMP Sun Dec 11 20:36:41 UTC 2011 x86_64 GNU/Linux

Mail Server VMs

  • debian
  • debian packages for the different software

    root@mail1:~# echo "mail1" > /etc/hostname
    root@mail1:~# apt-get install inotify-tools rsync openssh-server pgpool javascript-common apache2 libapache2-mod-php5 roundcube postgresql postfix postfix-pgsql mailman roundcube-pgsql libc-client2007e mlock php5-imap postgrey courier-authlib-postgresql sasl2-bin courier-authdaemon libsasl2-modules-sql courier-imap-ssl --no-install-recommends

  • use default options for roundcube, courier & mailman for now
    • ident authentication
    • dbconfig
    • pgsql as database choice
    • mailman language as you prefer
  • install postfixadmin :

    root@mail1:~# lynx 'http://downloads.sourceforge.net/project/postfixadmin/postfixadmin/postfixadmin-2.3.4/postfixadmin_2.3.4_all.deb'

  • use default options for now
  • just as of personal habit, some tools i use

    root@mail1:~# apt-get install lynx less mc vim

    root@mail1:~# cat /etc/debian_version
    wheezy/sid
    root@mail1:~# uname -a
    Linux mail1.test 3.1.0-1-amd64 #1 SMP Tue Jan 10 05:01:58 UTC 2012 x86_64 GNU/Linux

    root@mail2:~# cat /etc/debian_version
    wheezy/sid
    root@mail2:~# uname -a
    Linux mail2 3.1.0-1-amd64 #1 SMP Fri Dec 23 16:37:11 UTC 2011 x86_64 GNU/Linux

root@mail2:~# cat /etc/network/interfaces
  1. This file describes the network interfaces available on your system
  2. and how to activate them. For more information, see interfaces(5).
  1. The loopback network interface
    auto lo
    iface lo inet loopback
  1. The primary network interface
    allow-hotplug eth0
    iface eth0 inet static
    address 192.168.122.3
    netmask 255.255.255.0
    network 192.168.122.0
    broadcast 192.168.122.255
    gateway 192.168.122.1

dns setup on host

root@quadebian:/etc/bind# cat db.192.168.122
;
; BIND reverse data file for test
;
$TTL 604800
IN SOA master.test. root.master.test. (
1 ; Serial
604800 ; Refresh
86400 ; Retry
2419200 ; Expire
604800 ) ; Negative Cache TTL
;
 IN NS master.test.
1 IN PTR master.test.
2 IN PTR mail1.test.
3 IN PTR mail2.test.

root@quadebian:/etc/bind# cat db.test
;
; BIND data file for test
;
$TTL 604800
IN SOA master.test. info.master.test. (
2 ; Serial
604800 ; Refresh
86400 ; Retry
2419200 ; Expire
604800 ) ; Negative Cache TTL
;
 IN NS master.test.
test. IN MX 10 mail1.test.
test. IN MX 20 mail2.test.

master IN A 192.168.122.1
mail1 IN A 192.168.122.2
mail2 IN A 192.168.122.3

root@quadebian:/etc/bind# named-checkzone test db.test
zone test/IN: loaded serial 2
OK

  • pass kvm dns server in forward mode on host node (default net config)

root@quadebian:/etc/bind# virsh
Welcome to virsh, the virtualization interactive terminal.

Type: 'help' for help with commands
'quit' to quit

virsh # net-dumpxml default
<network>
<name>default</name>
<uuid>0529cc34-c2ad-9663-0f42-5b338b14a6e4</uuid>
<forward mode='nat'/>
<bridge name='virbr0' stp='on' delay='0' />
<mac address='52:54:00:37:85:D8'/>
<ip address='192.168.122.1' netmask='255.255.255.0'>
<dhcp>
<range start='192.168.122.2' end='192.168.122.254' />
</dhcp>
</ip>
</network>

vm dns config

  • change requires to reaffect NICs via virt-manager
    • remove nic (and /etc/udev/rules.d/70-persistent-net.rules - it keeps track of different nics on the system, avoids getting eth2/3/4...)
    • create new nic on default network
    • reboot vm
    • test connectivity & bind (set nameserver to 192.168.122.1 in /etc/resolv.conf)

tests to do

root@quadebian:/etc/bind# dig mx test

; <<>> DiG 9.7.3 <<>> mx test
;; global options: +cmd
;; Got answer:
;; >>HEADER<< opcode: QUERY, status: NOERROR, id: 26405
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 1, ADDITIONAL: 3

;; QUESTION SECTION:
;test. IN MX

;; ANSWER SECTION:
test. 604800 IN MX 20 mail2.test.
test. 604800 IN MX 10 mail1.test.

;; AUTHORITY SECTION:
test. 604800 IN NS master.test.

;; ADDITIONAL SECTION:
mail1.test. 604800 IN A 192.168.122.2
mail2.test. 604800 IN A 192.168.122.3
master.test. 604800 IN A 192.168.122.1

;; Query time: 2 msec
;; SERVER: 10.11.12.126#53(10.11.12.126)
;; WHEN: Tue Jan 24 09:55:25 2012
;; MSG SIZE rcvd: 135

Server configuration

postfix

root@mail2:/etc/postfix# mv main.cf main.cf.debian
root@mail2:/etc/postfix# vi main.cf
root@mail2:/etc/postfix# mkdir pgsql
root@mail2:/etc/postfix# vi pgsql/virtual_alias_maps.cf
root@mail2:/etc/postfix# vi pgsql/virtual_domain_maps.cf
root@mail2:/etc/postfix# vi pgsql/relay_domains.cf
root@mail2:/etc/postfix# vi pgsql/virtual_mailbox_limits.cf
root@mail2:/etc/postfix# vi pgsql/virtual_mailbox_maps.cf
root@mail2:/etc/courier# vi /etc/mailname
root@mail2:/etc/courier# cat /etc/postfix/transport
lists.test mailman:
root@mail2:/etc/courier# postmap /etc/postfix/transport
root@mail2:/etc/postfix# scp -r . mail1.test:/etc/postfix/

root@mail1:/etc/postfix# vi main.cf
  1. change following line :
    mydestination = test,mail1.test,localhost.test, localhost

saslauthd

  • change /etc/default/saslauthd

    START=yes
    MECHANISMS="rimap"
    OPTIONS="-c -r -O localhost -m /var/run/saslauthd"

postfixadmin

Only on mail1 : mail2 will be synced through logshipping/PITR ;)

  • open

http://mail1.test/postfixadmin/setup.php

  • set password and replace specified line in /etc/postfixadmin/config.inc.php :

    $CONF['setup_password'] = 'changeme';

  • create superadmin account using a local or valid email address (if you have internet access)
  • modify /usr/share/postfixadmin/functions.inc.php
    • this is in order to allow local domains, e.g. .test

lignes 232++

    if (!preg_match ('/^([-0-9A-Z]+\.)+' . '([0-9A-Z]){2,6}$/i', ($domain)))
    {
    if (!preg_match ('/^([-0-9A-Z]){3,16}$/i', ($domain)))
        {
        flash_error(sprintf($PALANG['pInvalidDomainRegex'], htmlentities($domain)));
        return false;
        }
    }

courier

root@mail1:/etc/courier# vi authdaemonrc
root@mail2:/etc/courier# mv authpgsqlrc authpgsqlrc.debian
root@mail2:/etc/courier# vi authpgsqlrc
root@mail2:/etc/courier# mv imapd imapd.debian
root@mail2:/etc/courier# vi imapd
root@mail2:/etc/courier# mv imapd-ssl imapd-ssl.debian
root@mail2:/etc/courier# vi imapd-ssl

roundcube

  • activate webapp
    • uncomment two alias directives inside /etc/apache2/conf.d/roundcube
    • adapt config :

      $rcmail_config['default_host'] = 'localhost';
      $rcmail_config['smtp_server'] = 'localhost';

  • /etc/init.d/apache2 reload

ssh

  • generate pair of keys on mail1 & mail2
  1. su mail
    $ bash
    mail@mail2:/etc/postfix$ ssh-keygen
    Generating public/private rsa key pair.
    Enter file in which to save the key (/var/mail/.ssh/id_rsa):
    Created directory '/var/mail/.ssh'.
    Enter passphrase (empty for no passphrase):
    Enter same passphrase again:
    Your identification has been saved in /var/mail/.ssh/id_rsa.
    Your public key has been saved in /var/mail/.ssh/id_rsa.pub.
    The key fingerprint is:
    b9:bf:63:05:c0:9f:4f:07:82:d9:fd:79:99:cf:20:20 mail@mail2
    The key's randomart image is:
    -[ RSA 2048]--- | . + . | | E + o | | + + o .o| | .+ o =o.| | S + o +.| | . o o| | . . | | .o | | .oo |
    ---------------
  • add mail1's public key to mail1's authorized keys

    mail@mail1:/$ cp /var/mail/.ssh/id_rsa.pub /var/mail/.ssh/authorized_keys

  • add mail1's public key to mail2's authorized keys

    mail@mail2:/$ vi /var/mail/.ssh/authorized_keys
    mail@mail2:/$ chmod 0600 /var/mail/.ssh/authorized_keys

  • test connection

mail@mail1:/etc/courier$ ssh mail2.test
The authenticity of host 'mail2.test (192.168.122.3)' can't be established.
ECDSA key fingerprint is cb:a6:dd:64:03:ba:45:61:a3:b8:14:3a:05:89:ab:b3.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'mail2.test,192.168.122.3' (ECDSA) to the list of known hosts.
Linux mail2 3.1.0-1-amd64 #1 SMP Fri Dec 23 16:37:11 UTC 2011 x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
$ hostname
mail2
$ logout

inotify/rsync

  1. gen ssh key for user mail on mail1 & copy public key to mail2
  • create sync script

mail@mail1:/etc/courier$ vi ~/sync.sh

#!/bin/sh
BASEDIR="$1" 
REMOTE_HOST="$2" 
RSYNC_OPTIONS="-rtlavz -e ssh --delete" 

# Initial sync
rsync ${RSYNC_OPTIONS} ${BASEDIR}/ ${REMOTE_HOST}:${BASEDIR}

# Wait for events to trigger rsync
inotifywait --format '%e %w' -e close_write -e move -e create -e delete -qmr $BASEDIR | while read EVENT
do
  # Fork off rsync proc to do sync
rsync  ${RSYNC_OPTIONS} ${BASEDIR}/ ${REMOTE_HOST}:${BASEDIR} &
done

root@mail1:/etc/courier# mkdir /var/log/mail
root@mail1:/etc/courier# chown mail:mail /var/log/mail
root@mail1:/etc/courier# vi /etc/rc.local
  1. ajout de la ligne :

    su mail -l -c " nohup sh ~/sync.sh /var/mail/ mail2.test 2>&1 >> /var/log/mail/sync.log &"

root@mail1:/etc/courier# sh /etc/rc.local
nohup: ignoring input and redirecting stderr to stdout
root@mail1:/etc/courier# su mail
mail@mail1:/etc/courier$ chmod 0700 ~/sync.sh

postgresql PITR

  1. gen ssh key for user postgres on mail1 & copy public key to mail2

http://www.postgresql.org/docs/8.4/static/continuous-archiving.html
http://wiki.postgresql.org/wiki/Warm_Standby

  • stop postresql on mail2.test
  • do a full sync of the database
  1. su postgres
    $ rsync -a /var/lib/postgresql/9.1/main/ ://var/lib/postgresql/9.1/main/

on mail1 :

archive_command = 'rsync -a /var/lib/postgresql/9.1/main/%p ://var/lib/postgresql/9.1/wal/pg_xlog/%f'

  • restart postgresql

on mail2 :

root@mail2:~# mkdir /var/lib/postgresql/9.1/wal
root@mail2:~# chown postgres:postgres /var/lib/postgresql/9.1/wal
root@mail2:/var/lib/postgresql/9.1/main# vi recovery.conf
restore_command = 'cp /var/lib/postgresql/9.1/wal/pg_xlog/%f "%p"'

  • on first startup,
  • recovery.conf will be renamed to recovery.done after recovery
    • rename recovery.done to recovery.conf and restart postgresql to sync with latest logs from master.

putting pieces together

  • recover postfixadmin on mail1 password from /etc/postfixadmin/config.inc.php :

    $CONF['database_password'] = 'GENERATED PASSWORD';

  • apply it to /etc/postfixadmin/config.inc.php on mail2
  • apply it to the different files (mail1 & mail2):

    for i in /etc/postfix/pgsql/virtual_alias_maps.cf /etc/postfix/pgsql/virtual_domain_maps.cf /etc/postfix/pgsql/relay_domains.cf /etc/postfix/pgsql/virtual_mailbox_limits.cf /etc/postfix/pgsql/virtual_mailbox_maps.cf ; do sed -i "s/PASSWORD/GENERATED PASSWORD/" $i ; done

    vi /etc/courier/authpgsqlrc

  • restart courier authdaemon :

    /etc/init.d/courier-authdaemon restart

  • create account via postfixadmin
  • verify domain & mailbox creation
  • send testmail in commandline on master (apt-get install bsd-mailx)
  • verify replication of maildir on mail2
  • roundcube
    • connect on http://mail1.test/roundcube with test@test
    • send test mail to outside (may be rejected/filtered as spam since "test" emaildomain isn't valid, should work with a public MX DNS entry)

vacation/responder

root@mail1:~# apt-get install git-core --no-install-recommends
root@mail1:~# cd /usr/share/roundcube/plugins/ && git clone https://github.com/bhuisgen/rc-vacation.git vacation

root@mail1:/usr/share/roundcube/plugins# mkdir /etc/roundcube/plugins/vacation
root@mail1:/usr/share/roundcube/plugins# ln -s /usr/share/roundcube/plugins/vacation/config.inc.php /etc/roundcube/plugins/vacation/
root@mail1:/usr/share/roundcube/plugins# cd vacation/
root@mail1:/usr/share/roundcube/plugins/vacation# cp config.inc.php.dist config.inc.php
root@mail1:/usr/share/roundcube/plugins/vacation# vi config.inc.php
root@mail1:/usr/share/roundcube/plugins/vacation# ln -s /usr/share/roundcube/plugins/vacation/ /var/lib/roundcube/plugins/

  • edit /etc/roundcube/main.inc.php

$rcmail_config['vacation_sql_dsn'] =
'pgsql://postfixadmin:PASSWORD@localhost/postfixadmin';

  • test in roundcube settings, you should have a new tab "vacation/répondeur"

failover

  • in case of a failover of mail1, mail2 should be available to receive mails and provide access to all the mails that were on mail1
    • when mail1 comes back up online, it needs to synchronize with mail2 before
  • in case of a failover of mail2, mail1 should not be impacted

References

http://chiliproject.tetaneutral.net/projects/tetaneutral/wiki/Serveur_Mail_tetalab

http://www.kutukupret.com/2011/06/28/postfix-one-way-maildir-replication-backup-using-inotify-and-rsync/

http://www.postgresql.org/docs/9.1/interactive/continuous-archiving.html