Projet

Général

Profil

Historique

HowTo Mail Backup - Ikujam » Historique » Version 5

« Précédent - Version 5/19 (diff) - Version actuelle
iku jam, 30/01/2012 14:23
layout

Presentation

several projects with mail servers
request of certain stability, needed documentation
free software user, activist and contributor
idea is to produce a complete test environment with vms on a single machine

CC-NC-SA

Requirements

to follow you need some linux admin skills:

  • basic shell (bash)
  • at least basic knowledge of debian package system (install & setup packages with apt-get, manage services)
  • able to setup ssh public key authentication
  • i don't like nano, feel free to use it - or another editor - instead of vi

Host system

  • debian
  • qemu-kvm
  • bind

This howto uses

 # cat /etc/debian_version 
 wheezy/sid
 # uname -a
 Linux master 3.1.0-1-amd64 #1 SMP Sun Dec 11 20:36:41 UTC 2011 x86_64 GNU/Linux

Mail Server VMs

  • debian
  • debian packages for the different software
 root@mail1:~# echo "mail1" > /etc/hostname
 root@mail1:~# apt-get install inotify-tools rsync openssh-server pgpool  javascript-common apache2 libapache2-mod-php5  roundcube postgresql postfix postfix-pgsql mailman roundcube-pgsql libc-client2007e mlock php5-imap postgrey courier-authlib-postgresql sasl2-bin courier-authdaemon  libsasl2-modules-sql courier-imap-ssl --no-install-recommends
  • use default options for roundcube, courier & mailman for now
    • ident authentication
    • dbconfig
    • pgsql as database choice
    • mailman language as you prefer
  • install postfixadmin :
 root@mail1:~# lynx 'http://downloads.sourceforge.net/project/postfixadmin/postfixadmin/postfixadmin-2.3.4/postfixadmin_2.3.4_all.deb'

either directly from within lynx, otherwise via

 root@mail1:~# dpkg -i postfixadmin_2.3.4_all.deb
  • use default options for now
  • just as of personal habit, some tools i use
 root@mail1:~# apt-get install lynx less mc vim screen
 root@mail1:~# cat /etc/debian_version 
 wheezy/sid
 root@mail1:~# uname -a
 Linux mail1.test 3.1.0-1-amd64 #1 SMP Tue Jan 10 05:01:58 UTC 2012 x86_64 GNU/Linux

root@mail2:~# cat /etc/debian_version 
wheezy/sid
root@mail2:~# uname -a
Linux mail2 3.1.0-1-amd64 #1 SMP Fri Dec 23 16:37:11 UTC 2011 x86_64 GNU/Linux
root@mail2:~# cat /etc/network/interfaces 
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
allow-hotplug eth0
iface eth0 inet static
        address 192.168.122.3
        netmask 255.255.255.0
        network 192.168.122.0
        broadcast 192.168.122.255
        gateway 192.168.122.1

dns setup on host node


root@quadebian:/etc/bind# cat  db.192.168.122
;
; BIND reverse data file for test
;
$TTL    604800
@    IN    SOA    master.test. root.master.test. (
                  1        ; Serial
             604800        ; Refresh
              86400        ; Retry
            2419200        ; Expire
             604800 )    ; Negative Cache TTL
;
@    IN    NS    master.test.
1    IN    PTR    master.test.
2    IN    PTR    mail1.test.
3    IN    PTR    mail2.test.

root@quadebian:/etc/bind# cat  db.test 
;
; BIND data file for test
;
$TTL    604800
@    IN    SOA    master.test. info.master.test. (
                  2        ; Serial
             604800        ; Refresh
              86400        ; Retry
            2419200        ; Expire
             604800 )    ; Negative Cache TTL
;
@    IN    NS    master.test.
test.    IN    MX    10    mail1.test.
test.    IN    MX    20    mail2.test.

master    IN    A    192.168.122.1
mail1    IN    A    192.168.122.2
mail2    IN    A    192.168.122.3

root@quadebian:/etc/bind# named-checkzone test db.test 
zone test/IN: loaded serial 2
OK
  • pass kvm dns server in forward mode on host node (default net config)
root@quadebian:/etc/bind# virsh 
Welcome to virsh, the virtualization interactive terminal.

Type:  'help' for help with commands
       'quit' to quit

virsh # net-dumpxml default
<network>
  <name>default</name>
  <uuid>0529cc34-c2ad-9663-0f42-5b338b14a6e4</uuid>
  <forward mode='nat'/>
  <bridge name='virbr0' stp='on' delay='0' />
  <mac address='52:54:00:37:85:D8'/>
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.122.2' end='192.168.122.254' />
    </dhcp>
  </ip>
</network>

vm dns config

  • change requires to reaffect NICs via virt-manager
    • remove nic (and /etc/udev/rules.d/70-persistent-net.rules - it keeps track of different nics on the system, avoids getting eth2/3/4...)
    • create new nic on default network
    • reboot vm
    • test connectivity & bind (set nameserver to 192.168.122.1 in /etc/resolv.conf)

tests to do

root@quadebian:/etc/bind# dig mx test

; <<>> DiG 9.7.3 <<>> mx test
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 26405
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 1, ADDITIONAL: 3

;; QUESTION SECTION:
;test.                IN    MX

;; ANSWER SECTION:
test.            604800    IN    MX    20 mail2.test.
test.            604800    IN    MX    10 mail1.test.

;; AUTHORITY SECTION:
test.            604800    IN    NS    master.test.

;; ADDITIONAL SECTION:
mail1.test.        604800    IN    A    192.168.122.2
mail2.test.        604800    IN    A    192.168.122.3
master.test.        604800    IN    A    192.168.122.1

;; Query time: 2 msec
;; SERVER: 10.11.12.126#53(10.11.12.126)
;; WHEN: Tue Jan 24 09:55:25 2012
;; MSG SIZE  rcvd: 135

Server configuration

postfix


root@mail2:/etc/postfix# mv main.cf main.cf.debian
root@mail2:/etc/postfix# vi  main.cf 
root@mail2:/etc/postfix# mkdir pgsql
root@mail2:/etc/postfix# vi pgsql/virtual_alias_maps.cf
root@mail2:/etc/postfix# vi pgsql/virtual_domain_maps.cf
root@mail2:/etc/postfix# vi pgsql/relay_domains.cf
root@mail2:/etc/postfix# vi pgsql/virtual_mailbox_limits.cf
root@mail2:/etc/postfix# vi pgsql/virtual_mailbox_maps.cf 
root@mail2:/etc/courier# vi /etc/mailname 
root@mail2:/etc/courier# cat /etc/postfix/transport
lists.test mailman:
root@mail2:/etc/courier# postmap  /etc/postfix/transport
root@mail2:/etc/postfix# scp -r . mail1.test:/etc/postfix/

root@mail1:/etc/postfix# vi  main.cf 
# change following line : 
mydestination = test,mail1.test,localhost.test, localhost

saslauthd

  • change /etc/default/saslauthd
 START=yes
 MECHANISMS="rimap" 
 OPTIONS="-c -r -O localhost -m /var/run/saslauthd"

postfixadmin

Only on mail1 : mail2 will be synced through logshipping/PITR ;)

  • open

http://mail1.test/postfixadmin/setup.php

  • set password and replace specified line in /etc/postfixadmin/config.inc.php :
 $CONF['setup_password'] = 'changeme';
  • create superadmin account using a local or valid email address (if you have internet access)
  • modify /usr/share/postfixadmin/functions.inc.php
    • this is in order to allow local domains, e.g. .test

lignes 232++

    if (!preg_match ('/^([-0-9A-Z]+\.)+' . '([0-9A-Z]){2,6}$/i', ($domain)))
    {
    if (!preg_match ('/^([-0-9A-Z]){3,16}$/i', ($domain)))
        {
        flash_error(sprintf($PALANG['pInvalidDomainRegex'], htmlentities($domain)));
        return false;
        }
    }

courier

root@mail1:/etc/courier# vi authdaemonrc 
root@mail2:/etc/courier# mv authpgsqlrc authpgsqlrc.debian
root@mail2:/etc/courier# vi authpgsqlrc
root@mail2:/etc/courier# mv imapd imapd.debian
root@mail2:/etc/courier# vi  imapd 
root@mail2:/etc/courier# mv imapd-ssl imapd-ssl.debian
root@mail2:/etc/courier# vi  imapd-ssl

roundcube

  • activate webapp
    • uncomment two alias directives inside /etc/apache2/conf.d/roundcube
    • adapt config :
 $rcmail_config['default_host'] = 'localhost';
 $rcmail_config['smtp_server'] = 'localhost';
  • /etc/init.d/apache2 reload

ssh

  • generate pair of keys on mail1 & mail2
# su mail
$ bash
mail@mail2:/etc/postfix$ ssh-keygen 
Generating public/private rsa key pair.
Enter file in which to save the key (/var/mail/.ssh/id_rsa): 
Created directory '/var/mail/.ssh'.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /var/mail/.ssh/id_rsa.
Your public key has been saved in /var/mail/.ssh/id_rsa.pub.
The key fingerprint is:
b9:bf:63:05:c0:9f:4f:07:82:d9:fd:79:99:cf:20:20 mail@mail2
The key's randomart image is:
+--[ RSA 2048]----+
|       . + .     |
|        E + o    |
|         + + o .o|
|         .+ o =o.|
|        S  + o +.|
|         .  o   o|
|        .  .     |
|         .o      |
|         .oo     |
+-----------------+
  • add mail1's public key to mail1's authorized keys
 mail@mail1:/$ cp /var/mail/.ssh/id_rsa.pub /var/mail/.ssh/authorized_keys
  • add mail1's public key to mail2's authorized keys
 mail@mail2:/$ vi /var/mail/.ssh/authorized_keys
 mail@mail2:/$ chmod 0600 /var/mail/.ssh/authorized_keys
  • test connection
mail@mail1:/etc/courier$ ssh mail2.test
The authenticity of host 'mail2.test (192.168.122.3)' can't be established.
ECDSA key fingerprint is cb:a6:dd:64:03:ba:45:61:a3:b8:14:3a:05:89:ab:b3.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'mail2.test,192.168.122.3' (ECDSA) to the list of known hosts.
Linux mail2 3.1.0-1-amd64 #1 SMP Fri Dec 23 16:37:11 UTC 2011 x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
$ hostname
mail2
$ logout

inotify/rsync

  1. gen ssh key for user mail on mail1 & copy public key to mail2
  • create sync script
mail@mail1:/etc/courier$ vi ~/sync.sh

#!/bin/sh
BASEDIR="$1" 
REMOTE_HOST="$2" 
RSYNC_OPTIONS="-rtlavz -e ssh --delete" 

# Initial sync
rsync ${RSYNC_OPTIONS} ${BASEDIR}/ ${REMOTE_HOST}:${BASEDIR}

# Wait for events to trigger rsync
inotifywait --format '%w' -e close_write -e move -e create -e delete -qmr $BASEDIR | while read EVENT_DIR
do
  # Fork off rsync proc to do sync
rsync  ${RSYNC_OPTIONS} ${EVENT_DIR} ${REMOTE_HOST}:${EVENT_DIR} &
done


root@mail1:/etc/courier# mkdir /var/log/mail
root@mail1:/etc/courier# chown mail:mail  /var/log/mail
root@mail1:/etc/courier# vi   /etc/rc.local
  • ajout de la ligne :
 su mail -l  -c " nohup sh ~/sync.sh /var/mail/ mail2.test  2>&1 >> /var/log/mail/sync.log &" 

root@mail1:/etc/courier# sh  /etc/rc.local 
nohup: ignoring input and redirecting stderr to stdout
root@mail1:/etc/courier# su mail
mail@mail1:/etc/courier$ chmod 0700 ~/sync.sh

postgresql PITR

  • gen ssh key for user postgres on mail1 & copy public key to mail2

http://www.postgresql.org/docs/8.4/static/continuous-archiving.html
http://wiki.postgresql.org/wiki/Warm_Standby

  • stop postresql on mail2.test
  • do a full sync of the database
# su postgres
$ rsync -a /var/lib/postgresql/9.1/main/ postgres@mail2.test://var/lib/postgresql/9.1/main/

on mail1 :

archive_command = 'rsync -a /var/lib/postgresql/9.1/main/%p ://var/lib/postgresql/9.1/wal/pg_xlog/%f'

  • restart postgresql

on mail2 :

root@mail2:~# mkdir /var/lib/postgresql/9.1/wal
root@mail2:~# chown postgres:postgres  /var/lib/postgresql/9.1/wal
root@mail2:/var/lib/postgresql/9.1/main# vi recovery.conf
restore_command = 'cp /var/lib/postgresql/9.1/wal/pg_xlog/%f "%p"'
  • on first startup,
  • recovery.conf will be renamed to recovery.done after recovery
    • rename recovery.done to recovery.conf and restart postgresql to sync with latest logs from master.

putting pieces together

  • recover postfixadmin on mail1 password from /etc/postfixadmin/config.inc.php :

    $CONF['database_password'] = 'GENERATED PASSWORD';

  • apply it to /etc/postfixadmin/config.inc.php on mail2
  • apply it to the different files (mail1 & mail2):
for i in /etc/postfix/pgsql/virtual_alias_maps.cf /etc/postfix/pgsql/virtual_domain_maps.cf /etc/postfix/pgsql/relay_domains.cf /etc/postfix/pgsql/virtual_mailbox_limits.cf /etc/postfix/pgsql/virtual_mailbox_maps.cf ; do   sed -i "s/PASSWORD/GENERATED PASSWORD/"  $i ;  done
 vi  /etc/courier/authpgsqlrc
  • restart courier authdaemon :

    /etc/init.d/courier-authdaemon restart

  • create account via postfixadmin
  • verify domain & mailbox creation
  • send testmail in commandline on master (apt-get install bsd-mailx)
  • verify replication of maildir on mail2
  • roundcube
    • connect on http://mail1.test/roundcube with test@test
    • send test mail to outside (may be rejected/filtered as spam since "test" emaildomain isn't valid, should work with a public MX DNS entry)

vacation/responder

root@mail1:~# apt-get install git-core  --no-install-recommends  
root@mail1:~# cd /usr/share/roundcube/plugins/ && git clone https://github.com/bhuisgen/rc-vacation.git vacation

root@mail1:/usr/share/roundcube/plugins# mkdir /etc/roundcube/plugins/vacation
root@mail1:/usr/share/roundcube/plugins# ln -s /usr/share/roundcube/plugins/vacation/config.inc.php /etc/roundcube/plugins/vacation/
root@mail1:/usr/share/roundcube/plugins# cd vacation/
root@mail1:/usr/share/roundcube/plugins/vacation# cp config.inc.php.dist config.inc.php
root@mail1:/usr/share/roundcube/plugins/vacation# vi config.inc.php
root@mail1:/usr/share/roundcube/plugins/vacation# ln -s /usr/share/roundcube/plugins/vacation/ /var/lib/roundcube/plugins/
  • edit /etc/roundcube/main.inc.php

$rcmail_config['vacation_sql_dsn'] =
'pgsql://postfixadmin:PASSWORD@localhost/postfixadmin';

  • test in roundcube settings, you should have a new tab "vacation/répondeur"

failover

  • in case of a failover of mail1, mail2 should be available to receive mails and provide access to all the mails that were on mail1
    • when mail1 comes back up online, it needs to synchronize with mail2 before
  • in case of a failover of mail2, mail1 should not be impacted

References

http://chiliproject.tetaneutral.net/projects/tetaneutral/wiki/Serveur_Mail_tetalab

http://www.kutukupret.com/2011/06/28/postfix-one-way-maildir-replication-backup-using-inotify-and-rsync/

http://www.postgresql.org/docs/9.1/interactive/continuous-archiving.html