Backup » Historique » Version 18
Version 17 (Laurent GUERBY, 28/04/2016 23:38) → Version 18/24 (Laurent GUERBY, 28/04/2016 23:39)
{{>toc}}
h1. Backup
h2. Liens
* [[Backup_infra]]
* [[Apt_Backports_TTNN]] borg debian packaging
* https://www.reddit.com/r/linux/comments/42feqz/i_asked_here_for_the_optimal_backup_solution_and/
h2. BackupPC
* http://backuppc.sourceforge.net/
h2. Attic
* https://attic-backup.org/
* https://lists.tetaneutral.net/pipermail/technique/2015-September/001971.html
** Logiciel de backup : choix de attic
h2. Borg
* https://github.com/borgbackup
* http://readthedocs.org/projects/borgbackup/
* http://puppet.tetaneutral.net/pool/main/b/borgbackup/
* http://puppet.tetaneutral.net/dists/
h2. Borg script
*Note 20160428* : le script est probablement inutile inutilen cf https://github.com/borgbackup/borg/issues/994
Pour une machine qui va etre eteinte et rallumée de maniere non controlable.
Creation initiale avec un user normal capable de ssh sur MACHINE:PORT :
<pre>
borg init --encryption keyfile ssh://USER@MACHINE:PORT/some/where/borg/NICK-repo
</pre>
Et setup cron + script :
<pre>
# crontab -l
@reboot /root/cron-borg.sh
# cat /root/cron-borg.sh
#!/bin/bash
export LANG=en_US.UTF-8
mkdir -p /root/borg >& /dev/null
sleep 300
echo === start === $(date) >> /root/borg/cron.log
NICK=myhost
REPO=ssh://USER@MACHINE:PORT/some/where/borg/${NICK}-repo
export BORG_PASSPHRASE=lalalala
if [ -f /root/borg/stamp ]; then
STAMP=$(cat /root/borg/stamp)
borg break-lock $REPO
else
STAMP=$(date '+%Y%m%dT%H%M%S')
if [ -f /root/borg/previous-stamp ]; then
PREVIOUS_STAMP=$(cat /root/borg/previous-stamp)
while [ "${STAMP%T*}" = "${PREVIOUS_STAMP%T*}" ]; do
STAMP=$(date '+%Y%m%dT%H%M%S')
echo === delay === $(date) >> /root/borg/cron.log
sleep 1h
done
fi
echo $STAMP > /root/borg/stamp
fi
borg create --compression lz4 --stats --verbose \
--exclude /root/borg --exclude '/home/*/.cache' --exclude-caches --one-file-system \
${REPO}::${NICK}-$STAMP / >> /root/borg/log-$STAMP 2>> /root/borg/err-$STAMP
res=$?
if [ $res -eq 0 -o $res eq 1 ]; then
mv -f /root/borg/stamp /root/borg/previous-stamp >& /dev/null
rm -f /root/borg/stamp >& /dev/null
fi
echo === done === $res === $(date) >> /root/borg/cron.log
exec "$0"
</pre>
h2. Migration Attic vers Borg
* https://github.com/borgbackup/borg/pull/231
* old: https://chiliproject.tetaneutral.net/projects/git-tetaneutral-net/repository/puppet-backup
* new: https://chiliproject.tetaneutral.net/projects/git-tetaneutral-net/repository/puppetmaster/revisions/master/entry/modules/ttnn/manifests/backup.pp
** git history BackupPC => Attic => Borg
<pre>
# apt-get -t jessie-backports install borgbackup
$ cd /backup/attic/
$ borg upgrade <repo>
$ borg check --repair <repo>
$ mv <repo> ../borg/
$ chown -R backupinfra: /backup/borg/<repo>
</pre>
Dans le module puppet, le changement le plus important est le parametre compression explicite pour correspondre au défaut de attic create :
<pre>
attic create ... <repo> -> borg create --compression zlib,6 ... <repo>
</pre>
Sinon le prochain backup sera non compressé, et aucun nouveau chucks ne correspondra aux anciens -> perte de la dedup. "zlib,6" étant le niveau de compression de attic.
https://github.com/jborg/attic/issues/299
http://borgbackup.readthedocs.org/en/stable/usage.html#environment-variables
<pre>
export ATTIC_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=yes
</pre>
h1. Backup
h2. Liens
* [[Backup_infra]]
* [[Apt_Backports_TTNN]] borg debian packaging
* https://www.reddit.com/r/linux/comments/42feqz/i_asked_here_for_the_optimal_backup_solution_and/
h2. BackupPC
* http://backuppc.sourceforge.net/
h2. Attic
* https://attic-backup.org/
* https://lists.tetaneutral.net/pipermail/technique/2015-September/001971.html
** Logiciel de backup : choix de attic
h2. Borg
* https://github.com/borgbackup
* http://readthedocs.org/projects/borgbackup/
* http://puppet.tetaneutral.net/pool/main/b/borgbackup/
* http://puppet.tetaneutral.net/dists/
h2. Borg script
*Note 20160428* : le script est probablement inutile inutilen cf https://github.com/borgbackup/borg/issues/994
Pour une machine qui va etre eteinte et rallumée de maniere non controlable.
Creation initiale avec un user normal capable de ssh sur MACHINE:PORT :
<pre>
borg init --encryption keyfile ssh://USER@MACHINE:PORT/some/where/borg/NICK-repo
</pre>
Et setup cron + script :
<pre>
# crontab -l
@reboot /root/cron-borg.sh
# cat /root/cron-borg.sh
#!/bin/bash
export LANG=en_US.UTF-8
mkdir -p /root/borg >& /dev/null
sleep 300
echo === start === $(date) >> /root/borg/cron.log
NICK=myhost
REPO=ssh://USER@MACHINE:PORT/some/where/borg/${NICK}-repo
export BORG_PASSPHRASE=lalalala
if [ -f /root/borg/stamp ]; then
STAMP=$(cat /root/borg/stamp)
borg break-lock $REPO
else
STAMP=$(date '+%Y%m%dT%H%M%S')
if [ -f /root/borg/previous-stamp ]; then
PREVIOUS_STAMP=$(cat /root/borg/previous-stamp)
while [ "${STAMP%T*}" = "${PREVIOUS_STAMP%T*}" ]; do
STAMP=$(date '+%Y%m%dT%H%M%S')
echo === delay === $(date) >> /root/borg/cron.log
sleep 1h
done
fi
echo $STAMP > /root/borg/stamp
fi
borg create --compression lz4 --stats --verbose \
--exclude /root/borg --exclude '/home/*/.cache' --exclude-caches --one-file-system \
${REPO}::${NICK}-$STAMP / >> /root/borg/log-$STAMP 2>> /root/borg/err-$STAMP
res=$?
if [ $res -eq 0 -o $res eq 1 ]; then
mv -f /root/borg/stamp /root/borg/previous-stamp >& /dev/null
rm -f /root/borg/stamp >& /dev/null
fi
echo === done === $res === $(date) >> /root/borg/cron.log
exec "$0"
</pre>
h2. Migration Attic vers Borg
* https://github.com/borgbackup/borg/pull/231
* old: https://chiliproject.tetaneutral.net/projects/git-tetaneutral-net/repository/puppet-backup
* new: https://chiliproject.tetaneutral.net/projects/git-tetaneutral-net/repository/puppetmaster/revisions/master/entry/modules/ttnn/manifests/backup.pp
** git history BackupPC => Attic => Borg
<pre>
# apt-get -t jessie-backports install borgbackup
$ cd /backup/attic/
$ borg upgrade <repo>
$ borg check --repair <repo>
$ mv <repo> ../borg/
$ chown -R backupinfra: /backup/borg/<repo>
</pre>
Dans le module puppet, le changement le plus important est le parametre compression explicite pour correspondre au défaut de attic create :
<pre>
attic create ... <repo> -> borg create --compression zlib,6 ... <repo>
</pre>
Sinon le prochain backup sera non compressé, et aucun nouveau chucks ne correspondra aux anciens -> perte de la dedup. "zlib,6" étant le niveau de compression de attic.
https://github.com/jborg/attic/issues/299
http://borgbackup.readthedocs.org/en/stable/usage.html#environment-variables
<pre>
export ATTIC_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=yes
</pre>