Projet

Général

Profil

Backup » Historique » Version 17

Laurent GUERBY, 28/04/2016 23:38

1 3 Laurent GUERBY
{{>toc}}
2 1 Laurent GUERBY
3 1 Laurent GUERBY
h1. Backup
4 1 Laurent GUERBY
5 1 Laurent GUERBY
h2. Liens
6 1 Laurent GUERBY
7 1 Laurent GUERBY
* [[Backup_infra]]
8 1 Laurent GUERBY
* [[Apt_Backports_TTNN]] borg debian packaging
9 6 Laurent GUERBY
* https://www.reddit.com/r/linux/comments/42feqz/i_asked_here_for_the_optimal_backup_solution_and/
10 1 Laurent GUERBY
11 1 Laurent GUERBY
h2. BackupPC
12 1 Laurent GUERBY
13 1 Laurent GUERBY
* http://backuppc.sourceforge.net/
14 1 Laurent GUERBY
15 1 Laurent GUERBY
h2. Attic
16 1 Laurent GUERBY
17 1 Laurent GUERBY
* https://attic-backup.org/
18 1 Laurent GUERBY
* https://lists.tetaneutral.net/pipermail/technique/2015-September/001971.html
19 1 Laurent GUERBY
** Logiciel de backup : choix de attic
20 1 Laurent GUERBY
21 1 Laurent GUERBY
h2. Borg
22 1 Laurent GUERBY
23 1 Laurent GUERBY
* https://github.com/borgbackup
24 1 Laurent GUERBY
* http://readthedocs.org/projects/borgbackup/
25 2 Laurent GUERBY
* http://puppet.tetaneutral.net/pool/main/b/borgbackup/
26 2 Laurent GUERBY
* http://puppet.tetaneutral.net/dists/
27 1 Laurent GUERBY
28 11 Laurent GUERBY
h2. Borg script
29 11 Laurent GUERBY
30 17 Laurent GUERBY
*Note 20160428* : le script est probablement inutilen cf https://github.com/borgbackup/borg/issues/994
31 17 Laurent GUERBY
32 11 Laurent GUERBY
Pour une machine qui va etre eteinte et rallumée de maniere non controlable.
33 11 Laurent GUERBY
34 12 Laurent GUERBY
Creation initiale avec un user normal capable de ssh sur MACHINE:PORT :
35 12 Laurent GUERBY
36 12 Laurent GUERBY
<pre>
37 12 Laurent GUERBY
borg init --encryption keyfile ssh://USER@MACHINE:PORT/some/where/borg/NICK-repo
38 12 Laurent GUERBY
</pre>
39 12 Laurent GUERBY
40 12 Laurent GUERBY
Et setup cron + script :
41 11 Laurent GUERBY
<pre>
42 11 Laurent GUERBY
# crontab -l
43 11 Laurent GUERBY
@reboot /root/cron-borg.sh
44 11 Laurent GUERBY
45 11 Laurent GUERBY
46 11 Laurent GUERBY
47 11 Laurent GUERBY
# cat /root/cron-borg.sh
48 11 Laurent GUERBY
#!/bin/bash
49 11 Laurent GUERBY
export LANG=en_US.UTF-8
50 11 Laurent GUERBY
mkdir -p /root/borg >& /dev/null
51 11 Laurent GUERBY
52 11 Laurent GUERBY
sleep 300
53 11 Laurent GUERBY
echo === start === $(date) >> /root/borg/cron.log
54 11 Laurent GUERBY
55 11 Laurent GUERBY
NICK=myhost
56 11 Laurent GUERBY
REPO=ssh://USER@MACHINE:PORT/some/where/borg/${NICK}-repo
57 11 Laurent GUERBY
export BORG_PASSPHRASE=lalalala
58 11 Laurent GUERBY
59 11 Laurent GUERBY
if [ -f /root/borg/stamp ]; then
60 11 Laurent GUERBY
    STAMP=$(cat /root/borg/stamp)
61 11 Laurent GUERBY
    borg break-lock $REPO
62 11 Laurent GUERBY
else
63 11 Laurent GUERBY
    STAMP=$(date '+%Y%m%dT%H%M%S')
64 11 Laurent GUERBY
    if [ -f /root/borg/previous-stamp ]; then
65 1 Laurent GUERBY
	  PREVIOUS_STAMP=$(cat /root/borg/previous-stamp)
66 14 Laurent GUERBY
	  while [ "${STAMP%T*}" = "${PREVIOUS_STAMP%T*}" ]; do
67 14 Laurent GUERBY
              STAMP=$(date '+%Y%m%dT%H%M%S')
68 13 Laurent GUERBY
              echo === delay === $(date) >> /root/borg/cron.log
69 14 Laurent GUERBY
	      sleep 1h
70 14 Laurent GUERBY
	  done
71 11 Laurent GUERBY
    fi
72 11 Laurent GUERBY
    echo $STAMP > /root/borg/stamp
73 11 Laurent GUERBY
fi
74 11 Laurent GUERBY
75 11 Laurent GUERBY
76 16 Laurent GUERBY
borg create --compression lz4 --stats --verbose \
77 16 Laurent GUERBY
   --exclude /root/borg --exclude '/home/*/.cache' --exclude-caches --one-file-system \
78 16 Laurent GUERBY
   ${REPO}::${NICK}-$STAMP / >> /root/borg/log-$STAMP 2>> /root/borg/err-$STAMP
79 1 Laurent GUERBY
80 16 Laurent GUERBY
res=$?
81 16 Laurent GUERBY
82 16 Laurent GUERBY
if [ $res -eq 0 -o $res eq 1 ]; then
83 11 Laurent GUERBY
    mv -f /root/borg/stamp /root/borg/previous-stamp >& /dev/null
84 11 Laurent GUERBY
    rm -f /root/borg/stamp >& /dev/null
85 1 Laurent GUERBY
fi
86 1 Laurent GUERBY
87 1 Laurent GUERBY
88 16 Laurent GUERBY
echo === done === $res === $(date) >> /root/borg/cron.log
89 14 Laurent GUERBY
90 14 Laurent GUERBY
exec "$0"
91 11 Laurent GUERBY
</pre>
92 1 Laurent GUERBY
93 1 Laurent GUERBY
h2. Migration Attic vers Borg
94 7 Laurent GUERBY
95 4 Mehdi Abaakouk
* https://github.com/borgbackup/borg/pull/231
96 4 Mehdi Abaakouk
* old: https://chiliproject.tetaneutral.net/projects/git-tetaneutral-net/repository/puppet-backup
97 1 Laurent GUERBY
* new: https://chiliproject.tetaneutral.net/projects/git-tetaneutral-net/repository/puppetmaster/revisions/master/entry/modules/ttnn/manifests/backup.pp
98 1 Laurent GUERBY
** git history BackupPC => Attic => Borg
99 4 Mehdi Abaakouk
100 5 Laurent GUERBY
<pre>
101 4 Mehdi Abaakouk
# apt-get -t jessie-backports install borgbackup
102 4 Mehdi Abaakouk
$ cd /backup/attic/
103 4 Mehdi Abaakouk
$ borg upgrade <repo>
104 4 Mehdi Abaakouk
$ borg check --repair <repo>
105 4 Mehdi Abaakouk
$ mv <repo> ../borg/
106 4 Mehdi Abaakouk
$ chown -R backupinfra: /backup/borg/<repo>
107 4 Mehdi Abaakouk
</pre>
108 8 Laurent GUERBY
109 4 Mehdi Abaakouk
Dans le module puppet, le changement le plus important est le parametre compression explicite pour correspondre au défaut de attic create :
110 4 Mehdi Abaakouk
111 8 Laurent GUERBY
<pre>
112 4 Mehdi Abaakouk
attic create ... <repo> -> borg create --compression zlib,6 ... <repo>
113 4 Mehdi Abaakouk
</pre>
114 4 Mehdi Abaakouk
115 9 Laurent GUERBY
Sinon le prochain backup sera non compressé, et aucun nouveau chucks ne correspondra aux anciens -> perte de la dedup. "zlib,6" étant le niveau de compression de attic.
116 9 Laurent GUERBY
117 10 Laurent GUERBY
https://github.com/jborg/attic/issues/299
118 9 Laurent GUERBY
http://borgbackup.readthedocs.org/en/stable/usage.html#environment-variables
119 9 Laurent GUERBY
<pre>
120 9 Laurent GUERBY
export ATTIC_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=yes
121 1 Laurent GUERBY
</pre>