Project

General

Profile

Backup » History » Version 24

Laurent GUERBY, 06/11/2018 08:27 AM

1 3 Laurent GUERBY
{{>toc}}
2 1 Laurent GUERBY
3 1 Laurent GUERBY
h1. Backup
4 1 Laurent GUERBY
5 1 Laurent GUERBY
h2. Liens
6 1 Laurent GUERBY
7 1 Laurent GUERBY
* [[Backup_infra]]
8 1 Laurent GUERBY
* [[Apt_Backports_TTNN]] borg debian packaging
9 6 Laurent GUERBY
* https://www.reddit.com/r/linux/comments/42feqz/i_asked_here_for_the_optimal_backup_solution_and/
10 23 Laurent GUERBY
* http://linuxfr.org/users/gbetous/journaux/raid-is-no-backup
11 1 Laurent GUERBY
12 1 Laurent GUERBY
h2. BackupPC
13 1 Laurent GUERBY
14 1 Laurent GUERBY
* http://backuppc.sourceforge.net/
15 1 Laurent GUERBY
16 1 Laurent GUERBY
h2. Attic
17 1 Laurent GUERBY
18 1 Laurent GUERBY
* https://attic-backup.org/
19 1 Laurent GUERBY
* https://lists.tetaneutral.net/pipermail/technique/2015-September/001971.html
20 1 Laurent GUERBY
** Logiciel de backup : choix de attic
21 1 Laurent GUERBY
22 1 Laurent GUERBY
h2. Borg
23 1 Laurent GUERBY
24 1 Laurent GUERBY
* https://github.com/borgbackup
25 1 Laurent GUERBY
* http://readthedocs.org/projects/borgbackup/
26 2 Laurent GUERBY
* http://puppet.tetaneutral.net/pool/main/b/borgbackup/
27 2 Laurent GUERBY
* http://puppet.tetaneutral.net/dists/
28 1 Laurent GUERBY
29 20 Nicolas BERTRAND
Pour un peu automatiser + cron, on peut utiliser borgmatic:
30 20 Nicolas BERTRAND
* https://github.com/witten/borgmatic
31 21 Laurent GUERBY
32 24 Laurent GUERBY
Doc intarnet :
33 24 Laurent GUERBY
* https://www.intarnet.fr/wiki/technique:admin-systeme:borg
34 24 Laurent GUERBY
35 22 Laurent GUERBY
h3. Borg tips
36 21 Laurent GUERBY
37 21 Laurent GUERBY
* On debian systems to avoid backuping cache (apt & others) use borg  --exclude-caches  and do once :
38 21 Laurent GUERBY
<pre>
39 21 Laurent GUERBY
echo Signature: 8a477f597d28d172789f06886806bc55 > /var/cache/CACHEDIR.TAG
40 21 Laurent GUERBY
</pre>
41 21 Laurent GUERBY
42 21 Laurent GUERBY
http://borgbackup.readthedocs.io/en/stable/usage.html#borg-create
43 21 Laurent GUERBY
http://www.brynosaurus.com/cachedir/spec.html
44 21 Laurent GUERBY
45 22 Laurent GUERBY
h3. Borg script
46 11 Laurent GUERBY
47 18 Laurent GUERBY
*Note 20160428* : le script est probablement inutile cf https://github.com/borgbackup/borg/issues/994
48 17 Laurent GUERBY
49 11 Laurent GUERBY
Pour une machine qui va etre eteinte et rallumée de maniere non controlable.
50 11 Laurent GUERBY
51 12 Laurent GUERBY
Creation initiale avec un user normal capable de ssh sur MACHINE:PORT :
52 12 Laurent GUERBY
53 12 Laurent GUERBY
<pre>
54 12 Laurent GUERBY
borg init --encryption keyfile ssh://USER@MACHINE:PORT/some/where/borg/NICK-repo
55 12 Laurent GUERBY
</pre>
56 12 Laurent GUERBY
57 19 Nicolas BERTRAND
On A 2 types d'encryption keyfile et repokey
58 19 Nicolas BERTRAND
* en keyfile: la clef est stockée 'localement' est doit aussi être backupé. mode "passphrase + key"
59 19 Nicolas BERTRAND
* repokey : la clef est stocké dans le repo only: mode "passphrase only"
60 19 Nicolas BERTRAND
<pre>
61 19 Nicolas BERTRAND
11.1026 < guerby> [09:39:25] zorun, tu as une ref dans la doc ? ce que j'ai trouvé http://borgbackup.readthedocs.io/en/stable/quickstart.html#encrypted-repos
62 19 Nicolas BERTRAND
11.1026 < guerby> [09:39:38] "so you still have the key in case it gets corrupted or lost. Also keep your passphrase at a safe place."
63 19 Nicolas BERTRAND
11.1026 < zorun> [09:50:10] guerby: cherche « repokey » dans https://borgbackup.readthedocs.io/en/stable/usage.html
64 19 Nicolas BERTRAND
11.1026 < zorun> [09:50:19] # Local repository (default is to use encryption in repokey mode)
65 19 Nicolas BERTRAND
11.1026 < zorun> [09:50:27] If you want “passphrase-only” security, use the repokey mode. The key will be stored inside the repository (in its “config” file). In above mentioned attack scenario, the attacker will have the key (but not the passphrase).
66 19 Nicolas BERTRAND
11.1026 < zorun> [09:50:43] If you want “passphrase and having-the-key” security, use the keyfile mode. The key will be stored in your home directory (in .config/borg/keys). In the attack scenario, the attacker who has just access to your repo won’t have the key (and also not the passphrase).
67 19 Nicolas BERTRAND
11.1026 < taziden> [09:51:21] et la méthode par défaut, c'est repokey
68 19 Nicolas BERTRAND
</pre>
69 19 Nicolas BERTRAND
  
70 19 Nicolas BERTRAND
71 12 Laurent GUERBY
Et setup cron + script :
72 11 Laurent GUERBY
<pre>
73 11 Laurent GUERBY
# crontab -l
74 11 Laurent GUERBY
@reboot /root/cron-borg.sh
75 11 Laurent GUERBY
76 11 Laurent GUERBY
77 11 Laurent GUERBY
78 11 Laurent GUERBY
# cat /root/cron-borg.sh
79 11 Laurent GUERBY
#!/bin/bash
80 11 Laurent GUERBY
export LANG=en_US.UTF-8
81 11 Laurent GUERBY
mkdir -p /root/borg >& /dev/null
82 11 Laurent GUERBY
83 11 Laurent GUERBY
sleep 300
84 11 Laurent GUERBY
echo === start === $(date) >> /root/borg/cron.log
85 11 Laurent GUERBY
86 11 Laurent GUERBY
NICK=myhost
87 11 Laurent GUERBY
REPO=ssh://USER@MACHINE:PORT/some/where/borg/${NICK}-repo
88 11 Laurent GUERBY
export BORG_PASSPHRASE=lalalala
89 11 Laurent GUERBY
90 11 Laurent GUERBY
if [ -f /root/borg/stamp ]; then
91 11 Laurent GUERBY
    STAMP=$(cat /root/borg/stamp)
92 11 Laurent GUERBY
    borg break-lock $REPO
93 11 Laurent GUERBY
else
94 11 Laurent GUERBY
    STAMP=$(date '+%Y%m%dT%H%M%S')
95 11 Laurent GUERBY
    if [ -f /root/borg/previous-stamp ]; then
96 1 Laurent GUERBY
	  PREVIOUS_STAMP=$(cat /root/borg/previous-stamp)
97 14 Laurent GUERBY
	  while [ "${STAMP%T*}" = "${PREVIOUS_STAMP%T*}" ]; do
98 14 Laurent GUERBY
              STAMP=$(date '+%Y%m%dT%H%M%S')
99 13 Laurent GUERBY
              echo === delay === $(date) >> /root/borg/cron.log
100 14 Laurent GUERBY
	      sleep 1h
101 14 Laurent GUERBY
	  done
102 11 Laurent GUERBY
    fi
103 11 Laurent GUERBY
    echo $STAMP > /root/borg/stamp
104 11 Laurent GUERBY
fi
105 11 Laurent GUERBY
106 11 Laurent GUERBY
107 16 Laurent GUERBY
borg create --compression lz4 --stats --verbose \
108 16 Laurent GUERBY
   --exclude /root/borg --exclude '/home/*/.cache' --exclude-caches --one-file-system \
109 16 Laurent GUERBY
   ${REPO}::${NICK}-$STAMP / >> /root/borg/log-$STAMP 2>> /root/borg/err-$STAMP
110 1 Laurent GUERBY
111 16 Laurent GUERBY
res=$?
112 16 Laurent GUERBY
113 16 Laurent GUERBY
if [ $res -eq 0 -o $res eq 1 ]; then
114 11 Laurent GUERBY
    mv -f /root/borg/stamp /root/borg/previous-stamp >& /dev/null
115 11 Laurent GUERBY
    rm -f /root/borg/stamp >& /dev/null
116 1 Laurent GUERBY
fi
117 1 Laurent GUERBY
118 1 Laurent GUERBY
119 16 Laurent GUERBY
echo === done === $res === $(date) >> /root/borg/cron.log
120 14 Laurent GUERBY
121 14 Laurent GUERBY
exec "$0"
122 11 Laurent GUERBY
</pre>
123 1 Laurent GUERBY
124 1 Laurent GUERBY
h2. Migration Attic vers Borg
125 7 Laurent GUERBY
126 4 Mehdi Abaakouk
* https://github.com/borgbackup/borg/pull/231
127 4 Mehdi Abaakouk
* old: https://chiliproject.tetaneutral.net/projects/git-tetaneutral-net/repository/puppet-backup
128 1 Laurent GUERBY
* new: https://chiliproject.tetaneutral.net/projects/git-tetaneutral-net/repository/puppetmaster/revisions/master/entry/modules/ttnn/manifests/backup.pp
129 1 Laurent GUERBY
** git history BackupPC => Attic => Borg
130 4 Mehdi Abaakouk
131 5 Laurent GUERBY
<pre>
132 4 Mehdi Abaakouk
# apt-get -t jessie-backports install borgbackup
133 4 Mehdi Abaakouk
$ cd /backup/attic/
134 4 Mehdi Abaakouk
$ borg upgrade <repo>
135 4 Mehdi Abaakouk
$ borg check --repair <repo>
136 4 Mehdi Abaakouk
$ mv <repo> ../borg/
137 4 Mehdi Abaakouk
$ chown -R backupinfra: /backup/borg/<repo>
138 4 Mehdi Abaakouk
</pre>
139 8 Laurent GUERBY
140 4 Mehdi Abaakouk
Dans le module puppet, le changement le plus important est le parametre compression explicite pour correspondre au défaut de attic create :
141 4 Mehdi Abaakouk
142 8 Laurent GUERBY
<pre>
143 4 Mehdi Abaakouk
attic create ... <repo> -> borg create --compression zlib,6 ... <repo>
144 4 Mehdi Abaakouk
</pre>
145 4 Mehdi Abaakouk
146 9 Laurent GUERBY
Sinon le prochain backup sera non compressé, et aucun nouveau chucks ne correspondra aux anciens -> perte de la dedup. "zlib,6" étant le niveau de compression de attic.
147 9 Laurent GUERBY
148 10 Laurent GUERBY
https://github.com/jborg/attic/issues/299
149 9 Laurent GUERBY
http://borgbackup.readthedocs.org/en/stable/usage.html#environment-variables
150 9 Laurent GUERBY
<pre>
151 9 Laurent GUERBY
export ATTIC_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=yes
152 1 Laurent GUERBY
</pre>