Projet

Général

Profil

Backup » Historique » Version 19

Nicolas BERTRAND, 11/01/2017 16:29

1 3 Laurent GUERBY
{{>toc}}
2 1 Laurent GUERBY
3 1 Laurent GUERBY
h1. Backup
4 1 Laurent GUERBY
5 1 Laurent GUERBY
h2. Liens
6 1 Laurent GUERBY
7 1 Laurent GUERBY
* [[Backup_infra]]
8 1 Laurent GUERBY
* [[Apt_Backports_TTNN]] borg debian packaging
9 6 Laurent GUERBY
* https://www.reddit.com/r/linux/comments/42feqz/i_asked_here_for_the_optimal_backup_solution_and/
10 1 Laurent GUERBY
11 1 Laurent GUERBY
h2. BackupPC
12 1 Laurent GUERBY
13 1 Laurent GUERBY
* http://backuppc.sourceforge.net/
14 1 Laurent GUERBY
15 1 Laurent GUERBY
h2. Attic
16 1 Laurent GUERBY
17 1 Laurent GUERBY
* https://attic-backup.org/
18 1 Laurent GUERBY
* https://lists.tetaneutral.net/pipermail/technique/2015-September/001971.html
19 1 Laurent GUERBY
** Logiciel de backup : choix de attic
20 1 Laurent GUERBY
21 1 Laurent GUERBY
h2. Borg
22 1 Laurent GUERBY
23 1 Laurent GUERBY
* https://github.com/borgbackup
24 1 Laurent GUERBY
* http://readthedocs.org/projects/borgbackup/
25 2 Laurent GUERBY
* http://puppet.tetaneutral.net/pool/main/b/borgbackup/
26 2 Laurent GUERBY
* http://puppet.tetaneutral.net/dists/
27 1 Laurent GUERBY
28 11 Laurent GUERBY
h2. Borg script
29 11 Laurent GUERBY
30 18 Laurent GUERBY
*Note 20160428* : le script est probablement inutile cf https://github.com/borgbackup/borg/issues/994
31 17 Laurent GUERBY
32 11 Laurent GUERBY
Pour une machine qui va etre eteinte et rallumée de maniere non controlable.
33 11 Laurent GUERBY
34 12 Laurent GUERBY
Creation initiale avec un user normal capable de ssh sur MACHINE:PORT :
35 12 Laurent GUERBY
36 12 Laurent GUERBY
<pre>
37 12 Laurent GUERBY
borg init --encryption keyfile ssh://USER@MACHINE:PORT/some/where/borg/NICK-repo
38 12 Laurent GUERBY
</pre>
39 12 Laurent GUERBY
40 19 Nicolas BERTRAND
On A 2 types d'encryption keyfile et repokey
41 19 Nicolas BERTRAND
* en keyfile: la clef est stockée 'localement' est doit aussi être backupé. mode "passphrase + key"
42 19 Nicolas BERTRAND
* repokey : la clef est stocké dans le repo only: mode "passphrase only"
43 19 Nicolas BERTRAND
<pre>
44 19 Nicolas BERTRAND
11.1026 < guerby> [09:39:25] zorun, tu as une ref dans la doc ? ce que j'ai trouvé http://borgbackup.readthedocs.io/en/stable/quickstart.html#encrypted-repos
45 19 Nicolas BERTRAND
11.1026 < guerby> [09:39:38] "so you still have the key in case it gets corrupted or lost. Also keep your passphrase at a safe place."
46 19 Nicolas BERTRAND
11.1026 < zorun> [09:50:10] guerby: cherche « repokey » dans https://borgbackup.readthedocs.io/en/stable/usage.html
47 19 Nicolas BERTRAND
11.1026 < zorun> [09:50:19] # Local repository (default is to use encryption in repokey mode)
48 19 Nicolas BERTRAND
11.1026 < zorun> [09:50:27] If you want “passphrase-only” security, use the repokey mode. The key will be stored inside the repository (in its “config” file). In above mentioned attack scenario, the attacker will have the key (but not the passphrase).
49 19 Nicolas BERTRAND
11.1026 < zorun> [09:50:43] If you want “passphrase and having-the-key” security, use the keyfile mode. The key will be stored in your home directory (in .config/borg/keys). In the attack scenario, the attacker who has just access to your repo won’t have the key (and also not the passphrase).
50 19 Nicolas BERTRAND
11.1026 < taziden> [09:51:21] et la méthode par défaut, c'est repokey
51 19 Nicolas BERTRAND
</pre>
52 19 Nicolas BERTRAND
  
53 19 Nicolas BERTRAND
54 12 Laurent GUERBY
Et setup cron + script :
55 11 Laurent GUERBY
<pre>
56 11 Laurent GUERBY
# crontab -l
57 11 Laurent GUERBY
@reboot /root/cron-borg.sh
58 11 Laurent GUERBY
59 11 Laurent GUERBY
60 11 Laurent GUERBY
61 11 Laurent GUERBY
# cat /root/cron-borg.sh
62 11 Laurent GUERBY
#!/bin/bash
63 11 Laurent GUERBY
export LANG=en_US.UTF-8
64 11 Laurent GUERBY
mkdir -p /root/borg >& /dev/null
65 11 Laurent GUERBY
66 11 Laurent GUERBY
sleep 300
67 11 Laurent GUERBY
echo === start === $(date) >> /root/borg/cron.log
68 11 Laurent GUERBY
69 11 Laurent GUERBY
NICK=myhost
70 11 Laurent GUERBY
REPO=ssh://USER@MACHINE:PORT/some/where/borg/${NICK}-repo
71 11 Laurent GUERBY
export BORG_PASSPHRASE=lalalala
72 11 Laurent GUERBY
73 11 Laurent GUERBY
if [ -f /root/borg/stamp ]; then
74 11 Laurent GUERBY
    STAMP=$(cat /root/borg/stamp)
75 11 Laurent GUERBY
    borg break-lock $REPO
76 11 Laurent GUERBY
else
77 11 Laurent GUERBY
    STAMP=$(date '+%Y%m%dT%H%M%S')
78 11 Laurent GUERBY
    if [ -f /root/borg/previous-stamp ]; then
79 1 Laurent GUERBY
	  PREVIOUS_STAMP=$(cat /root/borg/previous-stamp)
80 14 Laurent GUERBY
	  while [ "${STAMP%T*}" = "${PREVIOUS_STAMP%T*}" ]; do
81 14 Laurent GUERBY
              STAMP=$(date '+%Y%m%dT%H%M%S')
82 13 Laurent GUERBY
              echo === delay === $(date) >> /root/borg/cron.log
83 14 Laurent GUERBY
	      sleep 1h
84 14 Laurent GUERBY
	  done
85 11 Laurent GUERBY
    fi
86 11 Laurent GUERBY
    echo $STAMP > /root/borg/stamp
87 11 Laurent GUERBY
fi
88 11 Laurent GUERBY
89 11 Laurent GUERBY
90 16 Laurent GUERBY
borg create --compression lz4 --stats --verbose \
91 16 Laurent GUERBY
   --exclude /root/borg --exclude '/home/*/.cache' --exclude-caches --one-file-system \
92 16 Laurent GUERBY
   ${REPO}::${NICK}-$STAMP / >> /root/borg/log-$STAMP 2>> /root/borg/err-$STAMP
93 1 Laurent GUERBY
94 16 Laurent GUERBY
res=$?
95 16 Laurent GUERBY
96 16 Laurent GUERBY
if [ $res -eq 0 -o $res eq 1 ]; then
97 11 Laurent GUERBY
    mv -f /root/borg/stamp /root/borg/previous-stamp >& /dev/null
98 11 Laurent GUERBY
    rm -f /root/borg/stamp >& /dev/null
99 1 Laurent GUERBY
fi
100 1 Laurent GUERBY
101 1 Laurent GUERBY
102 16 Laurent GUERBY
echo === done === $res === $(date) >> /root/borg/cron.log
103 14 Laurent GUERBY
104 14 Laurent GUERBY
exec "$0"
105 11 Laurent GUERBY
</pre>
106 1 Laurent GUERBY
107 1 Laurent GUERBY
h2. Migration Attic vers Borg
108 7 Laurent GUERBY
109 4 Mehdi Abaakouk
* https://github.com/borgbackup/borg/pull/231
110 4 Mehdi Abaakouk
* old: https://chiliproject.tetaneutral.net/projects/git-tetaneutral-net/repository/puppet-backup
111 1 Laurent GUERBY
* new: https://chiliproject.tetaneutral.net/projects/git-tetaneutral-net/repository/puppetmaster/revisions/master/entry/modules/ttnn/manifests/backup.pp
112 1 Laurent GUERBY
** git history BackupPC => Attic => Borg
113 4 Mehdi Abaakouk
114 5 Laurent GUERBY
<pre>
115 4 Mehdi Abaakouk
# apt-get -t jessie-backports install borgbackup
116 4 Mehdi Abaakouk
$ cd /backup/attic/
117 4 Mehdi Abaakouk
$ borg upgrade <repo>
118 4 Mehdi Abaakouk
$ borg check --repair <repo>
119 4 Mehdi Abaakouk
$ mv <repo> ../borg/
120 4 Mehdi Abaakouk
$ chown -R backupinfra: /backup/borg/<repo>
121 4 Mehdi Abaakouk
</pre>
122 8 Laurent GUERBY
123 4 Mehdi Abaakouk
Dans le module puppet, le changement le plus important est le parametre compression explicite pour correspondre au défaut de attic create :
124 4 Mehdi Abaakouk
125 8 Laurent GUERBY
<pre>
126 4 Mehdi Abaakouk
attic create ... <repo> -> borg create --compression zlib,6 ... <repo>
127 4 Mehdi Abaakouk
</pre>
128 4 Mehdi Abaakouk
129 9 Laurent GUERBY
Sinon le prochain backup sera non compressé, et aucun nouveau chucks ne correspondra aux anciens -> perte de la dedup. "zlib,6" étant le niveau de compression de attic.
130 9 Laurent GUERBY
131 10 Laurent GUERBY
https://github.com/jborg/attic/issues/299
132 9 Laurent GUERBY
http://borgbackup.readthedocs.org/en/stable/usage.html#environment-variables
133 9 Laurent GUERBY
<pre>
134 9 Laurent GUERBY
export ATTIC_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=yes
135 1 Laurent GUERBY
</pre>