Management Cluster Ceph » Historique » Version 38
Mehdi Abaakouk, 05/09/2018 14:12
1 | 10 | Mehdi Abaakouk | {{>toc}} |
---|---|---|---|
2 | 1 | Mehdi Abaakouk | |
3 | 10 | Mehdi Abaakouk | h1. Management Cluster Ceph |
4 | 9 | Mehdi Abaakouk | |
5 | 8 | Mehdi Abaakouk | h2. Liens |
6 | 8 | Mehdi Abaakouk | |
7 | 8 | Mehdi Abaakouk | * [[Openstack Management TTNN]] |
8 | 8 | Mehdi Abaakouk | * [[Openstack Setup VM pas dans openstack]] |
9 | 8 | Mehdi Abaakouk | * [[Openstack Installation nouvelle node du cluster]] |
10 | 8 | Mehdi Abaakouk | * [[Openstack Installation TTNN]] |
11 | 8 | Mehdi Abaakouk | * "Openstack tools for ttnn":/projects/git-tetaneutral-net/repository/openstack-tools |
12 | 8 | Mehdi Abaakouk | |
13 | 2 | Mehdi Abaakouk | h2. Ajout d'un OSD classique |
14 | 2 | Mehdi Abaakouk | |
15 | 20 | Mehdi Abaakouk | |
16 | 20 | Mehdi Abaakouk | |
17 | 20 | Mehdi Abaakouk | <pre> |
18 | 1 | Mehdi Abaakouk | $ ceph-disk prepare --zap-disk --cluster-uuid 1fe74663-8dfa-486c-bb80-3bd94c90c967 --fs-type=ext4 /dev/sdX |
19 | 34 | Laurent GUERBY | $ tune2fs -c 0 -i 0 -m 0 /dev/sdX1 |
20 | 1 | Mehdi Abaakouk | $ smartctl --smart=on /dev/sdX # Pour le monitoring. |
21 | 1 | Mehdi Abaakouk | </pre> |
22 | 1 | Mehdi Abaakouk | |
23 | 1 | Mehdi Abaakouk | Récuperer l'id avec (c'est celui tout en bas pas accroché à l'arbre): |
24 | 1 | Mehdi Abaakouk | |
25 | 1 | Mehdi Abaakouk | <pre> |
26 | 1 | Mehdi Abaakouk | ceph osd tree |
27 | 1 | Mehdi Abaakouk | </pre> |
28 | 1 | Mehdi Abaakouk | |
29 | 26 | Mehdi Abaakouk | *DEBUT WORKAROUND BUG PREPARE* |
30 | 26 | Mehdi Abaakouk | |
31 | 26 | Mehdi Abaakouk | Dans le cas ou l'osd est DOWN après le prepare c'est surement ce bug |
32 | 26 | Mehdi Abaakouk | |
33 | 26 | Mehdi Abaakouk | ID est le premier numero libre d'osd en partant de zero (en bas du ceph osd tree) |
34 | 26 | Mehdi Abaakouk | |
35 | 26 | Mehdi Abaakouk | <pre> |
36 | 26 | Mehdi Abaakouk | mkdir /var/lib/ceph/osd/ceph-<ID> |
37 | 26 | Mehdi Abaakouk | chown ceph:ceph /var/lib/ceph/osd/ceph-<ID> |
38 | 26 | Mehdi Abaakouk | ceph-disk activate /dev/sd<X>1 |
39 | 26 | Mehdi Abaakouk | systemctl status ceph-osd@<ID> |
40 | 26 | Mehdi Abaakouk | </pre> |
41 | 26 | Mehdi Abaakouk | |
42 | 26 | Mehdi Abaakouk | *FIN WORKAROUND BUG PREPARE* |
43 | 26 | Mehdi Abaakouk | |
44 | 17 | Mehdi Abaakouk | Pour un HDD: |
45 | 2 | Mehdi Abaakouk | <pre> |
46 | 2 | Mehdi Abaakouk | $ ceph osd crush add osd.<ID> 0 root=default host=<host> |
47 | 1 | Mehdi Abaakouk | </pre> |
48 | 2 | Mehdi Abaakouk | |
49 | 17 | Mehdi Abaakouk | Pour un SSD: |
50 | 2 | Mehdi Abaakouk | <pre> |
51 | 2 | Mehdi Abaakouk | $ ceph osd crush add osd.<ID> 0 root=ssd host=<host>-ssd |
52 | 2 | Mehdi Abaakouk | </pre> |
53 | 2 | Mehdi Abaakouk | |
54 | 2 | Mehdi Abaakouk | Ensuite, autoriser Ceph à mettre des data dessus: |
55 | 2 | Mehdi Abaakouk | |
56 | 2 | Mehdi Abaakouk | <pre> |
57 | 2 | Mehdi Abaakouk | $ /root/tools/ceph-reweight-osds.sh osd.<ID> |
58 | 19 | Mehdi Abaakouk | </pre> |
59 | 19 | Mehdi Abaakouk | |
60 | 35 | Mehdi Abaakouk | h3. helper |
61 | 35 | Mehdi Abaakouk | |
62 | 36 | Mehdi Abaakouk | <pre> |
63 | 35 | Mehdi Abaakouk | add_osd(){ |
64 | 35 | Mehdi Abaakouk | dev="$1" |
65 | 35 | Mehdi Abaakouk | type="$2" |
66 | 35 | Mehdi Abaakouk | host=$(hostname -s) |
67 | 35 | Mehdi Abaakouk | [ "$type" == "ssd" ] && host="${host}-ssd" |
68 | 35 | Mehdi Abaakouk | next=-1 ; for i in $(ceph osd ls); do next=$((next+1)) ; [ $(($i - $next)) -gt 0 ] && break; done ; |
69 | 35 | Mehdi Abaakouk | mkdir /var/lib/ceph/osd/ceph-$next |
70 | 35 | Mehdi Abaakouk | chown ceph:ceph /var/lib/ceph/osd/ceph-$next |
71 | 35 | Mehdi Abaakouk | |
72 | 35 | Mehdi Abaakouk | ceph-disk prepare --zap-disk --cluster-uuid 1fe74663-8dfa-486c-bb80-3bd94c90c967 --fs-type=ext4 $dev |
73 | 37 | Mehdi Abaakouk | tune2fs -c 0 -i 0 -m 0 ${dev}1 |
74 | 1 | Mehdi Abaakouk | smartctl --smart=on $dev |
75 | 37 | Mehdi Abaakouk | |
76 | 37 | Mehdi Abaakouk | systemctl start ceph-osd@$next |
77 | 35 | Mehdi Abaakouk | systemctl status ceph-osd@$next |
78 | 35 | Mehdi Abaakouk | |
79 | 38 | Mehdi Abaakouk | sleep 1 |
80 | 35 | Mehdi Abaakouk | ceph osd crush add osd.$next 0 root=${type} host=${host} |
81 | 35 | Mehdi Abaakouk | } |
82 | 36 | Mehdi Abaakouk | </pre> |
83 | 35 | Mehdi Abaakouk | |
84 | 19 | Mehdi Abaakouk | h2. Vider un OSD: |
85 | 19 | Mehdi Abaakouk | |
86 | 1 | Mehdi Abaakouk | <pre> |
87 | 35 | Mehdi Abaakouk | vider_osd(){ |
88 | 19 | Mehdi Abaakouk | name="$1" |
89 | 19 | Mehdi Abaakouk | ceph osd out ${name} |
90 | 19 | Mehdi Abaakouk | ceph osd crush reweight ${name} 0 |
91 | 19 | Mehdi Abaakouk | ceph osd reweight ${name} 0 |
92 | 19 | Mehdi Abaakouk | } |
93 | 19 | Mehdi Abaakouk | </pre> |
94 | 19 | Mehdi Abaakouk | |
95 | 19 | Mehdi Abaakouk | h2. Suppression d'un OSD: |
96 | 19 | Mehdi Abaakouk | |
97 | 19 | Mehdi Abaakouk | <pre> |
98 | 19 | Mehdi Abaakouk | remove_osd(){ |
99 | 19 | Mehdi Abaakouk | name="$1" |
100 | 19 | Mehdi Abaakouk | ceph osd out ${name} |
101 | 19 | Mehdi Abaakouk | systemctl stop ceph-osd@${name#osd.} |
102 | 19 | Mehdi Abaakouk | ceph osd crush remove ${name} |
103 | 19 | Mehdi Abaakouk | ceph auth del ${name} |
104 | 19 | Mehdi Abaakouk | ceph osd rm ${name} |
105 | 19 | Mehdi Abaakouk | ceph osd tree |
106 | 19 | Mehdi Abaakouk | } |
107 | 19 | Mehdi Abaakouk | </pre> |
108 | 19 | Mehdi Abaakouk | |
109 | 19 | Mehdi Abaakouk | h2. Arrêter les IO de recovery: |
110 | 19 | Mehdi Abaakouk | |
111 | 19 | Mehdi Abaakouk | <pre> |
112 | 19 | Mehdi Abaakouk | ceph osd set nobackfill |
113 | 19 | Mehdi Abaakouk | ceph osd set norebalance |
114 | 19 | Mehdi Abaakouk | ceph osd set norecover |
115 | 19 | Mehdi Abaakouk | </pre> |
116 | 19 | Mehdi Abaakouk | |
117 | 23 | Mehdi Abaakouk | h2. Procédure d'upgrade |
118 | 23 | Mehdi Abaakouk | |
119 | 23 | Mehdi Abaakouk | + |
120 | 23 | Mehdi Abaakouk | _*/!\Lire la release note (contient très très souvent des trucs à faire en plus) /!\*_+ |
121 | 23 | Mehdi Abaakouk | |
122 | 23 | Mehdi Abaakouk | |
123 | 23 | Mehdi Abaakouk | h4. Upgrade des MONs: |
124 | 23 | Mehdi Abaakouk | |
125 | 23 | Mehdi Abaakouk | Mettre le flags noout: |
126 | 23 | Mehdi Abaakouk | |
127 | 23 | Mehdi Abaakouk | <pre>ceph osd set noout</pre> |
128 | 23 | Mehdi Abaakouk | |
129 | 23 | Mehdi Abaakouk | Sur chaque MONs (g1/g2/g3) |
130 | 23 | Mehdi Abaakouk | <pre> |
131 | 23 | Mehdi Abaakouk | apt-get upgrade -y |
132 | 23 | Mehdi Abaakouk | systemctl restart ceph-mon@g* |
133 | 23 | Mehdi Abaakouk | ceph -s |
134 | 23 | Mehdi Abaakouk | </pre> |
135 | 23 | Mehdi Abaakouk | |
136 | 23 | Mehdi Abaakouk | Note que seulement le node 'leader/master' va provoquer une micro/nano coupure, souvent c'est même invisible. |
137 | 23 | Mehdi Abaakouk | |
138 | 23 | Mehdi Abaakouk | h4. Upgrade des OSDs: |
139 | 23 | Mehdi Abaakouk | |
140 | 23 | Mehdi Abaakouk | Pour chaque machine |
141 | 23 | Mehdi Abaakouk | <pre> |
142 | 23 | Mehdi Abaakouk | apt-get upgrade -y |
143 | 23 | Mehdi Abaakouk | systemctl restart ceph-osd@* |
144 | 23 | Mehdi Abaakouk | </pre> |
145 | 23 | Mehdi Abaakouk | |
146 | 23 | Mehdi Abaakouk | Puis attendre que le recovery termine avant de faire la suivante. |
147 | 23 | Mehdi Abaakouk | |
148 | 23 | Mehdi Abaakouk | Une fois toutes les OSDs upgrader et relancer, faire: |
149 | 23 | Mehdi Abaakouk | |
150 | 23 | Mehdi Abaakouk | <pre>ceph osd unset noout</pre> |
151 | 23 | Mehdi Abaakouk | |
152 | 19 | Mehdi Abaakouk | h2. Remplacement à froid d'un tier cache: |
153 | 19 | Mehdi Abaakouk | |
154 | 19 | Mehdi Abaakouk | upstream doc: http://docs.ceph.com/docs/master/rados/operations/cache-tiering/ |
155 | 19 | Mehdi Abaakouk | |
156 | 19 | Mehdi Abaakouk | <pre> |
157 | 19 | Mehdi Abaakouk | ceph osd tier cache-mode ec8p2c forward |
158 | 19 | Mehdi Abaakouk | rados -p ec8p2c cache-flush-evict-all |
159 | 19 | Mehdi Abaakouk | ceph osd tier remove-overlay ec8p2 |
160 | 19 | Mehdi Abaakouk | ceph osd tier remove ec8p2 ec8p2c |
161 | 19 | Mehdi Abaakouk | |
162 | 19 | Mehdi Abaakouk | rados rmpool ec8p2c ec8p2c --yes-i-really-really-mean-ita |
163 | 19 | Mehdi Abaakouk | ceph osd pool create ec8p2c 128 128 replicated |
164 | 19 | Mehdi Abaakouk | |
165 | 19 | Mehdi Abaakouk | ceph osd tier add ec8p2 ec8p2c |
166 | 19 | Mehdi Abaakouk | ceph osd tier cache-mode ec8p2c writeback |
167 | 19 | Mehdi Abaakouk | ceph osd tier set-overlay ec8p2 ec8p2c |
168 | 19 | Mehdi Abaakouk | |
169 | 19 | Mehdi Abaakouk | ceph osd pool set ec8p2c size 3 |
170 | 19 | Mehdi Abaakouk | ceph osd pool set ec8p2c min_size 2 |
171 | 19 | Mehdi Abaakouk | ceph osd pool set ec8p2c hit_set_type bloom |
172 | 19 | Mehdi Abaakouk | |
173 | 19 | Mehdi Abaakouk | ceph osd pool set ec8p2c hit_set_count 1 |
174 | 19 | Mehdi Abaakouk | ceph osd pool set ec8p2c hit_set_period 3600 |
175 | 19 | Mehdi Abaakouk | ceph osd pool set ec8p2c target_max_bytes 200000000000 |
176 | 19 | Mehdi Abaakouk | ceph osd pool set ec8p2c target_max_objects 10000000 |
177 | 19 | Mehdi Abaakouk | ceph osd pool set ec8p2c cache_target_dirty_ratio 0.4 |
178 | 19 | Mehdi Abaakouk | ceph osd pool set ec8p2c cache_target_full_ratio 0.8 |
179 | 19 | Mehdi Abaakouk | </pre> |
180 | 19 | Mehdi Abaakouk | |
181 | 16 | Mehdi Abaakouk | h2. Ajout d'un OSD qui partage le SSD avec l'OS (OBSOLETE PLUS COMPATIBLE AVEC LES FUTURES VERSION DE CEPH) |
182 | 2 | Mehdi Abaakouk | |
183 | 2 | Mehdi Abaakouk | |
184 | 2 | Mehdi Abaakouk | En général avec ceph, on donne un disque, ceph créé 2 partitions une pour le journal de l'OSD, l'autre pour les datas |
185 | 2 | Mehdi Abaakouk | mais pour le SSD de tetaneutral qui a aussi l'OS, voici la méthode |
186 | 2 | Mehdi Abaakouk | |
187 | 2 | Mehdi Abaakouk | Création manuelle de la partition de data ceph /dev/sda2 ici |
188 | 7 | Mehdi Abaakouk | |
189 | 7 | Mehdi Abaakouk | Debian (MBR format): |
190 | 2 | Mehdi Abaakouk | <pre> |
191 | 5 | Mehdi Abaakouk | apt-get install partprobe |
192 | 5 | Mehdi Abaakouk | fdisk /dev/sda |
193 | 5 | Mehdi Abaakouk | |
194 | 2 | Mehdi Abaakouk | n |
195 | 14 | Mehdi Abaakouk | p |
196 | 14 | Mehdi Abaakouk | <enter> |
197 | 14 | Mehdi Abaakouk | <enter> |
198 | 14 | Mehdi Abaakouk | <enter> |
199 | 14 | Mehdi Abaakouk | <enter> |
200 | 14 | Mehdi Abaakouk | w |
201 | 14 | Mehdi Abaakouk | |
202 | 14 | Mehdi Abaakouk | $ partprobe |
203 | 14 | Mehdi Abaakouk | </pre> |
204 | 14 | Mehdi Abaakouk | |
205 | 14 | Mehdi Abaakouk | Ubuntu (GPT format): |
206 | 2 | Mehdi Abaakouk | <pre> |
207 | 2 | Mehdi Abaakouk | # parted /dev/sdb |
208 | 2 | Mehdi Abaakouk | GNU Parted 2.3 |
209 | 13 | Mehdi Abaakouk | Using /dev/sdb |
210 | 13 | Mehdi Abaakouk | Welcome to GNU Parted! Type 'help' to view a list of commands. |
211 | 13 | Mehdi Abaakouk | (parted) print |
212 | 18 | Mehdi Abaakouk | Model: ATA SAMSUNG MZ7KM480 (scsi) |
213 | 13 | Mehdi Abaakouk | Disk /dev/sdb: 480GB |
214 | 13 | Mehdi Abaakouk | Sector size (logical/physical): 512B/512B |
215 | 13 | Mehdi Abaakouk | Partition Table: msdos |
216 | 13 | Mehdi Abaakouk | |
217 | 13 | Mehdi Abaakouk | Number Start End Size Type File system Flags |
218 | 2 | Mehdi Abaakouk | 1 1049kB 20.0GB 20.0GB primary ext4 boot |
219 | 2 | Mehdi Abaakouk | 2 20.0GB 36.0GB 16.0GB primary linux-swap(v1) |
220 | 15 | Mehdi Abaakouk | |
221 | 15 | Mehdi Abaakouk | (parted) mkpart |
222 | 15 | Mehdi Abaakouk | Partition type? primary/extended? |
223 | 15 | Mehdi Abaakouk | Partition type? primary/extended? primary |
224 | 15 | Mehdi Abaakouk | File system type? [ext2]? xfs |
225 | 15 | Mehdi Abaakouk | Start? |
226 | 15 | Mehdi Abaakouk | Start? 36.0GB |
227 | 15 | Mehdi Abaakouk | End? 100% |
228 | 1 | Mehdi Abaakouk | (parted) print |
229 | 1 | Mehdi Abaakouk | Model: ATA SAMSUNG MZ7KM480 (scsi) |
230 | 1 | Mehdi Abaakouk | Disk /dev/sdb: 480GB |
231 | 1 | Mehdi Abaakouk | Sector size (logical/physical): 512B/512B |
232 | 1 | Mehdi Abaakouk | Partition Table: msdos |
233 | 1 | Mehdi Abaakouk | |
234 | 1 | Mehdi Abaakouk | Number Start End Size Type File system Flags |
235 | 1 | Mehdi Abaakouk | 1 1049kB 20.0GB 20.0GB primary ext4 boot |
236 | 1 | Mehdi Abaakouk | 2 20.0GB 36.0GB 16.0GB primary linux-swap(v1) |
237 | 1 | Mehdi Abaakouk | 3 36.0GB 480GB 444GB primary |
238 | 1 | Mehdi Abaakouk | |
239 | 1 | Mehdi Abaakouk | (parted) quit |
240 | 1 | Mehdi Abaakouk | Information: You may need to update /etc/fstab. |
241 | 1 | Mehdi Abaakouk | </pre> |
242 | 1 | Mehdi Abaakouk | |
243 | 1 | Mehdi Abaakouk | On prepare le disk comme normalement |
244 | 1 | Mehdi Abaakouk | |
245 | 1 | Mehdi Abaakouk | <pre> |
246 | 1 | Mehdi Abaakouk | ceph-disk prepare --fs-type=ext4 --cluster-uuid 1fe74663-8dfa-486c-bb80-3bd94c90c967 /dev/sda2 |
247 | 1 | Mehdi Abaakouk | ceph-disk activate /dev/sda2 |
248 | 1 | Mehdi Abaakouk | ceph osd crush add osd.<ID> 0 root=ssd host=g3-ssd |
249 | 1 | Mehdi Abaakouk | </pre> |
250 | 1 | Mehdi Abaakouk | |
251 | 1 | Mehdi Abaakouk | Ensuite, autoriser Ceph à mettre des data dessus: |
252 | 1 | Mehdi Abaakouk | |
253 | 1 | Mehdi Abaakouk | <pre> |
254 | 1 | Mehdi Abaakouk | $ /root/tools/ceph-reweight-osds.sh osd.<ID> |
255 | 1 | Mehdi Abaakouk | </pre> |
256 | 28 | Laurent GUERBY | |
257 | 28 | Laurent GUERBY | h2. inconsistent pg |
258 | 28 | Laurent GUERBY | |
259 | 32 | Laurent GUERBY | * Analyse d'une erreur de coherence detectee par ceph |
260 | 32 | Laurent GUERBY | ** https://lists.tetaneutral.net/pipermail/technique/2017-August/002859.html |
261 | 32 | Laurent GUERBY | |
262 | 28 | Laurent GUERBY | <pre> |
263 | 28 | Laurent GUERBY | root@g1:~# ceph health detail |
264 | 28 | Laurent GUERBY | HEALTH_ERR 1 pgs inconsistent; 2 scrub errors |
265 | 28 | Laurent GUERBY | pg 58.22d is active+clean+inconsistent, acting [9,47,37] |
266 | 28 | Laurent GUERBY | 2 scrub errors |
267 | 28 | Laurent GUERBY | root@g1:~# rados list-inconsistent-obj 58.22d --format=json-pretty |
268 | 28 | Laurent GUERBY | { |
269 | 28 | Laurent GUERBY | "epoch": 269000, |
270 | 28 | Laurent GUERBY | "inconsistents": [ |
271 | 28 | Laurent GUERBY | { |
272 | 28 | Laurent GUERBY | "object": { |
273 | 28 | Laurent GUERBY | "name": "rbd_data.11f20f75aac8266.00000000000f79f9", |
274 | 28 | Laurent GUERBY | "nspace": "", |
275 | 28 | Laurent GUERBY | "locator": "", |
276 | 28 | Laurent GUERBY | "snap": "head", |
277 | 28 | Laurent GUERBY | "version": 9894452 |
278 | 28 | Laurent GUERBY | }, |
279 | 28 | Laurent GUERBY | "errors": [ |
280 | 28 | Laurent GUERBY | "data_digest_mismatch" |
281 | 28 | Laurent GUERBY | ], |
282 | 28 | Laurent GUERBY | "union_shard_errors": [ |
283 | 28 | Laurent GUERBY | "data_digest_mismatch_oi" |
284 | 28 | Laurent GUERBY | ], |
285 | 31 | Laurent GUERBY | "selected_object_info": |
286 | 31 | Laurent GUERBY | "58:b453643a:::rbd_data.11f20f75aac8266.00000000000f79f9:head(261163'9281748 osd.9.0:6221608 dirty|data_digest|omap_digest s 4194304 uv 9894452 dd 2193d055 od ffffffff alloc_hint [0 0])", |
287 | 28 | Laurent GUERBY | "shards": [ |
288 | 28 | Laurent GUERBY | { |
289 | 28 | Laurent GUERBY | "osd": 9, |
290 | 28 | Laurent GUERBY | "errors": [], |
291 | 28 | Laurent GUERBY | "size": 4194304, |
292 | 28 | Laurent GUERBY | "omap_digest": "0xffffffff", |
293 | 28 | Laurent GUERBY | "data_digest": "0x2193d055" |
294 | 28 | Laurent GUERBY | }, |
295 | 28 | Laurent GUERBY | { |
296 | 28 | Laurent GUERBY | "osd": 37, |
297 | 28 | Laurent GUERBY | "errors": [ |
298 | 28 | Laurent GUERBY | "data_digest_mismatch_oi" |
299 | 28 | Laurent GUERBY | ], |
300 | 28 | Laurent GUERBY | "size": 4194304, |
301 | 28 | Laurent GUERBY | "omap_digest": "0xffffffff", |
302 | 28 | Laurent GUERBY | "data_digest": "0x05891fb4" |
303 | 28 | Laurent GUERBY | }, |
304 | 28 | Laurent GUERBY | { |
305 | 28 | Laurent GUERBY | "osd": 47, |
306 | 28 | Laurent GUERBY | "errors": [], |
307 | 28 | Laurent GUERBY | "size": 4194304, |
308 | 28 | Laurent GUERBY | "omap_digest": "0xffffffff", |
309 | 28 | Laurent GUERBY | "data_digest": "0x2193d055" |
310 | 28 | Laurent GUERBY | } |
311 | 28 | Laurent GUERBY | ] |
312 | 28 | Laurent GUERBY | } |
313 | 28 | Laurent GUERBY | ] |
314 | 28 | Laurent GUERBY | } |
315 | 29 | Laurent GUERBY | root@g1:~# ceph osd map disks rbd_data.11f20f75aac8266.00000000000f79f9 |
316 | 29 | Laurent GUERBY | osdmap e269110 pool 'disks' (58) object 'rbd_data.11f20f75aac8266.00000000000f79f9' -> pg 58.5c26ca2d (58.22d) -> up ([9,47,37], p9) acting ([9,47,37], p9) |
317 | 29 | Laurent GUERBY | |
318 | 30 | Laurent GUERBY | |
319 | 30 | Laurent GUERBY | root@g8:/var/lib/ceph/osd/ceph-9/current/58.22d_head# find . -name '*11f20f75aac8266.00000000000f79f9*' |
320 | 30 | Laurent GUERBY | ./DIR_D/DIR_2/DIR_A/DIR_C/rbd\udata.11f20f75aac8266.00000000000f79f9__head_5C26CA2D__3a |
321 | 30 | Laurent GUERBY | |
322 | 30 | Laurent GUERBY | root@g10:/var/lib/ceph/osd/ceph-37/current/58.22d_head# find . -name '*11f20f75aac8266.00000000000f79f9*' |
323 | 30 | Laurent GUERBY | ./DIR_D/DIR_2/DIR_A/DIR_C/rbd\udata.11f20f75aac8266.00000000000f79f9__head_5C26CA2D__3a |
324 | 30 | Laurent GUERBY | |
325 | 30 | Laurent GUERBY | $ scp g8:/var/lib/ceph/osd/ceph-9/current/58.22d_head/DIR_D/DIR_2/DIR_A/DIR_C/rbd*data.11f20f75aac8266.00000000000f79f9__head_5C26CA2D__3a g8data |
326 | 30 | Laurent GUERBY | $ scp g10:/var/lib/ceph/osd/ceph-37/current/58.22d_head/DIR_D/DIR_2/DIR_A/DIR_C/rbd*data.11f20f75aac8266.00000000000f79f9__head_5C26CA2D__3a g10data |
327 | 30 | Laurent GUERBY | |
328 | 30 | Laurent GUERBY | $ md5sum * |
329 | 30 | Laurent GUERBY | bd85c0ef1f30829ce07e5f9152ac2d2f g10data |
330 | 30 | Laurent GUERBY | 4297d0bc373e6603e0ad842702e0ecaa g8data |
331 | 30 | Laurent GUERBY | $ $ diff -u <(od -x g10data) <(od -x g8data) |
332 | 30 | Laurent GUERBY | --- /dev/fd/63 2017-08-13 10:43:52.837097740 +0200 |
333 | 30 | Laurent GUERBY | +++ /dev/fd/62 2017-08-13 10:43:52.833097808 +0200 |
334 | 30 | Laurent GUERBY | @@ -2617,7 +2617,7 @@ |
335 | 30 | Laurent GUERBY | 0121600 439b 14f4 bb4c 5f14 6ff7 4393 9ff8 a9a9 |
336 | 30 | Laurent GUERBY | 0121620 29a8 56a4 1133 b6a8 2206 4821 2f42 4b2c |
337 | 30 | Laurent GUERBY | 0121640 3d86 41a2 785f 9785 8b48 4243 e7b9 f0aa |
338 | 30 | Laurent GUERBY | -0121660 29b6 be0c 0455 bf97 1c0d 49e5 75dd e1ed |
339 | 30 | Laurent GUERBY | +0121660 29a6 be0c 0455 bf97 1c0d 49e5 75dd e1ed |
340 | 30 | Laurent GUERBY | 0121700 2519 d6ac 1047 1111 0344 38be 27a1 db07 |
341 | 30 | Laurent GUERBY | 0121720 dff6 c002 75d8 4396 6154 eba9 3abd 5d20 |
342 | 30 | Laurent GUERBY | 0121740 8ae4 e63a 298b d754 0208 9705 1bb8 3685 |
343 | 30 | Laurent GUERBY | </pre> |
344 | 30 | Laurent GUERBY | |
345 | 30 | Laurent GUERBY | Donc un seul bit flip 29b6 vs 29a6 |
346 | 30 | Laurent GUERBY | |
347 | 30 | Laurent GUERBY | <pre> |
348 | 30 | Laurent GUERBY | >>> bin(0xa) |
349 | 30 | Laurent GUERBY | '0b1010' |
350 | 30 | Laurent GUERBY | >>> bin(0xb) |
351 | 30 | Laurent GUERBY | '0b1011' |
352 | 28 | Laurent GUERBY | </pre> |
353 | 29 | Laurent GUERBY | |
354 | 29 | Laurent GUERBY | * http://cephnotes.ksperis.com/blog/2013/08/20/ceph-osd-where-is-my-data |
355 | 33 | Laurent GUERBY | * https://superuser.com/questions/969889/what-is-the-granularity-of-a-hard-disk-ure-unrecoverable-read-error |