Management Cluster Ceph » Historique » Version 22
Mehdi Abaakouk, 09/12/2016 17:33
1 | 10 | Mehdi Abaakouk | {{>toc}} |
---|---|---|---|
2 | 1 | Mehdi Abaakouk | |
3 | 10 | Mehdi Abaakouk | h1. Management Cluster Ceph |
4 | 9 | Mehdi Abaakouk | |
5 | 8 | Mehdi Abaakouk | h2. Liens |
6 | 8 | Mehdi Abaakouk | |
7 | 8 | Mehdi Abaakouk | * [[Openstack Management TTNN]] |
8 | 8 | Mehdi Abaakouk | * [[Openstack Setup VM pas dans openstack]] |
9 | 8 | Mehdi Abaakouk | * [[Openstack Installation nouvelle node du cluster]] |
10 | 8 | Mehdi Abaakouk | * [[Openstack Installation TTNN]] |
11 | 8 | Mehdi Abaakouk | * "Openstack tools for ttnn":/projects/git-tetaneutral-net/repository/openstack-tools |
12 | 8 | Mehdi Abaakouk | |
13 | 2 | Mehdi Abaakouk | h2. Ajout d'un OSD classique |
14 | 2 | Mehdi Abaakouk | |
15 | 22 | Mehdi Abaakouk | *DEBUT WORKAROUND BUG PREPARE* |
16 | 20 | Mehdi Abaakouk | |
17 | 20 | Mehdi Abaakouk | ID est le premier numero lib d'osd en partant de zero |
18 | 20 | Mehdi Abaakouk | |
19 | 20 | Mehdi Abaakouk | <pre> |
20 | 20 | Mehdi Abaakouk | mkdir /var/lib/ceph/osd/ceph-<ID> |
21 | 20 | Mehdi Abaakouk | chmod ceph:ceph /var/lib/ceph/osd/ceph-<ID> |
22 | 20 | Mehdi Abaakouk | </pre> |
23 | 1 | Mehdi Abaakouk | |
24 | 22 | Mehdi Abaakouk | *FIN WORKAROUND BUG PREPARE* |
25 | 21 | Mehdi Abaakouk | |
26 | 20 | Mehdi Abaakouk | |
27 | 2 | Mehdi Abaakouk | <pre> |
28 | 11 | Mehdi Abaakouk | $ ceph-disk prepare --zap-disk --cluster-uuid 1fe74663-8dfa-486c-bb80-3bd94c90c967 --fs-type=ext4 /dev/sdX |
29 | 2 | Mehdi Abaakouk | $ smartctl --smart=on /dev/sdX # Pour le monitoring. |
30 | 2 | Mehdi Abaakouk | </pre> |
31 | 2 | Mehdi Abaakouk | |
32 | 17 | Mehdi Abaakouk | Récuperer l'id avec (c'est celui tout en bas pas accroché à l'arbre): |
33 | 17 | Mehdi Abaakouk | |
34 | 17 | Mehdi Abaakouk | <pre> |
35 | 17 | Mehdi Abaakouk | ceph osd tree |
36 | 17 | Mehdi Abaakouk | </pre> |
37 | 17 | Mehdi Abaakouk | |
38 | 2 | Mehdi Abaakouk | Pour un HDD: |
39 | 1 | Mehdi Abaakouk | <pre> |
40 | 17 | Mehdi Abaakouk | $ ceph osd crush add osd.<ID> 0 root=default host=<host> |
41 | 2 | Mehdi Abaakouk | </pre> |
42 | 2 | Mehdi Abaakouk | |
43 | 1 | Mehdi Abaakouk | Pour un SSD: |
44 | 2 | Mehdi Abaakouk | <pre> |
45 | 17 | Mehdi Abaakouk | $ ceph osd crush add osd.<ID> 0 root=ssd host=<host>-ssd |
46 | 2 | Mehdi Abaakouk | </pre> |
47 | 2 | Mehdi Abaakouk | |
48 | 2 | Mehdi Abaakouk | Ensuite, autoriser Ceph à mettre des data dessus: |
49 | 2 | Mehdi Abaakouk | |
50 | 2 | Mehdi Abaakouk | <pre> |
51 | 2 | Mehdi Abaakouk | $ /root/tools/ceph-reweight-osds.sh osd.<ID> |
52 | 2 | Mehdi Abaakouk | </pre> |
53 | 2 | Mehdi Abaakouk | |
54 | 19 | Mehdi Abaakouk | |
55 | 19 | Mehdi Abaakouk | |
56 | 19 | Mehdi Abaakouk | h2. Vider un OSD: |
57 | 19 | Mehdi Abaakouk | |
58 | 19 | Mehdi Abaakouk | <pre> |
59 | 19 | Mehdi Abaakouk | vider_osp(){ |
60 | 19 | Mehdi Abaakouk | name="$1" |
61 | 19 | Mehdi Abaakouk | ceph osd out ${name} |
62 | 19 | Mehdi Abaakouk | ceph osd crush reweight ${name} 0 |
63 | 19 | Mehdi Abaakouk | ceph osd reweight ${name} 0 |
64 | 19 | Mehdi Abaakouk | } |
65 | 19 | Mehdi Abaakouk | </pre> |
66 | 19 | Mehdi Abaakouk | |
67 | 19 | Mehdi Abaakouk | h2. Suppression d'un OSD: |
68 | 19 | Mehdi Abaakouk | |
69 | 19 | Mehdi Abaakouk | <pre> |
70 | 19 | Mehdi Abaakouk | remove_osd(){ |
71 | 19 | Mehdi Abaakouk | name="$1" |
72 | 19 | Mehdi Abaakouk | ceph osd out ${name} |
73 | 19 | Mehdi Abaakouk | systemctl stop ceph-osd@${name#osd.} |
74 | 19 | Mehdi Abaakouk | ceph osd crush remove ${name} |
75 | 19 | Mehdi Abaakouk | ceph auth del ${name} |
76 | 19 | Mehdi Abaakouk | ceph osd rm ${name} |
77 | 19 | Mehdi Abaakouk | ceph osd tree |
78 | 19 | Mehdi Abaakouk | } |
79 | 19 | Mehdi Abaakouk | </pre> |
80 | 19 | Mehdi Abaakouk | |
81 | 19 | Mehdi Abaakouk | h2. Arrêter les IO de recovery: |
82 | 19 | Mehdi Abaakouk | |
83 | 19 | Mehdi Abaakouk | <pre> |
84 | 19 | Mehdi Abaakouk | ceph osd set nobackfill |
85 | 19 | Mehdi Abaakouk | ceph osd set norebalance |
86 | 19 | Mehdi Abaakouk | ceph osd set norecover |
87 | 19 | Mehdi Abaakouk | </pre> |
88 | 19 | Mehdi Abaakouk | |
89 | 19 | Mehdi Abaakouk | h2. Remplacement à froid d'un tier cache: |
90 | 19 | Mehdi Abaakouk | |
91 | 19 | Mehdi Abaakouk | upstream doc: http://docs.ceph.com/docs/master/rados/operations/cache-tiering/ |
92 | 19 | Mehdi Abaakouk | |
93 | 19 | Mehdi Abaakouk | <pre> |
94 | 19 | Mehdi Abaakouk | ceph osd tier cache-mode ec8p2c forward |
95 | 19 | Mehdi Abaakouk | rados -p ec8p2c cache-flush-evict-all |
96 | 19 | Mehdi Abaakouk | ceph osd tier remove-overlay ec8p2 |
97 | 19 | Mehdi Abaakouk | ceph osd tier remove ec8p2 ec8p2c |
98 | 19 | Mehdi Abaakouk | |
99 | 19 | Mehdi Abaakouk | rados rmpool ec8p2c ec8p2c --yes-i-really-really-mean-ita |
100 | 19 | Mehdi Abaakouk | ceph osd pool create ec8p2c 128 128 replicated |
101 | 19 | Mehdi Abaakouk | |
102 | 19 | Mehdi Abaakouk | ceph osd tier add ec8p2 ec8p2c |
103 | 19 | Mehdi Abaakouk | ceph osd tier cache-mode ec8p2c writeback |
104 | 19 | Mehdi Abaakouk | ceph osd tier set-overlay ec8p2 ec8p2c |
105 | 19 | Mehdi Abaakouk | |
106 | 19 | Mehdi Abaakouk | ceph osd pool set ec8p2c size 3 |
107 | 19 | Mehdi Abaakouk | ceph osd pool set ec8p2c min_size 2 |
108 | 19 | Mehdi Abaakouk | ceph osd pool set ec8p2c hit_set_type bloom |
109 | 19 | Mehdi Abaakouk | |
110 | 19 | Mehdi Abaakouk | ceph osd pool set ec8p2c hit_set_count 1 |
111 | 19 | Mehdi Abaakouk | ceph osd pool set ec8p2c hit_set_period 3600 |
112 | 19 | Mehdi Abaakouk | ceph osd pool set ec8p2c target_max_bytes 200000000000 |
113 | 19 | Mehdi Abaakouk | ceph osd pool set ec8p2c target_max_objects 10000000 |
114 | 19 | Mehdi Abaakouk | ceph osd pool set ec8p2c cache_target_dirty_ratio 0.4 |
115 | 19 | Mehdi Abaakouk | ceph osd pool set ec8p2c cache_target_full_ratio 0.8 |
116 | 19 | Mehdi Abaakouk | </pre> |
117 | 19 | Mehdi Abaakouk | |
118 | 16 | Mehdi Abaakouk | h2. Ajout d'un OSD qui partage le SSD avec l'OS (OBSOLETE PLUS COMPATIBLE AVEC LES FUTURES VERSION DE CEPH) |
119 | 2 | Mehdi Abaakouk | |
120 | 2 | Mehdi Abaakouk | |
121 | 2 | Mehdi Abaakouk | En général avec ceph, on donne un disque, ceph créé 2 partitions une pour le journal de l'OSD, l'autre pour les datas |
122 | 2 | Mehdi Abaakouk | mais pour le SSD de tetaneutral qui a aussi l'OS, voici la méthode |
123 | 2 | Mehdi Abaakouk | |
124 | 2 | Mehdi Abaakouk | Création manuelle de la partition de data ceph /dev/sda2 ici |
125 | 7 | Mehdi Abaakouk | |
126 | 7 | Mehdi Abaakouk | Debian (MBR format): |
127 | 2 | Mehdi Abaakouk | <pre> |
128 | 5 | Mehdi Abaakouk | apt-get install partprobe |
129 | 5 | Mehdi Abaakouk | fdisk /dev/sda |
130 | 5 | Mehdi Abaakouk | |
131 | 2 | Mehdi Abaakouk | n |
132 | 14 | Mehdi Abaakouk | p |
133 | 14 | Mehdi Abaakouk | <enter> |
134 | 14 | Mehdi Abaakouk | <enter> |
135 | 14 | Mehdi Abaakouk | <enter> |
136 | 14 | Mehdi Abaakouk | <enter> |
137 | 14 | Mehdi Abaakouk | w |
138 | 14 | Mehdi Abaakouk | |
139 | 14 | Mehdi Abaakouk | $ partprobe |
140 | 14 | Mehdi Abaakouk | </pre> |
141 | 14 | Mehdi Abaakouk | |
142 | 14 | Mehdi Abaakouk | Ubuntu (GPT format): |
143 | 2 | Mehdi Abaakouk | <pre> |
144 | 2 | Mehdi Abaakouk | # parted /dev/sdb |
145 | 2 | Mehdi Abaakouk | GNU Parted 2.3 |
146 | 13 | Mehdi Abaakouk | Using /dev/sdb |
147 | 13 | Mehdi Abaakouk | Welcome to GNU Parted! Type 'help' to view a list of commands. |
148 | 13 | Mehdi Abaakouk | (parted) print |
149 | 18 | Mehdi Abaakouk | Model: ATA SAMSUNG MZ7KM480 (scsi) |
150 | 13 | Mehdi Abaakouk | Disk /dev/sdb: 480GB |
151 | 13 | Mehdi Abaakouk | Sector size (logical/physical): 512B/512B |
152 | 13 | Mehdi Abaakouk | Partition Table: msdos |
153 | 13 | Mehdi Abaakouk | |
154 | 13 | Mehdi Abaakouk | Number Start End Size Type File system Flags |
155 | 2 | Mehdi Abaakouk | 1 1049kB 20.0GB 20.0GB primary ext4 boot |
156 | 2 | Mehdi Abaakouk | 2 20.0GB 36.0GB 16.0GB primary linux-swap(v1) |
157 | 15 | Mehdi Abaakouk | |
158 | 15 | Mehdi Abaakouk | (parted) mkpart |
159 | 15 | Mehdi Abaakouk | Partition type? primary/extended? |
160 | 15 | Mehdi Abaakouk | Partition type? primary/extended? primary |
161 | 15 | Mehdi Abaakouk | File system type? [ext2]? xfs |
162 | 15 | Mehdi Abaakouk | Start? |
163 | 15 | Mehdi Abaakouk | Start? 36.0GB |
164 | 15 | Mehdi Abaakouk | End? 100% |
165 | 1 | Mehdi Abaakouk | (parted) print |
166 | 1 | Mehdi Abaakouk | Model: ATA SAMSUNG MZ7KM480 (scsi) |
167 | 1 | Mehdi Abaakouk | Disk /dev/sdb: 480GB |
168 | 1 | Mehdi Abaakouk | Sector size (logical/physical): 512B/512B |
169 | 1 | Mehdi Abaakouk | Partition Table: msdos |
170 | 1 | Mehdi Abaakouk | |
171 | 1 | Mehdi Abaakouk | Number Start End Size Type File system Flags |
172 | 1 | Mehdi Abaakouk | 1 1049kB 20.0GB 20.0GB primary ext4 boot |
173 | 1 | Mehdi Abaakouk | 2 20.0GB 36.0GB 16.0GB primary linux-swap(v1) |
174 | 1 | Mehdi Abaakouk | 3 36.0GB 480GB 444GB primary |
175 | 1 | Mehdi Abaakouk | |
176 | 1 | Mehdi Abaakouk | (parted) quit |
177 | 1 | Mehdi Abaakouk | Information: You may need to update /etc/fstab. |
178 | 1 | Mehdi Abaakouk | </pre> |
179 | 1 | Mehdi Abaakouk | |
180 | 1 | Mehdi Abaakouk | On prepare le disk comme normalement |
181 | 1 | Mehdi Abaakouk | |
182 | 1 | Mehdi Abaakouk | <pre> |
183 | 1 | Mehdi Abaakouk | ceph-disk prepare --fs-type=ext4 --cluster-uuid 1fe74663-8dfa-486c-bb80-3bd94c90c967 /dev/sda2 |
184 | 1 | Mehdi Abaakouk | ceph-disk activate /dev/sda2 |
185 | 1 | Mehdi Abaakouk | ceph osd crush add osd.<ID> 0 root=ssd host=g3-ssd |
186 | 1 | Mehdi Abaakouk | </pre> |
187 | 1 | Mehdi Abaakouk | |
188 | 1 | Mehdi Abaakouk | Ensuite, autoriser Ceph à mettre des data dessus: |
189 | 1 | Mehdi Abaakouk | |
190 | 1 | Mehdi Abaakouk | <pre> |
191 | 1 | Mehdi Abaakouk | $ /root/tools/ceph-reweight-osds.sh osd.<ID> |
192 | 1 | Mehdi Abaakouk | </pre> |