Libvirt (ancien) » Historique » Version 59

« Précédent - Version 59/61 (diff) - Suivant » - Version actuelle
Mehdi Abaakouk, 10/04/2017 20:42

Libvirt¶

/!\ tetaneutral utilise Openstack maintenant /!\¶

https://chiliproject.tetaneutral.net/projects/tetaneutral/wiki/Openstack_Management_TTNN

Resources/Liens¶

Virtualisation du routage IPv4 pour les VM http://lists.tetaneutral.net/pipermail/technique/2013-April/000810.html

libvirt

http://builder.virt-tools.org/artifacts/libvirt-virshcmdref/html/index.html
http://docs.fedoraproject.org/en-US/Fedora/13/html/Virtualization_Guide/chap-Virtualization-Managing_guests_with_virsh.html

virt-install et preseed sans server web et pxe:

http://honk.sigxcpu.org/con/Preseeding_Debian_virtual_machines_with_virt_install.html

Backup d'un VM n7¶

sur h2:

# VM=XXX.gz; cd /backup/libvirt-n7-vm-raw/20140319/ ; if [ ! -f ${VM} ]; then socat -u tcp4-listen:50555 - > $VM ; zcat $VM | md5sum -; du -hs *; else echo $VM already present;fi

sur la machine n7:

# virsh shutdown XXX
# LV=XXX; cat /dev/raid5/$LV | gzip --fast|socat -u - tcp4:h2.tetaneutral.net:50555;md5sum /dev/raid5/$LV

Installation VM sans bridge avec routage static (en /32 pour l'ipv4)¶

Note: tous les fichiers utilisés et créés ne sont peut être pas à jour, voir ici pour les dernières version: http://git.tetaneutral.net/libvirt.git/

Le répertoire /etc/libvirt/ttnn cité plustard est géré dans ce git.

Configuration de la libvirt¶

On créé un réseau libvirt qui fait rien (déjà fait sur n7):

cat > libvirt_dump_network.xml <<EOF
<network ipv6='yes'>
<name>dump-net</name>
<bridge name='dumpbr0' />
</network>
EOF

virsh net-define libvirt_dump_network.xml
virsh net-autostart dump-net
virsh net-start dump-net

On créé un hook libvirt (déjà fait sur n7):

mkdir -p /etc/libvirt/ttnn/hooks
ln -s /etc/libvirt/ttnn/hooks /etc/libvirt/hooks
vi /etc/libvirt/hooks/qemu
----8<--------------->8----
#!/bin/bash

name="$1" 
operation="$2" 
suboperation="$3" 

[ "$operation" != "start" ] && exit 0
[ "$suboperation" != "begin" ] && exit 0

# get interface from the libvirt xml
interface=$(cat - | sed -ne "/vnet/s/.*dev='\([^']*\)'.*/\1/gp")

[ -f /etc/libvirt/ttnn/${name}.conf ] && source /etc/libvirt/ttnn/${name}.conf

[ "$ip" -o "$ip6" ] && brctl delif dumpbr0 $interface
[ "$ip" ] && ip route add $ip/32 dev $interface
[ "$ip6" ] && ip route add $ip6/56 dev $interface
[ "$ip6" ] && ip -6 addr add fe80::31/64 dev $interface
exit 0
----8<--------------->8----

Preparation de l'installer¶

Pour une squeeze on met à jour virtinst (déjà fait sur n7):

wget http://ftp.us.debian.org/debian/pool/main/v/virtinst/virtinst_0.600.3-3_all.deb
dpkg -i virtinst_0.600.3-3_all.deb

On install le preseed.cfg attaché à la page dans /etc/libvirt/preseed.cfg (déjà fait sur n7)

Installation d'une VM¶

Creation du disk (déjà fait sur n7):

lvcreate -n picttn -L 50G /dev/raid5

Configuration du routage static:

vi /etc/libvirt/ttnn/picttn.conf 
# le /32 ipv4
ip="91.224.148.252" 
# le /56 ipv6
ip6="2a01:6600:8080:FC00::"

Lancement de l'installation via le réseau natter de libvirt

virt-install -n picttn -r 2048 --os-type=linux \
    --autostart \
    --disk /dev/raid5/picttn,bus=virtio,cache=none,io=native \
    --network network=default,model=virtio \
    --nographics \
    --noreboot \
    --wait -1 \
    --location http://ftp.debian.org/debian/dists/wheezy/main/installer-amd64/ \
    --serial pty \
    --initrd-inject /etc/libvirt/ttnn/preseed.cfg \
    --initrd-inject /etc/libvirt/ttnn/picttn.conf \
    --initrd-inject /etc/libvirt/ttnn/postinstall.sh \
    --extra-args "console=ttyS0,9600n8 DEBIAN_FRONTEND=text auto priority=critical hostname=picttn domain=le-pic.org"

Une fois l'installation terminée, on change la conf réseau:

EDITOR="sed -ie s/default/dump-net/g" virsh edit picttn

On peux démarrer la VM

virsh start picttn
virsh console picttn

Ensuite on peux se connecte à la VM

ssh root@2a01:6600:8080:FC00::1

Le /etc/network/interfaces IPv4 après installation

iface eth0 inet static
    address 91.224.148.xxx
    netmask 32
    up ip route add 91.224.148.0/32 dev eth0
    up ip route add default via 91.224.148.0

iface eth0 inet6 static
    address 2a01:6600:8080:xx00::1
    netmask 56
    gateway fe80::31
    pre-up /sbin/sysctl -w net.ipv6.conf.eth0.accept_ra=0
    pre-up /sbin/sysctl -w net.ipv6.conf.eth0.autoconf=0

Misc¶

<H4Mm3r> si tu lance la création de la VM sans avoir créé le disk --disk /vmroot/testnat.qcow2,format=qcow2,size=12,bus=virtio,cache=writeback
<H4Mm3r> le disk se créé mais dans le XML le format est à RAW
<H4Mm3r> si tu le créé à l'avance avec qemu-img create -f qcow2
<H4Mm3r> et que tu lance ta commande --disk /vmroot/testnat.qcow2,format=qcow2
<guerby> ok donc c'etait bien ça :)
<guerby> bbl (miam)
<H4Mm3r> dans le fichier XML le format est à RAW
<H4Mm3r> il faut en effet dumper un XML, modifier la valeur à QCOW2
<H4Mm3r> et créer la VM depuis ce XML
<H4Mm3r> virsh create file.xml
<H4Mm3r> pourqu'enfin il comprenne que c'est du QCOW...
<H4Mm3r> et là mon disk fait bien 265ko, mais est détecté comme 12G :D

HOWTO by H4Mm3r (under contruction)¶

Usefull links¶

Misc

#OS Type
http://rhevdup.blogspot.fr/2012/05/list-available-os-types-for-kvm-on-rhel.html

#Libvirt TTN
http://chiliproject.tetaneutral.net/projects/tetaneutral/wiki/Libvirt

KVM management

#Install QEMU KVM + LIBVIRT
http://www.howtoforge.com/virtualization-with-kvm-on-a-debian-squeeze-server

#Edit XML file
http://vladnevzorov.com/2011/02/11/how-to-change-configuration-of-kvm-virtual-machine/

#Clone a KVM
https://help.ubuntu.com/community/KVM/CreateGuests#Cloning_a_virtual_machine
http://linux.die.net/man/1/virt-clone

Network

#NAT
http://www.linuxquestions.org/questions/linux-networking-3/kvm-qemu-and-nat-on-the-host-machine-mini-tutorial-697980/
http://wiki.libvirt.org/page/Networking
http://libvirt.org/formatnetwork.html#examplesNAT
https://help.ubuntu.com/community/KVM/Networking

#Bridge
http://toast.djw.org.uk/qemu.html
http://www.mediaonfire.com/en/tech_tips/QEMUNetworking.html (bridge, nat, arp)

Improve performance
http://blog.bodhizazen.net/linux/improve-kvm-performance/
http://www.linux-kvm.org/page/Tuning_KVM
http://www.ezunix.org/index.php?title=Boost_performance_of_KVM_guests

Storage
http://doc.opensuse.org/products/draft/SLES/SLES-kvm_sd_draft/cha.libvirt.storage.html

HOWTO¶

[H] = Host
[G] = Guest
[XML] = XML content
"command"

Diskfile Storage

#Create qcow2 disk

[H] "qemu-img create -f qcow2 /path/to/disk.qcow2 $SIZE" (ex: 20G)
[H] "chmod +x /path/to/disk.qcow2"

#Convert raw to qcow2

[H] "qemu-img convert -O qcow2 disk.img disk.qcow2"

#Disable cache to improve performance

[XML] <driver name='qemu' type='qcow2' cache='none'/>

LVM Storage

[H] Create "/etc/libvirt/storage/vmdata.xml" file :
      <pool type="logical">
        <name>my_pool</name>
        <source>
          <device path="/path/to/dev"/>
        </source>
        <target>
          <path>/target/path/path> (LV will be created there)
        </target>
      </pool>
[H] "virsh pool-define /etc/libvirt/storage/my_pool.xml" 
[H] "virsh pool-start my_pool" 
[H] "virsh pool-autostart my_pool" 
[H] "virsh pool-info my_pool" 
[H] "virsh vol-create-as my_pool my_logical_volume 12G" 
[H] You must create a LV to be able to give it to a KVM "virsh vol-create-as my_pool my_vol 12G --format=lvm2"

Create KVM¶

#Bridged with TAP + LVM

[H] "virt-install -d --hvm --vnc --name=my_kvm --vcpus=4 --cpuset=0-4 --ram 2048 --disk vol=my_pool/my_vol,bus=virtio,cache=none --network bridge=my_bridge,model=virtio -c /path/to/distrib.iso --keymap=fr --os-type=linux --noautoconsole"

#TAP (if you want to rename the default "vnetX" interface by "tapX"

[H] "virsh dumpxml my_guest > my_guest.xml
[H] "virsh undefine my_guest" 
[H] Edit the my_guest.xml file to rename VNETX by TAPX
[H] "virsh define my_guest.xml" 
[H] "virsh start my_guest"

#NATed + LVM

[H] "virt-install -d --hvm --vnc --name=my_kvm --vcpus=4 --cpuset=0-4 --ram 2048 --disk vol=my_pool/my_vol,bus=virtio,cache=none --network network=my_network,model=virtio -c /path/to/distrib.iso --keymap=fr --os-type=linux --noautoconsole"

#Clone VM

[H] Create the LVM dest
[H] Pause the KVM "virsh suspend my_kvm" 
[H] virt-clone -o my_orig_kvm -n my_cloned_kvm -f=/path/to/lvm/dev
[G] Go and change the hostname and resolv.conf

Command explanation¶

#Bridged with tap

virt-install -d --hvm --vnc --name=testlvm \
    --vcpus=4 --cpuset=0-4 \
    --ram 2048 \ 
    --disk vol=vmdata/testlvm,bus=virtio,cache=none \
    --network bridge=br0,model=virtio \
    -c /data/isos/debian-testing-amd64-netinst.iso \
    --keymap=fr --os-type=linux --os-variant=debiansqueeze --noautoconsole

#NATed

virt-install -d --hvm --vnc --name=testlvm \
    --vcpus=4 --cpuset=0-4 \
    --ram 2048 \ 
    --disk vol=vmdata/testlvm,bus=virtio,cache=none \
    --network network=frp_network,model=virtio \
    -c /data/isos/debian-testing-amd64-netinst.iso \
    --keymap=fr --os-type=linux --os-variant=debiansqueeze --noautoconsole

[OLD QCOW2]

[H] "virt-install --connect qemu:///system -n puppetnode1 -r 1024 --vcpus=4 --cpuset=4-7 --disk path=/vmroot/puppetnode1.qcow2 -c /data/isos/debian-6.0.6-amd64-netinst.iso --vnc --keymap=fr --noautoconsole --os-type linux --os-variant debiansqueeze --accelerate --network network=default --hvm" 
[H] "virt-install --connect qemu:///system -n testnat -r 1024 --vcpus=2 --cpuset=2-3 --disk /vmroot/testnat.qcow2,format=qcow2,size=12,bus=virtio,cache=writeback -c /data/isos/debian-6.0.6-amd64-netinst.iso --vnc --keymap=fr --noautoconsole --os-type linux --os-variant debiansqueeze --accelerate --network bridge=br0 --hvm"

#Start a KVM

[H] "virsh start my_kvm"

#Stop node

[H] Soft : "virsh shutdown my_kvm" 
[H] Hard : "virsh destroy my_kvm"

#Suppr node del

[H] "virsh undefine node" (+ remove files)

#Rename node

[H] "virsh dumpxml my_kvm > foo.xml" 
[H] <edit foo.xml, change the name, move storage>
[H] "virsh undefine my_kvm" 
[H] "virsh define foo.xml"

Connect to virsh qemu

[H] "virsh --connect qemu:///system" 
[H] "virsh # help"

#Start a network

[H] "virsh net-start my_network"

#Configure VNC

[H] Edit /etc/libvirt/qemu.conf and set vnc_listen = "your.ip.addd" 
[H] You need to restart service "/etc/init.d/libvirtd restart" 
[H] "ps aux | grep -i kvm" (find vnc display for your KVM)

#Connect to a KVM through VNC

tightvncviewer -compresslevel 7 -encodings "copyrect tight hextile zlib corre rre raw" x.xxx.xxx.xxx:x

#Keyboard FR for KVM

[H] "virsh edit my_kvm" 
[XML] <graphics type='vnc' port='-1' autoport='yes' keymap='fr'/>

#Create your own NAT network

[H] Create your_nat.xml file :
[XML] <network>
  <name>frp_network</name>
  <forward mode='nat'/>
  <bridge name='br1' stp='on' delay='0' />
  <ip address='10.16.42.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='10.16.42.2' end='10.16.42.50' />
    </dhcp>
  </ip>
</network>
[H] "virsh net-define your_nat.xml" 
[H] "virsh net-start your_nat.xml" 
[H] "virsh net-autostart your_nat.xml" 
[XML] Network type "interface type = network, source network=your_nat"

#IBM advisements to improve performance

[H] Use LVM (or other direct access to HDD) for VMs storage (from 45 to 3' to install a VM)
[H] Set the swappiness value to 0 ("echo 0 > /proc/sys/vm/swappiness" && edit "/etc/sysctl.conf" and add "vm.swappiness=0")
[H] Disable zone_reclaim ("echo 0 > /proc/sys/vm/zone_reclaim_mode" && edit "/etc/sysctl.conf" and add "vm.zone_reclaim_mode=0")
[H] Enable KSM ("echo 1 > /sys/kernel/mm/ksm/run") and set the check time to 200ms ("echo 200 > /sys/kernel/mm/ksm/sleep_millisecs")

PC velizy¶

root@tetaneutral1:~# fdisk /dev/sdb

Command (m for help): p

Disk /dev/sdb: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x9bf0c7df

   Device Boot      Start         End      Blocks   Id  System

Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p): p
Partition number (1-4, default 1): 
Using default value 1
First sector (2048-3907029167, default 2048): 
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-3907029167, default 3907029167): 
Using default value 3907029167

Command (m for help): n
Partition type:
   p   primary (1 primary, 0 extended, 3 free)
   e   extended
Select (default p): ^C
root@tetaneutral1:~# fdisk /dev/sdb

Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-3907029167, default 2048): 
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-3907029167, default 3907029167): 
Using default value 3907029167

Command (m for help): p

Disk /dev/sdb: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x9bf0c7df

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048  3907029167  1953513560   83  Linux

Command (m for help): m
Command action
   a   toggle a bootable flag
   b   edit bsd disklabel
   c   toggle the dos compatibility flag
   d   delete a partition
   l   list known partition types
   m   print this menu
   n   add a new partition
   o   create a new empty DOS partition table
   p   print the partition table
   q   quit without saving changes
   s   create a new empty Sun disklabel
   t   change a partition's system id
   u   change display/entry units
   v   verify the partition table
   w   write table to disk and exit
   x   extra functionality (experts only)

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): L

 0  Empty           24  NEC DOS         81  Minix / old Lin bf  Solaris        
 1  FAT12           27  Hidden NTFS Win 82  Linux swap / So c1  DRDOS/sec (FAT-
 2  XENIX root      39  Plan 9          83  Linux           c4  DRDOS/sec (FAT-
 3  XENIX usr       3c  PartitionMagic  84  OS/2 hidden C:  c6  DRDOS/sec (FAT-
 4  FAT16 <32M      40  Venix 80286     85  Linux extended  c7  Syrinx         
 5  Extended        41  PPC PReP Boot   86  NTFS volume set da  Non-FS data    
 6  FAT16           42  SFS             87  NTFS volume set db  CP/M / CTOS / .
 7  HPFS/NTFS/exFAT 4d  QNX4.x          88  Linux plaintext de  Dell Utility   
 8  AIX             4e  QNX4.x 2nd part 8e  Linux LVM       df  BootIt         
 9  AIX bootable    4f  QNX4.x 3rd part 93  Amoeba          e1  DOS access     
 a  OS/2 Boot Manag 50  OnTrack DM      94  Amoeba BBT      e3  DOS R/O        
 b  W95 FAT32       51  OnTrack DM6 Aux 9f  BSD/OS          e4  SpeedStor      
 c  W95 FAT32 (LBA) 52  CP/M            a0  IBM Thinkpad hi eb  BeOS fs        
 e  W95 FAT16 (LBA) 53  OnTrack DM6 Aux a5  FreeBSD         ee  GPT            
 f  W95 Ext'd (LBA) 54  OnTrackDM6      a6  OpenBSD         ef  EFI (FAT-12/16/
10  OPUS            55  EZ-Drive        a7  NeXTSTEP        f0  Linux/PA-RISC b
11  Hidden FAT12    56  Golden Bow      a8  Darwin UFS      f1  SpeedStor      
12  Compaq diagnost 5c  Priam Edisk     a9  NetBSD          f4  SpeedStor      
14  Hidden FAT16 <3 61  SpeedStor       ab  Darwin boot     f2  DOS secondary  
16  Hidden FAT16    63  GNU HURD or Sys af  HFS / HFS+      fb  VMware VMFS    
17  Hidden HPFS/NTF 64  Novell Netware  b7  BSDI fs         fc  VMware VMKCORE 
18  AST SmartSleep  65  Novell Netware  b8  BSDI swap       fd  Linux raid auto
1b  Hidden W95 FAT3 70  DiskSecure Mult bb  Boot Wizard hid fe  LANstep        
1c  Hidden W95 FAT3 75  PC/IX           be  Solaris boot    ff  BBT            
1e  Hidden W95 FAT1 80  Old Minix      
Hex code (type L to list codes): 8e
Changed system type of partition 1 to 8e (Linux LVM)

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

pvcreate /dev/sdb1
vgcreate vm-velizy /dev/sdb1
lvcreate -n velizy-vm1 -L 50G vm-velizy
apt-get install virtinst

/root/preseed.cfg @ http://git.tetaneutral.net/libvirt.git/

replace the late-command line at the end of the file by:
d-i preseed/late_command string cp -f /postinstall-velizy.sh /target/; in-target bash /postinstall-velizy.sh

vmname=velizy-vm1
ip=192.166.143.24

virt-install -n $vmname -r 2048 --os-type=linux \
    --debug \
    --autostart \
    --disk /dev/vm-velizy/$vmname,bus=virtio,cache=none,io=native \
    --network bridge=br0,model=virtio \
    --nographics \
    --noreboot \
    --wait -1 \
    --location http://ftp.debian.org/debian/dists/wheezy/main/installer-amd64/ \
    --serial pty \
    --initrd-inject /root/preseed.cfg \
    --initrd-inject /root/postinstall-velizy.sh \
    --extra-args "console=ttyS0,9600n8 DEBIAN_FRONTEND=text auto priority=critical hostname=$vmname domain=tetaneutral.net netcfg/get_ipaddress=$ip netcfg/get_netmask=255.255.252.0 netcfg/get_gateway=192.166.143.254 netcfg/get_nameservers=192.166.143.23 netcfg/disable_dhcp=true"

postinstall-velizy.sh

#!/bin/bash
sed -i -e 's/^.*Port.*$/Port 2222/' -e 's/^[# ]*PasswordAuthentication.*$/PasswordAuthenticati
on no/' /etc/ssh/sshd_config

mkdir -p /root/.ssh
cat > /root/.ssh/authorized_keys  <<EOF
ssh-dss AAAAB3NzaC1kc3MAAACBAJtnGLvuz4uVD6fnERDxDi/C0UyzwCiKmgNtEessopREYasAX4Gu6Fg10jAyIL5Nuc
7YDnqj//pOfxNjD7hp99a2ZmkRQgh/ltClxYML9fqhBHgsUCpVse9nOYDHDnDgvXIPRSDUHat2UFpdchHVrFURNLIlZnBz
tsr+GCDURTV/AAAAFQD9SrBcH49ltaKOm6V5ssCPmgs1SwAAAIAbAvkQ3/tMzzdACO5B9s5Yb3ZuM7XImm0iqufivVf2Xy
39g8HOBulUsN7eKEGPhVriqNolgIO7q7tVgYAff4/NE4yDP0Kp0SPg4tjt2yFJuL31Y3wzwHjnKrjgNlDSLl3uZnRXSipw
UMwGKbdHB6NZSqlq192VKbHilQ00pbiMlAAAAIB5dfB1lVHkJ0o5CcaVQRPbca9DvbbRwnoTSmKHc5DUcqsPqDhS07CkM9
ZcJuY1Nh4wGl4Q9kArj7Tnsvvygf/HReSUcIk4+nbDytJ8/pca/Qx4fzQQyppa94TylN62LSFT6MIJKLoMwYa0dQURT7Mv
5+9Qj2vk5pZ38w2iQ9zVCg== root@h1
EOF

passwd -d root

apt-get install debconf-utils
debconf-get-selections > preseed.cfg # marche pas

Fichiers

Projet

Général

Profil

Wiki

Observateurs (1)