h1. Libvirt

h2. /!\ tetaneutral utilise de nouveau libvirt /!\

https://chiliproject.tetaneutral.net/projects/tetaneutral/wiki/Cluster_with_libvirt

h2. Resources/Liens

* Virtualisation du routage IPv4 pour les VM http://lists.tetaneutral.net/pipermail/technique/2013-April/000810.html

* libvirt

http://builder.virt-tools.org/artifacts/libvirt-virshcmdref/html/index.html

http://docs.fedoraproject.org/en-US/Fedora/13/html/Virtualization_Guide/chap-Virtualization-Managing_guests_with_virsh.html

* virt-install et preseed sans server web et pxe:

http://honk.sigxcpu.org/con/Preseeding_Debian_virtual_machines_with_virt_install.html

h2. Backup d'un VM n7

sur h2:

<pre>

# VM=XXX.gz; cd /backup/libvirt-n7-vm-raw/20140319/ ; if [ ! -f ${VM} ]; then socat -u tcp4-listen:50555 - > $VM ; zcat $VM | md5sum -; du -hs *; else echo $VM already present;fi

</pre>

sur la machine n7:

<pre>

# virsh shutdown XXX

# LV=XXX; cat /dev/raid5/$LV | gzip --fast|socat -u - tcp4:h2.tetaneutral.net:50555;md5sum /dev/raid5/$LV

</pre>

h2. Installation VM sans bridge avec routage static (en /32 pour l'ipv4)

Note: tous les fichiers utilisés et créés ne sont peut être pas à jour, voir ici pour les dernières version: http://git.tetaneutral.net/libvirt.git/

Le répertoire /etc/libvirt/ttnn cité plustard est géré dans ce git.

h3. Configuration de la libvirt

On créé un réseau libvirt qui fait rien (déjà fait sur n7):

<pre>

cat > libvirt_dump_network.xml <<EOF

<network ipv6='yes'>

<name>dump-net</name>

<bridge name='dumpbr0' />

</network>

EOF

virsh net-define libvirt_dump_network.xml

virsh net-autostart dump-net

virsh net-start dump-net

</pre>

On créé un hook libvirt (déjà fait sur n7):

<pre>

mkdir -p /etc/libvirt/ttnn/hooks

ln -s /etc/libvirt/ttnn/hooks /etc/libvirt/hooks

vi /etc/libvirt/hooks/qemu

----8<--------------->8----

#!/bin/bash

name="$1"

operation="$2"

suboperation="$3"

[ "$operation" != "start" ] && exit 0

[ "$suboperation" != "begin" ] && exit 0

# get interface from the libvirt xml

interface=$(cat - | sed -ne "/vnet/s/.*dev='\([^']*\)'.*/\1/gp")

[ -f /etc/libvirt/ttnn/${name}.conf ] && source /etc/libvirt/ttnn/${name}.conf

[ "$ip" -o "$ip6" ] && brctl delif dumpbr0 $interface

[ "$ip" ] && ip route add $ip/32 dev $interface

[ "$ip6" ] && ip route add $ip6/56 dev $interface

[ "$ip6" ] && ip -6 addr add fe80::31/64 dev $interface

exit 0

----8<--------------->8----

</pre>

h3. Preparation de l'installer

Pour une squeeze on met à jour virtinst  (déjà fait sur n7):

<pre>

wget http://ftp.us.debian.org/debian/pool/main/v/virtinst/virtinst_0.600.3-3_all.deb

dpkg -i virtinst_0.600.3-3_all.deb

</pre>

On install le preseed.cfg attaché à la page dans /etc/libvirt/preseed.cfg  (déjà fait sur n7)

h4. Installation d'une VM

Creation du disk (déjà fait sur n7):

<pre>

lvcreate -n picttn -L 50G /dev/raid5

</pre>

Configuration du routage static:

<pre>

vi /etc/libvirt/ttnn/picttn.conf 

# le /32 ipv4

ip="91.224.148.252"

# le /56 ipv6

ip6="2a01:6600:8080:FC00::"

</pre>

Lancement de l'installation via le réseau natter de libvirt

<pre>

virt-install -n picttn -r 2048 --os-type=linux \

    --autostart \

    --disk /dev/raid5/picttn,bus=virtio,cache=none,io=native \

    --network network=default,model=virtio \

    --nographics \

    --noreboot \

    --wait -1 \

    --location http://ftp.debian.org/debian/dists/wheezy/main/installer-amd64/ \

    --serial pty \

    --initrd-inject /etc/libvirt/ttnn/preseed.cfg \

    --initrd-inject /etc/libvirt/ttnn/picttn.conf \

    --initrd-inject /etc/libvirt/ttnn/postinstall.sh \

    --extra-args "console=ttyS0,9600n8 DEBIAN_FRONTEND=text auto priority=critical hostname=picttn domain=le-pic.org"

</pre>

Une fois l'installation terminée, on change la conf réseau:

<pre>

EDITOR="sed -ie s/default/dump-net/g" virsh edit picttn

</pre>

On peux démarrer la VM

<pre>

virsh start picttn

virsh console picttn

</pre>

Ensuite on peux se connecte à la VM

<pre>

ssh root@2a01:6600:8080:FC00::1

</pre>

Le /etc/network/interfaces IPv4 après installation

<pre>

iface eth0 inet static

    address 91.224.148.xxx

    netmask 32

    up ip route add 91.224.148.0/32 dev eth0

    up ip route add default via 91.224.148.0

iface eth0 inet6 static

    address 2a01:6600:8080:xx00::1

    netmask 56

    gateway fe80::31

    pre-up /sbin/sysctl -w net.ipv6.conf.eth0.accept_ra=0

    pre-up /sbin/sysctl -w net.ipv6.conf.eth0.autoconf=0

</pre>

h2. Misc

<H4Mm3r> si tu lance la création de la VM sans avoir créé le disk --disk /vmroot/testnat.qcow2,format=qcow2,size=12,bus=virtio,cache=writeback

<H4Mm3r> le disk se créé mais dans le XML le format est à RAW

<H4Mm3r> si tu le créé à l'avance avec qemu-img create -f qcow2

<H4Mm3r> et que tu lance ta commande --disk /vmroot/testnat.qcow2,format=qcow2

<guerby> ok donc c'etait bien ça :)

<guerby> bbl (miam)

<H4Mm3r> dans le fichier XML le format est à RAW

<H4Mm3r> il faut en effet dumper un XML, modifier la valeur à QCOW2

<H4Mm3r> et créer la VM depuis ce XML

<H4Mm3r> virsh create file.xml

<H4Mm3r> pourqu'enfin il comprenne que c'est du QCOW...

<H4Mm3r> et là mon disk fait bien 265ko, mais est détecté comme 12G :D

h2. HOWTO by H4Mm3r (under contruction)

h3. Usefull links

#### Misc

#OS Type

http://rhevdup.blogspot.fr/2012/05/list-available-os-types-for-kvm-on-rhel.html

#Libvirt TTN

http://chiliproject.tetaneutral.net/projects/tetaneutral/wiki/Libvirt

#### KVM management

#Install QEMU KVM + LIBVIRT

http://www.howtoforge.com/virtualization-with-kvm-on-a-debian-squeeze-server

#Edit XML file

http://vladnevzorov.com/2011/02/11/how-to-change-configuration-of-kvm-virtual-machine/

#Clone a KVM

https://help.ubuntu.com/community/KVM/CreateGuests#Cloning_a_virtual_machine

http://linux.die.net/man/1/virt-clone

#### Network

#NAT

http://www.linuxquestions.org/questions/linux-networking-3/kvm-qemu-and-nat-on-the-host-machine-mini-tutorial-697980/

http://wiki.libvirt.org/page/Networking

http://libvirt.org/formatnetwork.html#examplesNAT

https://help.ubuntu.com/community/KVM/Networking

#Bridge

http://toast.djw.org.uk/qemu.html

http://www.mediaonfire.com/en/tech_tips/QEMUNetworking.html (bridge, nat, arp)

#### Improve performance

http://blog.bodhizazen.net/linux/improve-kvm-performance/

http://www.linux-kvm.org/page/Tuning_KVM

http://www.ezunix.org/index.php?title=Boost_performance_of_KVM_guests

#### Storage

http://doc.opensuse.org/products/draft/SLES/SLES-kvm_sd_draft/cha.libvirt.storage.html

h3. HOWTO

<pre>

[H] = Host

[G] = Guest

[XML] = XML content

"command"

</pre>

#### Diskfile Storage

#Create qcow2 disk

<pre>

[H] "qemu-img create -f qcow2 /path/to/disk.qcow2 $SIZE" (ex: 20G)

[H] "chmod +x /path/to/disk.qcow2"

</pre>

#Convert raw to qcow2

<pre>

[H] "qemu-img convert -O qcow2 disk.img disk.qcow2"

</pre>

#Disable cache to improve performance 

<pre>

[XML] <driver name='qemu' type='qcow2' cache='none'/>

</pre>

#### LVM Storage

<pre>

[H] Create "/etc/libvirt/storage/vmdata.xml" file :

      <pool type="logical">

        <name>my_pool</name>

        <source>

          <device path="/path/to/dev"/>

        </source>

        <target>

          <path>/target/path/path> (LV will be created there)

        </target>

      </pool>

[H] "virsh pool-define /etc/libvirt/storage/my_pool.xml"

[H] "virsh pool-start my_pool"

[H] "virsh pool-autostart my_pool"

[H] "virsh pool-info my_pool"

[H] "virsh vol-create-as my_pool my_logical_volume 12G"

[H] You must create a LV to be able to give it to a KVM "virsh vol-create-as my_pool my_vol 12G --format=lvm2"

</pre>

h3. Create KVM

#Bridged with TAP + LVM

<pre>

[H] "virt-install -d --hvm --vnc --name=my_kvm --vcpus=4 --cpuset=0-4 --ram 2048 --disk vol=my_pool/my_vol,bus=virtio,cache=none --network bridge=my_bridge,model=virtio -c /path/to/distrib.iso --keymap=fr --os-type=linux --noautoconsole"

</pre>

#TAP (if you want to rename the default "vnetX" interface by "tapX"

<pre>

[H] "virsh dumpxml my_guest > my_guest.xml

[H] "virsh undefine my_guest"

[H] Edit the my_guest.xml file to rename VNETX by TAPX

[H] "virsh define my_guest.xml"

[H] "virsh start my_guest"

</pre>

#NATed + LVM

<pre>

[H] "virt-install -d --hvm --vnc --name=my_kvm --vcpus=4 --cpuset=0-4 --ram 2048 --disk vol=my_pool/my_vol,bus=virtio,cache=none --network network=my_network,model=virtio -c /path/to/distrib.iso --keymap=fr --os-type=linux --noautoconsole"

</pre>

#Clone VM

<pre>

[H] Create the LVM dest

[H] Pause the KVM "virsh suspend my_kvm"

[H] virt-clone -o my_orig_kvm -n my_cloned_kvm -f=/path/to/lvm/dev

[G] Go and change the hostname and resolv.conf

</pre>

h3. Command explanation

#Bridged with tap

<pre>

virt-install -d --hvm --vnc --name=testlvm \

    --vcpus=4 --cpuset=0-4 \

    --ram 2048 \ 

    --disk vol=vmdata/testlvm,bus=virtio,cache=none \

    --network bridge=br0,model=virtio \

    -c /data/isos/debian-testing-amd64-netinst.iso \

    --keymap=fr --os-type=linux --os-variant=debiansqueeze --noautoconsole

</pre>

#NATed

<pre>

virt-install -d --hvm --vnc --name=testlvm \

    --vcpus=4 --cpuset=0-4 \

    --ram 2048 \ 

    --disk vol=vmdata/testlvm,bus=virtio,cache=none \

    --network network=frp_network,model=virtio \

    -c /data/isos/debian-testing-amd64-netinst.iso \

    --keymap=fr --os-type=linux --os-variant=debiansqueeze --noautoconsole

</pre>

[OLD QCOW2]

<pre>

[H] "virt-install --connect qemu:///system -n puppetnode1 -r 1024 --vcpus=4 --cpuset=4-7 --disk path=/vmroot/puppetnode1.qcow2 -c /data/isos/debian-6.0.6-amd64-netinst.iso --vnc --keymap=fr --noautoconsole --os-type linux --os-variant debiansqueeze --accelerate --network network=default --hvm"

[H] "virt-install --connect qemu:///system -n testnat -r 1024 --vcpus=2 --cpuset=2-3 --disk /vmroot/testnat.qcow2,format=qcow2,size=12,bus=virtio,cache=writeback -c /data/isos/debian-6.0.6-amd64-netinst.iso --vnc --keymap=fr --noautoconsole --os-type linux --os-variant debiansqueeze --accelerate --network bridge=br0 --hvm"

</pre>

#Start a KVM

<pre>

[H] "virsh start my_kvm"

</pre>

#Stop node

<pre>

[H] Soft : "virsh shutdown my_kvm"

[H] Hard : "virsh destroy my_kvm"

</pre>

#Suppr node del

<pre>

[H] "virsh undefine node" (+ remove files)

</pre>

#Rename node

<pre>

[H] "virsh dumpxml my_kvm > foo.xml"

[H] <edit foo.xml, change the name, move storage>

[H] "virsh undefine my_kvm"

[H] "virsh define foo.xml"

</pre>

# Connect to virsh qemu

<pre>

[H] "virsh --connect qemu:///system"

[H] "virsh # help"

</pre>

#Start a network

<pre>

[H] "virsh net-start my_network"

</pre>

#Configure VNC

<pre>

[H] Edit /etc/libvirt/qemu.conf and set vnc_listen = "your.ip.addd"

[H] You need to restart service "/etc/init.d/libvirtd restart"

[H] "ps aux | grep -i kvm" (find vnc display for your KVM)

</pre>

#Connect to a KVM through VNC

<pre>

tightvncviewer -compresslevel 7 -encodings "copyrect tight hextile zlib corre rre raw" x.xxx.xxx.xxx:x

</pre>

#Keyboard FR for KVM

<pre>

[H] "virsh edit my_kvm"

[XML] <graphics type='vnc' port='-1' autoport='yes' keymap='fr'/>

</pre>

#Create your own NAT network

<pre>

[H] Create your_nat.xml file :

[XML] <network>

  <name>frp_network</name>

  <forward mode='nat'/>

  <bridge name='br1' stp='on' delay='0' />

  <ip address='10.16.42.1' netmask='255.255.255.0'>

    <dhcp>

      <range start='10.16.42.2' end='10.16.42.50' />

    </dhcp>

  </ip>

</network>

[H] "virsh net-define your_nat.xml"

[H] "virsh net-start your_nat.xml"

[H] "virsh net-autostart your_nat.xml"

[XML] Network type "interface type = network, source network=your_nat"

</pre>

#IBM advisements to improve performance

<pre>

[H] Use LVM (or other direct access to HDD) for VMs storage (from 45 to 3' to install a VM)

[H] Set the swappiness value to 0 ("echo 0 > /proc/sys/vm/swappiness" && edit "/etc/sysctl.conf" and add "vm.swappiness=0")

[H] Disable zone_reclaim ("echo 0 > /proc/sys/vm/zone_reclaim_mode" && edit "/etc/sysctl.conf" and add "vm.zone_reclaim_mode=0")

[H] Enable KSM ("echo 1 > /sys/kernel/mm/ksm/run") and set the check time to 200ms ("echo 200 > /sys/kernel/mm/ksm/sleep_millisecs")

</pre>

h2. PC velizy

<pre>

root@tetaneutral1:~# fdisk /dev/sdb

Command (m for help): p

Disk /dev/sdb: 2000.4 GB, 2000398934016 bytes

255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0x9bf0c7df

   Device Boot      Start         End      Blocks   Id  System

Command (m for help): n

Partition type:

   p   primary (0 primary, 0 extended, 4 free)

   e   extended

Select (default p): p

Partition number (1-4, default 1): 

Using default value 1

First sector (2048-3907029167, default 2048): 

Using default value 2048

Last sector, +sectors or +size{K,M,G} (2048-3907029167, default 3907029167): 

Using default value 3907029167

Command (m for help): n

Partition type:

   p   primary (1 primary, 0 extended, 3 free)

   e   extended

Select (default p): ^C

root@tetaneutral1:~# fdisk /dev/sdb

Command (m for help): n

Partition type:

   p   primary (0 primary, 0 extended, 4 free)

   e   extended

Select (default p): p

Partition number (1-4, default 1): 1

First sector (2048-3907029167, default 2048): 

Using default value 2048

Last sector, +sectors or +size{K,M,G} (2048-3907029167, default 3907029167): 

Using default value 3907029167

Command (m for help): p

Disk /dev/sdb: 2000.4 GB, 2000398934016 bytes

255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0x9bf0c7df

   Device Boot      Start         End      Blocks   Id  System

/dev/sdb1            2048  3907029167  1953513560   83  Linux

Command (m for help): m

Command action

   a   toggle a bootable flag

   b   edit bsd disklabel

   c   toggle the dos compatibility flag

   d   delete a partition

   l   list known partition types

   m   print this menu

   n   add a new partition

   o   create a new empty DOS partition table

   p   print the partition table

   q   quit without saving changes

   s   create a new empty Sun disklabel

   t   change a partition's system id

   u   change display/entry units

   v   verify the partition table

   w   write table to disk and exit

   x   extra functionality (experts only)

Command (m for help): t

Selected partition 1

Hex code (type L to list codes): L

 0  Empty           24  NEC DOS         81  Minix / old Lin bf  Solaris        

 1  FAT12           27  Hidden NTFS Win 82  Linux swap / So c1  DRDOS/sec (FAT-

 2  XENIX root      39  Plan 9          83  Linux           c4  DRDOS/sec (FAT-

 3  XENIX usr       3c  PartitionMagic  84  OS/2 hidden C:  c6  DRDOS/sec (FAT-

 4  FAT16 <32M      40  Venix 80286     85  Linux extended  c7  Syrinx         

 5  Extended        41  PPC PReP Boot   86  NTFS volume set da  Non-FS data    

 6  FAT16           42  SFS             87  NTFS volume set db  CP/M / CTOS / .

 7  HPFS/NTFS/exFAT 4d  QNX4.x          88  Linux plaintext de  Dell Utility   

 8  AIX             4e  QNX4.x 2nd part 8e  Linux LVM       df  BootIt         

 9  AIX bootable    4f  QNX4.x 3rd part 93  Amoeba          e1  DOS access     

 a  OS/2 Boot Manag 50  OnTrack DM      94  Amoeba BBT      e3  DOS R/O        

 b  W95 FAT32       51  OnTrack DM6 Aux 9f  BSD/OS          e4  SpeedStor      

 c  W95 FAT32 (LBA) 52  CP/M            a0  IBM Thinkpad hi eb  BeOS fs        

 e  W95 FAT16 (LBA) 53  OnTrack DM6 Aux a5  FreeBSD         ee  GPT            

 f  W95 Ext'd (LBA) 54  OnTrackDM6      a6  OpenBSD         ef  EFI (FAT-12/16/

10  OPUS            55  EZ-Drive        a7  NeXTSTEP        f0  Linux/PA-RISC b

11  Hidden FAT12    56  Golden Bow      a8  Darwin UFS      f1  SpeedStor      

12  Compaq diagnost 5c  Priam Edisk     a9  NetBSD          f4  SpeedStor      

14  Hidden FAT16 <3 61  SpeedStor       ab  Darwin boot     f2  DOS secondary  

16  Hidden FAT16    63  GNU HURD or Sys af  HFS / HFS+      fb  VMware VMFS    

17  Hidden HPFS/NTF 64  Novell Netware  b7  BSDI fs         fc  VMware VMKCORE 

18  AST SmartSleep  65  Novell Netware  b8  BSDI swap       fd  Linux raid auto

1b  Hidden W95 FAT3 70  DiskSecure Mult bb  Boot Wizard hid fe  LANstep        

1c  Hidden W95 FAT3 75  PC/IX           be  Solaris boot    ff  BBT            

1e  Hidden W95 FAT1 80  Old Minix      

Hex code (type L to list codes): 8e

Changed system type of partition 1 to 8e (Linux LVM)

Command (m for help): w

The partition table has been altered!

Calling ioctl() to re-read partition table.

Syncing disks.

</pre>

<pre>

pvcreate /dev/sdb1

vgcreate vm-velizy /dev/sdb1

lvcreate -n velizy-vm1 -L 50G vm-velizy

apt-get install virtinst

/root/preseed.cfg @ http://git.tetaneutral.net/libvirt.git/

replace the late-command line at the end of the file by:

d-i preseed/late_command string cp -f /postinstall-velizy.sh /target/; in-target bash /postinstall-velizy.sh

vmname=velizy-vm1

ip=192.166.143.24

virt-install -n $vmname -r 2048 --os-type=linux \

    --debug \

    --autostart \

    --disk /dev/vm-velizy/$vmname,bus=virtio,cache=none,io=native \

    --network bridge=br0,model=virtio \

    --nographics \

    --noreboot \

    --wait -1 \

    --location http://ftp.debian.org/debian/dists/wheezy/main/installer-amd64/ \

    --serial pty \

    --initrd-inject /root/preseed.cfg \

    --initrd-inject /root/postinstall-velizy.sh \

    --extra-args "console=ttyS0,9600n8 DEBIAN_FRONTEND=text auto priority=critical hostname=$vmname domain=tetaneutral.net netcfg/get_ipaddress=$ip netcfg/get_netmask=255.255.252.0 netcfg/get_gateway=192.166.143.254 netcfg/get_nameservers=192.166.143.23 netcfg/disable_dhcp=true" 

</pre>

postinstall-velizy.sh

<pre>

#!/bin/bash

sed -i -e 's/^.*Port.*$/Port 2222/' -e 's/^[# ]*PasswordAuthentication.*$/PasswordAuthenticati

on no/' /etc/ssh/sshd_config

mkdir -p /root/.ssh

cat > /root/.ssh/authorized_keys  <<EOF

ssh-dss AAAAB3NzaC1kc3MAAACBAJtnGLvuz4uVD6fnERDxDi/C0UyzwCiKmgNtEessopREYasAX4Gu6Fg10jAyIL5Nuc

7YDnqj//pOfxNjD7hp99a2ZmkRQgh/ltClxYML9fqhBHgsUCpVse9nOYDHDnDgvXIPRSDUHat2UFpdchHVrFURNLIlZnBz

tsr+GCDURTV/AAAAFQD9SrBcH49ltaKOm6V5ssCPmgs1SwAAAIAbAvkQ3/tMzzdACO5B9s5Yb3ZuM7XImm0iqufivVf2Xy

39g8HOBulUsN7eKEGPhVriqNolgIO7q7tVgYAff4/NE4yDP0Kp0SPg4tjt2yFJuL31Y3wzwHjnKrjgNlDSLl3uZnRXSipw

UMwGKbdHB6NZSqlq192VKbHilQ00pbiMlAAAAIB5dfB1lVHkJ0o5CcaVQRPbca9DvbbRwnoTSmKHc5DUcqsPqDhS07CkM9

ZcJuY1Nh4wGl4Q9kArj7Tnsvvygf/HReSUcIk4+nbDytJ8/pca/Qx4fzQQyppa94TylN62LSFT6MIJKLoMwYa0dQURT7Mv

5+9Qj2vk5pZ38w2iQ9zVCg== root@h1

EOF

passwd -d root

</pre>

apt-get install debconf-utils

debconf-get-selections > preseed.cfg # marche pas