{{>toc}}

h1. DPDK

Traitement haute performance en logiciel, 10 Gbit/s en petit paquets line rate

h2. Liens

* http://dpdk.org

* [[AtelierPPS2012]]

* GIT http://chiliproject.tetaneutral.net/projects/git-tetaneutral-net/repository/dpdk

<pre>

$ git clone ssh://git@git.tetaneutral.net/dpdk.git

</pre>

* Nerim Antoine https://twitter.com/nerim_net/status/517592380841930752

** Du code maison fondé sur #DPDK en prod dans le réseau : première étape vers l'élimination des systèmes propriétaires ?

h2. Misc

* https://www.kernel.org/doc/Documentation/networking/switchdev.txt

* https://www.usenix.org/conference/osdi14/technical-sessions/presentation/belay

** IX: A Protected Dataplane Operating System for High Throughput and Low Latency

** http://www.linuxfoundation.org/images/1/1c/Network_data_flow_through_kernel.png

** http://www.linuxfoundation.org/collaborate/workgroups/networking/kernel_flow

* https://blog.cloudflare.com/how-to-receive-a-million-packets/

** How to receive a million packets per second

* https://blog.cloudflare.com/how-to-achieve-low-latency/

* http://www.cs.cmu.edu/~dongz/papers/cuckooswitch.pdf

** We show that CUCKOOSWITCH can process 92.22 million minimum sized packets per second on a commodity server equipped with eight 10 Gbps Ethernet interfaces while maintaining a forwarding table of one billion forwarding entries.

* Improving Linux networking performance https://lwn.net/Articles/629155/

** https://www.youtube.com/watch?v=3XG9-X777Jo

** http://people.netfilter.org/hawk/presentations/LCA2015/net_stack_challenges_100G_LCA2015.pdf

* http://www.bigswitch.com/sites/default/files/presentations/onug-baremetal-2014-final.pdf

* Intel PCI id http://www.intel.com/support/network/sb/cs-012904.htm

* http://www.intel.com/content/www/us/en/ethernet-controllers/ethernet-products.html?iid=embed_portal+hdprod_ethernet#s1=Gigabit%20Ethernet&s2=all&s3=all

* Intel Pro/1000 PT vs CT http://hardforum.com/showthread.php?t=1386042

* http://images.ncix.com/forumimages/C9864B87-583B-10FB-B0DE80742781CE07.jpg

* http://www.intel.com/content/www/us/en/ethernet-controllers/ethernet-controllers.html#s1=all&s2=82572EI&s3=all

* http://www.intel.com/content/www/us/en/ethernet-controllers/ethernet-controllers.html#s1=all&s2=82574L&s3=all

* https://github.com/Pktgen/Pktgen-DPDK

* Evaluating the Suitability of Server Network Cards for Software Routers http://kfall.net/ucbpage/papers/snc.pdf

* http://www.intel.fr/content/dam/www/public/us/en/documents/guides/intel-dpdk-sample-applications-user-guide.pdf

*  rte_eth_rx_burst

** http://dpdk.org/doc/api/rte__ethdev_8h.html#a5ac93faa0f3f69545ca121dcfd9433f5

** http://dpdk.info/ml/archives/dev/2014-January/001223.html

* http://www.intel.fr/content/dam/www/public/us/en/documents/white-papers/aes-ipsec-performance-linux-paper.pdf

* http://networkbuilders.intel.com/docs/Network_Builders_RA_vBRAS_Final.pdf

* https://github.com/SnabbCo/snabbswitch

** https://github.com/SnabbCo/snabbswitch/blob/master/src/apps/intel/intel10g.lua

** http://blog.ipspace.net/2014/06/snabb-switch-and-nfv-on-openstack-in.html

** https://github.com/SnabbCo/snabbswitch/wiki/Snabb-Lab

** https://twitter.com/lukego/status/327714050219847680

h2. Note sileht dpdk:

Extract from: http://www.intel.com/content/dam/www/public/us/en/documents/guides/intel-dpdk-getting-started-guide.pdf

h3. configuration hugepages: 

* 2M (1024*2k): hugepages=1024 (at runtime: echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages)

* 4G (4x1G): default_hugepagesz=1G hugepagesz=1G hugepages=4 (only works at boot time via grub)

<pre>

mkdir /mnt/huge

mount -t hugetlbfs nodev /mnt/huge

</pre>

h3. Compile and load modules: 

_Note: source tools/setup.sh is a helper tools for this but works only with IGB driver not e1000e_

<pre>

# apt-get install make gcc linux-headers-amd64

# make config T=x86_64-default-linuxapp-gcc

# make T=x86_64-default-linuxapp-gcc

..

Build complete

# modprobe uio  (I think this is not useful)

# insmod build/kmod/rte_kni.ko (I think this is not useful)

# insmod build/kmod/igb_uio.ko (I think this is not useful)

# ./tools/pci_unbind.py --status

Network devices using IGB_UIO driver

====================================

<none>

Network devices using kernel driver

===================================

0000:00:19.0 '82579LM Gigabit Network Connection' if=eth1 drv=e1000e unused=<none>

0000:01:00.0 '82599EB 10-Gigabit SFI/SFP+ Network Connection' if=eth6 drv=ixgbe unused=igb_uio *Active*

0000:01:00.1 '82599EB 10-Gigabit SFI/SFP+ Network Connection' if=eth7 drv=ixgbe unused=igb_uio

0000:04:00.0 '82571EB Gigabit Ethernet Controller' if=eth2 drv=e1000e unused=igb_uio *Active*

0000:04:00.1 '82571EB Gigabit Ethernet Controller' if=eth3 drv=e1000e unused=igb_uio

0000:05:00.0 '82571EB Gigabit Ethernet Controller' if=eth4 drv=e1000e unused=igb_uio

0000:05:00.1 '82571EB Gigabit Ethernet Controller' if=eth5 drv=e1000e unused=igb_uio

0000:06:00.0 'RTL8111/8168B PCI Express Gigabit Ethernet controller' if=eth0 drv=r8169 unused=<none> *Active*

Other network devices

=====================

<none>

</pre>

test sur eth7:

<pre>

# ip link set eth7 down

# ./tools/pci_unbind.py --bind=igb_uio 0000:01:00.1

</pre>

h3. Prepare examples programs:

<pre>

# export RTE_SDK=/root/sileht/dpdk-1.6.0r1

# export RTE_TARGET=build

# cd /root/sileht/

# cp -r $RTE_SDK/examples/helloworld my_rte_app

# cd my_rte_app

# make

</pre>

h3. Tests

<pre>

</pre>

h2. Limitation SPF carte intel:

Une fois le module igb_uio changé sur les:

<pre>

01:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

</pre>

Un flags de l'eeprom change et le module ixgbe ne charge plus les ports de la carte qui n'ont pas des SFP+ intel avec le message d'erreur suivant:

<pre>

[  563.069683] ixgbe 0000:01:00.0: failed to load because an unsupported SFP+ or QSFP module type was detected.

[  563.069786] ixgbe 0000:01:00.0: Reload the driver after installing a supported module.

</pre>

Pour contourner le problème:

<pre>

modprobe ixgbe allow_unsupported_sfp=1 # allow_any_sfp=1 suivant le kernel

</pre>

ou 

<pre>

echo "options ixgbe allow_unsupported_sfp=1" > /etc/modprobe.d/ixgbe_any_sfp.conf

update-initramfs -u -k all

</pre>

La carte fonctionnera à nouveau avec le message suivant:

<pre>

[  908.361767] ixgbe 0000:01:00.0 (unregistered net_device): WARNING: Intel (R) Network Connections are quality tested using Intel (R) Ethernet Optics.  Using untested modules is not supported and may cause unstable operation or damage to the module or the adapter.  Intel Corporation is not responsible for any harm caused by using untested modules.

</pre>

h2. Packet Journey

https://www.gandi.net/news/en/2015-12-16/6308-packet-journey_a_free_software_router_for_linux_based_on_dpdk/

https://github.com/gandi/packet-journey

h2. Atelier 2016

<pre>

core i7 2600 @ 3.40GHz

16G RAM

SSD 250G

</pre>

<pre>

host eth DHCP          10G-a                                           10G-b

t1   54:04:a6:ef:a8:c7 00:1b:21:d9:28:55 fe80::21b:21ff:fed9:2855(t2b) 00:1b:21:d9:28:54 fe80::21b:21ff:fed9:2854(t3b)

t2   e0:69:95:2e:e6:a2 00:1b:21:d9:1a:e1 fe80::21b:21ff:fed9:1ae1(t3a) 00:1b:21:d9:1a:e0 fe80::21b:21ff:fed9:1ae0(t1a)

t3   bc:ae:c5:e1:b2:6d 00:1b:21:d9:df:3d fe80::21b:21ff:fed9:df3d(t2a) 00:1b:21:d9:df:3c fe80::21b:21ff:fed9:df3c(t1b)

</pre>

http://dpdk.org/doc/guides/linux_gsg/build_dpdk.html

<pre>

modprobe uio_pci_generic

wget http://dpdk.org/browse/dpdk/snapshot/dpdk-2.2.0.tar.gz

git clone https://github.com/Gandi/packet-journey.git

git clone http://dpdk.org/git/apps/pktgen-dpdk

git clone http://dpdk.org/git/dpdk-stable

tar xf dpdk-2.2.0.tar.gz

cd dpdk-2.2.0

make config T=x86_64-native-linuxapp-gcc

make install T=x86_64-native-linuxapp-gcc

</pre>