Projet

Général

Profil

DPDK » Historique » Version 17

Laurent GUERBY, 02/10/2014 11:57

1 1 Laurent GUERBY
{{>toc}}
2 1 Laurent GUERBY
3 1 Laurent GUERBY
h1. DPDK
4 1 Laurent GUERBY
5 1 Laurent GUERBY
Traitement haute performance en logiciel, 10 Gbit/s en petit paquets line rate
6 1 Laurent GUERBY
7 1 Laurent GUERBY
h2. Liens
8 1 Laurent GUERBY
9 1 Laurent GUERBY
* http://dpdk.org
10 1 Laurent GUERBY
* [[AtelierPPS2012]]
11 16 Laurent GUERBY
* GIT http://chiliproject.tetaneutral.net/projects/git-tetaneutral-net/repository/dpdk
12 16 Laurent GUERBY
<pre>
13 16 Laurent GUERBY
$ git clone ssh://git@git.tetaneutral.net/dpdk.git
14 16 Laurent GUERBY
</pre>
15 16 Laurent GUERBY
16 17 Laurent GUERBY
* Nerim Antoine https://twitter.com/nerim_net/status/517592380841930752
17 17 Laurent GUERBY
** Du code maison fondé sur #DPDK en prod dans le réseau : première étape vers l'élimination des systèmes propriétaires ?
18 2 Laurent GUERBY
19 2 Laurent GUERBY
h2. Misc
20 2 Laurent GUERBY
21 2 Laurent GUERBY
* Intel PCI id http://www.intel.com/support/network/sb/cs-012904.htm
22 3 Laurent GUERBY
* http://www.intel.com/content/www/us/en/ethernet-controllers/ethernet-products.html?iid=embed_portal+hdprod_ethernet#s1=Gigabit%20Ethernet&s2=all&s3=all
23 3 Laurent GUERBY
* Intel Pro/1000 PT vs CT http://hardforum.com/showthread.php?t=1386042
24 4 Laurent GUERBY
* http://images.ncix.com/forumimages/C9864B87-583B-10FB-B0DE80742781CE07.jpg
25 4 Laurent GUERBY
* http://www.intel.com/content/www/us/en/ethernet-controllers/ethernet-controllers.html#s1=all&s2=82572EI&s3=all
26 4 Laurent GUERBY
* http://www.intel.com/content/www/us/en/ethernet-controllers/ethernet-controllers.html#s1=all&s2=82574L&s3=all
27 5 Laurent GUERBY
* https://github.com/Pktgen/Pktgen-DPDK
28 6 Laurent GUERBY
* Evaluating the Suitability of Server Network Cards for Software Routers http://kfall.net/ucbpage/papers/snc.pdf
29 13 Laurent GUERBY
* http://www.intel.fr/content/dam/www/public/us/en/documents/guides/intel-dpdk-sample-applications-user-guide.pdf
30 13 Laurent GUERBY
*  rte_eth_rx_burst
31 13 Laurent GUERBY
** http://dpdk.org/doc/api/rte__ethdev_8h.html#a5ac93faa0f3f69545ca121dcfd9433f5
32 13 Laurent GUERBY
** http://dpdk.info/ml/archives/dev/2014-January/001223.html
33 14 Laurent GUERBY
* http://www.intel.fr/content/dam/www/public/us/en/documents/white-papers/aes-ipsec-performance-linux-paper.pdf
34 15 Laurent GUERBY
* http://networkbuilders.intel.com/docs/Network_Builders_RA_vBRAS_Final.pdf
35 15 Laurent GUERBY
36 15 Laurent GUERBY
* https://github.com/SnabbCo/snabbswitch
37 15 Laurent GUERBY
** https://github.com/SnabbCo/snabbswitch/blob/master/src/apps/intel/intel10g.lua
38 15 Laurent GUERBY
** http://blog.ipspace.net/2014/06/snabb-switch-and-nfv-on-openstack-in.html
39 15 Laurent GUERBY
** https://github.com/SnabbCo/snabbswitch/wiki/Snabb-Lab
40 15 Laurent GUERBY
** https://twitter.com/lukego/status/327714050219847680
41 15 Laurent GUERBY
42 7 Mehdi Abaakouk
43 7 Mehdi Abaakouk
h2. Note sileht dpdk:
44 7 Mehdi Abaakouk
45 7 Mehdi Abaakouk
Extract from: http://www.intel.com/content/dam/www/public/us/en/documents/guides/intel-dpdk-getting-started-guide.pdf
46 7 Mehdi Abaakouk
47 7 Mehdi Abaakouk
h3. configuration hugepages: 
48 7 Mehdi Abaakouk
49 7 Mehdi Abaakouk
* 2M (1024*2k): hugepages=1024 (at runtime: echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages)
50 7 Mehdi Abaakouk
* 4G (4x1G): default_hugepagesz=1G hugepagesz=1G hugepages=4 (only works at boot time via grub)
51 7 Mehdi Abaakouk
52 7 Mehdi Abaakouk
<pre>
53 7 Mehdi Abaakouk
mkdir /mnt/huge
54 7 Mehdi Abaakouk
mount -t hugetlbfs nodev /mnt/huge
55 7 Mehdi Abaakouk
</pre>
56 7 Mehdi Abaakouk
57 7 Mehdi Abaakouk
h3. Compile and load modules: 
58 7 Mehdi Abaakouk
59 7 Mehdi Abaakouk
_Note: source tools/setup.sh is a helper tools for this but works only with IGB driver not e1000e_
60 7 Mehdi Abaakouk
61 1 Laurent GUERBY
<pre>
62 8 Mehdi Abaakouk
# apt-get install make gcc linux-headers-amd64
63 8 Mehdi Abaakouk
# make config T=x86_64-default-linuxapp-gcc
64 7 Mehdi Abaakouk
# make T=x86_64-default-linuxapp-gcc
65 7 Mehdi Abaakouk
..
66 7 Mehdi Abaakouk
Build complete
67 7 Mehdi Abaakouk
68 7 Mehdi Abaakouk
# modprobe uio  (I think this is not useful)
69 7 Mehdi Abaakouk
# insmod build/kmod/rte_kni.ko (I think this is not useful)
70 7 Mehdi Abaakouk
# insmod build/kmod/igb_uio.ko (I think this is not useful)
71 7 Mehdi Abaakouk
# ./tools/pci_unbind.py --status
72 7 Mehdi Abaakouk
73 7 Mehdi Abaakouk
Network devices using IGB_UIO driver
74 7 Mehdi Abaakouk
====================================
75 7 Mehdi Abaakouk
<none>
76 7 Mehdi Abaakouk
77 7 Mehdi Abaakouk
Network devices using kernel driver
78 7 Mehdi Abaakouk
===================================
79 9 Mehdi Abaakouk
0000:00:19.0 '82579LM Gigabit Network Connection' if=eth1 drv=e1000e unused=<none>
80 9 Mehdi Abaakouk
0000:01:00.0 '82599EB 10-Gigabit SFI/SFP+ Network Connection' if=eth6 drv=ixgbe unused=igb_uio *Active*
81 9 Mehdi Abaakouk
0000:01:00.1 '82599EB 10-Gigabit SFI/SFP+ Network Connection' if=eth7 drv=ixgbe unused=igb_uio
82 9 Mehdi Abaakouk
0000:04:00.0 '82571EB Gigabit Ethernet Controller' if=eth2 drv=e1000e unused=igb_uio *Active*
83 9 Mehdi Abaakouk
0000:04:00.1 '82571EB Gigabit Ethernet Controller' if=eth3 drv=e1000e unused=igb_uio
84 9 Mehdi Abaakouk
0000:05:00.0 '82571EB Gigabit Ethernet Controller' if=eth4 drv=e1000e unused=igb_uio
85 9 Mehdi Abaakouk
0000:05:00.1 '82571EB Gigabit Ethernet Controller' if=eth5 drv=e1000e unused=igb_uio
86 9 Mehdi Abaakouk
0000:06:00.0 'RTL8111/8168B PCI Express Gigabit Ethernet controller' if=eth0 drv=r8169 unused=<none> *Active*
87 1 Laurent GUERBY
88 1 Laurent GUERBY
Other network devices
89 1 Laurent GUERBY
=====================
90 1 Laurent GUERBY
<none>
91 9 Mehdi Abaakouk
</pre>
92 7 Mehdi Abaakouk
93 9 Mehdi Abaakouk
test sur eth7:
94 9 Mehdi Abaakouk
95 9 Mehdi Abaakouk
<pre>
96 9 Mehdi Abaakouk
# ip link set eth7 down
97 9 Mehdi Abaakouk
# ./tools/pci_unbind.py --bind=igb_uio 0000:01:00.1
98 7 Mehdi Abaakouk
</pre>
99 7 Mehdi Abaakouk
100 7 Mehdi Abaakouk
h3. Prepare examples programs:
101 7 Mehdi Abaakouk
102 7 Mehdi Abaakouk
<pre>
103 7 Mehdi Abaakouk
# export RTE_SDK=/root/sileht/dpdk-1.6.0r1
104 7 Mehdi Abaakouk
# export RTE_TARGET=build
105 7 Mehdi Abaakouk
# cd /root/sileht/
106 7 Mehdi Abaakouk
# cp -r $RTE_SDK/examples/helloworld my_rte_app
107 7 Mehdi Abaakouk
# cd my_rte_app
108 7 Mehdi Abaakouk
# make
109 7 Mehdi Abaakouk
</pre>
110 7 Mehdi Abaakouk
111 7 Mehdi Abaakouk
112 7 Mehdi Abaakouk
h3. Tests
113 7 Mehdi Abaakouk
114 7 Mehdi Abaakouk
<pre>
115 1 Laurent GUERBY
</pre>
116 10 Mehdi Abaakouk
117 11 Mehdi Abaakouk
h2. Limitation SPF carte intel:
118 10 Mehdi Abaakouk
119 11 Mehdi Abaakouk
Une fois le module igb_uio changé sur les:
120 10 Mehdi Abaakouk
121 10 Mehdi Abaakouk
<pre>
122 10 Mehdi Abaakouk
01:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
123 10 Mehdi Abaakouk
</pre>
124 10 Mehdi Abaakouk
125 11 Mehdi Abaakouk
Un flags de l'eeprom change et le module ixgbe ne charge plus les ports de la carte qui n'ont pas des SFP+ intel avec le message d'erreur suivant:
126 10 Mehdi Abaakouk
127 10 Mehdi Abaakouk
<pre>
128 10 Mehdi Abaakouk
[  563.069683] ixgbe 0000:01:00.0: failed to load because an unsupported SFP+ or QSFP module type was detected.
129 10 Mehdi Abaakouk
[  563.069786] ixgbe 0000:01:00.0: Reload the driver after installing a supported module.
130 10 Mehdi Abaakouk
</pre>
131 10 Mehdi Abaakouk
132 10 Mehdi Abaakouk
Pour contourner le problème:
133 10 Mehdi Abaakouk
134 10 Mehdi Abaakouk
<pre>
135 10 Mehdi Abaakouk
modprobe ixgbe allow_unsupported_sfp=1 # allow_any_sfp=1 suivant le kernel
136 10 Mehdi Abaakouk
</pre>
137 10 Mehdi Abaakouk
138 10 Mehdi Abaakouk
ou 
139 10 Mehdi Abaakouk
140 1 Laurent GUERBY
<pre>
141 1 Laurent GUERBY
echo "options ixgbe allow_unsupported_sfp=1" > /etc/modprobe.d/ixgbe_any_sfp.conf
142 1 Laurent GUERBY
update-initramfs -u -k all
143 11 Mehdi Abaakouk
</pre>
144 11 Mehdi Abaakouk
145 11 Mehdi Abaakouk
La carte fonctionnera à nouveau avec le message suivant:
146 11 Mehdi Abaakouk
147 11 Mehdi Abaakouk
<pre>
148 11 Mehdi Abaakouk
[  908.361767] ixgbe 0000:01:00.0 (unregistered net_device): WARNING: Intel (R) Network Connections are quality tested using Intel (R) Ethernet Optics.  Using untested modules is not supported and may cause unstable operation or damage to the module or the adapter.  Intel Corporation is not responsible for any harm caused by using untested modules.
149 10 Mehdi Abaakouk
</pre>
150 12 Laurent GUERBY
151 12 Laurent GUERBY
h2. Atelier 20140705
152 12 Laurent GUERBY
153 12 Laurent GUERBY
TODO