Projet

Général

Profil

AtelierPPS2012 » Historique » Version 78

Mehdi Abaakouk, 28/04/2014 12:38

1 1 Laurent GUERBY
{{>toc}}
2 1 Laurent GUERBY
3 1 Laurent GUERBY
h1. AtelierPPS2012
4 1 Laurent GUERBY
5 32 Laurent GUERBY
Une attaque sur le réseau gitoyen a eu lieu le 18 juin et une sur tetaneutral.net le 29 juin, ces deux attaques etaient en "paquet par seconde" (PPS) avec de petits paquets de 50-60 byte qui saturent les CPU des routeurs logiciels.
6 1 Laurent GUERBY
7 32 Laurent GUERBY
L'idée est d'étudier via des recherches sur le web et des laboratoires/ateliers le comportement des routeurs logiciels dans ce cas la : limites atteintes en fonction du paramétrage et du matériel (carte réseau, CPU et fréquence).
8 1 Laurent GUERBY
9 1 Laurent GUERBY
h2. Liens
10 1 Laurent GUERBY
11 16 Laurent GUERBY
12 2 Laurent GUERBY
* http://lists.tetaneutral.net/pipermail/technique/2012-July/000406.html
13 2 Laurent GUERBY
* http://guerby.org/ftp/dos-tetaneutral-20120629-12h33-13h03-pps.png
14 24 Laurent GUERBY
* http://networkstatic.net/the-sdn-impact-on-net-neutrality/
15 16 Laurent GUERBY
* http://blog.exceliance.fr/2012/04/24/hypervisors-virtual-network-performance-comparison-from-a-virtualized-load-balancer-point-of-view/
16 3 Laurent GUERBY
* http://www.spinics.net/lists/netdev/msg206077.html
17 3 Laurent GUERBY
** So with your patch, Eric's patch, and this most recent patch we are now at 11.8Mpps with 8 or 9 queues.  At this point I am staring to hit the hardware limits since 82599 will typically max out at about 12Mpps w/ 9 queues.
18 3 Laurent GUERBY
** 12e6 * 64 byte * 8  = 6.1 Gbit/s
19 18 Laurent GUERBY
** PATCH Remove the ipv4 routing cache http://www.spinics.net/lists/netdev/msg205545.html
20 32 Laurent GUERBY
* Intel® 82599 10 Gigabit Ethernet Controller http://ark.intel.com/products/series/32609
21 4 Laurent GUERBY
* more interrupts (lower performance) in bare-metal compared with running VM https://lkml.org/lkml/2012/7/27/490
22 3 Laurent GUERBY
23 3 Laurent GUERBY
100 Mbit/s = 195312 frames de 64 byte/s
24 3 Laurent GUERBY
1000 Mbit/s = 1953125 frames de 64 byte/s
25 31 Laurent GUERBY
* http://dpdk.org/ml/archives/dev/2013-May/000102.html
26 31 Laurent GUERBY
** In case of 64 byte packets (with Ethernet CRC), (64+20)*8 = 672 bits. So line rate is 10000/672 = 14.88 Mpps.
27 39 Laurent GUERBY
** Intel Data Plane Development Kit (Intel® DPDK) Overview Packet Processing on Intel® Architecture  http://www.intel.com/content/dam/www/public/us/en/documents/presentation/dpdk-packet-processing-ia-overview-presentation.pdf
28 35 Laurent GUERBY
* http://www.intel.com/content/www/us/en/intelligent-systems/intel-technology/packet-processing-is-enhanced-with-software-from-intel-dpdk.html
29 35 Laurent GUERBY
** 80 Mpps par processeur Xeon
30 35 Laurent GUERBY
** http://www.intel.com/content/www/us/en/communications/communications-packet-processing-brief.html
31 5 Laurent GUERBY
* discussion choix d'un routeur et attaque PPS : http://www.mail-archive.com/frnog@frnog.org/msg19673.html
32 10 Laurent GUERBY
* projet netmap http://info.iet.unipi.it/~luigi/netmap/
33 10 Laurent GUERBY
** http://lwn.net/Articles/484323/
34 6 Laurent GUERBY
** http://info.iet.unipi.it/~luigi/papers/20120503-netmap-atc12.pdf
35 7 Laurent GUERBY
*** "In our prototype, a single core running at 900 MHz can send or receive 14.88 Mpps (the peak packet rate on 10 Gbit/s links). This is more than 20 times faster than conventional APIs."
36 8 Laurent GUERBY
** http://info.iet.unipi.it/~luigi/netmap/20110729-rizzo-infocom.pdf
37 8 Laurent GUERBY
** VALE, a Virtual Local Ethernet http://info.iet.unipi.it/~luigi/vale/
38 1 Laurent GUERBY
*** http://info.iet.unipi.it/~luigi/papers/20120608-vale.pdf
39 1 Laurent GUERBY
*** " Our architecture, called VALE, implements a Virtual Local Ethernet that can be used by virtual machines such as QEMU, KVM and others, as well as regular processes, to achieve over 17 million packets per second (Mpps) between host processes, and over 2 Mpps between QEMU instances, without any hardware assistance"
40 1 Laurent GUERBY
** Towards a Billion Routing Lookups per Second in Software http://info.iet.unipi.it/~luigi/papers/20120601-dxr.pdf
41 13 Laurent GUERBY
** http://info.iet.unipi.it/~luigi/netmap/talk-hp.html
42 13 Laurent GUERBY
** http://marc.info/?a=133836981100006&r=1&w=4
43 14 Laurent GUERBY
** 10 Gbit/s Line Rate Packet Processing Using Commodity Hardware: Survey and new Proposals http://luca.ntop.org/10g.pdf
44 10 Laurent GUERBY
* http://www.intel.com/content/www/us/en/ethernet-controllers/82599-10-gbe-controller-datasheet.html
45 10 Laurent GUERBY
* ipfw 9-10 Mpps http://lists.freebsd.org/pipermail/freebsd-net/2012-July/032869.html
46 19 Laurent GUERBY
* projet PFQ
47 19 Laurent GUERBY
** http://netgroup.iet.unipi.it/software/pfq/index.html
48 17 Laurent GUERBY
* Ubiquity EdgeMax router
49 17 Laurent GUERBY
** http://www.ubnt.com/edgemax
50 17 Laurent GUERBY
** http://forum.ubnt.com/showthread.php?t=59312
51 17 Laurent GUERBY
** http://dl.ubnt.com/Tolly212127UbiquitiEdgeRouterLitePricePerformance.pdf
52 17 Laurent GUERBY
** http://dl.ubnt.com/Tolly212128UbiquitiEdgeRouterLitePricePerformanceVsMikroTik.pdf
53 25 Laurent GUERBY
* http://dpdk.org/
54 25 Laurent GUERBY
** Intel DPDK: Data Plane Development Kit
55 25 Laurent GUERBY
** Intel DPDK is a set of libraries and drivers for fast packet processing on x86 platforms. It runs mostly in Linux userland.
56 26 Laurent GUERBY
* http://www.slideshare.net/shemminger/uio-final
57 26 Laurent GUERBY
** Networking in Userspace : Living on the edge
58 27 Laurent GUERBY
* http://tech.slashdot.org/story/13/04/17/2014206/vint-cerf-sdn-is-a-model-for-a-better-internet
59 27 Laurent GUERBY
** http://slashdot.org/topic/datacenter/vint-cerf-sdn-is-a-model-for-a-better-internet/
60 28 Laurent GUERBY
* http://www.opendaylight.org/
61 28 Laurent GUERBY
** OpenDaylight's mission is to facilitate a community-led, industry-supported open source framework, including code and architecture, to accelerate and advance a common, robust Software-Defined Networking platform
62 10 Laurent GUERBY
63 30 Laurent GUERBY
* http://www.packetdam.com/
64 30 Laurent GUERBY
65 10 Laurent GUERBY
* http://www.cisco.com/web/partners/downloads/765/tools/quickreference/routerperformance.pdf
66 8 Laurent GUERBY
67 11 Laurent GUERBY
* http://osdir.com/ml/linux.drivers.e1000.devel/2007-05/msg00182.html
68 11 Laurent GUERBY
** "The network cards are perfectly capable of achieving much higher numbers than  135k pps. The linux network stack however is currently not."
69 11 Laurent GUERBY
* http://code.google.com/p/openpgm/
70 12 Laurent GUERBY
* http://afresh1.com/OpenBSD_49_Throughput_Latency/
71 5 Laurent GUERBY
72 20 Laurent GUERBY
* http://code.ettus.com/redmine/ettus/projects/public/wiki/Latency
73 20 Laurent GUERBY
74 32 Laurent GUERBY
* 10Gbps Open Source Routing » de Bengt Gördén, Olof Hagsand et Robert Olsson http://www.iis.se/docs/10G-OS-router_2_.pdf
75 22 Laurent GUERBY
* http://fr.slideshare.net/brouer/linuxcon2009-10gbits-bidirectional-routing-on-standard-hardware-running-linux
76 22 Laurent GUERBY
* 10 Gbit Hardware Packet Filtering Using Commodity Network Adapters http://ripe61.ripe.net/presentations/138-Deri_RIPE_61.pdf
77 23 Laurent GUERBY
* https://wiki.freebsd.org/NetworkPerformanceTuning
78 1 Laurent GUERBY
79 21 Laurent GUERBY
*  http://wiki.networksecuritytoolkit.org/nstwiki/index.php/LAN_Ethernet_Maximum_Rates,_Generation,_Capturing_%26_Monitoring
80 1 Laurent GUERBY
* http://www.cisco.com/web/about/security/intelligence/network_performance_metrics.html
81 1 Laurent GUERBY
82 32 Laurent GUERBY
* http://blog.erratasec.com/2013/12/ccc-100-gbps-and-your-own-private-shodan.html
83 33 Laurent GUERBY
* https://github.com/robertdavidgraham/masscan
84 32 Laurent GUERBY
* http://www.ntop.org/products/pf_ring/
85 29 Laurent GUERBY
86 34 Laurent GUERBY
* http://routebricks.org/pubs.html
87 34 Laurent GUERBY
88 36 Laurent GUERBY
* http://lwn.net/Articles/542643/
89 36 Laurent GUERBY
** Chelsio's T5 asic moves the architecture into 40GbE speeds. T5 is a 10/40GbE controller with full offload support of a complete Unified Wire solution comprising NIC, Virtualization, TOE, iWARP RDMA and FCoE.
90 36 Laurent GUERBY
** http://dpdk.org/ml/archives/dev/2014-January/001111.html fix atomic and out of order	execution
91 36 Laurent GUERBY
92 37 Laurent GUERBY
* http://blog.erratasec.com/2013/10/whats-max-speed-on-ethernet.html
93 37 Laurent GUERBY
** What's the max speed on Ethernet?
94 38 Laurent GUERBY
* http://bsdrp.net/documentation/examples/forwarding_performance_lab_of_a_superserver_5018a-ftn4
95 37 Laurent GUERBY
96 32 Laurent GUERBY
h2. Personnes interessées
97 32 Laurent GUERBY
98 1 Laurent GUERBY
# Laurent GUERBY
99 32 Laurent GUERBY
# Obinou (qui a déjà utilisé PF-RING et NTOP)
100 1 Laurent GUERBY
101 1 Laurent GUERBY
A priori il suffit de deux machines pour pouvoir commencer chez soi.
102 11 Laurent GUERBY
103 11 Laurent GUERBY
h2. Tests
104 11 Laurent GUERBY
105 11 Laurent GUERBY
e1000e D2500CC (squeeze) et core i5 DQ67SW (squeeze + kernel 3.2bpo)
106 11 Laurent GUERBY
iperf plafonne a 120-130k pps
107 78 Mehdi Abaakouk
108 78 Mehdi Abaakouk
h2. Note sileht dpdk:
109 78 Mehdi Abaakouk
110 78 Mehdi Abaakouk
Extract from: http://www.intel.com/content/dam/www/public/us/en/documents/guides/intel-dpdk-getting-started-guide.pdf
111 78 Mehdi Abaakouk
112 78 Mehdi Abaakouk
h3. configuration hugepages: 
113 78 Mehdi Abaakouk
* 2M (1024*2k): hugepages=1024 (at runtime: echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages)
114 78 Mehdi Abaakouk
* 4G (4x1G): default_hugepagesz=1G hugepagesz=1G hugepages=4 (only works at boot time via grub)
115 78 Mehdi Abaakouk
116 78 Mehdi Abaakouk
<pre>
117 78 Mehdi Abaakouk
mkdir /mnt/huge
118 78 Mehdi Abaakouk
mount -t hugetlbfs nodev /mnt/huge
119 78 Mehdi Abaakouk
</pre>
120 78 Mehdi Abaakouk
121 78 Mehdi Abaakouk
122 78 Mehdi Abaakouk
h3. Compile and load modules: 
123 78 Mehdi Abaakouk
124 78 Mehdi Abaakouk
_Note: source tools/setup.sh is a helper tools for this_
125 78 Mehdi Abaakouk
126 78 Mehdi Abaakouk
<pre>
127 78 Mehdi Abaakouk
# make T=x86_64-default-linuxapp-gcc
128 78 Mehdi Abaakouk
..
129 78 Mehdi Abaakouk
Build complete
130 78 Mehdi Abaakouk
131 78 Mehdi Abaakouk
# modprobe uio  (I think this is not useful)
132 78 Mehdi Abaakouk
# insmod build/kmod/rte_kni.ko (I think this is not useful)
133 78 Mehdi Abaakouk
# insmod build/kmod/igb_uio.ko (I think this is not useful)
134 78 Mehdi Abaakouk
# ./tools/pci_unbind.py --status
135 78 Mehdi Abaakouk
136 78 Mehdi Abaakouk
Network devices using IGB_UIO driver
137 78 Mehdi Abaakouk
====================================
138 78 Mehdi Abaakouk
<none>
139 78 Mehdi Abaakouk
140 78 Mehdi Abaakouk
Network devices using kernel driver
141 78 Mehdi Abaakouk
===================================
142 78 Mehdi Abaakouk
0000:00:19.0 'Ethernet Connection I217-LM' if=eth1 drv=e1000e unused=<none> *Active*
143 78 Mehdi Abaakouk
0000:04:00.0 'RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller' if=eth0 drv=r8169 unused=<none> 
144 78 Mehdi Abaakouk
145 78 Mehdi Abaakouk
Other network devices
146 78 Mehdi Abaakouk
=====================
147 78 Mehdi Abaakouk
<none>
148 78 Mehdi Abaakouk
149 78 Mehdi Abaakouk
# ip link set eth1 down
150 78 Mehdi Abaakouk
# lspci|grep -i 'Ethernet.*Intel'
151 78 Mehdi Abaakouk
00:19.0 Ethernet controller: Intel Corporation Ethernet Connection I217-LM (rev 05)
152 78 Mehdi Abaakouk
# ./tools/pci_unbind.py --bind=e1000e 00:19.0
153 78 Mehdi Abaakouk
</pre>
154 78 Mehdi Abaakouk
155 78 Mehdi Abaakouk
h3. Prepare examples programs:
156 78 Mehdi Abaakouk
157 78 Mehdi Abaakouk
<pre>
158 78 Mehdi Abaakouk
# export RTE_SDK=/root/sileht/dpdk-1.6.0r1
159 78 Mehdi Abaakouk
# export RTE_TARGET=build
160 78 Mehdi Abaakouk
# cd /root/sileht/
161 78 Mehdi Abaakouk
# cp -r $RTE_SDK/examples/helloworld my_rte_app
162 78 Mehdi Abaakouk
# cd my_rte_app
163 78 Mehdi Abaakouk
# make
164 78 Mehdi Abaakouk
</pre>
165 78 Mehdi Abaakouk
166 78 Mehdi Abaakouk
167 78 Mehdi Abaakouk
h3. Tests
168 78 Mehdi Abaakouk
169 78 Mehdi Abaakouk
<pre>
170 78 Mehdi Abaakouk
</pre>