diff options
| -rw-r--r-- | CodingStyle (renamed from Documentation/CodingStyle) | 0 | ||||
| -rw-r--r-- | Documentation/Downstream | 140 | ||||
| -rw-r--r-- | Documentation/KnownIssues | 97 | ||||
| -rw-r--r-- | Documentation/Mirrors | 17 | ||||
| -rw-r--r-- | Documentation/Performance | 278 | ||||
| -rw-r--r-- | Documentation/RelatedWork | 87 | ||||
| -rw-r--r-- | Documentation/Summary | 59 | ||||
| -rw-r--r-- | Documentation/Workflow | 64 | ||||
| -rw-r--r-- | Sponsors (renamed from Documentation/Sponsors) | 0 | ||||
| -rw-r--r-- | SubmittingPatches (renamed from Documentation/SubmittingPatches) | 51 |
10 files changed, 31 insertions, 762 deletions
diff --git a/Documentation/CodingStyle b/CodingStyle index 31265b4..31265b4 100644 --- a/Documentation/CodingStyle +++ b/CodingStyle diff --git a/Documentation/Downstream b/Documentation/Downstream deleted file mode 100644 index ba22cdd..0000000 --- a/Documentation/Downstream +++ /dev/null @@ -1,140 +0,0 @@ -Maintainer: -/////////// - -netsniff-ng operating system distribution package maintainers are listed here -with the following attributes: 1 - OS distribution top-level site, 2 - OS -distribution netsniff-ng site, M - Maintainers name, W - Maintainers website, -E - Maintainers e-mail, C - Maintainers country. - -We'd hereby like to express a huge thanks to our awesome maintainers! Kudos! -If you are a maintainer for netsniff-ng and not listed here, please contact -us at <workgroup@netsniff-ng.org>. - -Debian - * 1: http://www.debian.org/ - * 2: http://packages.debian.org/search?keywords=netsniff-ng - * M: Kartik Mistry - * W: http://people.debian.org/~kartik/ - * E: kartik@debian.org - * C: India - -Fedora / Fedora Security Lab Spin / Red Hat Enterprise Linux - * 1: http://fedoraproject.org/ - * 2: https://admin.fedoraproject.org/pkgdb/acls/name/netsniff-ng - * M: Jaroslav Škarvada - * W: https://admin.fedoraproject.org/pkgdb/users/packages/jskarvad - * E: jskarvad@redhat.com - * C: Czech Republic - -Ubuntu - * 1: http://www.ubuntu.com/ - * 2: https://launchpad.net/ubuntu/+source/netsniff-ng/ - * (pulled from Debian) - -Arch Linux - * 1: http://archlinux.org/ - * 2: http://aur.archlinux.org/packages.php?K=netsniff-ng - * M: Can Celasun - * W: http://durucancelasun.info/ - * E: dcelasun@gmail.com - * C: Turkey - -Linux Mint - * 1: http://www.linuxmint.com - * 2: http://community.linuxmint.com/software/view/netsniff-ng - * (pulled from Debian) - -Gentoo - * 1: http://www.gentoo.org/ - * 2: http://packages.gentoo.org/package/net-analyzer/netsniff-ng - * M: Michael Weber - * W: http://cia.vc/stats/author/xmw - * E: xmw@gentoo.org - * C: Germany - -Sabayon - * 1: http://www.sabayon.org/ - * 2: http://gpo.zugaina.org/net-misc/netsniff-ng - * M: Epinephrine - * E: epinephrineaddict@gmail.com - -Slackware - * 1: http://www.slackware.com/ - * 2: http://www.slackers.it/repository/netsniff-ng/ - * M: Corrado Franco - * W: http://conraid.net/ - * E: conraid@gmail.com - * C: Italy - -openSUSE / SUSE Linux Enterprise - * 1: http://opensuse.org/ - * 2: http://software.opensuse.org/search?baseproject=ALL&p=1&q=netsniff-ng - * M: Pascal Bleser - * W: http://linux01.gwdg.de/~pbleser/ - * E: pascal.bleser@skynet.be - * C: Belgium - -Mageia - * 1: http://www.mageia.org/ - * 2: https://bugs.mageia.org/show_bug.cgi?id=7268 - * M: Matteo Pasotti - * E: pasotti.matteo@gmail.com - * C: Italy - -Mandriva - * 1: http://www.mandriva.com/ - * 2: http://sophie.zarb.org/srpm/Mandriva,cooker,/netsniff-ng - * M: Dmitry Mikhirev - * E: dmikhirev@mandriva.org - * C: Russia - -Trisquel - * 1: http://trisquel.info/ - * 2: http://packages.trisquel.info/slaine/net/netsniff-ng - * (pulled from Debian) - -GRML - * 1: http://grml.org/ - * 2: http://grml.org/changelogs/README-grml-2010.04/ - * M: Michael Prokop - * E: mika@grml.org - * C: Austria - -Alpine Linux - * 1: http://alpinelinux.org/ - * M: Fabian Affolter - * W: http://affolter-engineering.ch - * E: fabian@affolter-engineering.ch - * C: Switzerland - -Network Security Toolkit - * 1: http://networksecuritytoolkit.org/ - * 2: http://networksecuritytoolkit.org/nst/links.html - * M: Ronald W. Henderson - * W: http://www.networksecuritytoolkit.org/nstpro/help/aboutus.html - * E: rwhalb@nycap.rr.com - * C: USA - -Network Forensic Analysis Tool (NFAT, Xplico) - * 1: http://www.xplico.org/ - * 2: http://www.xplico.org/archives/1184 - * M: Gianluca Costa - * E: g.costa@iserm.com - * C: Italy - -Backtrack - * 1: http://backtrack-linux.org/ - * 2: http://redmine.backtrack-linux.org:8080/issues/572 - * E: slyscorpion@gmail.com - -Scientific Linux by Fermilab / CERN - * 1: http://linux.web.cern.ch/linux/scientific.shtml - * E: linux.support@cern.ch - * C: Switzerland - -Security Onion - * 1: http://code.google.com/p/security-onion/ - * 2: http://code.google.com/p/security-onion/wiki/Beta - * M: Doug Burks - * E: doug.burks@gmail.com - * C: USA diff --git a/Documentation/KnownIssues b/Documentation/KnownIssues deleted file mode 100644 index eb17a3f..0000000 --- a/Documentation/KnownIssues +++ /dev/null @@ -1,97 +0,0 @@ -netsniff-ng's known issues: -/////////////////////////// - -Q: When I perform a traffic capture on the Ethernet interface, the PCAP file is - created and packets are received but without 802.1Q header. If I use - tshark, I get all headers but netsniff-ng removes 802.1Q headers. Is that - normal behavior? -A: Yes and no. The way how VLAN headers are handled in PF_PACKET sockets by the - kernel is somewhat problematic [1]. The problem in the Linux kernel is that - some drivers already handle VLAN, others not. Those who handle it have - different implementations, i.e. hardware acceleration and so on. So in some - cases the VLAN tag is even stripped before entering the protocol stack, in - some cases probably not. Bottom line is that the netdev hackers introduced - a "hack" in PF_PACKET so that a VLAN ID is visible in some helper data - structure that is accessible from the RX_RING. And then it gets really messy - in the user space to artificially put the VLAN header back into the right - place. Not mentioning about the resulting performance implications on that - of /all/ libpcap tools since parts of the packet need to be copied for - reassembly. A user reported the following, just to demonstrate this mess: - Some tests were made with two machines, and it seems that results depends on - the driver ... - - 1) AR8131 - * ethtool -k eth0 gives "rx-vlan-offload: on" - -> wireshark gets the vlan header - -> netsniff-ng doesn't get the vlan header - - * ethtool -K eth0 rxvlan off - -> wireshark gets twice the same vlan header (like QinQ even though - I never sent QinQ) - -> netsniff-ng gets the vlan header - - 2) RTL8111/8168B - * ethtool -k eth0 gives "rx-vlan-offload: on" - -> wireshark gets the vlan header - -> netsniff-ng doesn't get the vlan header - - * ethtool -K eth0 rxvlan off - -> wireshark gets the vlan header - -> netsniff-ng doesn't get the vlan header - - Even if we would agree on doing the same workaround as libpcap, we still - will not be able to see QinQ, for instance, due to the fact that only /one/ - VLAN tag is stored in this kernel helper data structure. We think that - there should be a good consensus on the kernel space side about what gets - transferred to the userland. - - [1] http://lkml.indiana.edu/hypermail/linux/kernel/0710.3/3816.html - - Update (28.11.2012): the Linux kernel and also bpfc has built-in support - for hardware accelerated VLAN filtering, even though tags might not be - visible in the payload itself as reported here. However, the filtering - for VLANs works reliable if your NIC supports it. bpfc example for filtering - for any tags: - - _main: - ld #vlanp - jeq #0, drop - ret #-1 - drop: - ret #0 - - Filtering for a particular VLAN tag: - - _main: - ld #vlant - jneq #10, drop - ret #-1 - drop: - ret #0 - - Where 10 is VLAN ID 10 in this example. Or, more pedantic: - - _main: - ld #vlanp - jeq #0, drop - ld #vlant - jneq #10, drop - ret #-1 - drop: - ret #0 - -Q: When I start trafgen, my kernel crashes! What is happening? -A: We have fixed this ``bug'' in the Linux kernel under commit - 7f5c3e3a80e6654cf48dfba7cf94f88c6b505467 (http://bit.ly/PcH5Nd). Either - update your kernel to the latest version, e.g. clone and build it from - git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git or don't - start multiple trafgen instances at once resp. start trafgen with flag -A - to disable temporary socket memory tuning! Although trafgen's mechanism is - written in a correct manner, some probably Linux internal side-effects - cause the tigger of the BUG macro. Why tuning? In general, if not otherwise - specified, the netsniff-ng suite tries to get a good performance on default. - For instance, this includes things like tuning the system's socket memory, - enabling the BPF JIT compiler, migrating the NIC's interrupt affinity and - so on. If you don't want netsniff-ng to do this, look at the relevant cmd - line options that disable them with ``--help'' and explicitly specify them - on the program start. diff --git a/Documentation/Mirrors b/Documentation/Mirrors deleted file mode 100644 index d61cc47..0000000 --- a/Documentation/Mirrors +++ /dev/null @@ -1,17 +0,0 @@ -Mirrors: -//////// - -Official mirrors for the netsniff-ng website: - - * Germany: http://netsniff-ng.{org,net,com} - -Official mirrors for the netsniff-ng Git repository: - - * Switzerland: git://git.distanz.ch/netsniff-ng.git - * Iceland: git://git.cryptoism.org/pub/git/netsniff-ng.git - * United States: git://github.com/borkmann/netsniff-ng.git - -Distribution specific maintenance/release Git repositories: - - * Debian Linux: git://anonscm.debian.org/collab-maint/netsniff-ng.git - * Fedora/RHEL Linux: git://pkgs.fedoraproject.org/netsniff-ng.git diff --git a/Documentation/Performance b/Documentation/Performance deleted file mode 100644 index e51411a..0000000 --- a/Documentation/Performance +++ /dev/null @@ -1,278 +0,0 @@ -Hitchhiker's guide to high-performance with netsniff-ng: -//////////////////////////////////////////////////////// - -This is a collection of short notes in random order concerning software -and hardware for optimizing throughput (partly copied or derived from sources -that are mentioned at the end of this file): - -<=== Hardware ====> - -.-=> Use a PCI-X or PCIe server NIC -`-------------------------------------------------------------------------- -Only if it says Gigabit Ethernet on the box of your NIC, that does not -necessarily mean that you will also reach it. Especially on small packet -sizes, you won't reach wire-rate with a PCI adapter built for desktop or -consumer machines. Rather, you should buy a server adapter that has faster -interconnects such as PCIe. Also, make your choice of a server adapter, -whether it has a good support in the kernel. Check the Linux drivers -directory for your targeted chipset and look at the netdev list if the adapter -is updated frequently. Also, check the location/slot of the NIC adapter on -the system motherboard: Our experience resulted in significantly different -measurement values by locating the NIC adapter in different PCIe slots. -Since we did not have schematics for the system motherboard, this was a -trial and error effort. Moreover, check the specifications of the NIC -hardware: is the system bus connector I/O capable of Gigabit Ethernet -frame rate throughput? Also check the network topology: is your network -Gigabit switch capable of switching Ethernet frames at the maximum rate -or is a direct connection of two end-nodes the better solution? Is Ethernet -flow control being used? "ethtool -a eth0" can be used to determine this. -For measurement purposes, you might want to turn it off to increase throughput: - * ethtool -A eth0 autoneg off - * ethtool -A eth0 rx off - * ethtool -A eth0 tx off - -.-=> Use better (faster) hardware -`-------------------------------------------------------------------------- -Before doing software-based fine-tuning, check if you can afford better and -especially faster hardware. For instance, get a fast CPU with lots of cores -or a NUMA architecture with multi-core CPUs and a fast interconnect. If you -dump PCAP files to disc with netsniff-ng, then a fast SSD is appropriate. -If you plan to memory map PCAP files with netsniff-ng, then choose an -appropriate amount of RAM and so on and so forth. - -<=== Software (Linux kernel specific) ====> - -.-=> Use NAPI drivers -`-------------------------------------------------------------------------- -The "New API" (NAPI) is a rework of the packet processing code in the -kernel to improve performance for high speed networking. NAPI provides -two major features: - -Interrupt mitigation: High-speed networking can create thousands of -interrupts per second, all of which tell the system something it already -knew: it has lots of packets to process. NAPI allows drivers to run with -(some) interrupts disabled during times of high traffic, with a -corresponding decrease in system load. - -Packet throttling: When the system is overwhelmed and must drop packets, -it's better if those packets are disposed of before much effort goes into -processing them. NAPI-compliant drivers can often cause packets to be -dropped in the network adaptor itself, before the kernel sees them at all. - -Many recent NIC drivers automatically support NAPI, so you don't need to do -anything. Some drivers need you to explicitly specify NAPI in the kernel -config or on the command line when compiling the driver. If you are unsure, -check your driver documentation. - -.-=> Use a tickless kernel -`-------------------------------------------------------------------------- -The tickless kernel feature allows for on-demand timer interrupts. This -means that during idle periods, fewer timer interrupts will fire, which -should lead to power savings, cooler running systems, and fewer useless -context switches. (Kernel option: CONFIG_NO_HZ=y) - -.-=> Reduce timer interrupts -`-------------------------------------------------------------------------- -You can select the rate at which timer interrupts in the kernel will fire. -When a timer interrupt fires on a CPU, the process running on that CPU is -interrupted while the timer interrupt is handled. Reducing the rate at -which the timer fires allows for fewer interruptions of your running -processes. This option is particularly useful for servers with multiple -CPUs where processes are not running interactively. (Kernel options: -CONFIG_HZ_100=y and CONFIG_HZ=100) - -.-=> Use Intel's I/OAT DMA Engine -`-------------------------------------------------------------------------- -This kernel option enables the Intel I/OAT DMA engine that is present in -recent Xeon CPUs. This option increases network throughput as the DMA -engine allows the kernel to offload network data copying from the CPU to -the DMA engine. This frees up the CPU to do more useful work. - -Check to see if it's enabled: - -[foo@bar]% dmesg | grep ioat -ioatdma 0000:00:08.0: setting latency timer to 64 -ioatdma 0000:00:08.0: Intel(R) I/OAT DMA Engine found, 4 channels, [...] -ioatdma 0000:00:08.0: irq 56 for MSI/MSI-X - -There's also a sysfs interface where you can get some statistics about the -DMA engine. Check the directories under /sys/class/dma/. (Kernel options: -CONFIG_DMADEVICES=y and CONFIG_INTEL_IOATDMA=y and CONFIG_DMA_ENGINE=y and -CONFIG_NET_DMA=y and CONFIG_ASYNC_TX_DMA=y) - -.-=> Use Direct Cache Access (DCA) -`-------------------------------------------------------------------------- -Intel's I/OAT also includes a feature called Direct Cache Access (DCA). -DCA allows a driver to warm a CPU cache. A few NICs support DCA, the most -popular (to my knowledge) is the Intel 10GbE driver (ixgbe). Refer to your -NIC driver documentation to see if your NIC supports DCA. To enable DCA, -a switch in the BIOS must be flipped. Some vendors supply machines that -support DCA, but don't expose a switch for DCA. - -You can check if DCA is enabled: - -[foo@bar]% dmesg | grep dca -dca service started, version 1.8 - -If DCA is possible on your system but disabled you'll see: - -ioatdma 0000:00:08.0: DCA is disabled in BIOS - -Which means you'll need to enable it in the BIOS or manually. (Kernel -option: CONFIG_DCA=y) - -.-=> Throttle NIC Interrupts -`-------------------------------------------------------------------------- -Some drivers allow the user to specify the rate at which the NIC will -generate interrupts. The e1000e driver allows you to pass a command line -option InterruptThrottleRate when loading the module with insmod. For -the e1000e there are two dynamic interrupt throttle mechanisms, specified -on the command line as 1 (dynamic) and 3 (dynamic conservative). The -adaptive algorithm traffic into different classes and adjusts the interrupt -rate appropriately. The difference between dynamic and dynamic conservative -is the rate for the 'Lowest Latency' traffic class, dynamic (1) has a much -more aggressive interrupt rate for this traffic class. - -As always, check your driver documentation for more information. - -With modprobe: insmod e1000e.o InterruptThrottleRate=1 - -.-=> Use Process and IRQ affinity -`-------------------------------------------------------------------------- -Linux allows the user to specify which CPUs processes and interrupt -handlers are bound. - -Processes: You can use taskset to specify which CPUs a process can run on -Interrupt Handlers: The interrupt map can be found in /proc/interrupts, and -the affinity for each interrupt can be set in the file smp_affinity in the -directory for each interrupt under /proc/irq/. - -This is useful because you can pin the interrupt handlers for your NICs -to specific CPUs so that when a shared resource is touched (a lock in the -network stack) and loaded to a CPU cache, the next time the handler runs, -it will be put on the same CPU avoiding costly cache invalidations that -can occur if the handler is put on a different CPU. - -However, reports of up to a 24% improvement can be had if processes and -the IRQs for the NICs the processes get data from are pinned to the same -CPUs. Doing this ensures that the data loaded into the CPU cache by the -interrupt handler can be used (without invalidation) by the process; -extremely high cache locality is achieved. - -NOTE: If netsniff-ng or trafgen is bound to a specific, it automatically -migrates the NIC's IRQ affinity to this CPU to achieve a high cache locality. - -.-=> Tune Socket's memory allocation area -`-------------------------------------------------------------------------- -On default, each socket has a backend memory between 130KB and 160KB on -a x86/x86_64 machine with 4GB RAM. Hence, network packets can be received -on the NIC driver layer, but later dropped at the socket queue due to memory -restrictions. "sysctl -a | grep mem" will display your current memory -settings. To increase maximum and default values of read and write memory -areas, use: - * sysctl -w net.core.rmem_max=8388608 - This sets the max OS receive buffer size for all types of connections. - * sysctl -w net.core.wmem_max=8388608 - This sets the max OS send buffer size for all types of connections. - * sysctl -w net.core.rmem_default=65536 - This sets the default OS receive buffer size for all types of connections. - * sysctl -w net.core.wmem_default=65536 - This sets the default OS send buffer size for all types of connections. - -.-=> Enable Linux' BPF Just-in-Time compiler -`-------------------------------------------------------------------------- -If you're using filtering with netsniff-ng (or tcpdump, Wireshark, ...), you -should activate the Berkeley Packet Filter Just-in-Time compiler. The Linux -kernel has a built-in "virtual machine" that interprets BPF opcodes for -filtering packets. Hence, those small filter applications are applied to -each packet. (Read more about this in the Bpfc document.) The Just-in-Time -compiler is able to 'compile' such an filter application to assembler code -that can directly be run on the CPU instead on the virtual machine. If -netsniff-ng or trafgen detects that the BPF JIT is present on the system, it -automatically enables it. (Kernel option: CONFIG_HAVE_BPF_JIT=y and -CONFIG_BPF_JIT=y) - -.-=> Increase the TX queue length -`-------------------------------------------------------------------------- -There are settings available to regulate the size of the queue between the -kernel network subsystems and the driver for network interface card. Just -as with any queue, it is recommended to size it such that losses do no -occur due to local buffer overflows. Therefore careful tuning is required -to ensure that the sizes of the queues are optimal for your network -connection. - -There are two queues to consider, the txqueuelen; which is related to the -transmit queue size, and the netdev_backlog; which determines the recv -queue size. Users can manually set this queue size using the ifconfig -command on the required device: - -ifconfig eth0 txqueuelen 2000 - -The default of 100 is inadequate for long distance, or high throughput pipes. -For example, on a network with a rtt of 120ms and at Gig rates, a -txqueuelen of at least 10000 is recommended. - -.-=> Increase kernel receiver backlog queue -`-------------------------------------------------------------------------- -For the receiver side, we have a similar queue for incoming packets. This -queue will build up in size when an interface receives packets faster than -the kernel can process them. If this queue is too small (default is 300), -we will begin to loose packets at the receiver, rather than on the network. -One can set this value by: - -sysctl -w net.core.netdev_max_backlog=2000 - -.-=> Use a RAM-based filesystem if possible -`-------------------------------------------------------------------------- -If you have a considerable amount of RAM, you can also think of using a -RAM-based file system such as ramfs for dumping pcap files with netsniff-ng. -This can be useful for small until middle-sized pcap sizes or for pcap probes -that are generated with netsniff-ng. - -<=== Software (netsniff-ng / trafgen specific) ====> - -.-=> Bind netsniff-ng / trafgen to a CPU -`-------------------------------------------------------------------------- -Both tools have a command-line option '--bind-cpu' that can be used like -'--bind-cpu 0' in order to pin the process to a specific CPU. This was -already mentioned earlier in this file. However, netsniff-ng and trafgen are -able to do this without an external tool. Next to this CPU pinning, they also -automatically migrate this CPU's NIC IRQ affinity. Hence, as in '--bind-cpu 0' -netsniff-ng will not be migrated to a different CPU and the NIC's IRQ affinity -will also be moved to CPU 0 to increase cache locality. - -.-=> Use netsniff-ng in silent mode -`-------------------------------------------------------------------------- -Don't print information to the konsole while you want to achieve high-speed, -because this highly slows down the application. Hence, use netsniff-ng's -'--silent' option when recording or replaying PCAP files! - -.-=> Use netsniff-ng's scatter/gather or mmap for PCAP files -`-------------------------------------------------------------------------- -The scatter/gather I/O mode which is default in netsniff-ng can be used to -record large PCAP files and is slower than the memory mapped I/O. However, -you don't have the RAM size as your limit for recording. Use netsniff-ng's -memory mapped I/O option for achieving a higher speed for recording a PCAP, -but with the trade-off that the maximum allowed size is limited. - -.-=> Use static packet configurations in trafgen -`-------------------------------------------------------------------------- -Don't use counters or byte randomization in trafgen configuration file, since -it slows down the packet generation process. Static packet bytes are the fastest -to go with. - -.-=> Generate packets with different txhashes in trafgen -`-------------------------------------------------------------------------- -For 10Gbit/s multiqueue NICs, it might be good to generate packets that result -in different txhashes, thus multiple queues are used in the transmission path -(and therefore high likely also multiple CPUs). - -Sources: -~~~~~~~~ - -* http://www.linuxfoundation.org/collaborate/workgroups/networking/napi -* http://datatag.web.cern.ch/datatag/howto/tcp.html -* http://thread.gmane.org/gmane.linux.network/191115 -* http://bit.ly/3XbBrM -* http://wwwx.cs.unc.edu/~sparkst/howto/network_tuning.php -* http://bit.ly/pUFJxU diff --git a/Documentation/RelatedWork b/Documentation/RelatedWork deleted file mode 100644 index ed7dba8..0000000 --- a/Documentation/RelatedWork +++ /dev/null @@ -1,87 +0,0 @@ -Work that relates to netsniff-ng and how we differ from it: -/////////////////////////////////////////////////////////// - -ntop - * W: http://www.ntop.org/ - - The ntop projects offers zero-copy for network packets. Is this approach - significantly different from the already built-in from the Linux kernel? - High likely not. In both cases packets are memory mapped between both address - spaces. The biggest difference is that you get this for free, without - modifying your kernel with netsniff-ng since it uses the kernel's RX_RING - and TX_RING functionality. Unfortunately this is not really mentioned on the - ntop's website. Surely for promotional reasons. For many years the ntop - projects lives on next to the Linux kernel, attempts have been made to - integrate it [1] but discussions got stuck and both sides seem to have no - interest in it anymore, e.g. [2]. Therefore, if you want to use ntop, you are - dependent on ntop's modified drivers that are maintained out of the Linux - kernel's mainline tree. Thus, this will not provide you with the latest - improvements. Also, the Linux kernel's PF_PACKET is maintained by a much bigger - audience, probably better reviewed and optimized. Therefore, also we decided - to go with the Linux kernel's variant. So to keep it short: both approaches - are zero-copy, both have similar performance (if someone tells you something - different, he would lie due to their technical similarities) and we are using - the kernel's built-in variant to reach a broader audience. - - [1] http://lists.openwall.net/netdev/2009/10/14/37 - [2] http://www.spinics.net/lists/netfilter-devel/msg20212.html - -tcpdump - * W: http://www.tcpdump.org/ - - tcpdump is probably the oldest and most famous packet analyzer. It is based on - libpcap and in fact the MIT team that maintains tcpdump also maintains libpcap. - It has been ported to much more architectures and operating systems than - netsniff-ng. However, we don't aim to rebuild or clone tcpdump. We rather focus - on achieving a higher capturing speed by carefully tuning and optimizing our - code. That said doesn't mean that tcpdump people do not take care of it. It - just means that we don't have additional layers of abstractions for being as - portable as possible. This already gives us a smaller code footprint. Also, on - default we perform some system tuning such as remapping the NIC's IRQ affinity - that tcpdump probably would never do due to its generic nature. By generic, we - mean to serve as many different user groups as possible. We rather aim at - serving users for high-speed needs. By that, they have less manual work to do - since it's already performed in the background. Next to this, we also aim at - being a useful networking toolkit rather than only an analyzer. So many other - tools are provided such as trafgen for traffic generation. - -Wireshark/tshark - * W: http://www.wireshark.org/ - - Probably we could tell you the same as in the previous section. I guess it is - safe to say that Wireshark might have the best protocol dissector out there. - However, this is not a free lunch. You pay for it with a performance - degradation, which is quite expensive. It is also based on libpcap (we are not) - and it comes with a graphical user interface, whereas we rather aim at being - used somewhere on a server or middle-box site where you only have access to a - shell, for instance. Again, offline analysis of /large/ pcap files might even - let it hang for a long time. Here netsniff-ng has a better performance also in - capturing pcaps. Again, we furthermore aim at being a toolkit rather than only - an analyzer. - -libpcap - * W: http://www.tcpdump.org/ - - Price question: why don't you rely on libpcap? The answer is quite simple. We - started developing netsniff-ng with its zero-copy capabilities back in 2009 - when libpcap was still doing packet copies between address spaces. Since the - API to the Linux kernel was quite simple, we felt more comfortable using it - directly and bypassing this additional layer of libpcap code. Today we feel - good about this decision, because since the TX_RING functionality was added to - the Linux kernel we have a clean integration of both, RX_RING and TX_RING. - libpcap on the other hand was designed for capturing and not for transmission - of network packets. Therefore, it only uses RX_RING on systems where it's - available but no TX_RING functionality. This would have resulted in a mess in - our code. Additionally, with netsniff-ng, one is able to a more fine grained - tuning of those rings. Why didn't you wrap netsniff-ng around your own library - just like tcpdump and libpcap? Because we are ignorant. If you design a library - than you have to design it well right at the beginning. A library would be a - crappy one if it changes its API ever. Or, if it changes its API, than it has - to keep its old one for the sake of being backwards compatible. Otherwise no - trust in its user or developer base can be achieved. Further, by keeping this - long tail of deprecated functions you will become a code bloat over time. We - wanted to keep this freedom of large-scale refactoring our code and not having - to maintain a stable API to the outer world. This is the whole story behind it. - If you desperately need our internal functionality, you still can feel free to - copy our code as long as your derived code complies with the GPL version 2.0. - So no need to whine. ;-) diff --git a/Documentation/Summary b/Documentation/Summary deleted file mode 100644 index 2863d60..0000000 --- a/Documentation/Summary +++ /dev/null @@ -1,59 +0,0 @@ -Tools: -////// - -The toolkit is split into small, useful utilities that are or are not -necessarily related to each other. Each program for itself fills a gap as -a helper in your daily network debugging, development or audit. - -*netsniff-ng* is a high-performance network analyzer based on packet mmap(2) -mechanisms. It can record pcap files to disc, replay them and also do an -offline and online analysis. Capturing, analysis or replay of raw 802.11 -frames are supported as well. pcap files are also compatible with tcpdump -or Wireshark traces. netsniff-ng processes those pcap traces either in -scatter-gather I/O or by mmap(2) I/O. - -*trafgen* is a high-performance network traffic generator based on packet -mmap(2) mechanisms. It has its own flexible, macro-based low-level packet -configuration language. Injection of raw 802.11 frames are supported as well. -trafgen has a significantly higher speed than mausezahn and comes very close -to pktgen, but runs from user space. pcap traces can also be converted into -a trafgen packet configuration. - -*mausezahn* is a performant high-level packet generator that can run on a -hardware-software appliance and comes with a Cisco-like CLI. It can craft -nearly every possible or impossible packet. Thus, it can be used, for example, -to test network behaviour under strange circumstances (stress test, malformed -packets) or to test hardware-software appliances for several kind of attacks. - -*bpfc* is a Berkeley Packet Filter (BPF) compiler that understands the original -BPF language developed by McCanne and Jacobson. It accepts BPF mnemonics and -converts them into kernel/netsniff-ng readable BPF ``opcodes''. It also -supports undocumented Linux filter extensions. This can especially be useful -for more complicated filters, that high-level filters fail to support. - -*ifpps* is a tool which periodically provides top-like networking and system -statistics from the Linux kernel. It gathers statistical data directly from -procfs files and does not apply any user space traffic monitoring that would -falsify statistics on high packet rates. For wireless, data about link -connectivity is provided as well. - -*flowtop* is a top-like connection tracking tool that can run on an end host -or router. It is able to present TCP, UDP(lite), SCTP, DCCP, ICMP(v6) flows -that have been collected by the kernel's netfilter connection tracking -framework. GeoIP and TCP/SCTP/DCCP state machine information is displayed. -Also, on end hosts flowtop can show PIDs and application names that flows -relate to as well as aggregated packet and byte counter (if available). No -user space traffic monitoring is done, thus all data is gathered by the kernel. - -*curvetun* is a lightweight, high-speed ECDH multiuser VPN for Linux. curvetun -uses the Linux TUN/TAP interface and supports {IPv4,IPv6} over {IPv4,IPv6} with -UDP or TCP as carrier protocols. Packets are encrypted end-to-end by a -symmetric stream cipher (Salsa20) and authenticated by a MAC (Poly1305), where -keys have previously been computed with the ECDH key agreement -protocol (Curve25519). - -*astraceroute* is an autonomous system (AS) trace route utility. Unlike -traceroute or tcptraceroute, it not only display hops, but also their AS -information they belong to as well as GeoIP information and other interesting -things. On default, it uses a TCP probe packet and falls back to ICMP probes -in case no ICMP answer has been received. diff --git a/Documentation/Workflow b/Documentation/Workflow deleted file mode 100644 index 9ff3c45..0000000 --- a/Documentation/Workflow +++ /dev/null @@ -1,64 +0,0 @@ -Repository Workflow: -//////////////////// - -Here are some guidelines for users or developers regarding the work with this -Git repository. - -* Users: - -- steps to verify a release tag can be done with GPG: - git show maint-dborkman-pgp-pub | gpg --import - git show maint-tklauser-pgp-pub | gpg --import - git tag -v <tag-name> - -* Developers: - -- steps to submit a patch: - we follow in general a similar submission process as in the Linux kernel - read https://www.kernel.org/doc/Documentation/SubmittingPatches - read Documentation/SubmittingPatches - if a commit was acknowledged by someone, add - Acked-by: Random J Developer <random@developer.example.org> - if a bug was reported by someone, add - Reported-by: Random J Developer <random@developer.example.org> - if a feature was suggested by someone, add - Suggested-by: Random J Developer <random@developer.example.org> - if a commit was tested by someone, add - Tested-by: Random J Developer <random@developer.example.org> - if a commit was reviewed by someone, add - Reviewed-by: Random J Developer <random@developer.example.org> - if a bug was bisected by someone, add - Bisected-by: Random J Developer <random@developer.example.org> - in case tagging as mentioned above already happened internally - in your company by different persons, you can add those tags - before submission into the patch commit message - in case someone on the mailing list adds his/her *-by tag, it's the - job of the maintainer to keep track of that and to apply this properly - -* Maintainer: - -- steps to add your public GPG key: - gpg --gen-key [...] - gpg --list-keys [take the pubkey id you want to add] - gpg -a --export <pubid> | git hash-object -w --stdin - <git-object-id> - git tag -a maint-<short-name>-pgp-pub <git-object-id> - <log: Public key of <full name>.> - git push --tags - -- steps to make a new release: - set proper versioning in the Makefile itself - make release - git push --tags - upload tarballs to public dir - update website - email generated .MAIL_MSG to the mailing list, include - bcc to the maintainers listed in Documentation/Downstream - -- apply patches from non-maintainers, basic rules to follow: - preferred are patches that have been sent via git send-email - never do an automatic Github merge in case the pull request came from there - in case of Github we need to do cherry-picking - if it's a pull request, make sure it does not contain merges itself - always add your Signed-off-by to the commit message in case of an apply - patches must be rejected in case the developer's Signed-off-by is missing diff --git a/Documentation/Sponsors b/Sponsors index 2d21600..2d21600 100644 --- a/Documentation/Sponsors +++ b/Sponsors diff --git a/Documentation/SubmittingPatches b/SubmittingPatches index fbe72c4..c43080a 100644 --- a/Documentation/SubmittingPatches +++ b/SubmittingPatches @@ -4,18 +4,23 @@ Checklist for Patches: Submitting patches should follow this guideline (derived from the Git project): If you are familiar with upstream Linux kernel development, then you do not -need to read this file, it's basically the same process. +need to read this file, it's basically the same process, send your patches +via git-send-email(1) to <netsniff-ng@googlegroups.com>. Always make sure they +meet high quality if you want them to be taken serious. * Commits: - make sure to comply with the coding guidelines (see CodingStyle) +- make sure, you are working on the latest repository version - make commits of logical units - check for unnecessary whitespace with "git diff --check" before committing - do not check in commented out code or unneeded files +- no binary files are allowed - the first line of the commit message should be a short description (50 characters is the soft limit, see DISCUSSION in git-commit(1)), and should skip the full stop -- the body should provide a meaningful commit message, which: + . it should have the first line like: "subject: short title" +- the body should _always_ provide a meaningful commit message, which: . explains the problem the change tries to solve, iow, what is wrong with the current code without the change. . justifies the way the change solves the problem, iow, why the result with @@ -30,27 +35,20 @@ need to read this file, it's basically the same process. - add a "Signed-off-by: Your Name <you@example.com>" line to the commit message (or just use the option "-s" when committing) to confirm that you agree to the Developer's Certificate of Origin (see also - http://linux.yyz.us/patch-format.html or below); this is mandatory -- make sure syntax of man-pages is free of errors: podchecker <tool>.c - -* For Patches via GitHub: - -- fork the netsniff-ng project on GitHub to your local GitHub account - (https://github.com/gnumaniacs/netsniff-ng) -- make your changes to the latest master branch with respect to the commit - section above -- if you change, add, or remove a command line option or make some other user - interface change, the associated documentation should be updated as well. -- open a pull request on (https://github.com/gnumaniacs/netsniff-ng) and send - a notification to the list (netsniff-ng@googlegroups.com) and CC one of the - maintainers if (and only if) the patch is ready for inclusion. -- if your name is not writable in ASCII, make sure that you send off a message - in the correct encoding. -- add a short description what the patch or patchset is about + http://linux.yyz.us/patch-format.html _or_ further below); this is mandatory + otherwise we cannot accept your patches! +- patches via mail resp. mailing list that have been sent via git-send-email(1) + are preferred over patches via Github. However, we also accept stuff via + Github, in case you do not know how to setup git-send-email(1). * For Patches via Mail: -- use "git format-patch -M" to create the patch +- use git-format-patch(1) to create the patch, just to give you a few examples: + . git format-patch HEAD~1 + `- this will create a git-send-email(1)'able patch of your last commit + . git format-patch --cover-letter HEAD~<num> + `- this will create a git-send-email(1)'able patchset of your last <num> commits + including a cover letter, that should be filled out by yourself; <num> > 1 - do not PGP sign your patch - do not attach your patch, but read in the mail body, unless you cannot teach your mailer to leave the formatting of the patch alone. @@ -64,6 +62,19 @@ need to read this file, it's basically the same process. - send the patch to the list (netsniff-ng@googlegroups.com) and CC one of the maintainers if (and only if) the patch is ready for inclusion. If you use git-send-email(1), please test it first by sending email to yourself. +- if you first want to have comments on your patch before it should be seriously + taken into account, add an RFC into the subject like ``[PATCH RFC] ...'' + +* For Patches via GitHub: + +- fork the netsniff-ng project on GitHub to your local GitHub account +- create a feature branch from the latest up to date master branch + . git checkout -b my-fancy-feature +- only work in this branch, so that you can keep your master always clean/in sync +- open a pull request and send a notification to the list + (netsniff-ng@googlegroups.com) and CC one of the maintainers if (and only if) + the patch is ready for inclusion. +- add a short description what the patch or patchset is about * What does the 'Signed-off-by' mean? |