summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorDaniel Borkmann <dborkman@redhat.com>2013-04-11 16:52:24 +0200
committerDaniel Borkmann <dborkman@redhat.com>2013-04-11 16:52:24 +0200
commitd2d2b5d96121ba427f6b23b64aa38a77d6bf7ace (patch)
tree14f23f33a4169627d65ca3f60dafa898140a3597
parent209b2300fcb22cda9a27d8b122015c2ae235514c (diff)
docs: move some of them to the root directory
Lets move CodingStyle, SubmittingPatches, and Sponsors into the root directory of netsniff-ng and remove the Documentation folder. Some of those files are quite bloated, and most of these things should be in the man-pages anyway. They should be the only big sources of documentation, nothing else. The rest is currently put here: http://pub.netsniff-ng.org/docs/ Signed-off-by: Daniel Borkmann <dborkman@redhat.com>
-rw-r--r--CodingStyle (renamed from Documentation/CodingStyle)0
-rw-r--r--Documentation/Downstream140
-rw-r--r--Documentation/KnownIssues97
-rw-r--r--Documentation/Mirrors17
-rw-r--r--Documentation/Performance278
-rw-r--r--Documentation/RelatedWork87
-rw-r--r--Documentation/Summary59
-rw-r--r--Documentation/Workflow64
-rw-r--r--Sponsors (renamed from Documentation/Sponsors)0
-rw-r--r--SubmittingPatches (renamed from Documentation/SubmittingPatches)51
10 files changed, 31 insertions, 762 deletions
diff --git a/Documentation/CodingStyle b/CodingStyle
index 31265b4..31265b4 100644
--- a/Documentation/CodingStyle
+++ b/CodingStyle
diff --git a/Documentation/Downstream b/Documentation/Downstream
deleted file mode 100644
index ba22cdd..0000000
--- a/Documentation/Downstream
+++ /dev/null
@@ -1,140 +0,0 @@
-Maintainer:
-///////////
-
-netsniff-ng operating system distribution package maintainers are listed here
-with the following attributes: 1 - OS distribution top-level site, 2 - OS
-distribution netsniff-ng site, M - Maintainers name, W - Maintainers website,
-E - Maintainers e-mail, C - Maintainers country.
-
-We'd hereby like to express a huge thanks to our awesome maintainers! Kudos!
-If you are a maintainer for netsniff-ng and not listed here, please contact
-us at <workgroup@netsniff-ng.org>.
-
-Debian
- * 1: http://www.debian.org/
- * 2: http://packages.debian.org/search?keywords=netsniff-ng
- * M: Kartik Mistry
- * W: http://people.debian.org/~kartik/
- * E: kartik@debian.org
- * C: India
-
-Fedora / Fedora Security Lab Spin / Red Hat Enterprise Linux
- * 1: http://fedoraproject.org/
- * 2: https://admin.fedoraproject.org/pkgdb/acls/name/netsniff-ng
- * M: Jaroslav Škarvada
- * W: https://admin.fedoraproject.org/pkgdb/users/packages/jskarvad
- * E: jskarvad@redhat.com
- * C: Czech Republic
-
-Ubuntu
- * 1: http://www.ubuntu.com/
- * 2: https://launchpad.net/ubuntu/+source/netsniff-ng/
- * (pulled from Debian)
-
-Arch Linux
- * 1: http://archlinux.org/
- * 2: http://aur.archlinux.org/packages.php?K=netsniff-ng
- * M: Can Celasun
- * W: http://durucancelasun.info/
- * E: dcelasun@gmail.com
- * C: Turkey
-
-Linux Mint
- * 1: http://www.linuxmint.com
- * 2: http://community.linuxmint.com/software/view/netsniff-ng
- * (pulled from Debian)
-
-Gentoo
- * 1: http://www.gentoo.org/
- * 2: http://packages.gentoo.org/package/net-analyzer/netsniff-ng
- * M: Michael Weber
- * W: http://cia.vc/stats/author/xmw
- * E: xmw@gentoo.org
- * C: Germany
-
-Sabayon
- * 1: http://www.sabayon.org/
- * 2: http://gpo.zugaina.org/net-misc/netsniff-ng
- * M: Epinephrine
- * E: epinephrineaddict@gmail.com
-
-Slackware
- * 1: http://www.slackware.com/
- * 2: http://www.slackers.it/repository/netsniff-ng/
- * M: Corrado Franco
- * W: http://conraid.net/
- * E: conraid@gmail.com
- * C: Italy
-
-openSUSE / SUSE Linux Enterprise
- * 1: http://opensuse.org/
- * 2: http://software.opensuse.org/search?baseproject=ALL&p=1&q=netsniff-ng
- * M: Pascal Bleser
- * W: http://linux01.gwdg.de/~pbleser/
- * E: pascal.bleser@skynet.be
- * C: Belgium
-
-Mageia
- * 1: http://www.mageia.org/
- * 2: https://bugs.mageia.org/show_bug.cgi?id=7268
- * M: Matteo Pasotti
- * E: pasotti.matteo@gmail.com
- * C: Italy
-
-Mandriva
- * 1: http://www.mandriva.com/
- * 2: http://sophie.zarb.org/srpm/Mandriva,cooker,/netsniff-ng
- * M: Dmitry Mikhirev
- * E: dmikhirev@mandriva.org
- * C: Russia
-
-Trisquel
- * 1: http://trisquel.info/
- * 2: http://packages.trisquel.info/slaine/net/netsniff-ng
- * (pulled from Debian)
-
-GRML
- * 1: http://grml.org/
- * 2: http://grml.org/changelogs/README-grml-2010.04/
- * M: Michael Prokop
- * E: mika@grml.org
- * C: Austria
-
-Alpine Linux
- * 1: http://alpinelinux.org/
- * M: Fabian Affolter
- * W: http://affolter-engineering.ch
- * E: fabian@affolter-engineering.ch
- * C: Switzerland
-
-Network Security Toolkit
- * 1: http://networksecuritytoolkit.org/
- * 2: http://networksecuritytoolkit.org/nst/links.html
- * M: Ronald W. Henderson
- * W: http://www.networksecuritytoolkit.org/nstpro/help/aboutus.html
- * E: rwhalb@nycap.rr.com
- * C: USA
-
-Network Forensic Analysis Tool (NFAT, Xplico)
- * 1: http://www.xplico.org/
- * 2: http://www.xplico.org/archives/1184
- * M: Gianluca Costa
- * E: g.costa@iserm.com
- * C: Italy
-
-Backtrack
- * 1: http://backtrack-linux.org/
- * 2: http://redmine.backtrack-linux.org:8080/issues/572
- * E: slyscorpion@gmail.com
-
-Scientific Linux by Fermilab / CERN
- * 1: http://linux.web.cern.ch/linux/scientific.shtml
- * E: linux.support@cern.ch
- * C: Switzerland
-
-Security Onion
- * 1: http://code.google.com/p/security-onion/
- * 2: http://code.google.com/p/security-onion/wiki/Beta
- * M: Doug Burks
- * E: doug.burks@gmail.com
- * C: USA
diff --git a/Documentation/KnownIssues b/Documentation/KnownIssues
deleted file mode 100644
index eb17a3f..0000000
--- a/Documentation/KnownIssues
+++ /dev/null
@@ -1,97 +0,0 @@
-netsniff-ng's known issues:
-///////////////////////////
-
-Q: When I perform a traffic capture on the Ethernet interface, the PCAP file is
- created and packets are received but without 802.1Q header. If I use
- tshark, I get all headers but netsniff-ng removes 802.1Q headers. Is that
- normal behavior?
-A: Yes and no. The way how VLAN headers are handled in PF_PACKET sockets by the
- kernel is somewhat problematic [1]. The problem in the Linux kernel is that
- some drivers already handle VLAN, others not. Those who handle it have
- different implementations, i.e. hardware acceleration and so on. So in some
- cases the VLAN tag is even stripped before entering the protocol stack, in
- some cases probably not. Bottom line is that the netdev hackers introduced
- a "hack" in PF_PACKET so that a VLAN ID is visible in some helper data
- structure that is accessible from the RX_RING. And then it gets really messy
- in the user space to artificially put the VLAN header back into the right
- place. Not mentioning about the resulting performance implications on that
- of /all/ libpcap tools since parts of the packet need to be copied for
- reassembly. A user reported the following, just to demonstrate this mess:
- Some tests were made with two machines, and it seems that results depends on
- the driver ...
-
- 1) AR8131
- * ethtool -k eth0 gives "rx-vlan-offload: on"
- -> wireshark gets the vlan header
- -> netsniff-ng doesn't get the vlan header
-
- * ethtool -K eth0 rxvlan off
- -> wireshark gets twice the same vlan header (like QinQ even though
- I never sent QinQ)
- -> netsniff-ng gets the vlan header
-
- 2) RTL8111/8168B
- * ethtool -k eth0 gives "rx-vlan-offload: on"
- -> wireshark gets the vlan header
- -> netsniff-ng doesn't get the vlan header
-
- * ethtool -K eth0 rxvlan off
- -> wireshark gets the vlan header
- -> netsniff-ng doesn't get the vlan header
-
- Even if we would agree on doing the same workaround as libpcap, we still
- will not be able to see QinQ, for instance, due to the fact that only /one/
- VLAN tag is stored in this kernel helper data structure. We think that
- there should be a good consensus on the kernel space side about what gets
- transferred to the userland.
-
- [1] http://lkml.indiana.edu/hypermail/linux/kernel/0710.3/3816.html
-
- Update (28.11.2012): the Linux kernel and also bpfc has built-in support
- for hardware accelerated VLAN filtering, even though tags might not be
- visible in the payload itself as reported here. However, the filtering
- for VLANs works reliable if your NIC supports it. bpfc example for filtering
- for any tags:
-
- _main:
- ld #vlanp
- jeq #0, drop
- ret #-1
- drop:
- ret #0
-
- Filtering for a particular VLAN tag:
-
- _main:
- ld #vlant
- jneq #10, drop
- ret #-1
- drop:
- ret #0
-
- Where 10 is VLAN ID 10 in this example. Or, more pedantic:
-
- _main:
- ld #vlanp
- jeq #0, drop
- ld #vlant
- jneq #10, drop
- ret #-1
- drop:
- ret #0
-
-Q: When I start trafgen, my kernel crashes! What is happening?
-A: We have fixed this ``bug'' in the Linux kernel under commit
- 7f5c3e3a80e6654cf48dfba7cf94f88c6b505467 (http://bit.ly/PcH5Nd). Either
- update your kernel to the latest version, e.g. clone and build it from
- git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git or don't
- start multiple trafgen instances at once resp. start trafgen with flag -A
- to disable temporary socket memory tuning! Although trafgen's mechanism is
- written in a correct manner, some probably Linux internal side-effects
- cause the tigger of the BUG macro. Why tuning? In general, if not otherwise
- specified, the netsniff-ng suite tries to get a good performance on default.
- For instance, this includes things like tuning the system's socket memory,
- enabling the BPF JIT compiler, migrating the NIC's interrupt affinity and
- so on. If you don't want netsniff-ng to do this, look at the relevant cmd
- line options that disable them with ``--help'' and explicitly specify them
- on the program start.
diff --git a/Documentation/Mirrors b/Documentation/Mirrors
deleted file mode 100644
index d61cc47..0000000
--- a/Documentation/Mirrors
+++ /dev/null
@@ -1,17 +0,0 @@
-Mirrors:
-////////
-
-Official mirrors for the netsniff-ng website:
-
- * Germany: http://netsniff-ng.{org,net,com}
-
-Official mirrors for the netsniff-ng Git repository:
-
- * Switzerland: git://git.distanz.ch/netsniff-ng.git
- * Iceland: git://git.cryptoism.org/pub/git/netsniff-ng.git
- * United States: git://github.com/borkmann/netsniff-ng.git
-
-Distribution specific maintenance/release Git repositories:
-
- * Debian Linux: git://anonscm.debian.org/collab-maint/netsniff-ng.git
- * Fedora/RHEL Linux: git://pkgs.fedoraproject.org/netsniff-ng.git
diff --git a/Documentation/Performance b/Documentation/Performance
deleted file mode 100644
index e51411a..0000000
--- a/Documentation/Performance
+++ /dev/null
@@ -1,278 +0,0 @@
-Hitchhiker's guide to high-performance with netsniff-ng:
-////////////////////////////////////////////////////////
-
-This is a collection of short notes in random order concerning software
-and hardware for optimizing throughput (partly copied or derived from sources
-that are mentioned at the end of this file):
-
-<=== Hardware ====>
-
-.-=> Use a PCI-X or PCIe server NIC
-`--------------------------------------------------------------------------
-Only if it says Gigabit Ethernet on the box of your NIC, that does not
-necessarily mean that you will also reach it. Especially on small packet
-sizes, you won't reach wire-rate with a PCI adapter built for desktop or
-consumer machines. Rather, you should buy a server adapter that has faster
-interconnects such as PCIe. Also, make your choice of a server adapter,
-whether it has a good support in the kernel. Check the Linux drivers
-directory for your targeted chipset and look at the netdev list if the adapter
-is updated frequently. Also, check the location/slot of the NIC adapter on
-the system motherboard: Our experience resulted in significantly different
-measurement values by locating the NIC adapter in different PCIe slots.
-Since we did not have schematics for the system motherboard, this was a
-trial and error effort. Moreover, check the specifications of the NIC
-hardware: is the system bus connector I/O capable of Gigabit Ethernet
-frame rate throughput? Also check the network topology: is your network
-Gigabit switch capable of switching Ethernet frames at the maximum rate
-or is a direct connection of two end-nodes the better solution? Is Ethernet
-flow control being used? "ethtool -a eth0" can be used to determine this.
-For measurement purposes, you might want to turn it off to increase throughput:
- * ethtool -A eth0 autoneg off
- * ethtool -A eth0 rx off
- * ethtool -A eth0 tx off
-
-.-=> Use better (faster) hardware
-`--------------------------------------------------------------------------
-Before doing software-based fine-tuning, check if you can afford better and
-especially faster hardware. For instance, get a fast CPU with lots of cores
-or a NUMA architecture with multi-core CPUs and a fast interconnect. If you
-dump PCAP files to disc with netsniff-ng, then a fast SSD is appropriate.
-If you plan to memory map PCAP files with netsniff-ng, then choose an
-appropriate amount of RAM and so on and so forth.
-
-<=== Software (Linux kernel specific) ====>
-
-.-=> Use NAPI drivers
-`--------------------------------------------------------------------------
-The "New API" (NAPI) is a rework of the packet processing code in the
-kernel to improve performance for high speed networking. NAPI provides
-two major features:
-
-Interrupt mitigation: High-speed networking can create thousands of
-interrupts per second, all of which tell the system something it already
-knew: it has lots of packets to process. NAPI allows drivers to run with
-(some) interrupts disabled during times of high traffic, with a
-corresponding decrease in system load.
-
-Packet throttling: When the system is overwhelmed and must drop packets,
-it's better if those packets are disposed of before much effort goes into
-processing them. NAPI-compliant drivers can often cause packets to be
-dropped in the network adaptor itself, before the kernel sees them at all.
-
-Many recent NIC drivers automatically support NAPI, so you don't need to do
-anything. Some drivers need you to explicitly specify NAPI in the kernel
-config or on the command line when compiling the driver. If you are unsure,
-check your driver documentation.
-
-.-=> Use a tickless kernel
-`--------------------------------------------------------------------------
-The tickless kernel feature allows for on-demand timer interrupts. This
-means that during idle periods, fewer timer interrupts will fire, which
-should lead to power savings, cooler running systems, and fewer useless
-context switches. (Kernel option: CONFIG_NO_HZ=y)
-
-.-=> Reduce timer interrupts
-`--------------------------------------------------------------------------
-You can select the rate at which timer interrupts in the kernel will fire.
-When a timer interrupt fires on a CPU, the process running on that CPU is
-interrupted while the timer interrupt is handled. Reducing the rate at
-which the timer fires allows for fewer interruptions of your running
-processes. This option is particularly useful for servers with multiple
-CPUs where processes are not running interactively. (Kernel options:
-CONFIG_HZ_100=y and CONFIG_HZ=100)
-
-.-=> Use Intel's I/OAT DMA Engine
-`--------------------------------------------------------------------------
-This kernel option enables the Intel I/OAT DMA engine that is present in
-recent Xeon CPUs. This option increases network throughput as the DMA
-engine allows the kernel to offload network data copying from the CPU to
-the DMA engine. This frees up the CPU to do more useful work.
-
-Check to see if it's enabled:
-
-[foo@bar]% dmesg | grep ioat
-ioatdma 0000:00:08.0: setting latency timer to 64
-ioatdma 0000:00:08.0: Intel(R) I/OAT DMA Engine found, 4 channels, [...]
-ioatdma 0000:00:08.0: irq 56 for MSI/MSI-X
-
-There's also a sysfs interface where you can get some statistics about the
-DMA engine. Check the directories under /sys/class/dma/. (Kernel options:
-CONFIG_DMADEVICES=y and CONFIG_INTEL_IOATDMA=y and CONFIG_DMA_ENGINE=y and
-CONFIG_NET_DMA=y and CONFIG_ASYNC_TX_DMA=y)
-
-.-=> Use Direct Cache Access (DCA)
-`--------------------------------------------------------------------------
-Intel's I/OAT also includes a feature called Direct Cache Access (DCA).
-DCA allows a driver to warm a CPU cache. A few NICs support DCA, the most
-popular (to my knowledge) is the Intel 10GbE driver (ixgbe). Refer to your
-NIC driver documentation to see if your NIC supports DCA. To enable DCA,
-a switch in the BIOS must be flipped. Some vendors supply machines that
-support DCA, but don't expose a switch for DCA.
-
-You can check if DCA is enabled:
-
-[foo@bar]% dmesg | grep dca
-dca service started, version 1.8
-
-If DCA is possible on your system but disabled you'll see:
-
-ioatdma 0000:00:08.0: DCA is disabled in BIOS
-
-Which means you'll need to enable it in the BIOS or manually. (Kernel
-option: CONFIG_DCA=y)
-
-.-=> Throttle NIC Interrupts
-`--------------------------------------------------------------------------
-Some drivers allow the user to specify the rate at which the NIC will
-generate interrupts. The e1000e driver allows you to pass a command line
-option InterruptThrottleRate when loading the module with insmod. For
-the e1000e there are two dynamic interrupt throttle mechanisms, specified
-on the command line as 1 (dynamic) and 3 (dynamic conservative). The
-adaptive algorithm traffic into different classes and adjusts the interrupt
-rate appropriately. The difference between dynamic and dynamic conservative
-is the rate for the 'Lowest Latency' traffic class, dynamic (1) has a much
-more aggressive interrupt rate for this traffic class.
-
-As always, check your driver documentation for more information.
-
-With modprobe: insmod e1000e.o InterruptThrottleRate=1
-
-.-=> Use Process and IRQ affinity
-`--------------------------------------------------------------------------
-Linux allows the user to specify which CPUs processes and interrupt
-handlers are bound.
-
-Processes: You can use taskset to specify which CPUs a process can run on
-Interrupt Handlers: The interrupt map can be found in /proc/interrupts, and
-the affinity for each interrupt can be set in the file smp_affinity in the
-directory for each interrupt under /proc/irq/.
-
-This is useful because you can pin the interrupt handlers for your NICs
-to specific CPUs so that when a shared resource is touched (a lock in the
-network stack) and loaded to a CPU cache, the next time the handler runs,
-it will be put on the same CPU avoiding costly cache invalidations that
-can occur if the handler is put on a different CPU.
-
-However, reports of up to a 24% improvement can be had if processes and
-the IRQs for the NICs the processes get data from are pinned to the same
-CPUs. Doing this ensures that the data loaded into the CPU cache by the
-interrupt handler can be used (without invalidation) by the process;
-extremely high cache locality is achieved.
-
-NOTE: If netsniff-ng or trafgen is bound to a specific, it automatically
-migrates the NIC's IRQ affinity to this CPU to achieve a high cache locality.
-
-.-=> Tune Socket's memory allocation area
-`--------------------------------------------------------------------------
-On default, each socket has a backend memory between 130KB and 160KB on
-a x86/x86_64 machine with 4GB RAM. Hence, network packets can be received
-on the NIC driver layer, but later dropped at the socket queue due to memory
-restrictions. "sysctl -a | grep mem" will display your current memory
-settings. To increase maximum and default values of read and write memory
-areas, use:
- * sysctl -w net.core.rmem_max=8388608
- This sets the max OS receive buffer size for all types of connections.
- * sysctl -w net.core.wmem_max=8388608
- This sets the max OS send buffer size for all types of connections.
- * sysctl -w net.core.rmem_default=65536
- This sets the default OS receive buffer size for all types of connections.
- * sysctl -w net.core.wmem_default=65536
- This sets the default OS send buffer size for all types of connections.
-
-.-=> Enable Linux' BPF Just-in-Time compiler
-`--------------------------------------------------------------------------
-If you're using filtering with netsniff-ng (or tcpdump, Wireshark, ...), you
-should activate the Berkeley Packet Filter Just-in-Time compiler. The Linux
-kernel has a built-in "virtual machine" that interprets BPF opcodes for
-filtering packets. Hence, those small filter applications are applied to
-each packet. (Read more about this in the Bpfc document.) The Just-in-Time
-compiler is able to 'compile' such an filter application to assembler code
-that can directly be run on the CPU instead on the virtual machine. If
-netsniff-ng or trafgen detects that the BPF JIT is present on the system, it
-automatically enables it. (Kernel option: CONFIG_HAVE_BPF_JIT=y and
-CONFIG_BPF_JIT=y)
-
-.-=> Increase the TX queue length
-`--------------------------------------------------------------------------
-There are settings available to regulate the size of the queue between the
-kernel network subsystems and the driver for network interface card. Just
-as with any queue, it is recommended to size it such that losses do no
-occur due to local buffer overflows. Therefore careful tuning is required
-to ensure that the sizes of the queues are optimal for your network
-connection.
-
-There are two queues to consider, the txqueuelen; which is related to the
-transmit queue size, and the netdev_backlog; which determines the recv
-queue size. Users can manually set this queue size using the ifconfig
-command on the required device:
-
-ifconfig eth0 txqueuelen 2000
-
-The default of 100 is inadequate for long distance, or high throughput pipes.
-For example, on a network with a rtt of 120ms and at Gig rates, a
-txqueuelen of at least 10000 is recommended.
-
-.-=> Increase kernel receiver backlog queue
-`--------------------------------------------------------------------------
-For the receiver side, we have a similar queue for incoming packets. This
-queue will build up in size when an interface receives packets faster than
-the kernel can process them. If this queue is too small (default is 300),
-we will begin to loose packets at the receiver, rather than on the network.
-One can set this value by:
-
-sysctl -w net.core.netdev_max_backlog=2000
-
-.-=> Use a RAM-based filesystem if possible
-`--------------------------------------------------------------------------
-If you have a considerable amount of RAM, you can also think of using a
-RAM-based file system such as ramfs for dumping pcap files with netsniff-ng.
-This can be useful for small until middle-sized pcap sizes or for pcap probes
-that are generated with netsniff-ng.
-
-<=== Software (netsniff-ng / trafgen specific) ====>
-
-.-=> Bind netsniff-ng / trafgen to a CPU
-`--------------------------------------------------------------------------
-Both tools have a command-line option '--bind-cpu' that can be used like
-'--bind-cpu 0' in order to pin the process to a specific CPU. This was
-already mentioned earlier in this file. However, netsniff-ng and trafgen are
-able to do this without an external tool. Next to this CPU pinning, they also
-automatically migrate this CPU's NIC IRQ affinity. Hence, as in '--bind-cpu 0'
-netsniff-ng will not be migrated to a different CPU and the NIC's IRQ affinity
-will also be moved to CPU 0 to increase cache locality.
-
-.-=> Use netsniff-ng in silent mode
-`--------------------------------------------------------------------------
-Don't print information to the konsole while you want to achieve high-speed,
-because this highly slows down the application. Hence, use netsniff-ng's
-'--silent' option when recording or replaying PCAP files!
-
-.-=> Use netsniff-ng's scatter/gather or mmap for PCAP files
-`--------------------------------------------------------------------------
-The scatter/gather I/O mode which is default in netsniff-ng can be used to
-record large PCAP files and is slower than the memory mapped I/O. However,
-you don't have the RAM size as your limit for recording. Use netsniff-ng's
-memory mapped I/O option for achieving a higher speed for recording a PCAP,
-but with the trade-off that the maximum allowed size is limited.
-
-.-=> Use static packet configurations in trafgen
-`--------------------------------------------------------------------------
-Don't use counters or byte randomization in trafgen configuration file, since
-it slows down the packet generation process. Static packet bytes are the fastest
-to go with.
-
-.-=> Generate packets with different txhashes in trafgen
-`--------------------------------------------------------------------------
-For 10Gbit/s multiqueue NICs, it might be good to generate packets that result
-in different txhashes, thus multiple queues are used in the transmission path
-(and therefore high likely also multiple CPUs).
-
-Sources:
-~~~~~~~~
-
-* http://www.linuxfoundation.org/collaborate/workgroups/networking/napi
-* http://datatag.web.cern.ch/datatag/howto/tcp.html
-* http://thread.gmane.org/gmane.linux.network/191115
-* http://bit.ly/3XbBrM
-* http://wwwx.cs.unc.edu/~sparkst/howto/network_tuning.php
-* http://bit.ly/pUFJxU
diff --git a/Documentation/RelatedWork b/Documentation/RelatedWork
deleted file mode 100644
index ed7dba8..0000000
--- a/Documentation/RelatedWork
+++ /dev/null
@@ -1,87 +0,0 @@
-Work that relates to netsniff-ng and how we differ from it:
-///////////////////////////////////////////////////////////
-
-ntop
- * W: http://www.ntop.org/
-
- The ntop projects offers zero-copy for network packets. Is this approach
- significantly different from the already built-in from the Linux kernel?
- High likely not. In both cases packets are memory mapped between both address
- spaces. The biggest difference is that you get this for free, without
- modifying your kernel with netsniff-ng since it uses the kernel's RX_RING
- and TX_RING functionality. Unfortunately this is not really mentioned on the
- ntop's website. Surely for promotional reasons. For many years the ntop
- projects lives on next to the Linux kernel, attempts have been made to
- integrate it [1] but discussions got stuck and both sides seem to have no
- interest in it anymore, e.g. [2]. Therefore, if you want to use ntop, you are
- dependent on ntop's modified drivers that are maintained out of the Linux
- kernel's mainline tree. Thus, this will not provide you with the latest
- improvements. Also, the Linux kernel's PF_PACKET is maintained by a much bigger
- audience, probably better reviewed and optimized. Therefore, also we decided
- to go with the Linux kernel's variant. So to keep it short: both approaches
- are zero-copy, both have similar performance (if someone tells you something
- different, he would lie due to their technical similarities) and we are using
- the kernel's built-in variant to reach a broader audience.
-
- [1] http://lists.openwall.net/netdev/2009/10/14/37
- [2] http://www.spinics.net/lists/netfilter-devel/msg20212.html
-
-tcpdump
- * W: http://www.tcpdump.org/
-
- tcpdump is probably the oldest and most famous packet analyzer. It is based on
- libpcap and in fact the MIT team that maintains tcpdump also maintains libpcap.
- It has been ported to much more architectures and operating systems than
- netsniff-ng. However, we don't aim to rebuild or clone tcpdump. We rather focus
- on achieving a higher capturing speed by carefully tuning and optimizing our
- code. That said doesn't mean that tcpdump people do not take care of it. It
- just means that we don't have additional layers of abstractions for being as
- portable as possible. This already gives us a smaller code footprint. Also, on
- default we perform some system tuning such as remapping the NIC's IRQ affinity
- that tcpdump probably would never do due to its generic nature. By generic, we
- mean to serve as many different user groups as possible. We rather aim at
- serving users for high-speed needs. By that, they have less manual work to do
- since it's already performed in the background. Next to this, we also aim at
- being a useful networking toolkit rather than only an analyzer. So many other
- tools are provided such as trafgen for traffic generation.
-
-Wireshark/tshark
- * W: http://www.wireshark.org/
-
- Probably we could tell you the same as in the previous section. I guess it is
- safe to say that Wireshark might have the best protocol dissector out there.
- However, this is not a free lunch. You pay for it with a performance
- degradation, which is quite expensive. It is also based on libpcap (we are not)
- and it comes with a graphical user interface, whereas we rather aim at being
- used somewhere on a server or middle-box site where you only have access to a
- shell, for instance. Again, offline analysis of /large/ pcap files might even
- let it hang for a long time. Here netsniff-ng has a better performance also in
- capturing pcaps. Again, we furthermore aim at being a toolkit rather than only
- an analyzer.
-
-libpcap
- * W: http://www.tcpdump.org/
-
- Price question: why don't you rely on libpcap? The answer is quite simple. We
- started developing netsniff-ng with its zero-copy capabilities back in 2009
- when libpcap was still doing packet copies between address spaces. Since the
- API to the Linux kernel was quite simple, we felt more comfortable using it
- directly and bypassing this additional layer of libpcap code. Today we feel
- good about this decision, because since the TX_RING functionality was added to
- the Linux kernel we have a clean integration of both, RX_RING and TX_RING.
- libpcap on the other hand was designed for capturing and not for transmission
- of network packets. Therefore, it only uses RX_RING on systems where it's
- available but no TX_RING functionality. This would have resulted in a mess in
- our code. Additionally, with netsniff-ng, one is able to a more fine grained
- tuning of those rings. Why didn't you wrap netsniff-ng around your own library
- just like tcpdump and libpcap? Because we are ignorant. If you design a library
- than you have to design it well right at the beginning. A library would be a
- crappy one if it changes its API ever. Or, if it changes its API, than it has
- to keep its old one for the sake of being backwards compatible. Otherwise no
- trust in its user or developer base can be achieved. Further, by keeping this
- long tail of deprecated functions you will become a code bloat over time. We
- wanted to keep this freedom of large-scale refactoring our code and not having
- to maintain a stable API to the outer world. This is the whole story behind it.
- If you desperately need our internal functionality, you still can feel free to
- copy our code as long as your derived code complies with the GPL version 2.0.
- So no need to whine. ;-)
diff --git a/Documentation/Summary b/Documentation/Summary
deleted file mode 100644
index 2863d60..0000000
--- a/Documentation/Summary
+++ /dev/null
@@ -1,59 +0,0 @@
-Tools:
-//////
-
-The toolkit is split into small, useful utilities that are or are not
-necessarily related to each other. Each program for itself fills a gap as
-a helper in your daily network debugging, development or audit.
-
-*netsniff-ng* is a high-performance network analyzer based on packet mmap(2)
-mechanisms. It can record pcap files to disc, replay them and also do an
-offline and online analysis. Capturing, analysis or replay of raw 802.11
-frames are supported as well. pcap files are also compatible with tcpdump
-or Wireshark traces. netsniff-ng processes those pcap traces either in
-scatter-gather I/O or by mmap(2) I/O.
-
-*trafgen* is a high-performance network traffic generator based on packet
-mmap(2) mechanisms. It has its own flexible, macro-based low-level packet
-configuration language. Injection of raw 802.11 frames are supported as well.
-trafgen has a significantly higher speed than mausezahn and comes very close
-to pktgen, but runs from user space. pcap traces can also be converted into
-a trafgen packet configuration.
-
-*mausezahn* is a performant high-level packet generator that can run on a
-hardware-software appliance and comes with a Cisco-like CLI. It can craft
-nearly every possible or impossible packet. Thus, it can be used, for example,
-to test network behaviour under strange circumstances (stress test, malformed
-packets) or to test hardware-software appliances for several kind of attacks.
-
-*bpfc* is a Berkeley Packet Filter (BPF) compiler that understands the original
-BPF language developed by McCanne and Jacobson. It accepts BPF mnemonics and
-converts them into kernel/netsniff-ng readable BPF ``opcodes''. It also
-supports undocumented Linux filter extensions. This can especially be useful
-for more complicated filters, that high-level filters fail to support.
-
-*ifpps* is a tool which periodically provides top-like networking and system
-statistics from the Linux kernel. It gathers statistical data directly from
-procfs files and does not apply any user space traffic monitoring that would
-falsify statistics on high packet rates. For wireless, data about link
-connectivity is provided as well.
-
-*flowtop* is a top-like connection tracking tool that can run on an end host
-or router. It is able to present TCP, UDP(lite), SCTP, DCCP, ICMP(v6) flows
-that have been collected by the kernel's netfilter connection tracking
-framework. GeoIP and TCP/SCTP/DCCP state machine information is displayed.
-Also, on end hosts flowtop can show PIDs and application names that flows
-relate to as well as aggregated packet and byte counter (if available). No
-user space traffic monitoring is done, thus all data is gathered by the kernel.
-
-*curvetun* is a lightweight, high-speed ECDH multiuser VPN for Linux. curvetun
-uses the Linux TUN/TAP interface and supports {IPv4,IPv6} over {IPv4,IPv6} with
-UDP or TCP as carrier protocols. Packets are encrypted end-to-end by a
-symmetric stream cipher (Salsa20) and authenticated by a MAC (Poly1305), where
-keys have previously been computed with the ECDH key agreement
-protocol (Curve25519).
-
-*astraceroute* is an autonomous system (AS) trace route utility. Unlike
-traceroute or tcptraceroute, it not only display hops, but also their AS
-information they belong to as well as GeoIP information and other interesting
-things. On default, it uses a TCP probe packet and falls back to ICMP probes
-in case no ICMP answer has been received.
diff --git a/Documentation/Workflow b/Documentation/Workflow
deleted file mode 100644
index 9ff3c45..0000000
--- a/Documentation/Workflow
+++ /dev/null
@@ -1,64 +0,0 @@
-Repository Workflow:
-////////////////////
-
-Here are some guidelines for users or developers regarding the work with this
-Git repository.
-
-* Users:
-
-- steps to verify a release tag can be done with GPG:
- git show maint-dborkman-pgp-pub | gpg --import
- git show maint-tklauser-pgp-pub | gpg --import
- git tag -v <tag-name>
-
-* Developers:
-
-- steps to submit a patch:
- we follow in general a similar submission process as in the Linux kernel
- read https://www.kernel.org/doc/Documentation/SubmittingPatches
- read Documentation/SubmittingPatches
- if a commit was acknowledged by someone, add
- Acked-by: Random J Developer <random@developer.example.org>
- if a bug was reported by someone, add
- Reported-by: Random J Developer <random@developer.example.org>
- if a feature was suggested by someone, add
- Suggested-by: Random J Developer <random@developer.example.org>
- if a commit was tested by someone, add
- Tested-by: Random J Developer <random@developer.example.org>
- if a commit was reviewed by someone, add
- Reviewed-by: Random J Developer <random@developer.example.org>
- if a bug was bisected by someone, add
- Bisected-by: Random J Developer <random@developer.example.org>
- in case tagging as mentioned above already happened internally
- in your company by different persons, you can add those tags
- before submission into the patch commit message
- in case someone on the mailing list adds his/her *-by tag, it's the
- job of the maintainer to keep track of that and to apply this properly
-
-* Maintainer:
-
-- steps to add your public GPG key:
- gpg --gen-key [...]
- gpg --list-keys [take the pubkey id you want to add]
- gpg -a --export <pubid> | git hash-object -w --stdin
- <git-object-id>
- git tag -a maint-<short-name>-pgp-pub <git-object-id>
- <log: Public key of <full name>.>
- git push --tags
-
-- steps to make a new release:
- set proper versioning in the Makefile itself
- make release
- git push --tags
- upload tarballs to public dir
- update website
- email generated .MAIL_MSG to the mailing list, include
- bcc to the maintainers listed in Documentation/Downstream
-
-- apply patches from non-maintainers, basic rules to follow:
- preferred are patches that have been sent via git send-email
- never do an automatic Github merge in case the pull request came from there
- in case of Github we need to do cherry-picking
- if it's a pull request, make sure it does not contain merges itself
- always add your Signed-off-by to the commit message in case of an apply
- patches must be rejected in case the developer's Signed-off-by is missing
diff --git a/Documentation/Sponsors b/Sponsors
index 2d21600..2d21600 100644
--- a/Documentation/Sponsors
+++ b/Sponsors
diff --git a/Documentation/SubmittingPatches b/SubmittingPatches
index fbe72c4..c43080a 100644
--- a/Documentation/SubmittingPatches
+++ b/SubmittingPatches
@@ -4,18 +4,23 @@ Checklist for Patches:
Submitting patches should follow this guideline (derived from the Git project):
If you are familiar with upstream Linux kernel development, then you do not
-need to read this file, it's basically the same process.
+need to read this file, it's basically the same process, send your patches
+via git-send-email(1) to <netsniff-ng@googlegroups.com>. Always make sure they
+meet high quality if you want them to be taken serious.
* Commits:
- make sure to comply with the coding guidelines (see CodingStyle)
+- make sure, you are working on the latest repository version
- make commits of logical units
- check for unnecessary whitespace with "git diff --check" before committing
- do not check in commented out code or unneeded files
+- no binary files are allowed
- the first line of the commit message should be a short description (50
characters is the soft limit, see DISCUSSION in git-commit(1)), and should
skip the full stop
-- the body should provide a meaningful commit message, which:
+ . it should have the first line like: "subject: short title"
+- the body should _always_ provide a meaningful commit message, which:
. explains the problem the change tries to solve, iow, what is wrong with
the current code without the change.
. justifies the way the change solves the problem, iow, why the result with
@@ -30,27 +35,20 @@ need to read this file, it's basically the same process.
- add a "Signed-off-by: Your Name <you@example.com>" line to the commit message
(or just use the option "-s" when committing) to confirm that you agree to
the Developer's Certificate of Origin (see also
- http://linux.yyz.us/patch-format.html or below); this is mandatory
-- make sure syntax of man-pages is free of errors: podchecker <tool>.c
-
-* For Patches via GitHub:
-
-- fork the netsniff-ng project on GitHub to your local GitHub account
- (https://github.com/gnumaniacs/netsniff-ng)
-- make your changes to the latest master branch with respect to the commit
- section above
-- if you change, add, or remove a command line option or make some other user
- interface change, the associated documentation should be updated as well.
-- open a pull request on (https://github.com/gnumaniacs/netsniff-ng) and send
- a notification to the list (netsniff-ng@googlegroups.com) and CC one of the
- maintainers if (and only if) the patch is ready for inclusion.
-- if your name is not writable in ASCII, make sure that you send off a message
- in the correct encoding.
-- add a short description what the patch or patchset is about
+ http://linux.yyz.us/patch-format.html _or_ further below); this is mandatory
+ otherwise we cannot accept your patches!
+- patches via mail resp. mailing list that have been sent via git-send-email(1)
+ are preferred over patches via Github. However, we also accept stuff via
+ Github, in case you do not know how to setup git-send-email(1).
* For Patches via Mail:
-- use "git format-patch -M" to create the patch
+- use git-format-patch(1) to create the patch, just to give you a few examples:
+ . git format-patch HEAD~1
+ `- this will create a git-send-email(1)'able patch of your last commit
+ . git format-patch --cover-letter HEAD~<num>
+ `- this will create a git-send-email(1)'able patchset of your last <num> commits
+ including a cover letter, that should be filled out by yourself; <num> > 1
- do not PGP sign your patch
- do not attach your patch, but read in the mail body, unless you cannot teach
your mailer to leave the formatting of the patch alone.
@@ -64,6 +62,19 @@ need to read this file, it's basically the same process.
- send the patch to the list (netsniff-ng@googlegroups.com) and CC one of the
maintainers if (and only if) the patch is ready for inclusion. If you use
git-send-email(1), please test it first by sending email to yourself.
+- if you first want to have comments on your patch before it should be seriously
+ taken into account, add an RFC into the subject like ``[PATCH RFC] ...''
+
+* For Patches via GitHub:
+
+- fork the netsniff-ng project on GitHub to your local GitHub account
+- create a feature branch from the latest up to date master branch
+ . git checkout -b my-fancy-feature
+- only work in this branch, so that you can keep your master always clean/in sync
+- open a pull request and send a notification to the list
+ (netsniff-ng@googlegroups.com) and CC one of the maintainers if (and only if)
+ the patch is ready for inclusion.
+- add a short description what the patch or patchset is about
* What does the 'Signed-off-by' mean?