Recently, I've been experimenting with Wireshark for my bachelor's thesis, monitoring the performance of TCP uploads from my notebook to my web server. A while ago, I had also swapped my router for a nicer model capable of gigabit ethernet and 5 GHz wifi (due to increasing congestion of the 2.4 GHz band in my apartment building), of course also running OpenWrt like the old one.

Soon after the switch, I noticed an oddity in the Wireshark captures: Some of the outgoing TCP segments were reported as having a length of more than 1500 bytes, which is the upper payload limit for Ethernet, and therefore also for most, if not all, residential Internet connections.

Since I didn't really expect the path between my notebook and my server to be capable of a MTU higher than 1500, the obvious explanation for this to work would be IP fragmentation occuring in my router, which would be very unfortunate to say the least.

To figure out if that was really the case, I started tcpdump on the server, with the following interesting result:

No sign of IP fragmentation whatsoever; the packets were arriving as if sent with an MTU of 1500! This also matches both the MTU configured on my notebook and explains the pattern of ACKs received from the server – usually, every other segment should be ACKed by the recipient, but in this case, I was receiving much more than that.

After a bit of googling, I found the explanation: TCP segementation offload (TSO). This is (theoretically) a nice feature of some NICs that allow the operating system to delegate TCP segmentation and TCP and IP header generation to the network interface, relieving the CPU of those duties and possibly also increasing performance quite a bit. However, if there are bugs in the NIC firmware, this could lead to very obscure and hard to debug transmission errors, and it also makes debugging other network behavior more difficult, as I had experienced myself with my measurements.

There is an easy way to disable TSO on Linux:

sudo ethtool -K eth0 tso off
sudo ethtool -K eth0 gso off

GSO is a very similar technology, which can be used to offload some of the higher-layer networking tasks from the kernel to the network interface for protocols other than TCP. It also has to be disabled, because according to my tests, it happily takes over TSO's duties once disabled, also causing strage results in Wireshark.

Apart from the confusion in my packet traces, I have yet to experience any side effects of TSO or GSO with my NIC (likely because of the driver which is commendably developed by the NIC vendor itself), and plan to leave them enabled while I'm not working with Wireshark.