[GL-MT6000 / Flint 2] GRO on WAN silently corrupts forwarded TCP payloads above ~25 MB/s (fw 4.9.1)

Summary

On firmware 4.9.1, the Flint 2 silently corrupts forwarded TCP traffic once
throughput exceeds roughly 25 MB/s (~200 Mbps). The corrupted bytes carry a valid
TCP checksum
, so nothing detects the damage — no retransmits, no checksum errors,
no interface error counters. Only protocols that authenticate their payload (TLS,
SSH) notice, and they fail with integrity errors.

Traffic terminating on the router itself is unaffected — the router downloads the
same file at 217 MB/s with a perfect checksum. Only forwarded traffic is damaged.

Disabling GRO on the WAN interface fixes it completely:

ethtool -K eth1 gro off

This is data corruption, not just a throughput problem. Plain HTTP, rsync, SMB and
NFS have no payload integrity check and will accept the corrupted bytes silently.

Hardware / firmware

Model GL.iNet GL-MT6000 (Flint 2)
Firmware 4.9.1
Base OpenWrt 21.02-SNAPSHOT r15812+1092-46b6ee7ffc, target mediatek/mt7986
Kernel 5.4.238
Driver mtk_soc_eth
WAN eth1, link 2500 Mbps
WAN offloads (default) gro on, tso on, gso on, rx/tx checksumming on

Reproducer

Any large HTTPS download to a LAN client (not the router) that sustains more than
~25 MB/s. Expected size 1682401421 bytes.

URL=https://github.com/ollama/ollama/releases/download/v0.5.7/ollama-linux-amd64.tgz

# From a LAN client — FAILS at a random offset
curl -sL --max-time 600 -o /dev/null -w '%{size_download} %{errormsg}\n' "$URL"
#   1412759534 OpenSSL SSL_read: error:0A000119:SSL routines::decryption failed or bad record mac

# Same client, rate-limited below the threshold — SUCCEEDS
curl -sL --limit-rate 25M --max-time 600 -o /dev/null -w '%{size_download} %{errormsg}\n' "$URL"
#   1682401421

Docker/container image pulls fail the same way:

failed to copy: local error: tls: bad record MAC

Failure offsets are random: 32 MB, 129 MB, 140 MB, 192 MB, 258 MB, 371 MB, 578 MB,
698 MB, 1.01 GB, 1.33 GB, 1.41 GB across runs.

Evidence

All tests used the same 1682401421-byte file. Two independent LAN hosts were used
(one bare-metal Ubuntu 24.04, one Proxmox VE 9 host) — both behave identically.

Path NAT/forwarded Rate Result
Router itself (curl on the router) no — terminated locally 163 / 217 MB/s :white_check_mark: complete, 2/2
LAN host → internet yes 25 MB/s :white_check_mark: complete
LAN host → internet yes 50 MB/s :cross_mark: died at 698 MB
LAN host → internet yes 89–108 MB/s :cross_mark: died, 6/6 runs
LAN host → LAN host (6 GB, nc) no 100 MB/s :white_check_mark: sha256 match
LAN host → internet, gro off on eth1 yes 104 / 108 MB/s :white_check_mark: complete, 2/2
LAN host → internet, gro off + HNAT on yes 31–40 MB/s :white_check_mark: complete, 6/6
4 parallel streams, gro off + HNAT on yes 70 MB/s aggregate (6.7 GB) :white_check_mark: complete, 4/4

The threshold sits between 25 and 50 MB/s. A 791 MB HTTPS download that happened to
run at 15.6 MB/s completed cleanly and repeatedly — destination is irrelevant, rate
is the only variable
.

The corruption is invisible to TCP

TCP counters on the receiving LAN host, measured across a transfer that corrupted:

DELTA:  segs_in=9613   retrans=0   csum_err=0

Zero retransmissions and zero checksum errors — the checksum is valid for the
corrupted data
. Router WAN interface counters were also flat across a failing
transfer (rx_errors 199 before and after, rx_dropped 5370 before and after,
rx_crc_errors 0).

Suggested mechanism

Forwarded packets are coalesced by GRO on WAN ingress, routed/NAT'd, then
re-segmented by GSO/TSO on LAN egress — and checksums are recomputed at that point.
Locally-terminated traffic is consumed by a socket and never re-segmented, which
matches the observation that the router's own downloads are perfect at 217 MB/s while
forwarded traffic corrupts at half that rate.

What was ruled out

Hypothesis Ruled out by
ISP / WAN link / modem Router downloaded 1.68 GB at 217 MB/s, checksum clean
Router SoC capacity Load average 0.15; MT6000 is rated well beyond 1 Gbps NAT
LAN switching path 6 GB host→host at 100 MB/s, sha256 match
Client NIC (e1000e) Disabling rx-checksumming on the client changed nothing
Client RAM / CPU / crypto 2 GB AES encrypt→decrypt round-trip byte-perfect on 3 machines at 227–409 MB/s
Client storage Failing transfers wrote to /dev/null / piped to sha256sum
MTU / fragmentation DF-ping clean at 1472 payload (MTU 1500) to all destinations
TLS itself 791 MB HTTPS download clean at 15.6 MB/s
A specific CDN or file Reproduced with unrelated hosts and file types
MediaTek HNAT Disabling it via /sys/kernel/debug/hnat/hook_toggle did NOT fix it — see matrix below

HNAT is not the cause

Disabling HNAT (echo 0 > /sys/kernel/debug/hnat/hook_toggle) did not fix the
corruption. Disabling GRO did. Tested matrix:

HNAT GRO (eth1) Result
on on :cross_mark: corrupt — many runs, 50–108 MB/s
off on :cross_mark: corrupt — 129 MB, 132 MB
off off :white_check_mark: clean — 2/2 full file at 104 and 108 MB/s
on off :white_check_mark: clean — 6/6 full file at 31–40 MB/s, plus 4 parallel streams totalling 6.7 GB at 70 MB/s aggregate

The controlled comparison is rows 2 and 3: identical conditions, same ~105 MB/s
single-stream rate, GRO the only variable — corrupt with it on, clean with it off.

With HNAT re-enabled and GRO off, throughput from this CDN happened to sit at
31–40 MB/s per stream, so a single stream above 50 MB/s was not reproduced in that
configuration; the 4-stream parallel test was used to drive the forwarding path to
70 MB/s aggregate instead, which completed cleanly. Noting that limitation for
completeness — GRO is clearly the variable that changes the outcome, and hardware
acceleration can stay enabled.

Throughout, hnat_entry showed the WAN download flows in state=UNBIND; the only
bound entries were unrelated LAN-to-LAN RTSP flows. The router was 97% idle
(load 0.20) while forwarding with GRO disabled, so gro off does not appear to cost
meaningful CPU on this hardware.

Hi,

Thank you for the detailed report.

Could you please confirm whether you have already contacted us by email? We recently received a report of a similar issue. If that was from you, we will continue following up through the existing email thread.

We have also reproduced the issue locally and will ask our R&D team to investigate it further.

No I did not.

Hi,

Thank you for the confirmation.

As mentioned previously, we are already investigating this issue and looking into whether it can be fixed in a future firmware release.

If there are any relevant updates or progress, we will try to share them here.

While we wait for the issue to be fixed, is it safe to disable GRO with:
ethtool -K eth1 gro off

What could be the potential downsides?

Yes, it is safe to disable GRO on eth1 as a temporary workaround:

ethtool -K eth1 gro off

The possible downside is slightly higher CPU usage and reduced peak forwarding performance under very heavy traffic, although the impact should be minor on the Flint 2. Normal routing functions should not otherwise be affected.

This change is not persistent and will be reset after a reboot. You can also revert it manually with:

ethtool -K eth1 gro on

We are continuing to investigate a permanent fix.

3 Likes