Woke up on July 21 to a completely dead wired LAN on my Flint 3 (4.9.0 release), while WiFi on the very same br-lan kept working just fine. No wired client (PC, Xbox) could get a DHCP lease anymore: tcpdump on the client showed DHCP discovers going out, dnsmasq on the router never saw a single one. Meanwhile the switch chip happily reported normal link events (rtl8372-mdio: lanX link up, speed 2500). Swapping cables and ports did nothing.
SSH'd in and found a state that's physically impossible: the parent interface eth1 (switch chip → CPU) had carrier, but its VLAN child eth1.1 — the only LAN port in br-lan — didn't:
eth1 UP <BROADCAST,MULTICAST,UP,LOWER_UP>
eth1.1 LOWERLAYERDOWN <NO-CARRIER,BROADCAST,MULTICAST,UP>
bridge link show had eth1.1 on "state disabled" while all the WiFi interfaces were on "forwarding". So packets arrived at the switch and simply never made it into the router's network stack. The fix turned out to be embarrassingly simple — no reboot needed:
ip link set eth1.1 down; sleep 2; ip link set eth1.1 up
Instantly back to "state forwarding", LAN alive again.
Now for the fun part: the syslog actually caught the exact moment eth1.1 got stuck, and the context matters. The router had been grinding through a VPN failover loop for hours at that point. Both my WireGuard tunnels (NordVPN) had died because the keys got invalidated server-side (NordVPN decided my token was no longer valid — separate story), and the failover logic kept restarting them every ~2 minutes, each time with a full firewall reload. In the log window from 06:47 to 09:54 that adds up to 191 REKEY-GIVEUP events and 193 firewall reloads. A typical cycle looks like this:
06:47:13 wireguard-debug: ifname=wgclient1 ACTION=REKEY-GIVEUP
06:47:21 netifd: Interface 'wgclient1' is now down
06:47:21 netifd: Interface 'wgclient1' is setting up now
06:47:21 firewall: Reloading firewall due to ifdown of wgclient1 ()
06:47:23 vpn-failover-trigger: action=schedule iface=wgclient1 source=hotplug reason=ifdown tunnel_id=8106
06:47:32 tunnel_id_8106: (tunnel-switch.sh:558) interface wgclient1 still down after 8s, proceeding with failover
06:47:32 tunnel_id_8106: (tunnel-switch.sh:901) only one candidate profile found, skip tunnel switch
At 08:09:40 — right in the middle of one of these cycles, while tunnel-switch.sh is holding its lock and three seconds after yet another firewall reload — the physical link on lan1 briefly drops and comes back one second later:
08:09:37 netifd: Interface 'wgclient2' is now down
08:09:37 netifd: Interface 'wgclient2' is setting up now
08:09:37 firewall: Reloading firewall due to ifdown of wgclient2 ()
08:09:40 tunnel_id_1090: (tunnel-switch.sh:52) acquired lock tunnel_id=1090
08:09:40 kernel: rtl8372-mdio 90000.mdio:1d: lan1 link down
08:09:40 kernel: br-lan: port 1(eth1.1) entered disabled state
08:09:40 netifd: VLAN 'eth1.1' link is down
08:09:41 kernel: rtl8372-mdio 90000.mdio:1d: lan1 link up, speed 2500
The carrier-down still gets propagated properly (bridge port disabled, VLAN link down) — but the carrier-up one second later? Nothing. In the entire rest of the log (6+ hours) there is not one more eth1.1 message. No "link is up", no "entered forwarding state". The up event just got swallowed while netifd was busy tearing down and re-setting up the wgclient interface, and eth1.1 stayed stuck in that inconsistent state until I kicked it manually. Also worth noting: a link flap shortly before (down 08:09:07, up 08:09:11) produced no bridge/VLAN messages at all — so carrier propagation was already getting flaky under the sustained load.
The way I see it, there are two problems here. First, the race itself: a carrier-up event for a VLAN interface must not get lost, no matter how busy netifd is — and if it does get lost anyway, the state "parent has carrier, VLAN child hasn't for minutes" should be detected and self-healed. Second, the failover logic is the engine of the whole cascade: a tunnel that dies with REKEY-GIVEUP and only has a single candidate profile ("only one candidate profile found, skip tunnel switch") still gets restarted every two minutes, forever, with zero backoff. Let that run overnight and you've got hundreds of interface and firewall reload cycles — exactly the kind of sustained churn that makes a race like this likely in the first place.
Repro idea for you guys: configure a WireGuard peer with a deliberately wrong server public key (produces exactly the same REKEY-GIVEUP loop), let it run, and flap a LAN port's link a few times while it's going.
For completeness: while rebuilding my VPN config after this incident I stumbled over an unrelated config bug, reported separately here: https://forum.gl-inet.com/t/4-9-0-gl-be9300-adding-a-server-to-a-wireguard-provider-group-destroys-the-existing-tunnel-configuration-wipes-peer-credentials-and-disables-policy-routing/70063. That one was not the cause of the loop (the REKEY-GIVEUP signature shows the interfaces were fully configured), but it lives in the same neighborhood.
Happy to provide the full logfile (system.log/kernel.log, 06:47–14:58) on request.
