Transmit queue timeout when WAN port is plugged in

When my MT3000 plugged into an upstream router using DHCP, the whole network becomes unusable, and the LAN port connection drops and connects multiple times. Any idea why? I’m currently using OpenWrt but it had the same issue with the stock firmware.
dmesg:

[  128.083426] ------------[ cut here ]------------
[  128.088056] NETDEV WATCHDOG: eth0 (mtk_soc_eth): transmit queue 0 timed out
[  128.088087] WARNING: CPU: 0 PID: 0 at 0xffffffc0086c4014
[  128.093389] Modules linked in: nf_flow_table_ipv4 uas nft_tproxy nft_socket nft_masq nft_flow_offload nft_chain_nat nf_tables nf_nat nf_flow_table nf_conntrack mt7915e mt76_connac_lib mt76 mac80211 cfg80211 usb_storage nfnetlink nf_tproxy_ipv6 nf_tproxy_ipv4 nf_socket_ipv6 nf_socket_ipv4 nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c compat pwm_fan ksmbd asn1_decoder oid_registry nls_utf8 md4 authencesn authenc leds_gpio xhci_plat_hcd xhci_pci xhci_mtk_hcd xhci_hcd gpio_button_hotplug usbcore usb_common [last unloaded: crc_ccitt]
[  128.093492] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.15.137 #0
[  128.093499] Hardware name: GL.iNet GL-MT3000 (DT)
[  128.093503] pstate: 20400005 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[  128.093510] pc : 0xffffffc0086c4014
[  128.093514] lr : 0xffffffc0086c4014
[  128.093517] sp : ffffffc008003db0
[  128.093520] x29: ffffffc008003db0 x28: 0000000000000140 x27: 00000000ffffffff
[  128.093530] x26: 0000000000000000 x25: 0000000000000000 x24: ffffff8000d6b4c0
[  128.093539] x23: 0000000000000000 x22: 0000000000000001 x21: ffffffc008a36000
[  128.093549] x20: ffffff8000d6b000 x19: 0000000000000000 x18: ffffffc008a4a2b8
[  128.093558] x17: ffffffc017494000 x16: ffffffc008000000 x15: 00000000000002d9
[  128.093568] x14: 00000000000000f3 x13: ffffffc008003ae8 x12: ffffffc008aa22b8
[  128.093578] x11: fffffffffffe40c0 x10: ffffffc008aa22b8 x9 : 0000000000000000
[  128.093587] x8 : ffffffc008a4a268 x7 : ffffffc008a4a2b8 x6 : 0000000000000001
[  128.093597] x5 : 0000000000000000 x4 : 0000000000000000 x3 : 0000000000000000
[  128.093606] x2 : 0000000000000040 x1 : 0000000000000002 x0 : 000000000000003f
[  128.093615] Call trace:
[  128.093619]  0xffffffc0086c4014
[  128.093623]  0xffffffc0080a5cc0
[  128.093626]  0xffffffc0080a6538
[  128.093630]  0xffffffc0080a6e88
[  128.093633]  0xffffffc00801010c
[  128.093637]  0xffffffc00803b360
[  128.093641]  0xffffffc00803b490
[  128.093644]  0xffffffc008089614
[  128.093647]  0xffffffc008422e84
[  128.093651]  0xffffffc00801584c
[  128.093655]  0xffffffc00801614c
[  128.093658]  0xffffffc008833050
[  128.093661]  0xffffffc008833804
[  128.093665]  0xffffffc008011374
[  128.093668]  0xffffffc008833de4
[  128.093672]  0xffffffc008069490
[  128.093675]  0xffffffc0080696a4
[  128.093679]  0xffffffc008834184
[  128.093683]  0xffffffc0089c0940
[  128.093686]  0xffffffc0089c0f0c
[  128.093690]  0xffffffc0089c034c
[  128.093694] ---[ end trace d400a76f5424077e ]---
[  132.490512] mtk_soc_eth 15100000.ethernet eth0: Link is Down
[  132.490630] mtk_soc_eth 15100000.ethernet eth1: Link is Down
[  132.623648] mtk_soc_eth 15100000.ethernet eth0: PHY [mdio-bus:05] driver [Maxlinear Ethernet GPY211C] (irq=POLL)
[  132.626367] mtk_soc_eth 15100000.ethernet eth0: configuring for phy/sgmii link mode
[  132.641762] mtk_soc_eth 15100000.ethernet eth1: PHY [mdio-bus:00] driver [MediaTek MT7981 PHY] (irq=POLL)
[  132.641776] mtk_soc_eth 15100000.ethernet eth1: configuring for phy/gmii link mode
[  132.649228] br-lan: port 1(eth1) entered disabled state
[  135.763803] mtk_soc_eth 15100000.ethernet eth1: Link is Up - 1Gbps/Full - flow control rx/tx
[  135.763850] br-lan: port 1(eth1) entered blocking state
[  135.763861] br-lan: port 1(eth1) entered forwarding state
[  135.763905] mtk_soc_eth 15100000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off
[  177.210637] mtk_soc_eth 15100000.ethernet eth0: Link is Down
[  177.210755] mtk_soc_eth 15100000.ethernet eth1: Link is Down
[  177.343646] mtk_soc_eth 15100000.ethernet eth0: PHY [mdio-bus:05] driver [Maxlinear Ethernet GPY211C] (irq=POLL)
[  177.346361] mtk_soc_eth 15100000.ethernet eth0: configuring for phy/sgmii link mode
[  177.361763] mtk_soc_eth 15100000.ethernet eth1: PHY [mdio-bus:00] driver [MediaTek MT7981 PHY] (irq=POLL)
[  177.361776] mtk_soc_eth 15100000.ethernet eth1: configuring for phy/gmii link mode
[  177.369233] br-lan: port 1(eth1) entered disabled state
[  180.483800] mtk_soc_eth 15100000.ethernet eth1: Link is Up - 1Gbps/Full - flow control rx/tx
[  180.483847] br-lan: port 1(eth1) entered blocking state
[  180.483858] br-lan: port 1(eth1) entered forwarding state
[  180.483903] mtk_soc_eth 15100000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off
[  225.050772] mtk_soc_eth 15100000.ethernet eth0: Link is Down
[  225.050950] mtk_soc_eth 15100000.ethernet eth1: Link is Down
[  225.183648] mtk_soc_eth 15100000.ethernet eth0: PHY [mdio-bus:05] driver [Maxlinear Ethernet GPY211C] (irq=POLL)
[  225.186499] mtk_soc_eth 15100000.ethernet eth0: configuring for phy/sgmii link mode
[  225.201822] mtk_soc_eth 15100000.ethernet eth1: PHY [mdio-bus:00] driver [MediaTek MT7981 PHY] (irq=POLL)
[  225.201834] mtk_soc_eth 15100000.ethernet eth1: configuring for phy/gmii link mode
[  225.227178] br-lan: port 1(eth1) entered disabled state
[  228.323739] mtk_soc_eth 15100000.ethernet eth1: Link is Up - 1Gbps/Full - flow control rx/tx
[  228.323786] br-lan: port 1(eth1) entered blocking state
[  228.323797] br-lan: port 1(eth1) entered forwarding state
[  231.443804] mtk_soc_eth 15100000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off
[  272.890465] mtk_soc_eth 15100000.ethernet eth0: Link is Down
[  272.890583] mtk_soc_eth 15100000.ethernet eth1: Link is Down
[  273.023647] mtk_soc_eth 15100000.ethernet eth0: PHY [mdio-bus:05] driver [Maxlinear Ethernet GPY211C] (irq=POLL)
[  273.026444] mtk_soc_eth 15100000.ethernet eth0: configuring for phy/sgmii link mode
[  273.041848] mtk_soc_eth 15100000.ethernet eth1: PHY [mdio-bus:00] driver [MediaTek MT7981 PHY] (irq=POLL)
[  273.041862] mtk_soc_eth 15100000.ethernet eth1: configuring for phy/gmii link mode
[  273.049221] br-lan: port 1(eth1) entered disabled state
[  276.163863] mtk_soc_eth 15100000.ethernet eth1: Link is Up - 1Gbps/Full - flow control rx/tx
[  276.163907] mtk_soc_eth 15100000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off
[  276.163912] br-lan: port 1(eth1) entered blocking state
[  276.163922] br-lan: port 1(eth1) entered forwarding state

logread:

Mon Feb  5 19:39:22 2024 kern.info kernel: [   49.891962] mtk_soc_eth 15100000.ethernet eth0: PHY [mdio-bus:05] driver [Maxlinear Ethernet GPY211C] (irq=POLL)
Mon Feb  5 19:39:22 2024 kern.info kernel: [   49.891982] mtk_soc_eth 15100000.ethernet eth0: configuring for phy/2500base-x link mode
Mon Feb  5 19:39:26 2024 daemon.notice netifd: Network device 'eth0' link is up
Mon Feb  5 19:39:26 2024 daemon.notice netifd: Interface 'wan' has link connectivity
Mon Feb  5 19:39:26 2024 daemon.notice netifd: Interface 'wan' is setting up now
Mon Feb  5 19:39:26 2024 kern.info kernel: [   54.083778] mtk_soc_eth 15100000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off
Mon Feb  5 19:39:26 2024 daemon.notice netifd: wan (2726): udhcpc: started, v1.36.1
Mon Feb  5 19:39:26 2024 daemon.notice netifd: wan (2726): udhcpc: broadcasting discover
Mon Feb  5 19:39:29 2024 daemon.notice netifd: wan (2726): udhcpc: broadcasting discover
Mon Feb  5 19:39:32 2024 daemon.notice netifd: wan (2726): udhcpc: broadcasting discover
Mon Feb  5 19:40:10 2024 kern.info kernel: [   98.169933] mtk_soc_eth 15100000.ethernet eth0: Link is Down
Mon Feb  5 19:40:10 2024 kern.info kernel: [   98.303528] mtk_soc_eth 15100000.ethernet eth0: PHY [mdio-bus:05] driver [Maxlinear Ethernet GPY211C] (irq=POLL)
Mon Feb  5 19:40:10 2024 kern.info kernel: [   98.306246] mtk_soc_eth 15100000.ethernet eth0: configuring for phy/sgmii link mode
Mon Feb  5 19:40:10 2024 kern.info kernel: [   98.321745] mtk_soc_eth 15100000.ethernet eth1: PHY [mdio-bus:00] driver [MediaTek MT7981 PHY] (irq=POLL)
Mon Feb  5 19:40:10 2024 kern.info kernel: [   98.321759] mtk_soc_eth 15100000.ethernet eth1: configuring for phy/gmii link mode
Mon Feb  5 19:40:10 2024 daemon.notice netifd: Network device 'eth0' link is down
Mon Feb  5 19:40:10 2024 daemon.notice netifd: Interface 'wan' has link connectivity loss
Mon Feb  5 19:40:10 2024 daemon.notice netifd: wan (2726): udhcpc: received SIGTERM
Mon Feb  5 19:40:10 2024 daemon.notice netifd: wan (2726): udhcpc: entering released state
Mon Feb  5 19:40:10 2024 daemon.notice netifd: wan (2726): Command failed: ubus call network.interface notify_proto { "action": 0, "link-up": false, "keep": false, "interface": "wan" } (Permission denied)
Mon Feb  5 19:40:10 2024 daemon.notice netifd: Interface 'wan' is now down
Mon Feb  5 19:40:13 2024 daemon.notice netifd: Network device 'eth0' link is up
Mon Feb  5 19:40:13 2024 daemon.notice netifd: Interface 'wan' has link connectivity
Mon Feb  5 19:40:13 2024 daemon.notice netifd: Interface 'wan' is setting up now
Mon Feb  5 19:40:13 2024 kern.info kernel: [  101.443757] mtk_soc_eth 15100000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off
Mon Feb  5 19:40:13 2024 daemon.notice netifd: wan (2804): udhcpc: started, v1.36.1
Mon Feb  5 19:40:13 2024 daemon.notice netifd: wan (2804): udhcpc: broadcasting discover
Mon Feb  5 19:40:16 2024 daemon.notice netifd: wan (2804): udhcpc: broadcasting discover
Mon Feb  5 19:40:20 2024 daemon.notice netifd: wan (2804): udhcpc: broadcasting discover
Mon Feb  5 19:40:57 2024 kern.info kernel: [  144.970771] mtk_soc_eth 15100000.ethernet eth0: Link is Down
Mon Feb  5 19:40:57 2024 kern.info kernel: [  145.103526] mtk_soc_eth 15100000.ethernet eth0: PHY [mdio-bus:05] driver [Maxlinear Ethernet GPY211C] (irq=POLL)
Mon Feb  5 19:40:57 2024 kern.info kernel: [  145.106248] mtk_soc_eth 15100000.ethernet eth0: configuring for phy/sgmii link mode
Mon Feb  5 19:40:57 2024 daemon.notice netifd: Network device 'eth0' link is down
Mon Feb  5 19:40:57 2024 daemon.notice netifd: Interface 'wan' has link connectivity loss
Mon Feb  5 19:40:57 2024 kern.info kernel: [  145.121645] mtk_soc_eth 15100000.ethernet eth1: PHY [mdio-bus:00] driver [MediaTek MT7981 PHY] (irq=POLL)
Mon Feb  5 19:40:57 2024 kern.info kernel: [  145.121657] mtk_soc_eth 15100000.ethernet eth1: configuring for phy/gmii link mode
Mon Feb  5 19:40:57 2024 daemon.notice netifd: wan (2804): udhcpc: received SIGTERM
Mon Feb  5 19:40:57 2024 daemon.notice netifd: wan (2804): udhcpc: entering released state
Mon Feb  5 19:40:57 2024 daemon.notice netifd: wan (2804): Command failed: ubus call network.interface notify_proto { "action": 0, "link-up": false, "keep": false, "interface": "wan" } (Permission denied)
Mon Feb  5 19:40:57 2024 daemon.notice netifd: Interface 'wan' is now down

Would be interesting what the upstream router does and what kind of DHCP server (and options) are enabled there.

I do not have access to the upstream router OS, it a fiber modem from Huawei. I have another openwrt router (XDR6088) connected to it with a similar setup. It works without any issues.

The whole network becomes unstable, do you meant that devices connected to the Huawei also not stable?

It just tested again and it just the LAN port of the router lost the connection when WAN port is down, the wifi is fine. I updated the description with more logs from logread.

What is your firmware version?

OpenWrt 23.05.2, I switched the LAN and WAN port and tested again, the 1G LAN port seems to be working with the DHCP server and the 2.5G WAN works with my MacBook dongle. Maybe the 2.5G controller? There seems a lot of similar issues on OpenWrt GitHub. And the immortalwrt-mt798x project mentioned GL.iNet MT3000’s WAN/LAN GMAC does not follows the reference design, might be the cause?

The design should be OK.

The 2.5G wan do have compatibility issues.

Any thing I can do to solve this issue?

Try a different cable first.

If don’t work, may be just reverse the wan and lan port

Tried multiple cables from cat5e to cat7, the problem still persists. I guess switching port is the only workaround for me. BTW, even after switching ports, the 2.5G port still has problems with DHCP.