Slowdown sft1200 slowing to half speed

during ‘half speed’ behavior

           CPU0       CPU1       CPU2       CPU3
  8:    3202502    3247926    4211584    3185209  MIPS GIC Local   1  timer
  9:       8457       8455       8455       8455  MIPS GIC Local   2  timer
 10:    7118524          0          0          0  MIPS GIC   7  IPI call
 11:          0    3740695          0          0  MIPS GIC   8  IPI call
 12:          0          0   27721814          0  MIPS GIC   9  IPI call
 13:          0          0          0    3719003  MIPS GIC  10  IPI call
 14:    2630367          0          0          0  MIPS GIC  11  IPI resched
 15:          0    2839298          0          0  MIPS GIC  12  IPI resched
 16:          0          0    2587394          0  MIPS GIC  13  IPI resched
 17:          0          0          0    2803205  MIPS GIC  14  IPI resched
 18:   25970178          0          0          0  MIPS GIC  39  10800000.gmac
 19:          0          0          0          0  MIPS GIC  41  10800000.gmac
 20:          1          0          0          0  MIPS GIC 135  17000000.usb, dwc2_hsotg:usb1
 21:      99332          0          0          0  MIPS GIC 224  18100000.i2c
 22:    1088235          0          0          0  MIPS GIC 232  sfax8-spi
 23:          0         11          0          0  MIPS GIC 233  sfax8serial
 29:          0          0          0          0  MIPS GIC 194  11c00000.wifi-rf
 31:          0          0       5916          0  MIPS GIC  96  umac-lb
 33:          0          0          0       5916  MIPS GIC 192  umac-hb
 34:          1          0          0          0  MIPS GIC 253  pinctrl:bank0
 36:          1          0          0          0  pinctrl-sfax8   1  BTN_0
 62:          1          0          0          0  pinctrl-sfax8  27  Key Restart
 86:          1          0          0          0  MIPS GIC 254  pinctrl:bank1
 87:          0          0          0          0  MIPS GIC 255  pinctrl:bank2
 88:          0          0          0          0  MIPS GIC 256  pinctrl:bank3
ERR:          0
1 Like

Nothing abnormal about the interrupt.
My colleague told me it may be related to hardware acceleration.
Please check if there’s a firewall rule like this:

root@GL-SFT1200:/tmp# iptables-save |grep -i offload
-A FORWARD -m comment --comment "!fw3: Traffic offloading" -m conntrack --ctstate NEW,RELATED,ESTABLISHED -j FLOWOFFLOAD --hw

Yes, looks like the rule is present, what’s the best way to adjust?

uci show | grep ‘offloading’
firewall.@defaults[0].flow_offloading=‘1’
firewall.@defaults[0].flow_offloading_hw=‘1’

this had no impact

uci set firewall.@defaults[0].flow_offloading_hw='0'
uci commit
/etc/init.d/firewall restart

uci show | grep ‘offloading’
firewall.@defaults[0].flow_offloading=‘1’
firewall.@defaults[0].flow_offloading_hw=‘0’

but this was even slower than current (maybe that’s expected)

uci set firewall.@defaults[0].flow_offloading='0'
uci commit
/etc/init.d/firewall restart

uci show | grep ‘offloading’
firewall.@defaults[0].flow_offloading=‘0’
firewall.@defaults[0].flow_offloading_hw=‘0’

not sure if any of the above are the correct steps to mitigate, but wanted to try something, not sure if needs reboot between changes (would take longer to test)

What do you have for Active Connections in LuCI → Status → Overview? I increased the max limit considerably for the Slate AX’s 393MB. OWRT 23.05.x now calculates a respectable value automatically… but we’re not yet on mainline.

root@slateax:~# cat /etc/sysctl.conf
# Defaults are configured in /etc/sysctl.d/* and can be customized in this file
net.netfilter.nf_conntrack_max=32768

/etc/init.d/firewall {stop, start &/or restart} should do it.

See service.

Added …

#Defaults are configured in /etc/sysctl.d/* and can be customized in this file
net.netfilter.nf_conntrack_max=32768

restarting firewall has no impact, and does not return speed when slow

this works to regain speed

/etc/init.d/network restart

and this works…

ifdown wan lan && ifup lan wan

and specifically only → down wan lan → up lan wan → in that order alone

Active Connections at highest value
1250 / 16384 (7%)

Usually less, at about ~500 connections (3%) … would extending nf_conntrack_max value have any impact?

It certainly won’t hurt. It’s highly recommended if you do any torrenting.

Thanks for the suggestion I was hoping due some type of root cause being known? However change the hf_conntrack_max value did not help

It seems like something it wrong with flow offloading, the speed should be consistent and the system is not being stressed

Post the output of ethtool eth0

Use three backticks (```) before & after as new line the output block for readability.

If you have offloading enabled, I’d disable it for the moment then reboot just to clear any caching/routes.

So I’m just going to drop this note before I forget w/ other things going in life.

You can change interface parameters using ethtool. In this particular case I’d disable auto negotiation & force the interface to a strict 1000mb/s. The trouble is I’ve never had need to use this tool as I’ve never had a interface fail like what you describe so I’m not 100% sure of the specific command string needed to disable auto-neg and force 1Gb full-duplex. It could be as simple as ethtool --change eth0 speed 1000 duplex full autoneg off per the die.net manual page.

Once you find it, added it to /etc/rc.local to set it on boot. IDK what the root cause may be on your device/network but this should be a viable workaround. Spot checking w/ OpenSpeedTest might be prudent.

https://linux.die.net/man/8/ethtool

here are the results,
same results during fast (or normal performance), as during slow performance …

# ethtool eth0
Settings for eth0:        
Supported ports: [ TP MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Supported pause frame use: No
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Half 1000baseT/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Link partner advertised link modes:  10baseT/Half 10baseT/Full
                                             100baseT/Half 100baseT/Full
                                             1000baseT/Full
        Link partner advertised pause frame use: Symmetric Receive-only
        Link partner advertised auto-negotiation: Yes
        Link partner advertised FEC modes: Not reported
        Speed: 1000Mb/s
        Duplex: Full
        Port: MII
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: on
        Link detected: yes

For comparision’s sake here’s the output on my Slate AX running f/w 4.4.6-release1. IDK how much of a difference there is in our respective devices:

root@slateax:~# ethtool eth0
Settings for eth0:
        Supported ports: [ TP AUI BNC MII FIBRE ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Supported pause frame use: Symmetric Receive-only
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Link partner advertised link modes:  10baseT/Half 10baseT/Full
                                             100baseT/Half 100baseT/Full
                                             1000baseT/Full
        Link partner advertised pause frame use: Symmetric Receive-only
        Link partner advertised auto-negotiation: Yes
        Link partner advertised FEC modes: Not reported
        Speed: 1000Mb/s
        Duplex: Full
        Port: MII
        PHYAD: 0
        Transceiver: external
        Auto-negotiation: on
        Link detected: yes

I would still try to disable your Opal’s link auto-negotiation + force 1Gb & see how that goes. A quick reboot/power cycle is all it would take to revert it.

Tried this, but did not help

@admon What f/w are you running on your Opal? This isn’t expected behaviour, is it?

If the Cat 5 is good & within length limits I’m tempted to start thinking this is some sort of ‘power saving’ BS if not a bug in the Siflower SDK. That’s speculation of course.

Newest beta and no, this isn’t expected nor the case on my device.

We should try to understand if the WAN or switching capacity is the problem. Could you enable Wi-Fi and doing some Wi-Fi ↔ LAN speedtest when the speed is reduced?

Would really love to know what the cause of this is, any further thoughts?

Cheers

I have an “ifdown wan lan && ifup lan wan” cronjob scheduled, which keeps the speed up but takes the network offline for 10 seconds. It’s set for once an hour as a very temporary solution, since this actually disrupts the network and drops connections.

Looking for others out there, I found this Reddit post which sounds very familiar. So maybe others out there are indeed experiencing this exact issue.

https://www.reddit.com/r/GlInet/comments/15s5sdj/not_getting_gigabit_speeds_on_the_glinet_opal/

My firmware is the same as the reddit post, I had upgraded to latest available prior to posting (prior to my first post in this thread). I’m also running firmware version 3.216, OpenWrt reports version 18.06

Hi Joet,

Have you tried the latest firmware, 4.3.7? After seeing your thread and checking firmware I realised I was still running 3.x and wondered if possibly it had been addressed. Weirdly the firmware doesn’t show up on the router as an upgrade and when you manually install it, the verification process before install says it’s older and isn’t recognised. I have many of these routers so decided to give it a shot anyway.

It installs absolutely fine and my speeds have been completely fine since. I was actually running a starlink through it and could only get about 80mbps, while getting 300mbps+ through the original starlink router. After putting the firmware on it’s consistant with the starlink router and continual. I was seeing the same issue as you, immediately I could get fast speed but it would drop off quickly after each boot.

link to firmware https://fw.gl-inet.com/firmware/sft1200/release4/openwrt-sft1200-4.3.7-1222-1703236005.tar

Anyway, see if that works for you

Might be worth a shot yeah. Is that still in beta, maybe why it does not show up?

seems to validate (going to wait to test), but also has the warning message “The uploaded firmware is older than the current firmware or is a 3rd-party firmware. It make break the router due to hardware incompatibility. You are also suggested to NOT keep settings.”

Odd, seems both newer and from GLiNet, but maybe not released. I’d also like to keep settings, it’s not a complicated setup.

Yeah I’m not sure. On the firmware page it shows up under stable and doesn’t show up when you select beta. Maybe it’s just been uploaded wrong or something and should be beta?