GL-MT1300 stops working after a couple of days connected

robotluo · 2021-08-26T08:16:31.493Z

Hi ,
I have done a lot of tests on the stability of MT1300 in the last few days. In the process, I found a problem with the Ethernet driver of MT1300. I have fixed it, but I can’t confirm whether the stability reported by everyone is due to the Ethernet driver.
So far, I haven’t had a problem with device rebooting, except that Ethernet negotiation rates are too low to be usable.
I have no way to repeat the stability problem that everyone reported, so I cannot fix this problem, I need some help, I provide a special firmware in the link below, this firmware fixes the Ethernet driver problem and logs the system crash as much as possible in the event of a system crash.
https://dl.gl-inet.com/firmware/mt1300/testing/openwrt-mt1300-3.203-0826-oops-track.bin

When you notice that your device is rebooting, you first need to log in to the router via SSH and then execute the following command from the console
dd if=/dev/mtd7 of=/last.log
After the command is executed, you can copy last.log to your computer via SCP or some other tool, and then send the log to me via email or forum.

The above steps are a little complicated for ordinary users, if you have no experience at all, you can contact me for remote assistance.

Matus · 2021-08-27T10:47:28.826Z

Hi
The problem is that at the time it hungs - there is no connectivity at all to device. It is frozen.
I can also confirm that negotation rates are a bit slow.
On one of my sites ISP provides gateway box for fibre - which in my setup working in DMZ + all port forward on MT1300 - but have wifi enabled. When it is frozen - I can see that negotated speed on this router is showing 10MB instead 1G. Yet - no connectivity untill rebooted.

Also - were trying to not use that ISP modem and use Huawei OTP device directly - but due to fact it required PPPoE and VLANing - hungs occured even more. So I presume it have to something to do with max number of conenction of etherenet driver …

Suggestion you are giving is the right one: deumping the kernel / cores /logs - but we sure ain’t gonna receive it when system os frozen…

My suggestion is that you redirect log writing to USB device or add tunning attribute that can be specified elsewhere… so that logs can be reviewed after router restart/hungs.

robotluo · 2021-08-30T02:15:19.040Z

Matus:

On one of my sites ISP provides gateway box for fibre - which in my setup working in DMZ + all port forward on MT1300 - but have wifi enabled. When it is frozen - I can see that negotated speed on this router is showing 10MB instead 1G. Yet - no connectivity untill rebooted.

I can confirm that the firmware above solves the Ethernet negotiation rate of 10M, you can try it

Matus · 2021-09-01T12:15:01.200Z

Sure. Will try.

One thing: you should really place a release notes for each version somewhere

Matus · 2021-09-15T14:29:38.669Z

It still hungs once in a while (once a day) on openwrt-mt1300-3.203-0826-oops-track.bin. No response via web interface/ssh

alzhao · 2021-09-17T10:18:58.212Z

robotluo:

dd if=/dev/mtd7 of=/last.log

Can you check if you can get the log using above command?

Matus · 2021-09-17T10:31:15.406Z

last.zip (14.5 KB)

Matus · 2021-09-17T10:31:53.420Z

4>[ 3499.512485] —[ end trace cc44022468960fd3 ]—
<0>[ 3499.521024] Kernel panic - not syncing: Fatal exception

robotluo · 2021-09-17T10:35:08.510Z

Thank you for your log, your device is always hanging, did not auto reboot, right?

Matus · 2021-09-17T10:36:46.363Z

Hmmm. From kernel trace looks like NewZeland site is getting this started… That one is not on gl.INET (goodCloud) - 3 are in Poland and now we have 4th in UK
IPv6: ADDRCONF(NETDEV_UP): NewZelan: link is not ready
<4>[ 2663.446029] Unhandled kernel unaligned access

Will check (by disabling ipv6) on it

robotluo · 2021-09-18T09:00:50.828Z

image1001×738 27.3 KB

According to the log, the device has been rebooting repeatedly instead of crashing.
Do you have any special network configuration?

robotluo · 2021-09-18T09:46:12.941Z

I think it may be caused by hardware NAT offload. Please run the following command to disable the hardware NAT offload function of the router.

uci set firewall.@defaults[0].flow_offloading='0'
uci set firewall.@defaults[0].flow_offloading_hw='0'
uci commit
fw3 reload

Matus · 2021-09-20T06:38:53.136Z

robotluo:

uci commit
fw3 reload

Will try and let you know if any improvements.

Matus · 2021-09-20T12:21:05.956Z

robotluo:

dd if=/dev/mtd7 of=/last.log

Still the same after disabling firewall offload:
uci show firewall | grep flow
firewall.@defaults[0].flow_offloading=‘0’
firewall.@defaults[0].flow_offloading_hw=‘0’

<0>[ 1284.348260] Kernel panic - not syncing: Fatal exception
last.zip (12.7 KB)

Matus · 2021-09-20T12:45:19.067Z

<1>[ 1283.876010] CPU 1 Unable to handle kernel paging request at virtual address 00000020, epc == 80132900, ra == 801328d4

<4>[ 1283.889137] CPU: 1 PID: 3601 Comm: gltertf Not tainted 4.14.221 #0

gltertf is one thing whcih was rather changed (that is at least the stuff I was using to erase from orphaned client list in early release 3.200). It also seem to have a lot of changes since early version…

Any option to stop glterf from launching (or it is being used by another processes)

What is about gltertf: service gl_tertf stop - why is it trying to load ipv6 while is not enabled:
cp: can’t stat ‘/tmp/tertf/traffic_info_bak’: No such file or directory
Warning: Section @zone[1] (wan) cannot resolve device of network ‘wan6’
Warning: Option ‘wireguard’.masq6 is unknown
Warning: Option ‘sambasharewan’.dest_proto is unknown
Warning: Section ‘sambasharewan’ does not specify a protocol, assuming TCP+UDP
Warning: Option ‘sambasharelan’.dest_proto is unknown
Warning: Section ‘sambasharelan’ does not specify a protocol, assuming TCP+UDP
Warning: Option @redirect[0].gl is unknown
…
…

Populating IPv6 nat table
Warning: fw3_ipt_rule_append(): Can’t find target ‘prerouting_lan_rule’
Warning: fw3_ipt_rule_append(): Can’t find target ‘postrouting_lan_rule’
Warning: fw3_ipt_rule_append(): Can’t find target ‘prerouting_wan_rule’
Warning: fw3_ipt_rule_append(): Can’t find target ‘postrouting_wan_rule’
Warning: fw3_ipt_rule_append(): Can’t find target ‘prerouting_guestzone_rule’
Warning: fw3_ipt_rule_append(): Can’t find target ‘postrouting_guestzone_rule’
Warning: fw3_ipt_rule_append(): Can’t find target ‘prerouting_wireguard_rule’
Warning: fw3_ipt_rule_append(): Can’t find target ‘postrouting_wireguard_rule’
Warning: fw3_ipt_rule_append(): Can’t find target ‘prerouting_allpermit_rule’
Warning: fw3_ipt_rule_append(): Can’t find target ‘postrouting_allpermit_rule’
Warning: fw3_ipt_rule_append(): Can’t find target ‘prerouting_rule’
Warning: fw3_ipt_rule_append(): Can’t find target ‘postrouting_rule’
Running script ‘/etc/firewall.user’
uci: Entry not found
uci: Entry not found
iptables: No chain/target/match by that name.
iptables: No chain/target/match by that name.
iptables: No chain/target/match by that name.

robotluo · 2021-09-23T02:54:47.696Z

This is a very useful message, I think it is multithreaded processing Linux pipeline caused by the exception, please try to close the system using the most frequent pipeline program, execute the following command.

/etc/init.d/gl_tertf stop
/etc/init.d/gl_tertf disable

The above command will disable the client statistics and offline client record functions. If you want to enable them again, use the following command

/etc/init.d/gl_tertf enable
/etc/init.d/gl_tertf start

Matus · 2021-09-23T07:07:40.497Z

I have did something new. Resigned from ISP’s given router/modem and used Huawei OTP box.
Unfortunatelly my ISP have PPPoE + VLAN tag. So while gl.inet support PPPoE it doesn’t work out of the box and I need to dig into LUCI anyway (switch setup). Bad thing is that default tag is set to 2 (eth0.2) so needed to manually edit /etc/config/network.

config interface ‘wan’
option hostname ‘GL-MT1300-3f8’
option metric ‘10’
option proto ‘pppoe’
option username ‘XXXXXXXXX’
option password ‘XXXXXXXXX’
option disabled ‘0’
option ifname ‘eth0.35’
option ipv6 ‘1’
option delegate ‘0’

config interface ‘wan6’
option proto ‘dhcpv6’
option reqprefix ‘auto’
option reqaddress ‘try’
option ifname ‘@wan’
option disabled ‘0’

config switch
option name ‘switch0’
option reset ‘1’
option enable_vlan ‘1’

config switch_vlan
option device ‘switch0’
option vlan ‘1’
option ports ‘6t 0 1 2 3’
option vid ‘1’

config switch_vlan
option device ‘switch0’
option vlan ‘2’
option ports ‘6t 4t’
option vid ‘35’

Also removed custom config for wireguard and asked friend in NewZeland to buy gl.inet to spin it into goodcloud. Also removed dhcp option 6 remove push domain server and did it through tagging (no need for washing machine or air conditioning to query domain controller)

robotluo · 2021-09-26T03:00:22.803Z

Does your configuration work properly? Can I help you?
Vlan Settings will be supported in future 4.x firmware.

Matus · 2021-10-06T15:19:07.101Z

What actually worked was to replace unit with GL-S1300
Same config (used luci clone) - working as VPN concentrator for 3 other sites: online for last 7 days and no hicckups (reboots)
I think that GL-MT1300 require a lot more stability fixes. Is fine with small number of devices connected (up to 5-10 active) - but problems start with more complex (60+ devices + a lot of traffic). Performance of MT13000 is fine. Just stability far from good.

Matus · 2021-11-22T14:36:03.752Z

Any luck in making router reboot instead freeze? Reboots (although not welcome) are tolerable: complete freeze is just worst scenario ever…