GL-AXT1800 (Slate) crashed with no crash log

ericswpark · October 8, 2024, 1:40am

Title; I was using my laptop when I noticed the Ethernet icon blink out. Checking the router admin page it seems like the router crashed and rebooted by itself, and the uptime showed in minutes. Weirdly enough, there is nothing underneath the "Crash Log" tab.

The router is running the latest firmware (4.6.4 - release1). Any ideas?

bruce · October 8, 2024, 9:38am

Even if the system crashes, the crash log still be there after the reboot, it recorded in flash.

Probably the system does not crash, but the power lost or power not enough, please use the factory attach power supply.

ericswpark · October 8, 2024, 12:17pm

I can confirm the router was using the factory power supply upon crash. It wasn't a power interruption.

bruce · October 9, 2024, 11:44am

Please login the SSH, and execute the command:
logread
it will real-time print the syslog, let see what kernel print when it 'crash'

ericswpark · October 9, 2024, 1:04pm

Here you go: syslog from GL-AXT1800 · GitHub

bruce · October 10, 2024, 10:36am

Based on the logs, it seems that eth2 (probably PHY LAN2) is constantly going up and down, but system doesn't seem to be crashing.
Try changing the network cable with a better one.

No other issues, WAN works fine.

ericswpark · November 21, 2024, 12:56am

I got another crash today and this time the crash logs populated.

Oops#1 Part1
<4>[1375643.308962] CPU: 1 PID: 20250 Comm: kworker/1:1 Not tainted 4.4.60 #0
<4>[1375643.309317] Hardware name: Generic DT based system
<4>[1375643.315749] Workqueue: events rproc_crash_handler_work
<4>[1375643.325886] task: 9bc13180 ti: 8f43c000 task.ti: 8f43c000
<4>[1375643.326201] PC is at ath11k_core_ssr_notifier_cb+0xa4/0xb4 [ath11k]
<4>[1375643.331571] LR is at ath11k_hal_dump_srng_stats+0x248/0x2c0 [ath11k]
<4>[1375643.338140] pc : [<7f7e0064>]    lr : [<7f7e2764>]    psr: 60000013
<4>[1375643.338140] sp : 8f43de28  ip : 00000000  fp : 9d816428
<4>[1375643.344570] r10: 8083a520  r9 : 9d816664  r8 : 00000000
<4>[1375643.356277] r7 : 8075fee0  r6 : 9e5af610  r5 : 8075fee0  r4 : 9aba092c
<4>[1375643.361836] r3 : 00000000  r2 : dc8ba63c  r1 : 9eb4a324  r0 : 0000007e
<4>[1375643.368608] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment kernel
<4>[1375643.375292] Control: 10c0383d  Table: 515a006a  DAC: 00000055
<0>[1375643.382668] Process kworker/1:1 (pid: 20250, stack limit = 0x8f43c210)
<0>[1375643.388399] Stack: (0x8f43de28 to 0x8f43e000)
<0>[1375643.395342] de20:                   00000002 9aba192c 00000000 fffffffd 00000000 80044da0
<0>[1375643.399602] de40: 8087d520 00000000 8075fee0 ffffffff 9d816428 80045094 00000000 8f43de6c
<0>[1375643.407935] de60: 00000000 9d816778 9d816604 00000001 9d816400 800450c8 00000000 803572ec
<0>[1375643.416269] de80: 9d81662c 8044c40c 9d81662c 9d816664 9d816400 9d81662c 9d8165c0 8044df58
<0>[1375643.424601] dea0: 00000001 80357180 8073f73c 8075efec 9d8074c0 00000000 3197f26c 9d81662c
<0>[1375643.432936] dec0: 9d816664 00000003 9d81662c 9d8165c0 9d816664 8083a520 9eb4cb40 8044e290
<0>[1375643.441268] dee0: 8044e164 9d81662c 9006de80 9eb4cb40 9eb50000 00000000 00000000 8083a520
<0>[1375643.449602] df00: 9eb4cb40 8003ed30 9eb4cb40 9eb4cb40 00000008 9006de80 9006de98 9eb4cb40
<0>[1375643.457935] df20: 00000008 9eb4cb54 8f43c000 8083a520 9eb4cb40 8003f1b0 80836100 9006de80
<0>[1375643.466268] df40: 8003eec8 00000000 9bdf9080 9006de80 8003eec8 00000000 00000000 00000000
<0>[1375643.474601] df60: 00000000 8004428c 8f43dd10 00000000 00000000 9006de80 00000000 00000000
<0>[1375643.482935] df80: 8f43df80 8f43df80 00000000 00000000 8f43df90 8f43df90 8f43dfac 9bdf9080
<0>[1375643.491268] dfa0: 8004419c 00000000 00000000 80009c28 00000000 00000000 00000000 00000000
<0>[1375643.499600] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<0>[1375643.507935] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000
<4>[1375643.516398] [<7f7e0064>] (ath11k_core_ssr_notifier_cb [ath11k]) from [<80044da0>] (notifier_call_chain+0x2c/0x70)
<4>[1375643.524610] [<80044da0>] (notifier_call_chain) from [<80045094>] (__blocking_notifier_call_chain+0x48/0x64)
<4>[1375643.535024] [<80045094>] (__blocking_notifier_call_chain) from [<800450c8>] (blocking_notifier_call_chain+0x18/0x20)
<4>[1375643.545097] [<800450c8>] (blocking_notifier_call_chain) from [<8044c40c>] (rproc_stop+0x2c/0xc4)
<4>[1375643.555683] [<8044c40c>] (rproc_stop) from [<8044df58>] (rproc_trigger_recovery+0x4c/0x258)
<4>[1375643.564622] [<8044df58>] (rproc_trigger_recovery) from [<8044e290>] (rproc_crash_handler_work+0x12c/0x1bc)
<4>[1375643.573220] [<8044e290>] (rproc_crash_handler_work) from [<8003ed30>] (process_one_work+0x220/0x3b8)
<4>[1375643.583113] [<8003ed30>] (process_one_work) from [<8003f1b0>] (worker_thread+0x2e8/0x4d0)
<4>[1375643.592224] [<8003f1b0>] (worker_thread) from [<8004428c>] (kthread+0xf0/0xf8)
<4>[1375643.600470] [<8004428c>] (kthread) from [<80009c28>] (ret_from_fork+0x14/0x2c)
<0>[1375643.608108] Code: eb2a20eb e3500000 0affffe9 ea000000 (e7f001f2) 
<4>[1375643.615487] ---[ end trace a44130f51c0a1937 ]---


Panic#2 Part1
 ip6t_NPT ip6t_MASQUERADE nf_nat_masquerade_ipv6 nf_nat nf_conntrack nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 bonding ifb ip6_udp_tunnel udp_tunnel qca_nss_drv qca_nss_dp udp_diag tcp_diag raw_diag inet_diag tun qca_ssdk vfat fat ntfs nls_utf8 nls_iso8859_1 nls_cp437 mtdoops uas usb_storage gl_fan_driver xhci_plat_hcd xhci_pci xhci_hcd dwc3 dwc3_qcom dbm uhci_hcd ohci_platform ohci_hcd phy_msm_ssusb_qmp phy_msm_qusb ehci_platform sd_mod scsi_mod kmwan ehci_hcd gpio_button_hotplug gl_sdk4_tertf gl_sdk4_black_white_list usbcore nls_base usb_common mii gl_sdk4_hw_info
<4>[1375643.308962] CPU: 1 PID: 20250 Comm: kworker/1:1 Not tainted 4.4.60 #0
<4>[1375643.309317] Hardware name: Generic DT based system
<4>[1375643.315749] Workqueue: events rproc_crash_handler_work
<4>[1375643.325886] task: 9bc13180 ti: 8f43c000 task.ti: 8f43c000
<4>[1375643.326201] PC is at ath11k_core_ssr_notifier_cb+0xa4/0xb4 [ath11k]
<4>[1375643.331571] LR is at ath11k_hal_dump_srng_stats+0x248/0x2c0 [ath11k]
<4>[1375643.338140] pc : [<7f7e0064>]    lr : [<7f7e2764>]    psr: 60000013
<4>[1375643.338140] sp : 8f43de28  ip : 00000000  fp : 9d816428
<4>[1375643.344570] r10: 8083a520  r9 : 9d816664  r8 : 00000000
<4>[1375643.356277] r7 : 8075fee0  r6 : 9e5af610  r5 : 8075fee0  r4 : 9aba092c
<4>[1375643.361836] r3 : 00000000  r2 : dc8ba63c  r1 : 9eb4a324  r0 : 0000007e
<4>[1375643.368608] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment kernel
<4>[1375643.375292] Control: 10c0383d  Table: 515a006a  DAC: 00000055
<0>[1375643.382668] Process kworker/1:1 (pid: 20250, stack limit = 0x8f43c210)
<0>[1375643.388399] Stack: (0x8f43de28 to 0x8f43e000)
<0>[1375643.395342] de20:                   00000002 9aba192c 00000000 fffffffd 00000000 80044da0
<0>[1375643.399602] de40: 8087d520 00000000 8075fee0 ffffffff 9d816428 80045094 00000000 8f43de6c
<0>[1375643.407935] de60: 00000000 9d816778 9d816604 00000001 9d816400 800450c8 00000000 803572ec
<0>[1375643.416269] de80: 9d81662c 8044c40c 9d81662c 9d816664 9d816400 9d81662c 9d8165c0 8044df58
<0>[1375643.424601] dea0: 00000001 80357180 8073f73c 8075efec 9d8074c0 00000000 3197f26c 9d81662c
<0>[1375643.432936] dec0: 9d816664 00000003 9d81662c 9d8165c0 9d816664 8083a520 9eb4cb40 8044e290
<0>[1375643.441268] dee0: 8044e164 9d81662c 9006de80 9eb4cb40 9eb50000 00000000 00000000 8083a520
<0>[1375643.449602] df00: 9eb4cb40 8003ed30 9eb4cb40 9eb4cb40 00000008 9006de80 9006de98 9eb4cb40
<0>[1375643.457935] df20: 00000008 9eb4cb54 8f43c000 8083a520 9eb4cb40 8003f1b0 80836100 9006de80
<0>[1375643.466268] df40: 8003eec8 00000000 9bdf9080 9006de80 8003eec8 00000000 00000000 00000000
<0>[1375643.474601] df60: 00000000 8004428c 8f43dd10 00000000 00000000 9006de80 00000000 00000000
<0>[1375643.482935] df80: 8f43df80 8f43df80 00000000 00000000 8f43df90 8f43df90 8f43dfac 9bdf9080
<0>[1375643.491268] dfa0: 8004419c 00000000 00000000 80009c28 00000000 00000000 00000000 00000000
<0>[1375643.499600] dfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<0>[1375643.507935] dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000
<4>[1375643.516398] [<7f7e0064>] (ath11k_core_ssr_notifier_cb [ath11k]) from [<80044da0>] (notifier_call_chain+0x2c/0x70)
<4>[1375643.524610] [<80044da0>] (notifier_call_chain) from [<80045094>] (__blocking_notifier_call_chain+0x48/0x64)
<4>[1375643.535024] [<80045094>] (__blocking_notifier_call_chain) from [<800450c8>] (blocking_notifier_call_chain+0x18/0x20)
<4>[1375643.545097] [<800450c8>] (blocking_notifier_call_chain) from [<8044c40c>] (rproc_stop+0x2c/0xc4)
<4>[1375643.555683] [<8044c40c>] (rproc_stop) from [<8044df58>] (rproc_trigger_recovery+0x4c/0x258)
<4>[1375643.564622] [<8044df58>] (rproc_trigger_recovery) from [<8044e290>] (rproc_crash_handler_work+0x12c/0x1bc)
<4>[1375643.573220] [<8044e290>] (rproc_crash_handler_work) from [<8003ed30>] (process_one_work+0x220/0x3b8)
<4>[1375643.583113] [<8003ed30>] (process_one_work) from [<8003f1b0>] (worker_thread+0x2e8/0x4d0)
<4>[1375643.592224] [<8003f1b0>] (worker_thread) from [<8004428c>] (kthread+0xf0/0xf8)
<4>[1375643.600470] [<8004428c>] (kthread) from [<80009c28>] (ret_from_fork+0x14/0x2c)
<0>[1375643.608108] Code: eb2a20eb e3500000 0affffe9 ea000000 (e7f001f2) 
<4>[1375643.615487] ---[ end trace a44130f51c0a1937 ]---
<0>[1375643.630705] Kernel panic - not syncing: Fatal exception
<2>[1375643.630729] CPU2: stopping
<4>[1375643.635068] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G      D         4.4.60 #0
<4>[1375643.638023] Hardware name: Generic DT based system
<4>[1375643.645063] [<8001f784>] (unwind_backtrace) from [<8001bad4>] (show_stack+0x10/0x14)
<4>[1375643.650271] [<8001bad4>] (show_stack) from [<80260c10>] (dump_stack+0x8c/0xa0)
<4>[1375643.657994] [<80260c10>] (dump_stack) from [<8001e6e0>] (handle_IPI+0x104/0x190)
<4>[1375643.665546] [<8001e6e0>] (handle_IPI) from [<800093bc>] (gic_handle_irq+0x80/0x94)
<4>[1375643.672924] [<800093bc>] (gic_handle_irq) from [<8000a4c0>] (__irq_svc+0x40/0x74)
<4>[1375643.680819] Exception stack(0x9e48df88 to 0x9e48dfd0)
<4>[1375643.688195] df80:                   00000001 00000000 00000000 8000b400 9e48c000 808364a0
<4>[1375643.693409] dfa0: 806d2f48 00000000 00000000 8082f30c 9e48dfe0 808364a8 60000013 9e48dfd8
<4>[1375643.701740] dfc0: 80019394 80019398 60000013 ffffffff
<4>[1375643.710071] [<8000a4c0>] (__irq_svc) from [<80019398>] (arch_cpu_idle+0x34/0x50)
<4>[1375643.715285] [<80019398>] (arch_cpu_idle) from [<80063664>] (cpu_startup_entry+0x154/0x228)
<4>[1375643.722925] [<80063664>] (cpu_startup_entry) from [<4100946c>] (0x4100946c)
<2>[1375643.731512] CPU0: stopping
<4>[1375643.738537] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G      D         4.4.60 #0
<4>[1375643.741494] Hardware name: Generic DT based system
<4>[1375643.748528] [<8001f784>] (unwind_backtrace) from [<8001bad4>] (show_stack+0x10/0x14)
<4>[1375643.753740] [<8001bad4>] (show_stack) from [<80260c10>] (dump_stack+0x8c/0xa0)
<4>[1375643.761465] [<80260c10>] (dump_stack) from [<8001e6e0>] (handle_IPI+0x104/0x190)
<4>[1375643.769016] [<8001e6e0>] (handle_IPI) from [<800093bc>] (gic_handle_irq+0x80/0x94)
<4>[1375643.776396] [<800093bc>] (gic_handle_irq) from [<8000a4c0>] (__irq_svc+0x40/0x74)
<4>[1375643.784291] Exception stack(0x80835f50 to 0x80835f98)
<4>[1375643.791666] 5f40:                                     00000001 00000000 00000000 8000b400
<4>[1375643.796881] 5f60: 80834000 808364a0 806d2f48 00000000 00000000 8082f30c 80835fa8 808364a8
<4>[1375643.805215] 5f80: 80835f48 80835fa0 80019394 80019398 60000013 ffffffff
<4>[1375643.813546] [<8000a4c0>] (__irq_svc) from [<80019398>] (arch_cpu_idle+0x34/0x50)
<4>[1375643.820493] [<80019398>] (arch_cpu_idle) from [<80063664>] (cpu_startup_entry+0x154/0x228)
<4>[1375643.827962] [<80063664>] (cpu_startup_entry) from [<807f7d28>] (start_kernel+0x430/0x43c)
<2>[1375643.836549] CPU3: stopping
<4>[1375643.844615] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G      D         4.4.60 #0
<4>[1375643.847744] Hardware name: Generic DT based system
<4>[1375643.854778] [<8001f784>] (unwind_backtrace) from [<8001bad4>] (show_stack+0x10/0x14)
<4>[1375643.859989] [<8001bad4>] (show_stack) from [<80260c10>] (dump_stack+0x8c/0xa0)
<4>[1375643.867714] [<80260c10>] (dump_stack) from [<8001e6e0>] (handle_IPI+0x104/0x190)
<4>[1375643.875266] [<8001e6e0>] (handle_IPI) from [<800093bc>] (gic_handle_irq+0x80/0x94)
<4>[1375643.882645] [<800093bc>] (gic_handle_irq) from [<8000a4c0>] (__irq_svc+0x40/0x74)
<4>[1375643.890540] Exception stack(0x9e48ff88 to 0x9e48ffd0)
<4>[1375643.897916] ff80:                   00000001 00000000 00000000 8000b400 9e48e000 808364a0
<4>[1375643.903132] ffa0: 806d2f48 00000000 00000000 8082f30c 9e48ffe0 808364a8 60000013 9e48ffd8
<4>[1375643.911462] ffc0: 80019394 80019398 60000013 ffffffff
<4>[1375643.919794] [<8000a4c0>] (__irq_svc) from [<80019398>] (arch_cpu_idle+0x34/0x50)
<4>[1375643.925006] [<80019398>] (arch_cpu_idle) from [<80063664>] (cpu_startup_entry+0x154/0x228)
<4>[1375643.932647] [<80063664>] (cpu_startup_entry) from [<4100946c>] (0x4100946c)
<0>[1375643.941288] Couldn't get reading for sensor 4
<0>[1375643.948263] The reading for sensor 5 is 0x002052c1
<0>[1375643.952604] The reading for sensor 7 is 0x002072b0
<0>[1375643.957812] The reading for sensor 8 is 0x002082a6
<0>[1375643.962760] The reading for sensor 13 is 0x0022d2b7
<0>[1375643.967755] Couldn't get reading for sensor 14
<0>[1375643.972656] The reading for sensor 15 is 0x0020f2b4

EDIT: Router was running version 4.6.8, release1.

ericswpark · November 21, 2024, 5:19am

Yet another crash just now. This time the crash log is empty. I was issuing a command on my HomePod Mini and then the second command was taking way too long, so I checked my phone and found that the WiFi signal was gone. Uptime reset to 0 again but there was no crash log. This is getting pretty frustrating.

bruce · November 22, 2024, 2:13am

Please export this whole syslog file (logread.tar) and share to us, it is crucial to our analysis.

Please let me know what features/applications enabled and third-party plugins installed?
Have you tried to reset the router firmware or brand-new flash the firmware in uboot mode?

ericswpark · November 22, 2024, 2:35am

@bruce that was the one before the latest crash. I have the logread.tar from the latest crash but that doesn't have an associated crash log unfortunately. I've uploaded it here just in case:

logread.tar (170 KB)

Most if not all of the features are disabled. I have an SD card plugged into the router itself but don't use it to store or serve files or anything like that. I don't believe I've installed any 3rd party plugins either.
Not yet. If I export the settings and then reimport it after a brand-new flash, might that allow the cause of the problem to carry over to the fresh environment? I'd really like to avoid setting it up from scratch because it's such a hassle.

If it does crash again and produce crash logs I will upload another logread.tar file.

bruce · November 22, 2024, 3:44am

Thank you.

This function is printed whenever the system crashes, but the cause of the crash has nothing to do with this function.

If possible, you can also try to remove the SD card and check again.
If the crash still occurs, need to get the syslog to check.

BTW, what client device is connected to the second LAN port (eht2)? Can it disconnect?

ericswpark · November 22, 2024, 4:34am

Okay, I've removed the SD card and will report back if it panics again.

eth2 should be my laptop, it disconnects often because I use a USB-C to Ethernet adapter from Ugreen (I think it has the AX88179A chip) and for some reason whenever Windows goes to sleep the adapter sometimes disconnects. I don't know if it's a Windows problem or a ASIX driver problem or maybe a faulty adapter. I can try replacing it, but I do need a wired connection for my laptop due to the latency.