Flint crashed and rebooted - from the crash log any ideas?

My Flint GL-AX1800 has been pretty stable running Code 4.5.0 Release 8

Here’s the crash log - any ideas why it rebooted?

Thanks

Oops#1 Part1
<4>[994317.166004] CPU: 1 PID: 30157 Comm: kworker/1:1 Not tainted 4.4.60 #0
<4>[994317.166359] Hardware name: Generic DT based system
<4>[994317.172787] Workqueue: events rproc_crash_handler_work
<4>[994317.182755] task: 9b301680 ti: 86500000 task.ti: 86500000
<4>[994317.182979] PC is at ath11k_core_ssr_notifier_cb+0xa4/0xb4 [ath11k]
<4>[994317.188352] LR is at ath11k_hal_dump_srng_stats+0x248/0x2c0 [ath11k]
<4>[994317.194833] pc : [<7f7a2064>] lr : [<7f7a4764>] psr: 60000013
<4>[994317.194833] sp : 86501e28 ip : 00000000 fp : 9d82a428
<4>[994317.201178] r10: 8083a520 r9 : 9d82a664 r8 : 00000000
<4>[994317.212798] r7 : 8075fee0 r6 : 9e5af210 r5 : 8075fee0 r4 : 9aea092c
<4>[994317.218182] r3 : 00000000 r2 : dc8ba63c r1 : 9eb4a324 r0 : 0000007d
<4>[994317.224521] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment kernel
<4>[994317.231118] Control: 10c0383d Table: 586c006a DAC: 00000055
<0>[994317.238754] Process kworker/1:1 (pid: 30157, stack limit = 0x86500210)
<0>[994317.244398] Stack: (0x86501e28 to 0x86502000)
<0>[994317.250907] 1e20: 80831764 9aea192c 00000000 fffffffd 00000000 80044da0
<0>[994317.255429] 1e40: 8087d520 00000000 8075fee0 ffffffff 9d82a428 80045094 00000000 86501e6c
<0>[994317.263675] 1e60: 00000000 9d82a778 9d82a604 00000001 9d82a400 800450c8 00000000 803572ec
<0>[994317.271921] 1e80: 9d82a62c 8044c40c 9d82a62c 9d82a664 9d82a400 9d82a62c 9d82a5c0 8044df58
<0>[994317.280167] 1ea0: 00000001 80357180 8073f73c 8075efec 9d802380 00000000 734d0751 9d82a62c
<0>[994317.288415] 1ec0: 9d82a664 00000003 9d82a62c 9d82a5c0 9d82a664 8083a520 9eb4cb40 8044e290
<0>[994317.296660] 1ee0: 8044e164 9d82a62c 9a9a4080 9eb4cb40 9eb50000 00000000 00000000 8083a520
<0>[994317.304907] 1f00: 9eb4cb40 8003ed30 9eb4cb40 9eb4cb40 00000008 9a9a4080 9a9a4098 9eb4cb40
<0>[994317.313154] 1f20: 00000008 9eb4cb54 86500000 8083a520 9eb4cb40 8003f1b0 80836100 9a9a4080
<0>[994317.321400] 1f40: 8003eec8 00000000 81ecfe80 9a9a4080 8003eec8 00000000 00000000 00000000
<0>[994317.329646] 1f60: 00000000 8004428c 86501d10 00000000 00000000 9a9a4080 00000000 00000000
<0>[994317.337893] 1f80: 86501f80 86501f80 00000000 00000000 86501f90 86501f90 86501fac 81ecfe80
<0>[994317.346139] 1fa0: 8004419c 00000000 00000000 80009c28 00000000 00000000 00000000 00000000
<0>[994317.354386] 1fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<0>[994317.362633] 1fe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000
<4>[994317.370998] [<7f7a2064>] (ath11k_core_ssr_notifier_cb [ath11k]) from [<80044da0>] (notifier_call_chain+0x2c/0x70)
<4>[994317.379133] [<80044da0>] (notifier_call_chain) from [<80045094>] (__blocking_notifier_call_chain+0x48/0x64)
<4>[994317.389461] [<80045094>] (__blocking_notifier_call_chain) from [<800450c8>] (blocking_notifier_call_chain+0x18/0x20)
<4>[994317.399447] [<800450c8>] (blocking_notifier_call_chain) from [<8044c40c>] (rproc_stop+0x2c/0xc4)
<4>[994317.409946] [<8044c40c>] (rproc_stop) from [<8044df58>] (rproc_trigger_recovery+0x4c/0x258)
<4>[994317.418799] [<8044df58>] (rproc_trigger_recovery) from [<8044e290>] (rproc_crash_handler_work+0x12c/0x1bc)
<4>[994317.427308] [<8044e290>] (rproc_crash_handler_work) from [<8003ed30>] (process_one_work+0x220/0x3b8)
<4>[994317.436768] [<8003ed30>] (process_one_work) from [<8003f1b0>] (worker_thread+0x2e8/0x4d0)
<4>[994317.446141] [<8003f1b0>] (worker_thread) from [<8004428c>] (kthread+0xf0/0xf8)
<4>[994317.454298] [<8004428c>] (kthread) from [<80009c28>] (ret_from_fork+0x14/0x2c)
<0>[994317.461503] Code: eb2b18eb e3500000 0affffe9 ea000000 (e7f001f2)
<4>[994317.468796] —[ end trace 330b1c930e4dc00a ]—

Panic#2 Part1
ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 bonding ifb ip6_udp_tunnel udp_tunnel qca_nss_drv qca_nss_dp udp_diag tcp_diag raw_diag inet_diag tun qca_ssdk vfat fat ntfs nls_utf8 nls_iso8859_1 nls_cp437 mtdoops uas usb_storage xhci_plat_hcd xhci_pci xhci_hcd dwc3 dwc3_qcom dbm uhci_hcd ohci_platform ohci_hcd phy_msm_ssusb_qmp phy_msm_qusb ehci_platform sd_mod scsi_mod kmwan ehci_hcd gpio_button_hotplug gl_sdk4_tertf gl_sdk4_black_white_list usbcore nls_base usb_common mii gl_sdk4_hw_info
<4>[994317.166004] CPU: 1 PID: 30157 Comm: kworker/1:1 Not tainted 4.4.60 #0
<4>[994317.166359] Hardware name: Generic DT based system
<4>[994317.172787] Workqueue: events rproc_crash_handler_work
<4>[994317.182755] task: 9b301680 ti: 86500000 task.ti: 86500000
<4>[994317.182979] PC is at ath11k_core_ssr_notifier_cb+0xa4/0xb4 [ath11k]
<4>[994317.188352] LR is at ath11k_hal_dump_srng_stats+0x248/0x2c0 [ath11k]
<4>[994317.194833] pc : [<7f7a2064>] lr : [<7f7a4764>] psr: 60000013
<4>[994317.194833] sp : 86501e28 ip : 00000000 fp : 9d82a428
<4>[994317.201178] r10: 8083a520 r9 : 9d82a664 r8 : 00000000
<4>[994317.212798] r7 : 8075fee0 r6 : 9e5af210 r5 : 8075fee0 r4 : 9aea092c
<4>[994317.218182] r3 : 00000000 r2 : dc8ba63c r1 : 9eb4a324 r0 : 0000007d
<4>[994317.224521] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment kernel
<4>[994317.231118] Control: 10c0383d Table: 586c006a DAC: 00000055
<0>[994317.238754] Process kworker/1:1 (pid: 30157, stack limit = 0x86500210)
<0>[994317.244398] Stack: (0x86501e28 to 0x86502000)
<0>[994317.250907] 1e20: 80831764 9aea192c 00000000 fffffffd 00000000 80044da0
<0>[994317.255429] 1e40: 8087d520 00000000 8075fee0 ffffffff 9d82a428 80045094 00000000 86501e6c
<0>[994317.263675] 1e60: 00000000 9d82a778 9d82a604 00000001 9d82a400 800450c8 00000000 803572ec
<0>[994317.271921] 1e80: 9d82a62c 8044c40c 9d82a62c 9d82a664 9d82a400 9d82a62c 9d82a5c0 8044df58
<0>[994317.280167] 1ea0: 00000001 80357180 8073f73c 8075efec 9d802380 00000000 734d0751 9d82a62c
<0>[994317.288415] 1ec0: 9d82a664 00000003 9d82a62c 9d82a5c0 9d82a664 8083a520 9eb4cb40 8044e290
<0>[994317.296660] 1ee0: 8044e164 9d82a62c 9a9a4080 9eb4cb40 9eb50000 00000000 00000000 8083a520
<0>[994317.304907] 1f00: 9eb4cb40 8003ed30 9eb4cb40 9eb4cb40 00000008 9a9a4080 9a9a4098 9eb4cb40
<0>[994317.313154] 1f20: 00000008 9eb4cb54 86500000 8083a520 9eb4cb40 8003f1b0 80836100 9a9a4080
<0>[994317.321400] 1f40: 8003eec8 00000000 81ecfe80 9a9a4080 8003eec8 00000000 00000000 00000000
<0>[994317.329646] 1f60: 00000000 8004428c 86501d10 00000000 00000000 9a9a4080 00000000 00000000
<0>[994317.337893] 1f80: 86501f80 86501f80 00000000 00000000 86501f90 86501f90 86501fac 81ecfe80
<0>[994317.346139] 1fa0: 8004419c 00000000 00000000 80009c28 00000000 00000000 00000000 00000000
<0>[994317.354386] 1fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<0>[994317.362633] 1fe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000
<4>[994317.370998] [<7f7a2064>] (ath11k_core_ssr_notifier_cb [ath11k]) from [<80044da0>] (notifier_call_chain+0x2c/0x70)
<4>[994317.379133] [<80044da0>] (notifier_call_chain) from [<80045094>] (__blocking_notifier_call_chain+0x48/0x64)
<4>[994317.389461] [<80045094>] (__blocking_notifier_call_chain) from [<800450c8>] (blocking_notifier_call_chain+0x18/0x20)
<4>[994317.399447] [<800450c8>] (blocking_notifier_call_chain) from [<8044c40c>] (rproc_stop+0x2c/0xc4)
<4>[994317.409946] [<8044c40c>] (rproc_stop) from [<8044df58>] (rproc_trigger_recovery+0x4c/0x258)
<4>[994317.418799] [<8044df58>] (rproc_trigger_recovery) from [<8044e290>] (rproc_crash_handler_work+0x12c/0x1bc)
<4>[994317.427308] [<8044e290>] (rproc_crash_handler_work) from [<8003ed30>] (process_one_work+0x220/0x3b8)
<4>[994317.436768] [<8003ed30>] (process_one_work) from [<8003f1b0>] (worker_thread+0x2e8/0x4d0)
<4>[994317.446141] [<8003f1b0>] (worker_thread) from [<8004428c>] (kthread+0xf0/0xf8)
<4>[994317.454298] [<8004428c>] (kthread) from [<80009c28>] (ret_from_fork+0x14/0x2c)
<0>[994317.461503] Code: eb2b18eb e3500000 0affffe9 ea000000 (e7f001f2)
<4>[994317.468796] —[ end trace 330b1c930e4dc00a ]—
<0>[994317.478211] Kernel panic - not syncing: Fatal exception
<2>[994317.479818] CPU0: stopping
<4>[994317.485194] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G D 4.4.60 #0
<4>[994317.487714] Hardware name: Generic DT based system
<4>[994317.495011] [<8001f784>] (unwind_backtrace) from [<8001bad4>] (show_stack+0x10/0x14)
<4>[994317.499789] [<8001bad4>] (show_stack) from [<80260c10>] (dump_stack+0x8c/0xa0)
<4>[994317.507773] [<80260c10>] (dump_stack) from [<8001e6e0>] (handle_IPI+0x104/0x190)
<4>[994317.514890] [<8001e6e0>] (handle_IPI) from [<800093bc>] (gic_handle_irq+0x80/0x94)
<4>[994317.522528] [<800093bc>] (gic_handle_irq) from [<8000a4c0>] (__irq_svc+0x40/0x74)
<4>[994317.529990] Exception stack(0x80835f50 to 0x80835f98)
<4>[994317.537626] 5f40: 00000001 00000000 00000000 8000b400
<4>[994317.542755] 5f60: 80834000 808364a0 806d2f48 00000000 00000000 8082f30c 80835fa8 808364a8
<4>[994317.551000] 5f80: 00000000 80835fa0 80019394 80019398 60000013 ffffffff
<4>[994317.559246] [<8000a4c0>] (__irq_svc) from [<80019398>] (arch_cpu_idle+0x34/0x50)
<4>[994317.566107] [<80019398>] (arch_cpu_idle) from [<80063664>] (cpu_startup_entry+0x154/0x228)
<4>[994317.573488] [<80063664>] (cpu_startup_entry) from [<807f7d28>] (start_kernel+0x430/0x43c)
<2>[994317.581641] CPU2: stopping
<4>[994317.589966] CPU: 2 PID: 0 Comm: swapper/2 Tainted: G D 4.4.60 #0
<4>[994317.592661] Hardware name: Generic DT based system
<4>[994317.599955] [<8001f784>] (unwind_backtrace) from [<8001bad4>] (show_stack+0x10/0x14)
<4>[994317.604733] [<8001bad4>] (show_stack) from [<80260c10>] (dump_stack+0x8c/0xa0)
<4>[994317.612718] [<80260c10>] (dump_stack) from [<8001e6e0>] (handle_IPI+0x104/0x190)
<4>[994317.619836] [<8001e6e0>] (handle_IPI) from [<800093bc>] (gic_handle_irq+0x80/0x94)
<4>[994317.627475] [<800093bc>] (gic_handle_irq) from [<8000a4c0>] (__irq_svc+0x40/0x74)
<4>[994317.634938] Exception stack(0x9e48df88 to 0x9e48dfd0)
<4>[994317.642575] df80: 00000001 00000000 00000000 8000b400 9e48c000 808364a0
<4>[994317.647703] dfa0: 806d2f48 00000000 00000000 8082f30c 9e48dfe0 808364a8 00000000 9e48dfd8
<4>[994317.655946] dfc0: 80019394 80019398 60000013 ffffffff
<4>[994317.664190] [<8000a4c0>] (__irq_svc) from [<80019398>] (arch_cpu_idle+0x34/0x50)
<4>[994317.669317] [<80019398>] (arch_cpu_idle) from [<80063664>] (cpu_startup_entry+0x154/0x228)
<4>[994317.676870] [<80063664>] (cpu_startup_entry) from [<4100946c>] (0x4100946c)
<2>[994317.685023] CPU3: stopping
<4>[994317.692310] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G D 4.4.60 #0
<4>[994317.694833] Hardware name: Generic DT based system
<4>[994317.702126] [<8001f784>] (unwind_backtrace) from [<8001bad4>] (show_stack+0x10/0x14)
<4>[994317.706903] [<8001bad4>] (show_stack) from [<80260c10>] (dump_stack+0x8c/0xa0)
<4>[994317.714889] [<80260c10>] (dump_stack) from [<8001e6e0>] (handle_IPI+0x104/0x190)
<4>[994317.722007] [<8001e6e0>] (handle_IPI) from [<800093bc>] (gic_handle_irq+0x80/0x94)
<4>[994317.729645] [<800093bc>] (gic_handle_irq) from [<8000a4c0>] (__irq_svc+0x40/0x74)
<4>[994317.737108] Exception stack(0x9e48ff88 to 0x9e48ffd0)
<4>[994317.744744] ff80: 00000001 00000000 00000000 8000b400 9e48e000 808364a0
<4>[994317.749873] ffa0: 806d2f48 00000000 00000000 8082f30c 9e48ffe0 808364a8 968efec9 9e48ffd8
<4>[994317.758118] ffc0: 80019394 80019398 60000013 ffffffff
<4>[994317.766360] [<8000a4c0>] (__irq_svc) from [<80019398>] (arch_cpu_idle+0x34/0x50)
<4>[994317.771487] [<80019398>] (arch_cpu_idle) from [<80063664>] (cpu_startup_entry+0x154/0x228)
<4>[994317.779039] [<80063664>] (cpu_startup_entry) from [<4100946c>] (0x4100946c)
<0>[994317.787200] The reading for sensor 4 is 0x0020431b
<0>[994317.794530] Couldn’t get reading for sensor 5
<0>[994317.799082] The reading for sensor 7 is 0x0020730a
<0>[994317.803646] Couldn’t get reading for sensor 8
<0>[994317.808372] The reading for sensor 13 is 0x0022d31b
<0>[994317.812934] Couldn’t get reading for sensor 14
<0>[994317.818007] The reading for sensor 15 is 0x0020f318

I see ip6_tables mentioned in the panic, but I do not have ipv6 enabled.

I only have WG server (no clients connected) AdGuard, 2g and 5g WiFi enabled, nothing fancy.

Thanks

Hmm it looks your wifi driver crashed.

But what bothers me is this here:

Since adguard is involved in your message can you check how much space your device has left or maybe if you see it increase?

It very much sound to me the device could not keep up anymore and locked up and i would not really see that by the stack trace but more by the error i quoted.

the last time i was using flint 1, i was worried about the space, its already so much packed that im even afraid to run adguardhome :wink:

Thanks xize11.

I will have to keep an eye on the memory.

I might even have to stop running adguard on this router and move to another service :frowning:

1 Like

maybe it seems still some space, but maybe the logging in adguard is on?

that could be your issue maybe because this can slowly impact your router.

I also noticed some other users recommending oisd only on top of other lists since oisd is basicly just a collection of multiple lists.

Yes that’s a good point.
Do you think that I should disable logging then?

I’ve lowered the logging retention to 6 hours for now, and will monitor memory usage over the next week.

Thanks xize11

best is to turn them fully off but I understand thats also not convient because if you need to check something theres no data.

but its better than nothing :slight_smile:

I decided to just take Adguard off the router completely, set it up on my Raspberry Pi running Home Assistant.

Let’s see if that helps with stability.

Thanks

Tbh for me the kernel panic sounds more like a summary of all active components during the time of fail. I would assume that the Wi-Fi driver was faulty, nothing with AdGuard.

I love ChatGPT for analyzing something like that:

The specific reason for this panic can be deduced from several key points in the log:

  1. Process and Workqueue Involved: The process kworker/1:1 (kernel worker thread) was executing a workqueue item rproc_crash_handler_work. This suggests that the panic is related to a remote processor (rproc) crash handling routine, which is part of the Linux Remote Processor Framework used for managing remote processors in SoCs (System on Chips).
  2. Faulting Code Location: The Program Counter (PC) was at ath11k_core_ssr_notifier_cb+0xa4/0xb4 [ath11k] when the crash occurred, and the Link Register (LR) points to ath11k_hal_dump_srng_stats+0x248/0x2c0 [ath11k]. Both are within the ath11k driver, which is a driver for Qualcomm Atheros 802.11ax wireless LAN chips. This indicates that the panic occurred in the context of the ath11k WiFi driver, specifically during a Subsystem Restart (SSR) notification callback.
  3. Kernel Panic Trigger: The kernel panic message Kernel panic - not syncing: Fatal exception indicates a fatal exception occurred that the kernel could not handle, leading to the system halt. This is followed by messages about stopping CPUs, which is part of the kernel panic sequence to halt the system safely.
  4. Subsystem and Hardware Involved: The ath11k driver’s involvement points towards an issue with the WiFi hardware or the driver itself, possibly triggered by a hardware malfunction, a bug in the driver, or issues handling certain conditions like remote processor crashes.
  5. Additional Context: The log mentions hardware name “Generic DT based system”, suggesting this system uses Device Tree for hardware description, common in embedded systems and ARM architecture. The kernel version 4.4.60 indicates an older Linux kernel, which might have known bugs fixed in later versions.

In summary, the kernel panic was likely caused by an issue within the ath11k WiFi driver or related hardware, triggered while handling a remote processor crash. Resolving this issue could involve updating the ath11k driver to a newer version, applying patches if available, checking for hardware issues related to the WiFi module, or upgrading the Linux kernel to a more recent version that may contain fixes for known bugs.

1 Like

Wow that’s really interesting, what a great idea to use ChatGPT !!

I guess it’s back to the gl-inet team then for comment on the Wi-Fi driver.

Thanks