[SFT1200] WiFi crashes occasionally [dmesg included]

Every now and then, I have to restart the router because the WiFi just stops working. Sometimes, the WLAN goes down after a day. At better times, it could last a week. At worst, it can be just a few hours. The whole WLAN interface appears to be down; no SSID is being broadcasted. I logged into the router using ssh. What I found is that the dmesg shows a backtrace of what appears to be related to the WiFi driver. But it really isn't helpful since there are no symbols in the backtrace.

Here is one an example of a last few lines of a freshly-dumped dmesg:

[399846.626215] umac task_dump_registers-baseaddr=0x820fd000 backtrace=0x820fcc00 gpr=0x820fce00 cpu=0x820fcf00
[399846.636176] 
[399846.636176] $ 0:	ffffffff	ffffffff	ffffffff	ffffffff	
[399846.636176] $ 4:	ffffffff	ffffffff	ffffffff	ffffffff	
[399846.636176] $ 8:	ffffffff	ffffffff	ffffffff	ffffffff	
[399846.636176] $12:	ffffffff	ffffffff	ffffffff	ffffffff	
[399846.636176] $16:	ffffffff	ffffffff	ffffffff	ffffffff	
[399846.636176] $20:	ffffffff	ffffffff	ffffffff	ffffffff	
[399846.636176] $24:	ffffffff	ffffffff	
[399846.636176] $28:	ffffffff	ffffffff	ffffffff	ffffffff	
[399846.636338] epc: ffffffff
[399846.680180] ra: ffffffff
[399846.682800] Status: ffffffff
[399846.685764] Cause: ffffffff
[399846.688640] BadVA: ffffffff
[399846.691565] Prid: ffffffff
[399846.694360] Call Trace:
[399846.696889] [<0xffffffff>] <-
[399846.699993] [<0xffffffff>] <-
[399846.703051] [<0xffffffff>] <-
[399846.706100] [<0xffffffff>] <-
[399846.709213] [<0xffffffff>] <-
[399846.712270] [<0xffffffff>] <-
[399846.715320] [<0xffffffff>] <-
[399846.718370] [<0xffffffff>] <-
[399846.721481] [<0xffffffff>] <-
[399846.724539] [<0xffffffff>] <-
[399846.727590] [<0xffffffff>] <-
[399846.730760] [<0xffffffff>] <-
[399846.733829] [<0xffffffff>] <-
[399846.736879] [<0xffffffff>] <-
[399846.739993] [<0xffffffff>] <-
[399846.743050] [<0xffffffff>] <-
[399846.746100] [<0xffffffff>] <-
[399846.749252] [<0xffffffff>] <-
[399846.752313] [<0xffffffff>] <-
[399846.755362] [<0xffffffff>] <-
[399846.758411] [<0xffffffff>] <-
[399846.761529] [<0xffffffff>] <-
[399846.764586] [<0xffffffff>] <-
[399846.767636] [<0xffffffff>] <-
[399846.770747] [<0xffffffff>] <-
[399846.773805] [<0xffffffff>] <-
[399846.776855] [<0xffffffff>] <-
[399846.779993] [<0xffffffff>] <-
[399846.783056] [<0xffffffff>] <-
[399846.786106] [<0xffffffff>] <-
[399846.789284] [<0xffffffff>] <-
[399846.792356] [<0xffffffff>] <-
[399846.795406] [<0xffffffff>] <-
[399846.798456] [<0xffffffff>] <-
[399846.801580] [<0xffffffff>] <-
[399846.804638] [<0xffffffff>] <-
[399846.807688] [<0xffffffff>] <-
[399846.810799] [<0xffffffff>] <-
[399846.813857] [<0xffffffff>] <-
[399846.816907] [<0xffffffff>] <-
[399846.820009] [<0xffffffff>] <-
[399846.823066] [<0xffffffff>] <-
[399846.826115] [<0xffffffff>] <-
[399846.829288] [<0xffffffff>] <-
[399846.832359] [<0xffffffff>] <-
[399846.835409] [<0xffffffff>] <-
[399846.838458] [<0xffffffff>] <-
[399846.841612] [<0xffffffff>] <-
[399846.844671] [<0xffffffff>] <-
[399846.847721] [<0xffffffff>] <-
[399846.850836] [<0xffffffff>] <-
[399846.853894] [<0xffffffff>] <-
[399846.856944] [<0xffffffff>] <-
[399846.860060] [<0xffffffff>] <-
[399846.863119] [<0xffffffff>] <-
[399846.866169] [<0xffffffff>] <-
[399846.869282] [<0xffffffff>] <-
[399846.872340] [<0xffffffff>] <-
[399846.875390] [<0xffffffff>] <-
[399846.878440] [<0xffffffff>] <-
[399846.881600] [<0xffffffff>] <-
[399846.884660] [<0xffffffff>] <-
[399846.887710] [<0xffffffff>] <-
[399846.890827] [<0xffffffff>] <-
[399846.893886] [<0xffffffff>] <-
[399846.896936] [<0xffffffff>] <-
[399846.900042] [<0xffffffff>] <-
[399846.903099] [<0xffffffff>] <-
[399846.906150] [<0xffffffff>] <-
[399846.909260] [<0xffffffff>] <-
[399846.912319] [<0xffffffff>] <-
[399846.915369] [<0xffffffff>] <-
[399846.918419] [<0xffffffff>] <-
[399846.921529] [<0xffffffff>] <-
[399846.924587] [<0xffffffff>] <-
[399846.927636] [<0xffffffff>] <-
[399846.930748] [<0xffffffff>] <-
[399846.933809] [<0xffffffff>] <-
[399846.936861] [<0xffffffff>] <-
[399846.940014] [<0xffffffff>] <-
[399846.943083] [<0xffffffff>] <-
[399846.946133] [<0xffffffff>] <-
[399846.949266] [<0xffffffff>] <-
[399846.952324] [<0xffffffff>] <-
[399846.955374] [<0xffffffff>] <-
[399846.958424] [<0xffffffff>] <-
[399846.961536] [<0xffffffff>] <-
[399846.964594] [<0xffffffff>] <-
[399846.967644] [<0xffffffff>] <-
[399846.970756] [<0xffffffff>] <-
[399846.973813] [<0xffffffff>] <-
[399846.976863] [<0xffffffff>] <-
[399846.979966] [<0xffffffff>] <-
[399846.983023] [<0xffffffff>] <-
[399846.986073] [<0xffffffff>] <-
[399846.989184] [<0xffffffff>] <-
[399846.992242] [<0xffffffff>] <-
[399846.995292] [<0xffffffff>] <-
[399846.998342] [<0xffffffff>] <-
[399847.001451] [<0xffffffff>] <-
[399847.004510] [<0xffffffff>] <-
[399847.007559] [<0xffffffff>] <-
[399847.010706] [<0xffffffff>] <-
[399847.013766] [<0xffffffff>] <-
[399847.016816] [<0xffffffff>] <-
[399847.019921] [<0xffffffff>] <-
[399847.022978] [<0xffffffff>] <-
[399847.026154] [<0xffffffff>] <-
[399847.029362] [<0xffffffff>] <-
[399847.032431] [<0xffffffff>] <-
[399847.035483] [<0xffffffff>] <-
[399847.038534] [<0xffffffff>] <-
[399847.041711] [<0xffffffff>] <-
[399847.044779] [<0xffffffff>] <-
[399847.047829] [<0xffffffff>] <-
[399847.050948] [<0xffffffff>] <-
[399847.054006] [<0xffffffff>] <-
[399847.057055] [<0xffffffff>] <-
[399847.060160] [<0xffffffff>] <-
[399847.063218] [<0xffffffff>] <-
[399847.066268] [<0xffffffff>] <-
[399847.069381] [<0xffffffff>] <-
[399847.072439] [<0xffffffff>] <-
[399847.075489] [<0xffffffff>] <-
[399847.078539] [<0xffffffff>] <-
[399847.081651] [<0xffffffff>] <-
[399847.084708] [<0xffffffff>] <-
[399847.087759] [<0xffffffff>] <-
[399847.090871] [<0xffffffff>] <-
[399847.093928] [<0xffffffff>] <-
[399847.096980] [<0xffffffff>] <-
[399847.100130] [<0xffffffff>] <-
[399847.103197] [<0xffffffff>] <-
[399847.106247] [<0xffffffff>] <-
[399847.109377] [<0xffffffff>] <-
[399847.112437] [<0xffffffff>] <-
[399847.115487] [<0xffffffff>] <-
[399847.118537] [<0xffffffff>] <-
[399847.121649] [<0xffffffff>] <-
[399847.124707] [<0xffffffff>] <-
[399847.127757] [<0xffffffff>] <-
[399847.130867] [<0xffffffff>] <-
[399847.133924] [<0xffffffff>] <-
[399847.136978] [<0xffffffff>] <-
[399847.140167] [<0xffffffff>] <-
[399847.143237] [<0xffffffff>] <-
[399847.146287] [<0xffffffff>] <-
[399847.149420] [<0xffffffff>] <-
[399847.152478] [<0xffffffff>] <-
[399847.155528] [<0xffffffff>] <-
[399847.158577] [<0xffffffff>] <-
[399847.161711] [<0xffffffff>] <-
[399847.164773] [<0xffffffff>] <-
[399847.167823] [<0xffffffff>] <-
[399847.170938] [<0xffffffff>] <-
[399847.173995] [<0xffffffff>] <-
[399847.177045] [<0xffffffff>] <-
[399847.180146] [<0xffffffff>] <-
[399847.183202] [<0xffffffff>] <-
[399847.186252] [<0xffffffff>] <-
[399847.189362] [<0xffffffff>] <-
[399847.192419] [<0xffffffff>] <-
[399847.195470] [<0xffffffff>] <-
[399847.198519] [<0xffffffff>] <-
[399847.201629] [<0xffffffff>] <-
[399847.204687] [<0xffffffff>] <-
[399847.207737] [<0xffffffff>] <-
[399847.210845] [<0xffffffff>] <-
[399847.213903] [<0xffffffff>] <-
[399847.216953] [<0xffffffff>] <-
[399847.220054] [<0xffffffff>] <-
[399847.223111] [<0xffffffff>] <-
[399847.226161] [<0xffffffff>] <-
[399847.229272] [<0xffffffff>] <-
[399847.232329] [<0xffffffff>] <-
[399847.235379] [<0xffffffff>] <-
[399847.238429] [<0xffffffff>] <-
[399847.241595] [<0xffffffff>] <-
[399847.244668] [<0xffffffff>] <-
[399847.247718] [<0xffffffff>] <-
[399847.250842] [<0xffffffff>] <-
[399847.253901] [<0xffffffff>] <-
[399847.256951] [<0xffffffff>] <-
[399847.260153] [<0xffffffff>] <-
[399847.263225] [<0xffffffff>] <-
[399847.266275] [<0xffffffff>] <-
[399847.269434] [<0xffffffff>] <-
[399847.272494] [<0xffffffff>] <-
[399847.275544] [<0xffffffff>] <-
[399847.278593] [<0xffffffff>] <-
[399847.281713] [<0xffffffff>] <-
[399847.284771] [<0xffffffff>] <-
[399847.287821] [<0xffffffff>] <-
[399847.290943] [<0xffffffff>] <-
[399847.294002] [<0xffffffff>] <-
[399847.297052] [<0xffffffff>] <-
[399847.300154] [<0xffffffff>] <-
[399847.303217] [<0xffffffff>] <-
[399847.306269] [<0xffffffff>] <-
[399847.309433] [<0xffffffff>] <-
[399847.312501] [<0xffffffff>] <-
[399847.315551] [<0xffffffff>] <-
[399847.318601] [<0xffffffff>] <-
[399847.321722] [<0xffffffff>] <-
[399847.324780] [<0xffffffff>] <-
[399847.327830] [<0xffffffff>] <-
[399847.330950] [<0xffffffff>] <-
[399847.334011] [<0xffffffff>] <-
[399847.337061] [<0xffffffff>] <-
[399847.340166] [<0xffffffff>] <-
[399847.343229] [<0xffffffff>] <-
[399847.346281] [<0xffffffff>] <-
[399847.349441] [<0xffffffff>] <-
[399847.352510] [<0xffffffff>] <-
[399847.355559] [<0xffffffff>] <-
[399847.358609] [<0xffffffff>] <-
[399847.361730] [<0xffffffff>] <-
[399847.364788] [<0xffffffff>] <-
[399847.367838] [<0xffffffff>] <-
[399847.370960] [<0xffffffff>] <-
[399847.374021] [<0xffffffff>] <-
[399847.377070] [<0xffffffff>] <-
[399847.380175] [<0xffffffff>] <-
[399847.383234] [<0xffffffff>] <-
[399847.386284] [<0xffffffff>] <-
[399847.389400] [<0xffffffff>] <-
[399847.392461] [<0xffffffff>] <-
[399847.395511] [<0xffffffff>] <-
[399847.398561] [<0xffffffff>] <-
[399847.401682] [<0xffffffff>] <-
[399847.404741] [<0xffffffff>] <-
[399847.407791] [<0xffffffff>] <-
[399847.410908] [<0xffffffff>] <-
[399847.413967] [<0xffffffff>] <-
[399847.417017] [<0xffffffff>] <-
[399847.420151] [<0xffffffff>] <-
[399847.423219] [<0xffffffff>] <-
[399847.426268] [<0xffffffff>] <-
[399847.429393] [<0xffffffff>] <-
[399847.432453] [<0xffffffff>] <-
[399847.435503] [<0xffffffff>] <-
[399847.438552] [<0xffffffff>] <-
[399847.441668] [<0xffffffff>] <-
[399847.444731] [<0xffffffff>] <-
[399847.447783] [<0xffffffff>] <-
[399847.450948] [<0xffffffff>] <-
[399847.454017] [<0xffffffff>] <-
[399847.457067] [<0xffffffff>] <-
[399847.460178] [<0xffffffff>] <-
[399847.463396] [<0xffffffff>] <-
[399847.466458] [<0xffffffff>] <-
[399847.469654] [<0xffffffff>] <-
[399847.472807] [<0xffffffff>] <-
[399847.475940] [<0xffffffff>] <-
[399847.479237] [<0xffffffff>] <-
[399847.482427] [<0xffffffff>] <-
[399847.485595] [<0x3c1a8210>] <-
[399847.488766] [<0x275ad000>] <-
[399847.492116] [<0x3c1b2000>] <-
[399847.495291] [<0x35bd025>] <-
[399847.498381] [<0x241b01fc>] <-
[399847.501675] [<0x35bd023>] <-
[399847.504775] [<0xaf40fffc>] <-
[399847.507953] [<0x10000001>] <-
[399847.511308] heart_beat  restart over. count 1
[399847.517465] hb-fmac 17800000.wifi-hb wlan1: restart Del sta 49 (88:54:1f:4c:f8:18)
[399847.525371] remove STA 88:54:1f:4c:f8:18 from hash table success
[399847.531737] hb-fmac 17800000.wifi-hb wlan1: restart Del sta 51 (dc:fb:48:0d:85:81)
[399847.539604] remove STA dc:fb:48:0d:85:81 from hash table success
[399847.546111] lmac_glue_stop(1)
[399847.549458] stop_task, 1
[399847.552521] stop aresetn 2 por_resetn 1 
[399847.556892] tkn[2020917]  flags:001b  result:  0  cmd:5132-ME_TRAFFIC_IND_REQ       - reqcfm(5133-ME_TRAFFIC_IND_CFM)
[399847.567847] tkn[2020918]  flags:001f  result:  0  cmd:5132-ME_TRAFFIC_IND_REQ       - reqcfm(5133-ME_TRAFFIC_IND_CFM)
[399847.580921] platform shutdown : 1
[399847.588144] tx headroom : 224, tx amsdu headroom : 50, rx headroom : 182, rx pd offset 224
[399847.600104] lmac_glue_start(1)
[399847.603299] start_task, 1
[399847.606057] task entry_addr=0x2000000
[399847.610011] start aresetn 0 por_resetn 0 
[399847.614282] wait lmac init(1)>>>>>>>>>>>>>>>>>>>>>>>

Running ip a when this happens just hangs indefinitely. It's like the whole network stack just went down. The heart_beat restart over. count 1 line in the dmesg output seems to suggest that this is a heart-beat failure due to whatever is failing to pet the watchdog. It appears that the only way out of this is to do a reboot.

I'm running the Opal SFT1200 in AP mode. It connects to my pfSense main router via an ethernet backhaul. I'm also using the latest firmware, i.e. 4.3.19, at the time of this writing. Both 2.4GHz and 5GHz radios are enabled in my case.

image

Has anyone encountered this before? This is borderline annoying as it doesn't seem to be robust and needs baby-sitting every now and then. Yes. I know I could set-up a scheduled reboot. However, I would love to avoid that as I sometimes have long running sessions/operations in queue which depends on the continuous availability of the network; doing a scheduled reboot is disruptive.

Probably the issue is related to the wireless driver.

May I know what client devices the router connected?

To temporary avoid it, may can try to create a reboot schedule to everyday.

Does the device have this problem when it is in routing mode?

@bruce The connected devices are just a Google Pixel phone and a Linux PC.

I know I could set up a scheduled reboot but that is disruptive as I sometimes have a long running task on the Linux PC that uses the network and I prefer if that doesn't get interrupted by the scheduled reboot.

A complete solution will be ideal.

It also happens when it is in routing mode.

Hi,

Please PM me the complete syslog and configurations:

Router configuration: Luci -> System -> Backup / Flash Firmware -> Download backup
GL logread: GL GUI -> System -> Log -> Export the file

Thanks for getting back to me @bruce . I appreciate it.

Unfortunately, the web configuration interface (doesn't matter if it is GL or LuCI) is not accessible whenever this issue gets triggered. What I meant by this is that accessing the web configuration interface page via a web browser will just yield a never-ending wait for the page to load (which never comes). Therefore, I'm not able to do the following as mentioned in your previous reply:

Router configuration: Luci -> System -> Backup / Flash Firmware -> Download backup
GL logread: GL GUI -> System -> Log -> Export the file

Is there a way to do the above using the command line interface? I have no issues using ssh and I'm very comfortable in using the command line. Let me know the commands I should be executing to get the backup of the firmware as well as the system log which you need. I'll send them to you as soon as I get the instructions and re-trigger the crash.

Thank you. I look forward to your next reply.

Does the SSH available to access when the issue occurs?

When the issue occurs, execute in the SSH:

logread > /www/system.log
dmesg > /www/kernel.log
sysupgrade -b /www/configuration_backup.tar.gz

Then, enter in the address in the PC browser will download these 3 files, like:
/system.log
Example:
http://192.168.8.1/system.log
http://192.168.8.1/kernel.log
http://192.168.8.1/configuration_backup.tar.gz