GL-AR300M becomes slow/unresponsive, needs reboot

ZBeeblebrox · 2022-06-01T01:03:04.294Z

I was not able to ssh into the router while it was unresponsive, but it came back after a while by itself. free and demsg don’t show much, but I see the following from logread. Not sure it is correlated with the outage, I need to catch when it happens next time.

Tue May 31 19:01:52 2022 daemon.notice wpa_supplicant[2536]: wlan-sta: CTRL-EVENT-BEACON-LOSS
Tue May 31 19:01:53 2022 daemon.notice netifd: Network device 'wlan-sta' link is down
Tue May 31 19:01:53 2022 daemon.notice netifd: Interface 'wwan' has link connectivity loss
Tue May 31 19:01:53 2022 daemon.notice wpa_supplicant[2536]: wlan-sta: CTRL-EVENT-DISCONNECTED bssid=fa:92:bf:ca:49:04 reason=4 locally_generated=1
Tue May 31 19:01:53 2022 daemon.notice netifd: wwan (2652): udhcpc: received SIGTERM
Tue May 31 19:01:53 2022 daemon.notice netifd: Interface 'wwan' is now down
Tue May 31 19:01:53 2022 daemon.notice netifd: Interface 'wwan' is disabled
Tue May 31 19:01:53 2022 daemon.notice netifd: Interface 'wwan' is enabled
Tue May 31 19:01:53 2022 daemon.warn dnsmasq[2381]: no servers found in /tmp/resolv.conf.auto, will retry
Tue May 31 19:01:53 2022 daemon.notice wpa_supplicant[2536]: wlan-sta: SME: Trying to authenticate with fa:92:bf:ca:49:04 (SSID='#PUBLIC WIFI - Holiday Trails' freq=2462 MHz)
Tue May 31 19:01:54 2022 kern.info kernel: [ 1487.117066] wlan-sta: authenticate with fa:92:bf:ca:49:04
Tue May 31 19:01:54 2022 kern.info kernel: [ 1487.136406] wlan-sta: send auth to fa:92:bf:ca:49:04 (try 1/3)
Tue May 31 19:01:54 2022 kern.info kernel: [ 1487.142980] br-lan: port 2(wlan0) entered disabled state
Tue May 31 19:01:54 2022 daemon.notice netifd: Network device 'wlan0' link is down
Tue May 31 19:01:54 2022 kern.info kernel: [ 1487.161876] wlan-sta: authenticated
Tue May 31 19:01:54 2022 daemon.notice wpa_supplicant[2536]: wlan-sta: Trying to associate with fa:92:bf:ca:49:04 (SSID='#PUBLIC WIFI - Holiday Trails' freq=2462 MHz)
Tue May 31 19:01:54 2022 kern.info kernel: [ 1487.189918] wlan-sta: associate with fa:92:bf:ca:49:04 (try 1/3)
Tue May 31 19:01:54 2022 daemon.notice netifd: Network device 'wlan-sta' link is up
Tue May 31 19:01:54 2022 daemon.notice netifd: Interface 'wwan' has link connectivity
Tue May 31 19:01:54 2022 daemon.notice netifd: Interface 'wwan' is setting up now
Tue May 31 19:01:54 2022 kern.info kernel: [ 1487.203047] wlan-sta: RX AssocResp from fa:92:bf:ca:49:04 (capab=0x1421 status=0 aid=1)
Tue May 31 19:01:54 2022 kern.info kernel: [ 1487.211735] wlan-sta: associated
Tue May 31 19:01:54 2022 daemon.notice wpa_supplicant[2536]: wlan-sta: Associated with fa:92:bf:ca:49:04
Tue May 31 19:01:54 2022 daemon.notice wpa_supplicant[2536]: wlan-sta: CTRL-EVENT-CONNECTED - Connection to fa:92:bf:ca:49:04 completed [id=0 id_str=]
Tue May 31 19:01:54 2022 daemon.notice hostapd: wlan0: AP-STA-DISCONNECTED b2:a2:bb:3b:5d:51
Tue May 31 19:01:54 2022 daemon.notice hostapd: wlan0: AP-STA-DISCONNECTED 38:00:25:8a:3b:df
Tue May 31 19:01:54 2022 daemon.notice hostapd: wlan0: AP-STA-DISCONNECTED 64:6e:e0:57:dc:8f
Tue May 31 19:01:54 2022 daemon.notice hostapd: wlan0: AP-STA-DISCONNECTED 9c:95:61:34:14:d8
Tue May 31 19:01:54 2022 kern.info kernel: [ 1487.240763] br-lan: port 2(wlan0) entered blocking state
Tue May 31 19:01:54 2022 kern.info kernel: [ 1487.246289] br-lan: port 2(wlan0) entered forwarding state
Tue May 31 19:01:54 2022 daemon.notice netifd: Network device 'wlan0' link is up
Tue May 31 19:01:54 2022 daemon.notice wpa_supplicant[2536]: wlan-sta: CTRL-EVENT-SUBNET-STATUS-UPDATE status=0
Tue May 31 19:01:54 2022 daemon.notice netifd: wwan (32704): udhcpc: started, v1.30.1
Tue May 31 19:01:54 2022 daemon.notice netifd: wwan (32704): udhcpc: sending discover
Tue May 31 19:01:55 2022 daemon.info hostapd: wlan0: STA 38:00:25:8a:3b:df IEEE 802.11: authenticated
Tue May 31 19:01:55 2022 daemon.info hostapd: wlan0: STA 38:00:25:8a:3b:df IEEE 802.11: associated (aid 1)
Tue May 31 19:01:55 2022 daemon.notice hostapd: wlan0: AP-STA-CONNECTED 38:00:25:8a:3b:df
Tue May 31 19:01:55 2022 daemon.info hostapd: wlan0: STA 38:00:25:8a:3b:df RADIUS: starting accounting session CE652079B997E328
Tue May 31 19:01:55 2022 daemon.info hostapd: wlan0: STA 38:00:25:8a:3b:df WPA: pairwise key handshake completed (RSN)
Tue May 31 19:01:55 2022 daemon.info dnsmasq-dhcp[2381]: DHCPREQUEST(br-lan) 192.168.8.247 38:00:25:8a:3b:df
Tue May 31 19:01:55 2022 daemon.warn dnsmasq-dhcp[2381]: Ignoring domain inside.xero.com for DHCP host name XLW-5CD924FNX4
Tue May 31 19:01:55 2022 daemon.info dnsmasq-dhcp[2381]: DHCPACK(br-lan) 192.168.8.247 38:00:25:8a:3b:df XLW-5CD924FNX4
Tue May 31 19:01:55 2022 daemon.info hostapd: wlan0: STA 64:6e:e0:57:dc:8f IEEE 802.11: authenticated
Tue May 31 19:01:55 2022 daemon.info hostapd: wlan0: STA 64:6e:e0:57:dc:8f IEEE 802.11: associated (aid 2)
Tue May 31 19:01:55 2022 daemon.notice hostapd: wlan0: AP-STA-CONNECTED 64:6e:e0:57:dc:8f
Tue May 31 19:01:55 2022 daemon.info hostapd: wlan0: STA 64:6e:e0:57:dc:8f RADIUS: starting accounting session 21EE24790F2167DD
Tue May 31 19:01:55 2022 daemon.info hostapd: wlan0: STA 64:6e:e0:57:dc:8f WPA: pairwise key handshake completed (RSN)
Tue May 31 19:01:55 2022 daemon.info dnsmasq-dhcp[2381]: DHCPREQUEST(br-lan) 192.168.8.236 64:6e:e0:57:dc:8f
Tue May 31 19:01:55 2022 daemon.info dnsmasq-dhcp[2381]: DHCPACK(br-lan) 192.168.8.236 64:6e:e0:57:dc:8f DESKTOP-9C3R0V5
Tue May 31 19:01:56 2022 daemon.info hostapd: wlan0: STA b2:a2:bb:3b:5d:51 IEEE 802.11: authenticated
Tue May 31 19:01:56 2022 daemon.info hostapd: wlan0: STA b2:a2:bb:3b:5d:51 IEEE 802.11: associated (aid 3)
Tue May 31 19:01:56 2022 daemon.notice hostapd: wlan0: AP-STA-CONNECTED b2:a2:bb:3b:5d:51
Tue May 31 19:01:56 2022 daemon.info hostapd: wlan0: STA b2:a2:bb:3b:5d:51 RADIUS: starting accounting session 7E574D2C631E5060
Tue May 31 19:01:56 2022 daemon.info hostapd: wlan0: STA b2:a2:bb:3b:5d:51 WPA: pairwise key handshake completed (RSN)
Tue May 31 19:01:57 2022 daemon.info dnsmasq-dhcp[2381]: DHCPREQUEST(br-lan) 192.168.8.150 b2:a2:bb:3b:5d:51
Tue May 31 19:01:57 2022 daemon.info dnsmasq-dhcp[2381]: DHCPACK(br-lan) 192.168.8.150 b2:a2:bb:3b:5d:51
Tue May 31 19:01:57 2022 daemon.notice netifd: wwan (32704): udhcpc: sending discover
Tue May 31 19:01:57 2022 daemon.notice netifd: wwan (32704): udhcpc: sending select for 10.50.10.108
Tue May 31 19:01:57 2022 daemon.notice netifd: wwan (32704): udhcpc: lease of 10.50.10.108 obtained, lease time 432600
Tue May 31 19:01:57 2022 daemon.notice netifd: Interface 'wwan' is now up
Tue May 31 19:01:57 2022 daemon.info dnsmasq[2381]: reading /tmp/resolv.conf.auto
Tue May 31 19:01:57 2022 daemon.info dnsmasq[2381]: using local addresses only for domain test
Tue May 31 19:01:57 2022 daemon.info dnsmasq[2381]: using local addresses only for domain onion
Tue May 31 19:01:57 2022 daemon.info dnsmasq[2381]: using local addresses only for domain localhost
Tue May 31 19:01:57 2022 daemon.info dnsmasq[2381]: using local addresses only for domain local
Tue May 31 19:01:57 2022 daemon.info dnsmasq[2381]: using local addresses only for domain invalid
Tue May 31 19:01:57 2022 daemon.info dnsmasq[2381]: using local addresses only for domain bind
Tue May 31 19:01:57 2022 daemon.info dnsmasq[2381]: using local addresses only for domain lan
Tue May 31 19:01:57 2022 daemon.info dnsmasq[2381]: using nameserver 10.50.0.1#53
Tue May 31 19:01:59 2022 daemon.info hostapd: wlan0: STA 9c:95:61:34:14:d8 IEEE 802.11: authenticated
Tue May 31 19:01:59 2022 daemon.info hostapd: wlan0: STA 9c:95:61:34:14:d8 IEEE 802.11: associated (aid 4)
Tue May 31 19:01:59 2022 daemon.notice hostapd: wlan0: AP-STA-CONNECTED 9c:95:61:34:14:d8
Tue May 31 19:01:59 2022 daemon.info hostapd: wlan0: STA 9c:95:61:34:14:d8 RADIUS: starting accounting session 99DA11795BAE1F52
Tue May 31 19:01:59 2022 daemon.info hostapd: wlan0: STA 9c:95:61:34:14:d8 WPA: pairwise key handshake completed (RSN)
Tue May 31 19:02:00 2022 daemon.info dnsmasq-dhcp[2381]: DHCPREQUEST(br-lan) 192.168.8.237 9c:95:61:34:14:d8
Tue May 31 19:02:00 2022 daemon.info dnsmasq-dhcp[2381]: DHCPACK(br-lan) 192.168.8.237 9c:95:61:34:14:d8 43TCLRokuTV
Tue May 31 19:02:03 2022 user.info mwan3rtmon[2264]: Detect rtchange event.
Tue May 31 19:02:05 2022 user.notice firewall: Reloading firewall due to ifup of wwan (wlan-sta)

eric · 2022-06-01T02:45:40.230Z

This is only showing a few seconds of your log from 19:01:54 to 19:02:05. It would be real helpful to see the time just before it hung till after it was hung, and the time just before it comes back to when its fully operational. How long was your router not reachable? Next time can you also run the command:

uptime

As that will tell us if the router rebooted itself.

Just as an FYI: I’m just a user who owns several GL iNet routers including some AR300Ms, but I am not part of the GL iNet staff. Some of the GL iNet staff reads this forum but you may also want to open a support ticket with them via email.

alzhao · 2022-06-01T04:21:27.267Z

As I understand, you only set up repeater and then after some time, the router is not responsive.

Can you do this:

Before the router is not responsive (you have to make a guess), ssh to the router and get some log and other data.

dmesg
logread
ps
top

ZBeeblebrox · 2022-06-01T06:04:59.325Z

eric: Those are the first real logs after it rebooted, aside from a few of these sporadically

daemon.notice wpa_supplicant[2536]: wlan-sta: CTRL-EVENT-BEACON-LOSS
daemon.warn dnsmasq[2381]: possible DNS-rebind attack detected:

ZBeeblebrox · 2022-06-01T06:07:29.909Z

The most recent outage latest about 1 hour and 20 minutes, starting at about 04:30 and ending at about 05:50. I have attached the output of the commands from just after recovery (too large to include inline).

GL.iNet after recovery 2.zip (8.5 KB)

alzhao: I will try to get data from just before it is unresponsive, but it can go for hours without issue, so it is very hard. Maybe I will try to write a script to query it repeatedly.

alzhao · 2022-06-01T07:26:06.065Z

Can you turn off DNS rebind protection?

This is usually required when you connect to network with captive portal.

ZBeeblebrox · 2022-06-01T18:19:09.816Z

I have switched off DNS rebind protection. It failed again. I am running top and ps every 5 seconds. Here are the last results before it became unresponsive.

e[He[JMem: 52948K use
d, 70636K free, 192K shrd, 7416K buff, 20072K cached
CPU:   0% usr   9% sys   0% nic  81% idle   0% io   0% irq   9% sirq
Load average: 0.56 0.56 0.50 1/51 846
e[7m  PID  PPID USER     STAT   VSZ %VSZ %CPU COMMANDe[m
  846   844 root     R     1224   1%   9% top -n 1
 2599     1 root     S     5600   5%   0% /usr/bin/gltertf
 2720     1 root     S     5336   4%   0% /usr/sbin/lighttpd -f /etc/lighttpd/li
 2728  2720 root     S     5088   4%   0% /www/api
 2536     1 root     S     4592   4%   0% /usr/sbin/wpa_supplicant -B -s -P /var
 2344     1 root     S     4568   4%   0% /usr/bin/gl_health
 1674     1 root     S     4488   4%   0% /usr/bin/gl_crond
 2407     1 root     S     4440   4%   0% /usr/sbin/hostapd -s -P /var/run/wifi-
 1887     1 root     S     4016   3%   0% /usr/sbin/usbmuxd --systemd
29111     1 dnsmasq  S     2748   2%   0% /usr/sbin/dnsmasq -C /var/etc/dnsmasq.
 1524     1 root     S     1748   1%   0% /sbin/netifd
 2403     1 root     S     1656   1%   0% /bin/sh /usr/bin/gl_monitor
    1     0 root     S     1568   1%   0% /sbin/procd
 1618     1 root     S     1444   1%   0% /usr/sbin/odhcpd
 2264     1 root     S     1416   1%   0% /bin/sh /usr/sbin/mwan3rtmon
 2975     1 root     S     1328   1%   0% /bin/sh /usr/sbin/mwan3track wwan wlan
 1194     1 root     S     1284   1%   0% /sbin/logd -S 64
 2895     1 root     S<    1224   1%   0% /usr/sbin/ntpd -n -N -S /usr/sbin/ntpd
 2496  1524 root     S     1224   1%   0% udhcpc -p /var/run/udhcpc-wlan-sta.pid
  843  2403 root     S     1224   1%   0% sleep 3

  PID USER       VSZ STAT COMMAND
    1 root      1568 S    /sbin/procd
    2 root         0 SW   [kthreadd]
    4 root         0 IW<  [kworker/0:0H]
    6 root         0 IW<  [mm_percpu_wq]
    7 root         0 SW   [ksoftirqd/0]
   90 root         0 SW   [oom_reaper]
   91 root         0 IW<  [writeback]
   93 root         0 SW   [kcompactd0]
   94 root         0 IW<  [crypto]
   96 root         0 IW<  [kblockd]
  108 root         0 IW<  [watchdogd]
  129 root         0 SW   [kswapd0]
  193 root         0 SW   [spi0]
  313 root         0 IW<  [ipv6_addrconf]
  326 root         0 IW<  [kworker/0:1H]
  392 root         0 IW<  [uas]
  467 root         0 SWN  [jffs2_gcd_mtd5]
  549 root      1208 S    /sbin/ubusd
  567 root       920 S    /sbin/askfirst /usr/libexec/login.sh
  617 root      1024 S    /sbin/urngd
  783 root      1224 S    sleep 5
  842 root      1224 S    sleep 5
  843 root      1224 S    sleep 3
  844 root      1144 S    /usr/sbin/dropbear -F -P /var/run/dropbear.1.pid -p
  845 root      1224 R    ps
  931 root         0 IW<  [cfg80211]
 1194 root      1284 S    /sbin/logd -S 64
 1323 root      1076 S    /usr/sbin/dropbear -F -P /var/run/dropbear.1.pid -p
 1524 root      1748 S    /sbin/netifd
 1618 root      1444 S    /usr/sbin/odhcpd
 1674 root      4488 S    /usr/bin/gl_crond
 1887 root      4016 S    /usr/sbin/usbmuxd --systemd
 2264 root      1416 S    /bin/sh /usr/sbin/mwan3rtmon
 2344 root      4568 S    /usr/bin/gl_health
 2403 root      1656 S    /bin/sh /usr/bin/gl_monitor
 2407 root      4440 S    /usr/sbin/hostapd -s -P /var/run/wifi-phy0.pid -B /v
 2496 root      1224 S    udhcpc -p /var/run/udhcpc-wlan-sta.pid -s /lib/netif
 2536 root      4592 S    /usr/sbin/wpa_supplicant -B -s -P /var/run/wpa_suppl
 2599 root      5600 S    /usr/bin/gltertf
 2720 root      5336 S    /usr/sbin/lighttpd -f /etc/lighttpd/lighttpd.conf
 2728 root      5088 S    /www/api
 2895 root      1224 S<   /usr/sbin/ntpd -n -N -S /usr/sbin/ntpd-hotplug -p 0.
 2975 root      1328 S    /bin/sh /usr/sbin/mwan3track wwan wlan-sta online 10
 5308 root         0 IW   [kworker/u2:1]
11796 root         0 IW   [kworker/u2:0]
29111 dnsmasq   2748 S    /usr/sbin/dnsmasq -C /var/etc/dnsmasq.conf.cfg01411c
29664 root         0 IW   [kworker/u2:2]
32732 root         0 IW   [kworker/0:1]
32733 root         0 IW   [kworker/0:3]

eric · 2022-06-02T00:47:10.964Z

I have looked at the zip file and nothing is standing out to me. Maybe @alzhao will see something.

alzhao · 2022-06-02T01:47:47.757Z

Can you turn the “Real time data statistics” off? It is in Client section.

ZBeeblebrox · 2022-06-02T03:13:29.217Z

Real time data statistics was already off.

alzhao · 2022-06-02T08:29:13.777Z

The log does not show anything special.
I am trying to replicate this problem in my side.

Here is a check list. Pls help to verify that you are using the same.

AR300M with 3.212 firmware
no openvpn, no wireguard
no mac clone
Just repeater to an open wifi network
DNS rebind protection is turned off
Client data statistics is turned off
The web panel is not activated in your browser all the time. Turn it off when not use.

If the above all matches, can you please check if any of your clients may make trouble? You can disconnect one of them to see if things improve.

ZBeeblebrox · 2022-06-02T17:02:57.417Z

AR300M with 3.212 firmware
Confirmed
no openvpn, no wireguard
Confirmed
no mac clone
Confirmed
Just repeater to an open wifi network
Confirmed (wifi is captive portal)
DNS rebind protection is turned off
Confirmed
Client data statistics is turned off
Confirmed
The web panel is not activated in your browser all the time. Turn it off when not use.
I have been keeping it open. I will not do so any more.

I will try disconnecting some of my devices.

alzhao · 2022-06-03T04:36:50.804Z

ZBeeblebrox:

The web panel is not activated in your browser all the time. Turn it off when not use.
I have been keeping it open. I will not do so any more.

Maybe this is the problem. Pls try and let me know.

ZBeeblebrox · 2022-06-03T05:45:46.181Z

I have closed all of the web panels, however the problem occurred again.

alzhao · 2022-06-06T09:02:13.327Z

Sorry I have no more clue.

Walliot · 2023-03-14T14:25:31.390Z

I am experiencing the same issue. In my case, the router (AR300m16-ext) becomes unresponsive after about 3 days. It doesn’t matter if it has wireguard vpn turned on or not. My router has also stopped responding while in AP mode (not repeater).
One interesting thing is that the middle green LED light blinks sporadically and randomly when it is hung up and the amber wifi light turns solid. I noticed the middle green light begins to randomly blink a few hours before the router becomes unresponsive.
While the router is unresponsive, if I disconnect a client from the wifi signal, it is not able to reconnect until I restart the router by unplugging and plugging back in.

Walliot · 2023-03-14T14:28:10.335Z

I own multiple AR150-ext white mini routers and they have never given me any problems after months at a time working as repeater or AP mode. I might go back to using my older stable AR150-ext routers if this issue persists with the AR300m16-ext.
*note: all routers are updated and running latest firmware available.

alzhao · 2023-03-15T05:12:42.057Z

You mean in AP mode this problem happens? not in router mode, right?

AR300M an AR150 all use ath9k wifi so the behavior should be pretty the same. AR300M takes much more power than AR150 so need to use a better power adapter.

Walliot · 2023-03-28T02:28:07.696Z

Hi Alzhao, my 300M16-ext has done this in both AP mode and in Repeater mode. I bought a dual-boot NAND used AR300M-ext on ebay a few weeks ago. It’s been working flawlessly for over 9 days so far without hanging up. Im using the NAND flash at the moment. Same power supply I was using when the 300M16-ext became unresponsive. I think it must be a hardware issue with my specific 300M16-ext.

alzhao · 2023-03-28T09:18:38.698Z

Walliot:

Same power supply I was using when the 300M16-ext became unresponsive. I think it must be a hardware issue with my specific 300M16-ext.

If you use the same firmware version, AR300M16 should perform the same as your old AR300M. Otherwise there may be issue.