GL-X750 reboot issues

pvgonzalez · 2022-10-01T08:38:50.142Z

Hello there !

Some months ago I decided to use GL-X750 with a SIM card to connect to the Internet and collect some data, I noticed that sometimes reboots from command line lead my device to a state where it does absolutely nothing, only the Power led is on and all my settings are unavailable, I can’t even SSH it.

I decided to schedule daily reboots and I confirmed that this issue is directly related to the reboot, as once the device restarts I have exactly the same problems:

All leds are off, besides de Power ON/OFF led
The device doesn’t connect to mobile network thus I can’t SSH it
None of the processes that are supposed to run after reboot works
The only solution (to date) is to unplug and plug again the device, it will work until de next reboot

I found a similar issue with another model GL-B1300 - Reboot Issue where the “solution” was to change the device. It does not seem to be an issue with a particular device but a hardware/firmware thing as I have other devices (same model) and all of them had the same behavior at lease once since I bough them.

I’m trying to gather more info about the behavior but have anyone had seen this before? Other models, same model.

If you have any guideline to find what is behind the issue, any info will be appreciated.

Theodopolus · 2022-10-01T09:29:01.587Z

Not sure what version of firmware you are running, but I was using 3.212 which ran seemingly well (only intermittently used when away in Motorhome).

Just updated to 3.215, early days though.

alzhao · 2022-10-01T09:39:19.845Z

First pls upgrade to firmware 3.215.

Do you have cable devices connected on the wan and lan port? If yes can you disconnect them and check only with WiFi.

pvgonzalez · 2022-10-04T09:13:44.691Z

Thank you for your answers. So I upgraded to 3.215 (I had 3.212) but the issue seems to be more random.

As before, I restart my x750 using a cron, an after reboot now I have 3 scenarios occurring randomly:

- The device reboots well all my config is still there
- The device reboots on factory settings (WiFi SSID, DCHP, Cellular, etc)
- The device comes to stale state as before (one led only, no ssh, no network)

For cases 2 and 3, the device comes back after unplugging it one or more times.

For the second question, WAN and LAN ports are disconnected. I can’t check WiFi in stale state as it doesn’t work, neither do WAN, LAN or Cellular.

I tried to check the logread after a state 3 but there’s no trace of what the problem might be, below you will see that there are no logs between 07:43 and 08:52. Reboot took place at 07:45 and I hard rebooted around 08:52

Tue Oct  4 07:43:34 2022 daemon.notice netifd: Network device 'wlan0' link is up
Tue Oct  4 07:43:34 2022 daemon.notice netifd: Network device 'wlan3' link is up
Tue Oct  4 07:43:34 2022 daemon.notice netifd: bridge 'br-guest' link is up
Tue Oct  4 07:43:34 2022 daemon.notice netifd: Interface 'guest' has link connectivity
Tue Oct  4 07:43:35 2022 daemon.info smsd[3461]: 2022-10-04 07:43:35,2, smsd: Smsd v3.1.21 started.
Tue Oct  4 07:43:35 2022 daemon.info smsd[3461]: 2022-10-04 07:43:35,2, smsd: Running as root:root (0:0).
Tue Oct  4 07:43:35 2022 daemon.info smsd[3461]: 2022-10-04 07:43:35,2, smsd: Running in terminal mode.
Tue Oct  4 07:43:36 2022 daemon.err usbmuxd[3518]: [07:43:36.510][3] usbmuxd v1.1.1 starting up
Tue Oct  4 07:43:36 2022 daemon.err usbmuxd[3518]: [07:43:36.535][3] Using libusb 1.0.22
Tue Oct  4 07:43:36 2022 daemon.err usbmuxd[3518]: [07:43:36.580][3] Initialization complete
Tue Oct  4 07:43:36 2022 daemon.err usbmuxd[3518]: [07:43:36.581][3] Enabled exit on SIGUSR1 if no devices are attached. Start a new instance with "--exit" to trigger.
Tue Oct  4 07:43:36 2022 daemon.notice procd: /etc/rc.d/S95done: killall: uqmi: no process killed
Tue Oct  4 07:43:38 2022 daemon.notice procd: /etc/rc.d/S96led: setting up led WAN
Tue Oct  4 07:43:38 2022 daemon.notice procd: /etc/rc.d/S96led: setting up led LTE
Tue Oct  4 07:43:38 2022 daemon.notice nodogsplash[2852]: Waiting for connections
Tue Oct  4 08:52:48 2022 user.info mwan3rtmon[3006]: Detect rtchange event.
Tue Oct  4 08:52:49 2022 user.info : gl_health start->
Tue Oct  4 08:52:50 2022 daemon.info dnsmasq[3293]: exiting on receipt of SIGTERM
Tue Oct  4 08:52:50 2022 daemon.info dnsmasq[4077]: started, version 2.80 cachesize 150
Tue Oct  4 08:52:50 2022 daemon.info dnsmasq[4077]: compile time options: IPv6 GNU-getopt no-DBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP conntrack ipset auth nettlehash DNSSEC no-ID loop-detect inotify dumpfile
Tue Oct  4 08:52:50 2022 daemon.warn dnsmasq[4077]: warning: interface wlan3 does not currently exist
Tue Oct  4 08:52:50 2022 daemon.warn dnsmasq[4077]: warning: interface wlan2 does not currently exist
Tue Oct  4 08:52:50 2022 daemon.warn dnsmasq[4077]: warning: interface wlan0 does not currently exist
Tue Oct  4 08:52:50 2022 daemon.warn dnsmasq[4077]: warning: interface wlan1 does not currently exist
Tue Oct  4 08:52:50 2022 daemon.info dnsmasq-dhcp[4077]: DHCP, IP range 20.0.0.100 -- 20.0.0.249, lease time 12h
Tue Oct  4 08:52:50 2022 daemon.info dnsmasq-dhcp[4077]: DHCP, IP range 10.0.0.100 -- 10.0.0.249, lease time 12h
Tue Oct  4 08:52:50 2022 daemon.info dnsmasq-dhcp[4077]: DHCP, IP range 10.0.0.128 -- 10.0.0.255, lease time 12h
Tue Oct  4 08:52:50 2022 daemon.info dnsmasq-dhcp[4077]: DHCP, IP range 20.0.0.128 -- 20.0.0.255, lease time 12h
Tue Oct  4 08:52:50 2022 daemon.info dnsmasq-dhcp[4077]: DHCP, IP range 10.0.0.2 -- 10.0.0.127, lease time 12h
Tue Oct  4 08:52:50 2022 daemon.info dnsmasq-dhcp[4077]: DHCP, IP range 10.0.0.128 -- 10.0.0.255, lease time 12h
Tue Oct  4 08:52:50 2022 daemon.info dnsmasq[4077]: using local addresses only for domain test
Tue Oct  4 08:52:50 2022 daemon.info dnsmasq[4077]: using local addresses only for domain onion
Tue Oct  4 08:52:50 2022 daemon.info dnsmasq[4077]: using local addresses only for domain localhost
Tue Oct  4 08:52:50 2022 daemon.info dnsmasq[4077]: using local addresses only for domain local
Tue Oct  4 08:52:50 2022 daemon.info dnsmasq[4077]: using local addresses only for domain invalid
Tue Oct  4 08:52:50 2022 daemon.info dnsmasq[4077]: using local addresses only for domain bind
Tue Oct  4 08:52:50 2022 daemon.info dnsmasq[4077]: using local addresses only for domain lan
Tue Oct  4 08:52:50 2022 daemon.info dnsmasq[4077]: using nameserver 8.8.8.8#53

alzhao · 2022-10-04T10:12:01.015Z

If the device boot with factory settings, it may be a problem that you installed some programs and the flash is full. If this is the case you can do a firmware reset and set up again.

If no led is on then it could be a power problem or hardware failure.

pvgonzalez · 2022-10-04T15:07:55.738Z

alzhao:

If no led is on then it could be a power pr

Normally I did the Extroot configuration, should it be enough ? What is weird is that once I unplug/plug the device it works as expected. For what is worth I just saw the device restart while it had factory settings and come back to my settings !

Here is the output of df -h and free -h:

root@GL-X750:~# df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/root                10.8M     10.8M         0 100% /rom
tmpfs                    60.3M    288.0K     60.1M   0% /tmp
/dev/sda1                14.5G    422.3M     13.4G   3% /overlay
overlayfs:/overlay       14.5G    422.3M     13.4G   3% /
tmpfs                   512.0K         0    512.0K   0% /dev
/dev/sda1                14.5G    422.3M     13.4G   3% /mnt/sda1
/dev/mtdblock6            3.1M      3.0M     88.0K  97% /rwm
root@GL-X750:~# free -h
              total        used        free      shared  buff/cache   available
Mem:         123584       63112       39028         160       21444       28188
Swap:        262140        2312      259828

I dont think its a power problem as Power led is on, and then again the device it works as expected if I unplug/plug it again.

pvgonzalez · 2022-10-07T10:16:21.916Z

So I compared two devices (router A is the x750 where I found the issue and router B is the new one), both reboot every 10 minutes and I found this:

Device A, having v3.215, sometime does not reboot properly and hard reboot is required
Device B, having v3.12, reboots properly every time

Could there be something wrong with device A: at hardware level, something with extroot, faulty SD card ?

Do you recommend to buy a probe to see at a lower level ?

pvgonzalez · 2023-02-16T10:15:53.818Z

Hi @alzhao
Short recap of the issue: I have many x750V2 and some of them do not reboot properly when restarting them from command line, and sometimes unplugging/pugging them do not work either, meaning I have to do it up to three times so the router can actually boot.

The device stays at this state forever

MicrosoftTeams-image (3)1807×1355 145 KB

I took your advice on this issue GL-B1300 - Reboot Issue, I opened the case, connected an uart adapter to get the output on the serial.

The weird thing is once I welded the adapter, the issue just dissapeared ! I soft rebooted the device every 5 minutes for 2 days and I had not a single problem… Of course, the serial output seems normal to me, I get this output at every reboot, what do you think ?

Blockquote
[12:10:33:437] [ 0.000000] Initrd not found or empty - disabling initrd␍␊
[12:10:33:437] [ 0.000000] Primary instruction cache 64kB, VIPT, 4-way, linesize 32 bytes.␍␊
[12:10:33:437] [ 0.000000] Primary data cache 32kB, 4-way, VIPT, cache aliases, linesize 32 bytes␍␊
[12:10:33:437] [ 0.000000] Zone ranges:␍␊
[12:10:33:437] [ 0.000000] Normal [mem 0x0000000000000000-0x0000000007ffffff]␍␊
[12:10:33:437] [ 0.000000] Movable zone start for each node␍␊
[12:10:33:437] [ 0.000000] Early memory node ranges␍␊
[12:10:33:463] [ 0.000000] node 0: [mem 0x0000000000000000-0x0000000007ffffff]␍␊
[12:10:33:463] [ 0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x0000000007ffffff]␍␊
[12:10:33:515] [ 0.000000] random: get_random_bytes called from start_kernel+0x98/0x494 with crng_init=0␍␊
[12:10:33:515] [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 32512␍␊
[12:10:33:515] [ 0.000000] Kernel command line: rootfstype=squashfs,jffs2 rootfstype=squashfs,jffs2␍␊
[12:10:33:515] [ 0.000000] PID hash table entries: 512 (order: -1, 2048 bytes)␍␊
[12:10:33:515] [ 0.000000] Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)␍␊
[12:10:33:515] [ 0.000000] Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)␍␊
[12:10:33:515] [ 0.000000] Writing ErrCtl register=00000000␍␊
[12:10:33:569] [ 0.000000] Readback ErrCtl register=00000000␍␊
[12:10:33:569] [ 0.000000] Memory: 122404K/131072K available (4427K kernel code, 156K rwdata, 1008K rodata, 1244K init, 218K bss, 8668K reserved, 0K cma-reserved)␍␊
[12:10:33:569] [ 0.000000] SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1␍␊
[12:10:33:569] [ 0.000000] NR_IRQS: 51␍␊
[12:10:33:569] [ 0.000000] CPU clock: 650.000 MHz␍␊
[12:10:33:569] [ 0.000000] clocksource: MIPS: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 5880801374 ns␍␊
[12:10:33:569] [ 0.000008] sched_clock: 32 bits at 325MHz, resolution 3ns, wraps every 6607641598ns␍␊
[12:10:33:639] [ 0.008255] Calibrating delay loop… 432.53 BogoMIPS (lpj=2162688)␍␊
[12:10:33:689] [ 0.074862] pid_max: default: 32768 minimum: 301␍␊
[12:10:33:689] [ 0.079934] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)␍␊
[12:10:33:689] [ 0.086910] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)␍␊
[12:10:33:689] [ 0.098707] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns␍␊
[12:10:33:689] [ 0.109127] futex hash table entries: 256 (order: -1, 3072 bytes)␍␊
[12:10:33:689] [ 0.115657] pinctrl core: initialized pinctrl subsystem␍␊
[12:10:33:729] [ 0.122382] NET: Registered protocol family 16␍␊
[12:10:33:729] [ 0.134012] PCI host bridge /ahb/pcie-controller@180c0000 ranges:␍␊
[12:10:33:729] [ 0.140444] MEM 0x0000000010000000…0x0000000013ffffff␍␊
[12:10:33:729] [ 0.145996] IO 0x0000000000000000…0x0000000000000000␍␊
[12:10:33:782] [ 0.168526] PCI host bridge to bus 0000:00␍␊
[12:10:33:782] [ 0.172861] pci_bus 0000:00: root bus resource [mem 0x10000000-0x13ffffff]␍␊
[12:10:33:782] [ 0.180162] pci_bus 0000:00: root bus resource [io 0x0000]␍␊
[12:10:33:782] [ 0.186031] pci_bus 0000:00: root bus resource [??? 0x00000000 flags 0x0]␍␊
[12:10:33:782] [ 0.193186] pci_bus 0000:00: No busn resource found for root bus, will use [bus 00-ff]␍␊
[12:10:33:782] [ 0.202012] pci 0000:00:00.0: BAR 0: assigned [mem 0x10000000-0x101fffff 64bit]␍␊
[12:10:33:782] [ 0.209773] pci 0000:00:00.0: BAR 6: assigned [mem 0x10200000-0x1020ffff pref]␍␊
[12:10:33:833] [ 0.220163] clocksource: Switched to clocksource MIPS␍␊
[12:10:33:833] [ 0.226527] NET: Registered protocol family 2␍␊
[12:10:33:833] [ 0.231383] IP idents hash table entries: 2048 (order: 2, 16384 bytes)␍␊
[12:10:33:833] [ 0.238936] TCP established hash table entries: 1024 (order: 0, 4096 bytes)␍␊
[12:10:33:833] [ 0.246350] TCP bind hash table entries: 1024 (order: 0, 4096 bytes)␍␊
[12:10:33:833] [ 0.253067] TCP: Hash tables configured (established 1024 bind 1024)␍␊
[12:10:33:833] [ 0.259918] UDP hash table entries: 256 (order: 0, 4096 bytes)␍␊
[12:10:33:898] [ 0.266132] UDP-Lite hash table entries: 256 (order: 0, 4096 bytes)␍␊
[12:10:33:898] [ 0.273028] NET: Registered protocol family 1␍␊
[12:10:33:898] [ 0.280716] Crashlog allocated RAM at address 0x3f00000␍␊
[12:10:33:898] [ 0.287551] workingset: timestamp_bits=30 max_order=15 bucket_order=0␍␊
[12:10:33:898] [ 0.299511] squashfs: version 4.0 (2009/01/31) Phillip Lougher␍␊
[12:10:33:898] [ 0.305701] jffs2: version 2.2 (NAND) (SUMMARY) (LZMA) (RTIME) (CMODE_PRIORITY) (c) 2001-2006 Red Hat, Inc.␍␊
[12:10:33:898] [ 0.326770] io scheduler noop registered␍␊
[12:10:33:943] [ 0.330958] io scheduler deadline registered (default)␍␊
[12:10:33:943] [ 0.338530] pinctrl-single 1804002c.pinmux: 576 pins at pa b804002c size 72␍␊
[12:10:33:943] [ 0.346581] gpio-export gpio-export: 3 gpio(s) exported␍␊
[12:10:33:943] [ 0.352879] Serial: 8250/16550 driver, 16 ports, IRQ sharing enabled␍␊
[12:10:33:943] [ 0.361975] console [ttyS0] disabled␍␊
[12:10:33:943] [ 0.365780] 18020000.uart: ttyS0 at MMIO 0x18020000 (irq = 9, base_baud = 1562500) is a 16550A␍

We use a lot of x750v2 so it was not long until I found two more devices with this behavior, they could be let plugged for hours and the router would not boot. This time I only opened the case, removed all the screws and fastened them again and the issue is gone.

It’s a long shot but I think the card may have a electronic/magnetic issue (whether induced or inherent) preventing the router to boot.

alzhao · 2023-02-17T09:32:38.647Z

First, pls keep the antenna connected while using.

Second, can you clarify that when this problem happens, what devices is connected to the router’s wan and lan? You can email us (support at glinet.biz) for this issue because it needs more and more private info.