How-To: Installing Vanilla OpenWRT on GL X3000

navalcadet · October 17, 2024, 12:55pm

@ChrisW sorry for just tagging you here, what's the easiest way to fix the kernel drivers on an existing vanilla OpenWRT installation? Is it possible to just recompile the kernel and install a new version like through a deb (I guess ipk) package? Or do I need to do a complete rebuild and build my own sysupgrade basically? I have no experience with OpenWRT but have compiled my own kernels for Ubuntu/Debian before.

I don't want to use those sketchy Quectel proprietary drivers, so I'm looking for the easiest way to throw in the vendor ID patch on my existing OpenWRT.

Much obliged, thanks!

navalcadet · October 18, 2024, 1:52pm

After some more digging, I may have figured out how to run the mainline drivers on OpenWRT.

You need to setup a build environment for OpenWRT as in the OP
You need to add the packages mentioned - through "make menuconfig"
Afterwards, you need to add the two kernel patches to your OpenWRT source
These patches need to be based on Linux 6.6.56 (6.10/6.11 diverges)

I'm building the images right now and will test whether this works. FWIW, I'm building the MBIM drivers as (patched) kernel modules instead of as compiled in, that might cause some issues but I'll have to see once I flash the image.

navalcadet · October 18, 2024, 6:26pm

Follow up: this indeed does the trick. Here's the correct patch for current master branch (kernel version 6.6.56):

--- a/drivers/bus/mhi/host/pci_generic.c
+++ b/drivers/bus/mhi/host/pci_generic.c
@@ -621,6 +621,9 @@
 		.driver_data = (kernel_ulong_t) &mhi_telit_fn980_hw_v1_info },
 	{ PCI_DEVICE(PCI_VENDOR_ID_QCOM, 0x0306),
 		.driver_data = (kernel_ulong_t) &mhi_qcom_sdx55_info },
+	/* RM520N-GL variant with Qualcomm vendor and subvendor ID */
+	{ PCI_DEVICE_SUB(PCI_VENDOR_ID_QCOM, 0x0308, PCI_VENDOR_ID_QCOM, 0x5201),
+		.driver_data = (kernel_ulong_t) &mhi_quectel_rm5xx_info },
 	/* Telit FN990 */
 	{ PCI_DEVICE_SUB(PCI_VENDOR_ID_QCOM, 0x0308, 0x1c5d, 0x2010),
 		.driver_data = (kernel_ulong_t) &mhi_telit_fn990_info },
--- a/drivers/net/wwan/mhi_wwan_mbim.c
+++ b/drivers/net/wwan/mhi_wwan_mbim.c
@@ -209,7 +209,7 @@
 	if (mbim->rx_seq + 1 != le16_to_cpu(nth16->wSequence) &&
 	    (mbim->rx_seq || le16_to_cpu(nth16->wSequence)) &&
 	    !(mbim->rx_seq == 0xffff && !le16_to_cpu(nth16->wSequence))) {
-		net_err_ratelimited("sequence number glitch prev=%d curr=%d\n",
+		net_dbg_ratelimited("sequence number glitch prev=%d curr=%d\n",
 				    mbim->rx_seq, le16_to_cpu(nth16->wSequence));
 	}
 	mbim->rx_seq = le16_to_cpu(nth16->wSequence);

Now I get a wwan0 netdev interface and wwan0mbim0 device. Time to figure out how to actually establish a cellular connection now. Modem manager doesn't show any devices.

Does anyone know whether I'm supposed to get a ttyUSB2 and /dev/cdc-wdm0 device? I see neither of these two right now. @ChrisW maybe you know? Sorry for pestering you

EDIT:

I am now successfully running Vanilla OpenWRT with Linux Mainline drivers. I previously attempted to change the kernel build to just compile in the WWAN drivers, which resulted in ModemManager not recognizing the modem as PCIe device, only as PPP (serial) modem. After changing this to kernel modules (through kmod) in the OpenWRT build config, I'm now successfully running MBIM PCIe mode.

Here's the relevant parts (+USB support) from my .config in OpenWRT:

CONFIG_PACKAGE_kmod-mhi-net=y
CONFIG_PACKAGE_kmod-mhi-wwan-ctrl=y
CONFIG_PACKAGE_kmod-mhi-wwan-mbim=y
CONFIG_PACKAGE_kmod-wwan=y
CONFIG_PACKAGE_kmod-usb-net-qmi-wwan=y
CONFIG_PACKAGE_kmod-usb-net=y
CONFIG_PACKAGE_kmod-usb-serial=y
CONFIG_PACKAGE_kmod-usb3=y
CONFIG_PACKAGE_kmod-usb-serial-wwan=y
CONFIG_PACKAGE_kmod-usb-serial-option=y

Interestingly, the USB controller on this device is XHCI, but the speed seems way below that. Benchmarks show that USB mode limits at ~300 Mbps download, while PCIe mode is pretty boring the SOC at even 1.2 Gbps.

Overall, have to say, I'm really happy with this OpenWRT setup now. I bought the device for its hardware and not the GL.Inet stock firmware and it feels great to run a minimal installation. Maybe on of these days, I will go a step further and get rid of OpenWRT too.

TL,DR: You don't need the Quectel drivers anymore with a bit of hackery. Once the driver fixes are in mainline, it should just work right away if you build the image with the right kmod packages.

ChrisW · October 19, 2024, 8:04am

Hi, sorry for the slow reply. I'm glad you've already worked out how to build a patched kernel/kernel modules on openwrt, because I have to admit I'm far from an openwrt expert and just build entire initramfs images from source when I'm testing backports to openwrt, rather than doing anything more sophisticated.

I wouldn't expect to see a /dev/cdc-wdm0, just a /dev/wwan0mbim0 - which can be used exactly the same way. (Nowadays you get the latter from a usb mbim device too.) However, I would still expect to see the four USB serial ports exactly as you get when it's configured as USB:

# ls -l /dev/ttyUSB*
crw------- 1 root root 188, 0 Jan  1  1970 /dev/ttyUSB0
crw------- 1 root root 188, 1 Jan  1  1970 /dev/ttyUSB1
crw------- 1 root root 188, 2 Jan  1  1970 /dev/ttyUSB2
crw------- 1 root root 188, 3 Jan  1  1970 /dev/ttyUSB3

I have

CONFIG_USB_SERIAL=y
CONFIG_USB_SERIAL_OPTION=y

compiled in, and you seem to have the relevant modules loaded, so I'm not quite sure why they're not showing up for you. Any clues in dmesg?

I believe the hardware usb link between the modem and soc is usb2 not usb3 for what it's worth, which would fit with the 300Mbps ceiling you found?

navalcadet · October 26, 2024, 1:18pm

You're right. ModemManager can drive the /dev/wwan0mbim0 just fine. For some strange reason, when compiling the patched drivers as part of the kernel config, it wouldn't recognize the modem even though the devices were there. Once I changed it to just standard kernel modules as part of OpenWRT, everything worked smoothly! It looks like the usb-serial-option kmod is all that's needed to get the ttyUSB devices, works fine without the qualcomm specific one.

It turned out to be an issue with the USB controller - all you need is the XHCI USB3.0 controller.

You might be right, I don't know. However, keep in mind that I think you need to poll the USB device for packets while you can use MSI-X on PCIe, which is way more efficient, so that would explain a thing or three too.

navalcadet · October 26, 2024, 1:22pm

So some feedback after a week of running OpenWRT stock on three X3000's with three different carriers... each operating in 5G NSA mode. Speeds are consistent, devices are mostly stable. For some reason, one of my X3000's likes to randomly reboot (maybe kernel panic?) under load. I will investigate some more later on, need to figure out how to get the panic logs (if that's what it is).

The X3000 with mainline drivers can easily push 1.2 Gbps+, after which there will be some jitter and load on the CPU. Maybe with some additional batching one could squeeze out a bit more.

I still need to get the pwm fan to work too, anyone tried that yet?

SpitzAX3000 · October 26, 2024, 9:11pm

For using MBIM, you can issue the commands (of put it in a boot script):

ifconfig wwan0 up
mbimcli -d /dev/wwan0mbim0 --query-subscriber-ready-status --no-close
mbimcli -d /dev/wwan0mbim0 --query-registration-state --no-open=3 --no-close
mbimcli -d /dev/wwan0mbim0 --attach-packet-service --no-open=4 --no-close
mbimcli -d /dev/wwan0mbim0 --connect=access-string='<YOUR ISP APN STRING>' --no-open=5 --no-close
udhcpc -q -f -i wwan0 -x lease:864000

You need to install kmod bbr and paste the following lines into /etc/sysctl.conf

net.ipv4.tcp_congestion_control = bbr
net.ipv4.tcp_adv_win_scale = -2
net.ipv4.tcp_notsent_lowat = 131072
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_mtu_probing = 1
net.ipv4.tcp_sack = 0
net.core.netdev_max_backlog = 1048576
net.core.rmem_max = 16777216
net.core.somaxconn = 65535
net.core.wmem_max = 16777216
net.ipv4.ip_local_port_range = 1024 65535
net.ipv4.tcp_fin_timeout = 5
net.ipv4.tcp_max_orphans = 1048576
net.ipv4.tcp_max_syn_backlog = 20480
net.ipv4.tcp_max_tw_buckets = 400000
net.ipv4.tcp_no_metrics_save = 1
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_syn_retries = 2
net.ipv4.tcp_tw_reuse = 1

Try these tweaks by pasting the following lines into /etc/rc.local

echo 524288 > /sys/module/nf_conntrack/parameters/hashsize
echo 'on' >  /sys/devices/platform/soc/11280000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0/power/control

moonshoes · October 31, 2024, 9:18am

May I ask to be pointed in the direction of how to band lock, view signal strength, etc on Vanilla OWRT please?
From what I gather above it will have to be done via command line but I have not found much information to help me out.

admon · October 31, 2024, 1:00pm

Should be possible using the AT commands. Afaik the GL GUI just calls AT commands as well.
See [Summary] AT commands

moonshoes · October 31, 2024, 1:17pm

I hate too needy but I am on the rookie side of things. I have not found where or how to enter AT commands in the Vanilla. I am likely just trying to issue them wrong through SSH. Firm documentation has not been easy to come by aside from the initial guide.

Any guidance would be appreciated.

admon · October 31, 2024, 1:21pm

It won't be fun - but that's what you get if you choose the vanilla path:
https://openwrt.org/docs/guide-user/network/wan/wwan/at_commands

kkvpn · November 6, 2024, 11:22pm

hi, guys i am thinking if i should change to openwrt vanilla, is there a good reason to do that? what are the cons and pros.

does it increase performance, speed?

nialv7 · November 8, 2024, 9:18am

Hi, just want to ask does the mainline drivers now have performance comparable to quectel's pcie_mhi? I gather if I use mainline drivers I won't need to run quectel-CM manually?

Thanks.

nialv7 · November 8, 2024, 1:27pm

I am able to bring the interface up using the "MBIM Cellular" protocol in luci. The mbim netifd script makes the assumption that the mbim interface is connected via usb, after I hacked that out it works.

         devname="$(basename "$device")"
-        devpath="$(readlink -f /sys/class/usbmisc/$devname/device/)"
+        devpath="$(readlink -f /sys/class/wwan/$devname/device/)"
         ifname="$( ls "$devpath"/net )"

Edit: never mind, spoke too soon. it disconnected itself after a while, don't know why.
Edit 2: I think the echo 'on' > power/control trick might have fixed it. Wish openwrt can get 6.10 soon...

nialv7 · November 9, 2024, 6:35am

Curiously, speedtest.net upload test seems to crash my modem completely. Afterwards it became irresponsive, even /dev/ttyUSB2 was stuck. I had to reset the modem to get it back, and (after i initiated reset) it gave me a ton of

[17733.057289] pcieport 0000:00:00.0: AER: Uncorrected (Non-Fatal) error message received from 0000:00:00.0
[17733.066805] pcieport 0000:00:00.0: PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, (Receiver ID)
[17733.077857] pcieport 0000:00:00.0:   device [14c3:1f32] error status/mask=00000020/00400000
[17733.086213] pcieport 0000:00:00.0:    [ 5] SDES                   (First)
[17733.092995] pcieport 0000:00:00.0: AER: broadcast error_detected message
[17733.099698] mhi-pci-generic 0000:01:00.0: PCI error detected, state = 1
[17733.106373] pcieport 0000:00:00.0: AER: broadcast resume message
[17733.112374] mhi-pci-generic 0000:01:00.0: PCI slot reset done
[17733.118254] pcieport 0000:00:00.0: AER: device recovery successful

But I think uploading files works just fine, even large ones. It's only speedtest.net that has this problem.

laroXX · November 20, 2024, 8:26pm

Hi @SpitzAX3000, , thanks for your guide. I've successfully updated my X3000 to OpenWrt SNAPSHOT (r28129-e6d483cd52).

In my case, first, I had to flash the default SYSUPGRADE IMAGE and then use apk add to install the package you suggested.

I have a question regarding the Quectel driver.

Will this process replace my "legacy" QMI Cellular/wwan0?
Do I need to run the command quectel-CM -4 -n 1 -s <apn> & every time to establish an LTE/5G connection?

Thanks!

Thx!

kkvpn · November 20, 2024, 11:19pm

Hi, why would you upgrade to openwrt does it bring any advantages with it? Just curious.

laroXX · November 21, 2024, 8:19am

In my case, the original firmware has some bugs:

The Gl.iNET interface is terrible, sometimes the LTE/5G connection page is not displayed

when I try to add another wifi, different from the default and guest ones, at the reboot it does not start and I have to reset the wifi to make it start

currently my carrier resets the connection every 4 hours, and I have to do this WORKAROUND, to restore the connection.
With this new version of Modemmanager this problem has disappeared. But this version of Modemmanager is for the new versions of OpenWrt

So in general I would prefer to stay on vanilla OpenWRT to have a cleaner experience and HOPEFULLY with fewer bugs.

bye!

nialv7 · November 28, 2024, 5:59pm

I've been running upstream mbim driver for a while. On average I lose connection about once every two days, and each time I have to reset the modem to recover connection. So I would recommend using the quectel drivers. Would be nice if someone can write a script to integrate it with netifd, maybe I can steal the script from the GL.iNET firmware

djsisson · December 20, 2024, 3:44pm

I've managed to get the pci drivers working using the patches above, as well as add the kmod bbr and alter /etc/sysctl.conf

however, under any kind of load the modem just stops responding, its still flashing but it stops working, only a restart on interfaces fixes it

i dont know anywhere near enough to know what i'm missing

any help would be much appreciated

thanks