A quiet Sunday and the cows were behaving themselves so I debugged it. The RM520N-GL in the X3000 has a different PCI id than the kernel is expecting, so it ends up with the wrong mhi channel config by default. The following works for me and gets it working with the mainline driver:
diff --git a/drivers/bus/mhi/host/pci_generic.c b/drivers/bus/mhi/host/pci_generic.c
index 08844ee7..7043d366 100644
--- a/drivers/bus/mhi/host/pci_generic.c
+++ b/drivers/bus/mhi/host/pci_generic.c
@@ -620,6 +620,9 @@ static const struct pci_device_id mhi_pci_id_table[] = {
.driver_data = (kernel_ulong_t) &mhi_telit_fn980_hw_v1_info },
{ PCI_DEVICE(PCI_VENDOR_ID_QCOM, 0x0306),
.driver_data = (kernel_ulong_t) &mhi_qcom_sdx55_info },
+ /* RM520N-GL variant with Qualcomm vendor and subvendor ID */
+ { PCI_DEVICE_SUB(PCI_VENDOR_ID_QCOM, 0x0308, PCI_VENDOR_ID_QCOM, 0x5201),
+ .driver_data = (kernel_ulong_t) &mhi_quectel_rm5xx_info },
/* Telit FN990 */
{ PCI_DEVICE_SUB(PCI_VENDOR_ID_QCOM, 0x0308, 0x1c5d, 0x2010),
.driver_data = (kernel_ulong_t) &mhi_telit_fn990_info },
(You might need to apply by hand to an older kernel as the diff context may be wrong if other cards have been added between 6.6.x and 6.10.x. The only thing that really matters is that the PCI_DEVICE_SUB() entries for PCI_VENDOR_ID_QCOM, 0x0308 (like this one) need to come before the default PCI_DEVICE() entry for PCI_VENDOR_ID_QCOM, 0x0308 as they're the more specific matches.)
I haven't done much testing with it yet, but unmodified tinymbim brought it up with both IPv4 and IPv6 exactly as with the USB interface. Probably means umbim will be fine with it too.
[Edit] I'm using the PCIe mode for my internet access as I write this now. Speed is identical to USB, just over 100Mbps because I'm limited by 4G reception. No obvious packet loss but then my maximum download speed is unlikely to stress anything.
I saw regular warnings of the form sequence number glitch prev=38 curr=0 (for various values of prev, but curr is always zero) which I will probably suppress. These are discontinuities in a sequence field only intended for debugging purposes. Nothing relies on it and the USB cdc-ndm driver logs the same issue with debug severity instead of warning. We should probably do the same in mhi_wwan_mbim. I've included this in the update patch I pushed to GitHub - arachsys-hosts/gl-x3000: Build recipe for GL-X3000 router.