GL-MT3000 procd memory leak triggered by heavy network traffic (gaming)

Router
1

Device: GL-MT3000 (Beryl AX)

Firmware: 4.8.1 (stable) Issue: procd memory leak during high network activity (online gaming)

When playing online games (Overwatch, 2 devices simultaneously), procd grows from ~2KB to 493MB within 2 hours, eventually exhausting all RAM and swap, causing OOM kills and network disconnection.

Evidence from memory monitoring (1-minute intervals):

Time Activity procd VSZ
11:13 Idle 2.3 KB
11:30 Gaming 28 MB
12:00 Gaming 118 MB
12:53 Gaming 283 MB
13:00 Stop gaming 308 MB (stable, not released)
17:30 Idle 5 hours 308 MB (never released)
18:30 Gaming again 380 MB
18:57 Gaming 479 MB
19:01 Swap exhausted 493 MB (OOM)

Key observations:

  1. Memory only grows during gaming (high UDP traffic), stable during idle

  2. Leaked memory is never released until reboot

  3. Disabled all VPN-related hotplug scripts (WireGuard, OpenVPN, Tailscale, Tor) - no improvement

  4. Network Acceleration disabled

  5. ZRam configured at 256MB with zstd - only delays the crash

Workaround: Daily scheduled reboot at 5 AM

Request: Could you check if this is a known issue with procd on MT7981/MTK SDK? Is it fixed in 4.8.3-op24?

2

Just to be sure adguardhome is not active?

I noticed aswell something odd on my Beryl 7, about 70% of memory usage is quite alot when not using any of the functions other than repeater.

But I never ran this device for a few days to see if there is a memory leak in their gl sdk.

1 Like
3

I have firmware version 4.8.3-op24 installed on my GL-MT3000; I’ve been using it since it was released and haven’t had any issues, even during long online gaming sessions.

image
image1085×678 47.2 KB

4

Hi

Sorry for the delayed reply.

Could you please:

  1. Export the device logs and send them to us via private message for analysis

  2. Based on your description, it seems that memory usage does not continue increasing after stopping the game. So, could you please follow the guide and share your device with us via GoodCloud when the memory usage has increased significantly but before an OOM occurs, so we can check it remotely?
    Technical Support via GoodCloud - GL.iNet Router Docs 4

    Kindly note to send us the MAC address and the router password via private message so we can access it.

How to export logs:

image
image1430×911 87 KB

How to send private messages:

image
image632×233 19.1 KB

7

Thank you! I just saw your reply. I will do taht when same problem happend again (honestly it already happened many time but let me try to catch the log next time :slight_smile:

1 Like
8

Thanks for your control experiment.:joy_cat:

1 Like
9

Nothing change. First time, adguardhome is active, it happened. Then turn off, it happened but later.

10

Interesting, I have had my Beryl 7 run for a small week but no crash but....

The memory is still a issue my other gl firmware routers don't seem to have it this high even when dpi is off and adgh.

So something gives me a red flag, I had 70% usage that is quite alot for basically factory settings and only vpn active.

@will.qiu do you know what requires so much ram is it the wifi allocation or a memory leak?

11

We confirmed with our development team that it is expected for the Beryl 7 to use around 70% of its memory after boot.

The main reason is the increased memory overhead of the MediaTek Wi-Fi driver.

3 Likes
12

I think it is normal.
I had 20% usage before long time gaming, it is very stable before we play a lot.

13

Hi,

Thank you for providing the logs.

Unfortunately, after checking them, we were unable to find any useful clues.

Currently, we can confirm the following:

  1. An OOM (Out Of Memory) event did occur, and nginx was repeatedly stopped to help release memory.
  2. However, nginx only consumed around 10.5 MB RSS, so it is unlikely to be the root cause.
  3. The OOM information in the logs is incomplete. The 56 visible processes only account for around 18.4 MB RSS + 4.6 MB swap, so we cannot currently identify which process consumed excessive memory and triggered the OOM.

Could you please help us with the following?

  1. Please run the following commands via SSH to store logs on the Flash memory. This way, if the issue occurs and the device needs to be rebooted, the logs will not be lost and can be exported and sent to us for analysis.
  1. If possible, please open an additional terminal window and run the following script in the background. This will allow us to monitor memory usage in real time and collect relevant information after the issue occurs for further analysis:
cat > mem_monitor.sh <<'EOF'
#!/bin/sh

INTERVAL="${1:-5}"

print_memory()
{
    awk '
        /^MemTotal:/     { total=$2 }
        /^MemFree:/      { free=$2 }
        /^MemAvailable:/ { available=$2 }
        /^Buffers:/      { buffers=$2 }
        /^Cached:/       { cached=$2 }
        /^SReclaimable:/ { reclaim=$2 }
        /^Shmem:/        { shmem=$2 }

        END {
            if (!available)
                available=free+buffers+cached+reclaim-shmem

            used=total-available
            percent=(total > 0) ? used*100/total : 0

            printf "Memory: %.1f/%.1f MiB used (%.1f%%), %.1f MiB available\n",
                   used/1024,
                   total/1024,
                   percent,
                   available/1024
        }
    ' /proc/meminfo
}

print_top_processes()
{
    printf "%-7s %-20s %12s %12s %12s %12s\n" \
        "PID" "PROGRAM" "VmSize" "VmRSS" "VmData" "VmSwap"

    for status in /proc/[0-9]*/status; do
        [ -r "$status" ] || continue

        pid="${status#/proc/}"
        pid="${pid%/status}"

        awk -v pid="$pid" '
            $1 == "Name:"   { name=$2 }
            $1 == "VmSize:" { vmsize=$2 }
            $1 == "VmRSS:"  { vmrss=$2 }
            $1 == "VmData:" { vmdata=$2 }
            $1 == "VmSwap:" { vmswap=$2 }

            END {
                vmsize += 0
                vmrss  += 0
                vmdata += 0
                vmswap += 0

                printf "%d\t%s\t%s\t%d\t%d\t%d\t%d\n",
                       vmrss, pid, name,
                       vmsize, vmrss, vmdata, vmswap
            }
        ' "$status"
    done |
        LC_ALL=C sort -nr |
        head -n 10 |
        awk -F '\t' '{
            printf "%-7s %-20s %9d kB %9d kB %9d kB %9d kB\n",
                   $2, $3, $4, $5, $6, $7
        }'
}

trap 'echo; echo "Memory monitor stopped."; exit 0' INT TERM

while true; do
    echo
    echo "===== $(date '+%Y-%m-%d %H:%M:%S') ====="

    print_memory
    echo
    print_top_processes

    echo "--------------------------------------------------------------------------"
    sleep "$INTERVAL"
done
EOF

chmod +x mem_monitor.sh
./mem_monitor.sh
1 Like