Beryl 7 (GL-MT3600BE) - TX speeds unstable

Routers
23

Status update. You can completely disregard my earlier findings. The issue was with the test client I was using on the VLAN tagged side. When adding a VLAN tag the NIC didn’t automatically adjust the MTU correctly so was discarding input frames. :roll_eyes:

Manually forcing a higher MTU to account for the additional VLAN overhead resolved the issue and I cant get the same speed results I was seeing before lowering the MTU on the Beryl.

Back to square one and can’t reproduce your issues on my unit so something else is at play in your case :frowning:

24

Oh dang it i thougth we are getting somewhere :smiley:

Anyhow here are the ethtool results - not from everything everything but endpoints so Cudy, Beryl and Linksys routers

Beryl: 1G

root@Gaming:~# ethtool eth1
Settings for eth1:
        Supported ports: [ ]
        Supported link modes:   10baseT/Full
                                100baseT/Full
                                1000baseT/Full
                                2500baseT/Full
        Supported pause frame use: Symmetric Receive-only
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Full
                                100baseT/Full
                                1000baseT/Full
                                2500baseT/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Link partner advertised link modes:  10baseT/Half 10baseT/Full
                                             100baseT/Half 100baseT/Full
                                             1000baseT/Full
        Link partner advertised pause frame use: No
        Link partner advertised auto-negotiation: Yes
        Link partner advertised FEC modes: Not reported
        Speed: 1000Mb/s
        Duplex: Full
        Port: MII
        PHYAD: 15
        Transceiver: external
        Auto-negotiation: on
        Current message level: 0x000000ff (255)
                               drv probe link timer ifdown ifup rx_err tx_err
        Link detected: yes
root@Gaming:~#
root@Gaming:~# ethtool -a eth1
Pause parameters for eth1:
Autonegotiate:  off
RX:             on
TX:             off

root@Gaming:~#
root@Gaming:~# ethtool --show-eee eth1
EEE Settings for eth1:
        EEE status: disabled
        Tx LPI: disabled
        Supported EEE link modes:  100baseT/Full
                                   1000baseT/Full
        Advertised EEE link modes:  Not reported
        Link partner advertised EEE link modes:  Not reported
root@Gaming:~#



 
-----------------------------------------------------
root@Gaming:~# ethtool -S eth1
NIC statistics:
     tx_bytes: 1721178
     tx_packets: 4108
     tx_skip: 0
     tx_collisions: 0
     rx_bytes: 1500171
     rx_packets: 4731
     rx_overflow: 0
     rx_fcs_errors: 0
     rx_short_errors: 0
     rx_long_errors: 0
     rx_checksum_errors: 0
     rx_flow_control_packets: 0
root@Gaming:~#

Beryl: 2.5G

Settings for eth1:
        Supported ports: [ ]
        Supported link modes:   10baseT/Full
                                100baseT/Full
                                1000baseT/Full
                                2500baseT/Full
        Supported pause frame use: Symmetric Receive-only
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Full
                                100baseT/Full
                                1000baseT/Full
                                2500baseT/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Link partner advertised link modes:  10baseT/Half 10baseT/Full
                                             100baseT/Half 100baseT/Full
                                             1000baseT/Full
                                             2500baseT/Full
        Link partner advertised pause frame use: No
        Link partner advertised auto-negotiation: Yes
        Link partner advertised FEC modes: Not reported
        Speed: 2500Mb/s
        Duplex: Full
        Port: MII
        PHYAD: 15
        Transceiver: external
        Auto-negotiation: on
        Current message level: 0x000000ff (255)
                               drv probe link timer ifdown ifup rx_err tx_err
        Link detected: yes
root@Gaming:~#
root@Gaming:~# ethtool -a eth1
Pause parameters for eth1:
Autonegotiate:  off
RX:             on
TX:             off

root@Gaming:~#
root@Gaming:~# ethtool --show-eee eth1
EEE Settings for eth1:
        EEE status: disabled
        Tx LPI: disabled
        Supported EEE link modes:  100baseT/Full
                                   1000baseT/Full
        Advertised EEE link modes:  Not reported
        Link partner advertised EEE link modes:  Not reported
root@Gaming:~#


NIC statistics:
     tx_bytes: 4924178
     tx_packets: 10627
     tx_skip: 0
     tx_collisions: 0
     rx_bytes: 3269268
     rx_packets: 10407
     rx_overflow: 0
     rx_fcs_errors: 0
     rx_short_errors: 0
     rx_long_errors: 0
     rx_checksum_errors: 0
     rx_flow_control_packets: 0
root@Gaming:~#

Cudy:

root@Basement:~# ethtool lan1
Settings for lan1:
        Supported ports: [ TP MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Supported pause frame use: Symmetric Receive-only
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Link partner advertised link modes:  10baseT/Half 10baseT/Full
                                             100baseT/Half 100baseT/Full
                                             1000baseT/Full
        Link partner advertised pause frame use: No
        Link partner advertised auto-negotiation: Yes
        Link partner advertised FEC modes: Not reported
        Speed: 1000Mb/s
        Duplex: Full
        Port: Twisted Pair
        PHYAD: 1
        Transceiver: external
        Auto-negotiation: on
        MDI-X: Unknown
        Supports Wake-on: d
        Wake-on: d
        Link detected: yes
root@Basement:~# ethtool -a lan1
Pause parameters for lan1:
Autonegotiate:  on
RX:             off
TX:             off
RX negotiated:  off
TX negotiated:  off
root@Basement:~# ethtool -S lan1
NIC statistics:
     tx_packets: 50
     tx_bytes: 3102
     rx_packets: 120
     rx_bytes: 16962
     TxDrop: 0
     TxCrcErr: 0
     TxUnicast: 2215
     TxMulticast: 78
     TxBroadcast: 12
     TxCollision: 0
     TxSingleCollision: 0
     TxMultipleCollision: 0
     TxDeferred: 0
     TxLateCollision: 0
     TxExcessiveCollistion: 0
     TxPause: 0
     TxPktSz64: 697
     TxPktSz65To127: 466
     TxPktSz128To255: 204
     TxPktSz256To511: 237
     TxPktSz512To1023: 190
     Tx1024ToMax: 511
     TxBytes: 1067356
     RxDrop: 0
     RxFiltering: 357
     RxUnicast: 2031
     RxMulticast: 248
     RxBroadcast: 229
     RxAlignErr: 0
     RxCrcErr: 0
     RxUnderSizeErr: 0
     RxFragErr: 0
     RxOverSzErr: 0
     RxJabberErr: 0
     RxPause: 0
     RxPktSz64: 847
     RxPktSz65To127: 672
     RxPktSz128To255: 281
     RxPktSz256To511: 154
     RxPktSz512To1023: 154
     RxPktSz1024ToMax: 400
     RxBytes: 895496
     RxCtrlDrop: 0
     RxIngressDrop: 0
     RxArlDrop: 0
root@Basement:~#

Linksys:

root@LivingRoom:~# ethtool lan1
Settings for lan1:
        Supported ports: [ TP MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Supported pause frame use: Symmetric Receive-only
        Supports auto-negotiation: Yes
        Supported FEC modes: Not reported
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: Yes
        Advertised FEC modes: Not reported
        Link partner advertised link modes:  10baseT/Half 10baseT/Full
                                             100baseT/Half 100baseT/Full
                                             1000baseT/Full
        Link partner advertised pause frame use: Symmetric Receive-only
        Link partner advertised auto-negotiation: Yes
        Link partner advertised FEC modes: Not reported
        Speed: 1000Mb/s
        Duplex: Full
        Port: Twisted Pair
        PHYAD: 2
        Transceiver: external
        Auto-negotiation: on
        MDI-X: on (auto)
        Link detected: yes
root@LivingRoom:~# ethtool -a lan1
Pause parameters for lan1:
Autonegotiate:  on
RX:             off
TX:             off
RX negotiated:  on
TX negotiated:  on

root@LivingRoom:~# ethtool --show-eee lan1
EEE Settings for lan1:
        EEE status: disabled
        Tx LPI: disabled
        Supported EEE link modes:  100baseT/Full
                                   1000baseT/Full
                                   10000baseT/Full
                                   1000baseKX/Full
                                   10000baseKR/Full
                                   20000baseMLD2/Full
                                   40000baseSR4/Full
                                   40000baseLR4/Full
                                   56000baseKR4/Full
                                   56000baseCR4/Full
                                   56000baseSR4/Full
                                   56000baseLR4/Full
                                   25000baseCR/Full
        Advertised EEE link modes:  1000baseT/Half
                                    10000baseT/Full
                                    1000baseKX/Full
                                    10000baseKX4/Full
                                    10000baseKR/Full
                                    10000baseR_FEC
                                    20000baseKR2/Full
                                    40000baseKR4/Full
                                    56000baseKR4/Full
                                    56000baseCR4/Full
                                    56000baseSR4/Full
                                    56000baseLR4/Full
        Link partner advertised EEE link modes:  10000baseKX4/Full
                                                 20000baseMLD2/Full
                                                 20000baseKR2/Full
                                                 40000baseCR4/Full
                                                 40000baseSR4/Full
root@LivingRoom:~# ethtool -S lan1
NIC statistics:
     rx_broadcast: 5
     rx_pause: 0
     rx_unicast: 50
     rx_multicast: 84
     rx_fcserr: 0
     rx_alignerr: 0
     rx_runt: 0
     rx_frag: 0
     rx_jmbfcserr: 0
     rx_jmbalignerr: 0
     rx_pkt64: 11
     rx_pkt65to127: 45
     rx_pkt128to255: 9
     rx_pkt256to511: 60
     rx_pkt512to1023: 10
     rx_pkt1024to1518: 4
     rx_pkt1519tox: 0
     rx_toolong: 0
     rx_pktgoodbyte: 43899
     rx_pktbadbyte: 0
     rx_overflow: 0
     tx_broadcast: 23
     tx_pause: 0
     tx_multicast: 578
     tx_underrun: 0
     tx_pkt64: 20
     tx_pkt65to127: 74
     tx_pkt128to255: 38
     tx_pkt256to511: 479
     tx_pkt512to1023: 35
     tx_pkt1024to1518: 2
     tx_pkt1519tox: 0
     tx_oversize: 0
     tx_pktbyte_h: 242600
     tx_collisions: 0
     tx_abortcol: 0
     tx_multicol: 0
     tx_singlecol: 0
     tx_exesdeffer: 0
     tx_deffer: 0
     tx_latecol: 0
     tx_unicast: 47
root@LivingRoom:~#

Yayyy that Linkysys supports everything energy efficient :smiley: didnt even know

25

Don’t see anything obviously wrong there.

Do you see the loss/slow speeds when running iperf3 from the Beryl itself? (you will probably want to do 4x 230M (-P 4) streams for the UDP rather than 1 stream when running from the router )

26

@m.v.petev have you used the Realtek NIC on OPNSense at 2.5G previously? I’m not really that familiar with the BSD kernel updates, but on the Linux side since a lot of the Realtek 2.5G stuff became more common there are tons and tons of driver fixes in newer Linux kernels for fixing issues running at the higher speeds, that do not manifest at 1G.

If you are on the default OPNSense kernel driver for Realtek it’s possible a lot of these fixes haven’t made it there maybe. It might be worth investigating if there is a way to use the Realtek vendor driver on OPNSense to see if this improves anything (If you are not already) as this will may have driver fixes that are not in the BSD kernel yet.

27

have you used the Realtek NIC on OPNSense at 2.5G previously?

It is true that OPNSense and Realtek was a bit of work to get functioning but in this case it is not the problem. In fact I actually switched from openwrt in the middle of the testing without any change in performance. I can go back to the original openwrt router installation i have it on a different SSD since it took me a while to get OPNSense to work :wink:

Do you see the loss/slow speeds when running iperf3 from the Beryl itself?

Yes at some point i did it but it was TCP and bidirectional (with the same results) - I will run another one tonight - will also reflash stock image from the recovery interface just to be sure that everything is as unedited as possible.

28

Yeah if recent OpenWRT 24 was on there then that should include all the recent Realtek fixes

Yes at some point i did it but it was TCP and bidirectional (with the same results) - I will run another one tonight - will also reflash stock image from the recovery interface just to be sure that everything is as unedited as possible.

Be good to try and identify which side of the Beryl is the problem. e.g. confirm its happening on egress from the Beryl itself or if we see it on ingress to the Beryl to narrow down where the losses are happening.

29

I have seen some similar behavior in my setup. It turned out to be a NIC driver issue on all PC’s that were using the 2.5gb nic from realtek. Once manually installed the latest driver the issue went away.

What is happening is for Windows and Linux the old realtek 8169 driver is being used by default (Windows driver from 2015! :joy: ). My NIC chip was realtek 8125.

I was able to speedtest (openspeedtest hosted in LAN, from WAN< - >LAN) through the Beryl7 at full speed 2.5gbs up and down, ofc with VPN disabled.

30

The ‘default’ driver in Linux is dependant on the kernel version. This will be set by the Linux distribution and Kernel will upgrade with different releases. With newer hardware like this it’s generally better to be on the HWE kernel option (if available for the specific distribution). You can install Realtek provided drivers but this is usually not recommended if you can just upgrade the kernel and should be used only if that’s not an option.

In Openwrt these Realtek fixes are back ported and included in the older kernel versions. E.g 6.6 in 24.10 will have Realtek updates from 6.12+

On windows however as you point out this does need specific drivers like with typical hardware where it’s best not to rely in the old ones provided by windows update.

it’s not impossible it’s the issue. But if it happened on openwrt and was a recent openwrt 24.10 or 25.12 RC these should include updated Realtek fixes.

31
  1. This was not the case, as i said. My Linux machine was on kernel 6.17 and was using realtek RTL8169 instead of RTL8125(both modules are available). I had to blacklist the RTL8169 so the system would use the RTL8125.
  2. Stated that the Beryl7 does not have an issue with 2.5gbs up or down

It’s up to you to keep looking elsewhere but if applicable in your test environment i would verify that the above mentioned driver issue is not present.

32

I’m pretty sure the 8125 module is the external Realtek vendor provided module and the 8169 is the in-kernel drivers which includes support for the 8125. Rather than 8169 being the ‘wrong’ driver. (Which is why you had to blacklist as they both support the same hardware)

Strange that you had issue on 6.17 still but in that scenario if your distro has no newer kernel then you do have no choice but to try then vendor driver which you have done.

As I said though this shouldn’t be done as standard unless absolutely required, as it was with 8168 vs 8169 modules in the past, the advice is to stick with the in-kernel driver unless you have no choice and switch back to in-kernel when a later kernel is available.

With OpenWRT a lot of 2.5G fixes have been back-ported even in 6.6 (Is apparently up to date with Realtek fixes from 6.19 that e.g. Ubuntu is missing still in 6.17) so I’d be surprised if using the out of kernel 8125 module made a difference there. Being a network focus distro they are generally ahead of Desktop for networking fixes, but still be worth a try of 8125 failing all other options. (or a later or snapshot build first depending on which version was tested previously)

OPNSense is probably a bit more difficult to get updated drivers being BSD though (Sounds like he already has the latest to get it working at all).

33

Hmm, I wonder if changing the negotiation rate fixes the realtek driver in windows.

Besides I re-terminated the cable that is exactly what I did after, and so on it was fixed.

Does that do a trick maybe?

You need to set it on windows, not on the Beryl 7.

34

Ah wow there is a lot of replies :smiley:

Blanched_Almond (I`m sorry apparently i cant mention more than 2 users) I have tested drivers before not the problem unless there is something like this one unicorn driver that only works.

Anyhow the reason drivers are for sure not the issue is that I get full speed in any configuration apart from when using the Beryl. And for some reason I can still get ok ish speeds when going via the Beryl to the NAS.

As for making sure just now tested 3 drivers - default windows, the asrock super duper gaming X670E one from their website as well as the current realtek 10.79.20 = also all resulting in the same speeds.

@oorweeg the OPNSense realtek is not the problem running on openwrt OpenWrt 24.10.2 and it is exactly the same as OPNSense

@xize11 Changing the negotiation speed does in fact fix the problem - still not a cable problem unless I have 10 different cables which have exactly the same issue. I tested by putting all the devices just in front of the main router/switch combo to exclude the house wiring and again the same results came up.

@oorweeg as for the tests you suggested running from the Beryl it self :

it is not possible to run the suggested multi-stream -u -P 4 -b 230M basically kills the device / session. I tested if it is a ssh getting overloaded scenario -u -P 4 -b 230M > /tmp/iperf.log 2>&1 & but it is not it just directly dies.

I can however run the same tests as before from within

Beryl7 → downstream PC (10.74)

iperf3 -c 192.168.10.74 -u -b 930M

[  5]   0.00-10.00  sec   988 MBytes   829 Mbits/sec  0.000 ms  0/715615 (0%)  sender
[  5]   0.00-10.00  sec   983 MBytes   825 Mbits/sec  0.021 ms  3779/715615 (0.53%)  receiver

iperf3 -c 192.168.10.74 -u -b 930M -R

[  5]   0.00-10.00  sec  1.08 GBytes   928 Mbits/sec  0.000 ms  0/800969 (0%)  sender
[  5]   0.00-10.00  sec  1.08 GBytes   925 Mbits/sec  0.008 ms  2269/800885 (0.28%)  receiver

Beryl7 → 2.5GLAN-port PC (10.10)

iperf3 -c 192.168.10.10 -u -b 930M

[  5]   0.00-10.00  sec  1014 MBytes   851 Mbits/sec  0.000 ms  0/734395 (0%)  sender
[  5]   0.00-10.00  sec  1014 MBytes   851 Mbits/sec  0.018 ms  0/734395 (0%)  receiver

iperf3 -c 192.168.10.10 -u -b 930M -R

[  5]   0.00-10.00  sec  1.08 GBytes   930 Mbits/sec  0.000 ms  0/802515 (0%)  sender
[  5]   0.00-10.00  sec  1.06 GBytes   913 Mbits/sec  0.004 ms  13844/801979 (1.7%)  receiver

Beryl7 → 2.5GLAN-port PC (10.10)

iperf3 -c 192.168.10.10 -u -b 2500M

[  5]   0.00-10.00  sec  1014 MBytes   851 Mbits/sec  0.000 ms  0/734642 (0%)  sender
[  5]   0.00-10.00  sec  1014 MBytes   851 Mbits/sec  0.027 ms  0/734642 (0%)  receiver

iperf3 -c 192.168.10.10 -u -b 2500M -R


[  5]   0.00-10.00  sec  1.79 GBytes  1.54 Gbits/sec  0.000 ms  0/1327018 (0%)  sender
[  5]   0.00-10.00  sec  1.46 GBytes  1.25 Gbits/sec  0.003 ms  242753/1325847 (18%)  receiver

And of course

TCP bidir (to show asymmetry clearly)

iperf3 -c 192.168.10.74 --bidir

[  5][TX-C]   0.00-10.00  sec   143 MBytes   120 Mbits/sec   31             sender
[  7][RX-C]   0.00-10.00  sec  1.09 GBytes   938 Mbits/sec                  receiver

iperf3 -c 192.168.10.10 --bidir

[  5][TX-C]   0.00-10.00  sec  1.65 GBytes  1.42 Gbits/sec    0             sender
[  7][RX-C]   0.00-10.00  sec  2.72 GBytes  2.34 Gbits/sec                  receiver

Btw the usecase which made me investigate is lan streaming from the PC to the TV (via the other PC) using Moonlight-Sunshine combo and i get solid 400M with no drops or anything without the Beryl.

I am starting to think given those results that it just can not handle the throughput.

35

Moonlight can be a difficult one, I learned that most tvs come with a Mediatek soc, but Mediatek isn't the strongest in encoding high bitrates, you may need a very low bitrate because the tv bottlenecks on its gpu, most tvs only have an 100mb port too.

To solve this issue on my own network I was forced to buy a nuc with a intel cpu/apu, Moonlight is alot better suited for these cpus to substain these bitrates, currently I have a Intel i5 1335U and around 130MB bitrate it can handle, sometimes I need to go a bit lower, a higher intel cpu will definitely help.

I know, but I was more asking this to see if the windows driver could be having this as a bug, did changing back permanently fix it, or did you went back to square one?

36

Ah if I plug in directly to the TV thats never going to work thst is why I have the other PC connected to it - so the pc is doing decoding and all TV is just a display. But anyhow again the whole system wolrs perfectly just not when the beryl is in it so its not a bottleneck.

As for the negotiation speed - it seems thst if I force 1G it also works as it should. 2.5G is where things get weird

37

What does ethtool show for this connected port on the Beryl 7?

Is EEE on?

I do know that some realtek chips on OpenWrt have had some issues with EEE, there was also alot of patch work going on I remember back then for the banana pi suffering a similar negotiation problem, the issue also went back once when they bumped the kernel, and later in the newer kernel it was finally fixed.

But this is MTK SDK, so maybe it is not patched, can you try this command:

ethtool --set-eee <the port the connection has issues on> eee off

Let me know if this works better for you, command is not persistent after a reboot it's undone, you could make it permanent by adding it to /etc/rc.local

38

@xize11 We checked EEE already, It’s supported but disabled and not running on the link when the Beryl is in place. See Beryl 7 (GL-MT3600BE) - TX speeds unstable - #24 by m.v.petev

root@Gaming:~# ethtool --show-eee eth1
EEE Settings for eth1:
EEE status: disabled
Tx LPI: disabled
Supported EEE link modes:  100baseT/Full
1000baseT/Full
Advertised EEE link modes:  Not reported
Link partner advertised EEE link modes:  Not reported
root@Gaming:~#
1 Like
39

@m.v.petev So this is odd.

The UDP tests work in this result, which is generally a more taxing than TCP, it seems to be something specific about trying to do bidirectional which tests both ways at the same time, the actual path in the UDP test seems to be better than the TCP result.
This makes it seem like the PC itself CPU is getting overloaded (By default without parallel stream iperf3 will use a single core). Can you try your bidirectional test but with multiple streams to spread the CPU load out across multiple cores?

perf3 -c 192.168.10.10 -P 4 --bidir

Needs to be iperf3 3.16 or newer to support multi-threading, otherwise iperf2.

iperf version potentially explains why the Beryl died as its only iperf 3.10 so will have probably overloaded a single CPU core and upset it.

40

Well i wouldn't call it working - it is better, true:

[  5]   0.00-10.00  sec  1.79 GBytes  1.54 Gbits/sec  0.000 ms  0/1327018 (0%)  sender
[  5]   0.00-10.00  sec  1.46 GBytes  1.25 Gbits/sec  0.003 ms  242753/1325847 (18%)  receiver

but still 18% as a receiver is quite bad to be honest it is in the margin error of the 32% when we had server PC to client PC if you double it. To me it looks like the PC is trying to push it but the beryl cant handle it

As for the CPU - nope that is actually one of the first things I checked even before posting - no bottleneck there. it hardly even reaches 13% on any single core.

I mean I am using AMD Ryzen 9 9950X on one side and Intel i7-11700k on the other this can not be bottlenecked by a 2.5g ethernet

iperf on the beryl is:

iperf 3.10.1 (cJSON 1.7.13)

However I can now 90% guarantee it is the if i do -c 192.168.1.1 -P 4 --bidir or -c 192.168.10.10 -P 4 --bidir so main router pushing or to the PC or vice versa it maybe does it … on a good day. Most of the times the beryl dies mid way (Keep in mind the beryl in this case id just a communication agent - nothing running on it) , just stops and reboots. When it doesn’t crash it always reboots right after.

[SUM][TX-C]   0.00-10.00  sec  327 MBytes   274 Mbits/sec  1800             sender
[SUM][TX-C]   0.00-10.00  sec  419 MBytes   352 Mbits/sec                   receiver

[SUM][RX-C]   0.00-10.00  sec  2.62 GBytes  2.25 Gbits/sec                  sender
[SUM][RX-C]   0.00-10.00  sec  2.62 GBytes  2.25 Gbits/sec                  receiver

And of course if i just directly plug in without the Beryl (Not via the Cudy as to keep the 2.5G)

[SUM][TX-C]   0.00-10.00  sec  2.73 GBytes  2.34 Gbits/sec                  sender
[SUM][TX-C]   0.00-10.00  sec  2.71 GBytes  2.33 Gbits/sec                  receiver

[SUM][RX-C]   0.00-10.00  sec  2.74 GBytes  2.36 Gbits/sec                  sender
[SUM][RX-C]   0.00-10.00  sec  2.73 GBytes  2.35 Gbits/sec                  receiver

Still makes me think I really want to love this device as it is pretty much exactly what i was looking for (and for a long time) but unfortunately it seems it can not quite reach the hype. I would give it one or two more attempts and then call it a day ant return it if this continues.

41

Does feel like some kind of driver issue or hardware fault at this point. I didn’t get a chance to try bidir specifically myself yet as I had been out yesterday and again today but will try and do so as soon as possible just to get another data point on this.

42

For what its worth, I still can’t replicate your performance/loss issues in my setup :frowning:

The below first test results

  • Bidirectional Test, 4 streams (–bidir -P 4)
  • VLAN tagged WAN interface (DHCP client), Firewall and NAT enabled
  • VLAN tagged LAN interface in a bridge (vlan10)
[SUM][TX-C]   0.00-10.00  sec  2.72 GBytes  2.34 Gbits/sec  217 sender
[SUM][TX-C]   0.00-10.00  sec  2.71 GBytes  2.33 Gbits/sec      receiver
[SUM][RX-C]   0.00-10.00  sec  2.73 GBytes  2.34 Gbits/sec  489 sender
[SUM][RX-C]   0.00-10.00  sec  2.71 GBytes  2.33 Gbits/sec      receiver

To spice things up, switched the WAN to PPPoE to add extra load, but still got the same result

  • Bidirectional Test, 4 streams (–bidir -P 4)
  • VLAN tagged WAN interface (PPPoE client), Firewall and NAT enabled
  • VLAN tagged LAN interface in a bridge (vlan10)
[SUM][TX-C]   0.00-10.00  sec  2.70 GBytes  2.32 Gbits/sec  261   sender
[SUM][TX-C]   0.00-10.00  sec  2.69 GBytes  2.31 Gbits/sec        receiver
[SUM][RX-C]   0.00-10.00  sec  2.71 GBytes  2.33 Gbits/sec  227   sender
[SUM][RX-C]   0.00-10.00  sec  2.70 GBytes  2.32 Gbits/sec        receiver

Test Setup

                       +---------------------------------+                       
                       |      iPerf3 Server/Client       |                       
                       |   (VRF Interface Seperation)    |                       
                       |       Ubuntu Noble 24.04        |                       
     ^                 +--------^--------------^---------+                  ^    
     |          Intel x553 SFP+ |              |Intel x553 SFP+             |    
     |                          |              |                            |    
     |                          |              |                            |    
     |                       10Gbps         10Gbps                          |    
     |                        DAC             DAC                           |    
     |                          |              |                            |    
     |                          |              |                            |    
     |         Intel x710 SFP+  |              |                            |    
     |        (PCIe Passthrough)|              |Intel x710 SFP+             |    
     |      +-------------------+---+      +---+-------------------+        |    
 VLAN 101   |   vyOS Virtual BNG    |      |   Linux VLAN-Aware    |    VLAN 10  
 WAN Side   |  (IPOE/PPPoE Server)  |      |        Bridge         |    LAN Side 
     |      +-------------------^---+      +---^-------------------+        |    
     |         Intel I226 RJ45  |              | Intel I226 RJ45            |    
     |       (PCIe Passthrough) |              |                            |    
     |                          |              |                            |    
     |                          |              |                            |    
     |                       2.5Gbps         2.5Gbps                        |    
     |                        Cat6            Cat6                          |    
     |                          |              |                            |    
     |                          |              |                            |    
     |                      WAN |              | LAN                        |    
     |                     +-----------------------+                        |    
     |                     |        Beryl 7        |                        |    
     v                     |                       |                        v    
                           +-----------------------+