Network traffic capture problem on AR750

Technical Support for Routers
1

I want to use AR750 to capture network traffic between LAN ports and save to file by tcpdump. The problem is network speed is about 11MB/s，TFcard/USB disk writting speed is slower than 10MB/s. So tcpdump dropped many packets without writting to file.

The possible solution is either limiting LAN ports speed or speeding up TF/USB writing speed. Seems the latter one is impossible which limited by SoC chip speed. As CPU isn’t in the data path between LAN ports, so can’t use ethtool to limit bandwidth. Do you known how to limit LAN port speed in this case？

This seems not a rare use case, how do you guys transparently capture traffic between hosts with OpenWrt devices? Thanks in advance!

2

Pipe the output of the packet-capture program (over SSH, for example) to a “desktop” host that has sufficient write speed and capacity.

Another approach for traffic between LAN hosts is a managed switch, mirroring what you want to a specific monitor port, and capturing on a desktop host.

Edit: Looks like the AR750S (note the S) switch chip supports mirroring. I have not tried this on either the GL.iNet or OpenWrt firmware

root@OpenWrt:~# swconfig dev switch0 help
switch0: mdio-bus.0(Atheros AR8337), ports: 7 (cpu @ 0), vlans: 128
     --switch
	Attribute 1 (int): enable_vlan (Enable VLAN mode)
	Attribute 2 (none): reset_mibs (Reset all MIB counters)
	Attribute 3 (int): enable_mirror_rx (Enable mirroring of RX packets)
	Attribute 4 (int): enable_mirror_tx (Enable mirroring of TX packets)
	Attribute 5 (int): mirror_monitor_port (Mirror monitor port)
	Attribute 6 (int): mirror_source_port (Mirror source port)
[...]
3

install the kernel module for the iptables tee extension

opkg update
opkg install iptables-mod-tee

receiver=[ip of your client with wireshark]
iptables -A POSTROUTING -t mangle -o br-lan -j TEE --gateway ${receiver}
iptables -A PREROUTING -t mangle -i br-lan -j TEE --gateway ${receiver}

this will forward ALL packets to the receiver ip where you can filter them with wireshark.

alternatively you can filter for the source or target devices by adding options to the iptables commands above.

1 Like
4

Wonder if there’s a way to merge threads…

5

This is a good approach - in the other thread, I mentioned using either (a) ethernet hub, or (b) a smart switch with port mirroring, and like the mod-tee approach, using a third machine to capture the traffic direct with wireshark.

That’s why I was asking if the threads can be merged, as good ideas and contribs on both.

6

Thanks for your help！
Actually my setup is similar as your suggestion.
I already set port-mirroring. Instead of forwarding to PC, Packets forwarded to eth1@router CPU.
I don’t want to use another PC, I want to do capture and file writing directly on the router.
So seems unable to limit LAN ports bandwidth.

7

I would suggest against writing to flash storage as it has a limited lifetime and is slow. An external hard drive (or SSD) would be the best, a microSD or USB stick a reasonable second choice, if the write speeds are sufficient. Most USB drives aren’t good for much more than 5-10 MB/s (40-80 Mbps) write.

8

Yes，I know flash storage has limited lifetime due to no IO balance algorithm compared to SSD. But this isn’t an issue in my scenario, I only need to use it to capture data for only several hours.

My flash storage writing speed is more than 30M on PC, it isn’t the bottleneck. I suspect the bottleneck is AR750 router CPU’s SDRAM/DMA.
If can’t limit LAN port bandwidth, I plan to try data capture on GL b1300 router which has a better CPU and DDR3. Do you have performance data of USB storage writing on B1300?

Thanks again for all you guys :slight_smile:

9

You’re likely bound there on IO and Compute - hence the recommendation to offload the traffic.

Whether one does the mod-tee (clever as it is, but it does change the workload of the SoC) or offloaded SmartSwitch Port-Mirror and the old-school Ethernet Hub approach…

All generally work to some degree - be mindful of the platform capabilities.

10

Probably. if you are using tcpdump - you can pipe the output through gzip or similar to compress the output before writing to storage - this should give you more headroom.

11

Thanks for your help!
I read tcpdump doc,the save file only can be compressed after being closed. So this doesn’t work and almost triple file IO data.

12

Right - I meant more like this:

13

Thanks nopro404! This method is interesting，it likely working on a desktop class CPU/RAM. I tried on AR750，ufortunately still many packets dropped by tcpdump without writing to file.

14

Seems you have quite a lot of traffic. I find that my SD card write speed on my AR300M (same CPU as ar750) is about 1MB/s.under NTFS.

If you want to go deeper. The formatting of the SD card matters quite a bit. EXT4 is the fastest supported that I’ve tested – around 2MB’s.

You could pipe that gzipped tcpdump output through pipeview then to /dev/null to see what sort of write speed it wants to see if changing SD formats would help.

Oh well - good luck to you.

15

ntfs and ext4 are journal filesys which have extra overhead and not suitable for flash storage. I used exfat, it’s optimized for flash storage. It is about 10MB tested with this command: dd if=/dev/zero of=./out.bin bs=2M c=50

16

I assumed the SD slot in the GL-750 is much faster than USB storage on the AR-300M.

Good to know – thanks.

And w/r to journaling that’s true. I use mine as a travel router so I’m frequently leaving a hotel room and I the router is downloading to the storage, and the power to the room shuts off - a non journaled filesystem is no good for me :slight_smile: