Converting eth1 into second WAN port but Ping from a specific interface isn't working

souravndp · January 26, 2019, 1:05pm

Little bit background

I am using GLinet AR-150. It has 3 physical hardware interfaces (?): eth0 (single WAN port), eth1 (single LAN port), and wlan0 (WIFI). eth1 and wlan0 are bridged together (br-lan).

I recently got 2 connections from 2 different ISPs and thought about removing eth1 from the bridge and use it as a 2nd WAN port.

Both of my ISPs provided static IPs for router configuration.

ISP-1 (172.22.36.151/25) (Planned to connect it to WAN-eth0)
ISP-2 (172.30.138.151/24) (Planned to connect it to WAN2-eth1)

I used the following network config `/etc/config/network`

config interface 'loopback'
    option ifname 'lo'
    option proto 'static'
    option ipaddr '127.0.0.1'
    option netmask '255.0.0.0'

config globals 'globals'
    option ula_prefix 'fd22:bbe3:e6d8::/48'

config interface 'wan'
    option ifname 'eth0'
    option proto 'static'
    option ipaddr '172.22.36.151'
    option netmask '255.255.255.128'
    option gateway '172.22.36.129'
    option dns '8.8.8.8'
    option metric '10'

config interface 'wan2'
    option proto 'static'
    option ifname 'eth1'
    option ipaddr '172.30.138.151'
    option netmask '255.255.255.0'
    option gateway '172.30.138.1'
    option dns '8.8.8.8'
    option macaddr 'E4:95:6E:42:97:33'
    option metric '20'

config interface 'wan6'
    option ifname 'eth0'
    option proto 'dhcpv6'

config interface 'lan'
#REMOVED THE eth1 wlan0 BRIDGE. I WILL USE THIS LOGICAL INTERFACE `lan` IN /ETC/CONFIG/WIRELESS
    option proto 'static'
    option netmask '255.255.255.0'
    option ip6assign '60'
    option hostname 'GL-AR150-732'
    option ipaddr '192.168.8.1'

The outputs of `ip a show` and `ip route show` are attached at the end of this post.

Now, I connected the ISP cables to the respective pre-planned ports.

Now, I am specifying the interface to use for ping using:

ping -I eth0 8.8.8.8
ping -I eth1 8.8.8.8

Expected outcome

In the above commands, the ping should be sent via eth0 and eth1 respectively. The expected outcome is also mentioned in the mwan3 documentation.

The problem I am facing

Both the ping commands are being sent from eth1

Most probably because eth1 gateway has a lower metric.

If I use lower metric for eth0 gateway, then all pings are being sent from eth0 regardless of what interface I specify.

This behavior is similar to IP aliasing on a single interface. But I am using 2 completely different interfaces and if I specify the interface to use, then the default gateway metric should not matter. (see the expected outcome screenshot, they have 2 gateway with different metrics) The ping should be sent from my specified interface.

Please help.

Output of “ip route show”

default via 172.30.138.1 dev eth1 src 172.30.138.151 metric 10 
default via 172.22.36.129 dev eth0 src 172.22.36.151 metric 20 
172.22.36.128/25 dev eth0 proto static scope link metric 10 
172.30.138.0/24 dev eth1 proto static scope link metric 20 
192.168.8.0/24 dev wlan0 proto kernel scope link src 192.168.8.1

Output of “ip a show”

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
   valid_lft forever preferred_lft forever
inet6 ::1/128 scope host 
   valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether e4:95:6e:42:97:32 brd ff:ff:ff:ff:ff:ff
inet 172.22.36.151/25 brd 172.22.36.255 scope global eth0
   valid_lft forever preferred_lft forever
inet6 fe80::e695:6eff:fe42:9732/64 scope link 
   valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether e4:95:6e:42:97:33 brd ff:ff:ff:ff:ff:ff
inet 172.30.138.151/24 brd 172.30.138.255 scope global eth1
   valid_lft forever preferred_lft forever
inet6 fe80::e695:6eff:fe42:9733/64 scope link dadfailed tentative 
   valid_lft forever preferred_lft forever
4: teql0: <NOARP> mtu 1500 qdisc noop state DOWN group default qlen 100
link/void 
9: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether e4:95:6e:42:97:32 brd ff:ff:ff:ff:ff:ff
inet 192.168.8.1/24 brd 192.168.8.255 scope global wlan0
   valid_lft forever preferred_lft forever
inet6 fd22:bbe3:e6d8::1/60 scope global noprefixroute 
   valid_lft forever preferred_lft forever
inet6 fe80::e695:6eff:fe42:9732/64 scope link 
   valid_lft forever preferred_lft forever

sfx2000 · January 27, 2019, 12:23am

Hmm…

eth0 and eth1 as WAN, so one has to create a virtual interface as the bonded WAN to LAN facing clients keeping the WLAN interface as the bridged…

OpenWRT could do this, but it’ll likely break the GL-iNet WebGUI in a big way as you also need to get deep into iptables here

kyson-lok · January 28, 2019, 3:14am

If you use multiple WAN, you have to configure mwan3 correctly.

alzhao · January 31, 2019, 2:23am

If you set up the correct metric and mwan3 should work.

But You may still need to set up your firewall. Can you pls post your firewall? wan2 interface should be added to wan zone.

souravndp · January 31, 2019, 1:39pm

It was actually not that hard, My above configuration is completely okay. I did a little mistake while issuing the command.

I wish both of you and the whole Gl-inet team a joyous year of the pig. I know that mwan3 can automate the process but I just wanted to write a simple script myself.

I actually used the interface IP instead of the interface name while using ping -I 172.30.138.151 8.8.8.8. On windows, we need to use the interface IP and I did the same on Linux. But later I found out that I need to specify it as ping -I eth1 8.8.8.8 Thus, the above network configuration is completely okay and everything is working as expected as can be verified from the below picture (left ISP1 right ISP2).

I did face some challenges while trying to reach my 2nd ISP IP from the internet. I specified both ISP gateway as default route in the routing table but ISP1 had a lower metric. So, even if the ping echo request came on eth1, the echo reply was sent out from eth0. On windows, it is not a problem because the response is sent out automatically from the original interface where it is received. But here I resolved it by deploying policy-based routing using ip rule

echo "1 isp2" >> /etc/iproute2/rt_tables
ip route add default via 172.30.138.1 dev eth1 table isp2
ip rule add from 172.30.138.151/32 table isp2

Now the configuration is complete. Thus if anyone wants to use a similar setup on AR150, they can just use the network configuration posted on my original question. It is valid.

I was also a little bit confused from the OpenWRT wiki page of AR150.

|eth0.1, vlan1|WAN port|
| — | — | — | — |
|eth0.0, vlan0|LAN port|

I thought WAN and LAN ports are not 2 VLANS on a 5 port switch but they are connected to 2 different pysical interfaces. From the ar9331 datasheet:

The AR9331 supports four LAN ports and one WAN port with integrated PHY. … The four LAN ports connect to the CPU through a GMII interface … The WAN port can be configured to connect to the CPU using a dedicated MII interface.

It starts to get more confusing for me after that.

Can someone please explain the internal configurations of the LAN and WAN in layman’s terms? Thank you

alzhao · January 31, 2019, 2:11pm

Seems the WiFi page is wrong about AR150