WireGuard Site to Site again

Mikey · March 29, 2022, 7:03pm

Hi,

after updating my AR300M to the newest Version i did a new try to get Wireguard working with a limited site 2 site setup and failed again.

Setup:
GLINET is working as Hotspot and the WAN is connected to a foreign wifi network.

Test 1: Working Base setup

Wireguard Tunnel works with allowed IPs 0.0.0.0/0 on GLINET
All traffic is routed through the tunnel.
Site A (homenetwork) can reach IP subent of Site B (glinet).
Site B (glinet) can reach all IP subents of Site A.
Site B can reach a public Website which has the same WAN IP as the WireGuard Server on Site A.

That means the server side is working and configured correctly.

Test 2: SITE B (glinet) should only route 3 private subnets through the tunnel (because of performance). The rest should go outsite the foreign WLAN WAN network.

Activate VPN policies and limit vpn traffic for example to 192.168.10.0/24, 192.168.20.0/24, 192.168.30.0/24

Problem a) Subnet of Site A(homenet) is reachable, but Site A can´t reach any IPs from Site B (glinet) anymore
Problem b) GLINET clients can´t reach the public Website on site A anymore (beacuse of same IP as wireguard server?)

Test 3: Same as Test 2 with different config

deactivated VPN policies again
manually edit wireguard.conf file and set allowed IP´s from 0.0.0.0/0 to the 3 subnets of Site B.

Problem a) Like last year the input field of Wireguard settings of Allowed IPs is again limited to one subnet only
Problem b) Site A (homenet) can´t reach the Site B (glinet) subnet anymore

I don´t find any solution again what i can modify to get this setup working.
Can somebody help please? @alzaho?

alzhao · March 30, 2022, 10:12am

Can you try using the Goodcloud platform?

Each time I met manual settings cases, it takes me a lot of time to understand and solve.

GLrs · March 30, 2022, 12:57pm

A better description of your base setup was very helpful, usually a (hand-)drawing is easiest to work with.
Your description misses some points like who or what does connect to the WG-server (on the GLinet?)? How are sites A and B configured? WG-server and router the same device in the same subnet? Where are the other subnets (.20.0/24 and .30.0/24) located, how are they routed to .10.0/24? etc. pp.

Your WG-clientconfig was helpful as well (snip the private key)

Mikey · March 30, 2022, 3:19pm

Ok heres my setup.

Here are my configs:

FIREWALL

[Interface]

Address = 192.168.102.1/24
ListenPort = 61xxx
PrivateKey = zxy=

[Peer]
PublicKey = xyz=
AllowedIPs = 192.168.102.2/32,192.168.8.0/24

GLINET

cat /etc/config/wireguard

config proxy
        option main_server 'myNET'
        option access 'ACCEPT'
        option enable '1'
        option host '77.77.77.77'

config peers 'wg_peer_3979'
        option name 'myNET'
        option dns '192.168.10.101'
        option persistent_keepalive '25'
        option listen_port '61xxx'
        option end_point 'vpn.myNET.de:61xxx'
        option address '192.168.102.2/24'
        option public_key 'xyz='
        option private_key 'zyx='
        option allowed_ips '192.168.10.0/24,192.168.20.0/24,192.168.30.0/24'

While typing i recognized that i missed to set also 192.168.102.1/32 on the glinet under allowed ip´s. Maybe thats the fault!? Can´t test today

EDIT: Tested. Isn´t working. Tunnel is up but the glinet dont put anything through the tunnel - i see it in my firewall that no data are incoming. And also i can´t reach any glinet client.

GLrs · March 31, 2022, 4:58am

that’s a nasty one, @Mikey…
GL-inet appears to have kinda own way to put multiple subnets.

Can you pls try

config peers 'wg_peer_3979'
        [...]
        option allowed_ips '192.168.10.0/24'
        option allowed_ips '192.168.20.0/24'
        option allowed_ips '192.168.30.0/24'

Manually editing the config as shown above works like a charm over here.

alzhao · March 31, 2022, 8:54am

In the Firewall side, pls make sure wg0 interface can access lan subnets.

This should be set in your Firewall device.

Mikey · March 31, 2022, 7:27pm

@GLrs I will try it tomorrow. Never thought of this notation because in an example someone connected two glinet site 2 site with same notation like in my config. Thank you!

@alzhao I verified it already. If i set allowed ip’s on glinet to 0.0.0 0/0 both sides can reach all clients and subnets on the other side. But ALL traffic is tunneled. Actually my Firewall Rules allow “any” on wg0 for testing purpose. Isn’t it possible to set the wireguard connection to 0.0.0.0/0 and use the VPN policies infront? Thanks also!

alzhao · April 1, 2022, 1:44am

I am not sure. I was under the impression that the Firewall is not a GL.iNet router. So I don’t know what kind of rules are there.

Mikey · April 2, 2022, 8:33pm

Try 1: Extending /etc/config/wireguard to separate for example:

option allowed_ips '192.168.102.1/32'
option allowed_ips '192.168.10.0/24'
option allowed_ips '192.168.20.0/24'
option allowed_ips '192.168.30.0/24'

Problem 1A)

After reboot, the GLINET scripts can´t interpret the configfile anymore and don´t show the established connection in the WebGUI and tell me to create one. With a single entry the config is “back”.

Problem 1B)
Independent from showing the connection in the WebGUI:
Only one route from the last entry of option allowed_ips got automatically created.
This subnet is correctly reachable.

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.179.1   0.0.0.0         UG    20     0        0 wlan-sta
46.xx.xx.xxx    192.168.179.1   255.255.255.255 UGH   0      0        0 wlan-sta
192.168.8.0     0.0.0.0         255.255.255.0   U     0      0        0 br-lan
192.168.9.0     0.0.0.0         255.255.255.0   U     0      0        0 br-guest
192.168.30.0    0.0.0.0         255.255.255.0   U     0      0        0 wg0
192.168.102.0   0.0.0.0         255.255.255.0   U     0      0        0 wg0
192.168.179.0   0.0.0.0         255.255.255.0   U     20     0        0 wlan-sta

Problem 1C:
After adding a new manual route to one of the “missing” subnets:
ip route add 192.168.10.0/24 dev wg0
no client in this subnet ist reachable. It seems that wireguard don´t get all allowed_ips correctly.

@GLrs: Does not work!

Try 2: Extending /etc/config/wireguard for example:
option allowed_ips ‘192.168.102.1/32, 192.168.10.0/24, 192.168.20.0/24, 192.168.30.0/24’

Problem 2A)
No route gets created. I think again because of incorrect interpretation of the config file.
192.168.102.1 UH gets created

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.179.1   0.0.0.0         UG    20     0        0 wlan-sta
46.xx.xx.xxx    192.168.179.1   255.255.255.255 UGH   0      0        0 wlan-sta
192.168.8.0     0.0.0.0         255.255.255.0   U     0      0        0 br-lan
192.168.9.0     0.0.0.0         255.255.255.0   U     0      0        0 br-guest
192.168.102.0   0.0.0.0         255.255.255.0   U     0      0        0 wg0
192.168.102.1   0.0.0.0         255.255.255.255 UH    0      0        0 wg0
192.168.179.0   0.0.0.0         255.255.255.0   U     20     0        0 wlan-sta

Solution:
ip route add 192.168.10.0/24 dev wg0
ip route add 192.168.20.0/24 dev wg0
ip route add 192.168.30.0/24 dev wg0

Every Subnet is reachable from GLINET Site.
GLINET subnet is reachable form other side.

Problem 2B)
After rebooting the routes are gone.

@All:
Whats the correct way to get that routes permanently, so that they don´t get lost after reconnecting / rebooting?
Temporary i insert
ip route add 192.168.20.0/24 dev wg0
ip route add 192.168.30.0/24 dev wg0
ip route add 192.168.181.0/24 dev wg0
under custom Rules in the firewlall settings. Seems to work after reboot.

Last but not least another big Problem (all my iot devices are actually stop working):
After get everything temporary working i can´t reach my webserver/databases (same WAN IP as the Wireguard Server) from any GLINET client.
DNS is working correctly. Traceroute stops after 192.168.8.1 with no route to host. Ping answers Host not reachable.
But via ssh on GLINET i can ping and traceroute to my webserver correctly. Same problem like last year. Also i can´t ping other domains which points to the same homenet IP.

I would be happy if someone give me a hint how solve this issues …

Edit: I tried to find something and activated Firewall logging on the LAN Zone but i don´t find any entry from the firewall in the system.log that anything is rejected. Now i need to turn erverything back to opnvpn until someone can help me. I think yesterday after manually start wireguard in the gui and manual adding the routing rules that problem doesn´t exist. As i said from the GLINET itself i can reach the webserver/databases correctly.

root@GL-AR300M:~# ping xyz.de
PING xyz.de (46.xx.xx.xxx): 56 data bytes
64 bytes from 46.xx.xx.xxx: seq=0 ttl=58 time=25.526 ms

Client:

pi@client:~ $ ping xyz.de
PING xyz.de (46.xx.xx.xxx) 56(84) bytes of data.
From 192.168.8.1 (192.168.8.1) icmp_seq=1 Destination Host Unreachable

pi@client:~ $ sudo traceroute -d xyz.de
traceroute to xyz.de (46.xx.xx.xxx), 30 hops max, 60 byte packets
 1  192.168.8.1 (192.168.8.1)  0.591 ms  0.697 ms  0.827 ms
 2  192.168.8.1 (192.168.8.1)  1.032 ms !H  1.250 ms !H  1.467 ms !H

pi@client:~ $ sudo ip route
default via 192.168.8.1 dev eth0 src 192.168.8.10 metric 202 
192.168.8.0/24 dev eth0 proto dhcp scope link src 192.168.8.10 metric 202

alzhao · April 4, 2022, 8:17am

Can you change option to list

option allowed_ips '192.168.102.1/32'
option allowed_ips '192.168.10.0/24'
option allowed_ips '192.168.20.0/24'
option allowed_ips '192.168.30.0/24'

It should be

list allowed_ips '192.168.102.1/32'
list allowed_ips '192.168.10.0/24'
list allowed_ips '192.168.20.0/24'
list allowed_ips '192.168.30.0/24'

Mikey · April 4, 2022, 12:40pm

OK. I will test it as soon as possible.
But i´m wondering because if i create a new Wireguard connection with the GLINET WebGUI without importing anything the GUI itself writes in config file 'option allowed_ips ‘xxx’ instead of list.

Who is interested in (maybe i’m the only one who don´t knew this before):
Repeating the same “option” is an error, repeating the same “list” adds values to the parameter. Using a single “list” with a list of values is the same as using “option”, and may not work with some parameters, when the values are parsed.

Is this also responsible for my problems that i can´t reach my domain from all glinet clients or only for creating the routes to the subnets?

Mikey · April 4, 2022, 4:28pm

Hello @alzhao,

i tested it. It´s not working.
If there is one entry with list or option i get a connection.
With two or more entrys of type list i get

VPN client failed to connect. This may be because of wrong configuration, unsupported parameters or terminated by the server.

Mikey · April 12, 2022, 5:04pm

I changed to clean openwrt 21.02.2 because nobody helps me.
With the same settings all is working as expected with my wireguard tunnel.

@alzhao: Can you confirm that the AR300-M16 ext has Wifi problems with v. 21.02 ath79 Version?
At home with strong Wifi i don´t recognized a problem. On my external Site the download speed is reduced to 1/2 till 1/3 compared to the glinet Version on my AR300M ext. I tested is twice - same settings, same position, same antenna - more than half download speed in STA mode on 21.02… For test i will change to clean 19.07.7 arxxx on weekend.

Is the hardware of the AR300-M16 ext and AR300M ext except of the memory identical?

alzhao · April 13, 2022, 8:15am

I don’t know this problem. Maybe you can adjust wifi channel, band width and check.

But one thing, when upgrading Linux kernel and swconfig changed to dsa, openwrt is becoming bigger and performance are affected, including wifi and wireguard.

So we are very careful to upload to latest versions.