'VPN Policy Based on Target Domain' not working

[Fox]

Problem: VPN Policy based on target domain not working. Policy based on IP is working.

I know people have posted similar topics, but I believe mine is unique...

• Device: GL-AXT1800, Panel v4.5.16, Kernel 4.4.60
• VPN: PrivateInternetAccess
• Policy: Accessing target domain or IP do not use VPN.
• Global option: Block non-VPN traffic OFF, Allow access WAN OFF, services from GL.iNet use VPN OFF.

• I'm not using AdGuard Home (toggled off).

• Under DNS, I have tried to toggle on 'Override DNS Settings for All Clients' and Mode changed to 'Encrypted DNS'. It made no difference.
  

  

• On my Ubuntu, I am using the router as the DNS resolver. The default gateway and DNS are both the LAN IP of the router.

• I tried clearing the DNS cache to no avail: sudo resolvectl flush-caches

Test:

1. Add ifconfig.me into the VPN policy to not use VPN.
2. curl ifconfig.me returns IP of VPN.
3. curl -v ifconfig.me to get IPv4 address. Add this IP to VPN policy.
4. curl ifconfig.me returns my real IP.

• I am using router as the DNS resolver.
• I believe nothing is overwriting my VPN policy (but please let me know if you think I missed something).
• VPN policy based on domain still doesn't work.
• Why / what can I do?

Thank you!!

[admon]

Override DNS Settings for All Clients should be enabled.

Try to test with other IP detection services (like ifconfig.co) as well.

[Fox]

Thank you very much @admon for your quick response!

• I confirm Override DNS Settings for All Clients is toggled on.
• I repeated the same test to ifconfig.co and got the same results:
  • Add ifconfig.co to VPN policy so this domain does not use VPN
  • curl -v ifconfig.co tells me my IP is that of VPN.
  • Add two IPs of ifconfig.co to the VPN policy.
  • curl ifconfig.co now gives me my real IP.

[admon]

Are you 100% sure that there is no other resolve way for your client?
If you do nslookup ifconfig.co what response do you get?

[Fox]

I am not 100% sure so I appreciate your kind guidance.

I see 127.0.0.53 resolving:

$ nslookup ifconfig.co
Server:		127.0.0.53
Address:	127.0.0.53#53

Non-authoritative answer:
Name:	ifconfig.co
Address: 104.21.54.91
Name:	ifconfig.co
Address: 172.67.168.106
Name:	ifconfig.co
Address: 2606:4700:3037::6815:365b
Name:	ifconfig.co
Address: 2606:4700:3030::ac43:a86a

In my original post, I was referring to the information provided when clicking on Ubuntu's GUI

[Fox]

$ cat /etc/resolv.conf
nameserver 127.0.0.53
options edns0 trust-ad

$ resolvectl status
...
Link 2 (enp1s0)
    Current Scopes: DNS
         Protocols: +DefaultRoute -LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: 192.168.8.1
       DNS Servers: 192.168.8.1
        DNS Domain: lan
...

[alzhao]

@Fox

Server:		127.0.0.53

Can you confirm what is this IP?

[teleney]

This is a issue, try to excute 'nslookup ifconfig.me' on the router after modifying the vpn policy. This solution could only be temporary.

[Fox]

Thank you for looking into my issue.

After much digging, it appears that 127.0.0.3 is the local stub resolver for systemd-resolved. To my understanding, it performs local caching of DNS results. If not found in cache, resolved will query upstream. We can check which is upstream with resolvectl status. (In my case this is pointing to the router)

For other's references:

• I turn off resolved's caching by adding Cache=no (case sensitive) to /etc/systemd/resolved.conf.
• Then, sudo systemctl restart systemd-resolved to apply changes.
• Confirm that cache is off with sudo resolvectl statistics.

[Fox]

It gives the same result except the DNS server is 127.0.0.1:

$ nslookup ifconfig.me
Server:         127.0.0.1
Address:        127.0.0.1#53

Name:      ifconfig.me
Address 1: 34.117.118.44
Address 2: 2600:1901:0:bbc3::

I also noticed that it's easier to remove a domain from whitelist than to add it. In many lucky cases:

$ # Already in whitelist
$ curl ifconfig.co
(real IP)
$ curl ifconfig.me
(real IP)

$ # Remove from whitelist
$ curl ifconfig.co
[VPN IP]
$ curl ifconfig.me
[VPN IP]

$ # Add to whitelist, did not take effect
$ curl ifconfig.co
[VPN IP]
$ curl ifconfig.me
[VPN IP]

I tried turning off/on the VPN, I still obtain the VPN IP despite having added to whitelist.

I wonder if it was using the wrong resolver, perhaps the VPN's resolver?

I also tried curl ifconfig.me on the router but I'm not certain this result is not heavily cached. I have seen it giving a different answer than on my Ubuntu (with caching turned off).

[teleney]

1. First, add 'ifconfig. me' to the list and apply it.
2. Go to the router and execute 'ipset list', it will most likely be empty.
3. Run the 'curl ifconfig. me' command on Ubuntu. The rusult should be wrong.
4. Run 'nslookup ifconfig.me' on your router.
5. By executing 'ipset list' on your router, you can get the IP address of 'ifconfig.me'.
6. Now, run the 'curl ifconfig' command on Ubuntu. The result will be correct.

You can follow the steps I said and you may find the cause of the problem.

[alzhao]

@Fox @teleney Here is the problem:

When you use encrypted DNS (including AdguardHome), the DNS query cannot be splited, meaning it goes to either vpn or non-vpn.
So in the above test, the DNS resolving surely goes to vpn, even you use it in vpn policy.
But after it resolves to IP, it will split.

For DNS leaks, we decided to faover the VPN connection so all encrypted dns goes to VPN.

If you want "domain-based vpn policy" has the highest priority, using it with encrypted dns will be a difficult. Two encrypted dns streams need to created. And we need to decide which stream to use at the very early stage.

[admon]

@alzhao

When you use encrypted DNS (including AdguardHome), the DNS query cannot be splited, meaning it goes to either vpn or non-vpn.

Something seems odd here.

I am using AGH + encrypted upstream servers using QUIC/DoH + VPN split policies and everything is totally working!

The only thing I don't use is encrypted DNS from my device to the AGH instance. But I doubt that @Fox uses it because configuration of encrypted DNS to AGH is really pain.

So in my opinion the issue is something else.

[Fox]

Thank you all for your detailed response.

I had great success following @teleney's suggestions.

1. ifconfig.me not in VPN whitelist. Curl result shows VPN IP as expected. ifconfig.me is then added to whitelist and applied.
2. ipset list on router returns 5 sets, all without any members.
3. curl ifconfig.me still returns VPN IP (wrong, but as you expected)
4. nslookup ifconfig.me on router.
5. ipset list now shows the IPs as member under bypass_vpn_domain set.
6. curl ifconfig.me on Ubuntu is now correctly returning my residential IP.

In your original comment where you said it is an issue, do you mean it is a known bug that will be fixed in the future? My concerns are as follows:

1. As you mentioned, this is temporary (and manual). I tried clicking Apply on the policy editor without making any changes, and found out that all domains I whitelisted disappeared from ipset list, while all IPs that I whitelisted remains.
2. This approach doesn't work with services where the domain is stable but the IP of the backend is load balanced / rotated frequently. (Unless I manually run nslookup again)

Implications:

• At least for the workaround now, since matching is based on IP and not domain, it appears that having a stub resolver with caching on local PC is still fine for the cases where the IP is static.
• Automatic subdomain matching will be broken for now unless manually processed.
• The config is written to /etc/config/vpnpolicy. I am tempted to run nslookup on the entries daily, but of course, running realtime checks is preferred, which handles subdomains as well.

Is there anything else I can help with to get this bug resolved?

Many thanks!

[teleney]

I did some tests to confirm that DNS traffic from the subnet would successfully update the ipset. I think it may be that your Ubuntu is not successfully sending DNS messages. You can try capturing packets on the router to ensure that DNS packets are being sent from Ubuntu (tcpdump -i br-lan port 53).

@Fox

• As you mentioned, this is temporary (and manual). I tried clicking Apply on the policy editor without making any changes, and found out that all domains I whitelisted disappeared from ipset list, while all IPs that I whitelisted remains.
• This approach doesn't work with services where the domain is stable but the IP of the backend is load balanced / rotated frequently. (Unless I manually run nslookup again)

I think as long as it can trigger IP updates, it can be used normally.