Different VPN per site? Flint AX & WireGuard

xize11 · 2023-11-10T01:11:43.022Z

This is probably not possible per site with custom routing from the GL Ui vpn policies.

However there is also a luci app which kinda replaces policies which should be possible to work with.

This app is called luci-app-pbr but in order to have it work more properly you need to set the global policies in the GL Ui to custom routing.

But theres also a other catch, you may also need luci-proto-wireguard but this might break the wgclient, why? because you need two instances of vpn and i don’t think its possible in GL’s version since it is unmanaged (could be wrong) but it looks different than vanilla OpenWrts wireguard opkg, in the worst scenario you need to rebuild wgclient in luci, a better scenario is only setting up your second instance so that both vpn interfaces can co-exist hence the software differences between vanillas version and GL’s version.

Then you also need to make sure all routing is disabled aswell from the new vpn interface, as from the default gateway checkbox in the advanced tab when you are in the interface edit, this is on all interfaces required except lan.

In pbr the priority is from top to down, so: your domain rule should always be on top of the rule that routes through the default vpn, Heres a example:

name: route site to vpn2
local ip: 192.168.8.0/24
remote ip: mysite.mydomain
chain: prerouting
interface: wgclient2

And

name: route all vpn
local ip: 192.168.8.0/24
remote ip:
chain: prerouting
interface: wgclient

I can see this is alot advanced so please let me know if you need help

GrannySmasher · 2023-11-10T21:57:25.244Z

Yeah that sounds way too complex for me, I will just stick with normal functionality. Your reply is much appreciated though.

bring.fringe18 · 2023-11-11T09:38:20.264Z

I can attest stangri’s PBR for LuCI is a rather impressive piece of software.

Yeah, @xize11 is right on point; it’s not going to be an easy time using GL firmware. luci-app-pbr will certainly conflict & for best results it’s best to have full nft tables for the firewall vs iptables… nft is now the default as of OpenWrt 23.05 but the Flint is still using 21.02 in firmware 4.4.6-release1. You’d also need dnsmasq-full (v2.39+) & not just stock/regular dnsmasq.

If you can afford the hit to the wallet, I’d pick up another Flint or a Slate AX & flash it with @solidus1983 's ‘pure’ OpenWrt firmware build for it. While there might be a hit to the device’s overall performance as it doesn’t have GL’s optimizations via the proprietary SDK, you’ll have full access to all things OpenWrt. You could then put this pure OpenWrt behind/downstream of your GL Flint to handle whatever device clients you want to use PBR.

Of course you can always just pick up a compatible OpenWrt router on the used market for experimentation… but it can be done & close to a case of ‘easier done than said.’

Welcome to docs.openwrt.melmac.net!

stangri’s OpenWrt Packages Documentation.

OpenWrt Forum – 24 Oct 22

Policy-Based-Routing (pbr) package discussion

Community Builds, Projects & Packages

The old vpn-policy-routing thread grew too big and contains a lot of outdated information, now that pbr is available, so I've decided to start a new thread. pbr is a next generation service supporting split-tunneling for multiple types of VPN...

Reading time: 246 mins 🕑 Likes: 216 ❤

4rtz1z · 2024-05-02T11:41:05.889Z

i would be very thankful if u could help me to setup PBR on GL-Inet firmware, i’m kinda noob to play with firewall and rules, i’ve used solidus’ openwrt but the having problem with crashing here and there, maybe i’ll try i again when it finally be officially approved and merged into openwrt git

xize11 · 2024-05-02T12:49:30.643Z

sure thing!.

do you know by chance how to export config files and show me a topology?

note with the gl version it is not possible yet to use multiple wireguard client interfaces or servers since it is structered different inside the uci system (the openwrt configuration system).

4rtz1z · 2024-05-02T13:02:07.866Z

the tar.gz backup files ? i know all the basic stuff, i just noob on firewall and rules thing

topology of my network ?

xize11 · 2024-05-02T13:05:18.775Z

yea you can post the contents of /etc/config/network and /etc/config/firewall just make sure you strip off the data you don’t want to share like vpn secret keys or mac addresses, these files can also be accessed if you open the backup tar file with something like 7zip.

with topology I mean a drawing what you want to accomplish you can do this via draw.io or a other site this gives me a idea.

4rtz1z · 2024-05-02T13:12:25.213Z

aaa okay, lemme flash back into glinet first,
well basically i just wanna route :

some ip/dns (e.g twitter and telegram) thru wireguard
local ip range to wireguard
banking site thru wan

here is my current usage of pbr


config policy
        option name 'Bank'
        option dest_addr 'XXXXXXXXXX'
        option interface 'wan'

config policy
        option name '150'
        option src_addr '192.168.1.150 192.168.1.151'
        option interface 'wg'

config policy
        option name '160'
        option src_addr '192.168.1.160 192.168.1.161'
        option interface 'wgp'

config policy
        option name 'Twitter IP'
        option dest_addr '192.229.237.0/24 117.18.232.0/24 146.75.0.0/16 103.252.112.0/22 104.244.40.0/21 185.45.4.0/22'
        option interface 'wg'

config policy
        option name 'Twitter DNS'
        option dest_addr 'ads-twitter.com cms-twdigitalassets.com periscope.tv pscp.tv t.co tellapart.com tweetdeck.com twimg.com twitpic.com twitter.biz twitter.com twitter.jp twittercommunity.com twitterflightschool.com twitterinc.com twitteroauth.com twitterstat.us twtrdns.net twttr.com twttr.net twvid.com vine.co x.com'
        option interface 'wg'

config policy
        option name 'Telegram IPv4'
        option dest_addr '91.105.192.0/23 91.108.4.0/22 91.108.8.0/21 91.108.16.0/21 91.108.56.0/22 149.154.160.0/20 185.76.151.0/24'
        option interface 'wgp'

config policy
        option name 'Telegram DNS'
        option dest_addr 'cdn-telegram.org comments.app contest.com fragment.com graph.org quiz.directory t.me tdesktop.com telega.one telegra.ph telegram-cdn.org telegram.dog telegram.me telegram.org telegram.space telesco.pe tg.dev tx.me usercontent.dev'
        option interface 'wg'

config policy
        option name '1337x.to'
        option dest_addr '104.31.16.0/24 1337x.to 1337x.gd 1337x.is 1337x.st 1337x.tw 1337xto.to 1337xx.to x1337x.eu x1337x.se x1337x.ws'
        option interface 'wg'

config policy
        option name 'ipleak'
        option dest_addr '95.85.16.0/24'
        option interface 'wg'

config policy
        option name 'openwrt.org'
        option dest_addr '64.226.122.0/24 139.59.210.0/24 46.101.214.0/24 forum.openwrt.org openwrt.org'
        option interface 'wgp'

xize11 · 2024-05-02T13:35:07.405Z

ah I see, this can be accomplished but in order to do that inside the gl ui you have to set the routing policies for the gl-inet to manual like in this screenshot:

image1123×795 53.2 KB

then in luci even though the wireguard protocol is different than normal openwrt you have to edit the interface via luci → network → interfaces → wgclient and click there on the tab advanced you should de-select Use Default Gateway my screenshot may be different since I use standalone OpenWrt.

image882×449 27.6 KB

this is recommended on all interfaces except wan and wwan if that is your wan connection and lan since that is a special interface, you want only one default gateway and not wireguard being the default gateway, let PBR handle this

now if you go to the luci-app-pbr web page you click on the tab advanced configuration and write down wgclient into the supported interfaces field, you can also add device names like so if you wish a smooth experience if you have alot of other interfaces not depending on pbr I recommend this because you may get a really long boot:

image368×502 3.93 KB

there is also one another tweak which makes reloading pbr a tiny bit faster and more stable under more interfaces which is the option in /etc/config/pbr:

config pbr 'config'
    option nft_file_mode '1'
    //there are more options here.

now if you look to my own configuration:

image900×548 25.5 KB

I use hierachy here, this is important to understand because if you want to bypass one thing it always has to be on top of the rule for the vpn.

The ignore rules let you bypass the killswitch feature this is especially necessary if you want to keep acccess to your local devices when you are using the vpn route.

this ignore rule can be enabled by going to Web UI Configuration tab.

4rtz1z · 2024-05-02T13:37:33.347Z

do u still need the cat of the config ? or do i just continue try as you described ?

xize11 · 2024-05-02T13:39:10.566Z

you can try to follow my message if you got questions please ask

4rtz1z · 2024-05-02T13:49:09.070Z

i’ve set the “Customize Routing Rules”, but when i go to Luci i can’t edit the wgclient, what should i do ? install luci-proto-wireguard first ?

Screenshot 2024-05-02 2044161227×103 13.1 KB

network

config interface 'loopback'
        option device 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fd82:3aa8:5b72::/48'

config device
        option name 'br-lan'
        option type 'bridge'
        option macaddr 'XXXXXXXXXXXXXXXXXXXXXXX'
        list ports 'eth1'
        list ports 'eth2'
        list ports 'eth3'
        list ports 'eth4'
        list ports 'eth5'

config device
        option name 'eth1'
        option macaddr 'XXXXXXXXXXXXXXXXXXXXXXX'

config device
        option name 'eth2'
        option macaddr 'XXXXXXXXXXXXXXXXXXXXXXX'

config device
        option name 'eth3'
        option macaddr 'XXXXXXXXXXXXXXXXXXXXXXX'

config device
        option name 'eth4'
        option macaddr 'XXXXXXXXXXXXXXXXXXXXXXX'

config interface 'lan'
        option device 'br-lan'
        option proto 'static'
        option netmask '255.255.255.0'
        option ip6assign '60'
        option isolate '0'
        option ipaddr '192.168.1.1'

config device
        option name 'eth0'
        option macaddr 'XXXXXXXXXXXXXXXXXXXXXXX'

config interface 'wan'
        option device 'eth0'
        option force_link '0'
        option ipv6 '0'
        option metric '10'
        option proto 'static'
        option ipaddr '192.168.55.2'
        option gateway '192.168.55.1'
        option vlanid '0'
        option netmask '255.255.255.0'
        option peerdns '0'
        option dns '192.168.1.4'

config interface 'wan6'
        option proto 'dhcpv6'
        option disabled '1'
        option device '@wan'

config switch
        option name 'switch0'
        option reset '0'
        option enable_vlan '0'

config interface 'tethering6'
        option proto 'dhcpv6'
        option disabled '1'
        option device '@tethering'

config interface 'wwan6'
        option proto 'dhcpv6'
        option disabled '1'
        option device '@wwan'

config interface 'guest'
        option force_link '1'
        option type 'bridge'
        option proto 'static'
        option ipaddr '192.168.9.1'
        option netmask '255.255.255.0'
        option ip6assign '60'
        option multicast_querier '1'
        option igmp_snooping '0'
        option isolate '0'
        option bridge_empty '1'
        option disabled '1'

config interface 'wwan'
        option proto 'dhcp'
        option metric '20'

config interface 'modem_1_1_6'
        option proto 'dhcpv6'
        option disabled '1'
        option device '@modem_1_1'

config rule 'policy_direct_rt'
        option lookup 'main'
        option suppress_prefixlength '0'
        option priority '1100'

config rule 'policy_default_rt_vpn'
        option mark '0x8000/0xc000'
        option lookup '8000'
        option priority '1101'
        option invert '1'

config rule6 'policy_direct_rt6'
        option lookup 'main'
        option suppress_prefixlength '0'
        option priority '1100'

config rule6 'policy_default_rt_vpn6'
        option mark '0x8000/0xc000'
        option lookup '8000'
        option priority '1101'
        option invert '1'

config rule 'policy_default_rt_vpn_ts'
        option lookup 'main'
        option priority '1099'
        option mark '0x80000/0xc0000'
        option invert '0'

config interface 'wgclient'
        option proto 'wgclient'
        option config 'peer_9842'
        option disabled '0'

firewall

config defaults
        option syn_flood '1'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'REJECT'

config zone
        option name 'lan'
        list network 'lan'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'ACCEPT'

config zone
        option name 'wan'
        list network 'wan'
        list network 'wan6'
        list network 'wwan'
        option output 'ACCEPT'
        option forward 'REJECT'
        option mtu_fix '1'
        option input 'DROP'
        option masq '1'

config forwarding
        option src 'lan'
        option dest 'wan'
        option enabled '1'

config rule
        option name 'Allow-DHCP-Renew'
        option src 'wan'
        option proto 'udp'
        option dest_port '68'
        option target 'ACCEPT'
        option family 'ipv4'

config rule
        option name 'Allow-IGMP'
        option src 'wan'
        option proto 'igmp'
        option family 'ipv4'
        option target 'ACCEPT'

config rule
        option name 'Allow-DHCPv6'
        option src 'wan'
        option proto 'udp'
        option src_ip 'fc00::/6'
        option dest_ip 'fc00::/6'
        option dest_port '546'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-MLD'
        option src 'wan'
        option proto 'icmp'
        option src_ip 'fe80::/10'
        list icmp_type '130/0'
        list icmp_type '131/0'
        list icmp_type '132/0'
        list icmp_type '143/0'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-ICMPv6-Input'
        option src 'wan'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        list icmp_type 'router-solicitation'
        list icmp_type 'neighbour-solicitation'
        list icmp_type 'router-advertisement'
        list icmp_type 'neighbour-advertisement'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-ICMPv6-Forward'
        option src 'wan'
        option dest '*'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-IPSec-ESP'
        option src 'wan'
        option dest 'lan'
        option proto 'esp'
        option target 'ACCEPT'

config rule
        option name 'Allow-ISAKMP'
        option src 'wan'
        option dest 'lan'
        option dest_port '500'
        option proto 'udp'
        option target 'ACCEPT'

config rule
        option name 'Support-UDP-Traceroute'
        option src 'wan'
        option dest_port '33434:33689'
        option proto 'udp'
        option family 'ipv4'
        option target 'REJECT'
        option enabled 'false'

config include
        option path '/etc/firewall.user'

config include 'nat6'
        option path '/etc/firewall.nat6'
        option reload '1'

config rule 'block_dns'
        option name 'block_dns'
        option src '*'
        option device 'br-+'
        option dest_port '53'
        option target 'REJECT'
        option enabled '0'

config rule 'process_mark'
        option name 'process_mark'
        option dest '*'
        option proto 'all'
        option extra '-m owner --gid-owner 65533'
        option target 'MARK'
        option set_xmark '0x8000/0xc000'

config rule 'wan_in_conn_mark'
        option name 'wan_in_conn_mark'
        option src 'wan'
        option dest '*'
        option set_xmark '0x8000/0xc000'
        option target 'MARK'
        option extra '-j CONNMARK --set-xmark 0x8000/0xc000'
        option enabled '1'

config rule 'lan_in_conn_mark_restore'
        option name 'lan_in_conn_mark_restore'
        option src 'lan'
        option dest '*'
        option set_xmark '0x8000/0xc000'
        option target 'MARK'
        option extra '-m connmark --mark 0x8000/0xc000 -j CONNMARK --restore-mark'
        option enabled '1'

config rule 'out_conn_mark_restore'
        option name 'out_conn_mark_restore'
        option dest '*'
        option set_xmark '0x8000/0xc000'
        option target 'MARK'
        option extra '-m connmark --mark 0x8000/0xc000 -j CONNMARK --restore-mark'
        option enabled '1'

config include 'swap_wan_in_conn_mark'
        option type 'script'
        option reload '1'
        option path '/etc/firewall.swap_wan_in_conn_mark.sh'
        option enabled '1'

config include 'gls2s'
        option type 'script'
        option path '/var/etc/gls2s.include'
        option reload '1'

config include 'glblock'
        option type 'script'
        option path '/usr/bin/gl_block.sh'
        option reload '1'

config zone
        option name 'guest'
        option network 'guest'
        option forward 'REJECT'
        option output 'ACCEPT'
        option input 'REJECT'

config forwarding
        option src 'guest'
        option dest 'wan'
        option enabled '1'

config rule
        option name 'Allow-DHCP'
        option src 'guest'
        option target 'ACCEPT'
        option proto 'udp'
        option dest_port '67-68'

config rule
        option name 'Allow-DNS'
        option src 'guest'
        option target 'ACCEPT'
        option proto 'tcp udp'
        option dest_port '53'

config include 'vpn_server_policy'
        option type 'script'
        option path '/etc/firewall.vpn_server_policy.sh'
        option reload '1'
        option enabled '1'

config zone 'wgclient'
        option name 'wgclient'
        option forward 'DROP'
        option output 'ACCEPT'
        option mtu_fix '1'
        option network 'wgclient'
        option enabled '1'
        option input 'DROP'
        option masq '1'
        option masq6 '1'

config forwarding 'wgclient2wan'
        option src 'wgclient'
        option dest 'wan'
        option enabled '1'

config forwarding 'lan2wgclient'
        option src 'lan'
        option dest 'wgclient'
        option enabled '1'

config forwarding 'guest2wgclient'
        option src 'guest'
        option dest 'wgclient'
        option enabled '1'

4rtz1z · 2024-05-02T13:50:33.008Z

or just add defaultroute=‘0’ manually ? and restart network service ?

xize11 · 2024-05-02T13:51:33.445Z

yes you can try using defaultroute=0 though it might be removed recently because it did not had a function just try add it to be extra sure

4rtz1z · 2024-05-02T13:53:20.966Z

and what way did u like me to do ? i’m just gonna follow u straight so u could easily help me LOL

xize11 · 2024-05-02T13:54:24.197Z

you can uncheck this checkbox on each interface except lan, wan, wwan or a other wan type of interface, I forgot to mentoin in my previous post that lan is also considered a special interface.

4rtz1z · 2024-05-02T13:55:44.355Z

how about the wgclient ? i can’t click edit

xize11 · 2024-05-02T13:57:40.527Z

i think it should be fine , if not pbr will report it too as a default gateway indicating with the ✓ symbol, then you must exclusively add defaultroute='0' inside the wgclient configuration inside /etc/config/network.

4rtz1z · 2024-05-02T14:16:03.537Z

my device suddenly went reboot and couldn’t boot after trying installing luci-app-pbr LOL, i need to start again

root@GL-AX1800:~# opkg install luci-app-pbr
Installing luci-app-pbr (1.1.5-1) to root...
Downloading https://repo.openwrt.melmac.net/luci-app-pbr_1.1.5-1_all.ipk
Installing kmod-lib-crc32c (4.4.60-1) to root...
Downloading https://fw.gl-inet.com/releases/v21.02-SNAPSHOT/kmod-4.2.1/arm_cortex-a7/ip60xx/kmod-lib-crc32c_4.4.60-1_arm_cortex-a7.ipk
Installing kmod-nft-core (4.4.60-1) to root...
Downloading https://fw.gl-inet.com/releases/v21.02-SNAPSHOT/kmod-4.2.1/arm_cortex-a7/ip60xx/kmod-nft-core_4.4.60-1_arm_cortex-a7.ipk
Installing kmod-nft-nat (4.4.60-1) to root...
Downloading https://fw.gl-inet.com/releases/v21.02-SNAPSHOT/kmod-4.2.1/arm_cortex-a7/ip60xx/kmod-nft-nat_4.4.60-1_arm_cortex-a7.ipk
Installing libnftnl11 (1.1.8-1) to root...
Downloading https://fw.gl-inet.com/releases/v21.02-SNAPSHOT/packages-4.0/arm_cortex-a7/packages/libnftnl11_1.1.8-1_arm_cortex-a7.ipk
Installing nftables-json (0.9.6-2) to root...
Downloading https://fw.gl-inet.com/releases/v21.02-SNAPSHOT/packages-4.0/arm_cortex-a7/packages/nftables-json_0.9.6-2_arm_cortex-a7.ipk
Installing pbr (1.1.5-1) to root...
Downloading https://repo.openwrt.melmac.net/pbr_1.1.5-1_all.ipk
Configuring kmod-lib-crc32c.
Configuring kmod-nft-core.
Configuring kmod-nft-nat.
Configuring libnftnl11.
Configuring nftables-json.
Configuring pbr.
//usr/lib/opkg/info/pbr.postinst: /usr/lib/opkg/info/pbr.postinst-pkg: line 5: fw4: not found
Installing rc.d symlink for pbr... OK
client_loop: send disconnect: Connection reset

xize11 · 2024-05-02T14:19:30.923Z

oh… I think you need some modifications for this I remember

you need to install first:

opkg install pbr-iptables and over this you install luci-app-pbr, if you don’t do it this way the nft version gets installed which does not work.