Maybe you need set clear allowed ips (default is 0.0.0.0/0) in your Wireguard clients?