So you use one as Wireguard server and one as Wireguard client, right?

It seems that your ISP limit your upload speed. So your Wireguard will be limited.

If this is the case, you should ask your isp give you a plan that has good upload speed.