Nowadays, applications and devices may infer the SMTP protocol from the port number and/or auto-negotiate the protocol. Port 25 is essentially obsolete with no encryption. Port 465 is probably SSL encryption. Port 587 is probably TLS encryption.
Since you have your own email server, you should be able to find how which SMTP protocols are supported (could be multiple protocols) and check if they match those port/encryption combinations. Another thing to check is whether your email server requires OAuth2 authorization.
I do not work for and I am not directly associated with GL.iNet