None of my GaN chargers (i have 3) run cooler than around 50 degrees on the surface of the charger, so you can imagine how hot they are inside.

The GaN chips can handle more current more efficiently yes, but since they can also handle a higher maximum temperature, they are also pushed harder by the designers. Further pushing the designs to be smaller means worse heat management, at least in the current designs.

Silicon has a maximum temperature of 170C (Max TJunction). Higher than that and the heat causes thermal agitation that tends to disrupt chemical bonds. The transistors start to loose their “identity” quickly and degrade in performance. Bonding wires also start to get damaged from the stress of the heat. On the other hand GaN has a maximum temperature of 300C. It is even encouraged to run GaN chips hotter for maximum efficiency (around 220C).