So I think I figured out what is going on. I tried again using a completely different third-party USB wall charger. Same router drops. Noticed this time it was the router actually restarting, as I have enabled the new feature of turning off the LED after 10pm, and I kept seeing it come back on.
Noticed this happening again after I disconnected my phone from the charging cable, also plugged into the bulk charger.
Realized that what is probably happening is that when the devices are unplugged (or maybe even when they are plugged in, too), the charger is changing up the power delivery to the ports. There must be a slight power interruption, causing the router to reboot.