networking

One of them will have to go through the other. Ultimately your modem only allows one device attached to it, which is likely because it assigns a public IP address directly. So if you could plug two in, they'd have to give you two IP addresses. Given the tendency of going CGNAT already, they probably don't want to do that.

So you need a router that will talk to the modem. Ideally that'd be the best one of them. So like maybe plug the IoT router into the LAN of the main router, so it appears as one client instead of dozens of light bulbs, and set up some QoS and firewall rules to keep it isolated and throttled down if needed. Ultimately if neither of the routers can support all that traffic, you'll need a router upgrade because it has to converge to that point.

If you can flash an custom firmware like OpenWRT on it, you can do some pretty fancy things and really tune your network. Usually you can broadcast multiple SSIDs and put them in different VLANs and the likes. On some crappier models, the software improvements can really make a difference, or at least let you tweak it in a way you can squeeze a bit more out of it.

If both of them can get OpenWRT you might be able to distribute the NAT load across them where the primary router only needs to forward the traffic from the IoT router. Usually stateful connection tracking is what blow up and you run out of memory to keep track of them all, and the router can't process any more connections. And all those IoT devices probably open one or more connection to the cloud each, using up precious conntrack slots. So if you can avoid having to track the state of connections from the IoT router on the main router, it's freed up to process the rest of your traffic. On the main router, allocate a range of ports that unconditionally get forwarded to the IoT router. Then make the IoT router use that port range for its own NAT, and voilà, you now have two router's worth of NAT capacity because the primary router can now just blindly forward IoT<>modem as-is.

That said, I feel like it's worth the investment in a good router. I have zero regrets about my EdgeRouterX and Unifi AC-Lite 4-5 years ago. I just have two VLANs configured on the router, two matching SSIDs on the AP, two completely isolated networks on the same hardware. So it can easily NAT the two networks to my ISP, no hassle and a ton of headroom. Barely uses 5% of the CPU doing a gigabit speed test