Piracy: ꜱᴀɪʟ ᴛʜᴇ ʜɪɢʜ ꜱᴇᴀꜱ

I have a Pi4B but with a m2 drive as the OS instead of the sdcard which I highly recommend due to the amount of read/write downloading files does.

A pi4 should have 0 trouble (via ethernet) doing anything you are saying.

From what I could tell your setup is internet -> pi4 -> appletv. If the pi & applet are on Wifi that would be a huge issue. The most any wifi can realistically do is 270mbps and that is when you are next to the damn thing.
As let's say you are streaming 1080p content which requires around 20-50mbps. Router to pi (20mbps) + pi to router (20mbps) + router to Apple TV 20mbps), so you'd already be pushing 60mbps at best internally all over wifi.

Temporarily add cables to everything if possible or move the pi onto a cable as a minimum. then you will cut down on the back-and-forth it needs to do over wifi so then your router can dedicate all its bandwidth to sending the data to the appletv.

If it's all on cables already then I guess it's a filesystem issue, as your not using a SSD but using a SD card. Run iostat via command line while streaming and see if there is any big "iowait" values. I'd there is then your sd card is not keeping up with the total bandwidth