I have had this plant for several years. Within the past couple months I noticed a few leaves yellowing, but lately it has accelerated greatly to where about half have fallen. I water regularly and keep the soil moist. The pot is large and does have drainage holes. I stopped watering about 2 weeks ago to let the soil dry out but that hasnt seemed to help. This year I have noticed earthworms and centipedes in the soil if I dig around.
When was the last time you repotted? If it's been in the same pot for years it might have gone through a lot of the good stuff in the soil already, if it hasn't been washed out through watering.
Have you looked I to fertilizers? Might help avoid fully repotting.
I'm in no way a professional at this stuff but I ran into similar issues with some pepper plants I've had for years and they definitely perked up after repotting.
Interesting. I have not repotted or fertilized. The pot is fairly large so repotting would be hard but I will definitely try some fertilizer
You definitely need to fertilize. It might not be the root problem, but any potted plant needs fertilizer because nitrogen is water soluble and washes out when you water it. The leaves look chlorotic, so lack of fertilizer tracks. I recommend a balanced water-soluble fertilizer. Use it in a dilute application any time the plant shows growth or is flowering.
Once that is going, you can further diagnose issues.