Holy cow! I’m not alone! I listen to music a lot on my AirPods Pro 2nd gen, and this happens to me often. Volume lowers a tad and i gotta turn it back up again.
I’m currently on iOS 17.1. Happens both on Spotify and Apple Music. And no I don’t have loud music alert turned on.