I thought I'd miss not having a headphone jack but I don't. I don't miss getting the wires caught on things and yanking out of my ears.
People will always be very vocal about things like this but I'd bet the vast majority of people don't care. It's a bit like smaller phones, I'm firmly in the "I need a smaller phone" camp but it seems like the market doesn't want one and most people prefer the big phones we have today.