Unless the overall mortage payment goes down or you have a serious need (had a baby and need more room or have a physical challenge and need less), I can't imagine abandoning a low interest rate.
I bought this year and the interest rate makes my payment so much higher that it is insane. I can't wait until rates lower and I can refinance.