I own both. I carry the XDs and gave the MP to my sister to carry. Reasons being, the MP had an external safety that I did not care for, the MP is larger, the MP was in 9mm. My XDs is much smaller and narrower, comes in a great .45, and only has a grip safety.
The MP did feel better in my hands but its a CC gun, not a range gun. Both are great guns but the XDs fit my needs better and now the MP stays in my sisters purse.
I shot it enough to break it in and shoot it every month or so to stay fresh. I do not bring it to the range every weekend. The XDs does a great job managing the recoil and followup shots are not difficult at all. I would honestly say it is on par with my G23, just smaller.
I question people who choose one over the other, there are not many XDs out there and it seems unlikely that many have shot both.
I don't suppose the safety on the M&P would be very good for carrying because it isn't the easiest or fasted thing to manipulate. I find it to be a little bit too high on the frame and I don't like the angle it's at, where as a 1911 safety is incredibly fast and easy for me even though I hardly never shoot them. I'm far from what I consider to be familiar with a 1911 and the safety is still easy as shite for me.
I would prefer it to the grip safety on the XD though. I don't like grip safeties.