A .45 XDS is my every day carry gun. Recoil is very manageable. Gun design keeps it from being flippy.
The only .45 I've had was a big frame ruger and it was a really sweet shooter (P90 I think it was). I bet it's a bullet size to pressure ratio that's favorable in the .45 and 9 but not in the .40. Just like bows, to each his own.
