Rear Sway?

Im tired of the weak side to side on my Cougar. If I was to remove the sway bar, would the only neg. be the body roll or could I damage something?