I think by the way that you have briefly described yourself, you might best enjoy the Jeep Wrangler. It's a very versatile Jeep, and it's a fun cruising vehicle, especially in the summer. Also, the obvious Jeep-ing is the most fun you can have with it. Don't even get me started on "customizing" a Jeep.
But because you mentioned that you drive a lot, you have to ask yourself if you'd better benefit from better gas mileage, or a cool car that can have its roof and doors removed, and that can run over things and park in the coolest places. Either way, it'll be a tough choice. Plus, everyone knows the Wrangler is the coolest Jeep.
I have always been a Jeep lover and hope to get a Wrangler Unlimited in my near future. I personally just find the 4 doors much more appealing than the 2 doors.
I never liked the look of the Patriot, so I'm sorry that this post was very biased.