The Wen Shan Hot Springs are the only hot springs I've ever been to in Taiwan that are still almost completely natural! There are three different temperature pools (hot, really hot, and scalding hot) at the bottom of Tarako Gorge next to the river. Some people even climb the fence and go down to the river to cool off (although the Chinese sign says you shouldn't). It's a great place to relax, soak, and enjoy the views. And the best part is, it's free!
There is a changing room halfway up the gorge, or you can just wear your swimsuit under your clothes.
Unfortunately, it's a little tricky to get there. It's a beautiful 45-60 minute walk. There is a bus that runs a couple times a day (check the schedule at the Tienshang bus station- they mostly run in the morning). But if you have your own car, that's the easiest way to get there.