There are quite a few thermal baths in Budapest. One of them in particular I've always wanted to visit (Rudas) but it's off-limits to me as woman. *BaH* to male chauvinism. Here are the ones I have experienced first hand.
Very famous, very pretty, very everything. There are separate spa sections for women and men and some common swimming pools. The spas are very beautiful and tranquil. It's the norm to go naked (in the spa section ONLY, of course). A very liberating experience. Just a tad gross when you see pubic hairs floating in the water. *gag*
Very huge, very popular, very oh-look-the-old-men-are-playing-chess-like-in-the-photo. This is a pretty good bath too, but I found it rather impersonal. It's in the Varosliget, near the zoo and everything so maybe you could make it part of your day there.
Near Batthanyi Metro (which my friend thought was Hungarian for "bath" :)). It is alternatively open to women and men (call to find out). It's small and in bad need of renovation, but it's a "charming" example of Turkish architecture. The 4 of us had a splendid time and all got massages. My foot massage was oh-so-delicious but the whole-body massage I heard was painful rather than relaxing.
While at a bath, do try the 1) hot-bath, 2) freezing-cold tub, 3) sauna cycle - it's good for circulation and rather addictive after the first time. Trust me.