If you only have one afternoon in Budapest, it should be spent in the Castle District on Castle Hill. Most of the key monuments and museums (and therefore unfortunately tourists) are here, alongside a quite magnificent view of Pest across the Danube to the Parliament and beyond (see photo below). The District is contained within walls and is split into two fairly distinct areas, namely the Old Town and the Royal Palace.
The Old Town is charming -- painted houses, decorative churches (chief of which is St Matthias or Matyas) and the Fishermen's Bastion (again, great for views although there is an additional charge to go up there and the view is not so very much improved -- you may be better spending the entry fee on a coffee in the cafe that sets out tables under the balustrade formed by the Bastion and enjoy the panorama from there).
The Bastion was originally designed to be a viewing platform, and was built surprisingly recently (1905). Apparently it was named to honour the medieval guild of fishermen who defended this stretch of wall. It comprises seven gleaming white turrets (where you climb up) which represent the seven Magyar tribes who entered the Carpathian Basin in the 9th century.
In the square outside the church is a great statue of St Stephen on horseback - look carefully at the carved frieze on the plinth as well as at the statue itself. Stephen is famous for having brought together the Magyar fiefdoms into one state in c.990. On his coronation in Christmas Day 1000, he received an apostolic cross and crown from the then-Pope, Slyvester II, which have remained the symbol of Hungary (even during Communist times). He was canonised soon after his death in 1038 and the mummified relic of his right hand was famous in Hungary for its holy, healing qualities. See photo below.
Then, there's the church itself (entry 9-5 200Ft) -- a striking riot of colour both inside and out. The spiky, angular roof and spire bear colourful diamond painted tiles and every inch of the interior -- domes, walls, columns -- seems to have been painted with leaves or clouds or geometric designs; there are marble and gilded statues and bright, stained-glass windows. Pride of place in the centre, behind railings, are the tombs of Bela III and his consort. I'd give the crypt a miss and head instead up into the Royal Oratory where there's a copy of Stephen's crown (the original is the Parliament) with a small but excellent exhibition explaining what it was for, who is portrayed on the many perfect miniatures on tiny cameos and so forth.
Back outside, there's a quaint little street -- look out for the Golden Eagle pharmacy (signposted as you'd expect by a large gilt bird). They don't get many visitors judging by the enthusiasm with which they pounced on us and gave us 1 to 1 attention around the small museum. The curator is a retired chemist and talked us through each and every piece (gets a little wearing after a while) from the large marble pestle and mortar and inky black cauldrons to the primitive microscope designed by a Dominican nun whose portrait hangs on the inside wall.
The second part of the area is filled by the Royal Palace -- it has undergone various incarnations since the 1300s and the remaining castle today dates from partly 1700s/partly 1900s constructions, merged and consolidated after WWII. As well as impressive from the outside, it's home to the National Gallery (mainly dedicated to Hungarian art), the National Library and Budapest History Museum. Just behind the palace building you can access the gardens (via Ferdinand Gate under the conical Mace Tower, you'll see a flight of steps down to a historic Turkish cemetery with graves dating back to the Independence battle for Buda in 1686).
To get to the Royal Palace, take the Siklo, a funicular dating from 1870, from Clark Adam; the more energetic will enjoy the walk up the Royal Steps (the wide staircase that goes to the southern end of the Royal Palace).