One of the best things to do in Atlanta is to drive around through the various neighborhoods in Atlanta - Ansley Park, Virginia Highlands, Morningside, Druid Hills, Decatur, etc. There are some beautiful (and rather pricey) homes and areas of unique shops. Tour (in car or on foot) one of the area universities, like Emory or Georgia Tech (or even drive out to Athens and tour UGA).
The typical touristy destination like the World of Coca-Cola are available, and probably a good idea if you have young kids. You may also want to check out Zoo Atlanta.
Another good option is visiting one of the numerous local parks and taking a walk (you can rent rollerblades and bikes at a shop at the southwest corner of Pearson Park). The Atlanta Botanical Gardens are near Pearson Park, but are closed to the public on Mondays, so plan accordingly.
Quick Tips:
Get some input from an Atlanta native about what to see. If you don't know anyone from Atlanta, check out what some of the natives have to say on these boards. Also, try to eat out at least one night at a non-chain restaurant. Atlanta is full of wonderful eateries, from fast-food to fine cuisine.
If you're new to town and are driving, get a good map of the area. There are a LOT of streets named Peachtree - Peachtree St., W. Peachtree St., Peachtree Center Blvd, etc. and it can be a little confusing. Also, as with many downtowns, a lot of the roads run one-way, and you'll need to know how to get around the block if you miss a turn.
Best Way To Get Around:
A car is essential for getting around Atlanta, though traffic can be a real pain, especially if you need to get out of the city around 5:30. MARTA is great for getting to/from the airport, but overall the public transportation system is rather weak for such a large city. Also, if you do have to get around in rush hour and are travelling with someone else, head for the HOV (high occupancy vehicle) lane immediately. The HOV lane has its own exit lanes for major interstate splits, which is very beneficial.