The French Market is at the end of Decatur Street. It's a large covered market and a good place to get cheap souvenirs, especially on Saturday when there are even more stalls.
It has all the usual souvenir stuff--T-shirts, voodoo dolls, stuffed alligator heads, beads--but at cheaper prices than the souvenir shops. It also has some surprisingly good and original craft stalls that don't just sell the usual tat, but also silver jewelry at bargain prices.
There's a food market nearby where you can get all the usual New Orleans hot sauces and spices at cheaper prices than in the shops. Theres are also fresh fruit and vegetable stalls.