This is the largest permanent market in Oaxaca City (the Mercado Abastos is the enormous one on Sundays). You can find anything here, from toasted grasshoppers to handmade rugs. Seemingly endless rows of stalls are filled with anything you might need or want while staying in Oaxaca, or as gifts or souvenirs.
Walk two blocks south from the Zocalo to Flores Magon Street. The market takes up an entire city block, and you can't miss it.
The local restauranteurs buy their chiles here, the hotels buy their rugs and curtains here. If you don't speak Spanish, just bring a dictionary. A portable calculator is helpful as well, as you can just show the shopkeeper the amount you want to spend or what you think they said.
When shopping, please keep in mind that Oaxaca is the poorest state in Mexico. Haggling is expected, but that dollar means a lot more to them than to you.
Also keep in mind the US customs laws -- dried goods such as chiles and coffee are find to bring into the states, but you might get in trouble if you try and bring home some Oaxacan string cheese . . .