Ah, markets: the haggling, the bargains, the choice, and the crowds. Well, two out of four isn't too bad.
Open 5 days a week, the Victoria Market has pretty much whatever you are looking for and a lot of stuff you aren't. The outdoor portion of the market has two sides, the clothing and gifts side and the fruit and veggie side. Across the street you'll find an indoor market filled with meats, fish, breads and specialty and organic food.
Much of the clothing is what you might expect in a market. It's not the best quality and the vendors don't really like to haggle. Many of the items don't have price tags, but when asked, the price quoted becomes a "fixed price." Best buys on this side of the market are the gifts and souvenirs. Winter hats with Australia logos can be found for $5. If you are looking for something a bit nicer, there is a stall that sells black-and-white pictures from around Melbourne. There are a range of sizes, and they are not terribly overpriced for an framed photo.
The reason for my highly recommended is based on the fruit and veggie side. Here is where you find the real deals: a dozen free range eggs for $3and 10 kiwifruit for $2, and if you hit the stalls around midday or 1pm, chances are you can get even better deals, like whole trays of tomatoes for $4. In my experience, since I don't need a whole tray, I just tell them what I do need and most times I get one or two of what I need for free.
This is a great alternative to the grocery stores, where produce seems to be marked up sky-high. Also, check out the market across the street for deals on meats and fresh fish. There are also a few bread stalls, but I've found that these aren't that much of a bargain.
One last suggestion: Bring a backpack. Before I got wise, I always left weighed down by heavy plastic bags. With a backpack, it's much easier to get back home on your bike.