Internet access in New Zealand is definitely not hard to come by. Even the smallest towns--those towns with hostels but no other amenities like food or petrol that I have mentioned in other reviews--will generally have at least one establishment that offers internet, whether it be a dedicated internet cafe, a restaurant, or your hostel itself.
The quality of these computers & internet connections and the price you will pay for them varies greatly throughout the country. In the bigger cities, like Christchurch, Nelson, and Queenstown, you will easily find internet cafes that offer prices as low as $3/hour (either post-pay or by buying a pre-paid card for a set number of hours). These cafes are generally pretty well-equipped; many will either offer USB access on all computers or have special computers set aside where users can connect to USB and burn CDs or DVDs. In Nelson, for instance, the internet cafe that I found had 6 computers with this special access, all of which were first-come, first-served. The connections here can sometimes be slow, depending on the load (many cafes have upwards of 30 computers), but they are the fastest you'll get in the country.
Outside the cities, internet and computer access are a whole different kettle of fish. You'll count yourself as lucky if you find anywhere that offers access for $4/hour; many places charge $1/10 minutes, or $6/hour. I even saw one place--the only lodge on the Abel Tasman Track--that offered access for a whopping $10/hour (although why you'd be on the internet when in Abel Tasman National Park is beyond me). Also, the facilities are always a lot smaller; many hostels will only offer one computer (although YHAs normally have a bank of around 4 computers, for which you buy blocks of time from the reception counter), so you often have to spend a bit of time waiting around while the person before you finishes their business.
Many public computers--especially those in hostels--have protections on them so they cannot be easily destroyed by malicious users. However, these protections come at a cost to the normal user. Often, USB ports are completely blocked, since owners make the CPU unit inaccessible and sometimes do not provide USB extension cords. This means that you can't upload your pictures at all, which is frustrating if that's what you've signed on to the computer to do! It's always best to ask if any computers have USB access or card readers before choosing one. Other protections put on computers include blocking the user from easily accessing Explorer (again, making it difficult to view your pictures) and only allowing users to have one IE window open at a time (making it difficult to multi-task and minimize dollars and time spent online). Even with these restrictions, though, I still managed to find enough computers that allowed me access to USB to back up my pictures onto my iPod relatively regularly, meaning I never worried about losing all of them. I also backed pictures up to DVDs (which I bought at OfficeMax at a price of $8/5 DVDs--much better than the $2/CD price offered at internet cafes) while in Nelson and Queenstown, since I knew this sort of access would be limited in other areas.
Overall, computer access throughout New Zealand is widespread; you won't have to worry about being out of contact for long periods of time unless you go out into the backcountry. This means that there is no need to take a laptop, especially if you're concerned about its security in hostels or space in/weight of your pack. Also, wireless often costs just as much as using the computers at the internet cafe, so you're still going to have to fork out for internet access.