If you do get over to Windsor, you should take a look at the Windsor-Cornish Covered bridge over the Connecticut River. While I'm apparently immune to the charms of the covered bridge, many people get very excited, perhaps more so since "The Bridges of Madison County" came out.
If coming from the south of Windsor you'd be driveling north on Route 5. At the first stop light in Windsor, turn right on Bridge Street and it's about a quarter mile down.
For photographers, there's a nice angle from the New Hampshire side where you can get the bridge in the center of your picture with Mount Ascutney in the background.
As long as you're on the New Hampshire side, look around for reclusive author J.D. Salinger ("Catcher in the Rye", "Franny and Zooey"). He lives in Cornish and occasionally shows up in Windsor.