Just after the bridge on the left by the Lobster House is the marina (before you get into the Cape May center of town) and here you can go parasiling, rent jet skiis and wave runners, catch a whale or dolphin watching cruise, or sign up for a fishing charter.
Tip: Hit this place on your way in or out of Cape May so that once you're in Cape May you don't have to get in your car again. If you want to eat at the Lobster House, now is the time to do that too (eat at a table on the Schooner...but make reservations!)