A bathing suit for the hot springs
Layers of clothing - Arequipa is a bit warmer than Cusco but Chivay is definitely colder, especially at night!
Shampoo, etc - You won't want to shower at the hostels in Chivay, which tend to only have very cold water, but you can shower at the hot springs.
Bring food for the long bus trips!
Extra money - Sometimes several tour groups are all put together, and what was promised to one group (in terms of meals, etc) wasn't promised to another. Consequently, you may end up paying more than you expected, and it can be difficult to change money in Chivay. Besides, maybe you'll want money for drinks at the Irish pub or the hot springs (see my other entries for details!).