There are many libraries related to parallel programming to make life easier, or at least more interesting, for you.
crumb trail: > libraries > SimGrid
SimGrid [simgrid-homepage] is a simulator for distributed systems. It can for instance be used to explore the effects of architectural parameters. It has been used to simulate large scale operations such as HPL [cornebize:EmulateHPCms] .
crumb trail: > libraries > Other
ParaMesh
Global Arrays
Hdf5 and Silo