Do you get that with all files? People have reported
similar problems at the nV News forum so it's probably a bug in the MPlayer/libavcodec code or the driver.
Edit: driver version 180.08 was just released with some fixes to VDPAU, so you might want to try it.