I have the same problem; Fast recompress has the same speed as Normal recompress (The only difference is that with fast recompress, I'm not able to see which scene is being encoded, and if I use normal recompress, disabling the "view in- and output" option, gets me some 1 till 2 FPS extra, ... Just as with Fast Recompress.)
The biggest difference I had, was putting in the 256MB DDR extra. (Avisynth is indeed rather greedy on RAM).
Anyway, I still haven't found the problem, but I'm happy with the speed I have now