2-pass picture quality problem

[acdarc]

Hi,

I have a 14G raw vid captured with Fraps, and I packed it with x264 2-pass.

I ran:

x264.exe --pass 1 --bitrate 1500 -o outfile_p1.mp4 video.avs
x264.exe --pass 2 --bitrate 1500 -o outfile_p2.mp4 video.avs

video.avs consists:
###
AviSource("infile.avi")
ConvertToYV12()
###

The problem is that the 1st pass file is smaller, and _far_ superior in quality. The 2nd pass file first fade-in looks absolutely horrifying. It's full of blocks, tiling etc. And its 3mb's bigger than the 1st pass. In overall quality it's like its ran with 30% of the original bitrate.

What did I do wrong? I didn't mess up the bitrate-settings in the middle etc. Previous videos I've done the same way have been quite fine.

x264 version is: core:78 r1318 fe83a90

This is what the encoding looked like in the command prompt:

###################################

>x264 --pass 1 --bitrate 1500 -o outfile_p1.mp4 video.avs
avis [info]: 960x600 @ 25.00 fps (15828 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.1 Cache64
x264 [info]: profile Main, level 3.1
mp4 [info]: initial delay 1 (scale 25)
x264 [info]: frame I:66 Avg QP:26.73 size: 70478
x264 [info]: frame P:8659 Avg QP:31.04 size: 10921
x264 [info]: frame B:7103 Avg QP:36.05 size: 2350
x264 [info]: consecutive B-frames: 15.6% 70.3% 8.3% 5.9%
x264 [info]: mb I I16..4: 28.0% 0.0% 72.0%
x264 [info]: mb P I16..4: 15.5% 0.0% 0.0% P16..4: 39.9% 0.0% 0.0% 0.0% 0.0% skip:44.6%
x264 [info]: mb B I16..4: 1.6% 0.0% 0.0% B16..8: 14.2% 0.0% 0.0% direct: 5.9% skip:78.3% L0:33.6% L1:50.1% BI:16.3%
x264 [info]: final ratefactor: 28.47
x264 [info]: coded y,uvDC,uvAC intra: 48.8% 53.3% 23.2% inter: 12.7% 11.1% 1.4%
x264 [info]: i16 v,h,dc,p: 20% 35% 26% 19%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 17% 29% 14% 6% 7% 5% 9% 5% 9%
x264 [info]: kb/s:1464.62

encoded 15828 frames, 25.54 fps, 1464.62 kb/s

>x264 --pass 2 --bitrate 1500 -o outfile_p2.mp4 video.avs
avis [info]: 960x600 @ 25.00 fps (15828 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.1 Cache64
x264 [info]: profile High, level 3.1
mp4 [info]: initial delay 1 (scale 25)
x264 [info]: frame I:66 Avg QP:26.92 size: 72312
x264 [info]: frame P:8659 Avg QP:31.39 size: 10723
x264 [info]: frame B:7103 Avg QP:36.22 size: 2972
x264 [info]: consecutive B-frames: 15.6% 70.3% 8.3% 5.9%
x264 [info]: mb I I16..4: 13.0% 48.0% 39.0%
x264 [info]: mb P I16..4: 2.5% 7.2% 1.8% P16..4: 27.9% 10.4% 5.3% 0.0% 0.0% skip:45.1%
x264 [info]: mb B I16..4: 0.2% 0.7% 0.1% B16..8: 23.9% 1.1% 1.3% direct: 3.4% skip:69.2% L0:38.2% L1:55.3% BI: 6.5%
x264 [info]: 8x8 transform intra:62.3% inter:66.3%
x264 [info]: coded y,uvDC,uvAC intra: 58.2% 62.3% 27.7% inter: 15.4% 13.1% 1.9%
x264 [info]: i16 v,h,dc,p: 23% 45% 9% 23%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 18% 20% 6% 8% 7% 11% 6% 10%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 16% 26% 17% 6% 8% 6% 9% 5% 7%
x264 [info]: ref P L0: 78.5% 11.9% 9.6%
x264 [info]: ref B L0: 87.9% 12.1%
x264 [info]: kb/s:1500.33

encoded 15828 frames, 20.82 fps, 1500.33 kb/s

###################################

Thank you for any assistance.

[nurbs]

You use an old version of x264 from before weighted p-frame prediction, which helps a lot with fades, was committed.

[Dark Shikari]

Start by updating your x264. If you still have a problem then, upload a sample. We aren't psychic and can't figure out your problem if we can't see it.

[acdarc]

Okay, this is odd.

I updated the x264 and encoded the file again. There was a significant improvement on the fade in, but still it was much more tiled than the no-pass-version.

Now, I tried to make a sample for you, and I enclosed a region on sony vegas and saved the spot as another avi w/o audio or anything other. Just a ~8 second clip.

When i encoded that clip with no-pass and 2-pass, they were almost identical, however this time the 2pass was a smaller file.

It seems that the scene I'm talking about is so fragile to compression, it looks immediately tiled (guessing the fog effect in the scene) when the encoder tries to determine what to do with it when the video is much longer.

Here are the clips:
http://acd.ydinsota.net/sample_nopass.mp4
http://acd.ydinsota.net/sample_pass2.mp4

I couldnt get virtualdub to grab video from the distorted file, but here are a few snapshots from VLC:

http://acd.ydinsota.net/vlcsnap-205035.png
http://acd.ydinsota.net/vlcsnap-204102.png

Those snapshots are from the exact same spot rendered and encoded with the same options as those clips above, the only difference is that when the video was full length, the beginning becomes really distorted.

Hope you make sense of this and tell me where I'm doing wrong :).

[Dark Shikari]

It isn't a sample if it isn't a part of the problematic video. The sample you posted has no problems whatsoever.

A sample means "take the video with problems, and cut it", not "encode a new video that doesn't have any problems".

[acdarc]

Any tools you can recommend I could use to make a sample of the ready file? It seems that VirtualDub doesn't do the trick.

[nm]

MP4Box can split MP4 files. Use Yamb (http://yamb.unite-video.com/) if you need a GUI.

[acdarc]

Okay, I got the sample. As you can see, there is much more tiling etc. This is a 2 pass 1.5m bitrate, the exact same file as the clips above but taken from a full 10 min length video.
The options I used to encode this are the same as the previous clips that didnt have any problems.

http://acd.ydinsota.net/sample_2pass_tiled.mp4

[Dark Shikari]

I don't see any "tiling" or anything nearly as bad as the images you posted above.

The intro is slightly lower quality than one might want, though that's probably due to the misprediction inherent at the beginning of a 2-pass encode, especially one in which the first pass was not done with slow-firstpass. But it's not a very big effect effect visually as far as I can tell...

[acdarc]

I don't see any "tiling" or anything nearly as bad as the images you posted above.

The intro is slightly lower quality than one might want, though that's probably due to the misprediction inherent at the beginning of a 2-pass encode, especially one in which the first pass was not done with slow-firstpass. But it's not a very big effect effect visually as far as I can tell...

I guess the tiling was due to the reason I had to pause the video and as it didn't flow it looked much worse than it was.

I'm not saying that its 'very big' as such, the reason I posted this in the first place was the fact that 2 pass looked worse than singlepass, and took more space. That didn't make sense.

I'll try with the veryslow now.

[Dark Shikari]

I guess the tiling was due to the reason I had to pause the video and as it didn't flow it looked much worse than it was.

I'm not saying that its 'very big' as such, the reason I posted this in the first place was the fact that 2 pass looked worse than singlepass, and took more space. That didn't make sense.Of course it makes sense: the bitrate was allocated differently. That particular part of the video was given far less bits in 2-pass, and so it looked a lot worse by comparison.

[acdarc]

Of course it makes sense: the bitrate was allocated differently. That particular part of the video was given far less bits in 2-pass, and so it looked a lot worse by comparison.

Yeah, I know that's the point of the 2-pass. My original assumption was just that scenes with loads of things happening (like that, with the fog effect) would get more priority.
Of course it's impossible for the pass algorithm to read my thoughts :). The thing that came to mind is that it would be fun to be able to prioritize the bitrate yourself aswell. You could tell it that frames 50-150 need special attention :P.

[nm]

The thing that came to mind is that it would be fun to be able to prioritize the bitrate yourself aswell. You could tell it that frames 50-150 need special attention :P.
You can use zones (http://mewiki.project357.com/wiki/X264_Settings#zones) for that, but I don't know if there's a GUI that helps setting them.

[poisondeathray]

I suspect disabling mb-tree will help with that and sections like that

[acdarc]

I tried with the veryslow-option. It smoothed out it a bit. The zone-thingie looks exactly what I've been looking for, and there seems to be presets for animation and other kind of sources. I'm sure I'll find a good combination for this type of videos :).

Thank you everyone for your assistance, again.

[kypec]

Let us know your findings once you get satisfying results on your encodes so that others may benefit also. ;)

[creamyhorror]

I guess the tiling was due to the reason I had to pause the video and as it didn't flow it looked much worse than it was.
Take some screenshots of the exact issue in the full encoded file? I'm half-wondering if yours is a decoder issue.

[nm]

Take some screenshots of the exact issue in the full encoded file? I'm half-wondering if yours is a decoder issue.
There are links to screenshots above. Doesn't look like a decoder issue.