x264 Pre scenecut Error

[ACoolie]

It's not a very significant bug, but with pre-scenecut on (or x264 is threaded), I get 38 of 51 frames as i-frames, and the others as p-frames. With pre-scenecut off I get only 3 i-frames. I observed the same 38/13 frame decision with threads=1 and pre-scenecut on.

The source is House Season 4 Episode 1 DVD. The clip is a dust cloud caused by a building collapsing.

This is the stats file and the x264 output:
http://pastebin.ca/1259842
Here is the source in y4m. I don't know how to frame-accurately cut a VOB (9MB): http://rapidshare.com/files/164818388/out.y4m.zip

[Dark Shikari]

Pre-scenecut tends to be more aggressive than scenecut, especially in the case of psy-RD, as psy-RD tends to bias against intra-blocks but is only taken into account in non-pre-scenecut.

It chose scenecuts there because those frames were basically completely uncorrelated: they were completely smoke-filled, leading to be basically nothing but grain, so this resulted in the entire sequence being encoded as I-frames.

Generally, I think pre-scenecut tends to be more accurate. It might be an interesting project, sometime, however, to make pre-scenecut more adaptive to the effects and properties of psy-RD.

Edit: Just tested, and even with psy-RD, there's no point in making them P-frames. The frames are over 80% i8x8 blocks at CRF18. It does get a bit less optimal at higher CRFs though.

[LoRd_MuldeR]

Isn't it expected to get a different result with pre-scenecut, compared to pre-scenecut disabled ???

Also: Why do you think it is a bug? 38 of 51 frames as I-Frames sound like very much, but does it actually look worse with that specific sample?

[ACoolie]

Thanks DarkShikari, I'll never doubt x264's pre-scenecut decision again! :p

@Lord_Mulder: I thought it was a bug because so many consecutive i-frames seemed suboptimal and the final bitrate of the pre-scenecut encode was over 2000kb/s greater than the scenecut encode.

[LoRd_MuldeR]

@Lord_Mulder: I thought it was a bug because so many consecutive i-frames seemed suboptimal and the final bitrate of the pre-scenecut encode was over 2000kb/s greater than the scenecut encode.

You must do 2-Pass encodes and compare two encodes of the same size/bitrate. Then decide which one looks better.
Looking at the size of an encode alone doesn't help anything. It's the "quality to bitrate" ratio that matters. And 2-Pass is the only feasible way to compare that.

A great number of consecutive I-Frames would only be sub-optimal for your source, if it looks worse at same bitrate/size.
If pre-scenecut produces a bigger file in CRF mode, then it's almost impossible to decide whether the increase in quality is worth the increase in size or not...

[Dark Shikari]

Thanks DarkShikari, I'll never doubt x264's pre-scenecut decision again! :pProbably a bad idea as well, it isn't perfect ;)

[foxyshadis]

It chose scenecuts there because those frames were basically completely uncorrelated: they were completely smoke-filled, leading to be basically nothing but grain, so this resulted in the entire sequence being encoded as I-frames.

This brings up a question I forgot to ask some time ago: When you have a run of uncorrelated frames like this, and you don't want to shovel a lot of extra bitrate into them, but don't want them to get all blurry, what can you do? I was thinking, encoder side, reduce deblocking so that it doesn't get too blurry on high quantizer, while selectively keeping high frequency bands for the illusion of a noisy source? (Blurring & blocking are far more noticeable than ringing at times like that, after all.) Without changing the encoder, I don't really know what to do even if I singled the frames out in a zone.

[Dark Shikari]

This brings up a question I forgot to ask some time ago: When you have a run of uncorrelated frames like this, and you don't want to shovel a lot of extra bitrate into them, but don't want them to get all blurry, what can you do? I was thinking, encoder side, reduce deblocking so that it doesn't get too blurry on high quantizer, while selectively keeping high frequency bands for the illusion of a noisy source? (Blurring & blocking are far more noticeable than ringing at times like that, after all.) Without changing the encoder, I don't really know what to do even if I singled the frames out in a zone.qcomp will already generally lower the bitrate of complex grainy sections like that.

[foxyshadis]

But doesn't that generally just make it blurry, with default matrix and sane AQ/RDO values?

[Dark Shikari]

But doesn't that generally just make it blurry, with default matrix and sane AQ/RDO values?Not really. As long as psy-RD has grain to grab from previous frames, things generally turn out fine as long as the quantizer doesn't go ridiculously high.