x264: First IDR frame of a CBR encode looks horrible

[Guest]

I am happy to report that my 2-pass encode without stuffing just finished. I had asked for a VBV model as follows:

buffer size: 6000000
initial fill: 5400000
bitrate: 10000000

The encode finished with a reported bitrate of 9588000.

The encode finished without any underflows, either during the first pass or the second. My standalone ES VBV checker shows that in fact a bitrate of 10002000 is required to prevent underflow. The small discrepancy is apparently due to x264 not taking into account the AUD overhead when --aud is specified. In any case, it's not significant because when muxing to transport a margin is typically allowed.

Inspecting the result for I-frame flicker reveals that it is substantially eliminated but not totally. There are still some challenging scenes where it is apparent if you know what to look for. I will isolate one such scene and upload a source sample. I plan to implement a flicker analyzer tool and investigate the causes with a view to eliminating this remaining problem.

Some kind of VBV lookahead would be very useful to allow acceptable one-pass VBV-compliant encodes.

[Dark Shikari]

There are multiple types of I-frame flicker that I've encountered during testing.

1. The most obvious: an I-frame is encoded at a much higher QP than x264 wants because of VBV restrictions.

2. A bit less obvious: I-frames in a section of mostly static video are encoded at varying QPs due to varying complexity of the video. This is especially problematic when part of the frame is static and part is not, the former being the place where you notice the quality jumping around, and the latter being why VBV is forced to change the quantizer.

3. Even less obvious: intra prediction inherently codes things differently from inter prediction, a fact which can actually come out to be a big issue in static scenes at higher quantizers. I have samples that reliably cause noticable "jumping" on keyframes even without VBV. The only way I can imagine to solve this is to encode each non-scenecut keyframe as a P-frame, and then re-encode as an I-frame with the "source" image being replaced by the reconstructed version of the P-frame.

[Guest]

1. The most obvious: an I-frame is encoded at a much higher QP than x264 wants because of VBV We can see that from --verbose output, comparing it to the previous P frame.

2. A bit less obvious: I-frames in a section of mostly static video are encoded at varying QPs due to varying complexity of the video. This is especially problematic when part of the frame is static and part is not, the former being the place where you notice the quality jumping around, and the latter being why VBV is forced to change the quantizer. I am seeing this in fully static scenes, typically darker ones. I will post a source sample.

3. Even less obvious: intra prediction inherently codes things differently from inter prediction, a fact which can actually come out to be a big issue in static scenes at higher quantizers. I have samples that reliably cause noticable "jumping" on keyframes even without VBV. The only way I can imagine to solve this is to encode each non-scenecut keyframe as a P-frame, and then re-encode as an I-frame with the "source" image being replaced by the reconstructed version of the P-frame. That solution is one described in the literature. Anyway, I don't think this case is involved, as the quantizers are not that high. The problem is that the P and following I quantizers are too different. As I said, we can explore this more with my sample and yours, if you are willing to make them available.

Make no mistake, though, the encode is stunning; I need special tools to see the flicker. If it can be fully eliminated, stunning becomes absolutely jaw-dropping.

EDIT: I muxed it to transport with the audio, specifying a video bitrate of 10002000. Everything proceeded without error and the file plays perfectly.

[Sagittaire]

Possible to have the source?

[Gabriel_Bouvigne]

3. Even less obvious: intra prediction inherently codes things differently from inter prediction, a fact which can actually come out to be a big issue in static scenes at higher quantizers. I have samples that reliably cause noticable "jumping" on keyframes even without VBV. The only way I can imagine to solve this is to encode each non-scenecut keyframe as a P-frame, and then re-encode as an I-frame with the "source" image being replaced by the reconstructed version of the P-frame.
I know about an encoder which used to do that, and I was not that pleased by the result. Of course, it significantly reduces flickering, but it also significantly decreases overall resolution.
There are also a few alternative ways to cure this, like:

*increase a bit the QP difference between I and P frames. At about 5, flickering on static scenes is significantly reduced. (with perhaps the addition of I MB within P frames being coded at about the same QP as the one of I frames)

*running a fast P-skip test on I frames. If the MB would have been coded as skip within a P frame, then instead of computing distortion as (distortion_of_current_MB), compute it as (a*distortion_of_current_source_MB + b*distortion_to_previous_reconstructed_MB).

[Guest]

*increase a bit the QP difference between I and P frames But all my experiments show that it is that very difference that constitutes the flicker. Did I misunderstand you?

[Guest]

Possible to have the source? It's Baraka BluRay disk. As I said, I'll upload a sample for the flicker case.

[Gabriel_Bouvigne]

But all my experiments show that it is that very difference that constitutes the flicker. Did I misunderstand you?
Of course Qp of I frames should be lower than Qp of P frames, unlike in the log you posted. Of course, if Qp of P frames is lower than Qp of I frames, it's likely to flicker on the I frame.

[Guest]

I'll experiment to see if it really matters whether I > P or P > I. I suspect either can lead to flicker if the difference is big enough.

[CruNcher]

Does somebody know how Ateme avoids this with their I-Frame flicker Patent, they also use Lookahead since years already so i guess this already lowers the possible I-Frame flicker especially in short GOPs 1 Pass Encoding scenarios (Broadcast) significantly ? at least every Quality Evaluation shows that.

[Guest]

Does somebody know how Ateme avoids this with their I-Frame flicker Patent Patents are public. Do you know the patent number?

[Gabriel_Bouvigne]

I'll experiment to see if it really matters whether I > P or P > I. I suspect either can lead to flicker if the difference is big enough.
Of course it matters.
If P > I, your P picture will be better quality than your I picture. It means that you'll have a low quality I picture, quickly refined within the next P frames. That will create a visual "pulse" around the I frame.

[Guest]

(I was referring to quantizer values.)

If the I picture is quantized much less than the preceding P, you can also get a pulse.

[cacepi]

Patents are public. Do you know the patent number?I only found two patents assigned to Ateme, both at the European Patent Office; one about CABAC and this one:

METHOD AND DEVICE FOR ADJUSTING THE ENCODING FLOWRATE OF VIDEO IMAGE SEQUENCES RELATIVE TO A TARGET FLOWRATE (http://v3.espacenet.com/publicationDetails/biblio?KC=A2&date=20081106&NR=2008132378A2&DB=EPODOC&locale=en_EP&CC=WO&FT=D)

Might be the one, but as only the abstract is available in English I can't say for sure.