SSIM based rate control

[cogman]

So, I have noticed that the SSIM measurement at the end of the film is almost always consistently what I see the quality as. However, sometimes the CRF value used will produce a varrying effect on the film.

I was wondering, How hard (and possible) would it be to have a target SSIM value for the rate control? I was thinking of something to the effect of taking the SSIM sample at the end of every ten frames (I believe x264 does this already) and then raising or lowering the CRF value based on the SSIM value to attempt to achieve a targeted SSIM value.

Would that be of any worth? Or is there something like that already implemented (I'm grossly uninformed about how CRF work, so It may already be implimented) Thanks for considering.

[LoRd_MuldeR]

Maybe RDRC can help to further improve CRF mode, in order to bring it closer to constant SSIM:
RCRD (rate-distortion optimized ratecontrol) - lower quants on frames that have a greater influence on future frames, higher quants on frames that have less influence

[RaynQuist]

http://forum.doom9.org/showthread.php?t=128421
I'm not sure what happened there but seems like cssim didn't work out.

[Dark Shikari]

Maybe RDRC can help to further improve CRF mode, in order to bring it closer to constant SSIM:No, it has the opposite effect; RDRC tends to make some frames higher quality than others in order to have maximum temporal effect.

[LoRd_MuldeR]

No, it has the opposite effect; RDRC tends to make some frames higher quality than others in order to have maximum temporal effect.
Oh! I'd assume raising quality in some frames while lowering quality in others won't effect the average SSIM much.
Furthermore I'd assume raising the quality in "important" frames (referenced intensely) should even improve the global SSIM...
Bummer!

http://forum.doom9.org/showthread.php?t=128421
I'm not sure what happened there but seems like cssim didn't work out.

BTW: Is "Target SSIM" mode dead?

[Dark Shikari]

http://forum.doom9.org/showthread.php?t=128421
I'm not sure what happened there but seems like cssim didn't work out.That was constant SSIM on a per-block basis.

[cogman]

Humm, so if I understand the limitations of cssim correctly, x264 would assign too much bitrate to high complexity areas and not enough to the lower complexity areas where it would potentially need it more (IE, big dark blue sky with some fog moving in the background, making lots of blocks)

I guess doing it on an entire frame would probably not give any better results. Though, maybe if having a constant crf value with a maximum change in crf and a minimum crf value for x frames might make x264 handle high motion areas a bit better.

Or, Maybe taking into account the PSNR rating as well might balance things out.

Basically, (if I didn't explain myself very well) the order of operation as I see it would be

1. encode the frame at a given crf.
2. get the SSIM value of the frame (or 10 frames if that is how it already works)
3. adjust the crf value for the next frames up or down to hit the targeted SSIM value.

That would be the fast version of it, the slow version would involve re-encoding the frames until a given SSIM value is acheived. Perhaps it could even be just a minimum SSIM value (nothing to prevent you from having a 1.)

But, again, cssim looks as if it might have completed the same functionality, so this might be a worthless suggestion. (hey, at least cssim was recommended only 6 months ago, :D)

[Sagekilla]

The VAQ patch currently solves the issue you're talking about (dark areas, like blue sky with fog moving around) And as for high complexity areas getting more bitrate, it does the exact opposite thing. CRF mode raises q on high complexity, lowers on low complexity.