Test the improved ratecontrol

[Dark Shikari]

Its not that much better, but every bit counts.

Test the new 2pass ratecontrol modifications; speed should be almost exactly equivalent to the original.

Original build, for comparison: Linkage (http://mirror05.x264.nl/Dark/force.php?file=./x264_orig.exe)

Modified build: Linkage (http://mirror05.x264.nl/Dark/force.php?file=./x264_new.exe)

Note that you might need PthreadGC2.dll (http://mirror05.x264.nl/Dark/force.php?file=./pthreadGC2.dll) if you don't have it, for pthreads support. Also note that this change makes all first pass files from the new version incompatible with older versions, and vice versa. Don't mix firstpass files.

Visual difference shouldn't be too major, it just moves around frame quantizers a bit among P-frames. And of course note its 2pass-only, not CRF, not CQP. Firstpass doesn't change at all, it only writes more data (which the second pass uses). In my tests I got a boost of about 0.025 - 0.15 OPSNR depending on source, equivalent to 0.5-3% bitrate.

Build is based on SVN715.

[Terranigma]

Will give it a go a.s.a.p. :)

[Terranigma]

Here Goes:
(note, that after the first pass, the only difference in my settings are between the number of reference frames used and subme level.)

Pass 1 (Turbo Pass):
http://img150.imageshack.us/img150/1125/pass1turbopasskc1.png

Pass 2 (direct auto, subme 6, ref 5):
http://img524.imageshack.us/img524/5743/pass2normalpasssubme6gk6.png

Pass 3 (direct auto, subme 7, ref 6):
http://img245.imageshack.us/img245/5811/pass3normalpasssubme76rxc7.png

Pass 4 (direct auto, subme 7, ref 7):
http://img239.imageshack.us/img239/8562/pass4normalpasssubme77rdd1.png

Pass 5 (direct auto, subme 7, ref 8):
http://img408.imageshack.us/img408/2889/pass5normalpasssubme78rbx8.png

Pass 6 (direct auto, subme 7, ref 9):
http://img408.imageshack.us/img408/4963/pass6normalpasssubme79rlo9.png

Pass 7 (direct auto, subme 7, ref 10):
http://img523.imageshack.us/img523/9901/pass7normalpasssubme710ad9.png

I'm not sure, but it seems like the quants for the 2nd pass is higher than all other passes, and I believe that's due to direct auto using more temporal frames than the other passes.

[Dark Shikari]

Where's the PSNR/SSIM values, and the comparison to the original?

This isn't improved ratecontrol as in improved control of bitrate, its a change in ratecontrol to improve overall PSNR slightly.

[Terranigma]

Where's the PSNR/SSIM values, and the comparison to the original?

This isn't improved ratecontrol as in improved control of bitrate, its a change in ratecontrol to improve overall PSNR slightly.

Oh yea, I need to enable that. Knew there was something I was forgetting, lol.

[IgorC]

Dark Shikari
Is it optimization based purely on any metric? If yes, then which one?
And could you say what exactly has changed?

[Dark Shikari]

Dark Shikari
Is it optimization based purely on any metric? If yes, then which one?
And could you say what exactly has changed?Its not metric-based. All it does is save the data about which frames are referenced the most (list0-only) and raise quants slightly on rarely referenced frames and lower quants slightly on more commonly referenced frames.

[Terranigma]

Update. I increased the bitrate, and changed direct auto to spatial to prevent unbiased results.

Pass 1 (Turbo Pass):
http://img263.imageshack.us/img263/6871/pass1turbopassza9.png

Pass 2 (subme 6, ref 5):
http://img139.imageshack.us/img139/6688/pass2normalpasssubme6vi7.png

Pass 3 (subme 7, ref 6):
http://img139.imageshack.us/img139/9716/pass3normalpasssubme76rlc3.png

Pass 4 (subme 7, ref 7):
http://img139.imageshack.us/img139/3158/pass4normalpasssubme77rth0.png

Pass 5 (subme 7, ref 8):
http://img124.imageshack.us/img124/9707/pass5normalpasssubme78rnj4.png

Pass 6 (subme 7, ref 9):
http://img258.imageshack.us/img258/7669/pass6normalpasssubme79rwa9.png

Pass 7 (subme 7, ref 10):
http://img124.imageshack.us/img124/408/pass7normalpasssubme710wn4.png

[Dark Shikari]

Update. I increased the bitrate, and changed direct auto to spatial to prevent unbiased results.

Pass 1 (Turbo Pass):
http://img263.imageshack.us/img263/6871/pass1turbopassza9.png

Pass 2 (subme 6, ref 5):
http://img139.imageshack.us/img139/6688/pass2normalpasssubme6vi7.png

Pass 3 (subme 7, ref 6):
http://img139.imageshack.us/img139/9716/pass3normalpasssubme76rlc3.png

Pass 4 (subme 7, ref 7):
http://img139.imageshack.us/img139/3158/pass4normalpasssubme77rth0.png

Pass 5 (subme 7, ref 8):
http://img124.imageshack.us/img124/9707/pass5normalpasssubme78rnj4.png

Pass 6 (subme 7, ref 9):
http://img258.imageshack.us/img258/7669/pass6normalpasssubme79rwa9.png

Pass 7 (subme 7, ref 10):
http://img124.imageshack.us/img124/408/pass7normalpasssubme710wn4.pngWell we need a comparison to the original for such numbers to be useful ;)

[Terranigma]

Well we need a comparison to the original for such numbers to be useful ;)

Comparison to the original? Are you talking about the actual encode and source clip?

[Dark Shikari]

Comparison to the original? Are you talking about the actual encode and source clip?No... I mean comparing the original x264 and the modified x264. Its impossible to judge the effectiveness of a change unless you compare it to the same encode without that change.

[Terranigma]

No... I mean comparing the original x264 and the modified x264. Its impossible to judge the effectiveness of a change unless you compare it to the same encode without that change.

Oh yea, sorry about that. I was confused there for a second, and actually forgot all about that other .exe you supplied. :)

[Terranigma]

With original x264


Pass 1 (Turbo Pass):
http://img155.imageshack.us/img155/434/pass1turbopasshd4.png

Pass 2 (subme 6, ref 5):
http://img523.imageshack.us/img523/4584/pass2normalpasssubme6zz0.png

Pass 3 (subme 7, ref 6):
http://img155.imageshack.us/img155/4130/pass3normalpasssubme76ruz5.png

Pass 4 (subme 7, ref 7):
http://img231.imageshack.us/img231/6135/pass4normalpasssubme77rvn5.png

Pass 5 (subme 7, ref 8):
http://img231.imageshack.us/img231/1230/pass5normalpasssubme78rnf3.png

Pass 6 (subme 7, ref 9):
http://img231.imageshack.us/img231/2014/pass6normalpasssubme79rak6.png

Pass 7 (subme 7, ref 10):
http://img86.imageshack.us/img86/1738/pass7normalpasssubme710gs5.png


Seems like the original won; don't know why though.
I'm just a guinea pig. :D

[Sagekilla]

I don't mean to dig up an old thread. But I was reading this and thinking: Wouldn't it be possible to implement a similar RC (Or would RD be the proper term?) in CRF mode?

I was thinking something along the lines of exactly what you did: Store which frames were referenced most internally, lower qps on most used and raise on least used. It seems like it would be completely possible to do, and the only real problem would be increased memory usage from having to store the data on which frames where referenced most.

[Dark Shikari]

I don't mean to dig up an old thread. But I was reading this and thinking: Wouldn't it be possible to implement a similar RC (Or would RD be the proper term?) in CRF mode?

I was thinking something along the lines of exactly what you did: Store which frames were referenced most internally, lower qps on most used and raise on least used. It seems like it would be completely possible to do, and the only real problem would be increased memory usage from having to store the data on which frames where referenced most.We discussed this in #x264dev. The problem is that most referencing is indirect--that is, X references Y which references Z, rather than X referencing Z directly.

This requires calculating and keeping a complex graph of dependencies, and more importantly, there are some inherent mathematical problems in this.

[Sagekilla]

Dark Shikari, is this new rate control a part of svn yet or no?

[Dark Shikari]

Dark Shikari, is this new rate control a part of svn yet or no?This thread was just a silly idea involving reference counting, it didn't turn out to be particularly useful.