Psy RDO: Official testing thread (version 0.6 out!)

[Dark Shikari]

Psychovisually optimized rate-distortion optimization

Experimental Patch: Psy RD 0.6 (updated for r950)
Build: r937, version 0.6

Version 0.6 changes:

1. Psy trellis (adjustable separately on commandline) included and on by default.
2. Automatically adjust chroma QP offset to compensate for the higher quants psy RD/trellis result in.
3. Psy trellis allows trellis=1 now.

Version 0.3-0.5 changes:

1. Much faster, by caching half the SATDs that need to be done.
2. Warn the user if trellis 1 is enabled--and disable trellis--if psy RD is on.
3. Psy RD strength is now a decimal value with default 1.0. Don't touch it unless you have good reason.
4. Psy RD strength now automatically scales based on quantizer. This is done internally--the "strength" is a multiplier to this internal value.

How to use it:

Its on by default. Adjust the strength with --psy-rd. That's it.

How it works (simply): the human eye doesn't just want the image to look similar to the original, it wants the image to have similar complexity. Therefore, we would rather see a somewhat distorted but still detailed block than a non-distorted but completely blurred block. The result is a bias towards a detailed and/or grainy output image, a bit like xvid except that its actual detail rather than ugly blocking.

How it works (full explanation): read the comment in the patch.

[gav1577]

Originally Posted by Dark Shikari

If it isn't obvious already, this is going to destroy your PSNR and SSIM.

Great work Dark Shikari is the above anything to worry about ? i am a noob when it comes x264

[Dark Shikari]

Originally Posted by gav1577

Great work Dark Shikari is the above anything to worry about ? i am a noob when it comes x264

It only matters if you think that numbers are more important than how the video actually looks.

[Sagittaire]

Only 84 lines for the patch ... ?

[gav1577]

Ha ha the picture is more important IMHO. is there any cmd switch
for this patch or is automatically activated with certain settings ?
Thanks nevermind you just edited and answered my question thanks

[Dark Shikari]

Originally Posted by gav1577

Ha ha the picture is more important IMHO. is there any cmd switch
for this patch or is automatically activated with certain settings ?
Thanks

Its on by default any time you're using RDO. As such this is a patch for testing purposes and not for something to apply to every modified build out there, since you can't turn it off

Originally Posted by Sagittaire

Only 84 lines for the patch ... ?

Yup, and a large portion is comments

As you can tell the code is heavily macroed to reduce duplication; unwrapped it'd be another dozen lines or two. But overall the patch isn't too complicated; I'm only changing how x264 views "distortion" in RDO.

[guada2]

"Dark is a photoshop fgox264 .
Very god job.

Some questions:

- at what bitrate is it useful?
- is it compatible with grain opt on quickly scene?

[desta]

Me and my "2pass encode" questions again.... Would there be a worthwhile benefit in enabling this for both passes, or would just the 2nd pass suffice?

[Sagittaire]

Originally Posted by desta

Me and my "2pass encode" questions again.... Would there be a worthwhile benefit in enabling this for both passes, or would just the 2nd pass suffice?

And as always ... no. Because first pass is just a statistical reference for 2nd pass Rate Control. Very fine first pass (crf mode with same setting with close bitrate) will produce marginal better overall quality (it's just Rate Control optimisation).

[desta]

Originally Posted by Sagittaire

And as always ... no. Because first pass is just a statistical reference for 2nd pass Rate Control. Very fine first pass (crf mode with same setting with close bitrate) will produce marginal better overall quality (it's just Rate Control optimisation).

It wasn't exactly a ridiculous question though either. We know AQ needs to be the same for both passes to work, and other settings need to be similar or the same to do their job properly. I have also had instances where using similar settings in both passes have made for a better encode, especially when using FGO, which this patch is to replace....

http://forum.doom9.org/showthread.ph...77#post1143777

[Adub]

Damn DS!! You rule!! Keep the awesome quality patches coming!!

[TheRyuu]

Originally Posted by guada2

- at what bitrate is it useful?
- is it compatible with grain opt on quickly scene?

1.) Any
2.) It is intended to replace fgo or at least 'should' replace it. (hope that answers your question since I do not understand it fully)

[Gabriel_Bouvigne]

Nitpicking: this is not really similar to "perceptual entropy", but quite similar to the "dropout prevention" used by several encoders (exemple: 3gpp HE-AACv2 reference encoder).

[Dark Shikari]

Originally Posted by Gabriel_Bouvigne

Nitpicking: this is not really similar to "perceptual entropy", but quite similar to the "dropout prevention" used by several encoders (exemple: 3gpp HE-AACv2 reference encoder).

Explain?

[techouse]

I <3 Dark

[Gabriel_Bouvigne]

PE is a kind of rough complexity estimation of the frame done at early stage, and is usually done on a less granular scale. It is usually (PE is mainly a concept rather than a specific metric) not used within the RD/quantization stage. On the other hand, dropout prevention is something (usually enabled at low bitrates) that tries to preserve power per subband, even if the individual frequency bins are different, in order to avoid holes within the spectrum.

A simple dropout prevention scheme is described there:
http://www.3gpp.org/ftp/Specs/archiv.../26403-700.zip
section 5.6.1.1.2 "Avoidance of spectral holes"

[techouse]

http://techouse.project357.com/build...do_techouse.7z

Source: x264 r859 GIT (git://git.videolan.org/x264.git)

Applied patches (current versions):

x264_2pass_vbv.9.diff

x264_fix_win_stdin.diff

x264.gaussian.cplxblur.01.diff

x264_hrd_pulldown.04_interlace.diff

x264_me-prepass_DeathTheSheep.diff

x264_progress.diff

x264_psy_rdo.diff (this patch is STILL IN TESTING but it pretty much replaces FGO; please test how it compares to FGO using my older build "x264_x86_r859_progress_techouse")

x264_rd-optimze_DeathTheSheep.diff


Please check http://forum.doom9.org/showthread.php?t=130364 and http://git.videolan.org/gitweb.cgi?p...git;a=shortlog for more info

Compiled by techouse on June 1st 2008, 01:21:49 CEST with GCC-4.3.0 on Windows Vista Business SP-1 32-bit.

Commandline used: ./configure&&make

Platform: X86
System: MINGW
avis input: yes
mp4 output: yes
pthread: yes
gtk: no
debug: no
gprof: no
PIC: no
shared: no
visualize: no

[IgorC]

Some undesirable results:

With psy RDO
http://rapidshare.com/files/11924617...o_RDO.mp4.html

Without psy RDO
http://rapidshare.com/files/11924713...NoRDO.mp4.html

Settings for 3 passes:

x264.exe --threads 3 --pass 3 --progress --stats "x264_stat.log" --qcomp 0.75 --bframes 3 --bime --weightb --subme 7 --keyint 500 --ref 16 --trellis 2 --mixed-refs --8x8dct --partitions all --b-rdo --direct auto --b-pyramid --bitrate 800 --no-fast-pskip --me umh --merange 16 --deblock -1:-1 --me-prepass -o 3xrdo.mp4 Ma.avs

If source is needed I can upload it later.

[Dark Shikari]

Originally Posted by IgorC

Some undesirable results:

With psy RDO
http://rapidshare.com/files/11924617...o_RDO.mp4.html

Without psy RDO
http://rapidshare.com/files/11924713...NoRDO.mp4.html

Settings for 3 passes:

x264.exe --threads 3 --pass 3 --progress --stats "x264_stat.log" --qcomp 0.75 --bframes 3 --bime --weightb --subme 7 --keyint 500 --ref 16 --trellis 2 --mixed-refs --8x8dct --partitions all --b-rdo --direct auto --b-pyramid --bitrate 800 --no-fast-pskip --me umh --merange 16 --deblock -1:-1 --me-prepass -o 3xrdo.mp4 Ma.avs

If source is needed I can upload it later.

Undesirable? The psy-RDO version looks a hell of a lot better (overall) to me. There's a little bit of increased ringing but the overall detail retention is an order of magnitude better.

A random example:





Notice the vastly decreased blocking on the chest and the increased graininess in the background--plus of course the obvious, the face.

[IgorC]

Too much ringing and artifacts for my eyes