Psy RDO: Official testing thread (version 0.6 out!)

[jult]

3. Psy RD strength is now a decimal value with default 1.0. Don't touch it unless you have good reason.
4. Psy RD strength now automatically scales based on quantizer. This is done internally--the "strength" is a multiplier to this internal value.
So, if I use CRF, does --psy-rd still pick the right scale? (I generally prefer crf over some static rate target.)
If not, what would you recommend over this with a very sharp 1280x720 HD camcorder source?
x264.exe --crf 22 --keyint 200 --min-keyint 25 --no-fast-pskip --no-dct-decimate --no-psnr --no-ssim
--subme 6 --ref 9 --mixed-refs --bframes 5 --b-pyramid --b-rdo --bime --weightb --direct auto
--filter -2,-1 --psy-rd --analyse all --8x8dct --me umh
--threads auto --thread-input --progress --output "%%~na.h264" "%%a"
Thanks in advance!

[LoRd_MuldeR]

Psy RDO works just fine with CRF mode! It will scale automatically, based on the quantizer.
The "--psy-rd" option will be multiplied to the internal scaling factor, so don't touch this value!
(...unless you have a very good reason to do so)

[jult]

OK, tnx. I just discovered though that both --psy-rdo and --hadamard were unknown commands to the x264 modified rev. 928 from x264.nl. Seems this patch is not included (yet)?

[LoRd_MuldeR]

OK, tnx. I just discovered though that both --psy-rdo and --hadamard were unknown commands to the x264 modified rev. 928 from x264.nl. Seems this patch is not included (yet)?

I don't know what "--hadamard" is supposed to do, but it's not an official x264 option. There only is "--me tesa" as mentioned in the other thread.
And "Psy RDO" is not in the official x264 git yet, it's available as a patch! Patched builds can be found in this thread, as always:

Current Patches, Where to get them, How they affect speed/output (http://forum.doom9.org/showpost.php?p=1165193&postcount=615)

[Sharktooth]

--psy-rd and only available in patched/modified builds... --hadamard does not exists.

[jult]

--psy-rd and only available in patched/modified builds...
I had the latest modified build, but I used --psy-rdo instead of --psy-rd. (Why the o was omitted is beyond me..)

[chainring]

If you have one of the builds from here (http://forum.doom9.org/showthread.php?t=130364), then psyRDO is included and on by default. Go to the end of the thread. Or, if you're using meGUI and updating from the development servers, it's updating to a version with psyRDO.

[Sharktooth]

I had the latest modified build, but I used --psy-rdo instead of --psy-rd. (Why the o was omitted is beyond me..)
the option is --psy-rd... not --psy-rdo... the "o" stands for optimization(s), it was omitted to make the option shorter (...i think...).

also --no-fast-pskip may not help (it doesnt help much if at all) but it will consistently slow down the encoder. is it worth it? IMHO only for insane settings.

[Avenger007]

So what's there left to do before Psy RDO is committed to git?

All the clips I've tested show that Psy RDO retains more detail even though SSIM is slightly lower. The lower SSIM seems to be as a result of the detail not being exactly like the original but it's still close enough that it looks legit.

[Dark Shikari]

So what's there left to do before Psy RDO is committed to git?Psy trellis (http://i36.tinypic.com/ickyz9.png) (regular trellis (http://i36.tinypic.com/2dsfhur.png))

[G_M_C]

Psy trellis (http://i36.tinypic.com/ickyz9.png) (regular trellis (http://i36.tinypic.com/2dsfhur.png))

Hmmm, i find the difference only marginal; Switching between the two makes the psy-trellis just an tat sharper, and the rest of the difference seems only to be the grain beeing different. I doubt if it was really clearly noticable with real moving images, as opposed to static screenshots.

[Avenger007]

Psy trellis (http://i36.tinypic.com/ickyz9.png) (regular trellis (http://i36.tinypic.com/2dsfhur.png))
Does that mean there's nothing left to do? Because Psy trellis looks more detailed and less blurred than regular trellis.

[G_M_C]

Does that mean there's nothing left to do? Because Psy trellis looks more detailed and less blurred than regular trellis.

I have the feeling that the "standard trellis"-image is not blurred, but the psy-trellis is sharpened (and i'm no fan of sharpening).

[Dark Shikari]

Does that mean there's nothing left to do? Because Psy trellis looks more detailed and less blurred than regular trellis.No, my point is that "standard trellis" is "psy RD + --trellis 2" and "psy trellis" is an unreleased algorithm that I have yet to perfect ;)

Neither is blurred or sharpened.

[DarkZell666]

Hmmm, i find the difference only marginal; Switching between the two makes the psy-trellis just an tat sharper, and the rest of the difference seems only to be the grain beeing different. I doubt if it was really clearly noticable with real moving images, as opposed to static screenshots.

I find the psy trellis potentially closer to the source than the regular trellis, too bad :p (the regular trellis screenshot shows detail loss, which seems to be "recovered" by the psy-trellis version).

Dark Shikari, could you post the source frame so everyone can see ? ;)

[Dark Shikari]

Source (http://i37.tinypic.com/6r6680.png)

I think I accidentally used the wrong color conversion matrix or something, so ignore the slightly off colors.

[CruNcher]

what for a QP is this frame and how would the regular trellis look like without deblocking ?

[Soichiro]

I think the old version actually looks closer to the original, but the new version (with psy trellis) actually seems to look better than the original (less ugly blocking and jagged edges). Though I've never been a fan of film grain to begin with, so maybe it's just my preference. :p

[DarkZell666]

I think the old version actually looks closer to the original, but the new version (with psy trellis) actually seems to look better than the original (less ugly blocking and jagged edges). Though I've never been a fan of film grain to begin with, so maybe it's just my preference. :p

:confused: That's not what I see, are you sure you didn't swap tabs in FF when comparing ? That has happened to me a couple of times already :D.

Ok, I'll compare again in different viewing conditions, I have too much light around me here ATM, but my vote goes to Psy-Trellis (aka. --trellis 3 ? :p)

[elguaxo]

my vote goes to psy-trellis

+1 :)

[Sharktooth]

the first picture looks more detailed to me too.

[refulgentis]

that's scary, I can see differences between the source and regular trellis, or regular trellis and psy trellis, but between psy trellis and the source I see absolutely no difference.

[Sharktooth]

there's indeed a difference... but it's not so visible.

[LoRd_MuldeR]

I wouldn't go without Psy RDO nowadays... :)

[tetsuo55]

These Psy optimisations are getting better and better!

Do you think there are more area's that you can psy-improve?

[Warpman]

These Psy optimisations are getting better and better!

Do you think there are more area's that you can psy-improve?

nothing is perfect so there is always room for improvements nearly everywhere...

there was a development thread here updated by DS with some things that could be done... try the search

[Avenger007]

No, my point is that "standard trellis" is "psy RD + --trellis 2" and "psy trellis" is an unreleased algorithm that I have yet to perfect ;)
Wow! :D
The complexity difference between "psy trellis" and "standard trellis" looks just like the complexity difference between psy RD and no psy RD in my test clips.

I have a batch of encodes to do and after seeing "psy trellis" I think I'll wait until it is "perfected" before I start encoding them. :)

[yesgrey]

between psy trellis and the source I see absolutely no difference.

Look at the neck and at the bottom left corner of the image. The differences are very subtle...

I also prefer the psy trellis, the difference to the standard is easilly noticed!

Great work! :thanks:

I am also waiting for doing some conversions...

[Dark Shikari]

Psy-RD + psy-trellis patch prototype (http://pastebin.com/f3e048abf).

changes:

1) automatically lower chroma QP offset with psy-rd and psy-trellis in order to compensate for higher overall quantizer for chroma quality
2) prototype psy trellis, not optimized at the moment
3) various cosmetics/etc
4) remove restriction against trellis 1, since psy trellis is now in

Please do not go around stuffing this one in megui builds, this is experimental.

[Sharktooth]

same as for bframes patch. it never went into megui auto-update.

[LoRd_MuldeR]

Psy-RD + psy-trellis patch prototype (http://pastebin.com/f3e048abf).

changes:

1) automatically lower chroma QP offset with psy-rd and psy-trellis in order to compensate for higher overall quantizer for chroma quality
2) prototype psy trellis, not optimized at the moment
3) various cosmetics/etc
4) remove restriction against trellis 1, since psy trellis is now in

Please do not go around stuffing this one in megui builds, this is experimental.

Doesn't compile for me :(

gcc -O4 -ffast-math -Wall -I. -DHAVE_MMX -DARCH_X86 -DSYS_MINGW -DHAVE_PTHREAD
-s -fomit-frame-pointer -c -o encoder/analyse.o encoder/analyse.c
In file included from encoder/analyse.c:35:
encoder/rdo.c: In function 'x264_quant_4x4_trellis':
encoder/rdo.c:575: error: 'struct <anonymous>' has no member named 'i_lambda_qp'

encoder/rdo.c: In function 'x264_quant_8x8_trellis':
encoder/rdo.c:586: error: 'struct <anonymous>' has no member named 'i_lambda_qp'

make: *** [encoder/analyse.o] Error 1

Full log: http://pastebin.com/m75ed9773

[gav1577]

Psy trellis defiantly looks better to me the grain stands out more nice job Dark Shikari :) I usually lower chroma QP offset to -4 found it helped with banding with fog clouds/sky does this mean with this patch there is no point in lowering the value because it will be done automatically ? and also is there a need for --trellis 2 --trellis 1 in the cmd line when using psy-trellis ? Thanks

[Sharktooth]

trellis 1 will be automatically disabled if you use psy-rdo that's enabled by default at subme > 5.

[gav1577]

trellis 1 will be automatically disabled if you use psy-rdo that's enabled by default at subme > 5.

My mistake i just realized that just meant --trellis 2 :)

[Dark Shikari]

Doesn't compile for me :(

gcc -O4 -ffast-math -Wall -I. -DHAVE_MMX -DARCH_X86 -DSYS_MINGW -DHAVE_PTHREAD
-s -fomit-frame-pointer -c -o encoder/analyse.o encoder/analyse.c
In file included from encoder/analyse.c:35:
encoder/rdo.c: In function 'x264_quant_4x4_trellis':
encoder/rdo.c:575: error: 'struct <anonymous>' has no member named 'i_lambda_qp'

encoder/rdo.c: In function 'x264_quant_8x8_trellis':
encoder/rdo.c:586: error: 'struct <anonymous>' has no member named 'i_lambda_qp'

make: *** [encoder/analyse.o] Error 1

Full log: http://pastebin.com/m75ed9773I'm an idiot, fixed (http://pastebin.com/f6795ade1).

[Dark Shikari]

trellis 1 will be automatically disabled if you use psy-rdo that's enabled by default at subme > 5.Read my post, --trellis 1 works now with psy-trellis and psy-rd (psy-trellis is just a modificiation to trellis that happens when psy-rd is on).

[gav1577]

(psy-trellis is just a modificiation to trellis that happens when psy-rd is on).


Got it now was a bit confused :)

[LoRd_MuldeR]

I'm an idiot, fixed (http://pastebin.com/f6795ade1).

Yup, that one compiles again :thanks:

[saint-francis]

Read my post, --trellis 1 works now with psy-trellis and psy-rd (psy-trellis is just a modificiation to trellis that happens when psy-rd is on).

So with psy-trellis is there still an advantage of --trellis 2 over --trellis 1?

[wyti]

Yes, trellis 1 is not applied everywhere, and trellis 2 is always active (but far slower)

[Sharktooth]

Read my post, --trellis 1 works now with psy-trellis and psy-rd (psy-trellis is just a modificiation to trellis that happens when psy-rd is on).
sorry, just missed it.

[bob0r]

I'm an idiot, fixed (http://pastebin.com/f6795ade1).

I agree.

[DarkZell666]

I'm an idiot, fixed. I agree.

rofl ... :p

[Dark Shikari]

The power of psy-trellis + psy-RD:

http://i35.tinypic.com/rk33wx.png







The above P-frame is from the first 500 frames of BlackPearl... encoded at 400kbps.

Here's the same frame from a certain widely-renowned commercial encoder (http://i34.tinypic.com/qp53s0.png).

[Sharktooth]

holy $#!7, are you sure psy-rdo and/or psy-trellis are not yet ready?

[Avenger007]

But that frame is rather dark so the extra complexity is probably hard to perceive during normal play. I'm guessing it's that darkness that makes it still look good @ 400kbps.

[Sharktooth]

nope. iy you bring up the brightness you'll see all the details.
http://www.webalice.it/f.corriga/temp/temp.png

[Avenger007]

I know, but that part of the scene wasn't meant to be viewed with high brightness.
Perhaps a sample from 300 might really show the power of psy-trellis + psy-RD.

[Sharktooth]

but details are still there at a low bitrate. that means the algo is good.

[Ranguvar]

Well, IMO, a great psychovisual algo would remove all details in areas the human eye can't see at the input brightness, and spend those bits elsewhere ;) But that's debatable (as all psy algos are). Point is, psy-trellis r0x0rz.