2nd Psy-RDO / Psy-trellis test results (Blu-ray, insane grain)

[Ranguvar]

Please vote (post)! Give details, too. Take a look at the full encoded clips, if you can.

Test was done with a short clip from the 30 Days of Night NTSC Blu-ray (frames 75054 to 75556, inclusive). You can grab it here (http://www.rapidspread.com/file.jsp?id=o8w240tbtu).

Tested with my r977 build (http://forum.doom9.org/showthread.php?p=1184352#post1184352).

Complete settings used here (http://paste2.org/p/75737).

PLEASE USE THESE - one frame can only show so much:
ZIP of encoded videos, logs, stats, and DGA index files (a35): here (http://www.megaupload.com/?d=ZYAIQM99).

Some example pics (frame 333) (I recommend Firefox's Ctrl+num_tab feature for quick switching comparison):


Source (http://ranguvar.ath.cx/Source.png)

--psy-rd 0.0:0.0 (http://ranguvar.ath.cx/0_0.png) (off)

--psy-rd 1.0:0.0 (http://ranguvar.ath.cx/10_0.png) (default)

--psy-rd 1.0:0.5 (http://ranguvar.ath.cx/10_05.png)

--psy-rd 1.0:0.7 (http://ranguvar.ath.cx/10_07.png)

--psy-rd 1.0:1.0 (http://ranguvar.ath.cx/10_10.png) (old default)

--psy-rd 1.15:0.5 (http://ranguvar.ath.cx/115_05.png)

--psy-rd 1.15:0.7 (http://ranguvar.ath.cx/115_07.png)

--psy-rd 1.15:1.0 (http://ranguvar.ath.cx/115_10.png)

--psy-rd 1.3:0.3 (http://ranguvar.ath.cx/13_03.png)

--psy-rd 1.3:0.5 (http://ranguvar.ath.cx/13_05.png)

--psy-rd 1.3:0.7 (http://ranguvar.ath.cx/13_07.png)

--psy-rd 1.3:1.0 (http://ranguvar.ath.cx/13_10.png)

--psy-rd 1.5:0.3 (http://ranguvar.ath.cx/15_03.png)

--psy-rd 1.5:0.5 (http://ranguvar.ath.cx/15_05.png)

--psy-rd 1.5:0.7 (http://ranguvar.ath.cx/15_07.png)

[burfadel]

I think 1.0:1.0 (old default) looks better than 1.0:0 (new default). Psy-trellis seems to make it slightly clearer/sharper. Its hard to tell with the other settings, but I think aq above 1.0 psy trellis may be better

[Ranguvar]

Agreed. The psy-trellis helps preserve small details that aren't grain/noise, for instance the topmost crease in the vampire's forehead.

My favorite, and new default (for me), is still 1.15:0.7, I think :)

[Sharktooth]

again 1.0:1.0 and 1.15:0.7 (even if the latter seems to oversharp a bit)

[lexor]

1.0:1.0 for me. It just seems like it moves grain a bit (which isn't a big deal since grain isn't stationary to begin with), everything else seems to actually deform the image (especially womans forehead/face) to varying degree.

[G_M_C]

Well i've been testing the new psy-trellis additions (set to 1.0:1.0, on a capture of a movie). I've noticed that lowering AQ-strength a bit seems to help overall visual quality (i had it on 0.9 and also tried 0.8).

I seem to remember a posting about this, but i cant find it again. What are your thoughts on this ?

EDIT:
Re-found the the earlier discussion on this;
http://forum.doom9.org/showthread.php?p=1183636#post1183636

[burfadel]

Doesn't AQ redistributes bits from lines where they are wasted to flat areas where lower quality is noticeable? so for a low bitrate encode the reduction of bits from the lines may be more noticeable as the encoder tries to balance quality for the flat areas / straight lines with the low bitrate available to do so!... With psy-trellis enabled, which seems to keep the sharpness of the image (which is better), the low bitrate of the lines become more pronounced and hence may lower the perceived quality of the encode in certain scenarios as others have found. Due to the lines and 'real' flat areas involved with anime, the sharpening caused by psy-trellis may show roughness '(aliasing)'? of the lines more when AQ is used (as is the default). There still could be an output issue without vaq if the bitrate is low and psy-trellis 'sharpens' the lines. A way around this could be to automatically reduce the aq slightly (say -0.3)when there's certain parameters detected in the frame - what these are I could not guess, but parameters that would suggest anime such as a low complexity?, which could also work in reverse and raise the aq fractionally (to 1.1 or even 1.05) if deemed necessary by the encoder logic. Psy-rd could also be slightly variable owing to differences people notice in encodes, and go from 0.9 to 1.05 or something by default. The logic could be calculated at every I frame, or maybe every few P frames, whatever is best.

This idea sounds good at least, its probably not possible...? (or relatively easy to implement)

[Quark.Fusion]

Agreed. The psy-trellis helps preserve small details that aren't grain/noise, for instance the topmost crease in the vampire's forehead.

My favorite, and new default (for me), is still 1.15:0.7, I think :)

1.15:0.7 shoulnd't be default as it introduces artifacts —*look at vampire's tooth.

[Ranguvar]

@Quark.Fusion: Sorry, I don't see that?

@burfadel: Interesting :p Talk to DS/pengvado!

@G_M_C: Actually yeah, I find myself using 0.7 or 0.8 sensitivity to avoid the edge noise/dirt that VAQ seems to introduce a lot. Hopefully DS will finish the quantizer smoothing patch soon :D

[Quark.Fusion]

Lower tooth appers to be longer that in the source or 1.0:1.0

It's may be invisible at low brighthess, but I'm clearly see diffirence on my LCD.

[G_M_C]

@G_M_C: Actually yeah, I find myself using 0.7 or 0.8 sensitivity to avoid the edge noise/dirt that VAQ seems to introduce a lot.
[...]

Ahh, that confirms what I saw; But 0.7 is a bit low imho. I've had best results with 0.8~0.9, but I guess it would depend on the source.

For the results in the topic: I favour 1.0:1.0, it is the best balanced. But my opinion is also partly based on my own experiance (and then combined with lower aq-strength).


[...]
Hopefully DS will finish the quantizer smoothing patch soon :D

That would be a very good addition indeed !

X264 is becoming extremely good quality-wise by all DS' work, my great thanx go out to him for all the work he's put in (and all the other developes also offcourse) !

[R3Z]

Lower tooth appers to be longer that in the source or 1.0:1.0

It's may be invisible at low brighthess, but I'm clearly see diffirence on my LCD.

I see it as well, you aint going crazy.

[Ranguvar]

Lower tooth appers to be longer that in the source or 1.0:1.0

It's may be invisible at low brighthess, but I'm clearly see diffirence on my LCD.

OK, I see it now. However, I'd say that all Psy RDO causes distortions from the original. That's a given. The question is, are you going to notice a tooth that's 2px longer on a 1080p screen from a few feet a way, at 24fps? ;) If you can, by all means, but remember that things like brushed-away grain are artifacts too, of a sort, and are probably much more noticeable.

[Quark.Fusion]

I don't watch movies few feet away — I watch them closely to screen (LCD is safe for eyes, right?) :) And if it's a still scene — yes, I will notice artifact. If it is not blurred image then I will able to see pixel aliasing too :)

I already wrote here somewhere that strong PSY-RD options produces more sharp image, but at cost of additional artifact-noise. I wonder what is better for a reference frame.

Imagine that you have "F" letter in a text string and after encoding it becoming "P" — what you prefer: slightly blurred "F", or a sharp "P"? :)