Psy RDO: Official testing thread (version 0.6 out!)

[Razorholt]

The link to the build is not valid... :(

[fields_g]

Thanks for giving me something to play around with this weekend! I feel like I'm missing something basic though...

2. Warn the user if trellis 1 is enabled--and disable trellis--if psy RD is on.


Trellis 2 is not a bad thing at all with psy RD; the problem before was that it encouraged blurring because RD would often prefer blurrier modes, and trellis would generate blurrier modes. But now, RD prefers *less* blurry modes.

I get that trellis 1 is bad and warning is thrown, but wonder what you mean by "--and disable trellis--if psy RD is on". You should use trellis 2, right? By the comment, it seems to say that you override trellis to 0 if psy RD is used ALWAYS. Clarify for me please!

[Dark Shikari]

By the comment, it seems to say that you override trellis to 0 if psy RD is used ALWAYS. Clarify for me please!I override trellis 1 to set it to zero, if psy RD is on.

I should probably update the patch so that it doesn't complain if RD is off entirely.

If the link to the build is down, blame jarod/b0bor.

0.5 will be coming soon; I am currently doing analysis in order to create a "lambda table" for psy RD, to vary its strength based on quantizer.

[fields_g]

Makes sense. Thanks!

[fields_g]

Is the Win32 build only GIT rev 906 + PsyRD 0.4? Or are there other patches in there also?

[bob0r]

git 906 + psy rdo 0.4

[Dark Shikari]

Version 0.5 is up.

[Blue_MiSfit]

Nice :) He kept good on his word - attaboy dark_shikari !

It's going on those encoder first thing monday!

~MiSfit

[Comatose]

That was fast =)
Will version 1 be the one to be committed?

[skystrife]

http://forum.doom9.org/showthread.php?p=1158143#post1158143
^-- win32 build with psyRDO 0.5 applied, if you're interested.

[ACrowley]

Is psyRDO enabled by default with RDO+subme 7 ?

[LaTo]

I get a much smaller bitrate with new psy-rdo in CRF mode:



Build: x264 v886 psy-rdo 0.22
Size: 414mo

Cmd:
start /low /b /wait x264.886.psyrdo.exe --crf 18 --keyint 300 --min-keyint 30 --ref 6 --mixed-refs --no-fast-pskip --bframes 2 --b-pyramid --b-rdo --bime --weightb --direct auto --subme 7 --trellis 2 --partitions all --8x8dct --ipratio 1.3 --qcomp 0.8 --me umh --threads auto --thread-input --progress --no-psnr --no-ssim --aq-strength 1.0 --rdcmp psy --output "psy-test.mp4" "test.avs"

Log:
avis [info]: 1280x720 @ 29.97 fps (9009 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 PHADD SSE4 Cache64
mp4 [info]: initial delay 667334 (scale 10000000)
x264 [info]: slice I:398 Avg QP:13.18 size: 5688100:00
x264 [info]: slice P:7782 Avg QP:15.43 size: 49859
x264 [info]: slice B:829 Avg QP:19.05 size: 28502
x264 [info]: mb I I16..4: 23.1% 53.5% 23.3%
x264 [info]: mb P I16..4: 4.2% 10.9% 4.8% P16..4: 36.9% 22.2% 15.9% 0.3% 0
.2% skip: 4.7%
x264 [info]: mb B I16..4: 0.1% 0.3% 0.1% B16..8: 41.7% 3.2% 9.7% direct:
15.0% skip:29.9%
x264 [info]: 8x8 transform intra:54.5% inter:59.9%
x264 [info]: direct mvs spatial:98.1% temporal:1.9%
x264 [info]: ref P 51.3% 16.4% 12.7% 7.1% 6.2% 6.3%
x264 [info]: ref B 75.5% 10.5% 7.0% 3.5% 3.4%
x264 [info]: kb/s:11557.4

encoded 9009 frames, 1.99 fps, 11557.63 kb/s



Build: x264 v910 psy-rdo 0.5
Size: 351mo

Cmd:
start /low /b /wait x264.910.psyrdo5.exe --crf 18 --keyint 300 --min-keyint 30 --ref 6 --mixed-refs --no-fast-pskip --bframes 2 --b-pyramid --b-rdo --bime --weightb --direct auto --subme 7 --trellis 2 --partitions all --8x8dct --ipratio 1.3 --qcomp 0.8 --me umh --threads auto --thread-input --progress --no-psnr --no-ssim --aq-strength 1.0 --psy-rd 1.0 --output "newpsy5-test.mp4" "test.avs"

Log:
avis [info]: 1280x720 @ 29.97 fps (9009 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 PHADD SSE4 Cache64
mp4 [info]: initial delay 667334 (scale 10000000)
x264 [info]: slice I:389 Avg QP:13.27 size: 4596900:00
x264 [info]: slice P:7672 Avg QP:15.54 size: 42965
x264 [info]: slice B:948 Avg QP:19.65 size: 22399
x264 [info]: mb I I16..4: 28.2% 57.8% 13.9%
x264 [info]: mb P I16..4: 5.8% 18.2% 3.3% P16..4: 38.4% 19.3% 9.8% 0.2% 0
.1% skip: 4.9%
x264 [info]: mb B I16..4: 0.2% 0.9% 0.2% B16..8: 39.6% 2.9% 4.4% direct:
13.1% skip:38.8% L0:36.1% L1:47.7% BI:16.2%
x264 [info]: 8x8 transform intra:65.3% inter:65.4%
x264 [info]: direct mvs spatial:97.8% temporal:2.2%
x264 [info]: ref P L0 54.0% 16.2% 12.0% 6.7% 5.6% 5.5%
x264 [info]: ref B L0 75.1% 10.0% 7.8% 3.9% 3.2%
x264 [info]: ref B L1 99.3% 0.7%
x264 [info]: kb/s:9813.5

encoded 9009 frames, 1.95 fps, 9813.65 kb/s

why? :confused:

[CruNcher]

why? :confused:

Because it's efficiency improved over time :)

[cogman]

I get a much smaller bitrate with new psy-rdo in CRF mode:



Build: x264 v886 psy-rdo 0.22
Size: 414mo




Build: x264 v910 psy-rdo 0.5
Size: 351mo


why? :confused:

Has quality dropped? No? Then quit complaining :P. My guess is because it manages the bitrate even better.

[RickA]

You are a great one Dark Shikari. Thanks for all your work.

Cheers,
Rick

[wata]

i am using megui latest x264 build 901 which use psy-rdo 0.22
so trellis should be set to 0 or 2?

currently using the following command to encode all my video, spot any setting wrong?

program --crf 19.0 --ref 5 --mixed-refs --no-fast-pskip --bframes 16 --b-pyramid --b-rdo --bime --weightb --subme 7 --trellis 2 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh --threads auto --thread-input --progress --no-psnr --no-ssim --output "output" "input"

[Dark Shikari]

i am using megui latest x264 build 901 which use psy-rdo 0.22
so trellis should be set to 0 or 2?2 is better but a lot slower. Its a tradeoff.

[ajp_anton]

Been trying to encode a DVD with insane grain. Psy 0.3 needed 2300 kbit/s to look somewhat close to the source. Psy 0.5 looks almost identical to the source at 2100 kbit/s!

[skystrife]

Is psyRDO enabled by default with RDO+subme 7 ?

How to use it:

Its on by default.

So... in other words, yes it is. You could have gathered that much though. =P

[Inventive Software]

How long am I gonna have to hold off to encode my DVDs, eh? :p

[Comatose]

Been trying to encode a DVD with insane grain. Psy 0.3 needed 2300 kbit/s to look somewhat close to the source. Psy 0.5 looks almost identical to the source at 2100 kbit/s!
Awesome work, Dark Shikari!

Any chance for some comparison screenshots, anton?

[turbojet]

Was there an issue found with psyRDO and trellis 1?
Or was there something added since 0.3 that definitely won't work with trellis 1?

I know it was never recommended to use trellis 1 with psyRDO (due to trellis 1 nature?) but I was getting noticably better quality over deadzones (even lower ones) with a very minimal speed hit. I never found any artifacts with it. I couldn't find enough of a difference between trellis 1 and trellis 2 to justify the <1 vs 2-2.5 fps 1080p on a 3800+ But I do notice enough of a quality difference between deadzones and trellis 2 in 0.5 to maybe take the <1 fps plunge or revert back to 0.3 until I get a better computer.

However if I can get the benefits I saw with trellis 1 along with the nice compression gain of 0.5 with no noticable quality difference so far, it would be much appreciated, unless there's some flaw.

[DarkZell666]

I override trellis 1 to set it to zero, if psy RD is on.
I think that answers your question ;)

[turbojet]

Ya I'm aware it switches off in 0.5 but what I'm asking is why was this change made?

I notice some people using trellis 1 in this thread, dark shikari not suggesting its use until more info is found (from old first post), and no issues found...

[J_Darnley]

[2008-07-12 16:53] <jarod> in case people ask me: why cant trellis 1 be used?
[2008-07-12 16:56] <Dark_Shikari> With trellis 1, during RDO, when picking the best mode to use, regular deadzone quantizer is used--trellis is only used in the final encode. This is for speed reasons--trellis is slow.
[2008-07-12 16:56] <Dark_Shikari> This is fine with regular RDO, because trellis and regular RDO optimize for the exact same thing
[2008-07-12 16:57] <Dark_Shikari> but with Psy RDO, Psy RDO is optimizing for something totally different than trellis is optimizing for
[2008-07-12 16:57] <jarod> what happens when you do use trellis 1?
[2008-07-12 16:57] <Dark_Shikari> so the shortcut of doing deadzone quant during RDO, and trellis only on the final encode, no longer works
[2008-07-12 16:57] <Dark_Shikari> no, trellis 1 isn't used.
[2008-07-12 16:57] <Dark_Shikari> trellis 2 showed to be consistently best
[2008-07-12 16:57] <jarod> x264 [warning]: psy RD is not compatible with trellis=1; use 0 or 2.
[2008-07-12 16:58] <jarod> x264 [warning]: psy RD is not compatible with trellis=1; use 0 or 2. [using trellis=0 now]
[2008-07-12 17:02] <jarod> ok, so its a feature, trellis 1 can never work with psy rd
[2008-07-12 17:03] <jarod> 2: trellis 1 = used by rules and many profile defaults
[2008-07-12 17:04] <jarod> whats better? psyrd+trellis2 or psyrd+trellis0 ratio speed/quality ? (going by your tests)
[2008-07-12 17:04] <Dark_Shikari> trellis 2 is always fuckslow
[2008-07-12 17:05] <jarod> if you encode 20fps on pass2, how much will trellis2 slow it down compared to 0?
[2008-07-12 17:06] <Dark_Shikari> anyways I'd rather people use psy rdo + trellis 0 than psy rd + trellis 1
[2008-07-12 17:07] <Dark_Shikari> and dropping trellis will give them the speed boost they need to not complain about psy RD... ;)

[Snowknight26]

[2008-07-12 17:06] <Dark_Shikari> anyways I'd rather people use psy rdo + trellis 0 than psy rd + trellis 1


But since 1 is overridden to 0....

I hope he meant 2 instead of 1.

[Dark Shikari]

But since 1 is overridden to 0....

I hope he meant 2 instead of 1.No, I didn't mean 2. 1 is overridden to zero--the reason being my above quote ;)

[LoRd_MuldeR]

But since 1 is overridden to 0....

I hope he meant 2 instead of 1.

Psy RDO + Trellis=1 simply is a bad combination. Therefore Psy RDO + Trellis=1 will be lowered to Trellis=0 and a warning is put out.
Nevertheless Psy RDO + Trellis=2 still is the recommended combination for maximum quality! But Trellis=2 is significant slower than Trellis=0 or Trellis=1.
So people that intentionally set up Psy RDO + Trellis=1 probably don't want their Trellis setting raised to Trellis=2 (major speed loss).

[Yoshiyuki Blade]

Here's a test clip with default psy-rd (v0.5). Not easy on the hardware: http://www.mediafire.com/?bfniddwmvrn

How does it look to you guys?

EDIT: I just noticed that you can actually see the illuminated rough surface texture on my LCD (its not glossy). x264 can certainly preserve the finer details!

[Atak_Snajpera]

eeee. At 16Mbps everything will look good :) Try to use 8 Mbps instead.

[rhester72]

Stupid question: Does PsyRDO require CABAC? (If so, why?)

Rodney

[Dark Shikari]

Stupid question: Does PsyRDO require CABAC?nope.

[cogman]

Another dumb question, I have copied the patch from pastebin (the text box on the bottom). What command are you guys using to apply the patch? I tried patch but it informed me that the file is only full of garbage.

I can't use a build because I am using linux, but I have some grainy videos that I really would like to test this on.

[LoRd_MuldeR]

I think "patch -p1 <file.diff" called from the x264 directory should to the job.

[Dark Shikari]

Click the "download" link on Pastebin instead of copying.

Also, you may have to use dos2unix.

[cogman]

all right, got the patch applied and I am running an encode now. I've seen a short clip and so far it looks good. Thanks for the help

Wow, I haven't finished encoding yet, but the differences are already pretty noticeable. Ill get some screen shots up in a moment.

First, no psy (http://i253.photobucket.com/albums/hh50/cogman10/firstnopsy.png)
First, psy (http://i253.photobucket.com/albums/hh50/cogman10/firstpsy.png)

Second, no psy (http://i253.photobucket.com/albums/hh50/cogman10/secondnopsy.png)
Second, psy (http://i253.photobucket.com/albums/hh50/cogman10/secondpsy.png)

Third, no psy (http://i253.photobucket.com/albums/hh50/cogman10/thirdnopsy.png)
Third, psy (http://i253.photobucket.com/albums/hh50/cogman10/thirdpsy.png)

As you can see, there is a big difference in grainy video. However, the size size did increase in this case from 261.1 -> 272.6 MB

Heres my encoding settings used on both versions (Git 913)

x264 test.y4m -o S01Ep19old.264 --no-psnr --no-ssim --threads auto --no-fast-pskip -t 2 --mixed-refs --me tesa -w --direct auto -A all --crf 20.0 -r 6 -m 7 -b 16 --b-pyramid --progress --b-rdo

[Selur]

However, the size size did increase in this case from 261.1 -> 227.6 MB
Where is the typo? Did you mean decrease or should 227.6 mean 272.6 ? ;)

[TheRyuu]

Where is the typo? Did you mean decrease or should 227.6 mean 272.6 ? ;)

Well then his tests are invalid I would think.
They are not at the same bitrate and therefor non-comparable. That is, if the size increase.

If it decreased the same still holds true but no one would care ;)

[bkman]

v0.5 still suffers from serious artefacting issues. Will have samples up to illustrate what I mean later.

Edit: After further testing it seems that the artefacts might be related to a bug with x264 and some low deblocking settings (-3:0), as I see some (different) artefacts with the fgo build too. Problem doesn't seem to occur at 0:0 deblocking settings. Will do some further testing.

Update: -2:-1 seems to be safe to use as well, and gives sharper results than 0:0.

[DarkZell666]

@cogman : you used CRF, and you're like the Nth person to compare Psy and NonPsy using CRF, and it has been said N times already that CRF shouldn't be used to compare Psy and NonPsy. 2-pass should be used to compare Psy and NonPsy !.

Wow, sorry about the bold and colors, your effort for posting the screenshots _is_ appreciated, but it isn't as valuable as you intended :rolleyes:

You actually noticed by yourself that there was a flaw :
However, the size size did increase in this case from 261.1 -> 227.6 MBIf the filesize is different by such a large amount, there's no way your eyes can judge if the bits moved around by psyrdo were well spent.

THE Question : Would pushing the bitrate up by the same amount without using psyrdo have the same effect or not ? (and we've done a full circle, now we're back to comparing two files with the same bitrate : one with psyrdo, one without).

If you want your tests to be valuable, use 2-pass ;)

[rhester72]

@cogman : you used CRF, and you're like the Nth person to compare Psy and NonPsy using CRF, and it has been said N times already that CRF shouldn't be used to compare Psy and NonPsy. 2-pass should be used to compare Psy and NonPsy !.

What is your source for this? I rewalked this entire thread and can find no such claim. In fact, that would largely reduce the value of PsyRDO in the first place!

Rodney

[LoRd_MuldeR]

It justs says that you cannot test Psy RDO in CRF mode, you can use Psy RDO in CRF mode of course!

If you test PsyRDO versus Non-PsyRDO in CRF mode, you will get two files of different size as your result most likely ;)
Quality of files of different size cannot be compared, the test would be useless. 2-Pass avoids that.

[Audionut]

What is your source for this?

The developers of x264. It's been said many times before as DarkZell666 has pointed out.

Keep searching!!!

[jeffy]

What is your source for this? Rodney

I think this is the source, akupenguin, aimed at FGO (and the posts following this one):
http://forum.doom9.org/showthread.php?p=1129955#post1129955

[cogman]

@cogman : you used CRF, and you're like the Nth person to compare Psy and NonPsy using CRF, and it has been said N times already that CRF shouldn't be used to compare Psy and NonPsy. 2-pass should be used to compare Psy and NonPsy !.

Wow, sorry about the bold and colors, your effort for posting the screenshots _is_ appreciated, but it isn't as valuable as you intended :rolleyes:

You actually noticed by yourself that there was a flaw :
If the filesize is different by such a large amount, there's no way your eyes can judge if the bits moved around by psyrdo were well spent.

THE Question : Would pushing the bitrate up by the same amount without using psyrdo have the same effect or not ? (and we've done a full circle, now we're back to comparing two files with the same bitrate : one with psyrdo, one without).

If you want your tests to be valuable, use 2-pass ;)

I disagree. I believe the test results are valid. First, it is clear that Psy-Rdo is at very least the cause of the bitrate increase. This data is useful for anyone that does a CRF encode, It was stated earlier in this thread that people's filesize decreased with getting better quality, I just gave an example where filesize increased (grainy video)

Because we know that psy-rdo cause the increase in bitrate, it is safe to assume that at the same crf setting it will always (ok, not everything has been tested, but it look like it will) give better results, but not always give smaller file-sizes.

Now, I do get your point, but I hope you understand mine. I use CRF to get similar quality across encodes, not similar bitrates. I doubt (in fact Im sure) that I am not the only one. It is safe to say the Psy-rdo is giving better quality per value of CRF

And to those earlier, yes it was a typo, the increase was about 10 mb. not huge but big enough to make some differences (I do doubt that it was enough to give the quality increase that I saw)

[Sharktooth]

no, CRF is not a valid test for psy-RDO.
if you like CRF, that doesnt mean everyone uses it, and it doesnt mean it should be exclusively and universally used to optimize the encoder.

[LoRd_MuldeR]

@cogman:

What if Psy-RDO produces better quality at same CRF, but produces a bigger file at the same time?
How do you want to decide whether Psy-RDO is actually better? The increase in quality might be due to the increase in size...
No, to test Psy-RDO you must use 2-Pass, even if you use CRF later ...

[cogman]

nO, CRF is not a valid test for psy-RDO.
if you like CRF, that doesnt mean everyone uses it, and it doesnt mean it should be exclusively and universally used to optimize the encoder.

I realize that it isn't, and I realized that not everyone uses it. But again, my point was to show that with grainy video at a given crf value you will see a higher bitrate due to psy-RDO, though the video will look better.

Thats it. Im not trying to say that psy-Rdo optimized bitrate useage or anything to the sort. My message was for the most part to other CRF users "Beware, you may see bigger files as a result" Thats it.

[Sharktooth]

read the Lord MuldeR comment.
it's also quite natural to have a bigger filesize. the more details the encoder keeps (lowering the quntizer), the higher the needed bits.
That also alter the final rate factor... so it's perfectly normal.

[Dark Shikari]

My message was for the most part to other CRF users "Beware, you may see bigger files as a result" Thats it.This is inherent in the psy RDO algorithm, which lowers the mathematical coding efficiency--thus raising quantizers at the same bitrate. Since CRF will have the roughly the same quantizers no matter what settings you use, this means the bitrate will rise.