When to use trellis? [Archive]

View Full Version : When to use trellis?

Chengbin

25th March 2009, 00:47

I heard it is a bad idea to use trellis on anime. When should I use trellis (1 or 2), and when should I not?

Sagekilla

25th March 2009, 01:04

When using the latest psy-rd enhancements, enabling just about any option will not affect a source negatively. Trellis used to favor a soft image, but now it tends to favor a sharp result if you use psy-trellis.

Most of those "It's bad to use X on Y" came from the days when x264 would produce a softer result when you used certain options. Things aren't like that anymore, thankfully.

LoRd_MuldeR

25th March 2009, 01:15

In short: Use Trellis whenever you have the time for it! Preferably Trellis=2. If you ever read that Trellis makes the image soft then that info is old ;)

Chengbin

25th March 2009, 01:19

Well, it is still a bad idea to use psy-rd on anime.

Dark Shikari

25th March 2009, 01:23

Well, it is still a bad idea to use psy-rd on anime.Psy-rd helps massively in retaining textures and dither, so I would highly recommend it when encoding anime at higher bitrates if you're trying to avoid banding.

Also, trellis (non-psy) probably helps most on anime.

LoRd_MuldeR

25th March 2009, 01:24

Well, it is still a bad idea to use psy-rd on anime.

I was referring to Trellis itself, which always should be enabled (except for very rare cases maybe).

Psy-Trellis is still off by default, even when "--trellis 2" is set. You must test whether Psy-Trellis is good or bad for your source/eyes...

Bathrone

25th March 2009, 01:29

LoRd_MuldeR

25th March 2009, 01:33

Hmmm I had a look at the mewiki x264 settings:

http://mewiki.project357.com/wiki/X264_Settings#psy-rd

Couldnt see how to enable psy-trellis? I run trellis=2, but considering LMs post how is this done?

psy-rd
Default: 1.0:0.0

So by default Psy-RDO is set to "1.0" and Psy-Trellis is set to "0.0" (aka disabled).

To enable both, Psy-RDO and Psy-Trellis, you would set "--psy-rd 1.0:1.0" for example ;)

Of course you need to experiment with the numbers...

J_Darnley

25th March 2009, 01:33

--psy-rd x:y
where y > 0

Or as stated in the link you posted:
"The second number is the strength of Psy-Trellis (requires trellis>=1 to activate)."

TheRyuu

25th March 2009, 01:34

Bathrone

25th March 2009, 01:36

Thanks I get it, its enabled through the psy-rd command.

Chengbin

25th March 2009, 01:49

How about extremely low bitrate (300-400Kbps) anime? Is it still a good idea to use trellis?

Mr VacBob

25th March 2009, 02:23

Trellis is probably more useful at lower bitrates. (since QPs are further apart and since more analysis is better at lower bitrates anyways)

Bathrone

25th March 2009, 08:06

I gave :psy-rd=1,0.1:trellis=2: a whirl and it seems decent. I'll try other values shortly.

MetalPhreak

25th March 2009, 10:24

I always use psy-rd on on anime and have never seen it look worse than an encode without psy-rd, in some areas it makes a very visible difference in quality. It's probably a different story at lower bitrates though. I would never use psy-trellis on anime though, it causes way too much ringing for my tastes.

Yoshiyuki Blade

25th March 2009, 10:53

Psy-rd helps massively in retaining textures and dither, so I would highly recommend it when encoding anime at higher bitrates if you're trying to avoid banding.

+1

Static background details look amazing at decent bitrates with psy-rd.

EDIT: Especially if you don't use god-awful spatial filters that royally kills fine details.

Sagekilla

25th March 2009, 14:30

Indeed. If anything, psy-trellis should be preferred for anime. Since x264 doesn't prefer a soft image (Which is PSNR optimal), the results will only be good.

Esurnir

25th March 2009, 23:30

Indeed. If anything, psy-trellis should be preferred for anime. Since x264 doesn't prefer a soft image (Which is PSNR optimal), the results will only be good.

trellis or rd ? Cause right now it's becoming very confusing about which of a:b is what it is about.

What does a fair rd bring to anime and what does trellis bring to it ^^;

LoRd_MuldeR

25th March 2009, 23:33

trellis or rd ? Cause right now it's becoming very confusing about which of a:b is what it is about.

The syntax is "--psy-rd a:b" where a configures Psy-RDO strength (default: 1.0) and b configures Psy-Trellis strength (default: 0.0).

Also "--trellis" must be set to at least 1 to make Psy-Trellis take effect (default is 0) and "--sub-me" must be set to at least 6 (the default) to make Psy-RDO take effect.

You can find a comparison of various "--psy-rd" combination here, although it's not an anime:
http://forum.doom9.org/showthread.php?t=141249

`Orum

27th March 2009, 18:21

This is interesting, as Sharktooth's profiles for anime, while keeping psy-rdo, lower the strength. Here's the high quality preset he has for anime for those that don't have it handy:
program --pass 2 --bitrate 1000 --stats ".stats" --ref 8 --mixed-refs --bframes 16 --b-pyramid --direct auto
--deblock 1:1 --subme 7 --trellis 2 --psy-rd 0.6:0 --partitions all --8x8dct --me umh --threads auto
--thread-input --aq-mode 0 --progress --no-psnr --no-ssim --output "output" "input"
Is 0.6 just where he found it to be a good balance for animated content? Should this value change depending on the bitrate used?

Edit: on another note, why does he disable AQ? I thought it was beneficial for anime at reasonable bitrates. Sorry for going off on a tangent :)

Sagekilla

27th March 2009, 19:27

The most likely reason would be because the new AQ has a tendency to ring on sharp lines in anime. It's very helpful in nearly all situations (It's rare that it ever degrades quality severely). But, VAQ -does- take bits away from sharp transitions (lines) and gives it to flat areas.

In anime, if you're bitrate constrained as is, your lines could end up ringing very badly.

IMO though, this is my idea of high quality for anime though:

x264 --crf 19 --ref 8 --mixed-refs --bframes 6 --b-adapt 2 --b-pyramid --weightb --deblock 0:1 --subme 9 --trellis 2 --psy-rd 0.6:0.6 --8x8dct --partitions all --me umh --threads auto --aq-strength 0.5 [...]

Try it yourself. Do you like the results you get for aq-strength 1? 0.5? 2? Experiment and learn.

Yoshiyuki Blade

29th March 2009, 01:24

For a clean animated source, I usually have the usual "maxed out" settings (--ref 16, --trellis 2, --subme 9, --b-adapt 2, etc), But scale down the AQ and psy-rd a bit. Probably 0.6 each. More psy-rd doesnt hurt as much as more AQ though. Strong AQ seems to ruin line details in motion. No AQ seems horrid no matter what the source is.