trellis question

[microchip8]

Hi,

x264 supports 2 trellis settings. trellis=1 and trellis=2. As I understand it, when using the first trellis setting, trellis only kicks in when needed. What I don't get is how does x264 "knows" when to use trellis and when not? Is there some threshold that must be met in something before trellis=1 kicks in?

Also, as I've read here, trellis doesn't work with deadzones. But isn't it possible to do something like...

1) use trellis and get the values
2) use deadzone and get the values
3) compare the values of trellis with those from deadzone
4) use the best value (either from trellis or from deadzone)

[Sergey A. Sablin]

Originally Posted by froggy1

Hi,

x264 supports 2 trellis settings. trellis=1 and trellis=2. As I understand it, when using the first trellis setting, trellis only kicks in when needed. What I don't get is how does x264 "knows" when to use trellis and when not? Is there some threshold that must be met in something before trellis=1 kicks in?

trellis = 1 works only for final transform (for decision between different mb modes it's not used)
trellis = 2 works every time transform is called.

Originally Posted by froggy1

Also, as I've read here, trellis doesn't work with deadzones. But isn't it possible to do something like...

1) use trellis and get the values
2) use deadzone and get the values
3) compare the values of trellis with those from deadzone
4) use the best value (either from trellis or from deadzone)

ha! the question is how to determine what's the best trellis tries to optimize RD function (and actually reaches the maximum for given mb), while dead-zones is kinda magic - you'll need (and you can) to tune them up for every movie in order to achieve whatever result you want.

[Dark Shikari]

Trellis always gives better results, PSNR-wise, than deadzone.

However, for strong grain or cases where you want more sharpness retained, low deadzone settings can look better than trellis (if that's what you want, of course). Also note that Trellis 2 has a habit of decimating grain/fine detail.

[Sagekilla]

Originally Posted by Dark Shikari

Trellis always gives better results, PSNR-wise, than deadzone.

However, for strong grain or cases where you want more sharpness retained, low deadzone settings can look better than trellis (if that's what you want, of course). Also note that Trellis 2 has a habit of decimating grain/fine detail.

But isn't Trellis "optimized" for PSNR anyway?

[ToS_Maverick]

Dark Shikari, didn't you mention somewhere that you develop(ed) an adaptive deadzone algorythm?

[Dark Shikari]

Originally Posted by Sagekilla

But isn't Trellis "optimized" for PSNR anyway?

That's the point; its optimized for PSNR vs bits, and as a result can decimate fine detail in favor of bitrate savings.

Originally Posted by ToS_Maverick

Dark Shikari, didn't you mention somewhere that you develop(ed) an adaptive deadzone algorythm?

Yeah, but it was pretty crappy IMO. I need a good "reason" to use a lower deadzone; i.e. a way to tell the algorithm what kind of blocks lower deadzone is good for.