Squaring with AVC at 1.6mbps

[Sirber]

http://www.detritus.qc.ca/H264_RE2_1.6mbps.png

Source: Resident Evil 2 DVD
Video: Nero AVC at 1.6mbps
Audio: Nero HE-AAC at 128kbps, 5.1

I disabled the postprocecssing in the nero decoder (not supposed to be needed at 1.6mbps).

:eek:

[edit]
Source: (almost same frame, took the picture while watching it)
http://www.detritus.qc.ca/snapshot20050112191120.png

[Leo 69]

My nero encodes suffer from squaring too, especially in the dark areas of the film...Seems like psychovisual enhancements are not so perfect:confused:

[Sirber]

I asked "low" for that. I'm sure xvid can beat AVC quality with no PP...

[Leo 69]

@ Sirber

I suppose you've managed to catch a b-frame.It looks really bad :D

[Tommy Carrot]

The blocking in AVC is in fact worse than in ASP. Hopefully high profile will solve the problem, but until then, it's better to use deblocking.

[akupenguin]

In-loop deblocking in AVC is inherently adaptive. (I assume that's what you mean by postprocessing, though it's no longer "post"). So you should never need to disable it: If your bitrate allows it to use a low enough quantizer, deblocking will disable itself.

[Sirber]

it is the post procecssing from nero decoder, used for mpeg1,2,4,avc. In theory, I don't need PP above 800kbps for that rez.

[Tommy Carrot]

Sirber, AVC is not just an improved ASP, it's completely different, you cannot expect the same behaviour from it. AVC was designed to be used together with the deblocking filter, so the researchers/developers did not care what will happen if you disable the filter.

[Sirber]

ok, but wasn't the deblocker supposed to be "inloop", not a "post processing filter"?

[Tommy Carrot]

Originally posted by Sirber
ok, but wasn't the deblocker supposed to be "inloop", not a "post processing filter"?
I don't know, i don't have nero installed, i still use the dshow filter from the ateme beta package. :D

[Sirber]

lol!!! :D

[bobololo]

Originally posted by Sirber
it is the post procecssing from nero decoder, used for mpeg1,2,4,avc. In theory, I don't need PP above 800kbps for that rez.

If I'm correct, the post processing option in the nero decoding filter doesn't apply for AVC video. You could verify this by enabling the post processing and check that the picture doesn't change.

It would be interesting to know which setting you used. Your problem seems to come from a Recode known issue. Were you using a profile different from "Maximum AVC Profile" ?

ps: your "I don't need PP above 800kbps for that rez" can't be applied to AVC deblocking filter which should be always enabled. Its behaviour is quite different from usual mpeg-2/4 post processing.

[Sirber]

Nero uses it, and is enabled by default on nero decoder. I know it makes no sense, but it's there :o. I used "Standart AVC".

[babayaga]

Originally posted by Sirber
Nero uses it, and is enabled by default on nero decoder. I know it makes no sense, but it's there :o. I used "Standart AVC".
Unfortunately , it might just be the point. The "Standard AVC" maximum bitrate is 2Mbits/s. With an average of 1.6Mbits/s, there is a good chance that hard parts of the sequence reach this limit. In this case, the behaviour of the rate-control is rather "harsh" and it ends up with bad frames.

Some methods have been used in the latest release to smooth the problem but it still exists and will continue to exist.

There is no such limit in "Maximum Definition AVC" (well there are some limits but they are on the 100Mbits/s area), so you can use it safely.

About the post-processing tab in the GUI, it's useless and does nothing when playing AVC.

[neo_anderson]

hey, is this "CABAC" AND "MACROBLOCK PARTITIONS" AND "WEIGHTED PARTITIONS" ALSO SLOW DOWN ENCODING? WHICH OF THESE CAN BE DISABLED FOR A DVD MOVIE SOURCE TO NERO DIGITAL CONVERSION FOR STANDARD-AVC ?

[bond]

Originally posted by neo_anderson
hey, is this "CABAC" AND "MACROBLOCK PARTITIONS" AND "WEIGHTED PARTITIONS" ALSO SLOW DOWN ENCODING?plz dont use caps ;)

yep they will slow down encoding and decoding

WHICH OF THESE CAN BE DISABLED FOR A DVD MOVIE SOURCE TO NERO DIGITAL CONVERSION FOR STANDARD-AVC ?well you can disable all of course if you want ;)
you have to test whether they really help quality noticable yourself (cabac will never hurt quality as its lossless)

seems its really time to update my sticky :D