black blocks within bright flashes with QuEnc

[cfriedl]

I encoded a VHS capture of Judge Dredd and got black blocks in some of the bright flashes during gun fire scenes.

http://img149.exs.cx/img149/9219/flashquenc0544mc.jpg

The filter part of my avs file follows. THe rest is source AVI and trimming
.
.
movie=movie.ConvertToYUY2()
movie=movie.smoothdeinterlace()
movie=movie.peachsmoother()
movie=movie.Crop(8,8,-24,-8)
movie=movie.AddBorders(16,8,16,8,0)
movie=movie.ConvertToYV12()
.
.
source is 704x576 avi (Pinnacle codec) captured with DV500
Quenc is v0.54
VBR
High Quality
Trellis Quant
2 Pass Encode
6388 kbps (calcd to fit a DVD)
max bitrate 8000
GOP 12
max B frames is 2
10 bit DC precision
4:3 aspect ratio

I've seen this in the earlier versions but not having seen it for a while I figured it was gone. Have a vague recollection of this having been reported some time ago but can't find the thread. IN thepast I tried changing all the setting above but to no change. Only fix was to encode with BBMpeg but its encodes aren't as nice as Quenc.

Anyone else experienced this? Anyone have any idea what to do to fix it? Quenc settings?

[dragongodz]

put and link to your images here
_http://www.imageshack.us/

do you see any anomoly in these parts if you play the avs in a media player or step to those frames in virtualdub ?

[cfriedl]

thanks for the imageshack link, and thanks to imageshack for the service.

No anomolies in the source avi in any player (WMP, VDub-MPEG2, MPC).
No anomolies in the output of avs in any player above.
Anomolies appear only in the mpeg file in players (PowerDVD, VDub-MPEG2)

Note that the problem isn't consistent (the worst kind). Here is a frame a fraction of a second later which is nominally identical, but which was encoded properly.

http://img53.exs.cx/img53/7459/flashquenc054good8uz.jpg

:o I'm afraid, not thinking, I deleted the source avi yesterday after running out of space. Thought I'd recapture the scene to investigate further but then remembered I returned the capture card which was on loan through my job which I left last week. So right now, in terms of being able to follow up on this issue I'm afraid I'm up shit creek without a paddle. :(

[Peter Cheat]

This is a well-known bug with trellis, certain matrices cause overflow/precision problems, especially in parts of the screen where there are bright flashes. Turning off trellis quantisation will fix the problem.

[cfriedl]

thanks. Will try when next I can. Just one question ... can you give me a link to discussion on this problem. I'd like to try and understand it a little more and understand what I'm losing by switching it off.

[Peter Cheat]

I can't point you too a specific thread about this, but I can tell you that trellis searched quantisation can improve perceptual quality, making a sequence appear sharper than without. On the downside it slows down encoding a lot. For very high bitrates, such as the bitrate you were targetting, I personally would not use it. The fact that blocking sometimes occurs is also a downside, but can be avoided by using a different matrix, which I don't think is an option in Quenc 0.54???

[Nic]

There has been a discussion on it in the ffmpeg-devel mailing list. There's even been a patch to try and help, but mainly now it throws a warning message stating that an overflow can occur which causes these problems...

The mailing list should be archived at www.sourceforge.net/projects/ffmpeg

-Nic

[dragongodz]

yes they flatly refuse to reduce precision(which fixes it). why not make the reduction an option even ? that would make more sense than letting a known and reproducable bug to continue with nothing more than a bloody warning.

[Mug Funky]

actually, the argument offered in the mailing list is a good one i think - if they were to reduce precision to increase range, what's saying another custom matrix wont appear with even more difficult numbers?

is there a good general set of rules for constructing a custom matrix (like don't put in numbers likely to cause overflows)?

[edit]

it also said that making the bit-shift a variable would significantly slow things down.

[dragongodz]

ok how about the fact that QuEnc 0.54 only has 1 custom matrix which is QLB(which is very conservative as custom matrices go) and has not produced the trellis bug before ? infact if you look at the settings cfriedl lists using QLB is not even mentioned. now if the bug is happening without even using a custom matrix then that is a major problem. obviously catching the overflow and clipping/correcting it would be ideal but until someone actually works out how to do that a small reduction in precision is better than leaving it as is IMHO, have a look at the latest xvid release and you will see they think the same(though they limit it to if quant 1). :)

it also said that making the bit-shift a variable would significantly slow things down.
and the fact that it already is ? #define QMAT_SHIFT
ok its really a constant but thats basically a variable that you cant change after its intialised. i have never seen anything mentioned about any speed difference, anyone else ?

is there a good general set of rules for constructing a custom matrix (like don't put in numbers likely to cause overflows)?
i think i saw someone posts some general rules a long time ago but forget where and when now.

[Peter Cheat]

Originally posted by Mug Funky
actually, the argument offered in the mailing list is a good one i think - if they were to reduce precision to increase range, what's saying another custom matrix wont appear with even more difficult numbers?


I posted the precision fix on the mailing list, and only matrices that use values <=4 will have problems. I've never seen anyone use matrices with values less than 8, and I don't even see any benefits of going so low anyway (use lossless encoding instead!).

Making QMAT_SHIFT dynamic will slow down trellis, the variable will no longer be a static constant and will be compiled differently. Another solution is to write 2 trellis_dct_encode functions, one that uses more precision and another that uses less precision. This will duplicate code, but at least it won't slow it down.


Is there a good general set of rules for constructing a custom matrix (like don't put in numbers likely to cause overflows)?


Use smaller values.

[dragongodz]

Making QMAT_SHIFT dynamic will slow down trellis, the variable will no longer be a static constant and will be compiled differently.
obviously it would compile slightly differently but has anyone ever tested o see if their is any real difference speed wise ? i can find no information on it and being the time of year it is have bugger all time to properly test it myself.

Another solution is to write 2 trellis_dct_encode functions, one that uses more precision and another that uses less precision. This will duplicate code, but at least it won't slow it down.
i would rather see some proof that there would be a noticable speed difference first but yes that is another solution.

[Peter Cheat]

The difference is Immediate vs. Absolute (or Direct) addressing. I know these terms apply to Motorola processors, Intel might have called it something else.

Immediate means the value is loaded straight into the accumulator (or index register), direct means its copied from a place in memory. Here's an example (for the Motorola 6502/6510 processors, my good old Commodore 64). Btw, 8 bit processors ruled :D.


LDA #$05 ; Load value 5(hex) into Accumulator (immediate)

or

LDA $0005 ; Load value stored in memory location 0005(hex) into Accumulator (absolute, direct, extended or whatever you call it)


Now the difference between the two is that immediate is 200% faster (2 cycles vs. 4 cycles). This is basically true for all Motorola processors I've ever programmed, such as the 680x0 series (Commodore Amiga), or 6800 processor.

I don't know Intel at all (I guess it is the same, maybe different endianess), but I can imagine it will take longer to copy a value from one part of memory to another, than to just load a value into memory. Since QMAT_SHIFT is used several times inside the trellis_dct function, and this is looped around for each MB, I can image encoding will slow down significantly. Considering trellis is already slow because it isn't MMX optimised (and I will have to learn intel asm before I have a go at it!), slowing it down more isn't good.

The problem is range vs. precision. I can see another solution I overlooked - increase the number of bits to increase the range. Unfortunately, this would also slow down encoding, but maybe not by much?

[dragongodz]

This is basically true for all Motorola processors
and we are not using Motorola processors now are we ? :)

I don't know Intel at all (I guess it is the same, maybe different endianess), but I can imagine it will take longer to copy a value from one part of memory to another, than to just load a value into memory
yes but i was asking has anyone actually ever tested or know of it being tested to see the real difference, especially with GCC because thats what it pertains to in this case. that is what i tried to find along with real cycle counts but no luck.

Considering trellis is already slow because it isn't MMX optimised
and i see no sign of that changing any time soon. well atleast it hasnt been mentioned on the mailing list.

Unfortunately, this would also slow down encoding, but maybe not by much?
though of course the guys on FFMpeg are so A.R. about anything that causes even the smallest speed decrease it would have to be absolutly tiny.