XviD with BLACK Solid Background looks Terrible !!

[Koepi]

*sigh* even if you do tv captures:

LoadPlugin("mpeg2dec3.dll")
LumaFilter()

Read more about avisynth plugins in the avisynth forum.

Koepi

[McQuaid]

Doh! I was going to try that.

thx Koepi

[McQuaid]

Actually, I just tried it but it's giving me an error saying unknown cmd for lumafilter.

I downloaded mpeg2dec from this thread

http://forum.doom9.org/showthread.php?s=&threadid=44308

as I couldn't find a version for 2.07 anywhere else.

In the readme for mpeg2dec there is mention of a param lumoff but no mention of lumafilter.

I notice in most people scripts they use the param when loading the mpg2 file. So I'm not sure what I'm doing wrong.

[Koepi]

LumaFilter is only in avisynth-2.5-compiles of mpeg2dec3.dll if I'm not mistaken :-/

Maybe you should switch over

Koepi

[trbarry]

I don't really get this problem anyway but I was playing with custom quant matrices today and noticed something I thought was interesting. The default quant number for the DC component(top left entry) in the Inter quant matrix is 16.

That seems high because whatever the DC component does spreads over the entire block, luma or chroma. Has anyone ever verified the effect on the black blocks problem using custom quants? That is, use a custom quant matrix and set the top left Inter matrix value back to 8?

Just a maybe dumb idea.

- Tom

[Didée]

I fiddled around with custom matrices lately. But I was looking for completely other things, not black-blocking ...

However, I found the following (bug?):

After several tests with custom quants, I must re-install XviD.
Why? Because, it (sometimes) happens that switching back to "normal" mpeg-quantization does not have the desired effect: somehow the last custom quantization table gets "locked".

If that happens, I confirm it the following way:
- Hit "load defaults"
- encode something with mpeg quants
- re-install same XviD build, hit "default"
- encode again with mpeg quants

The results of both encodes are not the same then, they have different size ...

Is someone able to reproduce that, er, feature?

[Didée]

Originally posted by trbarry
I don't really get this problem anyway but I was playing with custom quant matrices today and noticed something I thought was interesting. The default quant number for the DC component(top left entry) in the Inter quant matrix is 16.

That seems high because whatever the DC component does spreads over the entire block, luma or chroma. Has anyone ever verified the effect on the black blocks problem using custom quants? That is, use a custom quant matrix and set the top left Inter matrix value back to 8?

Just a maybe dumb idea.

- Tom

Yes and No - it's not that easy, alas.

The problem is that the frequency of a table's cell is not responsible/corresponding to specific content of the frame, but to the (motion-corrected) difference to ... etc. etc., you know.

So, if you simply set the very upper-left to "8", you are influencing dark/black/flat areas, indeed.
BUT you are also influencing blocks with more or even high detail, that simply didn't change that much from the preceeding frame.

Some short tests gave me an increase of filesize by 10%-20% by only changing that upper-left "16" to "8" ...

I believe that there still lies some unrevealed magic in that quantization stuff, but it's hard to find the right way and balance!


Didée

[Ewi]

Could it be a good idea to make a thread about this sticky? Since not only 2 or 3 have problems (me, too) with this strange thing it could be worth making it sticky. Perhaps a short answer how to prevent this (LumaFilter() / UnFilter()), what the reason is and if or when there will be a fix in Xvid itself (status of this thing at all). Otherwise there could be a lot more of threads like this, perhaps...

[angelyote]

[EDIT] Kilg0r3 has already done this in the XviD Q&A sticky. So don't read any further on my post please. No, really, there's nothing to see here. These are not the droids you're looking for. [/EDIT]

I'd also like to see a sticky made for 'Hints & Tips' or something similar. Perhaps in a format that gives the hint and then the link to the knowledge base under it.

ie:
To help keep pure black from looking blocky try:
loadplugin(mpeg2dec3.dll)
loadplugin(unfilter.dll)
mpeg2source(yourmovie.d2v)
lumafilter()
unfilter(-,-)
http://forum.doom9.org/showthread.ph...301&perpage=20

To help prevent injury:
don't run with scissors
http://www.uoguelph.ca/mediarel/archives/001249.html

Dave

[yaz]

Originally posted by Ewi
Could it be a good idea to make a thread about this sticky? Since not only 2 or 3 have problems (me, too) with this strange thing it could be worth making it sticky. Perhaps a short answer how to prevent this (LumaFilter() / UnFilter()), what the reason is and if or when there will be a fix in Xvid itself (status of this thing at all). Otherwise there could be a lot more of threads like this, perhaps...

motto : blocks are always there but only u're too far from the screen.

hi ewi (& all suffering from this :-)

time to time somebody opens a new thread about this topic which indicates (at least for me) that there's no unified solution of this problem. some ways work fine for some set of movies but not for others. i experienced a lot with it but i can't give a 'universal' solution (& imho, nobody else can)

anyway (kick me if i'm too lame here :-):
- blocking/tiling is inherent to all mpeg4 codec. so, u can't avoid/prevent but just minimize it & the 'minimum' is (of course) quite subjective. but (of course) there is a level unacceptable for anyone.
- this phenomenon has (can have) many different sources from hardware problems up to an improper player/postproc setting. of course, faulty filtering/encoding is also included. chasing down is quite tedious but worth to do. the most important sources are listed in the thread indicated above. check it out!

keep some things always in mind :
- improper hardware/software settings may degrade even the most perfect encode. make that clear 1st!
- removing details increases compressibility but increases (may increase) the chance of blocking. it means, e.g., never apply more filter than necessary & tune them very carefully! don't use heavy guns just for sure! sometimes, noise may be your friend not your enemy!
- u must learn (as much as possible) how the tools work. after 60(!) ugly encodes i noticed a misused parameter at a filter. if i read the manual a bit more carefully. yeah, a banging the wall.
- video encoding is a kinda lossy compression. some extent of degradation is unavoidable. we must live with it. say, the price of having a 2h movie on 1cd is some blocking here&there. does it worth or not is up to u.
- sometimes give yourself a break. lean back, take a drink & watch your encoded movie as 'normal watchers' do. u'd be surprised :-)))

i hope it helps u a bit
yaz

[Ewi]

Thank you for your post Yaz, but it was perhaps really a bit too lame...

That block based coding is inherent to MPEG4 is clear, I think; just like all the other DCT based coding schemes (In our compartment(university) was money freed for a new project of finding wavelet bases that are better suited than the actual ones for video compression (you have to know that there is not THE wavelet basis and all of them have different properties; in the last years there was a kind of competition finding new ones and we try to find bases that have properties that are esspecially suited for video comp); this project (and some others) are highly "observed" by the MPEG-C, so perhaps we can get rid of this block coding at all; but thats future...)

But I think esspecially Xvid has a problem with it. I didn't know that this is in the faq. My fault, so excuse me for being lazy... But what I miss there is a conclusion what's the reason and if somebody is on this issue to solve it (or not, whatever). Simply some kind of status info if something is happening to this or not, just informal...

Thank you for your answer...

[bobby8798]

Originally posted by Ewi
That block based coding is inherent to MPEG4 is clear,...

But I think esspecially Xvid has a problem with it

XviD is seriously deficient in that area, but the problem should be solved over time. IF the problem is extremely bad for a particular movie, just use DivX for the meantime.

Just a NOTE:
Be careful when using filters to get rid of the blocks. I just noticed someone with dark brown, or black hair will get all the details REMOVED. Sofas, Tables, and dark furniture are susceptible also.

The LumaFilter works well for the opening and end credits.

[Ewi]

Originally posted by bobby8798
Be careful when using filters to get rid of the blocks. I just noticed someone with dark brown, or black hair will get all the details REMOVED. Sofas, Tables, and dark furniture are susceptible also.

Exactly. Thats always (when it happens) my problem. It looks not only blured; it looks like if it was masked and "floodfilled" (as you know it from photoshop); the result looks very cold and computer-drawn (in these areas). I don't have screenshots at the moment cause I'm testing dev-builds of Xvid and I don't want the encodes to get in the wild (I fear later decoder incompabilities), so I delete them right after testing; but I think what the problem is, is clear....

[Holomatrix]

I'm trying to find away to fix this moving Blocky/Black areas to and was wondering what you would recommend for doing DVD to SVCD conversions?

lumafilter()
unfilter(-,-)

or Blockbuster (dither) I aleady use Blockbuster (Noise) should I use both ?

Thanks

[Koepi]

I use the lumafilter() + (optionally) Unfilter(-,-) solution (still) successfully.

Regards
Koepi

[Holomatrix]

Ok, I'll try it out. I'm testing it with low bitrates. I did a NTSC movie called Equilibrium 106 min on 1CD at 880 bitrate and for some reason it looks very good with just Lanczos Resize no other filters(5gig to 800mg) except for in the dark sceens I get this dancing black block effect. I'll try luma and unfilter and see what happens.
Thanks

[Holomatrix]

Also what setting should I use for Unfilter? I tried (-,-) but the script crashed. I'm trying with (-1,-1) now, that works. Is that ok?

[Koepi]

Unfilter(-5,-5) is a good start.

[yaz]

Originally posted by Holomatrix
Also what setting should I use for Unfilter? I tried (-,-) but the script crashed. I'm trying with (-1,-1) now, that works. Is that ok?

oops ... what does 'crash' mean here? have u got syntax error, exceptional, access violation or what? just ask cus i got problems with unfilter in yuy2 permanently but in yv12 it's ok.

when using blur/sharpen tricks take care of proper yuv scaling too. the same is true for blockbuster. i've just noticed that a proper yuv scaling at the very end of the script solves most of the blocking problems. u can check the scale with coloryuv(analyze=true). it is especially important if u (want to) watch the clips on tv.
btw, i don't recommend lumafilter. use tweak or coloryuv insted.

the bests
y

[kilg0r3]

1.)
hi, yaz
now you can help me out .

Originally posted by yaz
proper yuv scaling

what's that mean? is this some multiple of at least 4 thingy.

2.)
@Koepi
You are actually softening the image? I always that the opposite was necessary to 'keep the quantizer active'.