lumi masking

[iago]

Hello all,

Well, my purpose is absolutely not to revive an old thread, where some important conclusions regarding the issue have already been drawn. Still, I find it most proper to first discuss the matter in this thread as an introduction to the topic.

And I assume, most people are "generally" using their PC monitors to watch their encodes, just like me . However, when it comes to TV, the quality of any rip is totally another story, and even your most "trusted" and "perfect" looking rips may well frustrate you when you watch them on TV without lowering brightness and contrast, or sometimes even when you watch them on your monitor especially in a dark environment, with a rather high resolution, full brightness and contrast, and no extra lights in the surrounding! (So, those who don't bother watching their rips with such monitor settings and in totally dark environments, and those who don't want/need/have TV output for any reason might totally ignore this post! )

OK, lately I've been doing a couple of tests, *with* and *without* lumi masking; using Koepi's 04092002-1 build, and statsreader 1.6 to linear-scale the curve for external cc (with the already well known parameters to all: Motion search:6 / hi-lo:0-0 / altCC disabled, etc.), to have a look at the problem of blockiness in dark areas. Also, these tests are/were not limited to XviD only. (SBC and DivX502 Pro, more or less, but still absolutely exhibiting the same problem in dark areas, were also tested several times, just resulting in the same ongoing problem.)

Back to XviD, the aim of all these tries was to see if there would be any difference/improvement towards eliminating this problem by tweaking some settings, and mainly by disabling "lumi masking", my all-time 'usual suspect' concerning this issue .

Surprisingly, after all these tries with XviD and other codecs mentioned (well, actually these tries of mine have been going on for a very very long time ), I concluded that the problem hasn't got much to do with lumi masking/luminance correction in XviD/SBC respectively, nor with psychovisual enhancements in DivX502Pro. Whatever you do, you cannot eliminate them totally! It seems to be (as always discussed) more or less an innate problem to all codecs. And that's why there are options such as "noise" in ffdshow and "film-effect" in DivX5 decoder...

And now comes the interesting/striking point: Concerning XviD again, the only rips in all my tries that displayed this problem *least* (though certainly still having the problem to some extent) were the ones with MPEG/MPEG quantization and with Bicubic Resizing (=/> 0,0.5), which possibly "preserved" the original DVD noise more than the others. DVD noise is also clearly distinguishable especially in dark areas of the films (not in the form of ugly blocks of course but as "noise") when you watch the DVD carefully on monitor or on TV with the above mentioned settings.

Finally, these are only my "personal" observations for a long period of time, might contain mistakes, and absolutely open to/requires much discussion and sharing of experiences, which might even initiate a step towards the solution of a well-known but still unsolved problem.

Maybe, (in order to solve this major-irritating problem) we should totally change our attitude towards encoding and try to "keep" the *noise* of the original DVD as much as possible instead of trying to "eliminate" it...


Important Note: All the above argument is about "regular/normal movies" and "DVD rips of such movies"; not about TV content, anime, etc. Also, a solution "may" or "may not" be fully possible technically of course, but still this might be taken as just a different look/perspective to the encoding process and our encoding habits .

P.S.: You can try enabling "Noise" in ffdshow with "Old noise algorithm" and "Uniform noise" with "luminance noise strength=8" (chroma noise strength=0) to observe "the difference/the most possible original DVD-like effect" and see these blocks in dark areas almost completely disappear, though that's not a perfect solution to the problem of course .


kindest regards to all
and many thanks for taking the time to read such a long and boring post

iago

EDIT: Well, I guess I have to add SimpleResize, the great non-smoothing, non-filtering filter by trbarry, besides "Bicubic Resize =/> 0,0.5" too.

[Marc FD]

i noticed many times blocking in black scenes when i used my TV out.
for example in an anime (Noir) i wanted to see on my TV (16/9) :
It was SBC compressed and the blocking in dark areas was horrible.
Even ffdshow with new noise 80/40 PostPproc cpu-6 200% and the image was still horrible to see !!
Using Black-Expand on my TV help a bit.
In the end, i used Levels() and it was very good. but too slow for playback.

Now,i see a very simple way to get rid of this annoying effect :
Make Lumi-Masking in Post-Processing stage.
Could even be done fast using the DC value of each MB, no ?
I can try this with my MPEG2Dec PP mod, but i'm too lasy to move my computer near my TV

and it's only for DVD playback (i love to use BSplayer to see DVDs )
not to play avi's (the PP should be built in the decoder)

I suggest you to try Levels() (of course you tweak the values)
If you see improvements, we could (any avs dev.) MMX optimise it.

[-h]

I've been thinking of adding a switch to XviD that adds AC "noise" to blocks whose luminance is below a certain threshold. I too have just started watching MPEG-4 on a TV and the result is truly horrible, I agree.

I would rather the encoder handle improving the areas than leaving it up to the post-processor, because eventually we may not have such fine-grained control over how the post-processor behaves (think hardware MPEG-4 players).

-h

[Marc FD]

AC ? i don't remember the meaning.
can you enlighten me ?
thx.

[iago]

I've been thinking of adding a switch to XviD that adds AC "noise" to blocks whose luminance is below a certain threshold. I too have just started watching MPEG-4 on a TV and the result is truly horrible, I agree.

I would rather the encoder handle improving the areas than leaving it up to the post-processor, because eventually we may not have such fine-grained control over how the post-processor behaves (think hardware MPEG-4 players).

@-h

Yes, absolutely that could be the one and best possible solution to the problem, and it would actually be great. I totally agree with you that it's much better to make it possible in the encoding process adding noise to dark areas below a certain threshold, and keeping the rest of the clean MPEG-4 picture alone. Hence, post-processing by adding noise (which might not be handled with full-control and success as you pointed out) can only be an emergency-exit to watch "poor" encodes on TV, which still contain the blocks problem; but by eliminating this "horrible" problem in the encoding stage, the "original" encode will be much improved, and that would really be a great step in the ripping world imho and open a new era for XviD to display its real perfect performance.

best regards,
iago

[Marc FD]

the problem is dumb guys (me) like _deep_ more-black-than-black black and don't like noisy stuff. (it's why i'm a h263 fan )

[-h]

AC ? i don't remember the meaning.

The DCT output consists of 1 DC and 63 AC values. As the DC value is the "average" of the entire block, the only coefficients worth setting to simulate noise are the higher AC ones, which concern progressively higher frequencies.

Here is an image with a graphical representation of the 64 coefficients and how they represent the image itself.

..it's much better to make it possible in the encoding process adding noise to dark areas below a certain threshold, and keeping the rest of the clean MPEG-4 picture alone.

The trick will be adding the right amount, without having it look just as bad only with different artifacts.

-h

[iago]

Hello again,

Well, it's getting more complicated actually, and I hate to do that but I think I have to make some corrections on my long post above, especially regarding lumi-masking, based on some new test results; though the main points discussed so far and the solution suggested by -h are still consistent and valid imho.

First of all, based on the new test results carried out with Nic's and Koepi's latest builds, using 1-pass constant quantizer 2, encoding the very beginning scenes of Matrix, with and without lumi masking, using both MPEG and h263 quantization types, and watching the test results very carefully (with no post-processing) both "on TV" and "on PC monitor with high brightness/contrast, high resolution, in completely dark environment at night with no lights on" , I can absolutely say that lumi-masking introduces more blockiness in dark areas and certainly causes more harm than benefits.

Briefly we can list the quality of the several quantizer 2 encodes in terms of blockiness in dark areas, from the best to the worst as follows: MPEG-NOLUMI -> H263-NOLUMI -> MPEG-LUMI ->/= H263-LUMI.

And actually the MPEG-NOLUMI encodes (both Koepi's and Nic's latest builds) were the ones closest to the appearance of actual DVD output in dark areas, bearing only the DVD-like noise rather than ugly blocks!

However, although these tests clearly show that lumi-masking is absolutely not our best-friend, there is another important issue too. The tests mentioned above are done using constant quantizer 2 in order to achieve the best possible quality with the given parameters and after that to compare the results with each other and see if lumi masking causes trouble or not. As a result, it really did!

But the point is that whenever you go higher than quantizer 2 (also tested constant quantizer 3 and 4), blocks begin to appear even without lumi-masking and whatever the quantization type (MPEG or h263) is. And as in 2-pass variable bitrate mode it's almost impossible to have an average quant 2 for most movies to ensure quality and elimination of blocking in dark areas, the solution pointed out by -h is still of vital importance.

But at least, these last tests confirmed my never-ending suspicion that lumi-masking was somehow and to some extent responsible for blockiness in dark areas, and now I certainly find it more secure not to use lumi-masking in my rips, and continue with MPEG/MPEG quantization as I've been already doing for a while.

And one more important point: using postprocessing with deblock/dering either with "presets" or with "custom" in ffdshow absolutely ruins the quality and introduces many more ugly blocks in dark scenes besides some artifacts. The best way for decoding XviD is using Nic's decoder filter (safely with or without postprocessing) imho. But if you "really want" to use ffdshow, use it either without postprocessing at all or only with "automatic quality control" without touching the rest in the postprocessing tab!

Well, that's all for now, and I'm really exhausted at the moment. Directly going to bed for some sleep!


regards to all,
ciao,

iago

[colasonic]

Originally posted by iago
Hello all,
And I assume, most people are "generally" using their PC monitors to watch their encodes, just like me . However, when it comes to TV, the quality of any rip is totally another story, and even your most "trusted" and "perfect" looking rips may well frustrate you when you watch them on TV without lowering brightness and contrast, or sometimes even when you watch them on your monitor especially in a dark environment, with a rather high resolution, full brightness and contrast, and no extra lights in the surrounding! (So, those who don't bother watching their rips with such monitor settings and in totally dark environments

I think the reason is on the screen size. usually, a TV screen is much bigger than our PC screen. I don't have the experience to watch a DVDRip on a TV set, but I do know that some Rips which appear good-looking on my 17" monitor turn out to be crappy on my another PC with 19" trinitron. the reason may be the same.....

[Koepi]

As iago wrote on his first post on this thread, you should use ffdshow's noise option to get a propper image again.

Since iago saw these blocks already at quant. 3, this seems the only thing working for now. Maybe -h's attempt can help it, but I'm not sure if this doesn't blow up the video stream massively again (in size, I mean) :-/

In that case (using ffdshow's noise option) it might be possible to use lumimasking as well, but to be on the save side, for HQ TV playback you shouldn't use it.

The only option is:

You want DVD quality? Buy the DVD and keep it.
or: make a 5CD rip at quant 2.

:P

When puting movies on 1CD (from >5GB before) you have to sacrifice quality for sure. Just wanted to mention that again

Best regards,
Koepi

[Marc FD]

I think too that AC noise is not the solution.
Maybe iago can check it, but i think the whole problem in black areas is:
1) Black is very sensitive to quantization, because AC/DC values are near to zero and get easily quantized.
2) Many blocks are skipped int these areas (so AC noise is not a solution)
3) Lumi masking is increasing the quantizers, so it's even worse.
4) The DVD source is already bad in black areas. Rencoding it with XviD is amplificating this problem !

Maybe best than AC noise, i suggest to use a Black-Expand/Black-Normalisation Pre-Processing technique.
The Pre-Processing could be done in the DCT domain.
And using a stronger Pre-Processing with Lumi-Masking, you would even get rid of the bad black blocks...

@iago

i know it's not fair to ask it to a XviD fan, but can you compare with RV9 and it's "black pre processing" (i don't remeber the name) option. i think it worth a try to compare.
thx.

[iago]

Originally posted by Marc FD
@iago

i know it's not fair to ask it to a XviD fan, but can you compare with RV9 and it's "black pre processing" (i don't remeber the name) option. i think it worth a try to compare. thx.

@Marc FD,

I would absolutely do that if I had any experience with RV9 , but unfortunately I don't... Maybe someone else could try that and post the results, since RV9 (in Doom9's latest comparison) really seems to have solved this issue to some extent.

Yes, maybe all we need is "more black and black"!

best regards,
iago

[Marc FD]

Originally posted by iago

Yes, maybe all we need is "more black and black"!

Not exaclty. We need to correct the "bad black" on the sure to get "true black". DCT is allergic to "almost-black-but-not-exaclty-black black"

I think it's how RV9 makes the trick.
And i see how it could be done by XviD too

Since i'm going to implement XviD's Pre Processing, it could be a very usefull task to start with.

[iago]

Originally posted by iago
And one more important point: using postprocessing with deblock/dering either with "presets" or with "custom" in ffdshow absolutely ruins the quality and introduces many more ugly blocks in dark scenes besides some artifacts. The best way for decoding XviD is using Nic's decoder filter (safely with or without postprocessing) imho. But if you "really want" to use ffdshow, use it either without postprocessing at all or only with "automatic quality control" without touching the rest in the postprocessing tab!

Well, it seems that even Nic's decoder filter causes this (more) blocking problem in dark areas when deblock and dering options are enabled! So to be on the safer side, either ffdshow or Nic's XviD decoder I go *without* postprocessing .

best regards,
iago

P.S.: I really wonder if I'm lucky or not to have a TV output and a Philips 107T21 17" monitor with LightFrame option on it to notice *all* these ugly blocks in dark areas!. Perhaps, I could be much happier *without* them!

[Marc FD]

can you please try with Levels ?

[Dali Lama]

@all

This is a great topic. One which should be hammered out until resolved, I believe. I tried to start a similar thread in the AVIsynth forum about restoring the blackness of a film, but maybe I wasn't too clear.

@ Iago

Recently I found a filter called ColorYUY2. It is made by a Japanese person. It has a really useful feature that restores the levels of a film from TV-->PC or PC-->TV.

Simply input:

ColorYUY2(Levels="TV->PC")

I would be delighted to see what you find when you try this filter before resize and no other filtering concerning the blockiness in black areas. For me it seems to significantly reduce them.

@ MarcFD

I believe that you were mentioning this when you spoke of levels, correct? And yes, I have used RV9 a fair amount and can contest that it handles black much better than other codecs. However, their black restore filter simply darkens the movie too much. Also, try out that ColorYUY2 filter, it seems to darken the movie, but retain original contrast and saturation.

One other note, I am using the Lanczos3 Resize filter after ColorYUY2. This may also help in achieving the better encoding in dark areas.

Lets solve this pesky problem!!

Dali

[-h]

2) Many blocks are skipped int these areas (so AC noise is not a solution)

AC noise would prevent the block from being skipped.

Maybe best than AC noise, i suggest to use a Black-Expand/Black-Normalisation Pre-Processing technique.

How would it work though? If you blacked out a large area with thresholding to pure black, you're going to get the same ugly blocks at boundary areas (and an odd smooth block on TV where you expect to see some kind of texture).

MPEG-4 and pure black are nasty things to get working. Whatever technique gets used in the end, it'll just require a heap of testing I suppose.

-h

[Marc FD]

agreed.
we need testing.
It's why i propose :
- To pre-filter / post-filter with Levels just to see if it's good.
- To see how RV9 handles this problem with it's special tool.

PS : The Black Expanding technique i propose is different and more complex than Levels. I think you can smooth it spatially in the DCT domain to avoid blockiness. Need to be tested....

[iago]

Originally posted by Dali Lama
@ Iago

Recently I found a filter called ColorYUY2. It is made by a Japanese person. It has a really useful feature that restores the levels of a film from TV-->PC or PC-->TV.

@Dali Lama,

Thanks a lot for the info you provided about this filter. I tested it several times and it really did a great job, just suiting my needs! As you also said, it seems to darken the movie a bit, but at least it doesn't ruin the encode . I applied it just before SimpleResize, and made numerous tests "with" and "without" using it to compare the difference, with constant quantizers up to 10, and even with the "guilty" lumi masking, as well as using h.263 quantization type too. As a result, in all my tests, it performed a fantastic job, eliminating all blocks in dark areas, and converting them to pure clean black colour.

For the time being, until an innate solution within the codec itself is achieved, I guess I'll certainly use that filter without hesitation in my encodes.

And for now, until the revolutionary step of implementing an innate solution is taken, the other possibility stands as adding noise in ffdshow (uniform ~8 with old noise algo), which is absolutely an inferior solution compared to the above filter's job imho.


kindest regards and many thanks again,
iago

[-h]

For the time being, until an innate solution within the codec itself is achieved, I guess I'll certainly use that filter without hesitation in my encodes.

That sounds like a nice solution. I wouldn't like XviD's default behaviour to be altering the input (i.e. preprocessing) though, and as well as this method works I would still like to investigate one that doesn't resort to changing levels.

Of course the levels system could be integrated as a "preserve black" or "optimize for TV" option or however it should be categorized.

-h