lumi masking [Archive] - Doom9's Forum

iago

14th August 2002, 23:06

Hello everybody,

Here are the results of the lumi masking test done with the 08082002-1 build. The same movie is encoded twice, with the exact same parameters, except that in one encode lumi masking is used in both passes, and in the other no lumi masking is used at all. Respectively, I'll refer to these encodes as "LUMI" and "NOLUMI" here and in the screenshots I provide for comparison.

Movie: The Score (119 min)
resolution/resize: 640*272/neutral bicubic
video size: 645000 kb

First Pass:
UltraHigh
H.263

Second Pass:
UltraHigh
Modulated
Min-Max I-frame: 2-4
Min-Max P-frame: 2-8
I-frame boost: 0
Below I-frame: 10
I-frame bitrate reduction: 30
Bitrate payback: 250 (bias)
AltCC: medium/500/90/strength:30
Enabled automatic bonus bias calc.

Max-Min I-frame intervals: 250-1 (both passes)
SCD threshold: 50 (both passes)

credits: 31quant

Results (LUMI)

DebugView analyzer for XviD codec v0.10 by MoonWalker
e-mail : s_ilias@gmx.net

Quantizers Analisis
---------------------

Quantizers Used For Movie :
------------------------------
Quant 2 Used : 2783 Times, Percentage Used : 1.63%
Quant 3 Used : 101929 Times, Percentage Used : 59.86%
Quant 4 Used : 61221 Times, Percentage Used : 35.95%
Quant 5 Used : 3476 Times, Percentage Used : 2.04%
Quant 6 Used : 519 Times, Percentage Used : 0.30%
Quant 7 Used : 258 Times, Percentage Used : 0.15%
Quant 8 Used : 99 Times, Percentage Used : 0.06%

Average Quantizer Used for Movie : 3.402

Quantizers Used For Credits :
--------------------------------
Quant 31 Used : 8293 Times.

MPEG Quantization Type Used 113005 timed, Percentage Used : 63.28%

H.263 Quantization Type Used 65573 timed, Percentage Used : 36.72%

Quantizers prevented from rising too steeply 0 times

Intra-Frame (Key-Frame) Quantizers
------------------------------------

Movie
-------
Quant 3 Used : 536 Times, Percentage Used : 79.29%
Quant 4 Used : 98 Times, Percentage Used : 14.50%

Credits
---------
Quant 31 Used : 42 Times, Percentage Used : 6.21%

Number Of Consecutive I-Frames : 9

Inter-Frame (P-Frame) Quantizers
------------------------------------

Movie
-------
Quant 2 Used : 2783 Times, Percentage Used : 1.56%
Quant 3 Used : 101393 Times, Percentage Used : 56.99%
Quant 4 Used : 61123 Times, Percentage Used : 34.36%
Quant 5 Used : 3476 Times, Percentage Used : 1.95%
Quant 6 Used : 519 Times, Percentage Used : 0.29%
Quant 7 Used : 258 Times, Percentage Used : 0.15%
Quant 8 Used : 99 Times, Percentage Used : 0.06%

Credits
---------
Quant 31 Used : 8251 Times, Percentage Used : 4.64%

Frame Analisis
----------------

Number Of Intra-Frames (Key-Frames) : 676
Number Of Inter-Frames (P-Frames) : 177902
Total Number Of Frames : 178578

0.38% of the Movie is Intra-Frames (Key-Frames)
99.62% of the Movie is Inter-Frames (P-Frames)

Size Analysis
----------------

1-Pass Size : 1210679771 Bytes or 1182304 KBytes
Scaled Size : 656193554 Bytes or 640814 KBytes
Actual Size : 656176664 Bytes or 640797 KBytes

Usefull Statistics
------------------
Compressibility : 54.20%
Relative Quality of XviD avi : 58.79%
Absolute Quality of XviD avi : 95.79%

Results (NOLUMI)

DebugView analyzer for XviD codec v0.10 by MoonWalker
e-mail : s_ilias@gmx.net

Quantizers Analisis
---------------------

Quantizers Used For Movie :
------------------------------
Quant 2 Used : 1704 Times, Percentage Used : 1.00%
Quant 3 Used : 93312 Times, Percentage Used : 54.80%
Quant 4 Used : 73227 Times, Percentage Used : 43.00%
Quant 5 Used : 1925 Times, Percentage Used : 1.13%
Quant 6 Used : 113 Times, Percentage Used : 0.07%
Quant 7 Used : 3 Times, Percentage Used : 0.00%
Quant 8 Used : 1 Times, Percentage Used : 0.00%

Average Quantizer Used for Movie : 3.445

Quantizers Used For Credits :
--------------------------------
Quant 31 Used : 8293 Times.

MPEG Quantization Type Used 103309 timed, Percentage Used : 57.85%

H.263 Quantization Type Used 75269 timed, Percentage Used : 42.15%

Quantizers prevented from rising too steeply 0 times

Intra-Frame (Key-Frame) Quantizers
------------------------------------

Movie
-------
Quant 3 Used : 1218 Times, Percentage Used : 86.75%
Quant 4 Used : 144 Times, Percentage Used : 10.26%

Credits
---------
Quant 31 Used : 42 Times, Percentage Used : 2.99%

Number Of Consecutive I-Frames : 51

Inter-Frame (P-Frame) Quantizers
------------------------------------

Movie
-------
Quant 2 Used : 1704 Times, Percentage Used : 0.96%
Quant 3 Used : 92094 Times, Percentage Used : 51.98%
Quant 4 Used : 73083 Times, Percentage Used : 41.25%
Quant 5 Used : 1925 Times, Percentage Used : 1.09%
Quant 6 Used : 113 Times, Percentage Used : 0.06%
Quant 7 Used : 3 Times, Percentage Used : 0.00%
Quant 8 Used : 1 Times, Percentage Used : 0.00%

Credits
---------
Quant 31 Used : 8251 Times, Percentage Used : 4.66%

Frame Analisis
----------------

Number Of Intra-Frames (Key-Frames) : 1404
Number Of Inter-Frames (P-Frames) : 177174
Total Number Of Frames : 178578

0.79% of the Movie is Intra-Frames (Key-Frames)
99.21% of the Movie is Inter-Frames (P-Frames)

Size Analysis
----------------

1-Pass Size : 1265838928 Bytes or 1236170 KBytes
Scaled Size : 656193878 Bytes or 640814 KBytes
Actual Size : 656182560 Bytes or 640803 KBytes

Usefull Statistics
------------------
Compressibility : 51.84%
Relative Quality of XviD avi : 58.06%
Absolute Quality of XviD avi : 95.67%

-----------------------------------------------------------

Finally, I'm making no comments this time, and leaving the evaluation job up to you :). Also I'm providing screenshots (taken in VirtualDub) of both encodes (LUMI/NOLUMI) for those who are interested.

so long,
iago

iago

15th August 2002, 01:45

Sorry for the double post; it was not my fault, though. (Must be some problem occured related with the "preview reply" button.) Two posts above are exactly the same :(.

Hope the screenshots attachment gets approved soon so we can talk about the results more accurately.

best regards to all,
iago

Koepi

15th August 2002, 01:47

Which double post?

Just kidding. I delted one of them :)

Regards,
Koepi

iago

15th August 2002, 01:51

@Koepi

LOL! Either need my glasses or have to sober up soon ;). Thanks anyway.

iago :D

iago

15th August 2002, 13:38

Hello again,

Well, after so much trying and spending long hours on test encodes and after watching both encodes (LUMI and NOLUMI) very carefully, I would like to comment a bit on the results.

First of all, it is very clear that using lumi masking in both passes is very helpful in terms of increasing compressibility (LUMI: 54.20%, NOLUMI: 51.84%) and in terms of reaching lower quantizers (obviously more 2 and 3 quantizers) for the movie.

Secondly, comparing the two encodes frame by frame and judging by (only) the screenshots (of the darkest and most likely-to-be-problematic scenes) would be absolutely misleading (after all, you can't see much difference, can you? ;)), since when you watch and compare the two encodes (using ffdshow and of course choosing "decode using XviD" in the codecs tab and selecting "IDCT: XviD" in the Miscallenous tab), you can't spot any quality loss or artifacts in the "both passes LUMI" encode.

Next, the settings used in the first and second passes and the AltCC parameters (suggested by Koepi) really seem to work fine and both encodes (LUMI/NOLUMI) are really impressing and of pretty high quality :).

Finally, of course I choose to keep the "both passes LUMI" encode for my 1cd rip (with 0.100q ogg) of this movie. And my vote is now (with much more certainty) for "lumi masking in both passes" ;).

best regards to all,
iago

ookzDVD

16th August 2002, 04:48

@iago,

Thank you for your testing, I really apreciate that,
and I really support your test result. ;)

MaTTeR

16th August 2002, 13:06

Originally posted by BiaTch 5.0
I used 27072002-1 to encoded from hell (PAL/Field De)both passes & I got pinky arifacts in the background then just second pass and all was fine.

I'm actually seeing these artifacts also on 3 out of the last 6 movies I ripped using the 08-08 build. I thought the culprit might be the alt-cc settings so I disabled and still seeing the same issue. Hope to post full details and screenshots about it this weekend.

In regards to Lumi...I've never stopped using it since day one on both passes. It was always clear to me the advantages of using it, especially since the old code provides more compressibility over all. Remember, lumi-masking is going to work on bright scenes just as well as drak scenes. So for a bright movie(Mary Poppins? LOL), you should see some nice results with it as well.

BiaTch 5.0

16th August 2002, 23:33

Originally posted by MaTTeR

I'm actually seeing these artifacts also on 3 out of the last 6 movies I ripped using the 08-08 build. I thought the culprit might be the alt-cc settings so I disabled and still seeing the same issue. Hope to post full details and screenshots about it this weekend.

In regards to Lumi...I've never stopped using it since day one on both passes. It was always clear to me the advantages of using it, especially since the old code provides more compressibility over all. Remember, lumi-masking is going to work on bright scenes just as well as drak scenes. So for a bright movie(Mary Poppins? LOL), you should see some nice results with it as well.
Have you tried encoding with Lumi disabled or only In second pass?.

MaTTeR

17th August 2002, 01:48

Originally posted by BiaTch 5.0

Have you tried encoding with Lumi disabled or only In second pass?.
I actually tried with it disabled and still had nasty artifacts appearing. I have not tried just using it in the 2nd pass alone though. I'll give that a shot before I post the screenshots tomorrow.

Koepi

17th August 2002, 02:05

There might be no need for screenshots, it's possibly a bug which is investigated ATM.

Hopefully the fix is in CVS soon :)

Regards,
Koepi

MaTTeR

17th August 2002, 02:58

Thanks so much Koepi;) Do let me know if you need the screenshots and encoding details.

iago

10th September 2002, 16:07

Hello all,

Well, my purpose is absolutely not to revive an old thread, where some important conclusions regarding the issue have already been drawn. Still, I find it most proper to first discuss the matter in this thread as an introduction to the topic.

And I assume, most people are "generally" using their PC monitors to watch their encodes, just like me ;). However, when it comes to TV, the quality of any rip is totally another story, and even your most "trusted" and "perfect" looking rips may well frustrate you when you watch them on TV without lowering brightness and contrast, or sometimes even when you watch them on your monitor especially in a dark environment, with a rather high resolution, full brightness and contrast, and no extra lights in the surrounding! (So, those who don't bother watching their rips with such monitor settings and in totally dark environments, and those who don't want/need/have TV output for any reason might totally ignore this post! ;))

OK, lately I've been doing a couple of tests, *with* and *without* lumi masking; using Koepi's 04092002-1 build, and statsreader 1.6 to linear-scale the curve for external cc (with the already well known parameters to all: Motion search:6 / hi-lo:0-0 / altCC disabled, etc.), to have a look at the problem of blockiness in dark areas. Also, these tests are/were not limited to XviD only. (SBC and DivX502 Pro, more or less, but still absolutely exhibiting the same problem in dark areas, were also tested several times, just resulting in the same ongoing problem.)

Back to XviD, the aim of all these tries was to see if there would be any difference/improvement towards eliminating this problem by tweaking some settings, and mainly by disabling "lumi masking", my all-time 'usual suspect' concerning this issue ;).

Surprisingly, after all these tries with XviD and other codecs mentioned (well, actually these tries of mine have been going on for a very very long time ;)), I concluded that the problem hasn't got much to do with lumi masking/luminance correction in XviD/SBC respectively, nor with psychovisual enhancements in DivX502Pro. Whatever you do, you cannot eliminate them totally! It seems to be (as always discussed) more or less an innate problem to all codecs. And that's why there are options such as "noise" in ffdshow and "film-effect" in DivX5 decoder...

And now comes the interesting/striking point: Concerning XviD again, the only rips in all my tries that displayed this problem *least* (though certainly still having the problem to some extent) were the ones with MPEG/MPEG quantization and with Bicubic Resizing (=/> 0,0.5), which possibly "preserved" the original DVD noise more than the others. DVD noise is also clearly distinguishable especially in dark areas of the films (not in the form of ugly blocks of course but as "noise") when you watch the DVD carefully on monitor or on TV with the above mentioned settings.

Finally, these are only my "personal" observations for a long period of time, might contain mistakes, and absolutely open to/requires much discussion and sharing of experiences, which might even initiate a step towards the solution of a well-known but still unsolved problem.

Maybe, (in order to solve this major-irritating problem) we should totally change our attitude towards encoding and try to "keep" the *noise* of the original DVD as much as possible instead of trying to "eliminate" it...

Important Note: All the above argument is about "regular/normal movies" and "DVD rips of such movies"; not about TV content, anime, etc. Also, a solution "may" or "may not" be fully possible technically of course, but still this might be taken as just a different look/perspective to the encoding process and our encoding habits ;).

P.S.: You can try enabling "Noise" in ffdshow with "Old noise algorithm" and "Uniform noise" with "luminance noise strength=8" (chroma noise strength=0) to observe "the difference/the most possible original DVD-like effect" and see these blocks in dark areas almost completely disappear, though that's not a perfect solution to the problem of course ;).

kindest regards to all
and many thanks for taking the time to read such a long and boring post ;)

iago

EDIT: Well, I guess I have to add SimpleResize, the great non-smoothing, non-filtering ;) filter by trbarry, besides "Bicubic Resize =/> 0,0.5" too.

Marc FD

10th September 2002, 16:53

i noticed many times blocking in black scenes when i used my TV out.
for example in an anime (Noir) i wanted to see on my TV (16/9) :
It was SBC compressed and the blocking in dark areas was horrible.
Even ffdshow with new noise 80/40 PostPproc cpu-6 200% and the image was still horrible to see !!
Using Black-Expand on my TV help a bit.
In the end, i used Levels() and it was very good. but too slow :( for playback.

Now,i see a very simple way to get rid of this annoying effect :
Make Lumi-Masking in Post-Processing stage.
Could even be done fast using the DC value of each MB, no ?
I can try this with my MPEG2Dec PP mod, but i'm too lasy to move my computer near my TV :D

and it's only for DVD playback (i love to use BSplayer to see DVDs ;) )
not to play avi's (the PP should be built in the decoder)

I suggest you to try Levels() (of course you tweak the values)
If you see improvements, we could (any avs dev.) MMX optimise it.

-h

10th September 2002, 17:15

Marc FD

10th September 2002, 17:44

AC ? i don't remember the meaning.
can you enlighten me ?
thx.

iago

10th September 2002, 17:53

I've been thinking of adding a switch to XviD that adds AC "noise" to blocks whose luminance is below a certain threshold. I too have just started watching MPEG-4 on a TV and the result is truly horrible, I agree.

I would rather the encoder handle improving the areas than leaving it up to the post-processor, because eventually we may not have such fine-grained control over how the post-processor behaves (think hardware MPEG-4 players).@-h

Yes, absolutely that could be the one and best possible solution to the problem, and it would actually be great. I totally agree with you that it's much better to make it possible in the encoding process adding noise to dark areas below a certain threshold, and keeping the rest of the clean MPEG-4 picture alone. Hence, post-processing by adding noise (which might not be handled with full-control and success as you pointed out) can only be an emergency-exit to watch "poor" encodes on TV, which still contain the blocks problem; but by eliminating this "horrible" problem in the encoding stage, the "original" encode will be much improved, and that would really be a great step in the ripping world imho and open a new era for XviD to display its real perfect performance.

best regards,
iago

Marc FD

10th September 2002, 18:04

the problem is dumb guys (me) like _deep_ more-black-than-black black and don't like noisy stuff. (it's why i'm a h263 fan ;) )

-h

10th September 2002, 18:15

AC ? i don't remember the meaning.

The DCT output consists of 1 DC and 63 AC values. As the DC value is the "average" of the entire block, the only coefficients worth setting to simulate noise are the higher AC ones, which concern progressively higher frequencies.

Here (http://www.cs.sfu.ca/CourseCentral/365/li/material/notes/Chap4/Chap4.2/DCT_basis.gif) is an image with a graphical representation of the 64 coefficients and how they represent the image itself.

..it's much better to make it possible in the encoding process adding noise to dark areas below a certain threshold, and keeping the rest of the clean MPEG-4 picture alone.

The trick will be adding the right amount, without having it look just as bad only with different artifacts.

-h

iago

11th September 2002, 04:31

Hello again,

Well, it's getting more complicated actually, and I hate to do that but I think I have to make some corrections on my long post above, especially regarding lumi-masking, based on some new test results; though the main points discussed so far and the solution suggested by -h are still consistent and valid imho.

First of all, based on the new test results carried out with Nic's and Koepi's latest builds, using 1-pass constant quantizer 2, encoding the very beginning scenes of Matrix, with and without lumi masking, using both MPEG and h263 quantization types, and watching the test results very carefully (with no post-processing) both "on TV" and "on PC monitor with high brightness/contrast, high resolution, in completely dark environment at night with no lights on" ;), I can absolutely say that lumi-masking introduces more blockiness in dark areas and certainly causes more harm than benefits.

Briefly we can list the quality of the several quantizer 2 encodes in terms of blockiness in dark areas, from the best to the worst as follows: MPEG-NOLUMI -> H263-NOLUMI -> MPEG-LUMI ->/= H263-LUMI.

And actually the MPEG-NOLUMI encodes (both Koepi's and Nic's latest builds) were the ones closest to the appearance of actual DVD output in dark areas, bearing only the DVD-like noise rather than ugly blocks!

However, although these tests clearly show that lumi-masking is absolutely not our best-friend, there is another important issue too. The tests mentioned above are done using constant quantizer 2 in order to achieve the best possible quality with the given parameters and after that to compare the results with each other and see if lumi masking causes trouble or not. As a result, it really did!

But the point is that whenever you go higher than quantizer 2 (also tested constant quantizer 3 and 4), blocks begin to appear even without lumi-masking and whatever the quantization type (MPEG or h263) is. And as in 2-pass variable bitrate mode it's almost impossible to have an average quant 2 ;) for most movies to ensure quality and elimination of blocking in dark areas, the solution pointed out by -h is still of vital importance.

But at least, these last tests confirmed my never-ending suspicion ;) that lumi-masking was somehow and to some extent responsible for blockiness in dark areas, and now I certainly find it more secure not to use lumi-masking in my rips, and continue with MPEG/MPEG quantization as I've been already doing for a while.

And one more important point: using postprocessing with deblock/dering either with "presets" or with "custom" in ffdshow absolutely ruins the quality and introduces many more ugly blocks in dark scenes besides some artifacts. The best way for decoding XviD is using Nic's decoder filter (safely with or without postprocessing) imho. But if you "really want" to use ffdshow, use it either without postprocessing at all or only with "automatic quality control" without touching the rest in the postprocessing tab!

Well, that's all for now, and I'm really exhausted at the moment. Directly going to bed for some sleep! ;)

regards to all,
ciao,

iago

colasonic

11th September 2002, 05:40

Originally posted by iago
Hello all,
And I assume, most people are "generally" using their PC monitors to watch their encodes, just like me ;). However, when it comes to TV, the quality of any rip is totally another story, and even your most "trusted" and "perfect" looking rips may well frustrate you when you watch them on TV without lowering brightness and contrast, or sometimes even when you watch them on your monitor especially in a dark environment, with a rather high resolution, full brightness and contrast, and no extra lights in the surrounding! (So, those who don't bother watching their rips with such monitor settings and in totally dark environments

I think the reason is on the screen size. usually, a TV screen is much bigger than our PC screen. I don't have the experience to watch a DVDRip on a TV set, but I do know that some Rips which appear good-looking on my 17" monitor turn out to be crappy on my another PC with 19" trinitron. the reason may be the same.....

Koepi

11th September 2002, 07:23

As iago wrote on his first post on this thread, you should use ffdshow's noise option to get a propper image again.

Since iago saw these blocks already at quant. 3, this seems the only thing working for now. Maybe -h's attempt can help it, but I'm not sure if this doesn't blow up the video stream massively again (in size, I mean) :-/

In that case (using ffdshow's noise option) it might be possible to use lumimasking as well, but to be on the save side, for HQ TV playback you shouldn't use it.

The only option is:

You want DVD quality? Buy the DVD and keep it.
or: make a 5CD rip at quant 2.

:P

When puting movies on 1CD (from >5GB before) you have to sacrifice quality for sure. Just wanted to mention that again :)

Best regards,
Koepi

Marc FD

11th September 2002, 13:17

I think too that AC noise is not the solution.
Maybe iago can check it, but i think the whole problem in black areas is:
1) Black is very sensitive to quantization, because AC/DC values are near to zero and get easily quantized.
2) Many blocks are skipped int these areas (so AC noise is not a solution)
3) Lumi masking is increasing the quantizers, so it's even worse.
4) The DVD source is already bad in black areas. Rencoding it with XviD is amplificating this problem !

Maybe best than AC noise, i suggest to use a Black-Expand/Black-Normalisation Pre-Processing technique.
The Pre-Processing could be done in the DCT domain.
And using a stronger Pre-Processing with Lumi-Masking, you would even get rid of the bad black blocks...

@iago

i know it's not fair to ask it to a XviD fan, but can you compare with RV9 and it's "black pre processing" (i don't remeber the name) option. i think it worth a try to compare.
thx.

iago

11th September 2002, 13:26

Originally posted by Marc FD
@iago

i know it's not fair to ask it to a XviD fan, but can you compare with RV9 and it's "black pre processing" (i don't remeber the name) option. i think it worth a try to compare. thx.

@Marc FD,

I would absolutely do that if I had any experience with RV9 ;), but unfortunately I don't... Maybe someone else could try that and post the results, since RV9 (in Doom9's latest comparison) really seems to have solved this issue to some extent.

Yes, maybe all we need is "more black and black"! ;)

best regards,
iago

Marc FD

11th September 2002, 13:30

Originally posted by iago

Yes, maybe all we need is "more black and black"! ;)

Not exaclty. We need to correct the "bad black" on the sure to get "true black". DCT is allergic to "almost-black-but-not-exaclty-black black"

I think it's how RV9 makes the trick.
And i see how it could be done by XviD too :devil:

Since i'm going to implement XviD's Pre Processing, it could be a very usefull task to start with.

iago

11th September 2002, 14:56

Originally posted by iago
And one more important point: using postprocessing with deblock/dering either with "presets" or with "custom" in ffdshow absolutely ruins the quality and introduces many more ugly blocks in dark scenes besides some artifacts. The best way for decoding XviD is using Nic's decoder filter (safely with or without postprocessing) imho. But if you "really want" to use ffdshow, use it either without postprocessing at all or only with "automatic quality control" without touching the rest in the postprocessing tab!Well, it seems that even Nic's decoder filter causes this (more) blocking problem in dark areas when deblock and dering options are enabled! So to be on the safer side, either ffdshow or Nic's XviD decoder I go *without* postprocessing ;).

best regards,
iago

P.S.: I really wonder if I'm lucky or not to have a TV output and a Philips 107T21 17" monitor with LightFrame option on it to notice *all* these ugly blocks in dark areas!. Perhaps, I could be much happier *without* them! ;)

Marc FD

11th September 2002, 15:22

can you please try with Levels ?

Dali Lama

11th September 2002, 15:58

@all

This is a great topic. One which should be hammered out until resolved, I believe. I tried to start a similar thread in the AVIsynth forum about restoring the blackness of a film, but maybe I wasn't too clear.

@ Iago

Recently I found a filter called ColorYUY2. It is made by a Japanese person. It has a really useful feature that restores the levels of a film from TV-->PC or PC-->TV.

Simply input:

ColorYUY2(Levels="TV->PC")

I would be delighted to see what you find when you try this filter before resize and no other filtering concerning the blockiness in black areas. For me it seems to significantly reduce them.

@ MarcFD

I believe that you were mentioning this when you spoke of levels, correct? And yes, I have used RV9 a fair amount and can contest that it handles black much better than other codecs. However, their black restore filter simply darkens the movie too much. Also, try out that ColorYUY2 filter, it seems to darken the movie, but retain original contrast and saturation.

One other note, I am using the Lanczos3 Resize filter after ColorYUY2. This may also help in achieving the better encoding in dark areas.

Lets solve this pesky problem!!

Dali

-h

11th September 2002, 16:25

2) Many blocks are skipped int these areas (so AC noise is not a solution)

AC noise would prevent the block from being skipped.

Maybe best than AC noise, i suggest to use a Black-Expand/Black-Normalisation Pre-Processing technique.

How would it work though? If you blacked out a large area with thresholding to pure black, you're going to get the same ugly blocks at boundary areas (and an odd smooth block on TV where you expect to see some kind of texture).

MPEG-4 and pure black are nasty things to get working. Whatever technique gets used in the end, it'll just require a heap of testing I suppose.

-h

Marc FD

11th September 2002, 16:50

agreed.
we need testing.
It's why i propose :
- To pre-filter / post-filter with Levels just to see if it's good.
- To see how RV9 handles this problem with it's special tool.

PS : The Black Expanding technique i propose is different and more complex than Levels. I think you can smooth it spatially in the DCT domain to avoid blockiness. Need to be tested....

iago

11th September 2002, 18:19

Originally posted by Dali Lama
@ Iago

Recently I found a filter called ColorYUY2. It is made by a Japanese person. It has a really useful feature that restores the levels of a film from TV-->PC or PC-->TV.@Dali Lama,

Thanks a lot for the info you provided about this filter. I tested it several times and it really did a great job, just suiting my needs! As you also said, it seems to darken the movie a bit, but at least it doesn't ruin the encode ;). I applied it just before SimpleResize, and made numerous tests "with" and "without" using it to compare the difference, with constant quantizers up to 10, and even with the "guilty" ;) lumi masking, as well as using h.263 quantization type too. As a result, in all my tests, it performed a fantastic job, eliminating all blocks in dark areas, and converting them to pure clean black colour.

For the time being, until an innate solution within the codec itself is achieved, I guess I'll certainly use that filter without hesitation in my encodes.

And for now, until the revolutionary step of implementing an innate solution is taken, the other possibility stands as adding noise in ffdshow (uniform ~8 with old noise algo), which is absolutely an inferior solution compared to the above filter's job imho.

kindest regards and many thanks again,
iago

-h

11th September 2002, 19:12

For the time being, until an innate solution within the codec itself is achieved, I guess I'll certainly use that filter without hesitation in my encodes.

That sounds like a nice solution. I wouldn't like XviD's default behaviour to be altering the input (i.e. preprocessing) though, and as well as this method works I would still like to investigate one that doesn't resort to changing levels.

Of course the levels system could be integrated as a "preserve black" or "optimize for TV" option or however it should be categorized.

-h

Marc FD

11th September 2002, 19:30

i wouldn't be surprised at all if this filter is doing exactly what i described...
but it doesn't matter now.

can you compare the filesizes ?
can you try this filter for the playback of a bad compressed (with ugly blocks in dark scenes) source ?

Dali Lama

11th September 2002, 20:21

Originally posted by Marc FD
can you compare the filesizes ?

Yes! :D

I am really ecstatic about the results of this test comparing filesize/perceived video quality in dark scenes.

Clip used...

Opening of Rush Hour (2min 40s):

Dark and clean video, should be a good test for viewing any noise produced by the codec in dark scenes.

This is the structure of the AVS used:

# Plugins
LoadPlugin("C:\DOCUME~1\Desktop\DVDENC~1\Avisynth\mpeg2dec.dll")
LoadPlugin("C:\DOCUME~1\Desktop\DVDENC~1\Avisynth\decomb.dll")
LoadPlugin("C:\DOCUME~1\Desktop\DVDENC~1\Avisynth\ColorYUY2.dll")
LoadPlugin("C:\DOCUME~1\Desktop\DVDENC~1\Avisynth\lanczos3.vdf")
#
# SOURCE
mpeg2source("C:\Rush Hour\RushHour.d2v")
#
# IVTC
Telecide(guide=1,post=false)
Decimate(cycle=5)
#
# Filter
ColorYUY2(Levels="TV->PC")
#
# CROPPING
crop(5,57,712,367)
#
# RESIZING
Lanczos3Resize(640,272)

------------------------------------------

Test:

Used H.263 @ Quant 2

With ColorYUY2 filter - 47.3MB

Without ColorYUY2 filter - 40.2MB

Analysis:

I believe that these are excellent results. Why? Because this is exactly what I wanted this filter to do. By adjusting the levels suited to the PC monitor, the filter demands higher bitrate in dark scenes. I have always believed that the pragmatic solution
to blocking in dark scense is to give it more bits. At last, this filter does this.

Of course the differences in visual quality is very small at such a low quantization.
-------------------------------------------

Test 2:

Used H.263 @ Quant 8

With ColorYUY2 filter - 8.94MB

Without ColorYUY2 filter - 7.72MB

Analysis:

Once agains the filter is forcing the codec to give more bits to the dark scenes. The result? Even at quant 8, the ColorYUY2 filter is able to provide a relatively excellent picture in dark scenes. Without the filter, the blocks are prevailent in lots of dark shades. Furthermore, the picture of brighter scenes are not affected either!! This is the best part, it doesn't seem to damage the brighter scenes in favor of the dark ones.
-------------------------------------------

Note: Since the respected Iago and I find this filter useful and the fact that it is hard to find, I will attach the filter here for others to experiment or begin using in their encodings.

Satisfied and Exuberant,

Dali

iago

11th September 2002, 21:39

@Dali Lama and all,

I'm really glad that (after digging into the issues of lumi-masking and blockiness in dark areas again and again) we've finally reached a (temporary or maybe permanent ;)) solution regarding this matter, which imho is/was the most annoying problem for all kinds of rips. Though I haven't performed any 2-Pass/full-encode tests yet, the filter really gave the exact desired results in all my constant quantizer tests with the opening scenes of Matrix (up to quant 10 - with both MPEG and h.263 quantization types, and lumi-masking enabled or not).

Currently, I've started a two-pass encode of Matrix using MPEG quantization and no lumi-masking. When finished I will report back my opininons and comments on the full rip results.

As for compressibility, as far as I can comment based on my short tests, and parallel to Dali Lama's results, the filter seems to decrease compressibility by ~10-15; however, compared to its great job, this is no longer much of a problem I guess. Also, since the filter seems to work without causing any problems with h.263 and lumi masking too, as long as no trouble is encountered with these options, this decrease in compressibility may well be compensated by using these options as well.

Here is a sample avisynth script:

LoadPlugin("C:\filters\mpeg2dec.dll")
LoadPlugin("C:\filters\colorYUY2.dll")
LoadPlugin("C:\filters\SimpleResize.dll")
mpeg2source("D:\MOVIE\MOVIE.d2v")
crop(2,80,718,418)
ColorYUY2(Levels="TV->PC")
SimpleResize(640,272)

BTW, in my tests with SimpleResize, the filter's speed was pretty good too, almost close to the no-filter speed.

@Dali Lama,
You can also apply the filter after cropping and before resizing to gain some extra speed, no?

ciao,
iago

colasonic

11th September 2002, 22:21

Hi, where can I find this "colorYUY2.dll"?

thanks.

Koepi

12th September 2002, 08:30

I don't want to sound stupid, but which colour scale did you choose when creating the d2v project file in dvd2avi?

Did you choose "TV sclae" and "colorspace: YUV"?

Do you notice a difference when using different scales? (e.g., when selecting "PC scale" instead of "TV scale" compressability is ~10-15% worse.... reminds you of something? We back then choose the "TV scale" since it wasn't giving us bad artefacts... or was it vice versa?)

*whistle*

Regards,
Koepi

Sharro

12th September 2002, 10:47

I sound stupid...

I have been using pc scale and yuv colorspace . ..this is the correcT right?

Glad to see u back Koepi (still #doom9 Efnet is missing u).

All the best.

Take Care

Sharro

iago

12th September 2002, 12:31

@Koepi

In dvd2avi I used the same old procedure to create the d2v file: "YUV Colorspace" and "PC Scale". Then I used the above script to encode Matrix (MPEG/MPEG-No Lumi). The result is really good in terms of blocks in dark (or maybe better to say blocks in black or almost-black) areas. I mean, they are all gone, as mentioned before. So the main problem (for me) is solved, though the average quantizer jumped up a bit.

But, after reading your post, there comes the question: what happens (in terms of both compressibility and blockiness) when we choose "YUV Colorspace" and "TV Scale" to create the d2v file, which I have never tried so far actually (maybe the basic point I was/we were missing?)?

Is that what you mean Koepi? (Sorry if I haven't got it clearly?)

I'll do some tests with two different d2v files, both in YUV colorspace, but one "PC Scaled" and the other "TV Scaled", using constant quantizer mode (for example quant4). And I'll see how this affects compressibility and blockiness.

Anything you want to suggest for further convincement?

thanks,
iago

Bluedan

12th September 2002, 12:33

Using the above avisynth script with colorYUV2 filter, how does it decrease encoding speed ?
Still suitable for average user on PIII@667MHz?
And if you use Lanczos resize filter in addition? Speed issue?
Thx.

iago

12th September 2002, 12:37

@Bluedan

I guess I'd told that before in one of my above posts ;). It makes almost no difference in encoding speed (but I didn't use lanczos3, I used SimpleResize for resizing).

regards,
iago

Bluedan

12th September 2002, 12:43

@iago
sorry pal, maybe to excited to read carefully before. thank you though

so I address this exclusively to dali lama who obviously has worked with lanczos filter excessively ;) :

how does it affect speed ?

droolian01

12th September 2002, 13:41

Hi there.

I do all my computing though my tv (second hand sony trinitron 29 inch) using wireless keyboard and mouse as i like to lie around lazyly on my sofa! So everyone of my avs scripts ends with

levels(5,1,255,0,255)

I find this little tweak makes very little difference to the overall brightness of the xvid but does help to reduce the effect of the blocks in dark areas. I have always avoided lumi masking as it seemed that MORE bits were needed in dark parts of a film to correct this.

This is such a vital topic for xvid, and i will eagerly monitor this thread.

Good stuff!

iago

12th September 2002, 14:21

Originally posted by droolian01
This is such a vital topic for xvid, and i will eagerly monitor this thread.I totally agree with you. This problem has always been the most annoying and persistent one for me since my very first encodes. And I'm really glad that we've come to a point of overcoming it finally ;).

Now, back to Koepi's questions on dvd2avi PC or TV scaling matter: After another session of testing, initiated by the doubt that Koepi has managed to let loose onto me (LOL! ;)), here are the results:

Matrix (frames 0-3746)
Koepi's 04092002-1 binary - constant quantizer 4
dvd2avi 1.76 - VirtualDub 1.4.10 fast recompress

Compressibility:

Matrix_Mpeg_Lumi_PCScale(original): 19822 kb
Matrix_Mpeg_Lumi_TVScale(original): 19822 kb
Matrix_Mpeg_Lumi_PCScale_colorYUY2: 22922 kb
Matrix_Mpeg_Lumi_TVScale_colorYUY2: 22922 kb

Visuals:

No difference between the PC scaled and TV scaled original encodes: Both has the same annoying problem of ugly blocks all around in dark/black areas.

No difference between the PC scaled and TV scaled colorYUY2 encodes: Both has pure clean black color in such areas and has excellently eliminated the problem ;).

And some other results that could be useful (if read properly ;)):

Matrix_Mpeg_NoLumi_PCScale_colorYUY2: 23692 kb
Matrix_Mpeg_Lumi_PCScale_colorYUY2: 22922 kb
Matrix_h.263_Lumi_PCScale_colorYUY2: 22846 kb

Final comments:

1) Using PC Scale or TV Scale (YUV Colorspace) in dvd2avi has no influence on file size and quality (with or without colorYUY2). The results are absolutely the same.

2) colorYUY2 filter works with lumi masking and h.263 quantization without any problems, performing the same excellent job. So those who focus on "more" compressibility (which is quite understandable ;)) in addition to getting rid of ugly blocks in dark/black areas can use colorYUY2 with h.263 and lumi masking safely as well.

kindest regards,
iago

Koepi

12th September 2002, 14:31

Erm, the secret is not to use const quant... :) You won't notice a difference in "compressability" there. It's a matter where the YUV-ranges are cut. For TVscale, the scale is only from ~30-200, while pcscale should give all values from 0-255.

Strange that there is no visual difference... Maybe something to do with mpeg2dec.dll not respecting this value? I think those tests were done with dvd2avi 1.74 (or 1.76) and a "non tweaked" mpeg2dec.dll, sorry that I can't remember correctly (or was it even back in vfapi-times?).

Regards,
Koepi

Swede

12th September 2002, 14:58

Suiryc

12th September 2002, 15:00

If I remember well DVD2AVI 1.76 do not tell in the .d2v file if you choosed PC or TV Scale. That explains why there is no difference.

droolian01

12th September 2002, 15:35

Hi all.

I just tried colorYUY2 and from the preview window in vdub it just seemed to fiddle with the levels, probably something like

levels(20,1,250,0,255)

which made the whole video too dark for me. I've done lots of messing with levels, tweak and even deliberately oversaturating/increasing contrast on capture settings and my conclusion is you will always get blocky dark areas with low contrast/bright settings - thats why i compromise with

levels(5,1,255,0,255)

If your aim is maximum compressability then the bits have to be re-distributed from somewhere and lumi masking therefore has its place, but it seems to me that the default behaviour of xvid (and i assume all mpeg4 codecs) involves similar sorts of redistribution.

Now i wonder if it would be possible to have an option in xvid that is the opposite of lumi masking - gives more bits to dark scenes/doesn't redistribute as many? Or just take from bright scenes only (mild lumi masking?) - although this could be a problem as i noticed that the 'white outs' in six feet under can look ugly sometimes!

This option obviously would be for the 2-3 cd encoders and would probably bump up the quants on 1 cd encodes making things uglier.

If this is nonsense i appologise in advance - and tell me where i have gone wrong with my ideas please!

Marc FD

12th September 2002, 16:16

Maybe i'm totally wrong, but i think you could :
- Have a clean black
- Reduce Filesize
- Keep overall quality in bright scenes
all that in the meantime.

i don't see why you couldn't....

i'll create a little C avisynth filter for test purpose, just to see if my theory is working. when i would have time :(

EDIT : MPEG2Dec is using TV/PC scale. i saw it when i played with.

Swede

12th September 2002, 16:23

Originally posted by Suiryc
If I remember well DVD2AVI 1.76 do not tell in the .d2v file if you choosed PC or TV Scale. That explains why there is no difference. I tested *before* I posted... The difference *is* there, in 1.76.

trbarry

12th September 2002, 16:38

Since the only difference in the *.d2v-file is YUVRGB_Scale=0 or YUVRGB_Scale=1 (in the beginning of the file) perhaps someone that's familiar with mpeg2dec.dll could verify that the .dll don't respect this value?

IIRC that parm only affects how to do the conversion to RGB. Since MPEG2DEC.dll always converts to YUY2 instead of RGB that parm is ignored.

- Tom

Suiryc

12th September 2002, 16:40

Oops sorry in fact I misunderstood with the ColorSpace (YUV or RGB) choice.

Sorry again :(

Koepi

12th September 2002, 17:42

Ah, ok, so it was relevant back in vfapi days - too bad. Well, then you have to correct the values like in the example above :)

Regards,
Koepi

Dali Lama

12th September 2002, 19:37

Originally posted by trbarry

IIRC that parm only affects how to do the conversion to RGB. Since MPEG2DEC.dll always converts to YUY2 instead of RGB that parm is ignored.

That is exactly correct Tom. If I saw the posts earlier I would have mentioned this. Furthermore, look at it this way, most DVDs are mastered for viewing on TVs therefore, if you keep the YUV options checked in DVD2AVI then the effective scale is TV. Therefore, it is necessary to perform ColorYUY2 in the AVS if the encoding will be viewed on computer monitors.

"Ah, ok, so it was relevant back in vfapi days - too bad. Well, then you have to correct the values like in the example above."
-Koepi

Yes, Koepi you got it ;)

Bluedan, there is info about the Lanczos3Resize filter in the AVIsynth Forum. But I can assure you that there is a significant speed drop from using it, but also a significant quality increase. To me, it doesn't blur or sharpen too much. Its a well known high quality resizing algorithm for photographs. Try it out, I think you'll like it.

Take Care,

Dali

iago

12th September 2002, 23:40

Originally posted by droolian01
I just tried colorYUY2 and from the preview window in vdub it just seemed to fiddle with the levels, probably something like levels(20,1,250,0,255) which made the whole video too dark for me. I did some 2-pass tests with Matrix Chapter 3 with the same exact Xvid parameters (mpeg/mpeg, no lumi, quantizers 2-5/2-8, internal cc with default altCC values, and setting the filesize value in such a way that (13650 kb) it would correspond to an average bitrate of 625 kBit/s). As a result coloryuy2 doesn't seem to correspond to levels(20,1,250,0,255) as can be seen in the attachment. Also, you're right, coloryuy2 really darkens the movie and only thus it can totally eliminate the blocks. But a setting of levels(20,1,250,0,255) is not relevant and it not only makes the movie much darker than coloryuy2, but also still cannot eliminate the blocks as successfully.

Originally posted by droolian01
...i compromise with levels(5,1,255,0,255)In my tests, this setting simply cannot solve the problem, with still clearly visible blocks all around, though they may be a bit less than before.

Originally posted by droolian01This option obviously would be for the 2-3 cd encoders and would probably bump up the quants on 1 cd encodes making things uglier.Well, I guess what you mean by "this option" is "ColorYUY2". I guess this is possible, but imo it depends on the source. Based on my limited tests, especially for dark movies it may really decrease compressibility considerably and therefore cause a big increase in quantizers. (However, maybe luckily maybe as a result of capping quantizers, I didn't have much "visible" or "annoying" problems in terms of quality due to higher quantizers in my full Matrix encode.)

Originally posted by droolian01
Now i wonder if it would be possible to have an option in xvid that is the opposite of lumi masking - gives more bits to dark scenes/doesn't redistribute as many? Or just take from bright scenes only (mild lumi masking?)That would really be interesting and I would happily try it too if there were such an option, though I don't know of course if such a thing is possible or not ;).

And attached you can find the (jpeg) screenshots (with no post-processing, taken in VirtualDub) from original DVD, and levels(5,1,255,0,255) encode, levels (20,1,250,0,255) encode, and ColorYUY2 encode.

regards,
iago

Defiler

13th September 2002, 04:50

Is it feasible to add an ffdshow feature similar to ColorYUY(), or have I fallen behind the flow of the thread?

droolian01

13th September 2002, 09:46

Hi all.

@iago - Thanks for the tests - coloryuy2 definitely gets rid of the blocks in the dark areas, much more than using levels. (btw my suggestion of using levels(20,1,250,0,255) was just a guess at how i thought i could recreate the contrast narrowing that seemed at the time that coloryuy2 was doing - only a guess and clearly wrong!)

I'm so impressed with your results that i am going to use it on my next encode - altough i can't be too experimental as i have a backlog of half a dozen captures to process (hd now FULL!!!) and as buffy is on 5 times per week (no repeats!) and 6 feet under once per week i have no time to re-encode and rely on people like you to show what can be done with the new toys! (thanks)

My comment about 2-3 cd encoders refered to my idea of having 'anti-luming masking' option, it did not refer to coloryuy2 which i beleve like you will increase compressibility. I mentioned the 2-3 cd encoders as i thought my idea (if possible) would be ignored as it goes against the whole idea of video COMPRESSION but i just wanted to pre-empt that argument to hopefully get some feedback from developers (ok guys assuming it is desireable - is it possible to allocate more bits to dark scenes like that?)

Thanks again

Sharro

13th September 2002, 10:07

Hi Everybody,

I'm getting a xvid forum addict.

Are we just talking about getting rid of blocks? I cant really figure out what these screenshots mean in terms of playback.

Is it my impression or levels(5,1,255,0,255) are the ones who bring us the brightness closer to the dvd one? I understand that will decrease compressibility but ... wouldn't it be something to take in account for 2 cd encodes (my case most of the time).

I watch my movies only on TV... so brightness/contrast/sharpness are a main issue to me.

Sorry, I just cant stay quiet...

Take care.

Keep on Thinking, Keep on Testing, Keep on Posting.

Thks.

Sharro

Koepi

13th September 2002, 10:17

Hm... if xomeone would now put in all these usefull things into one dll it would be cool:

think of it this way...

LoadPlugin("MPEG4tool.dll")
MPEG4tool(source="myfile.lst",tvout=yes,crop="0,76,0,76",resize="lanczos,640,272")

(and all things that are vital which I forgot, maybe a convolution3d as well ;) )

the source is assumed to be for mpeg2decoder.dll from Nic, as well as the cropping. The tvout-option is just a switch if colorYUY2("TV->PC") should be invoked or not, and finally the resize should be clear ;) .

Does this sound stupid? I'd love to see such a "swiss army knife" (the important part is to have all in one plugin, with an easy-to-use commandline, so you don't need 5 dll's and need to remember ever command you have to use with it's parameters) for encoding issues.

Best regards,
Koepi

droolian01

13th September 2002, 10:19

Hi Sharro

If you turn your brightness to max on your tv and zoom into the guys back on the right you will see that the coloryuy2 one is block free (they are blue-ish on my screen)

All mpeg4 codecs afaik produce these ugly blocks in dark areas/scenes, its so bad that that it can look almost like a pixelated 'ghost' hovering there. It is more noticeable on tv than computer monitors and the usual way to 'get rid' of these artifacts is to turn down the brightness on the tv - not really satisfactory.

I used levels(5,1,255,0,255) as a way of just making the 'black part blacker' to reduce this a little - but it never got rid of it. Higher values of levels messed up the contrast/made the film too dark. Coloryuy2 looks to be the best so far at sorting this problem, but i wonder if the codec couldn't be tweaked for this - that all

Hope this helps

iago

13th September 2002, 14:19

Originally posted by Koepi
...I'd love to see such a "swiss army knife"...That would absolutely be great Koepi! ;)

* Nic's MPEGDecoder with correct framecount without depending on "trim", and yes together with NicCrop absolutely.
* Convolution3d with the most suitable settings discovered for regular movies to increase compressibility.
* Lanczos3 resize (compressibility is very close to sharp bicubic but quality is probably better) and absolutely SimpleResize as another option.
* ColorYUY2 or correspondent "levels" settings, dealing with black color as perfectly, but without darkening the movie that much.
* The add-noise option "during the encoding process" as suggested by -h, but only applying for dark and black areas, as an alternative.
* Inverse-lumi masking, stealing bits only from very bright scenes and distributing those bits to dark scenes.

and etc., etc... ;)

Who knows, maybe one day ;)...

kindest regards,
iago

"...There is a crack in everything... That's how the light gets in..." from a song by L.Cohen

EDIT: BTW, droolian01, I guess the closest levels setting to ColorYUY2 is something like levels(16,1,235,0,255) imo, but still not as successful as ColorYUY2. Maybe ColorYUY2 is also messing with the "gamma" setting a bit, I don't know...

iago

13th September 2002, 14:31

Please see the images attached: One with ColorYUY2(Levels="TV->PC"), the other with levels(16,1,235,0,255).

iago

EDIT: Where is my attachment?!! ;) It's just a very small zip with only two images included! ;)

Dali Lama

13th September 2002, 15:49

Originally posted by iago

* Inverse-lumi masking, stealing bits only from very bright scenes and distributing those bits to dark scenes.

Iago, I think the same way. I believe the opposite of lumi-masking will work better. Think about it, can you stare into the sun??? No, because its so bright. There are many times in movies (like explosions or bright flashes or sun shots) where the human eye cannot bear to stare or notice the blocks in it. On the other hand, dark scene blocks as this thread has pointed out IS a problem.

So, as of now, no lumi-masking for my encodes (never used it either). But hopefully as Iago put it, "Inverse lumi-masking"

Maybe a new thread should be started :sly: ,

Dali

Marc FD

13th September 2002, 16:11

About "inverse-lumi-masking".
If you want to achieve big sizes, encode with constant quant 1.
And if it's not big enough, use zero-me.
Adaptive quantization (like lumi-masking) is _only_ able to increase quantizers. something else is meaningfull.

BTW, using Layers, Levels, Tweak and Blur combo, maybe you can achieve to get a better black. could worth a try.

@Koepi

a rippack does exactly that, no ?

Maurizio

13th September 2002, 16:21

Adaptive quantization (like lumi-masking) is _only_ able to increase quantizers. something else is meaningfull.

Well, wasn't said that lumi-masking goal was to remove bits
where unneeded (not likely to be seen) and to redistribuite them
in more sensitive (to eye) places ?

Maurizio

Marc FD

13th September 2002, 16:28

no : it only removes data.

the reallocation is done by the curve scaler :
because the frames are less big at same quant, you can use lower quants.

BTW, doom9 totally screwed his XviD-encode for his comparison by using Lumi-Masking. it's one of the most experimental features of XviD. Because i know how it works, i would never use it for a real rip.

If people don't care, they do as they wish.
Never, _never_ forget than XviD is in alpha-stage.
(it's the best MPEG-4 codec, not the more safe)

Koepi

13th September 2002, 17:06

"rippack"?

You mean, some stupidly collected filters and programs, like a giant gknot-centered all-in-wonder installer?

Nope. I mean just a simple, single avisynth plugin which includes all those. I want to do that stuff by hand. And don't want any programs to give me suggestions.

(I coded expert systems some years ago and know that I don't trust them if I don't know the sources/databases).

Regards,
Koepi

droolian01

13th September 2002, 17:41

Hi all.

@ Koepi - this super plugin sounds very good, would make life easier for routine encodes

@ Marc FD - its a shame my idea of inverse lumi masking is a non starter, i wasn't aware that the procedure was just a data remover and not a data reallocator - oh well! Your point about encoding with quant 1 and incresing sizes misses the point, an excelent encode (few if any blocks at a 'good' resolution and filesize) can be spoilt by these 'darkblocks'. Sure we can prepare the source in such a way to reduce these (coloryuy2, levels, tweak etc) but if there was any way that the codec itself could allocate bits to these areas it would be worth doing. File sizes may not increase massively, and users would have the choice of how many cd they would shoot for anyway.

@ iago - i am experimenting with putting a 'tweak(bright=15) before the coloryuy2 line in order to compensate for the darkening effect.

Telecide(blend=false)
crop(4,4,360,568)
Convolution3d (0, 6, 6, 8, 8, 3, 0)
UnFilter(40,40)
tweak(bright=15)
Lanczos3Resize(512,384)
msharpen(threshold=10,strength=150)
ColorYUY2(Levels="tv->pc")

I think this would not cause many more (if any) blocks to appear as the coloryut2 comand seems to be doing more than just a simple 'levels' type command. Have a go with this and see what you think - remember i only encode tv captures so my numbers might need adjusting for dvd rips.

Thanks again

Swede

13th September 2002, 17:53

Originally posted by iago
EDIT: Where is my attachment?!! ;) It's just a very small zip with only two images included! ;)Well, you do know that each attachment has to be validated.?. And we (the mods) do have a real life, at least some of us :) (I had to stop by my local pub on my way home)

iago

13th September 2002, 17:57

Originally posted by Marc FD
About "inverse-lumi-masking".
If you want to achieve big sizes, encode with constant quant 1.
And if it's not big enough, use zero-me.I simply cannot understand why "such" a reply arrives, instead of just explaining "why" it can be possible or not; or if it "is" possible, then what the consequences will be.

It was just an idea after all, it may be possible or not. For example, ColorYUY2 also decreases compressibility considerably and ups the average quantizer, but still it can stand as a solution to the irritating problem of blocks in dark/black areas for many users.

BTW, I'm not a "coder" but just a "user" and my actual field of study is language and literature ;). And I really "respect" all the great effort and works of the coders here in "this world" (including you) regarding filters and codec development; but I guess that shouldn't lead to an undervaluing of "users", who also deserve some respect imho or at least who expect replies without disregarding some basic rules of "communication".

Thanks,
iago

EDIT: Just like Swede did, in reply to my "attachment" question, which was a much lighter matter actually ;). And I was kind of joking actually Swede, really sorry if that upset you. Thanks anyway ;)

Marc FD

13th September 2002, 18:01

Originally posted by droolian01

File sizes may not increase massively, and users would have the choice of how many cd they would shoot for anyway.

:confused:

That's the problem : filesize _will_ increase massively.
more needed bitrate (2x,3x!!) in dark scenes if you want very high quality.

I think you've forget that MPEG-4 is a _very_ lossy codec.
the actual ratio of an encode is around 1/500-1/5000 of original data.

The problem of "blocks in dark areas" is _wanted_ !!
Because "blocks in dark areas" is _much_ better than "blocks on the whole movie". It's how MJPEG/MPEG-1/MPEG-2/MPEG-4/ect... codecs work.

EDIT :
@iago
Sorry, but it's just i try to explain why you can't do some things.
I try to be the more effective possible. If you want to know all the details of why it's not possible, you'll need to learn a lot of stuff on video codecs. my own knowledge is small.
Please, don't say i'm bad because i'm having bad news.

PS : If you want, i can stop to post in this thread and let you dream. i'm not wanting to be rude, nor to be bad. i'm here to help and explain if i can.

iago

13th September 2002, 18:23

@Marc FD

If you want, i can stop to post in this thread and let you dream. i'm not wanting to be rude, nor to be bad. i'm here to help and explain if i can. Sorry Marc, but I really don't want to prolong this argument unnecessarily. Thanks for your explanations, anyway. But remember, without dreaming first, you cannot achieve anything. And sometimes the line between the real and the unreal (of "now") is very very thin.

kindest regards,
iago

Swede

13th September 2002, 19:28

Originally posted by iago
EDIT: Just like Swede did, in reply to my "attachment" question, which was a much lighter matter actually ;). And I was kind of joking actually Swede, really sorry if that upset you. Thanks anyway ;) I'm not upset, not at all. I just tried to keep your joke running. ;)

droolian01

13th September 2002, 19:30

Oh well....that's told me!:)

As Scotty would say to Captain Kirk "ye cana change the laws of physics"
This thread has been very useful, For one i have learned a little more (very good - thanks Marc FD), i think it has been established by iago that lumi masking will increase dark-blocks. Iago has also tested a possible solution with coloryuy2 and a possible codec related tweak was suggested by -h - introducing ac noise when blocks fall below a certain lumi value (or something like that:confused: !!)

No need for harsh word - but maybe it is difficult to avoid when people are so passionate about their interests, its this which motivates users and developers to do great stuff

Keep it up !!!!

iago

14th September 2002, 00:34

OK, some more test results: ;)
[regarding compressibility decrease and high quantizers issue with ColorYUY2(Levels="TV->PC")]

Matrix Chapter 3 (very dark content) / XviD 2-pass Koepi's 04092002-1 build / internal curve compression / 6.UltraHigh / MPEG-MPEG / No Lumi / Max-Min I-frame intervals: 300-5 / Quantizers capped: 2-4 2-8 / payback delay: 300 / payback with bias / AltCC with default parameters / target size: 13650kb ~ 625 kBit/s

script1
-------
LoadPlugin("C:\PROGRA~1\GORDIA~1\mpeg2dec.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\colorYUY2.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\SimpleResize.dll")
mpeg2source("D:\MATRIX-DE\CHAP3.d2v")
crop(2,80,718,418)
ColorYUY2(Levels="TV->PC")
SimpleResize(640,272)

1-Pass Size : 27860667 Bytes or 27207 KBytes or 26 MBytes
Average 1-Pass Size : 6223 bytes

Quantizer distribution for 2nd pass:
Q:2:783
Q:3:3369
Q:4:290
Q:5:29
Q:6:6

script2
-------
LoadPlugin("C:\PROGRA~1\GORDIA~1\mpeg2dec.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\convolution3d.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\colorYUY2.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\SimpleResize.dll")
mpeg2source("D:\MATRIX-DE\CHAP3.d2v")
crop(2,80,718,418)
Convolution3d(0,4,4,4,4,3,0)
ColorYUY2(Levels="TV->PC")
SimpleResize(640,272)

1-Pass Size : 20387750 Bytes or 19909 KBytes or 19 MBytes
Average 1-Pass Size : 4553 bytes

Quantizer distribution for 2nd pass:
Q:2:1547
Q:3:2897
Q:4:33

No comment! ;) (I hope vlad59 reads this thread too ;))
Screenshots from the two encodes are attached below.

regards,
iago

iago

14th September 2002, 01:11

Originally posted by droolian01
@ iago - i am experimenting with putting a 'tweak(bright=15) before the coloryuy2 line in order to compensate for the darkening effect.

Telecide(blend=false)
crop(4,4,360,568)
Convolution3d (0, 6, 6, 8, 8, 3, 0)
UnFilter(40,40)
tweak(bright=15)
Lanczos3Resize(512,384)
msharpen(threshold=10,strength=150)
ColorYUY2(Levels="tv->pc")

I think this would not cause many more (if any) blocks to appear as the coloryut2 comand seems to be doing more than just a simple 'levels' type command. Have a go with this and see what you think - remember i only encode tv captures so my numbers might need adjusting for dvd rips. @droolian01

I did some tests with the tweak values, with Matrix Chapter 3, trying to restore the original brightness of the movie as much as possible, and to see if ColorYUY2(Levels="tv->pc") still eliminates the blocks as properly as before when using a tweak value for brightness. I used the encoding parameters provided in my above post for the tests. As a result, tweak(bright=10) seemed to be the most matching value for brightness for this specific chapter from Matrix. However, this combination [tweak(bright=10) and ColorYUY2(Levels="tv->pc")] also exhibited some blocks in certain parts of the encode, and was not as successful as ColorYUY2(Levels="tv->pc") alone.

I tried with a setting as low as tweak(bright=6) but blocks were still visible, though the movie had already begun to get darker again ;).

So, "tweak" may be a solution to some extent of course, depending on the source, and it certainly helps to a point of compromise, but in my tests, it seems that ColorYUY2 hasn't liked it either as a companion! ;).

best regards,
iago

droolian01

14th September 2002, 10:01

Hi again.

I wasn't sure if using tweak(bright=whatever) would cause the blocking in dark areas to re-appear - seems it does unfortunately. This whole issue of source brightness/contrast/saturation etc is much more complicaated for me as i only process tv captures - i can adjust these settings individually for each capture whereas with a dvd rip you are presented with a 'fixed' source.

Getting authentic looking captures is something i've struggled with for some time - i feel closer now to this aim - and you have pointed me in the direction of coloryuy2: i thank you for this (and all your testing - you've probably seen certain parts of the matrix more than the editor and director did!!!)

Thanks again

drizztcanrender

14th September 2002, 15:12

Guys probably this will be off topic because it will involve maybe two more forums :)
I've been thinking about the effect ColourYUY2 has in a movie.It does help eliminate the blockiness in dark scenes of the film but also it makes the film even darker.So the ideal solution would be to use that in only some parts of the movie.Unfortunately the only filter i have seen doing that is the conditional filter of Mr Dmitri Schamschurko.
And i suppose ColourYUY2 works in yuv colour space,right?
So is there a way we can tweak this internally with avisynth or maybe a way to make it work with the conditional filter??
Anyways a kind of conditional filter for avisynth would be great :)
Well i guess you could do it with trim function,anyone tried that??Will be a real hassle though...

Excuse my mistakes,if i've made some,i'm still learning :) and i don't think i'll ever stop ;)

Marc FD

14th September 2002, 15:21

why not using ColorYUY2 in back parts of the image (Levels),
then you merge it with the brighter part (Levels,Layers) ?

Dali Lama

14th September 2002, 19:24

Not to hinder any possible positive results from this testing, but my feelings are that ColorYUY2 IS supposed to darken the movie. Maybe if we look at it this way. DVDs are designed to be viewed on televisions. Since, we need to watch it on PCs, it should be darkened. Similar to how images on PCs look darker than on MACs. Take a look at the DVD's darkenss on a new high end tv, with 3D comb filtering and antireflective screen. You will notice dark and rich colors. On the PC we have not been used to this. Attempts by software DVD players like PowerDVD have tried to add Theatre levels mode which darken the movie. As a result the use of ColorYUY2 to correct the levels is startling to some of us.

This comes downs to personal taste. However, lets not feel like we HAVE to lighten the movie. Maybe that's the way the film producers intended it to be?

Take Care,

Dali

drizztcanrender

14th September 2002, 20:26

I agree that ColorYUY2 darkens the movie,as the tests show us.But some people,as you said,dodn't want that effect.So what i proposed was to find a way to add the filter only where we have the dark-scenes,where the blocks would probably be.I'll try Marc's advice and see what happens.

iago

14th September 2002, 23:10

Originally posted by Dali Lama
Not to hinder any possible positive results from this testing, but my feelings are that ColorYUY2 IS supposed to darken the movie...
...This comes downs to personal taste. However, lets not feel like we HAVE to lighten the movie. Maybe that's the way the film producers intended it to be? I guess I agree with Dali. Since the main problem that was discussed at the re-beginning ;) of this thread was that even the most good-looking encodes on PC monitor, actually displayed many ugly blocks in black areas (or almost-black/impure-black areas, however we put it) when watched on TV. So the problem was "mainly" TV-viewing oriented, though it is possible that it can be observed on PC monitors too, but much less of course.

What ColorYUY2 does is solve "this" problem, and provide pure/true black color at those areas, hence eliminating the annoying blocks appearing at those areas. Also, the darkening effect is actually (more) visible on PC monitor; not on TV, which already provides a brighter viewing environment. So, after applying ColorYUY2, the picture you get "on TV" is actually not so far from the brightness of the original DVD, and also free of all these ugly blocks.

As for me, considering all that's mentioned above, the real problem with ColorYUY2 is that it decreases compressibility. But after a limited number of test encodes (absolutely more tests should be done to derive a better conclusion), I'm hopeful that, that decrease resulting from ColorYUY2 can be compensated by the use of some light Convolution3d settings, such as (0,4,4,4,4,3,0), without softening the picture much and keeping enough details, but increasing compressibility.

For watching DVD-rips (also) on TV, atm ColorYUY2 seems the best possible solution to me.

best regards,
iago

drizztcanrender

15th September 2002, 02:50

I guess I agree with Dali. Since the main problem that was discussed at the re-beginning of this thread was that even the most good-looking encodes on PC monitor, actually displayed many ugly blocks in black areas (or almost-black/impure-black areas, however we put it) when watched on TV. So the problem was "mainly" TV-viewing oriented, though it is possible that it can be observed on PC monitors too, but much less of course.

Try viewing them in my 21" monitor.They are visible,trust me.

iago

15th September 2002, 10:34

@drizztcanrender

Try viewing them in my 21" monitor.They are visible,trust me.I do ;).

iago

Didée

15th September 2002, 16:03

Sorry for not trying this on my on, but honestly, I do have some problems with using "layers", and unfortunaly, by now I don´t have hours and hours available for testing :(

Well, the most annoying aspect of those "dark blocks" is that they are often color-casted: greenish or purple-ish.

Wouldn´t it be helpful to remove all color from the very dark parts, say in the range [0 < luma < 16] ???

I suppose chroma information in the almost-black areas is hardly noticeable. But removing chroma in these areas probably could make the blocks less dominant?

IF this should be useful, I assume it´s not such a big effort for all the great coders out there to make a new filter out of it.

Perhaps someone could give a quick shot on this, and report the results? As I said, I have some problems with "layers".
(If someone gives a 100% working script for this purpose, using "layer", I will perform the test myself.)

iago

20th September 2002, 13:59

Hello all,

As long as the "black-color" problem (I guess it's better to call it this way rather than "blockiness in almost-black/impure-black areas" problem) is not totally and understandably solved I'll keep this thread up-topic ;), since I guess ColorYUY2 results are not too satisfactory for me either (darkening effect, worse color representation, etc.), though at first sight/tries it seemed the only possible "encoding-stage" solution to the problem for the time being.

What colorYUY2 does (in our example) is that it converts the levels of a TV scaled (!) encode (maybe a bit more than this considering the better effect than levels and the worse side-effects as well) to PC scale:

ColorYUY2(Levels="TV->PC") is actually something like levels(16,1,235,0,255), where 16 and 235 are the inputlow-inputhigh values (representing TV scale range), 0 and 255 are outputlow-outputhigh values for PC (representing PC scale range), and 1 stands for Gamma (correction).

And imho the whole problem may be actually related to this: the difference between the TV values for black (16) and PC values for black (0)?..

However, the interesting point is why do we get (or do we get) a TV scale in our encodes although we choose "PC scale" in DVD2AVI (as Dali Lama points out), which must accept 0 and 255 for PC scale values and therefore already "should" have solved the problem (?). Is it DVD2AVI (1.74 or 1.76 version as used by most) related, or mpeg2dec.dll related, or both, or none? And imho here lies the problem *and solution*, considering what Koepi, trbarry, and Swede discussed in their *very valuable* previous posts.

Ok, latest versions of ffdhow provides levels correction, adding noise is another alternative for post-processing; and ColorYUY2 filtering, messing with levels/layers etc. are other filtering-based choices, though none of these are the real and perfectly-working solutions! There *must* be another solution/way for the encoding stage to overcome this ugly problem.

Currently I'm doing some tests with DVD2AVI 1.74, DVD2AVI 1.76, using different scales, and frameserving with both avisynth and VFAPI. I've already got some ideas, but in order to be able to make better comments, I want to discuss these when the test encodes are over.

best regards,
iago

Maurizio

20th September 2002, 14:33

Well, I've got very good results using Virtual Dub filters,
simply enhancing the saturation and the contrast,
and applying the smoother filter with 0 threshold.
Regards
Maurizio

iago

20th September 2002, 17:35

@Hello again
@Koepi, trbarry, -h, Swede, etc. (and all other experts ;))
(especially your views would highly be appreciated)

So, about the tests:

Test parameters:
Color Space: YUV
XviD (Koepi's 04092002-1) constant quantizer 3 - MPEG quantization - no lumi
one vob from Baraka / 512*240
VFAPI -> full processing mode (crop-resize in VDub, Precise Bilinear)
AVISYNTH -> fast recompress (crop-resize in AVS, Bilinear)

After the tests, I could only reach some limited results:

1) Either with DVD2AVI 1.74 or 1.76, there *is* a difference in terms of black color (values) between the PC Scale and TV Scale modes when frameserving with VFAPI; and VFAPI seems to consider the values given by the .d2v file:

PC Scale: YUVRGB_Scale=1 (same in the .d2v file both with 1.74 and 1.76)
TV Scale: YUVRGB_Scale=0 (same in the .d2v file both with 1.74 and 1.76)

2) With DVD2AVI 1.76, there *isn't* any difference in terms of black color treatment between the PC Scale and TV Scale modes when frameserving with avisynth (mpeg2dec.dll), and the effective Scaling mode seems to be already "PC Scale" (not TV Scale) whether you choose PC Scale or TV Scale. However, as stated above, both 1.74 and 1.76 includes the difference in the d2v files: (PC Scale: YUVRGB_Scale=1 / TV Scale: YUVRGB_Scale=0). So it "seems" that it is mpeg2dec.dll that doesn't respect these values (as trbarry pointed out before if I'm interpreting his statement correctly).

3) As a result, that's why ColorYUY2(Levels="TV->PC") gives an excessive darkening effect imho. It is scaling an already PC Scaled film once more to PC Scale (blackening the movie once again), though the input values are not 16-235, but already 0-255!?

4) When watching the TV Scaled and PC Scaled encodes where VFAPI is used for frameserving, they are obviously much blockier and worse than the avisynth encodes (both avisynth encodes are exactly the same either PC Scale or TV Scale chosen: effective scaling is already PC Scale as mentioned above). Also, TV Scaled VFAPI encode looks much worse than the PC Scaled VFAPI one.

So, maybe the problem (with our regular avisynth encodes, already PC Scaled -> 0-255) is somewhere else and perhaps inevitable, which is also negating my argument in the former post and some previous arguments by others in the thread. Possibly the only solution: trying to keep quantizers as low as "possible" and apply some noise (if possible, more to dark areas as an option in the future releases of ffdshow).

Hoping more replies arrive to this thread and more opinions are put forward. (All stated above and previously are my personal opinions and experience, only an attempt with my limited knowledge, trying to derive a conclusion about the issue, and may of course contain some mistakes, which will hopefully be corrected as well).

kindest regards,
iago

P.S.: Images are attached to provide some visual aid especially to compare the different "black color treatment" with different options stated above.

trbarry

21st September 2002, 00:59

PC scale means the range 0-255 and TV scale is the NTSC range 16-whatever. But I have never really paid attention to what DVD2AVI does with it because I always use MPEG2DEC.dll through Avisynth, which ignores it.

The reason it ignores it is because it tells how you should convert to RGB and I avoid doing this when possible, staying with YUY2 in Avisynth and also with Fast Recompress in Vdub.

But if I was doing a conversion to RGB then I don't know whether DVD2AVI PC Scale means to convert to PC Scale or to expect PC Scale. ??

And I think at least NTSC DVD's are stored in TV Scale in a color space the same as YUY2. (same colors different bytes)

FWIW. ;)

- Tom

Marc FD

21st September 2002, 10:45

i've made a little test with french friends with a specific Avisynth filter, designed to enhance black a smooth way (linear) without modifying at all the other colors. the problem is, it makes the image bigger, because it increase contrast. (i can provide the bin if you want)

but i can try something else. a sort of search'n'destroy filter :
- find wide black areas
- smooth them
- darken them
- normalize chroma saturation

i need to test it, but i hope this approach would not lead to compressibility degradation.

and i still want to try to make Msmooth-like filter. don say it's not easy, i agree, but i have a lil'idea :D

and i've a lot of things to do for XviD. the problem is XviD work in YUV12 space, and avisynth not. so i'm waiting for avisynth 2.1
(because i want to make some pre-filtering and i want it avaible for XviD and Avisynth)

iago

21st September 2002, 10:51

@Marc FD

So nice to hear that you're working on it :). I hope your efforts give good results so that we can start using/trying your filter against this ugly problem ;)

best regards,
iago

P.S.: Looking forward to hear about the results of your tests.

soulfx

21st September 2002, 16:30

Hi, has anyone tried doing some tests with a levels command that would put the video back into the TV Scale and then use the ColorYUY2? Also I think the code is out there for ColorYUY2, maybe we could take a look at it and see what exactly it does.

Something like:
Levels(0,1,255,16,235)
ColorYUY2(Levels="TV->PC")

Wouldn't this keep the video from becoming too dark, but still allow ColorYUY2 to do it's "magic" since people have reported that it reduces blocks better if not completely comparied to a similar levels adjustment.

One other thing to mention. I know something was said about ColorYUY2 and it's strange effects on color. Since it's a level adjuster the color saturation will increase as you decrease the InputLevel range (ie: InputLow from 0 -> 16, InputHigh 255 -> 235). So to get about the same color as before a Tweak(sat="whateva") would have to be done.

Peace,
SoulFX

iago

25th September 2002, 12:56

-> Hi, has anyone tried doing some tests with a levels command that would put the video back into the TV Scale and then use the ColorYUY2? Also I think the code is out there for ColorYUY2, maybe we could take a look at it and see what exactly it does.

Something like:
Levels(0,1,255,16,235)
ColorYUY2(Levels="TV->PC")

Wouldn't this keep the video from becoming too dark, but still allow ColorYUY2 to do it's "magic" since people have reported that it reduces blocks better if not completely comparied to a similar levels adjustment.

@soulfx

Actually ColorYUY2 doesn't "reduce" blocks-in-black better than a levels adjustment, but it totally "eliminates" them, which is the real magic ;). Regarding your above arguments:

1)

Levels(0,1,255,16,235)
ColorYUY2(Levels="TV->PC")

I have tried it exactly like that (and then with some tweaking as well), but it simply doesn't work. Blocks are all around again as a result.

2)

I have also tried tweak(bright="whateva") ;), together with ColorYUY2(levels="TV->PC") with no success.

I haven't tried tweaking the saturation yet, and it may be worth a try ;). I think what you mean by Tweak(sat="whateva") is "decreasing" the saturation with that parameter (default: 1.0), and use it together with ColorYUY2(levels="TV->PC") ?

best regards,
iago

Marc FD

25th September 2002, 13:16

Hi iago.

could you try if playing with lumagain/lumaoff in my mod can help ??
i implemented it only to simplify testing for you in fact ;)

maybe something like lumaoff=-20,lumagain=138 ??

if this does not work, i've a new idea to test (darkensmoother) :D

JimiK

25th September 2002, 13:17

Iago,

I know you're an excessive tester and I agree with you in most of the points. But my tests (only short test encodes) show slightly different results.

For me the level settings supposed by soulfx reduce the blockiness in the movie. You're right, it's not completely the case, but it's better than an "ordinary" encode while it's still on the same brightness level. If you increase the brightness to compensate for the darkening of the ColorYUY2 some blocks will show up again.

I also tested by setting the luminance gain under DVD2Avi to a higher level. The compressibility suffered badly and the results were not perfect.

The problem I have with ColorYUY2 is that blocks seem to remain at the transition from dark parts to lighter parts. You know, where things are still dark, but not dark enough for it to paint it black.

Right now I'm encoding a full movie with the 'soulfx-settings'. I'll keep you posted about the results.

Btw. where is 'tweak' included? Is it in another special version of Avisynth or is it an ordinary plugin?

Best regards,

JimiK

iago

25th September 2002, 13:31

-> Hi iago.

could you try if playing with lumagain/lumaoff in my mod can help ??
i implemented it only to simplify testing for you in fact

maybe something like lumaoff=-20,lumagain=138 ?? ;)

if this does not work, i've a new idea to test (darkensmoother) :D

Hello Marc,

First, thanks a lot for your ongoing efforts and valuable works both in general and on this ugly problem! ;) I'm sure, sooner or later, we're gonna solve it. BTW, I have already started using your mpeg2dec.dll, with cpu=6, to achieve really better results than the regular mpeg2dec in terms of blockiness and deringing, but I haven't tried it on this "blocks-in-black/impure black" issue yet.

Can you confirm or correct the below sample script please:

----------------------------------------------------------------
mpeg2source("D:\MOVIE\MOVIE.d2v",cpu=6,lumaoff=-20,lumagain=138)
----------------------------------------------------------------

Also, do you suggest using it with or without ColorYUY2?

thanks and best regards,
iago

Marc FD

25th September 2002, 13:42

definitivelly without.

ColorYUY2 is not designed for that at all.
the benefit you see is a side effect.

i'll try a specialized filter to do intense, pure, dark, sweet, ect... more black than black black.

you script line seems good. but the values i gave are just a start point. some tweaking is needed.

i recommand you to use avisynth's Histogram() function to help with tweaking. it's very usefull.

iago

25th September 2002, 13:42

@JimiK

You are right, the below script helps to retain the original brightness of the movie, but in my tests it didn't resolve the blocks-in-black issue (maybe a bit better compared to an ordinary encode, but absolutely much inferior to ColorYUY2 alone), so it simply didn't achieve its aim.

Levels(0,1,255,16,235)
ColorYUY2(Levels="TV->PC")

Still, I'd be glad to hear about your test results.

Regarding "tweak", it's an internal filter of avisynth, which you can use simply by adding such a line to your avs script:

tweak(bright=5) or tweak(sat=0.5) just for example...

best regards,
iago

EDIT: Thanks for confirming and clearing it up Marc, I'll take your values as a starting point for my tests.

iago

25th September 2002, 15:08

Marc,

The script that you confirmed:

mpeg2source("D:\MOVIE\MOVIE.d2v",cpu=6,lumaoff=-20,lumagain=138)

gives the following error message when trying to load into VirtualDub:

Script error: mpeg2source dos not have a named argument "lumaoff" (or "lumagain")

iago

kilg0r3

25th September 2002, 15:20

hi iago.

man i really don't know how you manage to accomplish these gigantomaniac testing efforts. great!

one short thing did you ever try to use lumamasking together with ColorYUY2, and, if so , what were your results? thanks so much.

JimiK

25th September 2002, 15:55

What we needed is a "movie" room, where you could have the same light conditions all the time. The problem is: when it's day and everything around is light, you almost see only black in dark scenes when using ColorYUY2. But when it's dark around and you didn't use it for your encode, you can see so many blocks in dark areas. Maybe we should encode all movies twice and label them with "view only in dark environment" or "... in light environment" ;)
What I forgot to mention in my last post is, that when I used the new ffdshow and set the levels to 20, I got a decent picture (lighter) without such a big increase in blocks, as when adjusting the brightness (movie was encoded with ColorYUY2).

EDIT: Thank you iago. Tweak was introduced in Avisynth 2.05 and I was still using 2.04

Greetings,
JimiK

P.S. off topic: what I wanted to say, I've been following this forum for almost a year now. It's so cool how many test are done and how many good ideas come up in this forum. You guys are really amazing, both, developers and testers. Keep up the great work!

Marc FD

25th September 2002, 18:12

sorry iago.
it was lumgain and lumoff.
take a look at the readme file ;)

droolian01

25th September 2002, 21:42

Hi there.

I've written my first 'function' or whatever you call it. Please note that i am not a programmer (last time i did anything like that was at school with a BBC micro!!!!) hence some probably obvious errors (but not to me!).

History/idea - this script will produce selective darkening in only the dark areas of a video and leave the rest of the video 'as is' so as to negate the 'dark block' problem. It uses 3 masks each derived from the source clip to incrementally darken the selected area. I used just one mask originally but to get any serious block reduction i noticed the edges between masked and unmasked areas to be too sharp, hence the 3 gradually stronger (and gradually smaller) mask areas. If you could view them they would look like a contour map of a hill with the 3 boundry lines equivalent to contour lines.

###############################################################################
# DBK (or Dark Block Killer) #
# Attempts to reduce the appearance of 'dark blocks' by selectively darkening #
# dark areas and leaving the rest of the video untouched #
# All values have the same funtion as in LEVELS, with "threshold" being the #
# value that gets treated as 'dark' and "amount" is the amount of darkening #
# in these areas #
###############################################################################

function DBK(clip "clip", int "threshold", int "amount")
{
overlaya=levels(clip,amount/3,1,255,0,255)
overlaya=resetmask(overlaya)
mask_clipa=levels(greyscale(clip),threshold,1,threshold+1,255,0)
overlay_clipa=Mask(overlaya,mask_clipa)
finala=layer(clip,overlay_clipa,"add",255,0,0,0)
overlayb=levels(clip,(amount/3)*2,1,255,0,255)
overlayb=resetmask(overlayb)
mask_clipb=levels(greyscale(clip),(threshold/3)*2,1,((threshold/3)*2)+1,255,0)
overlay_clipb=Mask(overlayb,mask_clipb)
finalb=layer(finala,overlay_clipb,"add",255,0,0,0)
overlayc=levels(clip,amount,1,255,0,255)
overlayc=resetmask(overlayc)
mask_clipc=levels(greyscale(clip),threshold/3,1,(threshold/3)+1,255,0)
overlay_clipc=Mask(overlayc,mask_clipc)
finalc=layer(finalb,overlay_clipc,"add",255,0,0,0)
return finalc

}

#
# how to use
# avisource("source.avi")
# filters.......
# Resize(xxx,xxx)
# ConvertToRGB32()
# DBK(10,15)
# converttoyuy2() (or rgb etc...)

If you substitute the 'return finalc' with 'return mask_clipc' (or mask_clipa or mask_clipb) you can see where the masks work (white = where the darkening will take effect).

Threshold - is the value that any pixel in the source clip will be 'filtered' (actually <= this value) - i suggest a value of 10.

Amount - is the value by which the filtered area is darkened.

All these values are a in LEVELS so must be 0 - 255 (i havent put any silly values in yet, but why put them in there anyway???)

Hope you serial testers out there get your teeth into this and give me some feedback.

Yours

scott

iago

25th September 2002, 21:59

-> sorry iago.
it was lumgain and lumoff.
take a look at the readme file

Marc,

They are defined as "lumaoff" and "lumagain" in the readme file of "mpeg2dec pp mod b2" too ;), which is the one I use. I don't know whether you've updated the regarding info in the readme file of "mpeg2dec pp mod b3" that you have announced in the avisynth forum, but it hasn't been validated yet I guess.

Anyway, thanks for the information: I'll try them as "lumoff" and "lumgain" ;).

best regards,
iago

EDIT: Well, "lumoff" and "lumgain" works, yes... But in the readme file of "mpeg2dec pp mod b3", these parameters are still named "lumaoff" and "lumagain" ;). BTW, I have upgraded to "mpeg2dec pp mod b3" now.

iago

25th September 2002, 22:09

@kilg0r3

In my tests using lumi masking with colorYUY2 didn't lead to any problems concerning the blocks-in-black issue: colorYUY2 still performed its job perfectly (when lumi masking enabled) regarding the problem. Imho it may only be an option to increase compressibility and to compensate for the compressibility decrease introduced by ColorYUY2.

But in general, so far my experience is that lumi masking "decreases the overall quality" of the encode, introducing more blockiness and artifacts, and *personally* I always leave it unchecked ;).

@droolian1

That's very nice. I really wonder what the results will be both in terms of the blocks-in-black ;) issue (as I call it) and in tems of compressibility.

@soulfx and all,

using ColorYUY2(levels="TV->PC") with tweak(sat=0.75/0.80) gave me some good results in terms of restoring back/down the saturation increase introduced by colorYUY2, and it also gained back some ~3% of the compressibility that was lost due to colorYUY2 use. But I still didn't like the colors, that seem to have changed noticably.

best regards,
iago

JimiK

25th September 2002, 23:04

Hi,
I just wanted to share my results. Avisynth script:
mpeg2source("E:\video_ts\13th\vts_01.d2v",cpu=6)
crop(0,9,702,553)
SimpleResize(656,352)
Levels(0,1,255,16,235)
ColorYUY2(Levels="TV->PC")

In dark scenes there are quite little pixels. It's not perfect, but you have to see that the compression ratio is 2.5:1. So I think it almost impossible to get away with no blocks. The brightness is all right, just as if no level tweaks were used. But the picture is very soft. I used h263 in both passes, but I didn't think it could smooth the movie that much, because I used SimpleResize which should deliver a sharp picture.

So long,
JimiK

yaz

26th September 2002, 10:52

hi all !

i follow this thread for awhile & i decided to summarize my findings on "deblocking the dark regions" many times but these days i'm pretty short in time. maybe this weekend.

now about some things just occured.

Originally posted by JimiK
Levels(0,1,255,16,235)
ColorYUY2(Levels="TV->PC")
:rolleyes:

imho, u're on a wrong way with it. afaik, levels is a brightness/contrast filter in rgb & it does luma transformation in yuv. coloryuy2 does the same. i've done a short test & i guess, coloryuy2("tv->pc") does something like levels(~16,1,~240,0,255). not xactly, but it's very very close. (more tests needed). so ...
accordingly, the script above does (almost) nothing. 1st u shrink the scale & then xpand. of course, "it's regain the original contrast" (iago) but, imho, distorts the scale (yaz:-).
in add, putting levels 1st would increase the noise in the low luma region. maybe the revers order, but ... i should think of it more thoroughly

as far as i remember, the original topic was 'how to decrease blocking seen on tv", but i have the bad feeling that some of u test results only on pc. it's never the same! pictures looking awful on pc may look good on tv (that's what i should sum up:-) pls, check your results always(!) on tv.

@droolian
converting rgb<->yuv may result in blocking as it introduces some kinda scale distortion. check it out, a script is given for it in avs_help.
anyway, your script is quite impressive, but i don't really understand what's it for. gimme some time, i'll check it out.
i use something similar for doing a luma masked noise removal, but the script is not at hand, i'd drop it if u're interested.

@iago
'black-blocking' on tv has many different sources. once we summed them up in the nandub forum & we put together lotsa tricks to decrease them, but i haven't found this thread :-( ok, it was a year ago or so.
until i brush up my mind, try these tricks:
- do not(!) remove fine noise from the picture. some filters can thresholding not only the upper but the lower value for smoothing (say, tempcleaner, dnr2ver06). try to keep back some fine noise in the low luma regions & encode this. for me, it decreases dark blocking. i got some avs script to do this, i try to collect them. imho, it's much better than imposing noise in postproc (say, with ffdshow)
- use some kinda xtra sharpening. it causes a kinda twinkling/ringing on pc screen, but increase contrast on tv. in add, it increses the noise in the dark regions which may help preventing that shitty blocking.
- use always the max resolution available & try to beat the limits given in some maquals. (of course, keep it in a reasonable range:-) it may help agains blocking
- ...
oops, i got some other 'stupid' trick, but i'm running outta time, if u're interested i try to collect my memories.

hormonally yours
yaz

ps before u start lame me, i must confess, i'm really that :-)

JimiK

26th September 2002, 13:02

yaz,
the idea was, that ColorYUY2 does more than levels. So you could hold the brightness by adjusting it with "levels" and get rid of most of the dark blocks with ColorYUY2. But after some more short tests I have to say you're right. There is almost no difference and on one scene I would even say the one without these level tweaks is better. And you're right with the guess that I don't watch the movies on my TV. I'd love to (couch (TV) more comfortable than chair (PC)), but sadly I don't have a graphics card with TV out. So I can only contribute with tests how to eliminate blocks on PC and hope it works as well for TV. I'm sorry, but there is no PC-block thread.
@ Marc FD
I'm curious if your new project "darkensmoother" :) will work. I've tried an test encode with your mpeg2dec luma settings and have to say, you already got a filter that makes the blackest black I've ever seen (whole picture) ;) Well, I guess some more testing and tweaking from my side is required.

Best regards,
JimiK

iago

26th September 2002, 20:05

@yaz

It's really nice to see you step into this thread, share your opinions with us, and provide everyone with your valuable experiences :).
(the yaz from those old nandub times?..;))

Well, just a few words for now: ;)

* Yep,
Levels(0,1,255,16,235)
ColorYUY2(Levels="TV->PC")
is really of "no" use in terms of solving the black-blocking issue, as I also had stated in one (or more than one? ;)) of my previous posts.

* ColorYUY2(levels="TV->PC") indeed does something "like" levels(~16,1,~235-240,0,255), but not exactly. ColorYUY2 eliminates the black-blocks problem (with its a lot discussed side-effects of course); however levels(16,1,235,0,255) for example cannot achieve the same result as ColorYUY2 in my tests.

* I usually encode 640*... and very rarely go below that and I don't use any extra filtering (except test purposes), therefore I hadn't tried any extra sharpening filter before either. I'll try that. Maybe together with LanczosResize as well?! (But what about compressibility? ;))

* I agree that it would be much better to solve the problem "as much as possible" during the encoding stage, trying to keep the original noise, rather than post-processing with noise during decoding. I'd be very glad to try your avs scripts for that purpose.

* Also, imho, lumi-masking is a real "no-no" if we want less black-blocks in our encodes; and MPEG quantization is also more effective than h263 regarding the issue.

@JimiK

Since the "look" of an encode on PC and on TV "may be" totally different in terms of visual quality and black-blocking, regarding the solution of this issue, I guess we "have to" take TV as the reference viewing source.

Also, Marc's new mpeg2dec pp "is" as effective as ColorYUY2 in solving the black-blockiness issue, but still suffers from the same problems: compressibility decrease and losing the original colors. lumoff=-20 was an effective value in my tests to solve blocks-in-black the problem, but lumgain=138 (you're right) makes the picture even darker than ColorYUY2. When you go higher (lumgain=160-170) you begin to get closer to the original brightness of the movie, but paying the price with even more decrease in compressibility. As a result, you can try lumoff=-20,lumgain=170 as a starting point for your tests and tweak these values as you desire, but remember the higher the lumgain value, the less the compressibility!

@all

I guess we can now put aside ColorYUY2 and start tweaking with lumoff and lumgain values of Marc's "mpeg2dec pp mod b3" to achieve the same powerful effect as ColorYUY2 (and with the same loss in compressibility ;)), and besides that, there are much more tweaking possibilities with it to regain the original brightness as much as possible using "lumgain" parameter.

OK, that's all for now,
kindest regards,
iago

iago

27th September 2002, 01:50

Hello everybody,

I just wanted to share a good news with you. I guess finally I have found a way to beat down the ugly blocks-in-black problem as effectively as possible, without darkening out the encode as ColorYUY2 does, keeping the brightness as close to original as possible, and without decreasing compressibility drastically.

I don't want to say that it is the perfect or the only method, but at least it is the method that has worked best for "me" so far, and wanted to share it with everyone.

First of all, I want to thank to yaz, who inspired me to try this (sharpening) method with his comments in the "lumi-masking" thread; and of course to Marc FD, who coded "mpeg2dec pp" with the lumoff and lumgain parameters, and to trbarry for his great UnFilter filter.

Well, for the tests, I used Marc FD's mpeg2dec.dll pp, Koepi's 23092002-1 build, MPEG-custom HVS matrix for quantization type, constant quantizers up to 8, both BicubicResize(0,0.6) and BilinearResize, trbarry's UnFilter.dll in script3, and checked the visuals "only" in terms of black-blocking issue.

Here are the test results:
(1500 frames from a dark source to best spot the black blocks, and constant quantizer 4)

script1 (normal encode without filtering):
------------------------------------------
LoadPlugin("C:\MPEG2DEC_PP\MPEG2DEC.dll")
mpeg2source("D:\TEST\TEST.d2v",cpu=6)
crop(4,8,-4,-8)
BicubicResize(640,352,0,0.6)

file size: 3332 kb
visuals: full of black-blocking/a real shitty look

script2 (ColorYUY2 encode):
---------------------------------
LoadPlugin("C:\MPEG2DEC_PP\MPEG2DEC.dll")
LoadPlugin("C:\MPEG2DEC_PP\ColorYUY2.dll")
mpeg2source("D:\TEST\TEST.d2v",cpu=6)
crop(4,8,-4,-8)
BicubicResize(640,352,0,0.6)
ColorYUY2(levels="TV->PC")

file size: 4046 kb (~20% decrease in compressibility)
visuals: no black-blocking/encode too dark and many details lost

script3 (UnFilter(5,5) and lumoff=-2 encode):
--------------------------------------------
LoadPlugin("C:\MPEG2DEC_PP\MPEG2DEC.dll")
LoadPlugin("C:\MPEG2DEC_PP\UnFilter.dll")
mpeg2source("D:\TEST\TEST.d2v",cpu=6,lumoff=-2)
crop(4,8,-4,-8)
UnFilter(5,5)
BicubicResize(640,352,0,0.6)

file size: 3436 kb (~3% decrease in compressibility)
visuals: no black-blocking/almost the same as or very close to original brightness/almost all details preserved

Yes, these are the results I have got. It seems "script 3" is the "Ultimate Script" ;), but of course still open to tweaking and maybe even bettering ;). I hope you give it a try too, and report back your comments. Atm, I'm very pleased with the results I've got and starting a full 2-pass encode using the parameters in script3.

I hope this works for you too, to free your encodes from black-blocking problem, without decreasing the compressibility noticably and without darkening out your encode. At least, just give it a try! ;)

kindest regards to all,
iago

P.S.: lumoff=-2 and UnFilter(5,5) cannot eliminate the problem when used separately. It is the above combination of two that provided the solution for me.

soujir0u

27th September 2002, 02:39

Great! What about the encoding speed?

ookzDVD

27th September 2002, 03:31

@iago,
Sorry, I was not notice the forum recently,
what is "black-blocking" problem exactly ?

I have problem encoding "panic room", the movie almost dark
in every scene, every time I try to encode I always got bad
blockiness in the dark scene :(.

Dali Lama

27th September 2002, 04:50

Iago my friend. Nice name, Ultimate Script!! :sly:

I just wanted to mention that before I tried full encode with it I noticed vertical lines on the image due to the Unfilter. It shows up when using MPEG/Custom matrices. I just wanted to let you know, I don't believe its due to your script though, because that unfilter by itself caused that for me.

Keep up the good work,

Dali

Update: When I use Unfilter(10,10) those vertical lines are gone??

yaz

27th September 2002, 09:29

hi iago (& all) !

1st of all : your pleasure is my pleasure :-)) anyway, i'm not happy about that u split the original thread. somebody (a moderator ?) should close that officially & reference/link it here. otherwise we loose the opinions/results posted there or we must devide ourselfs. that's my bad experience. ok, i'd stop grumblin :-)

sorry for not having time to go into details but lemme some quick adds
your solution here
- is(can be) effective if the source is a good quality dvd (even that have some kinda noise espec in the low luma regions. it's our luck:-))) a 'noisy' source is a different part of life
- solves only one possible source of blocking. that's what i call 'black isn't black' problem. it's the result of the improper scaling. the luma scale on tv is narrower than on pc. if u wanna get the same 'look' on tv u must(!) (down)scale. afaik, levels(16,1,240,0,255) is the proper transformation, but correct me pls, if i'm wrong. it was discussed xtensively in the dvd2avi forum earlier but i can't find that thread anymore (why can't i find that good old threads?:-(

Qs:
- what does your script whit shallow luma gradients ? i mean, where luma changes very slowly. (e.g. a dim room, light in one corner) i guess, it'd help that too but it'd be tested.
- what about sharp luma gradients ? (e.g. dark scene & then a sudden lumma change; a bomb, flashlight or so. say, intro scene in matrix, when the torchlight zooms in) my guess is the same as above, but ... test.
- what about sources with heavy noise ?
(i don't blame u ! i just wanna outline further problems i know:-)

adds:
- 1500 frames ... seems too short for me. maybe, it's enough for one effect, but may be misleading if u consider encoding too. the bitrate distribution may be quite different if u take only that short part or a bigger one. (think of people like me making always 2pass encoding:-) i use 10-20 mins from a film trying to take it so as to contain as many problematic effects as possible.
- u can be much 'braver' with unfilter. try harder settings too, but be careful, at some args it crashes for me ('illegal calls' & so, dunno why)

lemme know your opinions about

hormonally yours
yaz
(yeah, the lamer from the 'good old nandub times':-)))

iago

27th September 2002, 10:31

@Koepi

I think yaz is right. Can you please attach this new thread to the end of "lumi-masking" thread? I first thought it "might" be better to start a new thread and discuss this black-blocking problem separately, but yeah it's likely to break the unity this way, and possibly will be better to go on in the old lumi-masking thread ;)...

@yaz

You are right. It was only a very short test, focusing only on "one" problem and a bit on compressibility, and leaving the rest to be checked later "after" the main issue is resolved. (BTW, I always do full 2-pass encodings too ;).) Absolutely more tests should be done, especially two-pass tests, with different quantization types such as h263 and MPEG (default), with different resizing methods (I tested not only BicubicResize(0,0.6) but BilinearResize too, as an option for more compressibility and to compensate for the possible extra-sharpening effects), and with tweaking the Unfilter strength parameters as well.

I guess I'll try (or we'll all have to try) to answer the questions in the "Qs:" part of your post when we finish our two-pass test encodes and check the visuals, this time concentrating on "other aspects" of the encode as well, since the major problem, the "black-blocking" problem, seems to be really beatable with the above method.
(Imho, noisy sources might require to be treated "completely" in a different way and with additional filters too. Currently "I" prefer to focus on clean sources actually.)

@Dali Lama

Ahoy my friend! ;) I also noticed those vertical lines with Unfilter(5,5) and MPEG-custom matrix. But do you say they don't occur in other quantization types (which is good imho)?? Maybe tweaking the Unfilter vertical sharpen parameter a bit down can also be a solution for that?

@ookzDVD

Well, I see better now that it wasn't a good idea to part from the original "lumi-masking" thread, where the issue was discussed in detail.

@soujir0u

All I can say is that, Unfilter is really fast even on my poor Cel900! ;)

kindest regards to all,
iago

fANMIR

27th September 2002, 10:52

@iago

I am looking forward to test your scripts but I don't have access to my encoding computer in a few days. So do you mind if I ask you to bring us some pics of your results? I would be very happy to compare screenshots with my eyes 15 cm before monitor ;) .I wonder how those operations affect picture quality ( I like very much sharp images but I hate those black blocks :) )

cya !

Koepi

27th September 2002, 12:45

I merged the splittet thread upon iago's request.

Regards,
Koepi

HarryM

27th September 2002, 12:49

Luma-masking (actual version implemented in XviD) is FINALLY AND REALLY correct, please?
Can I use it without risk? :)

Koepi

27th September 2002, 13:17

You have always the risk that you can see compression artefacts...

But it's "more correct" than the "in-between" one which was in the code as doom9 tested the codecs last time...

Regards,
Koepi

rui

27th September 2002, 15:54

Originally posted by iago
...I used Marc FD's mpeg2dec.dll pp...

I am sorry for asking for this, but i looked and looked and coudn't find any link to download MarcFd's mpeg2dec.dll :(
Could someone point where can i get it? (i already visited Marc's site, but it wasn't there)

Many thanks.

iago

27th September 2002, 15:54

@Koepi,

Thanks a lot for removing back the thread ;).

@Dali Lama,

You're right Dali, mpeg/custom quantization seems to cause/increase vertical lines issue of UnFilter. Imho, it seems safer to go with h263 quantization for now. Also, the more you sharpen, the more you have vertical lines, and the more you lose compressibility. Therefore, a combination of UnFilter(5,5) and lumoff=-2 used with h263 quantization type (and as a personal preference with LanczosResize) seems to be the most reliable and effective combination for me, in terms of both eliminating black-blocking very successfully and still keeping compressibility decrease within a very small range (and also in terms of avoiding the vertical-lines effect as much as possible ;)).

Currently I'm doing some two-pass tests with Matrix Chapter 2, I will post back the results and write my comments when finished. (BTW, I have already encoded a full movie "Ghost Dog" with great success using the previously mentioned script, with no black-blocking problem and with no annoying side-effects for this specific rip.)

best regards,
iago

edit: rui: you can find it here: http://forum.doom9.org/showthread.php?s=&threadid=34253&pagenumber=2

rui

27th September 2002, 15:58

thanks buddy :)

And sorry for not being hable to find it. The rules are for everyone, so my post was a little off :o

iago

27th September 2002, 16:33

Hello everybody again,

After my full two-pass encode of Ghost Dog (a rather dark movie in general) came out really good, without any black-blocking, without much compressibility decrease, and without any visible side-effects, using the above mentioned script; I went for trying it on "another source" to see if it can still achieve its aim.

And the result was much better and really much more impressing than the regular/unfiltered encode.

Well, here are the test parameters and results:
-----------------------------------------------
* Matrix Chapter 2 (generally dark content with both high-motion and low-motion scenes)

* Koepi's 23092002-1 build, two-pass - internal curve compression and linear scaling ("alt. curve system" disabled and high/low values in "two pass" is set 0/0), h263 quantization, no lumi, Max-Min I-frame intervals: 300-5, quantizers capped: 2-6/2-12, payback proportionally, payback delay: 300, aiming for 14336 kb (which corresponds to a bitrate of ~645 kBit/s)

* visuals checked on TV.

* script1 (normal encode/no filtering)
----------------------------------------
LoadPlugin("C:\MPEG2DEC_PP\MPEG2DEC.dll")
mpeg2source("D:\MATRIX\MATRIX.d2v",CPU=4)
crop(0,84,716,408)
LanczosResize(640,256)
----------------------------------------
Quantizer distribution for 2nd pass:
Q:2:939
Q:3:3601
Q:4:9

Visuals: full of black-blocking problem as expected, very ugly look.

* script2 (UnFilter(5,5) and lumoff=-2 encode)
-----------------------------------------------
LoadPlugin("C:\MPEG2DEC_PP\MPEG2DEC.dll")
LoadPlugin("C:\MPEG2DEC_PP\UnFilter.dll")
mpeg2source("D:\MATRIX\MATRIX.d2v",CPU=4,lumoff=-2)
crop(0,84,716,408)
UnFilter(5,5)
LanczosResize(640,256)
-----------------------------------------------
Quantizer distribution for 2nd pass:
Q:2:713
Q:3:3808
Q:4:28

Visuals: totally free of black-blocking problem, sharp and impressing quality with no other annoying side-effect

At least for me, this ugly problem seems to be solved (the solution is represented by script2 here) and I'm really glad for that ;). Hope you find it useful too, to free your encodes from those ugly blocks-in-black! ;)

best regards to everyone,
iago

Marc FD

27th September 2002, 16:56

@iago
can you try a little Blur2() 0.8x1 or 0.4x2 in the place of Unfilter ??
i wonder if it's faster and/or give the same good result.

Swede

27th September 2002, 21:00

I just wanted to say a big thank you to all you guys. Especially the infamous serial-tester Iago. I've been following this thread for a long time and so far my personal tests have confirmed your conclusions, using MarcFD's mpeg2dec has clearly shown an improvement of my encodes.
Thnx for your efforts guys!

Koepi

27th September 2002, 23:49

Iago,

that's good news :)
Can you test that with lumi masking activated as well please, just to see if we still can try to gain compression? ;)

Thanks a million,

serefe,
Koepi

iago

28th September 2002, 01:52

@Koepi

Yes friend, tested and confirmed! ;) (using the same above script for the same source with the same encoding parameters) It works with lumi-masking too. And below is the quantizer distribution of the lumi-masking enabled encode:

Quantizer distribution for 2nd pass:
Q:2:734
Q:3:3784
Q:4:31

sevgiler, ;)
iago

@Marc,

Sorry I couldn't test Blur() yet, but I'll try it asap. Once again, thanks for your "mpeg2dec pp mod" with lumoff/lumgain parameters, which actually helped a lot.

regards,
iago

@Swede,

Thanks for your kind comments, with a big friendly :) from notorious iago...

Koepi

28th September 2002, 09:59

Thanks iago,

that's really nice to know.
Let's see if a "[ ] optimize for TV output" switch will make it into XviD... setting luma offset to -2 can be easily done I think, so you just need post processing on the source, and unfilter.dll to work against the softening effect of that...

Wow, that's quite useful knowledge.. :)

Thanks a million again,

best regards,
Koepi

iago

28th September 2002, 10:10

@Koepi,

Yes my friend, that would be great actually, making it internal to XviD ;). But I'm still waiting for some more feedback from other people to confirm "my" test results. I hope good news and/or some potential problems that I may not have noticed (if there are any) with this method are reported back here so that we can all feel more comfortable and sure about the effectiveness of the method, which has worked without any problems for me so far.

kindest regards "dostum" ;)
ciao,
iago

Koepi

28th September 2002, 10:21

@iago,
I'm currently testing this with "the last boyscout" - with lumi masking, and no unfilter to work against a small softening effect of post processing (the picture suffered when using that).
Unfortunately, I can't test on TV out as I have no TV set, but I can at least report if it looks ok on PC ;)

I already wrote to the XviD developer list to make the core developers aware of this possibly simple solution to this problem :)

Serefe my friend,

take care,
Koepi

Koepi

28th September 2002, 12:10

Iago,

since I don't have a TV set, I may upload a small clip from the final encoding - I don't know which bandwidth you have, but could you test that sequence for "errors" then? I'll PM/mail you the link then...

Thanks,
Koepi

iago

28th September 2002, 12:25

@Koepi

I'd be glad to test it on TV (especially in terms of black-blocking issue I guess?) ;). Can you also please attach your full avs script and your encoding parameters with the uploaded clip? (I guess you didn't use UnFilter, but only "mpeg2dec pp" from Marc and luma offset: -2 in your encode?)

Anyway, waiting for it pal ;),
iago

Koepi

28th September 2002, 12:31

The encoding will last another 2 1/2 hours from now, after that some mux'ing, cutting out an appropiate scene,... in about 3 hours from now (~17:30h your time) you'll have mail ;)

Yes, you're right, my AVS script looks like this:

mpeg2source("D:\Video_TS\lastboyscout.d2v",CPU=4,lumoff=-2)
Crop(6,73,702,430)
Trim(0,145980).LanczosResize(640,272)+Trim(145981,0).BilinearResize(640,272)

Lumi masking activated, search precision 6, h.263 quant type, external linear scaling, targeted size 627102kb, compression ratio a little below 2:1, const. credits @quant 20.

Regards,
Koepi

iago

28th September 2002, 12:45

@Koepi,

OK. After 17:30 I'll check my e-mail. ;)

regards,
iago

P.S.: In my tests so far, lumoff=-2 alone (without sharpening with UnFilter) was not enough to get rid of the black-blocking problem.

iago

28th September 2002, 13:27

@Marc,

@iago
can you try a little Blur2() 0.8x1 or 0.4x2 in the place of Unfilter ??
i wonder if it's faster and/or give the same good result. I'd be glad if you make it clear which parameters do you want me to test with Blur2(?) and/or Sharpen2(?). I guess a range of -1.0/1.58 is accepted for Blur2, and a range of -1.58/1.0 for Sharpen2 ?..

Thanks,
iago

rui

28th September 2002, 14:09

Well, i have made some small tests using iago's script to the letter, but using Referencedivx HVS "best" matrix.
I still haven't come into any problems with those vertical lines that were mentioned when using this script with mpeg/costum mpeg. Also, in comparison to mpeg, Referencedivx costum matrix lowers a little bit the size of the final avi when making constant quantizer encoding (after reading the readme, i saw that he is trying to achieve the same mpeg quality but with a lower filesize), but i couldn't spot any problems in the image, like those that were reported when using "ultimate" matrix from Dali. So, this could be good to counter the little increase in size that happens when using Lanczos and iago's script to avoid the "black" problem.

EDIT: i just wanted to say that i experienced the clips i made in my tv-out, and iago is a true wizard :)

Marc FD

28th September 2002, 14:46

Blur(a) = Sharpen(-a)

i think Unfilter(+,+) could be replaced by Blur2(+,+)
just a though. i'm not sure at all.

trbarry

28th September 2002, 14:54

With positive numbers UnFilter is a sharpen filter. I haven't used Blur2 but it sounds like a soften filter, so you would think the signs would be reversed from UnFilter.

- Tom

Koepi

28th September 2002, 14:56

[992] Quantizer distribution for 2nd pass:
[992] Q:2:922
[992] Q:3:86002
[992] Q:4:58672
[992] Q:5:385

mux'ing the movie now and doing a visual check, then cutting out a piece for iago to review (if I don't fall above some bad things[tm] myself).

Regards,
Koepi

Marc FD

28th September 2002, 15:17

sorry tom, i didn't know.
strange that sharpening helps !!
i don't understand at all. it's like "black magic" for me :D

iago

28th September 2002, 15:20

@Marc

I guess if we gonna give it a try, we should use either Sharpen2(+,+) or Blur2(-,-) (which are the same after all, no?), since as trbarry pointed out UnFilter(+,+) is "sharpening" the picture, which is one of the keypoints of the above method to get rid of black-blocking problem.

regards,
iago

(P.S.: I have tested Sharpen2(1) to see if it could be an alternative to Unfilter(5,5), but with no success at all. File size almost doubled, but still it could not eliminate the black-blocking issue.)

Marc FD

28th September 2002, 15:24

i think it's because Unfilter use a bigger kernel.
i've not the skills to modify magic spells yet ;)

iago

28th September 2002, 15:47

@rui

That's great news friend! :) I'm really glad that you also tested the script, checked the visuals on TV, and confirmed that this method (a combination of taking the luma offset down by 2 and sharpening the picture with UnFilter) is effective against the black-blocking issue without hurting the picture quality.

best regards,
iago

@Koepi,

I'm downloading the clip now. I'll report back my comments after I check it on TV.

ciao,
iago

Didée

28th September 2002, 15:55

Iago,

first of all, a big THANK YOU for all your valuable testing.

One thing I noticed, and I wondered, because you´re a HiQuality guy:[...]
Matrix Chapter 2 [...]
mpeg2source("D:\MATRIX\MATRIX.d2v",CPU=4)
From my (little) experiences, Marc´s PP mod of mpeg2dec looses much detail with PP on.
See also this (http://forum.doom9.org/showthread.php?s=&postid=187229) post of mine.

Are the results really OK for you?

Koepi

28th September 2002, 16:00

On "Last Boyscout" that PP with CPU=4 indeed looses too much detail, I've to redo that movie.
Currently retesting "Contact" with unfilter(5,5) added to the script, and using the "good-picture-hvs" matrix from ReferenceDivx (which indeed looks yummie, dunno what picture quality that produces, but the matrix itself looks nice and promising) - let's see how it works out. Maybe using lumi masking is a bad idea in these scenarios?

Best regards,
Koepi

EDIT: using that unfilter(5,5) gives back a little detail, Didée :)

EDIT2: stupid me! restarting contact without unfilter-usage. the movie is "over-sharp" on the DVD already so this slight softening effect is just helping the quality - i did it all wrong way ;) last boyscout needs it, contact doesn't. *sigh* I've to think a little more before encoding ;)

iago

28th September 2002, 16:33

@Didée,

I read your post in the avisynth forum too. Actually I agree with you. That's why I started my testings with cpu=6, then went on with cpu=4, and finally came to test with no cpu ;)... However, Koepi is right, using UnFilter and LanczosResize (maybe MPEG quantization type as well) helps to compensate a bit for this loss of detail. But as many times discussed before, finally it comes to personal preferences: some like it sharp, some like it soft; some prefer to use Lanczos, some prefer to use Bilinear. And regarding this issue, there "is" already a solution: using no cpu parameter at all ;).

Since my (and maybe everyone's) "major aim" in this thread was to especially focus on the black-blocking issue, I actually discarded a bit the remaining issues such as the one you mentioned, as there were already some solutions for these, but there was still no working/effective solution for the ugliest problem, that is blocks-in-black.

And happily "a combination of sharpening with UnFilter(5,5) and lowering luma offset by 2" helped to resolve this problem finally. In all the above scrips tested and used, I think these two (sharpening with UnFilter(5,5) and luma offset-> -2) are the most reasonable, effective, and generally-agreeable-upon keypoints and all the rest can be more freely tweaked as to suit your taste.

best regards,
iago

P.S.: Btw, I guess you were right when saying (in another thread) that the actual AR of Matrix is something like 2.50:1, rather than 2.35:1 ;)... Now I'm resizing it 640*256...

iago

28th September 2002, 17:00

@Koepi

I watched the test clip on TV. Surprisingly, I didn't notice much black-blocking in it (just a bit on the left side of the screen when Bruce is opening the door, and only a bit on the jacket of the boy at the right side when the two boys are looking into the car), although you didn't use UnFilter but only applied lumoff=-2. However, I think the clip was not that dark in general ;). I would like to have a look at some more dark scenes actually to be sure. In my tests so far (with Ghost Dog and Matrix), lumoff=-2 alone (without UnFilter) was not enough the get rid of black-blocking problem...

Though it didn't display much black-blocking, the overall quality was not so impressing as you remarked. Too much smearing and blockiness in general (when Bruce turns his back, his white shirt looks terrible for example). Maybe lumi-masking was the cause of this "general decrease" in quality?..

As a result, I think that we must take "sharpening with UnFilter" and "lumoff=-x" as a *method* to solve the black-blocking problem, but of course the values can be changed as well according to the source, as I conclude from Koepi's test clip.

But as to my tests, UnFilter(5,5) and lumoff=-2 seems to be a generally-agreeable-upon combination suitable for "most" sources, in terms of keeping compressibility within a reasonable range, avoiding an excessive darkening effect, and not hurting the picture quality.

of course these are just my personal opinions ;)
kindest regards,
iago

MaTTeR

28th September 2002, 17:07

@iago,

It sounds like you think we don't need the PP CPU=x function now to help eliminate the darnened blocks if I understand your last 2 posts correctly. Guess I'll try again using Unfilter and lumoff=-2 alone. Wonder how mSharpen would compare to Unfilter in terms of quality and performance?

iago

28th September 2002, 17:26

@MaTTeR,

Regarding specifically black-blocking problem, I don't think pp levels have much effect on it. But I'll do some 2-pass and constant quant. tests with Matrix to confirm this feeling ;).

I think cpu=x mostly helps reduce blockiness "in general" (not in black areas), and using it we also gain more compressibility, however as discussed above, (concerning different personal tastes) this can lead to an oversmoothing effect as well.

If we come back to black-blocking again, since our *method* ;) is based on sharpening the image with UnFilter(+,+) and (for clean sources) trying to keep the original noise as much as possible rather than eliminating it, it "might" be even better to use no cpu=x at all and apply only lumoff=-2. But as I said, it requires some more testing! ;)

kind regards,
iago

Koepi

28th September 2002, 17:27

Thanks for the comments iago, I agree with them and your observations.

But as stated in my edit above, it depends on the movie if you need to sharpen it again or not - contact is nearly uncompressable because it's "oversharpened" in the sources, the noise is amplified by it etc., so there it's not dangerous to not sharpen it additionally to lnaczos resize.

I'll redo last boyscout after this 2pass without PP, that should keep the sharpness as well (or with PP and unfilter(5,5)).

Regards,
Koepi

iago

28th September 2002, 17:34

@Koepi,

OK friend! ;) I'll be looking forward to hearing about the results of your new encodes. Please keep us all informed about the results you get.

best wishes,
iago

edit: I agree with you, for sources that are already razor-sharp ;), why use LanczosResize!? ;)

Dali Lama

28th September 2002, 17:58

Iago,

I have tried your method and it looks great. Although I havn't had a chance to check it on TV. I was wondering if you could give this setup a shot.

ColorYuy2(cont_u=30,cont_v=30,hscale=50,x=200,y=200)

It doesn't darken the movie and I think there isn't much blocking in the dark areas, but I don't have a way to check on tv.

If this works then there are two ways to do it. If not, your method is great.

Good Job,

Dali

iago

28th September 2002, 19:06

@Dali

Sure, I'd be glad to try it. But can you please check the ColorYUY2 parameters that you provided? I get the below error message when using the parameters in your post:

ColorYUY2 does not have a named argument "hscale"

thanks,
iago

@MaTTeR,

Using cpu=x or not using it at all does not have an effect "on black-blocking issue" as I can see from my tests. Currently I'm testing UnFilter(5,5) and lumoff=-2 without cpu=x, and I notice neither positive nor negative impact of it on black-blocking issue.

(However, compressibility, deblocking, deringing and smoothing due to the use of cpu=x is a different issue to be discussed imho, as already being done in the related avisynth thread.)

I'd also be very glad to hear about your test results.

kind regards,
iago

cjv

29th September 2002, 05:03

Well I have to say that a combination of luma -2 and unfilter(5,5) combined with MPEG quant sure make a beautiful picture. :) I have tested a few 2 min samples from LOTR, both at constant quant 2, on the TV. Even without Unfilter, that annoying mpeg-4 blocking seems drastically diminished, and in my eyes unfilter only serves to make the picture more detailed, (which at quant 2 is not a bad thing)

Most of my 2-min clips are approx 40 megs, but once unfilter is added they jump up to 48-50 megs, and over the 178 mins of the movie, I feel this will hurt compressibility too much. I could always move to h263 quant possibly, but I feel this introduces too much smoothing for a 3CD rip. Anyways, I will use MPEG, lanczos resize w/luma -2 tomorrow, and see how it turns out visually.

cjv

trbarry

29th September 2002, 06:35

I have been following this thread with some interest but no understanding of what is going on or why UnFilter sharpening could be helping here.

But capture cards and TV circuits have something called Coring that can be set to change all luma values under a certain threshhold to zero. But if, say in DScaler, the coring level is set too high then it will produce black blocks on the screen in dark grey areas as certain pixels fall under the threshhold. I wonder if a similar phenomena is at work here for some reason?

If so it might be fixable with a filter that ensured there were no blacker than black pixels, or no pixels lower than some given level.

- Tom

Dali Lama

29th September 2002, 23:26

Iago,

You need the new version that just came out, I think its v12 or something. If you can't find it I will attach it once I get to my computer.

bye,

Dali

cjv

29th September 2002, 23:50

About the LOTR, using hvs best matrix, lanczos, lumaoff -2 and unfilter, it unfortunately drops compressibility to 47%. :(
I must say,though, that at const quant 2, it reproduces the DVD 100%!! But using 4 cds is too much for me and I want to keep the AC3.

I will try this combo on Training Day and Fight Club, as they seem to compress well when using 2CDs, and report back the visuals tomorrow (yea, video compression on a P4 laptop just doesn't cut it!)

cjv

yaz

30th September 2002, 11:53

hi all !

i'm happy about unifying this thread again. thanx, koepi !

some short notes now (time's not on my side:-))

@trbarry:
i suggested this trick of sharpening to iago. i don't know xactly why does it work, but it does. i guess, it's beacause of sharpening the noise being always present whatever source we use. this way the codec can't smooth out so harshly the regions having low luma noise. anyway, it's just a guess.
btw, i don't use unfilter, as it doesn't work for me. it accepts only some distinct values, say, between 30-40 it works only with 31 and 39. all other give an 'illegal call' in avisynth. can u xplain me, why does it do?

@iago:
use only filters u xactly know what they do. i don't like coloryuy2, cus i don't know what does the different arguments mean & how are they defaulted. (actually, i know only 6 japanese words, but i don't even know how are they written:-)
no blame on u(!) but cyuy2 has lotsa arguments. i guess, tv-pc transformation differs from the corresponding levels() just because of its arguments not set properly.

@koepi:
just a (sh)idea. maybe i'm wrong, but ... afaik, lumi masking is based on the idea of using different(higher) quants on low luma macroblocks. would it be possible to invert this effect? i mean a kinda 'inverse masking' applying lower quants on these blocks. it may help against black-blocking. maybe i'm completely wrong, but it'd worth a try.

@all:
try hq-quant ! it gave me xxlent results concerning black-blocking. the idea (reverting 'modulation' effect) is great! i luv it ! (the question above was inspired by this:-)

about compressibility. it's the factor everybody seems to be anxious about. yes, the tricks we use against blocking would decrease it (in some instances significantly) but ...
xvid (2pass) is extremely good in keeping target filesize. i'm not a perfectionist (like koepi:-), i accept differences in the 100kB range. xvid has never hit this range whatever i tried against blocking. so, no use of being so anxious !

hormonally yours
yaz

ps don't lame me, i do it on my own !

rui

30th September 2002, 14:10

Originally posted by yaz
...about compressibility. it's the factor everybody seems to be anxious about. yes, the tricks we use against blocking would decrease it (in some instances significantly) but ...
xvid (2pass) is extremely good in keeping target filesize. i'm not a perfectionist (like koepi:-), i accept differences in the 100kB range. xvid has never hit this range whatever i tried against blocking. so, no use of being so anxious...

Well, the problem is that, when trying to keep the filesize, with a movie that is less compressible, i believe xvid will rise the quantizers. So the movie will look worse. If one can increase compressibility (without smoothing too much, but this implicates personal choices and tastes), then the movie would look better.

Koepi

30th September 2002, 14:32

Dunno if I posted that already, but XviD is so flexible... e.g. you can circumvent using lumi masking, which particularly overquantizes some regions, with using the HVS_GOOD matrix (if heading for 1CD using h.263 quant type). My first results are promising.
Setting luma offset to -2 like iago suggests helps compressability, too, though for PC monitor playback you have to increase brightness on playback.
For TV playback, some more noise is always helpful, so using unfilter(5,5) increases the sharpness even of noise, so it gets encoded (well, if not quantized in bad heights like quant=10 or something), so alltogether you can tweak your encodings to look good quite easily it seems ;)

Just wanted to mention what I experienced so far when testing different things..

Best regards,
Koepi

yaz

30th September 2002, 14:34

hi rui !

Originally posted by rui

Well, the problem is that, when trying to keep the filesize, with a movie that is less compressible, i believe xvid will rise the quantizers. So the movie will look worse. If one can increase compressibility (without smoothing too much, but this implicates personal choices and tastes), then the movie would look better.

ehmm .. yes/no. imho, u oversimplify the things. a bit :-) of course, if a movie is less compressible, it needs higher quantizers to fit to the same target size. but, imho, it does not mean worse looking in general. say, with the sharpen trick black-blocking can be decreased. of course, the bits lost in this region must be regained somewhere else (so as to fit the same target size). i don't know how xvid spreads bits over the movie, but i can imagine even that it takes these bits from completely different scenes. but(again:-), say, it takes from different regions of the same frame. who can tell whether the new look will(!) be better or worse than that of having those shitty black-blocks. it may be better or worse. as compressibility can't be established before 1st pass so the effect of these kinda changes can't be predicted so easily. u should give it a try. in add, lookin good or bad is quite subjective. just as u wrote :-)

hormonally yours
yaz

iago

30th September 2002, 14:39

@Koepi

Well, OK, I decided to break my silence! ;) And you have a "new" e-mail and a "new" PM. Please check them! ;)

@Dali Lama

Neither ColorYuy2(cont_u=30,cont_v=30,hscale=50,x=200,y=200) nor ColorYUY2(opt="coring") could solve the black-blocking problem in my tests (v0.12 used).

@yaz

* Once again, many thanks for inspiring this (sharpening) method. And I "think" the same as you do about how/why sharpening helps handling this black-blocking issue.

* -> ...'inverse masking' applying lower quants on these blocks. it may help against black-blocking...
I really would like to have such an option one day too, though I don't know if it's possible! ;)

* OK, we'll keep discussing ColorYUY2 with Dali in private from now on ;)...

@all

I agree with yaz that new modHQ is really good, but the good-HVS matrix posted by ReferenceDivx is really worth a try imho, in terms of increasing compressibility even better than h263, and still keeping a pretty good image quality as well as eliminating noise successfully. (Koepi also discussed that in the related thread.)

Also, "lumoff=-2 and UnFilter(5,5)" combination "doesn't" lead to such a drastic decrease in compressibility but only to a minor one, and compared to its benefits on solving one of the ugliest problems -> black-blocking, it's nothing to bother imho. Moreover, this minor decrease may well be compensated using different encoding and resizing parameters/methods (HVS-good-matrix, not LanczosResize but another resizing method, setting some cpu levels in Marc FD's mpeg2dec.dll, etc.)

kindest regards,
iago

EDIT: Sorry, I didn't notice the last two posts by yaz and Koepi before mine, which discuss similar things as well ;)...

rui

30th September 2002, 14:43

yaz, i didn't explained myself very well.
I am big fan of iago's script. I see all my movie in tv-out, so this is a great improvement to me :)
I was only talking about the compressibility thing.
I too believe that the gains we have by using iago's script are much higher than the losses we have in compressibility.

To iago: sorry, i didn't wanted to sound like a critic :o
Your work is VERY apreciated. Thanks :)

yaz

1st October 2002, 16:14

hi all !

instincted by the enthusiasm & efforts of iago (& being angry with myself on being so loose-minded) i returned back to the roots. i captured some short clips from a noisy tv channel & i tried to brush my mind about what to do with them to make acceptable lookin encodes.

i tried 4 different codecs (all in 2pass, wout any filter, all shots checked on tv). wout further detailes, xvid far the best in keeping target size & in handling low luma regions (i've known it, but why not strengthen ?:-)
it is very interesting to watch how the different codecs suffers from the low luma parts. only one surprising experiment: encoding a steady clip (only minimal, slow movements, say, clouds on the sky or so) meant serious problem for most of the codecs. they can't do anything with the dark(er) parts of the scene. they were full of black blocks sometimes almost steady but sometimes even whirling around.
xvid made also heavy blocks, but this blocks gave back always the contours and sometimes even the shades in the dark parts. i luv it, really! but ...
when i tried to use anything but modHQ (my present favourite) everything went wrong. the blocks went more blurry (h263/mod) or got a quite funny whirling pattern (mpeg). the new hvs matrices gave something very close to mod.
when i switched lumi masking on i was able to reproduce the contours but the shades had gone (independently of what kinda quant matrix i used). in add, these parts became flat somehow. i can't put it all together now, but it seems, as if lumi masking would not only raise the quants on dark regions but 'sweep' them somehow. on the other hand, it works definitely against HQ. (blocking is more heavy with it than wout)

okay, just some short experiments, maybe digestable :-)

hormonally yours
yaz

iago

1st October 2002, 16:40

@hello yaz and everybody

Just a very short confirmation based on my own experiences and observations so far: lumi-masking really does more harm than good and decreases the overall quality of the encode whatever quantization type or encoding parameters you use! You'll be much better off if you keep your hands off it! ;)

Well, modHQ being really nice, I still like built-in MPEG quantization actually and in my tests with DVD sources so far, MPEG was the one which seems to handle black-blocking issue more successfully than others, keeping more of the original source noise in those low luma regions as well as more details and a better colour representation throughout the movie.

When it comes to compressibility, I usually prefer to pay the price with BilinearResize in really difficult/extreme cases, rather than using another quantization type, about most of which (including h263) I still have my doubts (or maybe better to say, find less appealing to "my eyes") unfortunately.

best regards,
iago

EDIT: btw, quantization -> detail removal factor ? ;)

If so, imho here lies the real dilemma actually. For instance:

* You choose MPEG quantization (which keeps more details and noise) -> compressibility down -> higher quantizers used in the encode (which sweeps away more details again) -> result: details are lost due to higher quantizers.

* You choose h263 quantization (which removes some details and noise) -> compressibility increase -> lower quantizers used in the encode (which removes less details/or keeps more details) -> result: details were already lost with h263 quantization in the beginning.

So far I have usually preferred the first routine (MPEG quantization) and applying bilinear resize or some light filtering such as TemporalSoften(1,5,0) with it in really difficult, hard-to-compress cases, which seemed "a better working" solution to me, but I really would like to hear about your comments and opinions on this issue...

MoonWalker

1st October 2002, 18:23

Originally posted by iago
So far I have usually preferred the first routine (MPEG quantization) and applying bilinear resize or some light filtering such as TemporalSoften(1,5,0) with it in really difficult, hard-to-compress cases, which seemed "a better working" solution to me, but I really would like to hear about your comments and opinions on this issue...

At last :)..

100% of my movies are done with TemporalSoften(1,5,0) and MPEG...It removes almost all the MPEG noise and it has a very very small loss of detail(I only observe it at Vdub with frame-by-frame comparison)..Note that my rips are all 2CD's cause I keep the AC3 track :)

MoonWalker

kilg0r3

1st October 2002, 18:36

i usually use temporalsoften 2,3,3 or 2,2,2 which IMO removes the same amount of noise but leaves more detailes than 1,5.

another setting i like is convolution3d 0,1,8,3,5,3,0. to smoothe on the chroma plane only, is quite effective regarding a gain of compressability while the detail, for the eyes, comes mainly from luma and hence remains. just for the fun of it try convolution3d 0,1,50,3,50,3,0. it blurres not as much as expected, and, strangely enough produces no color ghosting. (iknow this expresses only my lack of understanding regarding the inner workings of the filter).

cheers to all

iago

1st October 2002, 19:34

Well, perhaps some of these issues about filtering are more suitable to be discussed in the avisynth forum; however, discussing these methods together with their effects when using different quantization types and custom matrices and from a xvid-encoding focus makes it inevitable for them to find their way into this XviD thread ;), and this is really useful imho...

So... ;) filtering luma and/or chroma to what extent, with which parameters, and what results in the end?! Which offers more compressibility without losing fine details, is it possible keeping the noise especially in low luma regions (adaptive denoising?) and filtering the rest, etc. etc... ;)

AFAIK, human eye is more sensitive to changes in the luma plane rather than chroma, and most image information/data is stored already in the luma plane regarding the encoding process? So...

Well, it would really be nice to discuss all these imho! ;)

kindest regards
iago

edit: To be honest, recently I'm conducting some really weird (:D) tests on filtering, trying some aggressive parameters with different filters (using MPEG quantization type) just to see their effects ;)...

cjv

1st October 2002, 20:21

I've been playing around with some custom matricies lately, but for archival l've been using (for the time being) h263/bicubic0.6 for 1CD rips, and MPEG/tsoften(1,5,0)/lanczos (if there's bits to spare) or h263/lanczos for 2CD. And yea, I think temporalsoften w/MPEG looks really great, really helps to remove that MPEG noise.

In my opinion, iago's c3d(0,4,4,4,4,2.8,0) will without a doubt get rid of 99% of background noise/grain, (I was shocked at how well it works), at the expense of a bit of details lost in the faces..but then again, you also want to keep the grain because that is also part of the DVD..a tough call sometimes.

I'm debating whether to try lumi masking (or something crazy) on a movie I'm currently trying to encode. Even with c3d(0,4,4,4,4,2.8,0), bicubic 0.5 and h263, first pass is still 2.8+ gigs!..and at quant 2 still looks somehwat crappy and blurry.

@iago: are you still using lumoff/unfilter even with MPEG quant?

cjv

Lefungus

1st October 2002, 20:27

Useless post. Can be deleted by moderators. Sorry !

cjv

1st October 2002, 20:33

@lefungus: In avisynth 2.06 (the newest) temporalsoften2() is implemented as temporalsoften().
or
http://users.win.be/dividee/TemporalSoften2.zip

cjv

iago

1st October 2002, 20:47

Originally posted by cjv
@iago: are you still using lumoff/unfilter even with MPEG quant?@cjv

Absolutely! ;) (unless I can saturate the codec with MPEG quantization ;), which seems to be the only alternative to "lumoff=-2 and unfilter(5,5)" trick I have seen so far, with the difference that MPEG quant2 mostly "keeps" the source noise in dark/black areas as well and that is something tolerable/not annoying/or even preferable/ but almost impossible! ;))

regards,
iago

edit: However, some sources that are already sharp enough or that contain some heavier film noise (I'm not sure about that, maybe both) might not require any UnFilter at all and only lumoff=-2 can be the adequate solution.

yaz

2nd October 2002, 09:52

hi all !

i went on my little ugly clips. next i tried to combine filtering & encoding. i used some very basic avs scripts and fast recompress. results :
- icreasing stepwise the 'strength' of denoising has a quite 'strange' effect. light denoising improves quality (i mean the overall look of the encoded clips, special attention to black-blocks, watched on tv) but after awhile it degrades again. the 'mirror-cleaned' clips seem almost as ugly as the original noisy ones.
my guess : complete removal of such heavy noise i have here removes too much detailes & increases too much the compressibility, so the codec starts 'blocking' in the low luma(noise) ranges. indeed, these almost noiseless clips seem on pc rather like noisy cartoons. contours okay, but almost no shades anywhere. this overfiltration effect i call 'cartoonizing'. some people like this effect (i don't) but it's definitely increases blocking.
- filtering chroma has much less effect on quality than that of luma. it seems not only the human eye is less sensitive to chroma noise :-)
- denoising must be in tune with quant type. say, lighter denoise+h263 is (almost) equivalent to stronger denoise+mpeg. compare() shows some difference but i can't see it on tv.
- hq improves not only the look of dark regions but that of the light ones. it's not as surprising. this mode is intended to 'mpegging' the high quant blocks which are not necessarily dark. say, at the same level of denoising the shot of a(n almost) clear blue sky looks much(!) better with hq than with h263 or hvs. it prompts, we should concentrate on low luma-noise regions. i mean the regions where luma changes gradually & (very) slowly. anyway, blocking in light regions is much less visible than in dark ones (esp on tv). but it's there!
- instead of noise removal i tried noise decreasing. as i'm not aware of any avs filter making it, i merged the clean and the orig (noisy) clip. at some combination of merging weight and filter strength the look is almost what i want. however, these factors must be tuned very carfully. so as to make it less tedious, i suggest to examine the noise patterns removed by the filters. it's quite easy and straightforward with vd_avs. i used a script like this.

orig=avisource(clip)
filt=filter(orig,args)
subtract(orig,filt)

replace clip/filter()/args with the appropriate names. it gives back the noise removed, imposed in a flat mid-grey background. for improving contrast, u can add levels at the end.

- btw, levels() is always decreases blocking even if no denoising is applied. i used levels(~16,1,~240,0,255). when using symmetrical expansion, it doesn't decrease the average brightness. i guess, the effect is very similar to 'over-sharpen', as it increses the contrasts all over the frame. anyway, it's more effective with noise-decrease than wout & it improves the look of slightly noisy clips than that of the 'mirror-cleaned'. (so, it may help with dvd-like sources too)

that's what i have so far. pls, comment/criticize as heavy as u want. just to make it clear; i have very noisy sources, i take it intendly + i don't think that any of the aboves would be a general conclusion. we have no 'ultimate solutions' or 'magic sticks' (i think, we'll never have:-)

y

iago

2nd October 2002, 19:35

hello again,

I tested TemporalSoften(1,5,25) with MPEG quantization on a longish chapter of U-Turn which I find representative of the whole movie, (Btw, it is a very hard-to-compress source that has almost all the features to force the limits of a codec and includes a good deal of source grain/noise) just to see its (possible negative) effects due to chroma filtering. I think/agree that TemporalSoften(1,5,0) will do well on "most" sources, filtering luma lightly and increasing compressibility without hurting details, especially when used with MPEG quantization.

When I tried to filter luma a bit heavier with TemporalSoften(1,~10,0), the result really turned out to be crappy. However, TemporalSoften(1,5,25) also worked very well with this source, without decreasing the quality, and it brought some additional increase in compressibility due to chroma filtering too.

Some pre-work and analyzing/testing on our sources with some different filtering and encoding parameters before we start the actual encoding process will always be really useful imho.

I think the most important factor that determines "how and with what parameters" we will apply filtering is nothing but the source itself.

best regards,
iago

Didée

2nd October 2002, 22:40

U-turn really was 'fun' to encode - and the fact it was a TV-cap made things not easier.

More onTopic:
I have to check this one again for darkblocks - I didn´t bother by that time.
BUT I remember EXACTLY that, in few scenes, some large shadows (in high contrast scenes) turned sometimes to greenish, up to next KF. Hah, caught!
(And the raw cap was already deleted when I realized...scream!!)

yaz

3rd October 2002, 08:31

hi all !

@iago:
use tempsoft() only if your source has really temp-noise. in general, the filter must(should) be fitted to the noise intended to remove. problems :
- what to call noise? the noise is an integral part of the picture holding relevant information/details.
- how to characterize it? i mean its distribution/character along (at least) the time&space scale. anyway, i would appreciate other characterizations too.
as i'm not aware of any tool for that,
-i use that small script (outlined in my former post). it doesn't characterize the noise but makes visible that part of the picture the filter would change/remove. i try it for all filter having any sense to use, all alone & in chain. yeah, it's a bit tedious, but it's worth.
-i prefer decreasing the noise instead of its complete removal esp with heavy noise. its complete removal turns the look almost as ugly as the original.

tempsoft(>5) is a heavy weapon. ghosting/edge degradation/tiling/fake chroma/... , some examples i've faced.
try filtering Y&U(V) separately & in this order. in most instances, a proper luma filtering makes quite unnecessary the chroma filter. if needed, a reverse order is better. filter chroma 1st & then luma. a step-by-step approach works much better than doing all at once.
say, compare tempsoft(1,x,y) with tempsoft(1,0,y).tempsoft(1,x,0) ! sometimes there is a significant difference.

@Didée
greenish fake can be caused by many different sources : ivtc problems, side-effect of luma filter, chroma filter scaled improperly, some filter have such kinda side effect (say, earlier versions of resizers did it). make a step-by-step investigation by removing/replacing the filters in the chain. if it does not help, try to change the args one-by-one.

y

Didée

3rd October 2002, 13:07

yaz,

I still have the script, but not the source ...

BUT I am almost sure that this was one of the (very, very) few cases where I used lumi-masking!
Seems lumi-masking and me will never become friends ...

trbarry

3rd October 2002, 18:28

I'm still wondering if this is some sort of coring issue. If the blocks are only a problem on TV's then it may be that some TV's have a problem with displaying values < 16. (blacker than black)

If so then it would be a simple matter to just handle it during output. Optionally ffdshow could have an option to change all values less than 16 (luma, chroma?) to either 0 or 16. I'm not sure which one, probably 16.

But that way clips that really were supposed to be in PC Scale (Full Luma Range) would still display properly on VGA. And it only takes about 3 assembler instructions to force 8 pixels to either 0 or 16 if they are below that.

What I'm really wondering is if due to luma masking or some other processing you can have black (at 16) blocks fall below the threshhold, amplifying small changes. But if that's what is happening it could be easily fixed with a display time option.

It might be an interesting experiment, but one I can't do here.

- Tom

Marc FD

3rd October 2002, 19:53

ffdshow new levels filter is exactly what i needed. :devil:
it's the ultimate solution for TV ouput :cool:

kilg0r3

4th October 2002, 08:31

@marcFD

how do you configure it for tv?

yaz

4th October 2002, 08:57

hi all !

Originally posted by trbarry
I'm still wondering if this is some sort of coring issue. If the blocks are only a problem on TV's then it may be that some TV's have a problem with displaying values < 16. (blacker than black)
If so then it would be a simple matter to just handle it during output. Optionally ffdshow could have an option to change all values less than 16 (luma, chroma?) to either 0 or 16. I'm not sure which one, probably 16. ...

yes/no (again) most(!) tvs have such kinda luma scaling problem. i'm not an expert of it but as some guys explained it (years ago) it's a problem of different spread of scales. the luma scale on (most) tv is narrower(~224) than on pc(~256). dunno whether it still holds true, but i have the same tv as was that time :-)
in add, there can be many other sources. just think of the way the picture gets to the tv screen. player/decoder/tvout driver(s)/card driver(s)/... most of us have also a vcr somewhere along the line.
the tvo card itself can generate such blocks. once we tested 3 different tvo cards and they were quite different as regards look on tv. say, my good old asus is xtremly good in blocking. & there's the ever problem of support. cards having good output (~matrox) may be very weak in software support and reversly, good support for weak output (~asus)

(@marc)
postproc levelling is not the same as making it preproc as the latter changes the bit(rate)/quantizer distribution resulting in quite different encodes.

@both of u
would u be so kind to explain me why your filters (mostly unfilter, focus2) give me exceptionals & illegal calls at some settings. (ok, it's offtopic here, but my question washed away with 0 answer in the avisynth forum)

y

iago

4th October 2002, 15:00

-> If the blocks are only a problem on TV's then it may be that some TV's have a problem with displaying values < 16 ... (trbarry)

I really have big doubts (or I am almost sure) that the black-blocking issue is NOT only a problem on TVs. TV viewing makes them only more visible than they are on PC monitor.

Say you have an encode displaying black-blocking problem on TV. If you watch this same encode on PC monitor, BUT in a (totally) dark environment and with increased brightness/contrast settings of your monitor (I mean with no self-deception ;)), you can see that those blocks are already there in the encode itself!..

I'm pretty sure of this, since with the LightFrame option of my 17" Philips 107T 21 monitor (even not necessarily in a dark viewing environment) I can create almost the same TV-like viewing conditions and observe the black-blocking issue clearly on my monitor too.

It's more related with the encoding process (of dark/black areas) and codecs' behaviour imho.

best regards,
iago

EDIT: @yaz

Can you please modify the below script so that I can use your denoising routine below that you mentioned before in one of your posts:

orig=avisource(clip)
filt=filter(orig,args)
subtract(orig,filt)

(I really wanna test it, but sorry I couldn't exactly understand how to use the parameters you gave - I'm a bit short of time and overly confused/exhausted due to my working conditions these days ;))

Script:

LoadPlugin("C:\MPEG2DEC_PP\MPEG2Dec3.dll")
mpeg2source("D:\TEST\UTURN\CHAPTER18.d2v",cpu=0)
crop(12,16,696,544)
BicubicResize(640,352,0,0.5)

Modified Script with your routine: ?
(using TemporalSoften or UnFilter(-,-) for example)

Thanks in advance,
and best regards,
iago

yaz

7th October 2002, 15:31

Originally posted by iago
-> If the blocks are only a problem on TV's then it may be that some TV's have a problem with displaying values < 16 ... (trbarry)

I really have big doubts (or I am almost sure) that the black-blocking issue is NOT only a problem on TVs. TV viewing makes them only more visible than they are on PC monitor.
...
It's more related with the encoding process (of dark/black areas) and codecs' behaviour imho.

yes, one of its reason is the codec itself. however, if u see that block only under such conditions as u outlined, imho, it's not (so) serious on pc, is it ? :-)

Originally posted by iago
yaz : Can you please modify the below script so that I can use your denoising routine below that you mentioned before in one of your posts:

orig=avisource(clip)
filt=filter(orig,args)
subtract(orig,filt)

(I really wanna test it, but sorry I couldn't exactly understand how to use the parameters you gave - I'm a bit short of time and overly confused/exhausted due to my working conditions these days ;))

Script:

LoadPlugin("C:\MPEG2DEC_PP\MPEG2Dec3.dll")
mpeg2source("D:\TEST\UTURN\CHAPTER18.d2v",cpu=0)
crop(12,16,696,544)
BicubicResize(640,352,0,0.5)

Modified Script with your routine: ?
(using TemporalSoften or UnFilter(-,-) for example)

sorry for my being so ... short :-) try something ...

LoadPlugin("C:\MPEG2DEC_PP\MPEG2Dec3.dll")
orig=mpeg2source("D:\TEST\UTURN\CHAPTER18.d2v",cpu=0)
crop(12,16,696,544)

filt=TemporalSoften(r,t,0)

return subtract(orig,filt).BicubicResize(640,352,0,0.5)

... like this. change r(radius) and t(threshold) to see what tempsoft removes at that certain setting. it is a greyish picture with the noise pattern.

y

ps if u want the 'noise-reductor' i can send it too :-)

iago

7th October 2002, 22:54

Originally posted by yaz
yes, one of its reason is the codec itself [...] imho, it's not (so) serious on pc, is it ? :-)@yaz

You're right, as it (hardly) seems (on monitors), it's not so serious on PC :-). But I just wanted to point out once more that the problem is still there if you don't take special care of it by lumoff and/or sharpening etc, though it is much less visible on PC :-).

Btw, I still couldn't make your modified script work: it gives avisynth errors all the time, and I gave in/up in the end (but not without a fight ;)).

And of course I'd be glad to get your noise-reductor.

regards,
iago

yaz

8th October 2002, 08:37

Originally posted by iago

...
Btw, I still couldn't make your modified script work: it gives avisynth errors all the time, and I gave in/up in the end (but not without a fight ;)).

And of course I'd be glad to get your noise-reductor.

oops. what kinda 'avisynth error' u got ? ouuyeah, i see, u must (i should have) specify clip for tempsoft. use : TemporalSoften(orig,r,t,0), it must work. (f...me:-(( just to make everything clear, u must put numbers instead of 'r' & 't'. this way tempsoft clears only the luma channel.

noise-reduction :
it goes in the same way, but instead of subtracting u should use MeregLuma/Chroma() or Layer(). say, add this ...
MergeLuma(filt,orig,w)
... to the script, insted of subtract(). here, w is the 'weight' governing what extent orig(inal noise) is merged. it varies 0-1.
another possibility is dnr2(). here u can specify a lower threshold for filtering, thus the low level noise is kept back, so there's no need for post-merge. check it with the 'noise-checker' :-)))

y

iago

8th October 2002, 12:15

@yaz

sample script:
--------------------------------------------------------------------
LoadPlugin("C:\PROGRA~1\GORDIA~1\MPEG2Dec3.dll")
orig=mpeg2source("D:\TEST\UTURN\CHAPTER18.d2v",cpu=0).crop(12,16,696,544)
filt=TemporalSoften(orig,1,7,0)
return subtract(orig,filt).BicubicResize(640,352,0,0.5)
--------------------------------------------------------------------

Well, finally I made it work as above :). It's really interesting to see/observe the filtered/removed noise pattern this way, tweaking the radius and luma threshold values of TemporalSoften. Thanks again.

ciao,
iago

yaz

8th October 2002, 16:16

@iago :
(it worx:-))

pls,
- read my reply above
- have a look at the '...psychovisual...' thread in avisynth forum. the tool is coming up what we really need :-) my biggest problem at the moment is that i can't make smoothing dependent gradually on luma. amof, i can't make stepwise/thresholded/any kinda smoothing of that kind:-( luma-masking makes terrific artifacts for me.
--there's a vd filter (kcnr?) making the opposite i want, so maybe it's not impossible.
--the 'conditional' filter in vd may help, but actually i don't know how to make it work in avisynth.

y

JimiK

8th October 2002, 19:48

Sorry I can't contribute to the latest discussion, but there is an observation I made I want to share. I'm using MarcFDs' mpeg2dec3.dll (beta5). With fast=true, it's really lightning fast, but I observed an increase of blocks in dark areas and unfilter doesn't help. With fast=false everything is O.K.
@MarcFD
In the Avisynth forum you wrote, that you think your memory would be the speed limiting factor. I've also got a AthlonXP 1600+, but only 256Mb SDRam and I also get 50fps. So I think CPU optimization will still help.

Best regards,
JimiK

kilg0r3

8th October 2002, 20:37

@Jimik

i think the speed of memeory not the amount is the limiting factor.

-h

8th October 2002, 21:07

I just looked at the source from Nic's site, and things could be sped up by using dequantization assembly from XviD (or libavcodec if you want to convert it to nasm format).

I assume transfer and interpolation assembly is inside the .obj files regarding motion compensation?

-h

JimiK

8th October 2002, 22:07

I'm sorry I abused this thread. I just wanted to address this to MarcFD and just appended it to my "real" post. Just this before I stop abusing the thread: I think MarcFD also thinks that the speed of his memory is the limiting factor. So I told him I've only got SDRam (he's got DDRRam) and the same CPU and I get the same decoding speed as he does.
I'm afraid I did not understand what -h posted. Was it addressed to what I posted or to some of the post above? Was it posted in the right thread?
-h If you were referring to the stuff I posted: I was not talking about Nic's work. Can XviD (Mpeg4) code be reused to decode Mpeg2? I'm sorry if I got it all wrong.

Sorry again,
JimiK

iago

8th October 2002, 22:22

[...] have a look at the '...psychovisual...' thread in avisynth forum. the tool is coming up what we really need :-) my biggest problem at the moment is that i can't make smoothing dependent gradually on luma [...] (yaz)

@yaz,

I've been following the related thread for a while, and I'm really hopeful about the results to come out too. Also, if I'm not mistaken, we're referring to a similar process when I say what we "most urgently" ;) need is an "adaptive/selective denoising" method/filter with the flexibility/option of selecting what areas in the image to denoise depending on the luma levels. (i.e. denoising light/un-dark parts of the image more while keeping the noise as untouched as possible in dark regions.)

@JimiK,

Did you use lumoff (for example "-2") too? And if everything is fine with the default setting (fast=false), it might be a problem occuring due to "fast=true". However, I can't say this for certain since I haven't tried fast=true yet.

best regards,
iago

-h

8th October 2002, 22:57

-h If you were referring to the stuff I posted: I was not talking about Nic's work. Can XviD (Mpeg4) code be reused to decode Mpeg2? I'm sorry if I got it all wrong.

Yeah my bad, but I thought both decoding plugins were based off the same source code, and the source Nic's working with lacks dequantization assembly (which can be nabbed from XviD). Then I started writing and got a bit ahead of myself.

But anyway, a while ago Isibaar replaced libmpeg2's assembly with XviD's and got a 50% speedup. I don't think he hung on to the work, but at least it shows that several parts of libmpeg2 could be significantly faster.

-h

Emp3r0r

8th October 2002, 23:01

I did three encodes last night with nic's new Qpel build and I must say, I can tell a big increase in quality and decrease in filesize. I'd also like to mention that I used lumi masking and (maybe it is these HO fluoresent lights) didn't see any blocking in the dark area's. I'll look again when I can turn the lights off :D

sample script
source=mpeg2source("C:\VOBs\poa\poa.d2v").crop(3,60,-2,-60)
source=source.LanczosResize(640,272)
source=source.Convolution3d(0,3,3,3,3,2.2,0)
return source.freezeframe(0,0,1697)
#size 753301
#credits 162383 172579

JimiK

9th October 2002, 11:09

@Iago
Yes, I did use lumoff=-2. But as cjv in the Avisynth forum mentioned, lumoff seems to have no effect when using fast=true, so I would suggest to keep the hands off this option until it's fixed.
@Emp3r0r
I'm not sure if your good results are due to Nic's new build. After doing some test encodes I'm not sure that QPel is really activated. With constant quant, I had no decrease in filesize and the encoding speed dropped neither. I encoded a part with a dark scene, where I noticed as many blocks as with other builds. Setting lumoff=-2 helped this issue. You did not use that switch, so I wonder why your encodes don't seem to have that problem.

Sincerely,
JimiK

Gaia

9th October 2002, 12:23

QPel is NOT activated in latest NIC build!

Koepi

9th October 2002, 12:48

iago

9th October 2002, 15:26

QPEL is a little buggy in it's current state, Isibaar tries to add a fix today - I could upload a devel (unstable) binary with qpel activated additionally to the stable build then on my site. (Koepi)

@Koepi

That would be really good and I'd be very glad to try it as usual ;). I wonder if it is also possible to activate B-frames in addition to QPel (in your new devel. binary to be released) and use these two "together" for testing purposes?

Thanks again for all your efforts,
iago

Koepi

9th October 2002, 16:18

@iago:

unfortunately, there's no qpel code for bframes added. So you have the choice between qpel and bframes - but they don't work together yet, when activating bframes by setting it >-1, qpel isn't used.
(I for one test bframes now as qpel crashes for me on monsters inc. ;) )

Regards,
Koepi

trbarry

9th October 2002, 16:23

Yeah my bad, but I thought both decoding plugins were based off the same source code, and the source Nic's working with lacks dequantization assembly (which can be nabbed from XviD). Then I started writing and got a bit ahead of myself.

But anyway, a while ago Isibaar replaced libmpeg2's assembly with XviD's and got a 50% speedup. I don't think he hung on to the work, but at least it shows that several parts of libmpeg2 could be significantly faster.

-h -

How similar is dequant between mpeg2 & mpeg4/xvid? And which Xvid functions are you talking about here? I've never looked at that part, in either codec. But you've mentioned it a few times. ;)

- Tom (who mostly doesn't know how dequant works)

-h

9th October 2002, 17:07

How similar is dequant between mpeg2 & mpeg4/xvid? And which Xvid functions are you talking about here? I've never looked at that part, in either codec. But you've mentioned it a few times. ;)

MPEG dequantization is the same between MPEG-2 and MPEG-4 (but not MPEG-1.. I'm just going by code in libavcodec since MPEG-2 specs are hard to come by), as is interpolation, transfer and iDCT. I've no idea where the code is in libmpeg2 (ew), but XviD reads 64 quantized coefficients from the bitstream in mbcoding.c -> get_intra_block()/get_inter_block(), then dequantizes them in quant_mpeg4.c / quantize4_mmx.asm. Also I doubt libmpeg2 is using a lookup table to decode bitpacked coefficients, that could also speed decoding.

Just checked some spec-ish material for MPEG-2, and dequantization is different but could be simulated with XviD's code by increasing the final shift right by 1, or doubling the stored quantizer. Not sure why they changed that.

Also given that libavcodec already has optimizations for edging, dequant, iDCT, transfers, bitstreams and interpolation that are at least as fast as XviD's, I figured it'd be much faster than libmpeg2 already. I think once this avisynth filter is finished (or I get bored with it - much more likely), I'll make a "lavcSource()" plugin that'll read pretty much anything under the sun - MPEG1/2/4, AVI, ASF, WMV, MOV, DV, etc., most of them with audio as well.

-h

Marc FD

9th October 2002, 18:07

>I'll make a "lavcSource()" plugin that'll read pretty much anything under the sun - MPEG1/2/4, AVI, ASF, WMV, MOV, DV, etc., most of them with audio as well.

hehe. i'm a very lazy guy. if you're really gonna do this, say me if i can help ^^.
Because MPEG2Dec in the current state need a complete rewrite to have a more than 10 % speed increase.

Emp3r0r

9th October 2002, 18:16

I turned the lights out and then I saw the blocks. :( Oh well, I'm going to kept trying to get quant 2 encodes to look better. I wish it was easier.

iago

9th October 2002, 18:43

Originally posted by Emp3r0r
I turned the lights out and then I saw the blocks. :( Oh well, I'm going to kept trying to get quant 2 encodes to look better. I wish it was easier. @Emp3r0r,

Or you can still try lumoff=-2 (that will imho be enough in most cases without extra sharpening) with Marc's MPEG2Dec3.dll ;). (Personally, I prefer MPEG -or sometimes ModHQ- quantization with LanczosResize to compensate for the lack of an extra sharpening filter.)

@Koepi,

Thanks for the explanation, I guess -h had also mentioned it somewhere before. I'm looking forward to your new binary with QPel activated.

regards,
iago

JimiK

9th October 2002, 22:08

Emp3r0r,
try lumoff=-2. I encoded some dark scenes without it, using constant quant 2 and the blocks where still there (though I think they were smaller).
@ -h Such an Avisynth filter does not sound boring to me :) But after I'm only frameserving DVDs, I'm happy with the available filters. It's good enough for me when XviD is becoming perfect ;)

Best regards,
JimiK

SansGrip

20th October 2002, 20:06

Bet you thought you'd seen the last of this thread ;)

While my interests lie mainly in DVD/capture -> VCD, this thread sparked my curiosity in finding out exactly what's happening to the signal in terms of range as it passes through my processing chain.

It seems that DVD2AVI (1.76) and Dividee's mpeg2dec, regardless of the "scale" setting -- which I believe applies only to YUY/RGB conversion anyway -- produces luma and chroma with the full range of 0-255.

I don't know whether this is the result of mpeg2dec's conversion to YUV2 or whether this is the actual range used on the DVD. Since DVDs are designed for playback on televisions it would seem strange if they contained levels which fall outside the legal range for TVs (apparently 16-235 for luma, 16-240 for chroma). One possibility is that DVD players do this range compression automatically on playback. Does anyone know more on this?

The reason I've never noticed this in my encodes, it seems, is because TMPGEnc automatically converts from 0-255 to 8-235 (why 8? I don't know, but that's what the tooltip says) if you leave the "Output YUV data sa Basic YCbCr not CCIR601" option unchecked. This would (partially, since the blackest pixels should be 16, not 8) correct the range in the resulting MPEG.

With this in mind I tried adding

Levels(0, 1, 255, 16, 235)

to my Avisynth script and my plugin verifies that no illegal values are being produced. In order to get correct output one must then check the "Output YUV data as Basic YCbCr not CCIR601" option in TMPGEnc, otherwise it will compress the range a second time.

This seems to produce a brightness level that looks right on my monitor (using my ViewSonic's UltraBrite feature, which approximately emulates the brightness level of a TV set). I've yet to try the output on my standalone DVD player.

I know this contributes nothing to the dark blocks problem, but it's something I've been mildly curious about for a while and finally got round to testing :)

(Incidentally, would Levels(2, 1, 255, 16, 235) produce similar output to lumoff=-2, or am I misunderstanding what lumoff does?)

iago

20th October 2002, 20:15

@SansGrip

Welcome man, to one of the most enthralling discussions of the encoding scene ;). I'm really glad one more tester arrived to work on this ugly problem!

regards,
iago

Marc FD

20th October 2002, 20:20

lumoff/lumgain is not the same as levels. it processes in 16 bits mmx and it's finally clipped.

lumoff=-2 will reduce all luma values by 2. it's the theorical equivalent of Levels(2,1,258,0,255) but 258 is impossible.
so levels can't do it because it's a scaler, not a clipper.

Regards,
MarcFD

SansGrip

20th October 2002, 21:18

iago: Welcome man, to one of the most enthralling discussions of the encoding scene ;)

I agree. The non-geek side of me is currently pointing at my geek side and laughing uncontrollably, but I don't care ;)

iago: I'm really glad one more tester arrived to work on this ugly problem!

As the unofficial CIA motto goes: in God we trust, all others we polygraph ;). Theory's great in theory, but I like to try it before I make up my mind.

A month or so ago I actually wrote a filter to add noise back in to low-luma areas. It seemed to remove the blocks but was too noticible to be useful. It was a very simple rand()-based noise, though (adding between -5 and 5 to each pixel in the luma channel, if I remember correctly), so it's probable one would achieve better results using other algorithms. Something like Photoshop's Gaussian noise might be a candidate.

A better solution might be to leave the original noise in the dark areas, if there is any. Another filter I wrote might be useful for this, which mixes two clips based on luma levels. I need to fix it up before releasing it though, and might even generalize it to x number of clips for x different luma ranges.

SansGrip

20th October 2002, 21:22

Marc FD: lumoff=-2 will reduce all luma values by 2. it's the theorical equivalent of Levels(2,1,258,0,255) but 258 is impossible.
so levels can't do it because it's a scaler, not a clipper.

Thanks for the explanation. Incidentally, what is luma gain?

Would tweak() be able to do this? If not, I think I'm going to throw together a filter for those circumstances when I'm not using your mpeg2dec.

iago

20th October 2002, 22:02

@SansGrip

Well, actually such a filter (an "add-noise-to-low-luma-regions" ;) filter) is something I'm very willing to try for a long long time. Hope you continue working on it and even make a beta release perhaps ;). Btw, something else I wonder is: What was its effect on compressibility? Did it cause a big drop or was the effect somewhat negligible?

iago

SansGrip

20th October 2002, 22:46

iago: Well, actually such a filter (an "add-noise-to-low-luma-regions" ;) filter) is something I'm very willing to try for a long long time. Hope you continue working on it and even make a beta release perhaps ;).

I'll lay it on the table and make some incisions and see what happens ;)

Actually what just occurred to me is that it might be better to write a general noise generator, then use it with the luma threshold mixing filter. (I like to take a UNIX approach to stuff I write.)

iago: Btw, something else I wonder is: What was its effect on compressibility? Did it cause a big drop or was the effect somewhat negligible?

I didn't even look. At the time I was making CBR VCDs, so short of looking at the average Q level there's no real way of telling. But now I'm using TMPGEnc's CQ mode so when I've got the filter at least semi-functional I'll run some tests. I'm guessing it won't be pretty, but if you're more concerned about quality than size it might be worth it.

Defiler

20th October 2002, 23:35

When I run into an overly dark DVD (Macross Plus is a good example), I've found myself using positive lumoff values.. up to 20. Am I insane for doing this? I tend to use high bitrates, so "black blocking" is rare. I find the output to be much more pleasing.. but perhaps I should be pushing the levels during postprocessing, rather than during the encode?

SansGrip

21st October 2002, 01:01

When I run into an overly dark DVD (Macross Plus is a good example), I've found myself using positive lumoff values.. up to 20. Am I insane for doing this?

I think the best argument against it is it's "not what the director wanted" but even that's a pretty lame argument considering you're making these for yourself. I say you should do whatever processing makes the output most to your liking :)

@All:

Investigating further with the help of a filter that turns pixels lower than a certain luma threshold bright green, I find that this DVD (American Pie) has *some* pixels that are out of legal range -- perhaps 20 or so in each frame -- particularly in the "shadow" part of titles.

The Universal logo background, however, which is supposed to be pure black, is -- as it should be -- a uniform 16, and so are any fades-to-black in the movie.

For this reason a

Levels(0, 1, 255, 16, 235)

screws things up. Fades-to-black become luma 30, which is way too high. I'd say what we need is a filter to clip instead of scale, like Mark FD (sort of) said :)

(At the risk of stating the obvious, note this 16-235 range limit only applies to movies intended to be viewed on a TV. As trbarry has said, some TVs have a problem with "superblack" and "superwhite".)

Defiler

21st October 2002, 02:37

Originally posted by SansGrip

I think the best argument against it is it's "not what the director wanted" but even that's a pretty lame argument considering you're making these for yourself. I say you should do whatever processing makes the output most to your liking :)I'm sure this isn't what the director intended. The crappy U.S. DVD release is very different from the original Japanese laserdisc. Thanks for the reply, though.

SansGrip

21st October 2002, 02:44

Defiler: I'm sure this isn't what the director intended. The crappy U.S. DVD release is very different from the original Japanese laserdisc.

Funnily enough a few minutes after I posted it I thought about editing to include the line "Unless it's just a bad transfer" ;)

SansGrip

21st October 2002, 03:12

It seems I had a bug in my filter which wasn't showing all the out-of-range pixels in each frame :rolleyes:, and the situation isn't so simple now. I've made some grabs of particularly interesting bits.

Frame 450 (http://www.jungleweb.net/~sansgrip/ap/450.png) clearly shows that while the background is mostly 16, there are a significant number of pixels that fall below the legal range. They seem to be clustered around the stars (that now aren't visible because of all the lime green ;)). Is it just me or do those look like DCT blocks?

Frame 2379 (http://www.jungleweb.net/~sansgrip/ap/2379.png) is very interesting. This is several frames into a cut-to-black, yet we still see what I assume is residue. The previous few frames are identical, and the next few frames are totally black (luma 16).

Frame 12028 (http://www.jungleweb.net/~sansgrip/ap/12028.png) is typical of a "proper" frame from the movie. Here too there's a significant number of out-of-range pixels.

Frame 132008 (http://www.jungleweb.net/~sansgrip/ap/132008.png) shows the clustering of out-of-range pixels around the credit titles. The black background is solid luma 16. To me this almost looks like the output from a mosquito noise detector, but surely that's a coincidence...

My conclusion from this is that the main movie should be range compressed with Levels(0, 1, 255, 16, 235) in order to retain as much detail as possible, while the opening animation and end credits should instead be clipped to 16-235 to retain a true-black background.

I know this all isn't directly relevant to dark blocks, but my thinking is that before I can start considering that problem I need to understand the pixel value range produced by mpeg2dec. Perhaps Marc FD or someone else familiar with the mpeg2dec source could shed some light on this (no pun intended :D)?

trbarry

21st October 2002, 03:40

I know this all isn't directly relevant to dark blocks, but my thinking is that before I can start considering that problem I need to understand the pixel value range produced by mpeg2dec. Perhaps Marc FD or someone else familiar with the mpeg2dec source could shed some light on this (no pun intended )?

Marc may have added other code but most of the MPEG2DEC.dll versions only convert from 4:2:0 to YUY2. This does not change the color space though it will upsample the chroma pixels, making 2 lines out of one. But this upsampling is a vertical interpolation only, not a color conversion.

At least in NTSC DVD's I think that most values below 16 are compression artifacts, sharpening or mosquito noise, and should maybe be set to 16, but MPEG2DEC does not do that.

I am not sure of the luma range as stored on PAL DVD's.

- Tom

SansGrip

21st October 2002, 03:55

trbarry: Marc may have added other code but most of the MPEG2DEC.dll versions only convert from 4:2:0 to YUY2.

This is what I figured.

This does not change the color space though it will upsample the chroma pixels, making 2 lines out of one. But this upsampling is a vertical interpolation only, not a color conversion.

Thanks for the explanation. Now I have a better idea of what's going on (i.e. the out-of-range pixels are there in the source, not introduced by mpeg2dec), but should probably still look at the source code :).

At least in NTSC DVD's I think that most values below 16 are compression artifacts, sharpening or mosquito noise, and should maybe be set to 16, but MPEG2DEC does not do that.

This sentence made me rather excited, and seemed to confirm my suspicion that the out-of-range pixels in those frame grabs look a lot like DCT blocks and mosquito noise. I hesitate to say that this sounds like a foolproof way to remove artifacts from the source material, but... Doesn't it?

I am not sure of the luma range as stored on PAL DVD's.

If what I was reading on the web today is accurate, the luma and chroma ranges specified in CCIR-601 apply to all formats.

Marc FD

21st October 2002, 07:52

i didn't added code. luma filtering is a MPEG2Dec feature (you can play with it in DVD2AVI)

i just fixed the filter, because it was really buggy.
it very easy to port to YUY2 (a pand more ^^)

i can do it if you want ^_^

for the "forbidden values" pixels, i have a very simple idea why they are here :

MPEG2 encoding is done with a range of 0-255 for Y,U and V.
the clean source is free from luma below 16 and above 235

but MPEG2 is lossy. during the encoding, quant/dequant steps change values. worse, the precision of the fDCT is never very big, so fDCT/quant/dequant/iDCT stage is lossy. so the reference frames are modified. and the P_VOPs are more and more altered.

this is only a very subtle loss, so the values should stay between 10 and 240, but the DC quantzation is a factor of noise too.

i think using lumoff = -2 helps too keep the values between 16 and 235, and they are not too much quantised. i don't think it the best solution.

A better way to correct this is too clamp the pixels values to 16-235, then aplying a levels to expand to 0-255, and finally to use a level to regain 16-235 after. using a very fast mmx Avisynth filter, this should be pretty trivial. i could even ask Nic to add a "correct luma for TV" checkbox in his XviD DShow filter to do the dirty job.

Say me if you're interressed. i can code that in 30 min.

just remeber that XviD is lossy and would clamp to 0-255, like in the MPEG-4 specs, not 16-235 (and there is a very simple explaination for that ^^)

trbarry

21st October 2002, 13:54

A better way to correct this is too clamp the pixels values to 16-235, then aplying a levels to expand to 0-255, and finally to use a level to regain 16-235 after. using a very fast mmx Avisynth filter, this should be pretty trivial. i could even ask Nic to add a "correct luma for TV" checkbox in his XviD DShow filter to do the dirty job.

I'm not sure why you would want to expand the values to 0-255 if you were eventually going to compact them back to 16-235. Just because MPEG4 can compress all 256 values doesn't meant there is an advantage to doing so if there are not that many values (validly) present in the source.

What am I missing?

Maybe it would be better to just have pre and post processing that optionally clamp the values to the correct range. And some Avisynth filters (sharpness?) could also maybe force a 16 to a 15 if we were not careful.

Maybe it would be useful to just coopt the TV-Scale option that is already set in DVD2AVI and in MPEG2DEC define it to say that it means to clamp (not compress) the values to TV range. We would then be saying we believe all values outside the NTSC range are in error, and should be fixed.

A similar option could be added to ffdshow for the Xvid output.

If you can really write it in 30 minutes it might be an interesting experiment.

- Tom

SansGrip

21st October 2002, 15:11

trbarry: What am I missing?

Was wondering that myself :)

Maybe it would be better to just have pre and post processing that optionally clamp the values to the correct range. And some Avisynth filters (sharpness?) could also maybe force a 16 to a 15 if we were not careful.

Now I have a filter for highlighting out-of-range values I'm going to try out a bunch of different filters and see which ones can produce them from clean source. I'm almost certain one of mine will ;)

Maybe it would be useful to just coopt the TV-Scale option that is already set in DVD2AVI and in MPEG2DEC define it to say that it means to clamp (not compress) the values to TV range.

Might be an idea if the DVD2AVI menu item were renamed to something more generic, otherwise the mention of RGB could cause confusion.

Could this be a candidate for an internal state flag in Avisynth? Since I usually only produce output for TV, I want the range to stay within 16-235/240. People producing for output to monitor would and should want to use the full 0-255.

Perhaps what might be best (and maybe this is what Marc FD was saying) is for the various producing filters to output the full 0-255 range, all filters to operate in that range, then simply adjust the levels at the end of the chain for TV. Or maybe I'm just rambling ;)

We would then be saying we believe all values outside the NTSC range are in error, and should be fixed.

First I'm pretty sure that CCIR-601 applies this range to all formats, not just NTSC. What I'm not sure about is whether those out-of-range pixels I see in American Pie (and there are a lot of them) are meant to be there or not. I'd hate to lose detail by clipping instead of compressing, if that's the case.

Marc FD

21st October 2002, 15:13

>I'm not sure why you would want to expand the values to 0-255 if you
>were eventually going to compact them back to 16-235. Just because
>MPEG4 can compress all 256 values doesn't meant there is an advantage
>to doing so if there are not that many values (validly) present in the
>source.

i'm not 100% sure but if you begin to use heavy quantizers - because you compress much more in MPEG-4 than in MPEG-2 - you'll end with a _lot_ of values not clamped between 16-235. in fact i'm afraid to see DC values quantized to 0 with high dquants (if lumi masking enabled) or even without lumi-masking, only because of the noise created by the fdct/quant/dequant/idct stage. of course it's only ideas, it need real testing (iago?^^)

>What am I missing?

nothing ^_^
we should try with clamping too.

>Maybe it would be better to just have pre and post processing that
>optionally clamp the values to the correct range. And some Avisynth
>filters (sharpness?) could also maybe force a 16 to a 15 if we were
>not careful.

maybe that's what unfiler(5,5) was correcting ??

>Maybe it would be useful to just coopt the TV-Scale option that is
>already set in DVD2AVI and in MPEG2DEC define it to say that it means
>to clamp (not compress) the values to TV range. We would then be
>saying we believe all values outside the NTSC range are in error, and
>should be fixed.

yes i think clamping is MUCH better than scaling, because if you scale a good value like 16, you end with 18 :( and it creates some false luminosity

>A similar option could be added to ffdshow for the Xvid output.

there is a levels filter in ffdshow. but i think it's a scaler.
we should try with avisynth scripts for playback.

>If you can really write it in 30 minutes it might be an interesting >experiment.

okay i start my watch ^^
it's so easy to clamp values in MMX ^^

Marc FD

21st October 2002, 15:53

this plug is clamping every luma value under 16 to 16
and over 235 to 235

mmx optimised

use Clamp2TVrange()
or Clamp2TVrange(false) to disable MMX (if you have a non-mmx capable cpu ^_^)

it was long because i tested it ^^

(ps : argh i can't attach a dll. need to zip it.... done ^^)

SansGrip

21st October 2002, 16:10

Marc FD: this plug is clamping every luma value under 16 to 16
and over 235 to 235

Good work! Does it do chroma 16-240 too?

iago

21st October 2002, 16:10

Marc,

I'm really anxious to test it! ;) Waiting for the attachment.

iago

Marc FD

21st October 2002, 16:31

>Good work! Does it do chroma 16-240 too?

wait 10 min.

i will add free-values clamping support.

@iago

i don't think it would lead to miracles :(

but if i can help you to test, and because trbarry wanted it, i've made it ^_^

Marc FD

21st October 2002, 16:52

clamp.dll
syntax:
clamp(int "low", int "high", bool "mmx")
default : clamp(16,235,true)

trbarry

21st October 2002, 17:08

okay i start my watch ^^
it's so easy to clamp values in MMX ^^

Marc -

Gong! Nope, I saw the time stamps on you post. Don't want it anymore because it took 40 minutes. :) ;) ;) ;) ;) ;)

You are fast!

in fact i'm afraid to see DC values quantized to 0 with high dquants (if lumi masking enabled) or even without lumi-masking, only because of the noise created by the fdct/quant/dequant/idct stage.

I've never really understood what lumi masking does, though I sometimes use it. But I hope it is not setting ANYTHING to zero in dark areas. That would be a very bad idea in NTSC range video, equivalent to coring.

Eagerly awaiting the results of your Clamping filter.

- Tom

SansGrip

21st October 2002, 18:30

Marc FD: i will add free-values clamping support.

That's a cool idea too, but what we really need is separate ranges for luma and choma. You see, while luma can be from 16-235, chroma can be from 16-240.

SansGrip

21st October 2002, 18:32

trbarry: But I hope it is not setting ANYTHING to zero in dark areas. That would be a very bad idea in NTSC range video, equivalent to coring.

My RangeInfo diagnostic filter (to be posted shortly) shows at least that zero pixel values are definitely produced by mpeg2dec.

@All:

Shouldn't this:

Levels(16, 1, 235, 0, 255)

cause the pixel value range to be expanded from 16-235 to 0-255, with all pixels that were previously 16 now 0?

Emp3r0r

21st October 2002, 18:37

I just tried clamp on matrix R1 and using clamp or not using clamp produced idential outputs when using the default parameters. I took some grabs and did a difference calculation between clamped and unclamped which resulted in pure black (no difference?). I'm not sure I know what I'm doing ;) so I posted the screenshots for futher examination. Shots were grabbed with VirtualDub copy source frame to clipboard.

noclamped.png = default avs script
clamped.png = avsscript + Clamp()
moreclamped = avsscript + Clamp(32,215,true)

The 3rd obviously had differences and resulted in a png size of 85KB smaller.

http://jvance.com/files/frame38171.zip (http://jvance.com/hitme.aspx?url=http://jvance.com/files/frame38171.zip)

Marc FD

21st October 2002, 19:16

>That's a cool idea too, but what we really need is separate ranges for
>luma and choma. You see, while luma can be from 16-235, chroma can be >from 16-240.

okay i'll do that too.
but keep in mind i'm not using this at all ^^

using 0-255 range makes more sense to me ^_^

SansGrip

21st October 2002, 19:19

Here's a little something I whipped together. Synopsis:

---------------------

RangeInfo is a diagnostic filter for Avisynth. Its purpose is to allow fairly detailed inspection of pixel value ranges within a clip. That is to say, it will tell you various pieces of information such as the pixels with the lowest and highest values (for Y, U and V) in the frame and for the clip overall, amongst others.

It can also optionally highlight pixels within a certain range in a very attractive vivid green.

I wrote this filter to investigate the output from various sources to see if it conformed to the CCIR-601 specification that broadcast video be limited to using certain ranges of pixel values. It can also be useful for checking the range of a clip to help determine what levels adjustment might be necessary.

---------------------

See the readme for more info. Source code is included, as are both the release and debug DLLs.

It's unoptimized and probably has bugs, though I've tested it somewhat and it seems to behave. I figure it should help a lot in figuring out a) what's being handed to us by the producing filters, and b) the exact luma range of those dark blocks (see, I told you I'd get back to those eventually ;)).

Oh, and thanks to the author of Info.h for the excellent code, and to Donald Graft for finding it :).

Hope someone finds this useful. Feedback would be greatly appreciated.

Marc FD

21st October 2002, 19:48

attached last clamp.dll

you can now set luma/chroma low/high values you want.

hope it helps.

good luck @all

Cheers,
MarcFD

metallikop

21st October 2002, 22:50

Marc-

could you add this to your website as well?

-mk

trbarry

22nd October 2002, 01:38

But does clamping help at all with the lumi blocks problem? I don't see the problem so can't test it.

Note you can also use an Avisynth script to display an Xvid file to your TV and see if clamping helps there also, at display time.

- Tom

iago

22nd October 2002, 01:45

-> But does clamping help at all with the lumi blocks problem? (trbarry)

In my tests with clamp(), clamp(8,235,true), clamp(16,240,true), etc., unfortunately no.

I still find, for the time being, lumoff=-2 -and UnFilter(-,-) or UnFilter(+,+)- the most successful way to defeat the lumi blocks problem. (And the only other option seems to be using a constant quantizer of 2 with MPEG quantization type, which preserves the noise, and masks the problems in those low-luma areas :p )

regards,
iago

SansGrip

22nd October 2002, 02:39

Iago: I still find, for the time being, lumoff=-2 -and UnFilter(-,-) or UnFilter(+,+)- the most successful way to defeat the lumi blocks problem.

It might be interesting to run the various combinations (e.g. vanilla, lumoff alone, lumoff + unfilter, etc.) with a RangeInfo at the end to see how each one affects the luma range. Perhaps that way we might be a step closer to figuring out why UnFilter helps. Is it a direct result of the sharpening, or is it somehow adjusting the luma (or even chroma) in a subtle way?

Whatever is the case, it's presumably possible to isolate the cause and effect to low-luma areas rather than having to apply processing to the entire image.

yaz

22nd October 2002, 10:04

hi all !

(afais, the thread deviates fairly from the original topic, but still very interesting.)

black-blocking is a complex problem, i.e. it has (many) different sources. correct luma scaling is only one of that. this problem occurs never on pc but only on tv. imho, it's basically a problem of the i/o cards, as these always transform (&thus) rescale the source before outputting. (the cards i know have quite funny "native" i/o formats) in add, some users forget to scale the tvo channel correctly. (if the card&monitor does not support dual output mode, it's pretty hard, anyway.)
the other problem is handling of regions with shallow luma/chroma gradients. in general, handling the low luma noise parts. simply, where the differences are too small to distinguish for the codec. it's an ever problem of all mpeg codecs. some month ago -h explained this, and dropped some simple examples. the point is that the codec(s) will(!) tile this parts, it's inherent.
a third problem (not independent of the aboves) is compressibility. if a scene is compressed "too much" it looks ugly. in general, the most compressible parts are the low luma/chroma (noise) frames. we should check the stat file(s) very thorughly for that but it's quite tedious. at least, i've never been able to complete such work :-) instead, i always check the average effective bits/pixel of the 1st pass. imho, if it's below 0.5 xvid hasn't done its best. (oops ... not the best for me, as it always does its best:-) 10of12 of my last encodes had this problem. &they looked much much better when the effective bits/pixel rate was tortured upon 0.5 (yeah, sometimes it's pretty hard to reach) with a heavy low bitrate rescaling. that's why i always grin when everybody seems to be anxious about loosing compressibility. i got the opposite problem more frequently. (or maybe i just prefer compressible movies:-))

summing up so far, if u(we:-)'re thinking about solution, we can't expect only one tool/setting/solution/... these must be always fitted to the actual film/clip & the system we (plan to) use for watching it.

y

SansGrip

22nd October 2002, 11:40

yaz: black-blocking is a complex problem, i.e. it has (many) different sources. correct luma scaling is only one of that. this problem occurs never on pc but only on tv.

You're right about the complexity. The only reason I introduced the topic of pixel value range is because I wanted to understand how the range was being changed during the encoding process. This way I felt I could get a better understanding of the original luma values of the dark blocks, and how those are affected by various filters.

As for this being a TV-only issue, well, that's true for the luma range but other posts in this thread have confirmed that these dark blocks are also visible on monitors, especially large ones. I would guess this only occurs if the gamma of the source has been increased to compensate for the difference between monitors and TVs. Perhaps in some cases that modification was too powerful.

imho, it's basically a problem of the i/o cards, as these always transform (&thus) rescale the source before outputting.

In my case I also see dark blocks on my TV via (X)VCD discs in my standalone. I guess that counts as an I/O device ;).

in general, the most compressible parts are the low luma/chroma (noise) frames.

Yes, and this thread is also trying to address that, both by adjusting the luma of those areas and, perhaps, by retaining or even adding a little noise.

they looked much much better when the effective bits/pixel rate was tortured upon 0.5

Yes, a higher SansGrip Conjecture quotient results in better quality (you'll only get that if you read the CCE thread) :D.

these must be always fitted to the actual film/clip & the system we (plan to) use for watching it.

I agree, but I still think there's a lot we don't understand about the exact combination of factors that go in to producing these blocks. The fact that professionally made video discs are generally much better in this regard tells me that there's a lot that can be done to reduce them.

iago

22nd October 2002, 12:37

@yaz, SansGrip and all,

Maybe not only the encoding process (codec/mpeg2decX/avs/filters/encoder) but also the whole MPEG-2 decoding/encoding process (DVD2AVI(VFAPI)/codec/mpeg2decX/avs/filters/encoder) considering the source ranges, artifacts, etc. as well should be analyzed (or at least rechecked) thoroughly to understand exactly what's going on regarding the black/lumi-blocking issue.

regards,
iago

edit: And I still think that the amount of "noise" especially in low luma regions plays a very important role while discussing all the above mentioned processes. A "noising" filter to be applied finally after all the pre-processing/filtering/resizing, etc. to adaptively/selectively add noise to dark (low luma) regions might be an important point to focus on, at least to be tried imho ;).

edit2:Maybe Marc does something about it (the add-noise filter) in 30 minutes again, or SansGrip lets us use his own young and fresh one ;)...

SansGrip

22nd October 2002, 19:50

iago: also the whole MPEG-2 decoding/encoding process (DVD2AVI(VFAPI)/codec/mpeg2decX/avs/filters/encoder) considering the source ranges, artifacts, etc. as well should be analyzed (or at least rechecked) thoroughly to understand exactly what's going on regarding the black/lumi-blocking issue.

I absolutely agree. From what Tom and Marc say it seems that at least as far as MPEG-2 is concerned pixels outside CCIR-601 range are artifacts and should be clipped anyway. For me that's a pretty important discovery, as my first instinct was to compress the range instead, which I now know results in incorrect luma.

A "noising" filter to be applied finally after all the pre-processing/filtering/resizing, etc. to adaptively/selectively add noise to dark (low luma) regions might be an important point to focus on, at least to be tried imho ;).

I agree again, except I feel that it would be preferable to retain (legal range) noise from the source instead of adding it artificially. Of course if there's no noise there to begin with, it has to be added. If this even turns out to be a good idea ;).

or SansGrip lets us use his own young and fresh one ;)...

heheh it's not quite there yet. I'm not satisfied with the algorithm I'm using to generate the noise. I've yet to find a balance between eliminating blocks (which does happen with sufficient noise) and retaining a fairly clean-looking image. I'm going to experiment with adding only chroma noise to see if that helps -- it should certainly be less noticible to the eye.

You'll be the first to know when I have something promising ;).

SansGrip

24th October 2002, 21:09

@iago and all:

Finally starting to get somewhere with my noise generating filter. But I have some issues that I'd really like peoples' opinions on.

I decided for testing purposes just to generate one frame of noise and average it with the source frame in dark areas. This "spatial noise" actually appears to work pretty well, but I've not done extensive testing yet.

The question is, would "temporal noise" (i.e. generating new random noise for each frame) be better? Worse? The same? It would certainly be slower.

Right now the threshold works as a hard cut-off. This means the luma borders are pretty noticible. Would it be better to weight it so that the noise is applied less strongly as one approaches the threshold? Is there a canonical algorithm for this or should I make one up? ;)

At the moment I'm only applying luma noise. I think I shall add [luma]b and [chroma]b flags to indicate in which component to apply noise. I'm hoping that chroma noise will have the same effect on the encoder without being so apparent to the eyes...

I'm still using a simple rand()-based algorithm. Is that what Photoshop means by "uniform" noise? Does anyone have any pointers to information on implementing a Gaussian noise algorithm?

Any input would be appreciated. I'm hoping to release something preliminary in a few days.

iago

24th October 2002, 22:49

@SansGrip

Great news! ;) Looking forword to trying your release.

regards,
iago

SansGrip

25th October 2002, 03:25

Ok, here's the first version of my noise generator, MakeSomeNoise. It's not terribly sophisticated but preliminary results using MPEG-1 show significant reduction of DCT blocks -- and, I think, improved visual quality -- obviously at the expense of compressability.

Syntax:

MakeSomeNoise(clip, threshold, lamount, camount, temporal, show)

Default:

MakeSomeNoise(last, 25, 2, 4, true, false)

See the included readme.

While the threshold can be set to only "noise-up" dark areas, I've had good results from doing the whole frame at low settings.

As always, YMMV :)

SansGrip

28th October 2002, 06:56

Now I'm satisfied I know what I'm doing with noise generation (including the Gaussian variety) I'm going to factor MakeSomeNoise out into two different filters, UniformNoise and GaussianNoise. I'm going to remove the luma threshold code and move it into a separate filter, probably called LumaThreshold ;).

I think that would give maximum flexibility.

Any reports on MakeSomeNoise? It hasn't crashed for me yet...

iago

28th October 2002, 12:24

@SansGrip

Well, I'd like to report that so far I haven't got any crashes either! ;) As for the job that MakeSomeNoise does, it seems OK with quant 2 (default values seem to be a bit light for some sources); but as soon as the quantization increases MakeSomeNoise generates very visible artifacts, apparent bleeding, and even extra-blocking especially in low luma regions, the very region it aims at. I really appreciate your work and your attempts, but just wanted to report that problem.

Also, nice to hear that you keep up working on the case ;).

best regards,
iago

wotef

29th October 2002, 20:17

SansGrip

30th October 2002, 21:03

is the idea of this filter to achieve something similar to what poptones described in this thread? (http://arstechnica.infopop.net/OpenTopic/page?a=tpc&s=50009562&f=67909965&m=3890938134&r=5470927074#5470927074)

Precisely! I remember reading something similar a while ago and thought making bitmaps too much hassle. Now I can see the benefits of adding some noise, and I'm glad someone else does too ;).

Filter update: I've completed UniformNoise() and GaussianNoise() (writing the docs now), but they've mutated into new "producing" filters. They just make noise within the parameters you specify, and will need to be combined with Layer() and/or my soon-to-be-written LumaBlend()/ChromaBlend() filters.

iago

30th October 2002, 23:39

@SansGrip

Good news, pal! ;) I'm anxious as usual to try them as soon as you're finished. (Noise is something really precious, no one can underestimate it! ;)) I believe, as a valuable option to improve at least some of or some parts of our encodes, we absolutely need some good "noising" filters as well.

regards,
iago

SansGrip

31st October 2002, 00:46

I'm anxious as usual to try them as soon as you're finished.

See this thread (http://forum.doom9.org/showthread.php?s=&threadid=37003) for the noise generators. You can use these in combination with Layer to selectively add noise until I've finished my blending filters.

I believe, as a valuable option to improve at least some of or some parts of our encodes, we absolutely need some good "noising" filters as well.

I agree. I hope these fit the bill ;).

SansGrip

1st November 2002, 22:42

@iago:

I'd be grateful if you could run some tests with my new AddNoise filter (http://forum.doom9.org/showthread.php?s=&threadid=37135), which is specifically designed to add sensible levels of Gaussian noise to a clip with a view to eliminating DCT blocks.

Thanks :).

iago

1st November 2002, 23:08

@SansGrip

As you also mentioned in the related thread, unfortunately I couldn't get good results in my test encodes with XviD using Noise Generators -> GaussianNoise with Layer and Levels.

Certainly I'll try your new AddNoise filter. However, I'm so confused and tired because of some other tests ;) atm that I actually cannot concentrate on setting the right parameters with it. Considering that I want to add noise to the 0-20 area of a full 0-255 levels range, can you please modify the below sample script:

---------------------------------------
LoadPlugin("C:\FILTERS\mpeg2dec2.dll")
LoadPlugin("C:\FILTERS\AddNoise.dll")

mpeg2source("C:\MOVIE\MOVIE.d2v")
crop(12,12,698,552)
BicubicResize(576,320,0,0.5)

AddNoise(?)
---------------------------------------

Thanks a lot in advance,
regards,
iago

Emp3r0r

1st November 2002, 23:35

I think I mentioned this before but wouldn't the best solution to this problem be some post processing that blurs or adds noise to VERY dark areas? I'd love to have this in ffdshow or xvid.ax

SansGrip

2nd November 2002, 00:04

Considering that I want to add noise to the 0-20 area of a full 0-255 levels range, can you please modify the below sample script

AddNoise is designed to process the entire luma range. I can't think of a way to apply a filter to a certain luma range using existing filters (if anyone knows of a way please tell me), but LumaBlend and ChromaBlend are on my todo list...

If you wanted to try AddNoise anyway (to see if noise helps in the dark areas) I recommend using almost the default settings first:

AddNoise(loop=4) # here ym=um=vm=0, yv=uv=vv=1

If you find the noise too strong you can pull it down in any or all of the components:

AddNoise(yv=0.5, uv=0.2, vv=0.2)

This will reduce the noise to 0.5 in Y and 0.2 in U and V.

If you want more noise, increase the variance parameters.

I'd be very interested to hear your experiences with it. I've had extremely good results so far, at least with MPEG-1. MPEG-4 might be a different beast entirely.

Sorry I have no answer for the selective-noise part of your question right now. I think I'll make LumaBlend and ChromaBlend my next project :).