lumi masking

[-h]

-h If you were referring to the stuff I posted: I was not talking about Nic's work. Can XviD (Mpeg4) code be reused to decode Mpeg2? I'm sorry if I got it all wrong.

Yeah my bad, but I thought both decoding plugins were based off the same source code, and the source Nic's working with lacks dequantization assembly (which can be nabbed from XviD). Then I started writing and got a bit ahead of myself.

But anyway, a while ago Isibaar replaced libmpeg2's assembly with XviD's and got a 50% speedup. I don't think he hung on to the work, but at least it shows that several parts of libmpeg2 could be significantly faster.

-h

[Emp3r0r]

I did three encodes last night with nic's new Qpel build and I must say, I can tell a big increase in quality and decrease in filesize. I'd also like to mention that I used lumi masking and (maybe it is these HO fluoresent lights) didn't see any blocking in the dark area's. I'll look again when I can turn the lights off :D

sample script
source=mpeg2source("C:\VOBs\poa\poa.d2v").crop(3,60,-2,-60)
source=source.LanczosResize(640,272)
source=source.Convolution3d(0,3,3,3,3,2.2,0)
return source.freezeframe(0,0,1697)
#size 753301
#credits 162383 172579

[JimiK]

@Iago
Yes, I did use lumoff=-2. But as cjv in the Avisynth forum mentioned, lumoff seems to have no effect when using fast=true, so I would suggest to keep the hands off this option until it's fixed.
@Emp3r0r
I'm not sure if your good results are due to Nic's new build. After doing some test encodes I'm not sure that QPel is really activated. With constant quant, I had no decrease in filesize and the encoding speed dropped neither. I encoded a part with a dark scene, where I noticed as many blocks as with other builds. Setting lumoff=-2 helped this issue. You did not use that switch, so I wonder why your encodes don't seem to have that problem.

Sincerely,
JimiK

[Gaia]

QPel is NOT activated in latest NIC build!

[Koepi]

QPEL is a little buggy in it's current state, Isibaar tries to add a fix today - I could upload a devel (unstable) binary with qpel activated additionally to the stable build then on my site. I'll keep you posted then.

regards,
Koepi

[iago]

QPEL is a little buggy in it's current state, Isibaar tries to add a fix today - I could upload a devel (unstable) binary with qpel activated additionally to the stable build then on my site. (Koepi)

@Koepi

That would be really good and I'd be very glad to try it as usual ;). I wonder if it is also possible to activate B-frames in addition to QPel (in your new devel. binary to be released) and use these two "together" for testing purposes?

Thanks again for all your efforts,
iago

[Koepi]

@iago:

unfortunately, there's no qpel code for bframes added. So you have the choice between qpel and bframes - but they don't work together yet, when activating bframes by setting it >-1, qpel isn't used.
(I for one test bframes now as qpel crashes for me on monsters inc. ;) )

Regards,
Koepi

[trbarry]

Yeah my bad, but I thought both decoding plugins were based off the same source code, and the source Nic's working with lacks dequantization assembly (which can be nabbed from XviD). Then I started writing and got a bit ahead of myself.

But anyway, a while ago Isibaar replaced libmpeg2's assembly with XviD's and got a 50% speedup. I don't think he hung on to the work, but at least it shows that several parts of libmpeg2 could be significantly faster.

-h -

How similar is dequant between mpeg2 & mpeg4/xvid? And which Xvid functions are you talking about here? I've never looked at that part, in either codec. But you've mentioned it a few times. ;)

- Tom (who mostly doesn't know how dequant works)

[-h]

How similar is dequant between mpeg2 & mpeg4/xvid? And which Xvid functions are you talking about here? I've never looked at that part, in either codec. But you've mentioned it a few times. ;)

MPEG dequantization is the same between MPEG-2 and MPEG-4 (but not MPEG-1.. I'm just going by code in libavcodec since MPEG-2 specs are hard to come by), as is interpolation, transfer and iDCT. I've no idea where the code is in libmpeg2 (ew), but XviD reads 64 quantized coefficients from the bitstream in mbcoding.c -> get_intra_block()/get_inter_block(), then dequantizes them in quant_mpeg4.c / quantize4_mmx.asm. Also I doubt libmpeg2 is using a lookup table to decode bitpacked coefficients, that could also speed decoding.

Just checked some spec-ish material for MPEG-2, and dequantization is different but could be simulated with XviD's code by increasing the final shift right by 1, or doubling the stored quantizer. Not sure why they changed that.

Also given that libavcodec already has optimizations for edging, dequant, iDCT, transfers, bitstreams and interpolation that are at least as fast as XviD's, I figured it'd be much faster than libmpeg2 already. I think once this avisynth filter is finished (or I get bored with it - much more likely), I'll make a "lavcSource()" plugin that'll read pretty much anything under the sun - MPEG1/2/4, AVI, ASF, WMV, MOV, DV, etc., most of them with audio as well.

-h

[Marc FD]

>I'll make a "lavcSource()" plugin that'll read pretty much anything under the sun - MPEG1/2/4, AVI, ASF, WMV, MOV, DV, etc., most of them with audio as well.

hehe. i'm a very lazy guy. if you're really gonna do this, say me if i can help ^^.
Because MPEG2Dec in the current state need a complete rewrite to have a more than 10 % speed increase.

[Emp3r0r]

I turned the lights out and then I saw the blocks. :( Oh well, I'm going to kept trying to get quant 2 encodes to look better. I wish it was easier.

[iago]

Originally posted by Emp3r0r
I turned the lights out and then I saw the blocks. :( Oh well, I'm going to kept trying to get quant 2 encodes to look better. I wish it was easier. @Emp3r0r,

Or you can still try lumoff=-2 (that will imho be enough in most cases without extra sharpening) with Marc's MPEG2Dec3.dll ;). (Personally, I prefer MPEG -or sometimes ModHQ- quantization with LanczosResize to compensate for the lack of an extra sharpening filter.)


@Koepi,

Thanks for the explanation, I guess -h had also mentioned it somewhere before. I'm looking forward to your new binary with QPel activated.

regards,
iago

[JimiK]

Emp3r0r,
try lumoff=-2. I encoded some dark scenes without it, using constant quant 2 and the blocks where still there (though I think they were smaller).
@ -h Such an Avisynth filter does not sound boring to me :) But after I'm only frameserving DVDs, I'm happy with the available filters. It's good enough for me when XviD is becoming perfect ;)

Best regards,
JimiK

[SansGrip]

Bet you thought you'd seen the last of this thread ;)

While my interests lie mainly in DVD/capture -> VCD, this thread sparked my curiosity in finding out exactly what's happening to the signal in terms of range as it passes through my processing chain.

It seems that DVD2AVI (1.76) and Dividee's mpeg2dec, regardless of the "scale" setting -- which I believe applies only to YUY/RGB conversion anyway -- produces luma and chroma with the full range of 0-255.

I don't know whether this is the result of mpeg2dec's conversion to YUV2 or whether this is the actual range used on the DVD. Since DVDs are designed for playback on televisions it would seem strange if they contained levels which fall outside the legal range for TVs (apparently 16-235 for luma, 16-240 for chroma). One possibility is that DVD players do this range compression automatically on playback. Does anyone know more on this?

The reason I've never noticed this in my encodes, it seems, is because TMPGEnc automatically converts from 0-255 to 8-235 (why 8? I don't know, but that's what the tooltip says) if you leave the "Output YUV data sa Basic YCbCr not CCIR601" option unchecked. This would (partially, since the blackest pixels should be 16, not 8) correct the range in the resulting MPEG.

With this in mind I tried adding

Levels(0, 1, 255, 16, 235)

to my Avisynth script and my plugin verifies that no illegal values are being produced. In order to get correct output one must then check the "Output YUV data as Basic YCbCr not CCIR601" option in TMPGEnc, otherwise it will compress the range a second time.

This seems to produce a brightness level that looks right on my monitor (using my ViewSonic's UltraBrite feature, which approximately emulates the brightness level of a TV set). I've yet to try the output on my standalone DVD player.

I know this contributes nothing to the dark blocks problem, but it's something I've been mildly curious about for a while and finally got round to testing :)

(Incidentally, would Levels(2, 1, 255, 16, 235) produce similar output to lumoff=-2, or am I misunderstanding what lumoff does?)

[iago]

@SansGrip

Welcome man, to one of the most enthralling discussions of the encoding scene ;). I'm really glad one more tester arrived to work on this ugly problem!

regards,
iago

[Marc FD]

lumoff/lumgain is not the same as levels. it processes in 16 bits mmx and it's finally clipped.

lumoff=-2 will reduce all luma values by 2. it's the theorical equivalent of Levels(2,1,258,0,255) but 258 is impossible.
so levels can't do it because it's a scaler, not a clipper.

Regards,
MarcFD

[SansGrip]

iago: Welcome man, to one of the most enthralling discussions of the encoding scene ;)

I agree. The non-geek side of me is currently pointing at my geek side and laughing uncontrollably, but I don't care ;)

iago: I'm really glad one more tester arrived to work on this ugly problem!

As the unofficial CIA motto goes: in God we trust, all others we polygraph ;). Theory's great in theory, but I like to try it before I make up my mind.

A month or so ago I actually wrote a filter to add noise back in to low-luma areas. It seemed to remove the blocks but was too noticible to be useful. It was a very simple rand()-based noise, though (adding between -5 and 5 to each pixel in the luma channel, if I remember correctly), so it's probable one would achieve better results using other algorithms. Something like Photoshop's Gaussian noise might be a candidate.

A better solution might be to leave the original noise in the dark areas, if there is any. Another filter I wrote might be useful for this, which mixes two clips based on luma levels. I need to fix it up before releasing it though, and might even generalize it to x number of clips for x different luma ranges.

[SansGrip]

Marc FD: lumoff=-2 will reduce all luma values by 2. it's the theorical equivalent of Levels(2,1,258,0,255) but 258 is impossible.
so levels can't do it because it's a scaler, not a clipper.

Thanks for the explanation. Incidentally, what is luma gain?

Would tweak() be able to do this? If not, I think I'm going to throw together a filter for those circumstances when I'm not using your mpeg2dec.

[iago]

@SansGrip

Well, actually such a filter (an "add-noise-to-low-luma-regions" ;) filter) is something I'm very willing to try for a long long time. Hope you continue working on it and even make a beta release perhaps ;). Btw, something else I wonder is: What was its effect on compressibility? Did it cause a big drop or was the effect somewhat negligible?

iago

[SansGrip]

iago: Well, actually such a filter (an "add-noise-to-low-luma-regions" ;) filter) is something I'm very willing to try for a long long time. Hope you continue working on it and even make a beta release perhaps ;).

I'll lay it on the table and make some incisions and see what happens ;)

Actually what just occurred to me is that it might be better to write a general noise generator, then use it with the luma threshold mixing filter. (I like to take a UNIX approach to stuff I write.)

iago: Btw, something else I wonder is: What was its effect on compressibility? Did it cause a big drop or was the effect somewhat negligible?

I didn't even look. At the time I was making CBR VCDs, so short of looking at the average Q level there's no real way of telling. But now I'm using TMPGEnc's CQ mode so when I've got the filter at least semi-functional I'll run some tests. I'm guessing it won't be pretty, but if you're more concerned about quality than size it might be worth it.

[Defiler]

When I run into an overly dark DVD (Macross Plus is a good example), I've found myself using positive lumoff values.. up to 20. Am I insane for doing this? I tend to use high bitrates, so "black blocking" is rare. I find the output to be much more pleasing.. but perhaps I should be pushing the levels during postprocessing, rather than during the encode?

[SansGrip]

When I run into an overly dark DVD (Macross Plus is a good example), I've found myself using positive lumoff values.. up to 20. Am I insane for doing this?

I think the best argument against it is it's "not what the director wanted" but even that's a pretty lame argument considering you're making these for yourself. I say you should do whatever processing makes the output most to your liking :)

@All:

Investigating further with the help of a filter that turns pixels lower than a certain luma threshold bright green, I find that this DVD (American Pie) has *some* pixels that are out of legal range -- perhaps 20 or so in each frame -- particularly in the "shadow" part of titles.

The Universal logo background, however, which is supposed to be pure black, is -- as it should be -- a uniform 16, and so are any fades-to-black in the movie.

For this reason a

Levels(0, 1, 255, 16, 235)

screws things up. Fades-to-black become luma 30, which is way too high. I'd say what we need is a filter to clip instead of scale, like Mark FD (sort of) said :)

(At the risk of stating the obvious, note this 16-235 range limit only applies to movies intended to be viewed on a TV. As trbarry has said, some TVs have a problem with "superblack" and "superwhite".)

[Defiler]

Originally posted by SansGrip

I think the best argument against it is it's "not what the director wanted" but even that's a pretty lame argument considering you're making these for yourself. I say you should do whatever processing makes the output most to your liking :)I'm sure this isn't what the director intended. The crappy U.S. DVD release is very different from the original Japanese laserdisc. Thanks for the reply, though.

[SansGrip]

Defiler: I'm sure this isn't what the director intended. The crappy U.S. DVD release is very different from the original Japanese laserdisc.

Funnily enough a few minutes after I posted it I thought about editing to include the line "Unless it's just a bad transfer" ;)

[SansGrip]

It seems I had a bug in my filter which wasn't showing all the out-of-range pixels in each frame :rolleyes:, and the situation isn't so simple now. I've made some grabs of particularly interesting bits.

Frame 450 (http://www.jungleweb.net/~sansgrip/ap/450.png) clearly shows that while the background is mostly 16, there are a significant number of pixels that fall below the legal range. They seem to be clustered around the stars (that now aren't visible because of all the lime green ;)). Is it just me or do those look like DCT blocks?

Frame 2379 (http://www.jungleweb.net/~sansgrip/ap/2379.png) is very interesting. This is several frames into a cut-to-black, yet we still see what I assume is residue. The previous few frames are identical, and the next few frames are totally black (luma 16).

Frame 12028 (http://www.jungleweb.net/~sansgrip/ap/12028.png) is typical of a "proper" frame from the movie. Here too there's a significant number of out-of-range pixels.

Frame 132008 (http://www.jungleweb.net/~sansgrip/ap/132008.png) shows the clustering of out-of-range pixels around the credit titles. The black background is solid luma 16. To me this almost looks like the output from a mosquito noise detector, but surely that's a coincidence...

My conclusion from this is that the main movie should be range compressed with Levels(0, 1, 255, 16, 235) in order to retain as much detail as possible, while the opening animation and end credits should instead be clipped to 16-235 to retain a true-black background.

I know this all isn't directly relevant to dark blocks, but my thinking is that before I can start considering that problem I need to understand the pixel value range produced by mpeg2dec. Perhaps Marc FD or someone else familiar with the mpeg2dec source could shed some light on this (no pun intended :D)?

[trbarry]

I know this all isn't directly relevant to dark blocks, but my thinking is that before I can start considering that problem I need to understand the pixel value range produced by mpeg2dec. Perhaps Marc FD or someone else familiar with the mpeg2dec source could shed some light on this (no pun intended )?

Marc may have added other code but most of the MPEG2DEC.dll versions only convert from 4:2:0 to YUY2. This does not change the color space though it will upsample the chroma pixels, making 2 lines out of one. But this upsampling is a vertical interpolation only, not a color conversion.

At least in NTSC DVD's I think that most values below 16 are compression artifacts, sharpening or mosquito noise, and should maybe be set to 16, but MPEG2DEC does not do that.

I am not sure of the luma range as stored on PAL DVD's.

- Tom

[SansGrip]

trbarry: Marc may have added other code but most of the MPEG2DEC.dll versions only convert from 4:2:0 to YUY2.

This is what I figured.

This does not change the color space though it will upsample the chroma pixels, making 2 lines out of one. But this upsampling is a vertical interpolation only, not a color conversion.

Thanks for the explanation. Now I have a better idea of what's going on (i.e. the out-of-range pixels are there in the source, not introduced by mpeg2dec), but should probably still look at the source code :).

At least in NTSC DVD's I think that most values below 16 are compression artifacts, sharpening or mosquito noise, and should maybe be set to 16, but MPEG2DEC does not do that.

This sentence made me rather excited, and seemed to confirm my suspicion that the out-of-range pixels in those frame grabs look a lot like DCT blocks and mosquito noise. I hesitate to say that this sounds like a foolproof way to remove artifacts from the source material, but... Doesn't it?

I am not sure of the luma range as stored on PAL DVD's.

If what I was reading on the web today is accurate, the luma and chroma ranges specified in CCIR-601 apply to all formats.

[Marc FD]

i didn't added code. luma filtering is a MPEG2Dec feature (you can play with it in DVD2AVI)

i just fixed the filter, because it was really buggy.
it very easy to port to YUY2 (a pand more ^^)

i can do it if you want ^_^


for the "forbidden values" pixels, i have a very simple idea why they are here :

MPEG2 encoding is done with a range of 0-255 for Y,U and V.
the clean source is free from luma below 16 and above 235

but MPEG2 is lossy. during the encoding, quant/dequant steps change values. worse, the precision of the fDCT is never very big, so fDCT/quant/dequant/iDCT stage is lossy. so the reference frames are modified. and the P_VOPs are more and more altered.

this is only a very subtle loss, so the values should stay between 10 and 240, but the DC quantzation is a factor of noise too.

i think using lumoff = -2 helps too keep the values between 16 and 235, and they are not too much quantised. i don't think it the best solution.

A better way to correct this is too clamp the pixels values to 16-235, then aplying a levels to expand to 0-255, and finally to use a level to regain 16-235 after. using a very fast mmx Avisynth filter, this should be pretty trivial. i could even ask Nic to add a "correct luma for TV" checkbox in his XviD DShow filter to do the dirty job.

Say me if you're interressed. i can code that in 30 min.

just remeber that XviD is lossy and would clamp to 0-255, like in the MPEG-4 specs, not 16-235 (and there is a very simple explaination for that ^^)

[trbarry]

A better way to correct this is too clamp the pixels values to 16-235, then aplying a levels to expand to 0-255, and finally to use a level to regain 16-235 after. using a very fast mmx Avisynth filter, this should be pretty trivial. i could even ask Nic to add a "correct luma for TV" checkbox in his XviD DShow filter to do the dirty job.

I'm not sure why you would want to expand the values to 0-255 if you were eventually going to compact them back to 16-235. Just because MPEG4 can compress all 256 values doesn't meant there is an advantage to doing so if there are not that many values (validly) present in the source.

What am I missing?

Maybe it would be better to just have pre and post processing that optionally clamp the values to the correct range. And some Avisynth filters (sharpness?) could also maybe force a 16 to a 15 if we were not careful.

Maybe it would be useful to just coopt the TV-Scale option that is already set in DVD2AVI and in MPEG2DEC define it to say that it means to clamp (not compress) the values to TV range. We would then be saying we believe all values outside the NTSC range are in error, and should be fixed.

A similar option could be added to ffdshow for the Xvid output.

If you can really write it in 30 minutes it might be an interesting experiment.

- Tom

[SansGrip]

trbarry: What am I missing?

Was wondering that myself :)

Maybe it would be better to just have pre and post processing that optionally clamp the values to the correct range. And some Avisynth filters (sharpness?) could also maybe force a 16 to a 15 if we were not careful.

Now I have a filter for highlighting out-of-range values I'm going to try out a bunch of different filters and see which ones can produce them from clean source. I'm almost certain one of mine will ;)

Maybe it would be useful to just coopt the TV-Scale option that is already set in DVD2AVI and in MPEG2DEC define it to say that it means to clamp (not compress) the values to TV range.

Might be an idea if the DVD2AVI menu item were renamed to something more generic, otherwise the mention of RGB could cause confusion.

Could this be a candidate for an internal state flag in Avisynth? Since I usually only produce output for TV, I want the range to stay within 16-235/240. People producing for output to monitor would and should want to use the full 0-255.

Perhaps what might be best (and maybe this is what Marc FD was saying) is for the various producing filters to output the full 0-255 range, all filters to operate in that range, then simply adjust the levels at the end of the chain for TV. Or maybe I'm just rambling ;)

We would then be saying we believe all values outside the NTSC range are in error, and should be fixed.

First I'm pretty sure that CCIR-601 applies this range to all formats, not just NTSC. What I'm not sure about is whether those out-of-range pixels I see in American Pie (and there are a lot of them) are meant to be there or not. I'd hate to lose detail by clipping instead of compressing, if that's the case.

[Marc FD]

>I'm not sure why you would want to expand the values to 0-255 if you
>were eventually going to compact them back to 16-235. Just because
>MPEG4 can compress all 256 values doesn't meant there is an advantage
>to doing so if there are not that many values (validly) present in the
>source.

i'm not 100% sure but if you begin to use heavy quantizers - because you compress much more in MPEG-4 than in MPEG-2 - you'll end with a _lot_ of values not clamped between 16-235. in fact i'm afraid to see DC values quantized to 0 with high dquants (if lumi masking enabled) or even without lumi-masking, only because of the noise created by the fdct/quant/dequant/idct stage. of course it's only ideas, it need real testing (iago?^^)

>What am I missing?

nothing ^_^
we should try with clamping too.

>Maybe it would be better to just have pre and post processing that
>optionally clamp the values to the correct range. And some Avisynth
>filters (sharpness?) could also maybe force a 16 to a 15 if we were
>not careful.

maybe that's what unfiler(5,5) was correcting ??

>Maybe it would be useful to just coopt the TV-Scale option that is
>already set in DVD2AVI and in MPEG2DEC define it to say that it means
>to clamp (not compress) the values to TV range. We would then be
>saying we believe all values outside the NTSC range are in error, and
>should be fixed.

yes i think clamping is MUCH better than scaling, because if you scale a good value like 16, you end with 18 :( and it creates some false luminosity

>A similar option could be added to ffdshow for the Xvid output.

there is a levels filter in ffdshow. but i think it's a scaler.
we should try with avisynth scripts for playback.

>If you can really write it in 30 minutes it might be an interesting >experiment.

okay i start my watch ^^
it's so easy to clamp values in MMX ^^

[Marc FD]

this plug is clamping every luma value under 16 to 16
and over 235 to 235

mmx optimised

use Clamp2TVrange()
or Clamp2TVrange(false) to disable MMX (if you have a non-mmx capable cpu ^_^)

it was long because i tested it ^^

(ps : argh i can't attach a dll. need to zip it.... done ^^)

[SansGrip]

Marc FD: this plug is clamping every luma value under 16 to 16
and over 235 to 235

Good work! Does it do chroma 16-240 too?

[iago]

Marc,

I'm really anxious to test it! ;) Waiting for the attachment.

iago

[Marc FD]

>Good work! Does it do chroma 16-240 too?

wait 10 min.

i will add free-values clamping support.

@iago

i don't think it would lead to miracles :(

but if i can help you to test, and because trbarry wanted it, i've made it ^_^

[Marc FD]

clamp.dll
syntax:
clamp(int "low", int "high", bool "mmx")
default : clamp(16,235,true)

[trbarry]

okay i start my watch ^^
it's so easy to clamp values in MMX ^^

Marc -

Gong! Nope, I saw the time stamps on you post. Don't want it anymore because it took 40 minutes. :) ;) ;) ;) ;) ;)

You are fast!

in fact i'm afraid to see DC values quantized to 0 with high dquants (if lumi masking enabled) or even without lumi-masking, only because of the noise created by the fdct/quant/dequant/idct stage.

I've never really understood what lumi masking does, though I sometimes use it. But I hope it is not setting ANYTHING to zero in dark areas. That would be a very bad idea in NTSC range video, equivalent to coring.

Eagerly awaiting the results of your Clamping filter.

- Tom

[SansGrip]

Marc FD: i will add free-values clamping support.

That's a cool idea too, but what we really need is separate ranges for luma and choma. You see, while luma can be from 16-235, chroma can be from 16-240.

[SansGrip]

trbarry: But I hope it is not setting ANYTHING to zero in dark areas. That would be a very bad idea in NTSC range video, equivalent to coring.

My RangeInfo diagnostic filter (to be posted shortly) shows at least that zero pixel values are definitely produced by mpeg2dec.

@All:

Shouldn't this:

Levels(16, 1, 235, 0, 255)

cause the pixel value range to be expanded from 16-235 to 0-255, with all pixels that were previously 16 now 0?

[Emp3r0r]

I just tried clamp on matrix R1 and using clamp or not using clamp produced idential outputs when using the default parameters. I took some grabs and did a difference calculation between clamped and unclamped which resulted in pure black (no difference?). I'm not sure I know what I'm doing ;) so I posted the screenshots for futher examination. Shots were grabbed with VirtualDub copy source frame to clipboard.

noclamped.png = default avs script
clamped.png = avsscript + Clamp()
moreclamped = avsscript + Clamp(32,215,true)

The 3rd obviously had differences and resulted in a png size of 85KB smaller.

http://jvance.com/files/frame38171.zip (http://jvance.com/hitme.aspx?url=http://jvance.com/files/frame38171.zip)

[Marc FD]

>That's a cool idea too, but what we really need is separate ranges for
>luma and choma. You see, while luma can be from 16-235, chroma can be >from 16-240.

okay i'll do that too.
but keep in mind i'm not using this at all ^^

using 0-255 range makes more sense to me ^_^

[SansGrip]

Here's a little something I whipped together. Synopsis:

---------------------

RangeInfo is a diagnostic filter for Avisynth. Its purpose is to allow fairly detailed inspection of pixel value ranges within a clip. That is to say, it will tell you various pieces of information such as the pixels with the lowest and highest values (for Y, U and V) in the frame and for the clip overall, amongst others.

It can also optionally highlight pixels within a certain range in a very attractive vivid green.

I wrote this filter to investigate the output from various sources to see if it conformed to the CCIR-601 specification that broadcast video be limited to using certain ranges of pixel values. It can also be useful for checking the range of a clip to help determine what levels adjustment might be necessary.

---------------------

See the readme for more info. Source code is included, as are both the release and debug DLLs.

It's unoptimized and probably has bugs, though I've tested it somewhat and it seems to behave. I figure it should help a lot in figuring out a) what's being handed to us by the producing filters, and b) the exact luma range of those dark blocks (see, I told you I'd get back to those eventually ;)).

Oh, and thanks to the author of Info.h for the excellent code, and to Donald Graft for finding it :).

Hope someone finds this useful. Feedback would be greatly appreciated.

[Marc FD]

attached last clamp.dll

you can now set luma/chroma low/high values you want.

hope it helps.

good luck @all

Cheers,
MarcFD

[metallikop]

Marc-

could you add this to your website as well?

-mk

[trbarry]

But does clamping help at all with the lumi blocks problem? I don't see the problem so can't test it.

Note you can also use an Avisynth script to display an Xvid file to your TV and see if clamping helps there also, at display time.

- Tom

[iago]

-> But does clamping help at all with the lumi blocks problem? (trbarry)

In my tests with clamp(), clamp(8,235,true), clamp(16,240,true), etc., unfortunately no.

I still find, for the time being, lumoff=-2 -and UnFilter(-,-) or UnFilter(+,+)- the most successful way to defeat the lumi blocks problem. (And the only other option seems to be using a constant quantizer of 2 with MPEG quantization type, which preserves the noise, and masks the problems in those low-luma areas :p )

regards,
iago

[SansGrip]

Iago: I still find, for the time being, lumoff=-2 -and UnFilter(-,-) or UnFilter(+,+)- the most successful way to defeat the lumi blocks problem.

It might be interesting to run the various combinations (e.g. vanilla, lumoff alone, lumoff + unfilter, etc.) with a RangeInfo at the end to see how each one affects the luma range. Perhaps that way we might be a step closer to figuring out why UnFilter helps. Is it a direct result of the sharpening, or is it somehow adjusting the luma (or even chroma) in a subtle way?

Whatever is the case, it's presumably possible to isolate the cause and effect to low-luma areas rather than having to apply processing to the entire image.

[yaz]

hi all !

(afais, the thread deviates fairly from the original topic, but still very interesting.)

black-blocking is a complex problem, i.e. it has (many) different sources. correct luma scaling is only one of that. this problem occurs never on pc but only on tv. imho, it's basically a problem of the i/o cards, as these always transform (&thus) rescale the source before outputting. (the cards i know have quite funny "native" i/o formats) in add, some users forget to scale the tvo channel correctly. (if the card&monitor does not support dual output mode, it's pretty hard, anyway.)
the other problem is handling of regions with shallow luma/chroma gradients. in general, handling the low luma noise parts. simply, where the differences are too small to distinguish for the codec. it's an ever problem of all mpeg codecs. some month ago -h explained this, and dropped some simple examples. the point is that the codec(s) will(!) tile this parts, it's inherent.
a third problem (not independent of the aboves) is compressibility. if a scene is compressed "too much" it looks ugly. in general, the most compressible parts are the low luma/chroma (noise) frames. we should check the stat file(s) very thorughly for that but it's quite tedious. at least, i've never been able to complete such work :-) instead, i always check the average effective bits/pixel of the 1st pass. imho, if it's below 0.5 xvid hasn't done its best. (oops ... not the best for me, as it always does its best:-) 10of12 of my last encodes had this problem. &they looked much much better when the effective bits/pixel rate was tortured upon 0.5 (yeah, sometimes it's pretty hard to reach) with a heavy low bitrate rescaling. that's why i always grin when everybody seems to be anxious about loosing compressibility. i got the opposite problem more frequently. (or maybe i just prefer compressible movies:-))

summing up so far, if u(we:-)'re thinking about solution, we can't expect only one tool/setting/solution/... these must be always fitted to the actual film/clip & the system we (plan to) use for watching it.

y

[SansGrip]

yaz: black-blocking is a complex problem, i.e. it has (many) different sources. correct luma scaling is only one of that. this problem occurs never on pc but only on tv.

You're right about the complexity. The only reason I introduced the topic of pixel value range is because I wanted to understand how the range was being changed during the encoding process. This way I felt I could get a better understanding of the original luma values of the dark blocks, and how those are affected by various filters.

As for this being a TV-only issue, well, that's true for the luma range but other posts in this thread have confirmed that these dark blocks are also visible on monitors, especially large ones. I would guess this only occurs if the gamma of the source has been increased to compensate for the difference between monitors and TVs. Perhaps in some cases that modification was too powerful.

imho, it's basically a problem of the i/o cards, as these always transform (&thus) rescale the source before outputting.

In my case I also see dark blocks on my TV via (X)VCD discs in my standalone. I guess that counts as an I/O device ;).

in general, the most compressible parts are the low luma/chroma (noise) frames.

Yes, and this thread is also trying to address that, both by adjusting the luma of those areas and, perhaps, by retaining or even adding a little noise.

they looked much much better when the effective bits/pixel rate was tortured upon 0.5

Yes, a higher SansGrip Conjecture quotient results in better quality (you'll only get that if you read the CCE thread) :D.

these must be always fitted to the actual film/clip & the system we (plan to) use for watching it.

I agree, but I still think there's a lot we don't understand about the exact combination of factors that go in to producing these blocks. The fact that professionally made video discs are generally much better in this regard tells me that there's a lot that can be done to reduce them.

[iago]

@yaz, SansGrip and all,

Maybe not only the encoding process (codec/mpeg2decX/avs/filters/encoder) but also the whole MPEG-2 decoding/encoding process (DVD2AVI(VFAPI)/codec/mpeg2decX/avs/filters/encoder) considering the source ranges, artifacts, etc. as well should be analyzed (or at least rechecked) thoroughly to understand exactly what's going on regarding the black/lumi-blocking issue.

regards,
iago


edit: And I still think that the amount of "noise" especially in low luma regions plays a very important role while discussing all the above mentioned processes. A "noising" filter to be applied finally after all the pre-processing/filtering/resizing, etc. to adaptively/selectively add noise to dark (low luma) regions might be an important point to focus on, at least to be tried imho ;).

edit2:Maybe Marc does something about it (the add-noise filter) in 30 minutes again, or SansGrip lets us use his own young and fresh one ;)...