Fixing NTSC DV 4:1:1 Chroma Problems

[FredThompson]

I've just uploaded 2 10 frame samples.

http://www.geocities.com/fredthompson6 The site has an hourly bandwidth limit. If you can't get there, wait an hour or so. The site will come back up.

These are interlaced NTSC DV captures by the Canopus ADVC-100 from the Star Wars laserdisc.

@TRBarry, do you think your idea will work for interlaced as well?

[FredThompson]

IanD just sent me two more samples. One is Princess Leia's hand inserting a memeory card into R2D2. There's a red light reflecting on his "head." The other is C3PO next to a bunch of panel lights.

I can't promise to have these up before I leave Wednesday morning but I'll try to get it done.

[trbarry]

Well, I've got the first 2 samples but the only way I've been able to look at them so far is with the DirectShowSource using (probably) the M$ codec. It looks like I have to go codec hunting.

Any suggestions which codec to focus on first?

- Tom

[trbarry]

@TRBarry, do you think your idea will work for interlaced as well?

It would not help whatever mistakes a codec might have made vertically. Theoretically they should not have made any if they went from 4:1:1 to 4:2:2 cause there is nothing really to do vertically. But some of those other threads suggest problems have been seen so I really don't know yet.

If Avisynth asked for it could those codec just pass data in a 4:1:1 format and let Avisynth convert it to 4:2:2 on input? It seems like such a fast simple conversion that I'm sure I don't really understand the problems yet.

- Tom

[cweb]

Is this just an NTSC DV problem or does it occur with PAL DV too?

I can't say I've seen it with PAL DV and Lanczos, at least so far..

[FredThompson]

@trbarry,

The only free DV codecs I know of are the DirectShow and Canopus read-only one. Canopus won't do full-range luma. There is an old one from Panasonic available but I don't know the legal status. MainConcept has a demo, maybe Avid does as well.

@cweb,

PAL DV has a similar problem: http://www.hometheaterhifi.com/volum...ug-4-2001.html

FredThompson

link to freeware video codecs,...sometimes is "out of service"
but is very complete!

http://mirror01.iptelecom.net.ua/~video/

http://www.kmstudio.com.ua/video/codecs.htm

[Xesdeeni]

Fred, the ugly script I posted was really just a "proof-of-concept." It should be relatively simple to create a filter that will do the same thing without all the filter trickery. An actual interpolation would probably be a bit better.

But one problem is that you can't interpolate across 4 chroma samples in YUY2 (4:2:2) color space, because there are only 2 chroma samples, not 4. In the VirtualDub codec I wrote, I used RGB (I had to, VD only supports RGB), and converted it internally to 4:4:4 (actually 4:4:4:4), did the interpolation, and then went back to RGB. Something similar would have to be done in an AVISynth filter, since 4:4:4 isn't supported.

However, you should realize that if the destination is YV12 (4:2:0 for MPEG), then you will undo part of your smoothing when you encode the resulting image. Instead of 1 chroma for every 4 lumas horizontally (4:1:1), you'll have 1 chroma for 4 lumas in a square (4:2:0). But they'll all share the same chroma. Just like the DV codecs, if the decoder you use replicates and doesn't interpolate, you'll get 4 pixels with the same chroma, although in a square instead of a line.

Xesdeeni

[trbarry]

I don't think there is much point in interpolating to get 4:4:4 unless you are actually going to store it that way. It maybe would look better in vdub RGB preview but that's really all that's gained.

But creating the best 4:2:2 and subsequently 4:2:0 is still needed.

And for those helping, I found the Canopus & MainConcept codecs. That should be enough for testing for now.

- Tom

[FredThompson]

@xesdeeni, yes, I understand.

[FredThompson]

@trbarry,

Am I correct that you plan to pull 4:1:1 then interpolate the chroma? IOW, bypass the codec's attempt to do so?

I ask because the codecs all seem to take a different approach and preventing them from "breaking" the chroma in the first place seems like a good idea.

[Xesdeeni]

Fred,

Take a look at the sample pictures mentioned above (http://www.geocities.com/xesdeeni2001/: "4:1:1 Comparison"). The codecs mentioned above, MainConcept and Canopus, both seem to do the same thing. Namely, they replicate the chroma across all 4 pixels. Neither interpolates. Almost anything would be better than that

Xesdeeni

[FredThompson]

I ask because I've heard the Sony DV codec doesn't show this problem but does mess up luma.

[trbarry]

Am I correct that you plan to pull 4:1:1 then interpolate the chroma? IOW, bypass the codec's attempt to do so?

Fred -

This is maybe a last resort, but would certainly work if the codecs will even deliver as 4:1:1. But I don't know yet if that much will be needed anyway. I'm busy selling my house and moving so I don't have as much time for this right now.

However if the main problem is just horizontal pixels being replicated then I'll at least provide a quickie filter that does a re-convert to YUY2 properly since that is very simple to do using either a linear or cubic horizontal interpolation. And no data would have been actually lost in duplicating chroma pixels to YUY2 as long as we knew what happened. (and didn't crop or resize first)

However if the problem is at the time of the original 4:1:1 compression I don't know at all what to do about it. There is an inevitable chorma loss in first decimating 4:1 horizontally for 4:1:1 and then again in a square pattern for 4:2:0 MPEG-2/4.

That would be like trying to put a straight MPEG in a square hole.
<bad pun>

Dunno yet.

- Tom

[trbarry]

Well, it was easier to just write a quickie filter and see what it does. It seems to help, at least for the horizontal dup problem.

I've only tested with the MainConcept decoder so far, with Fred's data. But it does seem that even chroma pixels are just being duplicated in that codec. The new filter will help that by discarding the odd chroma pixels and recreating them as the average of the 2 adjacent even pixels.

It doesn't matter whether the material is interlaced. There are no parms, only Avisynth 2.5 YUY2 supported. Probably use it right after Avisource as:

ReInterpolate411()

Try the new barely tested test version from:

www.trbarry.com/ReInterpolate411.zip (source but no readme included)

As always, let me know if I obviously screwed up somewhere.

- Tom

[FredThompson]

Cool. I'm literally walking out the door and won't be back for a week. I'll ask IanD to test this. Thanks.

[trbarry]

I already found a silly bug that would make things look uglier than needed. If anyone is testing this please download v 0.1.1 from the same loc:

www.trbarry.com/ReInterpolate411.zip

- Tom

[IanD]

Originally posted by trbarry
Well, it was easier to just write a quickie filter and see what it does. It seems to help, at least for the horizontal dup problem.

I've only tested with the MainConcept decoder so far, with Fred's data. But it does seem that even chroma pixels are just being duplicated in that codec. The new filter will help that by discarding the odd chroma pixels and recreating them as the average of the 2 adjacent even pixels.

Tom,

The Canopus and Mainconcept DV decoders appear very similar in that they both produce the chroma bug for 4:1:1.

Have now tried ReInterpolate411() on a Canopus software DV decode of the material I sent to Fred Thompson and it does seem to fix the chroma bug.

However, when comparing the Canopus decode and the reinterpolated version of the Star Wars laser blast space scene as bmp enlarged to 1600% (to make the pixels visible) and flipping between the two, the laser blast seems to move left by 2 pixels even though the white rocket engines remain more or less in the same positions.

I have noticed a similar effect when comparing the Panasonic and Canopus decoders: one will have a slippage of a few pixels compared to the other.

Could this be a result of the different decoders (and your interpolation) applying interpolation in opposite directions (if that makes any sense)?

Anyway, ReInterpolate411() at least removes the obvious chroma bug problem and brings the decode back to a quality similar to the Sony decoder (which doesn't have the chroma bug but which shifts the brightness upwards).

Each decoder seems to do something different to the original DV image.

So far I haven't found a way to view the original DV source without the influence of decoder peculiarities. Even using Directshowsource in Avisynth (to make use of the inbuilt DV decoder in windows as comparison) seems to return only a 320x240 image and I can't find an option to supply the full 720x480.

Ian

[trbarry]

However, when comparing the Canopus decode and the reinterpolated version of the Star Wars laser blast space scene as bmp enlarged to 1600% (to make the pixels visible) and flipping between the two, the laser blast seems to move left by 2 pixels even though the white rocket engines remain more or less in the same positions.

IanD -

I think the Canopus and Main Concept decoders duplicate the left (even) chroma pixels. In YUY2 this effectively moves the chroma image 1 (YUY2) chroma pixel (2 luma pixels) to the right. So hopefully ReInterpolate411 is just putting it back.

- Tom

[trbarry]

So far I haven't found a way to view the original DV source without the influence of decoder peculiarities. Even using Directshowsource in Avisynth (to make use of the inbuilt DV decoder in windows as comparison) seems to return only a 320x240 image and I can't find an option to supply the full 720x480.

I actually don't use DV yet and am not an expert on it. But I'm almost certain I've heard this problem discussed before. Just don't remember where. IIRC, there is something in the DXSDK that let's you get around this somehow but I think the discussion was on AVS HTPC Forum and probably over a year ago.

- Tom