Fixing NTSC DV 4:1:1 Chroma Problems
 

Is there an available Lanczos resize which runs faster than the AVS internal?

Silly question, I know, if such a thing exists it'll get snarfed in the core quickly. Still...

No, not that I know of.

I'm not aware of one but I did post a few months ago about how Avisynth resize could be speeded up somewhat in the case where both the H & V dimensions were both being resized. This would be done by slightly integrating the H & V resize steps to avoid some conversion steps and temporary results.

But I don't exactly remember the details or where the post is.

- Tom

Benchmarking on a hyperthreading P4 shows lanczosresize is pretty much identical in CPU usage as bicubicresize.

I have spend a _lot_ of time on the resize filters, as they are very widely used, and I cannot see any way of gaining more speed without sacrificing quality. The dynamicly compiled YV12 (horizontal) resizer should be as fast as it gets.

As trbarry suggests a combined horizontal/vertical resizer might speed up things - however there are several things that make me doubt it would be faster.

1) Very bad cache performance - it would require massive lookup tables, and make data access very scattered. We would no longer be able to unpack data and keep it in cache, as we do with the horizontal resizer.

2) Complexity - it will make the algorithm quite complex with a lot of loop overhead.

@Fred: Where did you get the idea that there should be something faster?

@sh0dan

Not that it SHOULD be any faster, I was wondering if it COULD be any faster.

I'm wondering if there's a way to speed this up:

LanczosResize(1440,480).MergeChroma(ConvertToRGB().BilinearResize(180,480).BilinearResize(1440,480).ConvertBackToYUY2()).ConvertToRGB().LanczosResize(720,480).ConvertBackToYUY2().

Sh0dan -

I found the old thread. What I was actually proposing was what we talked about here.

At the time I thought you were partially agreeing with me but that was way back last fall and I know you've done a bunch of optimizing since then. And I haven't looked at the code since.

- Tom

Let's say you've got a 720 x 480 DVD and you crop to 704x464, and resize to 704x304, will this be faster since you're only resizing in one direction? If there's a difference, is it marked or marginal?

The fragment I posted is a modified version of Xesdeeni's correction for NTSC DV chroma bleed. In this particular case, the resizing is done in both directions because the double interpolation all but removes the problem of one chroma value for 4 adjacent pixels. I'm wondering if there is a faster way to do this. My hunch is the biggest return would be optimizing the resizing process. If it's already optimized about as much as it can be, ok.

wrt cropping and resizing DVD source, sure, the fewer processing steps, the faster processing can be completed. Your scenario is quite a bit different from what I'm working on. Good point, though.

BTW, I think my SimpleResize is probably still the fastest by a bit. But it doesn't smooth much.

But if you just want to fuzz up the chroma a bit would it help if I made a version of UnFilter that can operate just on chroma? UnFilter already has separate filter parms for horizontal & vertical and for 4:1:1 I guess you only want a fast horizontal smoother.

- Tom

You know, that just might be fantastic. The problem we're seeing is a grid-like pattern that's annoying as all heck. You can see an example near the top of this page: http://www.geocities.com/fredthompson6 (the area which compares Canopus DV and FlyVideo) and Xesdeeni's script and screenshots are in this thread: http://forum.doom9.org/showthread.php?s=&threadid=57013 and this website: http://www.geocities.com/xesdeeni2001

Xesdeeni's original script worked on luma and had a softening effect. IanD did a tremendous amount of experimentation and we've finally settled on Lanczos chroma-only.

The grid pattern seems to have a vertical aspect as well.

If you are willing to give it a shot, sure, let's see if there's another way. This problem is going to be common to anyone using DV as an NTSC capture codec so any advancement helping to correct this would help quite a few people.

I haven't tested unfilter on this problem. After two weeks of tweaking and testing and comparing notes, my head is still swimming.

I don't have time to study it properly this morning but it occurs to me that interlaced 4:1:1 to interlaced 4:2:0 would be a horrible color converson, easy to screw up. It would lose color info even in the best case. Maybe going first to YUY2 and deinterlacing there might be better?

Anybody know how this data is entering Avisynth and what codecs are converting what? I may also be working with some DVcam data soon so I'll check this later today. I'm not sure a modified UnFilter is the best solution by itself.

- Tom

I'm sorry - time is quite sparse right now - so a quick answer:

@Fred: This is to be expected - sharper resizer = more aliasing. No way around that. It is hard to give a universally best resize algorithm. I usually go for either lanczos (clean, not sharpened material) or neutral bicubic. Never sharp bicubic, as it tends to have a bigger tendency at aliasing.

I hope to get time to look more closely at Xesdeeni's tests. In my head a pointresize downscale and a lanczos/bicubic upscale for chroma would make the most sense - not sure if the pointresizer is precise enough though.

If speed is a problem you might try:

v = Avisource()
chU = utoy(v)
chV = vtoy(v)
# resize these two
YToUV(chU, chV, v)

Instead of mergechroma (which will make the resizer work on the full image size). This should make it between twice and 4 times as fast. D*mn - I'm good at optimizing the resizer! :)

Btw, moved thread to where it belongs.

Fred -

I don't have any DV/4:1:1 files to play with. Could you post about 10 frames with that red ray somewhere?

- Tom

sh0dan -

Hope you are still around for the moment.

IIRC, for Avisynth 2.5 we (you) went to a lot of trouble making sure we preferentially get YV12 input into Avisynth instead of YUY2. For MPEG-2 or MPEG-4 data that is a very good idea.

But for 4:1:1 data that would be a very bad idea. Is Avisynth getting 4:1:1 data as YV12 4:2:0? And does it have to?

- Tom

@sh0dan, thanks, we'll give it a whirl.

@trbarry, IanD has shipped a CD to me with test clips. I doubt it will be here by tomorrow and I leave on Wednesday for a week. I'll ask him if there's a quicker way to get me a few test frames. The challenge is he's in Australia and has a monthly bandwidth cap.

Any improvement in speed/quality would be fantastic. If it can be done for both progressive and interlaced great. Interlaced could always be split into 2 streams for processing then woven back together.

FWIW, there are 3 really good test clips from the Star Wars laserdisc:

1) The opening scene where the Imperial ship shoots a red light beam weapon at the fleeing Rebel ship.
2) Red light reflecting off R2D2's "head" when Princess Leia puts a memory card in him.
3) A blue light ring "stunning" Princes Leia.

Don't remember if he's also sending me one of C3PO and a bunch of colored panel lights.

trbarry: Of course! (How silly of me to miss that one) 4:1:1 should preferable be YUY2, and remain so - at least until chroma is finished.

The functions i describe above does NOT support YUY2, but they should. When I get my new computer up, I will hopefully soon be able to make the additions, so users are able to also extract/merge U/V from YUY2 material.

Not sure I understand here. If Avisource can actually get the 4:1:1 data before any other conversions then it should be easy to make a decent YUY2 version of it. It would only need to be re-arranged and horizontally averaged to make up the missing chroma pixels. 4:2:2 is happily a proper superset of 4:1:1.

I was worried that it was also being converted to YV12, thus again losing half the chroma resolution (down to 1/8), this time in the vertical direction. I would avoid that like the plague, at least until we had a chance to deinterlace it and gain back some vertical chroma resolution.

Of course it's probably going to be eventually converted to 4:2:0 MPEG-4 anyway but we don't want to lose any more before we have to.

I won't do anything now if it can just be fixed in the input routines but it would be very easy to write a very quick optional filter that assumed the chroma had been interpolated on input by nearest neigbor, and fixed it. That is, assume it had not been cropped and that every other horizontal chroma pixel was more or less a duplicate. Ignore those odd (starting from 0) chroma pixels and re-create them from an average of their 2 left and right neighbors, which we assume were the originals. I think this should work. ???

- Tom

I don't think any DV codecs deliver as YV12 - I only know of YUY2 capable codecs. The missing chroma in YUY2 is probably a 50/50 of the two neightboring pixels - if not a copy (which obviously gives lower visual quality).

sh0dan -

I got the impression from some of the links above from some of the links above that some of the 4:1:1 codecs were doing just that, just making copies instead of 50/50 averages. That's what my suggested filter would fix.

And I think you are saying that Avisynth always first sees the data as YUY2, right?

- Tom