Pixelation on edges on NTSC Film, I wish to blur it

[dan_joe]

I still consider myself a newbie in all video and audio related stuff, but I wasn't sure about posting over there, so I posted here since this is mainly an AviSynth question.

Anyways, I should explain myself in regards to blurring a pixelated edge. A friend of mine told me of a strange problem with the US LazyTown DVDs. The show is filmed with a green screen in the background. and it seems that whenever a characters head or hair would go in front of this green screen, the edge would be pixelated or blocky; I'm a newbie with the terms too, so I might be using the wrong words. When I finally bought a DVD, I only saw the problem after processing it. I never saw it when I played in my DVD player or when using VLC Media Player (with Deinterlace disabled) or with Media Player Classic anywhere else.

Here's an example of what I'm talking about:

http://img167.imageshack.us/img167/9350/forcedfilmpcscalevdubmoxq5.th.png (http://img167.imageshack.us/my.php?image=forcedfilmpcscalevdubmoxq5.png)

This is an untouched and not resized yet of the Full Frame Aspect Ratio of 4:3. This is what I did to get that screenshot. I used DVD Decrypter to get the full VOBs of this episode out. I used the DGIndex 1.4.8 with Forced Film selected and PC Scale chosen. I'm not sure if this could affect this, but in the Options menu I have these checkmarked:

Correct Field Order
Use Full Paths
Use DirectDraw Overlay

And in Help, I have these checkmarked:

DirectDraw Overlay
VFAPI Plug-in.

Then I did "Save Project" (not any other option), after that I opened the .avs file with VirtualDubMod, went to frame 10299, then I chose Video>'Snapshot source frame'. And just in case here's .avs script without the full path:

DGDecode_mpeg2source("VTS_03_1.d2v",info=3)

Here's the BMP captured with DGIndex and then converted to PNG:

http://img93.imageshack.us/img93/4127/dgindexsamplepcscalebmpav9.th.png (http://img93.imageshack.us/my.php?image=dgindexsamplepcscalebmpav9.png)

The problem is not as present as the first picture I linked, you can see a little pixelation on the leftside of the pink wig.

Finally, my question. How do I get the processed video to look like it does in DGIndex? I have tried Unfilter and it did what it was supposed to do and protect the edges. Same with MSmooth. I tried using the Blur filter in VirtualDubMod and it did blur that strange pixelation but I'll use that as a last resort. Here's the result of the blurring in VirtualDubMod (I had to "Copy Source frame to clipboard" here, pasted it in Adobe Photoshop, then saved as PNG):

http://img90.imageshack.us/img90/3372/forcedfilmpcscalevdubmovb8.th.png (http://img90.imageshack.us/my.php?image=forcedfilmpcscalevdubmovb8.png)

I had to "Copy Source frame to clipboard" here, paste it in Adobe Photoshop, then save as PNG because making a snapshot of the output frame didn't work, I should go and test that out some more though.

Is there a filter or plugin I can use that will detect the pixelated edges and blur it? I did try to search for an answer with keywords like pixelated edges, but I think they were mainly about deinterlacing and jaggy edges. And other searches showed edge protection. Another search showed up with MVTools with an edge mask, but does that involve protecting the edges? Thanks for any help or nudging in the right direction.

EDIT (11-24-2006): Didée pointed out that I didn't finish describing how I got the first screenshot, that's been fixed now. It's strange how I missed that... Also added how I got the third one, which I need to test out some more.

[scharfis_brain]

Wat you see there are the limitations of the YUV 4:2:0 color space (YV12).

You should be happy. It is a fairly crisp source! (That's why you see the stairstepping)

you may blur it using AVISynth:
mergechroma(blur(1))

but the better solution is to choose rgb output in ffdshow and enable 'high quality yuv to rgb conversion' for playback

[wonkey_monkey]

You should be happy. It is a fairly crisp source! (That's why you see the stairstepping)

For some reason I found myself at the Wikipedia page for this show yesterday. It's shot in Iceland in one of the "most advanced HDTV facilities in Europe."

David

[Didée]

you may blur it using AVISynth:
mergechroma(blur(1))
Suboptimal. It's slower than needed, and Blur(1) has this rounding issue (at least in AS 2.56, dunno about 2.57) which will shift all colors by +1 or -1 in some direction...

When going that way, then instead of "mergechroma(blur(1))", use

RemoveGrain(0,11)

Your colors will thank you.


Edit:

dan_joe, from your description it is not fully clear what the difference has been in the way how you did the 1st and the 2nd screenshot. The 2nd one is rather clear: DGindex->SaveBMP, okay. But not the first one: you're mentioning Decrypter and DGIndex, too, but then you don't finish the description how the screenshot actually was done.

In any case, the first screenshot shows _wrong_ upsampling from YV12 to RGB. The 2nd one is correct.
The what/how/why isn't yet clear, until you finish your description. ;)

[dan_joe]

Thanks for the replies, I'm gonna give them all a try so I can see what happens.

Wat you see there are the limitations of the YUV 4:2:0 color space (YV12).
You should be happy. It is a fairly crisp source! (That's why you see the stairstepping)
Thanks for the information, I always wondered what color spaces were all about and this reminded me that I should go and look it up!

For some reason I found myself at the Wikipedia page for this show yesterday. It's shot in Iceland in one of the "most advanced HDTV facilities in Europe."

If you're interested in reading more about that, you can read this PDF article at High Definition magazine (4th link down):
http://www.definitionmagazine.com/issue_pdfs/def20/def20.htm

And here's an html article that has the HD information too: http://www.avid.co.uk/uk/profiles/LazyTown/index.asp

Unfortunately I only understood about one fourth of the HD information. :p

dan_joe, from your description it is not fully clear what the difference has been in the way how you did the 1st and the 2nd screenshot.
Thanks for telling me, I don't know how I missed that. :confused: It's fixed now.

[IanB]

and Blur(1) has this rounding issue (at least in AS 2.56, dunno about 2.57) which will shift all colors by +1 or -1 in some direction...Tell me more.

Which mode (YUY2 or YV12 or Both)?

Do you have a BlankClip's example script to show the issue?

Any posts that discuss this issue?

[Didée]

Kassandro had pointed this out looong time ago already.

I'm not that elegant at creating minimalistic showcases as you are, IanB, sorry;) - but switching between these should show a difference:

http://img180.imageshack.us/img180/9375/mule1lr0.th.png (http://img180.imageshack.us/my.php?image=mule1lr0.png) http://img99.imageshack.us/img99/3414/mule2ot4.th.png (http://img99.imageshack.us/my.php?image=mule2ot4.png) http://img138.imageshack.us/img138/8550/mule3nd4.th.png (http://img138.imageshack.us/my.php?image=mule3nd4.png) http://img91.imageshack.us/img91/5894/mule4qc9.th.png (http://img91.imageshack.us/my.php?image=mule4qc9.png)

In slightly noisy content, it can often be seen that a single "blur(1)" creates "swimming blotches" in flat areas, where "RemoveGrain(11)" appears much smoother.
Both YUY2 and YV12 show the same issue.

Note that blur values around 1.0 show most of this effect, but then blur(1.58) seems rather precise again. Could be that it's related to the translation of the float value into a blur kernel. (?)

I never did any sophisticated tests about this, but surely I don't like what I see there. Fact is that Blur(1) is unnecessarily (and unacceptable) inprecise, and it has been like that since ever.
(That's the reason why I'm using RemoveGrain(11) exclusively in my scripts, never Blur(1). That, and the possibility of specifying which planes to process ;) )

[Chainmax]

I'm glad to see you acknowledged the source of the test image ;) :p.

[IanB]

@Didée,

I am not sure what I am supposed to be looking for. I have the 4 images, I have an ImageSource script to "flash" between them but I see no movement, I only see the blur effect.

I checked Bluring 5 times (I guess you blured 5 times) the original image and it is the same as the respective .png's

I subtract the 3 blur .png's in turn and the Blur(1.58) has more blur so I see edge lines. The RemoveGrain and Blur(1) while different have no edge lines just speckle noise differences. :confused:

Maybe the PNG encode (YUV->RGB) has masked the error?

[Alain2]

Just switching tabs in firefox show me a difference of color.. For instance green is fading with blur(1), whereas it stays the same green with removegrain(11) or blur(1.58); also the donkey seems more redish.. Look at the change in the average luma / chroma values

[IanB]

Okay by looking thru tissue paper to blur my eyes I think I can see what you are talking about. Man this is really subtle!

My first impression was that it is an artifact of gamma, linearly averaging pixels is not really the correct thing to do, but it is fast. Really we should apply inverse gamma, then average, then reapply the gamma. Fudging up a test with levels.blur.blur.blur.blur.blur.levels sort of helps but not completely.

Delving into the MMX code the function is evaluated thusValue is the blur factor
CW is the centre pixel weight
OW is the side pixel weight
L, C, R are the Left, Centre and Right pixles.
MMX is done using 16 bit signed arithmetic.

a = int(32768*pow(2.0, -value)+0.5)
CW = (a+256)>>9
OW = (32768-a+256)>>9
C = (C*CW+32)>>6 + ((L+R)*OW+64)>>7The net effect is the code only has 7 bits of precision. It will need to be rewritten.

The 2 effects combine to cause a noticable level shift.

[scharfis_brain]

I also noticed banding artifacts in areas with slow gradients with Blur() when I wanted to blur the movie 'Ice Age'.

Is it possible to avoid this banding?

[G_M_C]

To come back onto the original problem of pixellated edges ;)

I've cured a similar problem using FFT3DGPU by only setting the Sigma value for high-frequency cleaning (i.e. on edges and sharp transitions).

fft3dgpu(sigma=1,sigma2=0,sigma3=0,sigma4=0, plane=4,precision=2,bw=32,bh=32,ow=16,oh=16)?

The red sigma=1 adjusts the filterstrength for hight freq. noise. Sigma2 ~sigma4 deal with increasingly (or is it decreasingly :D) lower frequency noise.

It might help in curing the problem .... I think ;)

[foxyshadis]

For this you'd only need plane=3, all of the aliasing is in the chroma planes (if you test with mergeluma(blankclip(last)) you won't see it). I'd have never noticed that, I think, if scharfi hadn't pointed it out.

[G_M_C]

For this you'd only need plane=3, all of the aliasing is in the chroma planes (if you test with mergeluma(blankclip(last)) you won't see it). I'd have never noticed that, I think, if scharfi hadn't pointed it out.

Hmmm, if that is is true, than i've got a question. Would it make a difference when I use plane=4, in stead of plane=3 (aside from speed differences) ?

[IanB]

As a quick workaround the next (RC-2) release will have a blur(mmx=false) option to force use of the slower but more accurate C++ code. I have commited these changes to CVS.