Blocks on "flash frames", pixiedust artefacts or bug? Not consistent.

[FlimsyFeet]

Previewing an avs script with VirtualDub, stepping through frame-by-frame with the right arrow key, when I reach frames containing explosions or flashes I get horrible black blocks all over the place:

http://img217.exs.cx/img217/5462/bad8tb.th.jpg (http://img217.exs.cx/my.php?loc=img217&image=bad8tb.jpg)

(I get the same blocks in my encode, as well).

Now I did a search and I've seen it mentioned that Pixiedust can cause artefacts, but what makes me think it's a more serious bug - if I use alt+left arrow to go back a dozen frames or so, then use alt+right arrow to return to exactly the same frame number as before, this time it's alright:

http://img217.exs.cx/img217/3572/good5bs.th.jpg (http://img217.exs.cx/my.php?loc=img217&image=good5bs.jpg)

That shouldn't happen, should it? If it was pixiedust artefacts, then they would be consistent.

I'm still only a beginner when it comes to scripting, so it's a bit of a mess (the main part of the function is Didée's workaround to the pixiedust cropping problem):
clip1=loop(avisource("d:\capture\thx_trailer.avi"),47,0,0).trim(0,370)
clip2=AviSource("d:\capture\side_1.avi").trim(0,46328).crop(16, 124, 704, 324)
clip3=AviSource("d:\capture\side_2.avi").trim(322,45738).crop(16, 126, 704, 324)
clip4=avisource("d:\capture\side_3.avi").trim(57,43395).crop(16, 126, 704, 324)
clip5=avisource("d:\capture\side_4.avi").trim(249,39290).crop(16, 126, 704, 324)
clipA=clip1.pixiedust(limit=7).limitedsharpen(dest_x=480).addborders(120,0,120,0)
clipB=process(clip2+clip3+clip4+clip5)
clipA+clipB.addborders(8, 72, 8, 72)

function process( clip clp )
{
ox = clp.width
oy = clp.height
xcorr = int( 16. - ox%16 - 4. )
ycorr = int( 8. - oy%8 - 4. )
top = clp.crop( 0, 0,-0, 4).flipvertical()
bot = ycorr > 0
\ ? clp.crop( 0,oy-ycorr,-0,-0).flipvertical()
\ : top
ycorr > 0 ? stackvertical( top, clp, bot)
\ : stackvertical( top, clp)
left = last.crop(0, 0, 4,-0).fliphorizontal()
right = xcorr > 0
\ ? last.crop(ox-xcorr,0,-0,-0).fliphorizontal()
\ : left
xcorr > 0 ? stackhorizontal(left,last,right)
\ : stackhorizontal(left,last)
PixieDust(limit=7)
crop(4,4,-xcorr,-ycorr)
limitedsharpen(dest_y=432)
return last
}

[Didée]

Nice artefacting, really :)

While limit=7 is rather high and prone to produce the typical PixieDust-Blocking, the artefacts you are showing are *not* that kind of "typical" blocking from Dust. In fact it looks more like a RAM problem or something ...

But let's make sure that the function isn't guilty - perhaps it is.

What happens if you, just for testing, crop everything to MOD16 values, and use plain PixieDust(7) directly?

What happens if you leave out LimitedSharpen?

What happens if you place SetMemoryMax(xyz) at the script's beginning, with xyz = 50% of available RAM?

[FlimsyFeet]

Didée, thanks for taking the time to make some suggestions. I will try these out later when I have time.

[FlimsyFeet]

Originally posted by Didée
What happens if you, just for testing, crop everything to MOD16 values, and use plain PixieDust(7) directly?VirtualDub crashes out after the end of clip1 - no error message, just disappears completely.
Originally posted by Didée
What happens if you leave out LimitedSharpen?Still get blocks.
Originally posted by Didée
What happens if you place SetMemoryMax(xyz) at the script's beginning, with xyz = 50% of available RAM? Still get blocks.

:: Edit - it seems that if I seperate out each clip into it's own avs file, I don't get the artefacts. Now, I believe it's possible with the latest CCE to select several input files to be combined into one MPEG file. Let's hope it works!

[FlimsyFeet]

*bump* and, er... help!

I don't understand what's going on!

Now I've got 5 seperate avs files that do the same thing as my original script. Previewing in VirtualDub shows none of these black blocks.

I set up CCE to combine multiple source files to make one MPEG file. I get blocks on the white flash frame (not as bad as the pic above - but still bad), and, only about 25% is encoded (it must have crashed when I left it to encode).

I don't think it's my RAM, some time ago I set my Motherboard to use high performance RAM settings and tested it extensively with Memtest86, it seemed very stable. I can test again to see if anything has degraded, but if it was a RAM problem, wouldn't it be more random and not always on the flash frames?

[Kagura]

Actually, I experienced a very similar problem a while. I had been encoding stuff for a while with a computer and the output seemed fine. Then, when I ran a few projects that were *huge* memory hogs (long filter chain, long project) and outputed as huffyv12, I had corruption in certain frames (random though). The culprit in that case was indeed bad RAM. So, even if your RAM seems good, it can either degrade as you said or was bad originally but was never picked up because of the type of your encode or an error not detectable by memtest.

Then again, try isolating the problem. Process the video without any filtering. Do the blocks appear then? Slowly add filters and see which aspect of the encode creates the problem.

[FlimsyFeet]

Well, I just did one pass with Memtest86. Took about half an hour, and there were no errors reported.

It's very easy to see where the problem is - if I replace PixieDust with, just for example, NoMoSmooth, then I no longer get the blocks.

[Didée]

There is a slight chance the problem derives from the fact you're calling Dust 2 times in the script (one time directly, one time through the function).
Generally that shouldn't be a problem: while one cannot call Dust 2 times on the same parts of a clip, seperate calls on different parts or different clips should be okay. However, in the end, you never know ...

If you indeed can nail the problem down to PixieDust, you might give a try on LRemoveDust (http://forum.doom9.org/showthread.php?s=&postid=620669#post620669) (2nd/3rd function) as a replacement. Drops a little more detail than PixieDust, but never shows blocking, gives even better compressability, and is so much faster.

[FlimsyFeet]

Well I solved the problem (I think), I simply moved the crop after PixieDust. Had a quick look through the encoded output and it seems OK. I know there's probably a speed disadvantage to de-noising the letterbox bars that are later cropped out, but hey, at least it works now!

[Boulder]

You could try adding the parameter align=true to Crop.