Need some critique on a cleanup script, and some help with AddGrain

[HeHateM3]

Hi everyone. I'm doing a DV to DVD conversion of a laserdisc capture, and was looking for some constructive criticism of the script I'm using. The source is excellent (Yotoden remastered box set), and the capture came out looking very nice, but being that I had to use a DV box I'm basically just getting rid of the DV & analog noise, and some nasty dotcrawl.

Also, I had a question about Trbarry's AddGrain filter, which I absolutely LOVE. Is there a YUY2 version, or someone who might be interested enough in it to make one? It's fantastic for helping to eliminate blocking in flat shaded areas, but the required colorspace conversion for the filter is a weakness.

Anyways, here's what I'm using:

AviSource("M:\yotoden1.avi")
DoubleWeave().Trim2(0,-1).SelectOdd()
Reinterpolate411()
Chromashift(C=-2)
Levels(0, 1, 255, 16, 235)
DeDot()
UnFoldFieldsVertical(true)
Faerydust(2)
MipSmooth(preset="AnimeHQ")
ConvertToYV12()
AddGrain(2,0,0)
FoldFieldsVertical(true)
ConvertToYUY2(interlaced=true)

Is there any real difference between using unfoldfields or seperatefields? Unfoldfields seems to be much slower, and I really can't tell the difference.

I'm not sure if AddGrain should be included while the fields are unfolded or after... either way, it would be nice to eliminate the colorspace conversion.

Brian

[stickboy]

Originally posted by HeHateM3
Is there any real difference between using unfoldfields or seperatefields? Unfoldfields seems to be much slower, and I really can't tell the difference.Yes, it's slower. (JDL_UnfoldFieldsVertical calls SeparateFields!)

JDL_UnfoldFieldsVertical is fundamentally different from SeparateFields. Its purpose is to avoid blending together scanlines if you use spatial-temporal smoothers.

See this thread (http://forum.doom9.org/showthread.php?s=&postid=354834#post354834) for a few more details.

[Didée]

Welcome on board, HeHateM3.

Blockbuster might be an even better alternative than AdGrain for your given task, as it will only add noise to flat areas with little or no detail, and leave detailed areas alone (where the noise only will disturb). And it works in YUY2 directly.

About the colorspace conversions ... the less, the better, yes.
However, your encoding application (no matter which one [your script smells like CCE]) will encode to YV12 *anyways*. So, if any given encoder doesn't accept YV12 as input, it's a weakness of the encoder application in the first place.

- Didée

[HeHateM3]

Originally posted by stickboy
Yes, it's slower. (JDL_UnfoldFieldsVertical calls SeparateFields!)

JDL_UnfoldFieldsVertical is fundamentally different from SeparateFields. Its purpose is to avoid blending together scanlines if you use spatial-temporal smoothers.

See this thread (http://forum.doom9.org/showthread.php?s=&postid=354834#post354834) for a few more details.

Not really sure I completely understand all the particulars of that thread, but I had read through it previously when I was trying to figure out how to approach this encode (and it's where I learned about the function in the first place), but for this specific case, would my encode benefit from using unfoldfields?

Brian

[HeHateM3]

Originally posted by Didée
Welcome on board, HeHateM3.

Blockbuster might be an even better alternative than AdGrain for your given task, as it will only add noise to flat areas with little or no detail, and leave detailed areas alone (where the noise only will disturb). And it works in YUY2 directly.

About the colorspace conversions ... the less, the better, yes.
However, your encoding application (no matter which one [your script smells like CCE]) will encode to YV12 *anyways*. So, if any given encoder doesn't accept YV12 as input, it's a weakness of the encoder application in the first place.

- Didée

Yes, I am using CCE... so no YV12. :(

I actually tried to use Blockbuster first, but for some reason I can't get it to work at all. It keeps giving me an error that it needs YV12 or YUY2 input.

Brian

[Mug Funky]

hmm... what colourspace is your DV codec returning? a lot of them return RGB24, though DV is _mostly_ 4:1:1 or 4:2:0 (ntsc or pal), and occasionally 4:2:2 (i forget which DV standard allows this... DVCPRO50 i think).

if you're using plain ordinary DV, then you should be maintaining yv12 through your script. i know ffdshow can do yv12 DV decoding, not sure about the commercial ones (does mainconcept do it?).

[HeHateM3]

Originally posted by Mug Funky
hmm... what colourspace is your DV codec returning? a lot of them return RGB24, though DV is _mostly_ 4:1:1 or 4:2:0 (ntsc or pal), and occasionally 4:2:2 (i forget which DV standard allows this... DVCPRO50 i think).

if you're using plain ordinary DV, then you should be maintaining yv12 through your script. i know ffdshow can do yv12 DV decoding, not sure about the commercial ones (does mainconcept do it?).

I'm using the Canopus DV codec, but I don't know what colorspace it's returning. How can I check?

Brian

[Boulder]

You can check the colorspace by creating a script with two lines, the first one loads the clip and the second one is Info(). Open it in VDub for example and you'll see what's inside;)

[stickboy]

Originally posted by HeHateM3
Not really sure I completely understand all the particulars of that thread, but I had read through it previously when I was trying to figure out how to approach this encode (and it's where I learned about the function in the first place), but for this specific case, would my encode benefit from using unfoldfields?Yes, because you call MipSmooth, a spatial-temporal smoother.

If you have interlaced material and don't do anything, your frames look like:A A A
B B B
A A A
B B B

time --->(Each column represents a different frame.) Applying a spatial smoother will blend A scanlines into B scanlines, which is undesirable. A temporal-smoother is okay, because it blends A scanlines into A scanlines and B scanlines into B scanlines.

If you call SeparateFields, then you get:A B A B A B A B
A B A B A B A BNow you have the opposite problem: spatial-smoothing is okay, because it blends A scanlines with A scanlines, but now temporal-smoothing can blend A with B.

So if you have a function that filters in both the temporal and spatial arenas, neither method is really suitable.

In practice, maybe the effects of blending the wrong scanlines together isn't noticeable enough for you with the filter strengths you use. And, of course, it's possible for a spatial-temporal smoother to know how to handle field-separated video on its own.

[HeHateM3]

Originally posted by stickboy
Yes, because you call MipSmooth, a spatial-temporal smoother.

If you have interlaced material and don't do anything, your frames look like:A A A
B B B
A A A
B B B

time --->(Each column represents a different frame.) Applying a spatial smoother will blend A scanlines into B scanlines, which is undesirable. A temporal-smoother is okay, because it blends A scanlines into A scanlines and B scanlines into B scanlines.

If you call SeparateFields, then you get:A B A B A B A B
A B A B A B A BNow you have the opposite problem: spatial-smoothing is okay, because it blends A scanlines with A scanlines, but now temporal-smoothing can blend A with B.

So if you have a function that filters in both the temporal and spatial arenas, neither method is really suitable.

In practice, maybe the effects of blending the wrong scanlines together isn't noticeable enough for you with the filter strengths you use. And, of course, it's possible for a spatial-temporal smoother to know how to handle field-separated video on its own.

Ok, I think I'm following you, but just let me confirm.

I'm using FaeryDust, which according to the documentation is Temporal only. Am I correct in saying that I can apply that before I unfold or seperate fields?

Mipsmooth, on the other hand, does both temporal *and* spatial smoothing (I thought it only did spatial), so unfolding the fields would be the preference here.

Does that about cover it?

[HeHateM3]

Originally posted by Mug Funky
hmm... what colourspace is your DV codec returning? a lot of them return RGB24, though DV is _mostly_ 4:1:1 or 4:2:0 (ntsc or pal), and occasionally 4:2:2 (i forget which DV standard allows this... DVCPRO50 i think).

if you're using plain ordinary DV, then you should be maintaining yv12 through your script. i know ffdshow can do yv12 DV decoding, not sure about the commercial ones (does mainconcept do it?).

Ok, thanks to Boulder, I can say that the original colorspace for the DV file is YUY2. So to minimize loss it should be kept as YUY2, right?

Brian

[HeHateM3]

Oh, one more question, then hopefully I'll be done. ^_^ Assuming I use unfoldfields, where should the AddGrain filter be placed, or does it matter?

Brian

[Didée]

Originally posted by HeHateM3
Am I correct in saying that I can apply that before I unfold or seperate fields?

You can do so, but it's not at all recommended. The magic of the Dust filters lie in the fact that they're doing motion estimation & compensation (except for SpaceDust). Applying FaeryDust directly to interlaced video will disturb the motion engine noticeably. It won't harm your source to do so, but you'll loose a lot of Dust's efficiency. Better use it on single fields.

- Didée

[Boulder]

FYI,

if you process interlaced video, I'd like to recommend scharfis_brain's method. Basically it is this:


#Load your source here
AssumeTFF() # AssumeBFF() for bottom field first video
KernelBob(7)
#Filter and resize here the same way as when processing progressive material
SeparateFields()
SelectEvery(4,1,2) # for TFF video, use SelectEvery(4,0,3) for BFF
Weave()

function kernelbob(clip a, int th)
{ ord = getparity(a) ? 1 : 0
f=a.kerneldeint(order=ord, sharp=true, twoway=false, threshold=th)
e=a.separatefields().trim(1,0).weave().kerneldeint(order=1-ord, sharp=true, twoway=false, threshold=th)
interleave(f,e).assumeframebased()
}


You'll keep more details this way. The function needs neuron2's KernelDeint.

Here the original thread: http://forum.doom9.org/showthread.php?s=&threadid=74906

[trbarry]

As long as there is no vertical correlation (3rd parm is 0) then it should not matter whether the fields are folded for AddGrain(). But if you were making vertical streaks (some call AddRain() ;) ) then you would want it used on progressive frames.

- Tom