Sharpening of interlaced NTSC DV

[zambelli]

I'm editing a video that consists of two sources - one shot in HD, one shot in DV. They have been scaled down to 720x480 (HD) and deinterlaced (DV), for consistency. Even at the same resolution, the HD-originated footage looks a lot sharper, which is to be expected. My goal is to minimize the apparent difference between HD and DV footage. I'm not expecting miracles - but any bit of help is welcome.

I'm focusing on sharpening the DV footage without going crazy with the filters and inadvertently making it look worse.

Here's my script:

AVISource("FriAct2DV.avi")
video.Reinterpolate411()
video.SeparateFields()
video.Undot()
video.Weave()

FFT3DFilter(OH=16,OW=16,BH=48,BW=48,BT=3,Sigma=1,Sharpen=1,interlaced=true)
LeakKernelDeint(order=0, threshold=10, sharp=true, map=false)


Why FFT3DFilter and not LimitedSharpen? I experimented with both and was happier with FFT3DFilter results. An added bonus is the fact that FFT3DFilter can work natively with interlaced sources - which saves me the trouble of bobbing the video and then re-weaving it.

Why am I sharpening before deinterlacing? Also, I experimented with both orders and noticed that I was getting more detail when sharpening first. This is probably due to the fact that the sharpening is performed with interlaced content in mind.

Comments, suggestions for improvement?

[Pookie]

You're wasting a lot of time doing things that way. Besides, the results of that Undot will be close to invisible with FFT3d running after it.

Have you tried several different deinterlacers? It can make a big difference when you find one you like.

[Boulder]

Comments, suggestions for improvement?
Deinterlace first, then sharpen. You're screwing things up that way, regardless of what you've seen ;)

Try the LimitedSharpen-Soothe -combo for sharpening, you might like it. You could use FFT3DFilter for smoothing first.

[zambelli]

You're wasting a lot of time doing things that way. Besides, the results of that Undot will be close to invisible with FFT3d running after it.

But won't FFT3D be more successful on a denoised source?

Have you tried several different deinterlacers? It can make a big difference when you find one you like.
I have trouble keeping up with all the new deinterlacers. You stop reading doom9 for a month and you end up 5 deinterlacers behind the cutting edge. :)

But seriously... I've tried FieldDeinterlace (Decomb), TomsMoComp and LeakKernelDeint and the last one gives me by far the sharpest image with pretty much no jaggies or interlace artifacts. If you have a particular one in mind, I'm all ears (eyes).

[zambelli]

Deinterlace first, then sharpen. You're screwing things up that way, regardless of what you've seen ;)
I totally agree that deinterlace-sharpen order makes most sense - but the results are exactly opposite. If the end result looks better, how can I be screwing things up?

Try the LimitedSharpen-Soothe -combo for sharpening, you might like it. You could use FFT3DFilter for smoothing first.
What would be the advantage of smoothing with FFT3D first, then sharpening with LimitedSharpen, then sharpening again with Soothe? Wouldn't FFT3D destroy a lot of detail?

[Boulder]

I'd give TDeint a go as well, it's slower than LeakKernelDeint but apparently results in less flickering.

By the way, what's your destination format?

[zambelli]

By the way, what's your destination format?

Destination format is NTSC progressive MPEG-2 DVD. I have plenty of bitrate, up to 8.25Mbps.

[Boulder]

I totally agree that deinterlace-sharpen order makes most sense - but the results are exactly opposite. If the end result looks better, how can I be screwing things up?

In general, it's recommended to deinterlace the video without any preprocessing. There's only a couple of exceptions to the rule, such as GuavaComb and DeDot. I'd suppose it can mess the comb detection badly.

Wait, I thought Soothe did smoothing and FFT3DFilter did sharpening? What would be the advantage of smoothing with FFT3D, then sharpening with LimitedSharpen, then smoothing again with Soothe?
As the Soothe thread says, it can help with certain LimitedSharpen issues, you just need to increase the LS strength to compensate the effect in terms of sharpness.

I actually thought that FFT3DFilter should rather be used for denoising instead of simple sharpening.

[zambelli]

Based on Boulder's suggestions, I've revised my script as follows:

AVISource("FriAct2DV.avi")

# Correct chroma upsampling in MainConcept codec
Reinterpolate411()

# Deinterlace
LeakKernelDeint(order=0, sharp=true)

# Compress luma range in order to preserve all luma info recorded by the camcorder
ColorYUV(levels="PC->TV")

# Convert to YV12 due to MaskTools requirements
ConvertToYV12(interlaced=false)

dull = last

# Sharpen
sharp = dull.LimitedSharpen( ss_x=1.5, ss_y=1.5, strength=200, overshoot=1 )

# Temporal stabilization
Soothe( sharp, dull, 25 )

# Remove dots and noise
Undot()

I did a back-to-back comparison (I prefer Interleave over StackVertical because it allows me to focus my eyes on the same area) between LeakKernelDeint, TomsMoComp and TDeint. TomsMoComp was noticably softer so I easily dismissed it. LeakKernel and TDeint were both were good in deinterlacing performance. I didn't see any combing artifacts left over at all. One difficulty in comparing the two was that they seem to be interpolating information from different fields, so TDeint was always about one field ahead of LeakKernelDeint. However, in the end I chose LeakKernel again because it just seemed a tad sharper.

Here's what it looks like:

Just deinterlaced video (http://www.citizeninsomniac.com/images/LeakKernelDeint.png)
After Soothe (http://www.citizeninsomniac.com/images/Soothe.png)

Is that the most detail I can hope for? Would adding a denoiser help at all?

[zambelli]

On a slightly different subject but along the lines of trying to make the difference between HD and DV source as subtle as possible...

What do you guys think about adding some grain to both videos? Adding grain, while technically adding noise to the image, would hide the relative softness of the DV source, don't you think? I like TrBarry's AddGrain plugin and I'm thinking something like AddGrain(15,0,0) would do the trick.

[mic]

If you're tackling editing in a NLE, and the difference is noticable at cuts/transitions, I think one approach I might look at (depending on scene length etc) is to convert some of the HD to DV, blending HD into the converted HD/DV at transitions to sharpened DV. Might also play with colors/luminance/saturation a little bit to minimize perception of differences. Think grain might help, but would it be better added to edited render?

[zambelli]

If you're tackling editing in a NLE, and the difference is noticable at cuts/transitions, I think one approach I might look at (depending on scene length etc) is to convert some of the HD to DV, blending HD into the converted HD/DV at transitions to sharpened DV.
What exactly do you mean by "convert HD to DV"? And are you suggesting I always do dissolve transitions instead of straight cuts?

[mic]

What exactly do you mean by "convert HD to DV"? And are you suggesting I always do dissolve transitions instead of straight cuts?
Just an idea I might try really, with Vegas I'd import HD that had been converted to DV on a separate track, probably just above the reg HD. At the end of a clip where it transitioned to DV, I'd cut out a few frames of the converted HD/DV, drag it on top of the HD, then set the crossfade so the HD very quickly changd to DV before the actual DV footage. Wouldn't want it visible really, but since our eyes are designed to detect contrast, the actual jump from HD to DV might not be as noticable.

[zambelli]

Just an idea I might try really, with Vegas I'd import HD that had been converted to DV on a separate track, probably just above the reg HD. At the end of a clip where it transitioned to DV, I'd cut out a few frames of the converted HD/DV, drag it on top of the HD, then set the crossfade so the HD very quickly changd to DV before the actual DV footage. Wouldn't want it visible really, but since our eyes are designed to detect contrast, the actual jump from HD to DV might not be as noticable.

I'm still not sure I'm understanding correctly. "Convert HD to DV" is somewhat non-sensical because HD is a resolution and DV is a format/codec. Are you saying "scale HD down to SD resolution" or are you saying "scale down to SD and save with a DV codec"? Sorry if I'm nitpicking, but I'm just trying to make sure we're on the same page.

If the NLE project is 720x480, an actual imported 1280x720 HD video will automatically be scaled down to 720x480 by the NLE when dragged onto the timeline. I'm skipping this step (because I don't trust NLE scaling) and flattening all sources to 720x480 29.97fps progressive before importing them into the NLE (preprocessing, really).

I think I get the general gyst of your idea; I'm just having trouble getting on the same page and figuring out the details of your implementation.

[Pookie]

"What do you guys think about adding some grain to both videos?"

Addgrain is good, as is MplayerNoise, which is in BergFilterCollection.Dll

Sometimes grain and even interlace lines create the illusion of detail and sharpness. You can see what it would look like by saving to an Xvid file, then using FFdshow to play the file back -->choose the "Noise" checkbox in FFdshow to see what it would look like. I know a lot of people prefer to add noise as part of Playback, rather than encode, but you can't do that on a regular DVD player.

[Shinigami-Sama]

what he means is to take your scaled HD stream and covert it to a DV stream and intermix them so you don't notice the transistion as much

[zambelli]

what he means is to take your scaled HD stream and covert it to a DV stream and intermix them so you don't notice the transistion as much

But how would you convert it? Just loading it in VirtualDub and saving it with a DV codec isn't going to affect the quality much, except probably mess up the chroma.