Triggered LimitedSharpen usage.

[naz69]

Hi,

I'm trying to backup Independence Day Special Edition to xvid. My problem is that the material consists of intelaced scenes and not interlaced ones at the same time.
I've found the solution in this (http://forum.doom9.org/showthread.php?t=28900) thread.

After using telecide(post=0) movie comes out nicely compressed and with no interlacing at all. The only thing is that the parts which were oryginally interlaced (the ones telecide() was active on) come out less sharp than the normal ones (oryginally noninterlaced).

So my question is, If anybody here could recommend me a script in which the use of LimitedSharpen is triggered by the use of telecide()? so i could achieve an equal like level of sharpnes in my xvid.

[Mug Funky]

could you maybe post your script? the one in the other thread has a "reduceby2()" call in it, which will destroy any interlacing. if you're running telecide after this, i can imagine you're getting blur in motion. limitedsharpen wouldn't help on ghosty motion anyway, unless the movement is around 1-2 pixels per frame.

[naz69]

Here you go. Mine doesnt use reduceby2()
import("c:\limited.avs")
LoadPlugin("C:\PROGRA~1\GORDIA~1\DGMPGDec\DGDecode.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\MaskTools.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\decomb.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\hqdn3d.dll")
mpeg2source("C:\ID4xvid\projekt.d2v")
Telecide(post=0)
crop(6,76,708,424)
LanczosResize(672,288)
hqdn3d(1)
LimitedSharpen(ss_x=1.0,ss_y=1.0,Smode=4,strength=70)

[Mug Funky]

hmm. that is a little confusing... i wonder why certain frames are coming out blurry. unless it's a pretty poor DVD, it shouldn't be doing that.

maybe try knock everything out of the script (comment lines out with a # at the beginning) and see if the problem is still there.

if the problem's not visible, then re-introduce the filters 1 line at a time until you see it again.

if the problem is there when there's only an mpegsource("blah.d2v"), then your video stream is buggered in some way (post a clip if that's the case - the latest versions of DGindex can cut chunks out for you).

[naz69]

Some frames are coming out less sharp (blury) compared to the other because they come from deinterlaced parts and other are from normal video, and my problem is to set the sharpening on the scenes that are deinterlaced and those are placed randomly

[Didée]

It's probably not related, but I recall that here in Germany, up to now ALL airings of Independance Day have been fieldblended crappola. (Didn't bother yet to get any DVD in my hands, though.)

naz69, the easiest way would be to post a short VOB sample that shows the problem. Better have something to play with than poking in the blue ...

[naz69]

Interlaced VOB sample (http://www.jotem.net/grim/id4int.vob)
Normal VOB sample (http://www.jotem.net/grim/id4nor.vob)

Hope that helps a little.

[naz69]

I could also try to find and post one file where normal video changes to interlaced if that would be required.

[Didée]

Generally, this seems like a normal dynamic phase shift. But: the phase shifted parts, which you say they were blurry, are not really "blurry", but seem to only have half vertical resolution. After field matching, you can see that each frame consists of line pairs that are almost identical. Look closely, this is a zoom into a fieldmatched frame:

http://img470.imageshack.us/img470/9065/ipfieldmatched5gg.th.png (http://img470.imageshack.us/my.php?image=ipfieldmatched5gg.png)

Chunky, isn't it. And moreover, you should not go ahead and just sharpen this chunkyness, as it would only make the image worse.
First, the affected parts should be smartly re-sampled. There are several possible methods, and this is where our interlacing experts should jump in (I am not, definetly). My quick try was to smartbob the stream and blend adjacent frames together:

tdeint(mode=1,type=1,mthreshL=0,mthreshC=0).deleteframe(0)
merge(last.selecteven(),last.selectodd(),0.5)
Result:

http://img353.imageshack.us/img353/9616/ipbobmerged9nn.th.png (http://img353.imageshack.us/my.php?image=ipbobmerged9nn.png)

Whether or not this needs further sharpening is a matter of taste.

Lastly a method is needed to process only the fieldshifted parts like this, leaving the not phaseshifted parts alone. I could put something ugly together that would somehow work, but the most simple and elegant solutions probably would come from scharfis_brain or Mug Funky (these are said to even dream interlaced).

[naz69]

Thanks for help and good explanation Didée, guess I'll have to wait for the solution from one of the other experts on how to make this work/look good and most important to me, how to make the overall movie quality equal on all parts.

[foxyshadis]

Is that the actuall size of the video, not resized? Yeesh, it looks like a cinepak or ms video 1 encode! In which case I found this to be very effective on cinepak's 2x2 blocks:

eedi2().turnleft().eedi2().turnright().lanczosresize(width,height)

It'll be slow as a mofo, especially if used on the entire video and not just bad frames. Of course you could always throw motion compensation at it too. Combine the two if you feel like making your computer hate you for a few days. :p

[naz69]

Thx, but the screens posted by Didée are just the zoomed in parts of the frames. The size of the video is the native pal dvd resolution.

[foxyshadis]

Well, in that case, you could still probably use eedi2 and resize without turnleft/right to reconstruct just the vertical. (Resize after works better than resize before, even though it seems like it should be the other way around.) Hopefully it wouldn't look any worse in the rest of the video. Not sure whether combining it with tdeint would also improve anything in this case.

[naz69]

So I understand it would have to be sth like this from what you're saying:
eedi2().lanczosresize(width,height)
any idea how to not apply that to the rest of normal, noninterlaced video so it doesnt get hurt ?

ps. thx for help :)

[Didée]

Lately, Mug Funky had posted a rather simple script function for phaseshift detection. Using its method, and porting your original processing as close as possible, we get

mpeg2source("X:\yoursource.d2v")

source = crop(0,72,720,432)
swapped = source.doubleweave()

bobblend = source.tdeint(mode=1,type=3,mthreshL=0,mthreshC=0).deleteframe(0)
# bobblend = swapped.EEDI2(field=-2) # alternative to the line above
bobblend = merge(bobblend.selecteven(),bobblend.selectodd(),0.5)

swapped = swapped.selectodd()

global cmetric = source .mt_convolution(horizontal="1",vertical="-1 2 -1")
global smetric = swapped.mt_convolution(horizontal="1",vertical="-1 2 -1")

global clean = source .LanczosResize(672,288, 6,4,708,424) .hqdn3d(1)
\ .LimitedSharpen(ss_x=1.0,ss_y=1.0,Smode=4,strength=70) # normal strength for normal parts
global shifted = bobblend.LanczosResize(672,288, 6,4,708,424) .hqdn3d(1)
\ .LimitedSharpen(ss_x=1.0,ss_y=1.0,Smode=4,strength=140) # double strength for phaseshifted parts

scriptclip(clean,"cmetric.averageluma > smetric.averageluma? shifted : clean")
This is without any warranty: it's office time, and the script is just typed with *zero* testing.

The method of resampling still is to discuss, of course. However I had tried "simple" EEDI2 (i.e. upsampling one field only, instead of upsampling both fields + merging the results), and found that this method did not get rid of e.g. the banding in the shadows on the walls. Only processing both fields plus merging seems to deliver a smooth result here. A smooth base is necessary, because if you start out with "ridges in the hillside", sharpening will even enhance these.

But keep in mind all of this is "interlacing", and so I've no clue what I am doing here ... :D


edit: put in EEDI2 processing as an alternative. But either way it could be that the phaseshifted parts actually are "off" by one frame - please check.
Also, MaskTools v2 is needed.

[naz69]

Thx, will try that and report the resaults.

[naz69]

Scirpt worked pretty well, but it introuced this artifact in the places which were deinterlaced should i crop more from the top or did I change something wrong in the scirpt couse I altered it a little.
http://img365.imageshack.us/img365/1315/ssid42ca.jpg
import("c:\limited.avs")
import("c:\dehalo.avs")
LoadPlugin("C:\PROGRA~1\GORDIA~1\DGMPGDec\DGDecode.dll")

LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\convolution3d.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\hqdn3d.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\tdeint.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\masktools.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\repair.dll")
LoadPlugin("C:\PROGRA~1\GORDIA~1\AviSynthPlugins\MTv2\mt_masktools.dll")

mpeg2source("C:\ID4xvid\projekt.d2v")

source = crop(6,76,708,424)
swapped = source.doubleweave()

bobblend = source.tdeint(mode=1,type=3,mthreshL=0,mthreshC=0).deleteframe(0)
#bobblend = swapped.EEDI2(field=-2) # alternative to the line above
bobblend = merge(bobblend.selecteven(),bobblend.selectodd(),0.5)

swapped = swapped.selectodd()

global cmetric = source .mt_convolution(horizontal="1",vertical="-1 2 -1")
global smetric = swapped.mt_convolution(horizontal="1",vertical="-1 2 -1")

global clean = source .LanczosResize(640,272).convolution3d(preset="movielq")
\ .LimitedSharpen(ss_x=1.0,ss_y=1.0,Smode=4,strength=20) # normal strength for normal parts
global shifted = bobblend.LanczosResize(640,272).hqdn3d(1)
\ .LimitedSharpen(ss_x=1.0,ss_y=1.0,Smode=4,strength=60) # double strength for phaseshifted parts

scriptclip(clean,"cmetric.averageluma > smetric.averageluma? shifted : clean")

[Mug Funky]

hey cool! somebody noticed that stodgy field-matcher i posted :)

i've no idea where the red lines came from though... reading through that script it shouldn't be appearing at all. maybe some plugin in the chain has a bug or is the wrong version (compared to the one the script was written for). maybe try chase down the same versions of all the plugins used.

does that happen all through the video or just on filtered frames? to find out you can stick a ".subtitle("swapped") on the "swapped" clip. remember to remove it before your final encode though :)

[naz69]

The line only appears on the first frame after scene change in deinterlaced/filtered parts same in xvid and while viewing avs in vdub, also it seems to contain some artifact looking like the part of the previous frame (the last one before scene change).

Here are the screens (http://www.jotem.net/grim/id4sss2.png).

First two showing the last frame before the scene change and a following frame with the line and the artefacts (look at guys right hand).

Another two display the scenechange frame and the following frame where the top line dissapears and the overal quality gets a lot better. (Frames are from encoded xivd so I dont know if a dissaperance of the noise in the second frame of the last two ss is a quality drop after keyframe or quality gain after changing from frame that contained the artifacts from the one before as in the first two ss.)

[fight2win]

can someone tell me where to get dehalo.avs or dehalo.avsi from?