Need help cleaning up some footage. (pic included)

[Chainmax]

Source:
http://img467.imageshack.us/img467/7356/source2sy.png (http://imageshack.us)

Result:
http://img467.imageshack.us/img467/8161/result6uy.png (http://imageshack.us)

Script:
CDeBlend()
Deblock(quant=35)
DeGrainMedian(mode=0)
Crop(8,2,700,478,align=true)
GaussResize(640,480,p=100)
aWarpSharp(depth=16,cm=1)
LimitedSharpen()

What I am doing wrong, should I decimate after CDeBlend, tweak the defaults or do both?

[Chainmax]

Why doesn't CDeBlend work with MaskTools v1.5.8?

[MOmonster]

@Chainmax
Does it not? I don´t know why, will have a look at the new specs. I just integrated the last version I had on my pc.
But to your problem. Not restore24, not Crestore or Cdeblend or any other tool I know can restore this source. It isn´t a simple blend conversion. Only every 4th or 5th field is a clear one anbd the quality is that bad, that a restoring like restorefps does it won´t work.
The only thing I can do for you is to modificate Cdeblend to output only the clear fields (->12fps). This isn´t that good for the motion, but with a good decimating and maybe motion compensating fps conversion it will look much better than now.
You can use omode 4 of Cdeblend to find allready the optimized konfiguration for Cdeblend (highest val -> clear).

Edit: Where is the bobber in your source. Did you read the readme? You should begin with the second code example, the first is only for progressive blended material, maybe I should make it more clear.

[Chainmax]

Yeah, I didn't understand the readme. I'll try with TDeint bobbing and report back.

About MaskTools, since you included it in the package I assumed it was a special version. I wil try v1.5.8 then.

[Chainmax]

The following script:

ord = last.getparity() ? 1 : 0
a = tdeint(mode=1)
c = leakkernelbob(order=ord).crop(8,8,-8,-8).BilinearResize(480,288)
Cdeblend(a, clip2=c, thresh=50)
DeBlock_QED()
DeGrainMedian(mode=0)
Crop(8,2,700,478,align=true)
GaussResize(640,480,p=100)

Looks much better, but there's still too much blending (should I go to 23.976 or 29.97 afterwards?). Don't bother doing a customized version for me, it's not worth it in my opinion. Sorry mimage, but this is way out of my league :(.

[mimage]

Thanks for helping me in trying to figure this out, I appreciate it a lot! Same goes for everyone else that helped. I'm very thankful! ^__^

[MOmonster]

Looks much better, but there's still too much blending (should I go to 23.976 or 29.97 afterwards?). Don't bother doing a customized version for me, it's not worth it in my opinion.
Sorry, but I was faster. It´s just a easy modification, but I think it will help. Here is the script:
leakkernelbob(order=0, sharp=false, twoway=true, threshold=2, map=false)
a = last.crop(8,8,-8,-8).bilinearresize(288,240)
Cdeblend_mod(clip2=a, perc=6, thresh=1.8)


Function Cdeblend_mod(clip input, int "dmode", float "thresh", int "perc", float "bweight", clip "clip2")
{
###### PREPARATION ######
global output = input
global blendclip = default(clip2, input).trim(4,0)
global predup = input.duplicateframe(0).tweak(hue=0.11)
global predup2 = predup.duplicateframe(0).tweak(hue=-0.11)
global predup3 = predup2.duplicateframe(0).tweak(hue=0.08)
global postdup = input.trim(1,0).tweak(hue=0.22)
global postdup1 = input.trim(2,0).tweak(hue=-0.22)
global postdup2 = input.trim(3,0).tweak(hue=0.10)

global thresh = default(thresh,2)
global bweight = default(bweight,15)/100.0
global perc = default(perc,3)
global dmode = default(dmode,0)

###### Mask-Preparation ######
fr_diff = YV12Subtract(blendclip,blendclip.trim(1,0),tol=0)
frn_diff = YV12Subtract(blendclip,blendclip.trim(2,0),tol=0)
subtract = Average(fr_diff, 1.0, fr_diff.trim(1,0), 1.0, frn_diff, -1.0)

global ystring = "x y - 2 ^"
global cstring = "x"
global ssdiff = YV12lUTxy(blendclip, blendclip.trim(1,0), yexpr=ystring, uexpr=cstring, vexpr=cstring)
global m2_mask = YV12lUT(subtract, yexpr=ystring, uexpr=cstring, vexpr=cstring)

###### VAR.. ######
global difc1 = 1.0
global temp = 1.0
global btest3 = 1.0
global btest2 = 1.0
global btest1 = 1.0
global counter = 0


###### Conditional Function Chain, evaluated from bottom to top (!) ######
b99=scriptclip(input, " counter==3 ? postdup2 : (counter==2 ? postdup1 : (counter==1 ? postdup : (counter==-1 ? predup : (counter==-2 ? predup2 :
\ (counter==-3 ? predup3 : output)))))")

b5=FrameEvaluate(b99, " global counter = (btest1 > btest2 && btest1 > btest3 && btest1 > btest0 && btest1 > btestb) ||
\ counter < -1 && btest1 > btest0 && (btest1 > btest2 || btest1 > btest3) ||
\ btest1 > btest2*thresh && btest1 > btest0*thresh ? 3 : counter-1")

b4=FrameEvaluate(b5, " global btest3 = dmode == 0 ? (difc1+difc0-diffn)/(difc1<difc0 ? difc1+difc0/perc : difc0+difc1/perc) :
\ (dmode == 1 ? AverageLuma(m2_mask)/(difc1<difc0 ? difc1+difc0/perc : difc0+difc1/perc) :
\ (dmode == 2 ? LumaDifference(MergeLuma(blendclip,blendclip.trim(2,0),blw), blendclip.trim(1,0)) /
\ (difc1<difc0 ? difc1+difc0/perc : difc0+difc1/perc):
\ AverageLuma(YV12LUTxy(MergeLuma(blendclip, blendclip.trim(2,0), blw),blendclip.trim(1,0),
\ yexpr=ystring, uexpr=cstring, vexpr=cstring))/(difc1<difc0 ? difc1+difc0/perc : difc0+difc1/perc) ))")

b3=FrameEvaluate(b4, " global blw = difc1/(difc1+difc0) < bweight ? bweight : (difc1/(difc1+difc0) > 1-bweight ?
\ 1-bweight : difc1/(difc1+difc0))
global temp = btest0
global btest2 = btest3")

b2=FrameEvaluate(b3, " global difc1 = dmode < 3 ? YDifferenceToNext(blendclip.trim(1,0)) : AverageLuma(ssdiff.trim(1,0))
global diffn = dmode != 0 ? 0 : LumaDifference(blendclip,blendclip.trim(2,0))
global btest1 = btest2")

b1=FrameEvaluate(b2, " global difc0 = difc1
global btestb = temp
global btest0 = btest1")
return(b1.changefps(29.97))
}
You need the same filters for it. There is no dthresh or mthresh anymore and the simple thresh has another meaning (percentage to previous and next frame).
It´s much clearer and motion doesn´t hack so bad on some parts. Dmode 0 or 1 is prefered, the optimal perc value I don´t know. The output is 29.97 fps and it should be decimated to 23.976 fps for example with tdecimate(mode=1). If the decimation process results in blends, where there aren´t blends without decimating, try this (http://forum.doom9.org/showthread.php?p=745599#post745599) . The quality of the source is that bad, that a motion compensation could make no sense, but of course you can try it yourself ;) .

Edit:
Looks the whole source that overblended or is it just the intro? Sometimes only the beginning is that bad and therefore it would be better to use the simple Cdeblend or better restore24 for the good parts.

[Chainmax]

Just when I thought that I was out they pull me back in.

I hate you :).


mimage, can you post a sample from an episode rather than the intro?

[Chainmax]

How about this (http://www7.rapidupload.com/d.php?file=dl&filepath=5129)?

The following script was used:

ord = last.getparity() ? 1 : 0
leakkernelbob(order=0, sharp=false, twoway=true, threshold=2, map=false)
a = last.crop(8,8,-8,-8).bilinearresize(288,240)
Cdeblend_mod(clip2=a, perc=6, thresh=1.8)
TDecimate(mode=1)
DeBlock_QED()
DeGrainMedian(mode=0)
Crop(8,2,700,478,align=true)
GaussResize(640,480,p=100)
aWarpSharp(depth=16,cm=1)
LimitedSharpen()

[MOmonster]

ord = last.getparity() ? 1 : 0 This line isn´t necessary.
What is DeBlock_QED() for a function or filter? I don´t know it. Works it well?

[Didée]

What is DeBlock_QED() for a function or filter? I don´t know it. Deblock_QED is proof-of-concept ... it should not be a script. See this thread (http://videoprocessing.11.forumer.com/viewtopic.php?p=413#413) over at kassandro's.


Works it well? One of those cases where I was told "we must not do it this way" ... nonetheless it does its job. ;)

[mimage]

Alright. I'll try to get a sample up later today.

[Chainmax]

MOmonster: indeed, LeakKernelBob is set to order=0. Isn't it better to use that line instead of arbitrarily setting the field order?

Damn, Didée beat me to the punch :).

[mimage]

http://stormmetropolis.com/test/TheGuyverVOB.vob

[MOmonster]

Thanks for the quick answer Didee.
Normally it would be better, but this function is only for this special case. Also there is a problem with the fieldorder in the clip. The motion hacks sometimes. I was not sure if it gives the right order (and didn´t test it).

Edit: I downloaded the clip and it looks better than the intro, but I can only view it with vlc here. I´ll test it tomorrow.
But of course you can do it yourself. Just bob the source and have a closer look on the frames. If nearly every second is clear than restore24 or Cdeblend are the right choice. I there are more blends Cdeblend_mod should be used.

[Chainmax]

Has anyone watched my encode in order to see the results of CDeBlend_Mod?

[mimage]

I just checked and you did a really really really nice job on it. Congrats!! ;)

[MOmonster]

Decimating to 12 fps and than doing motion compensated framerateconversion back should improve the motion a bit.
But indeed much better than the original. The blenddetection is still tweakable, but I don´t think that it is worth much more time, the source is just too bad for it.

[Chainmax]

MOmonster, how difficult would it be to implement decimation to 12fps and motioncompensating back in the script I tried? Also, could I replace the LeakKernelBob line with TDeint(mode=1,type=1)?


mimage: is this a series or a movie? In either case, how long is it?

[foxyshadis]

Cdeblend_mod().Tdecimate(mode=1,cycleR=3,cycle=5).MotionProtectedFPS (http://forum.doom9.org/showthread.php?p=750037#post750037)(23.976)

=D

May as well give Tdeint a shot, see if it makes any difference.

[mimage]

mimage: is this a series or a movie? In either case, how long is it?
It's a twelve episode series.

[MOmonster]

@Chainmax
Foxyshadis gives you the answer (-> not difficult). Maybe the decimating makes some problems, than use the tip given by LogicDeLuxe (divided avs).
Of course you can use tdeint, but it is recommed to use leakkernelbob(order=0,twoway=true,threshold=2).crop.resize for the detection (clip2), because of speed reasons and the simple interpolating.
tdeint(mode=1,type=3)
a = leakkernelbob(order=0,twoway=true,threshold=2).crop(...).resize(...)

Cdeblend_mod(...,clip2=a)

@foxyshadis
Because of your source: I´ll take a look at the given clips and maybe improve FixBlendIVTC for it, but I don´t know when I can report back.

[mg262]

Be warned, motion compensation doesn't always work very well with non-CG animated material... or at least, backgrounds should be fine but foreground objects are built of solid blocks of colour so they often don't have enough detail to latch onto. It won't hurt to try both MVFlowFPS and MotionProtectedFPS, but you might find that one or both just falls back to blending...

[MOmonster]

@mg262
Yes I know, but the results of my first tests weren´t bad and if I should decide between 12 fps with less blends and 24fps with some more blends but better motion my choice is clear. :D

@mimage
Also your second clip is so badly overblended, so indeed using the default Cdeblend wouldn´t work. :(

[mimage]

It's alright.. I don't need to get rid of all the blending.. I could just cut the frames that blend when I edit the footage if I need to. I don't know if it's as messed up(srry I'm not well with deinterlacing) but is there away to fix this at all so there isn't blending. I can't figure it out.. well I've been trying, but I'm unsure what to do with it.

http://stormmetropolis.com/test/NinjaScrollvob.vob

[Chainmax]

I'm currently encoding the sample with the "12fps and backconversion" suggestion, once it's done I'll upload it and you can decide for yourself what looks better.

By the way, my suggestion is that if each episode lasts up to 25min, you should be able to fit 4 episodes per CD with Vorbis @ q4 sound (~112kbps, more or less equivalent to MP3 @ 160kbps). If each episode lasts more than 25mins, you could fit 3 episodes per CD with Vorbis @ q6 sound (~165kbps, more or less equivalent to MP3 @ alt-preset-standard). Finally, what I'd recommend as best alternative would be to put everything into a single DVD-R, that way you could keep all three AC3 tracks.

[MOmonster]

@mimage
And the next aweful source. The pal version I saw little time ago, looks much better.
The lumachannel shows a standard norm conversion, the right source for restore24 and Cdeblend. But the chromachannel is totally destroyed. A temporal cleaning is the reason for this chromablending and it is nearly irreversible.
Maybe a lumalinked mask can help a bit, but I don´t if it´s worth the time.

[mimage]

Oops, I was mistaken. I deleted most of this post because it was misleading.

[Chainmax]

The "12fps and backconversion" doesn't result in smooth motion. Check it out here (http://www7.rapidupload.com/d.php?file=dl&filepath=5249).

[MOmonster]

:eek:
Are this artefacts because of the motioncompensation or is it because I don´t have the decoder up to date? So or so, indeed there seems to be no positiv effect, so the last script seems to be ok (just using tdeint instead of leakkernelbob).

@mimage
This chroma problem isn´t that seldom. MaybeI´ll try to find a restoring methode, but next week or so.

[Chainmax]

Ok mimage, here's the final filterchain then:

SetMemoryMax(Your Ram / 2)

LoadPlugin("X:\wherever\DGDecode.dll")

LoadPlugin("X:\wherever\MaskTools.dll")
LoadPlugin("X:\wherever\Average_24Oct05.dll")
LoadPlugin("X:\wherever\LeakKernelDeint.dll")
Import("X:\wherever\CDeBlend_Mod.avs")
LoadPlugin("X:\wherever\TIVTC.dll")

LoadPlugin("X:\wherever\DCTFilter.dll")
Import("X:\wherever\DeBlock_QED.avs")

LoadPlugin("X:\wherever\DeGrainMedian.dll")

LoadPlugin("X:\wherever\aWarpSharp.dll")

Import("X:\wherever\LimitedSharpen.avs")



MPEG2Source("X:\wherever\Guyver.d2v")

leakkernelbob(order=0, sharp=false, twoway=true, threshold=2, map=false)
a = last.crop(8,8,-8,-8).bilinearresize(288,240)
Cdeblend_mod(clip2=a, perc=6, thresh=1.8)
Tdecimate(mode=1)

DeBlock_QED()

DeGrainMedian(mode=0)

Crop(8,2,700,478,align=true)

Lanczos4Resize(640,480)

aWarpSharp(depth=16,cm=1)

LimitedSharpen()

As codec, I recommend you to to use the latest x264 with the latest MeGUI. Select the AE-Maxquality profile with deblocking set at 0/0, keyframe interval of 240 and Min GOP size 24. As for bitrate, I already suggested number of episodes per CD. If you decide to backup to CD and not DVD, let me know and I'll instrcut you onwhere to find the latest Ogg Vorbis encoder and good Azid settings to use. If you have any further questions don't hesitate to ask.

[mimage]

OK then, thanks a bunch! I really really appreciate it! ^__^

[MOmonster]

@mimage
Sorry, I was wrong. Not only the chroma is destroyed by a temporal filter also the luma is. I don´t see any possibilty for a nice restoring. Go with restore24 or Cdeblend(omode=3, dthresh=60, thresh=66) + decimating, more work it isn´t worth. Chainmax filtercombi should also work not bad with this source. :(

[Chainmax]

The filterchain I posted is for the Guyver sample, I didn't check out the Ninja Scroll one.

[mimage]

The filterchain I posted is for the Guyver sample, I didn't check out the Ninja Scroll one.I know. I'm trying out the script now.. I'm having a few problems in making it all work.. I should work it all out, but if not I'll see to reporting back here and give a report on how it's working out. Anyway thanks a bunch for the help! I really appreciate it! :)

[Chainmax]

Glad to be of help. If you wind up using Vorbis audio, let me know and I'll tell you where to download the appropriate software and the settings to use it with,

[mimage]

lol, I don't need help with ninja scroll. I found a good method to deinterlace it.

[Chainmax]

Indeed, I've been trying to send him PMs for a while as well.

[XadoX]

Is there already a way to deinterlace Ninja Scroll PAL?
Because i do not get rid of the interlaced artifacts.

[MOmonster]

I don´t remember so good, but I think converttoyuy2(interlaced=true) + tdeint(mode=1)+Crestore(?) worked for me. Working with yv12 the chroma looks cruel.

[XadoX]

The deinterlaced worked good with Tdeint(mode=3), but the damn bledings.
I will give a trie to your method.

Also with mrestore() i did not get rid of those blendings.

[MOmonster]

Try the profiles for anime and if this doesn´t work, please upload a small sample the blenddetection fails on.

[XadoX]

I've tried the anime profiles also but with no good result.

Sample (http://rapidshare.de/files/24444451/Ninja.mpv.html)

[Backwoods]

The NTSC discs of Ninja Scroll have the same problems.

MPEG2Source("NS_16x9.d2v")
Cdeblend(dmode=8,omode=1,sd=true)
Decimate(5)

That gets rid of a good amount of the blends, but some still remain. If you find a better method, please let me know.

[MOmonster]

This is a heavy blended 12fps Source. This should work:
leakkernelbob(order=1,sharp=false,twoway=true,threshold=4)
Cdeblend(dmode=3, omode=3, sd=true, clip2=last.bilinearresize(352,288))
#(or tdeint(mode=1) and no clip2 or maybe only little bit soften)

Some blending artefacts will be left and the motion is not so good because the technical 12fps but the blends are gone. If there are 24fps pans or similar 24fps scenes in the source you should also set the dthresh parameter.

@Backwoods
Using decimate after Cdeblend is not recommed. Smartdecimate and different tdecimate modes works better together with Cdeblend.

[XadoX]

great work MOmonster. Most of the blend are gone, but should be the Output realy 50FPS then?

[MOmonster]

You have to decimate it after the Cdeblend call, with tdecimate or smartdecimate (to 24/25fps or direct to 12fps), sometimes a simple changefps(25) looks already ok. Do what works best for you. You don´t need to keep the 50fps.;)

[Backwoods]

Using decimate after Cdeblend is not recommed. Smartdecimate and different tdecimate modes works better together with Cdeblend.

Hmm, alright. I'll try each and see the difference. Thank you for the tip.