How to handle a fade applied to hard telecine?

[sirt]

Hello,

It's me again (!) ; What would you do to correct something like this ? Otherwise, I don't know how to call this :horizontal interlaced lines ?

http://i1240.photobucket.com/albums/gg481/abel413/2-1.png
http://i1240.photobucket.com/albums/gg481/abel413/1-1.png

I've tried several filters with no sucess :

vinverse(sstr=0.4,amnt=255,uv=3,scl=0.25)
vinverse(sstr=1.6,amnt=255,uv=3,scl=0.25)
vinverse(sstr=3,amnt=255,uv=3,scl=0.25)

(Depending on how marked are the artefacts)

But I don't get any satisfying result because each variant is either blurred either not corrected enough

blur(1)

One word comes to my mind : awful !

QTGMC(preset="slower")

It simply doesn't work well

So I have no idea what more I could do ; I want to delete those lines. It usually works well with vinverse but this time I have no luck.

[jmac698]

Maybe

blur(amountV=.5)


It almost looks like a purposeful "video effect". I've never seen anything like it as a video problem.

[Didée]

Maybe it's an intended Effect, maybe it's just interlaced content that has been stupidly resized progressively.

In any case, a 1023x682 screenshot tells pretty much nothing.

All screenshots shall be posted at 1:1 original pixels. Always. You shall not resize.

[Wilbert]

It looks a bit like interference. Where did you get the footage from? Try DeFreq for example: http://avisynth.org.ru/defreq/defreq.html

[sirt]

It looks a bit like interference. Where did you get the footage from? Try DeFreq for example: http://avisynth.org.ru/defreq/defreq.html

Thanks for your reply. As explained in another thread, it is another footage I use at my work. Mostly are Uncompressed YUV videos and MPEG2 like that one ; I took several pictures and also made a small sample but it's not possible to share the entire source. Most of this stuff is destined for broadcast or TV channels. I do have to correct the defaults then do encoding tests on them with a new h264 encoder.

[jmac698]

The picture posted created the black lines, they are not in the video. The actual problem is what you might call aliasing. It seems like some of the fields are a blend of two fields. Yes, it looks like the top fields are ok and the bottom fields are a blend of the top and bottom fields.
A quick solution would be to deinterlace with qtgmc.
Another idea is to try a subtraction.

Ok here's what it is, I suppose you would call it a blend deinterlace. There 3 clean progresive frames followed by 2 aliased frames. It seems these came from the original combed frames of a 3:2 pulldown sequence. They've been blended so that the top and bottom fields are one.

[sirt]

It looks a bit like interference. Where did you get the footage from? Try DeFreq for example: http://avisynth.org.ru/defreq/defreq.html

I've tried something like this but I have no idea how I could parameter DeFreq :

Defreq(fx=0.0, dx=3.0, fy=63.9, dy=4.0, sharp=8.0, fx2=0.0, dx2=8.0, fy2=0.0, dy2=3.0, sharp2=8.0, show=0, plane=2)

But it doesn't do anything.

The picture posted created the black lines, they are not in the video. The actual problem is what you might call aliasing. It seems like some of the fields are a blend of two fields. Yes, it looks like the top fields are ok and the bottom fields are a blend of the top and bottom fields.
A quick solution would be to deinterlace with qtgmc.
Another idea is to try a subtraction.

Ok here's what it is, I suppose you would call it a blend deinterlace. There 3 clean progresive frames followed by 2 aliased frames. It seems these came from the original combed frames of a 3:2 pulldown sequence. They've been blended so that the top and bottom fields are one.

Well when deinterlacing with QTGMC :

QTGMC(Preset="very slow",TR0=2,TR1=2,TR2=0,Sharpness=0.1,SourceMatch=3,EZKeepGrain=1.4,NoisePreset="faster",EdiMode="NNEDI3",Lossless=2).selectodd()

It result in flickering, so I guess I don't know how to configure it !

EDIT : so i've decided to upload this because it may help you more even though I am reluctant to do this : http://www.sendspace.com/file/6wdv9i

[Wilbert]

Could you upload the sample somewhere?

[sirt]

Yes, please check my EDIT above. I have forgotten to say the pictures refer to the iVTC result.

[jmac698]

It's fieldblended film->telecine->NTSC
I tried this

srestore(frate=23.976,omode="pp3")
decimate(cycle=5, quality=0)

It didn't do anything about the 2 of 5 frames that are bad.
I tired Scharfi's manual fix, that didn't work well either.

[sirt]

Thanks jmac698, I have identical result anywhere in the video e.g. anything is corrected.

In fact the only good filter seems to be vinverse (see my settings above) but it is not enough there. When using sstr near 1 or below it blurs too much even if the interference lines are deleted. I think we need something similar to vinverse there

[jmac698]

I don't know what vinverse does, but I understand the problem now.
I think you could use something I made before, which I called DOCI
http://forum.doom9.org/showthread.php?t=158230

Except not that exactly; but modified to work on luma instead of color.

This is the normal 3:2 pulldown pattern:

t0...t0...t1...t2...t3...t4...t4 top fields of NTSC video
b0...b1...b2...b2...b3...b4...b5 bot fields of NTSC video
cln lac lac cln cln| cln lac

t0 means the top field of film frame 0, b0 means the bottom field of film frame 0.

What that means is, NTSC frame 0 has film frame 0 as the top field and film frame 0 as the bottom field.
Since both NTSC fields come from the same film frame, it's a progessive or clean (cln) frame.
Next is a combed or interlaced (lac) frame. The two NTSC fields come from different film frames.

Your problem is that in this frame, the two fields are blended. This means
NTSC top field=(t0+b1)
NTSC bot field=(t0+b1)
Since they are both the same value, it makes a tall pixel, which looks blocky. (I wouldn't call them lines)
The solution is to use a subtraction.
We must recreate film frame 1.

The answer is the equation:
b1=(t0+b1)-t0
t1=(t1+b2)-b2
This means to separatefields, then subtract the first blended frame, top field from the last top field. This becomes a new bottom field.
For the 2nd blended frame, subtract the bottom field from the next bottom field. This becomes a new top field.
Erase the two blended frames, and replace with the new frame.
Now you have only 4 of the 5 frames, and the framerate is now 23.976fps, and you've recovered the film frames.
You could then do your job of encoding it directly to AVC. Otherwise, for mpeg you should encode with soft telecine.

[jmac698]

First version, not finished
--
First bug, is subtracting rec601 levels luma correctly.
--
Ok, can't finish this right now.

MPEG2Source("C:\project001a\aliasing sample\awful.d2v")
#Reconstruction of field-blended film to pulldown to NTSC filter by jmac698 (unblendfilm2telecine)
#v0.1
#Field-blended deinterlacing of telecined video leads to two bad looking frames out of every 5
#This filter will correct the problem
#the original film frames are output. The input is 29.97 and the output is 23.976
#Known issues: still a bit of garbage left, non-adaptive - but can use editing to make it work for real, restored color has lost 1bit of accuracy,
#Set the phase here, the filter works on a fixed pattern

trim(2,0)
destroyed=last
#reconstruct frames
restored=unblendfilm2telecine
#Show the ivtc'd video; compare against non-restored version
restored.simpleivtc
Crop(0, 64, -0, -64)

function restoreluma(clip destroyed, clip clean) {
#restores merged chroma from destoyed by subtracting clean, one of the mergee's
#destroyed=(clean+unknown)/2, unkown=(destroyed-clean/2)*2
#First, remove luma offset of 16
destroyed=destroyed.tweak(bright=-16)
clean=clean.tweak(bright=-16)
Overlay(destroyed, clean.tweak(cont=.5), mode="Difference", pc_range=true).tweak(bright=-128).tweak(cont=2).tweak(bright=16)
#now merge with existing luma
#note: there is no precision loss compared with d*2-c because d and c are always even;
# because the destruction was d=(c+unknown)/2
mergechroma(last,destroyed)
}

function unblendfilm2telecine(clip in) {
#Expects and returns a frame based clip
#Restore destroyed chroma; four fields out of 10
#note that if the pulldown pattern is distrupted, you may have to trim/edit the output
#Extract all the fields
#Need f1, f3, f4, f6 to reconstruct
#Output 0,1,0,newf3,newf4,7,6,7,8,9
#Note: I considered using DoubleWeave, but that doesn't give me the combinations two fields apart which I need
in.assumetff.separatefields
f0=selectevery(10,0)
f1=selectevery(10,1)
f2=selectevery(10,2)
f3=selectevery(10,3)
f4=selectevery(10,4)
f5=selectevery(10,5)
f6=selectevery(10,6)
f7=selectevery(10,7)
f8=selectevery(10,8)
f9=selectevery(10,9)
newf3=restoreluma(f3,f1)#new 1bot
newf4=restoreluma(f4,f6)#new 1top
interleave(f0,f1,f0,newf3,newf4,f7,f6,f7,f8,f9)
weave
}

function simpleivtc(clip in) {
#IVTC for a fixed 3:2 pulldown pattern, non-adaptive, returns 4/5 framerate clip (i.e. 24fps from 30fps)
in.swapfields.doubleweave
pulldown(0,3)
}

[sirt]

Thanks a lot for your explanations. But I am not able to do that myself I think.

I went to your DOCI post and I savec your script in an AVS file. Then I wanted to apply Demo() but it shows something that I don't understand ; I don't know how to your script.

I also tried your last script but with no success, it seems it highlights problematic frames.

EDIT : Didée made some function called "vinverse2" in my previous Dot crawl thread :


function Vinverse2(clip clp, float "sstr", int "amnt", int "uv")
{
uv = default(uv,3)
sstr = default(sstr,2.7)
amnt = default(amnt,255)
uv2 = (uv==2) ? 1 : uv
STR = string(sstr)
AMN = string(amnt)
vblur = clp.sbrV()
vblurD = mt_MakeDiff(clp,vblur,U=uv2,V=uv2)
Vshrp = mt_LutXY(vblur,vblur.mt_convolution("1","1 2 1"),expr="x x y - "+STR+" * +",U=uv2,V=uv2)
VshrpD = mt_MakeDiff(Vshrp,vblur,U=uv2,V=uv2)
VlimD = mt_LutXY(VshrpD,VblurD,expr="x 128 - y 128 - * 0 < x 128 - abs y 128 - abs < x y ? 128 - 0.25 * 128 + x 128 - abs y 128 - abs < x y ? ?",U=uv2,V=uv2)
mt_AddDiff(Vblur,VlimD,U=uv,V=uv)
(amnt>254) ? last : (amnt==0) ? clp : mt_LutXY(clp,last,expr="x "+AMN+" + y < x "+AMN+" + x "+AMN+" - y > x "+AMN+" - y ? ?",U=uv,V=uv)
return(last)
}

function sbr(clip o) {
rg11=o.removegrain(11)
rg11D=mt_makediff(o,rg11)
rg11DD=mt_makediff(rg11D,rg11D.removegrain(11)).mt_lutxy(rg11D,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?")
o.mt_makediff(rg11DD,U=2,V=2)
}

function sbrV(clip o) {
rg11=o.mt_convolution("1","1 2 1")
rg11D=mt_makediff(o,rg11)
rg11DD=mt_makediff(rg11D,rg11D.mt_convolution("1","1 2 1")).mt_lutxy(rg11D,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?")
o.mt_makediff(rg11DD,U=2,V=2)
}

It is similar to vinverse and call solve some frames.

[jmac698]

vinverse is for removing combing, that's not the problem, but if it helps it's just a coincidence.
I'd like to prove if my script works or not but I don't have time right now.

[Didée]

I don't see anything "blended" in this sample. The problem is that the fades were done after the telecine. I.e. those same-(24p)frame fields that are spread between two 30i frames have different brightness because of that, and don't fit together anymore.

Words too complicated. Diagram!

a b b c d < fields
a b c d d < fields

8 6 4 2 0 < brightness (due to fading)

Frame c cannot be IVTC'ed without artifacts, since the present fields of c have different brightness.

Also, an IVTC filter will have a hard time to do "correct" matching in the first place.

Not to speak of the multiple fieldorder transitions in the sample. "Broken", that is.

[sirt]

No worries I will also try on my own some things and report the potential results there. Thanks for your time spent on this.

[jmac698]

Didée
Thanks for pointing this out, however with my technique the frame c can still be recovered, just use different coeficients.
You say it's not blended, but I'm not using the term properly. I don't believe that a current and previous frame is blended, but two fields in one frame are spatially blended. You can also consider this as blended of two adjacent fields temporally.
Also, if my model is correct, the blends are easy to find; it's any frame where the top and bottom fields equal each other.
You can quickly see that my model is pretty close by subtracting adjacent fields; if there's results that are nearly neutral grey, then the two fields were almost equal.

[sirt]

I don't see anything "blended" in this sample. The problem is that the fades were done after the telecine. I.e. those same-(24p)frame fields that are spread between two 30i frames have different brightness because of that, and don't fit together anymore.

Words too complicated. Diagram!

a b b c d < fields
a b c d d < fields

8 6 4 2 0 < brightness (due to fading)

Frame c cannot be IVTC'ed without artifacts, since the present fields of c have different brightness.

Also, an IVTC filter will have a hard time to do "correct" matching in the first place.

Not to speak of the multiple fieldorder transitions in the sample. "Broken", that is.

Thanks for having checked this Didée. What do you mean by "broken" ? Isn't there anyhting to do ? I'm surprised jmac698 infers vinverse and also your vinverse2 are not excepted filters to use there in spite of the fact they give an acceptable result on some frames. It is probably some kind of "luck" I guess.

[jmac698]

My theory doesn't seem right. I looked at a subtracted fields. There does seem to be some pattern however.

[sirt]

My theory doesn't seem right. I looked at a subtracted fields. There does seem to be some pattern however.

Thanks for reflecting about this ! Anyway it is too complex for me, the only thing I can do is test and see what happens.

[mandarinka]

if it is regular enough, fft3dfilter could remove it if you supply a proper pattern image (the pattern mode).

[sirt]

if it is regular enough, fft3dfilter could remove it if you supply a proper pattern image (the pattern mode).

Thanks, do you have a configuration suggestion ?

[Wilbert]

I don't see anything "blended" in this sample. The problem is that the fades were done after the telecine. I.e. those same-(24p)frame fields that are spread between two 30i frames have different brightness because of that, and don't fit together anymore.

I completely agree with Didee. You will have to undo the fading before ivtc-ing it (but i'm not sure whether this is possible to do in an acceptable way):

SeparateFields()
UnFade()
Weave()
Telecide(order=1, guide=1).Decimate(cycle=5) #IVTC

[sirt]

I completely agree with Didee. You will have to undo the fading before ivtc-ing it (but i'm not sure whether this is possible to do in an acceptable way):

SeparateFields()
UnFade()
Weave()
Telecide(order=1, guide=1).Decimate(cycle=5) #IVTC


Thanks for your answer. What is that UnFade() ?

[Wilbert]

Thanks for your answer. What is that UnFade() ?
The opposite of fade, but sadly such a filter doesn't exist.

[Guest]

[bump after re-open]

[mirkosp]

I know it's meant to deal with moving text, but wouldn't ivtc_txt60mc (http://forum.doom9.org/showthread.php?p=1466105#post1466105) get a perfect result anyway? A fade should hopefully be easy enough to interpolate with good results. That said, even if you don't get the fade back to linear it's not that big of a problem, it's going to look fine anyway in motion. So alternatively to ivtc_txt60mc, the simple solution is just to use vinverse (as it was already suggested) after you do proper ivtc (you most likely want to tell your matcher to avoid postprocessing the frames and perhaps you might have to manually make sure the matches are correct).
Hell, I'd say ivtc_txt60mc isn't really required to get back to linear. Ideally you could do this by doing a high quality bob, then just doing an overlay at 0.5 with the fields that need to be overlayed to get a linear fade.

[cretindesalpes]

On a 3:2 telecine pattern, vinverse() the 2-field image, and for the 3-field image, average the 1st and 3rd fields, match with the 2nd field and vinverse() the result to smooth the remaining combing. But sometimes vinverse() cannot cancel the combing, especially on oblique, thin and and highly contrasted lines (mostly with anime/cartoon sources). In this case you could replace vinverse() with daa(). You could also limit the changes by the difference between the average lumas of the two fields.

[sirt]

I know it's meant to deal with moving text, but wouldn't ivtc_txt60mc (http://forum.doom9.org/showthread.php?p=1466105#post1466105) get a perfect result anyway? A fade should hopefully be easy enough to interpolate with good results. That said, even if you don't get the fade back to linear it's not that big of a problem, it's going to look fine anyway in motion. So alternatively to ivtc_txt60mc, the simple solution is just to use vinverse (as it was already suggested) after you do proper ivtc (you most likely want to tell your matcher to avoid postprocessing the frames and perhaps you might have to manually make sure the matches are correct).
Hell, I'd say ivtc_txt60mc isn't really required to get back to linear. Ideally you could do this by doing a high quality bob, then just doing an overlay at 0.5 with the fields that need to be overlayed to get a linear fade.

Thanks for your help, I saved the filter as an AVS file but I don't know how to make it work. I've tested ivtc_txt60mc(23.976) on the iVTCed video but it doesn't work (invalid parameters) and with ivtc_txt60mc(24) it doesn't work as well because it doesn't appl on the frame I need.



On a 3:2 telecine pattern, vinverse() the 2-field image, and for the 3-field image, average the 1st and 3rd fields, match with the 2nd field and vinverse() the result to smooth the remaining combing. But sometimes vinverse() cannot cancel the combing, especially on oblique, thin and and highly contrasted lines (mostly with anime/cartoon sources). In this case you could replace vinverse() with daa(). You could also limit the changes by the difference between the average lumas of the two fields.

Thanks, this is a comparison with daa() :

http://img254.imageshack.us/img254/3809/test2ud.png

http://img9.imageshack.us/img9/2068/test3bp.png

http://img40.imageshack.us/img40/8149/test1yq.png

http://img256.imageshack.us/img256/3648/testyn.png

As you see the lines are solved but it looks like the video is also filtered, probably because it is an aliasing filter. But, at least, it seems to always work without needing to be adjusted (like vinverse). Do you maybe know something similar like this ? I think we are on the way !

[sirt]

Any idea ?

[GMJCZP]

sirt, try this version of daa() that I modified to make it more smooth.

# Anti-aliasing with contra-sharpening by Didée (thanks to Didée, modded for this case with nnedi3)
# Is required VerticalCleaner
function DAAMod(clip c)
{
nn = c.nnedi3(field=-2) #or use nnedi2
dbl = mt_average(selecteven(nn),selectodd(nn),U=3,V=3)
dblD = mt_makediff(c,dbl,U=3,V=3)
shrpD = mt_makediff(dbl,dbl.removegrain((width(c)>1100) ? 20 : 14),U=3,V=3).verticalcleaner(mode=2)
DD = shrpD.repair(dblD,9)
return dbl.mt_adddiff(DD,U=3,V=3)
}

[feisty2]

@sirt
got an accurate solution (http://forum.doom9.org/showthread.php?t=174151) for this issue.
but you'll have to switch to vaporsynth to test it cuz I don't do avisynth plugins