Strange Source

[egrimisu]

OK, when i open with mplayerc(MPEG 2 internal filter) the m2v file looks like this http://img46.imageshack.us/img46/5075/buncs0.png
And when i open the avs(usign dgdindex) in with mpc i get these http://img384.imageshack.us/img384/7440/raulv9.png , even after encode remains like this. Even if i force the mpeg2 to be open with fddshow i gotthose nasty orizontal line Why?

The orizontal lines are vizible only when the light fades in some scenes, but i can feel that they are present everytime.

[neuron2]

Post a link to an unprocessed stream sample that contains the problematic area.

[TheRyuu]

Post a link to an unprocessed stream sample that contains the problematic area.

Don't need a sample to see what's going on there. It's common to see the combing on fades. Normally it can be easily delt with in yatta on anime by either sectioning out the fade and applying some deinterlacer to it or you can use a custom list (like reblend or a hacked up vinverse into reblend).

The following is the latter. Some of the options don't work because of it not using the original function. Vinverse I've found to be higher quality then the original which would blur the crap out of some stuff.

Please read the last line of this post for what I think is best for you in this case.

You can use it fairly easily in your case. Just:
Reblend_Vin(startframe,endframe)
Replace startframe/endframe with the real start/end frames.

It should be applied post-telecide and before decimate.
Of course in yatta this is done by the press of a button. In your case you might need to look for the stuff manually.

function ReBlend_Vin(clip clip, int "start", int "end", int "order", int "mode", int "ythresh", int "cthresh", float "sstr", int "amnt", int "uv"){

###############################################################################
# #
# ReBlend #
# #
# Requires MPEG2DEC, mSharpen, ChromaShift, MaskTools and #
# vd_warpsharp which you can find below *slaps head - changes order* #
# #
# Uses MPEG2DEC's BlendFields() to kill hardest interlacing, like in #
# interlaced fades. But BlendFields is very blurry and comes with evil #
# chroma artifacts. So I use it only for fades with movement and sometimes #
# for pattern-blends. ReFadeIn/Out should be weapon of choice if possible. #
# #
# To workaround the chroma artifacts the top/bottom-field of the #
# original chroma gets doubled in height and then merged with the #
# luma blended by BlendFields. #
# mSharpen and vd_warpsharp somewhat help with the blurryness. If you #
# don't like vdub's warpsharp for some odd reason feel free to change it #
# with some other warpsharp in the script below. (mode1) #
# #
# Mode2 uses simple doubled fields on planar surfaces #
# merged with "mode1'd" edges using MaskTools' MaskedMerge. #
# Can produce a weird look especially on extremely blocky scenes, #
# so mode1 is still an option. #
# #
# Mode3 is a dumb-mode and is just using resized fields. Looks best #
# in some very rare cases when there are scenes with no edges at all. #
# #
# Mode4(default) uses blur(0,1) and is fast and produces way less blocks #
# #
# "order" parameter was chosen according to decomb. #
# Just means "1" uses top-field and "0" uses bottom-field. #
# ythresh and cthresh are thresholds for EdgeMask's edge-detection. #
# #
# History #
# v0.1-0.4 too lazy to list #
# v0.5 mode2+3 added #
# v0.6 mode4 #
# #
# #
###############################################################################


#LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\reblend_files\ChromaShift.dll")
#LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\reblend_files\MaskTools.dll")
#LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\reblend_files\MPEG2DEC.dll")

start = default(start,0)
end = default(end,framecount(clip))
order = default(order,1)
mode = default(mode,4)
ythresh = default(ythresh,3)
cthresh = default(cthresh,5)
sstr = default(sstr,2.7) #strength for vinverse
amnt = default(amnt,255)
uv = default(uv,3)

new = clip.Vinverse(sstr, amnt, uv).trim(start,end)

new = clip.isYV12() ? new.converttoyv12() : clip.isYUY2() ? new.converttoyuy2() : new.converttorgb32()

final = (start == 0) ? new + clip.trim(end+1,0) :
\ (start == 1) ? clip.trim(0,1).deleteframe(1) + new + clip.trim(end+1,0) :
\ (end == clip.framecount) ? clip.trim(0,start-1) + new :
\ (end == clip.framecount-1) ? clip.trim(0,start-1) + new.deleteframe(0) + clip.trim(c.framecount,c.framecount)
\ : clip.trim(0,start-1) + new + clip.trim(end+1,0)

assert(clip.framecount == final.framecount, "ReBlend(): warning! wrong framecount! beat the author for writing buggy scripts >_>")

return final }

However there is a solution for the lazy which isn't guaranteed to work, but works most if not all of the time (I still don't trust it but meh).

###############################################################################################################
#
# Smartfade v0.1
#
# Aimed at removing interlaced fades in anime. Uses luma difference between two fields as activation threshold.
#
# Parameters:
#
# threshold (default: 0.4): Threshold for fade detection.
# tempsmooth (default: false): Enables temporalsoften on anything but fades (temporalsoften tends to create
# artifacts in fades).
# show (default: false): Displays luma difference between fields without processing anything.
#
###############################################################################################################


function smartfade(clip clip, float "threshold", bool "tempsmooth", bool "show") {

global threshold = default(threshold,0.4)
global tempsmooth = default(tempsmooth,false)
show = default(show,false)

global c = clip.isYV12() ? clip : clip.converttoyv12()

show ? scriptclip(c, "subtitle(string(c.separatefields().selectodd().averageluma() \
- c.separatefields().selecteven().averageluma()))") : \
scriptclip(c, "sep = c.separatefields() \
avg = sep.selectodd().averageluma() - sep.selecteven().averageluma() \
abs(avg) > threshold ? interleave(sep.selecteven(),sep.selectodd()). \
weave().fieldfade().lanczosresize(width(c)*2,height(c)*2).sangnom(). \
lanczosresize(width(c),height(c)).degrainmedian(mode=3).converttoyuy2(). \
convolution3d(0,floor(abs(avg/2)),0,0,0,2.8,0).converttoyv12() : \
tempsmooth ? temporalsoften(3,3,5,8,2) : last")

final = clip.isYV12() ? last : clip.isYUY2() ? converttoyuy2() : converttorgb32()

return final
}

My suggestion for you would to be to just use smart fade. Apply it post telecide before decimate and see what kind of results you get with it.
If your source is progressive just apply it after the input as martino has said below.
I'm sorry I did not recognize the source >_>

[martino]

The orizontal lines are vizible only when the light fades in some scenes, but i can feel that they are present everytime.
On that source they are present on all fades/cross-fades.

Also to the above poster there is nothing to decimate. The source is progressive, so just apply either of those after the input.

[TheRyuu]

Furthermore, to elaborate on smart fade.
I am not sure whether using those bunch of denoisers in there provides the best result, or it just replacing them with something like vinverse would provide better results.

So you can try this and see which one gives a better result:
###############################################################################################################
#
# Smartfade v0.1
#
# Aimed at removing interlaced fades in anime. Uses luma difference between two fields as activation threshold.
#
# Parameters:
#
# threshold (default: 0.4): Threshold for fade detection.
# tempsmooth (default: false): Enables temporalsoften on anything but fades (temporalsoften tends to create
# artifacts in fades).
# show (default: false): Displays luma difference between fields without processing anything.
#
###############################################################################################################


function smartfade(clip clip, float "threshold", bool "show") {

global threshold = default(threshold,0.4)
show = default(show,false)

global c = clip.isYV12() ? clip : clip.converttoyv12()

show ? scriptclip(c, "subtitle(string(c.separatefields().selectodd().averageluma() \
- c.separatefields().selecteven().averageluma()))") : \
scriptclip(c, "sep = c.separatefields() \
avg = sep.selectodd().averageluma() - sep.selecteven().averageluma() \
abs(avg) > threshold ? interleave(sep.selecteven(),sep.selectodd()).weave().vinverse() : last")

final = clip.isYV12() ? last : clip.isYUY2() ? converttoyuy2() : converttorgb32()

return final
}

[Gavino]

function smartfade(clip clip, float "threshold", bool "tempsmooth", bool "show") {

global threshold = default(threshold,0.4)
global tempsmooth = default(tempsmooth,false)
show = default(show,false)

global c = clip.isYV12() ? clip : clip.converttoyv12()

show ? scriptclip(c, "subtitle(string(c.separatefields().selectodd().averageluma() \
- c.separatefields().selecteven().averageluma()))") : \
scriptclip(c, "sep = c.separatefields() \
avg = sep.selectodd().averageluma() - sep.selecteven().averageluma() \
abs(avg) > threshold ? interleave(sep.selecteven(),sep.selectodd()). \
weave().fieldfade().lanczosresize(width(c)*2,height(c)*2).sangnom(). \
lanczosresize(width(c),height(c)).degrainmedian(mode=3).converttoyuy2(). \
convolution3d(0,floor(abs(avg/2)),0,0,0,2.8,0).converttoyv12() : \
tempsmooth ? temporalsoften(3,3,5,8,2) : last")
...
OK, your function probably works fine, but if you really must use a global variable inside a function, especially a function for public use, it's a good idea to use some sort of naming scheme to avoid clashes with variables elsewhere. In any case, it's best to avoid a name like c.

In fact here the global c is unnecessary and (since it is the parameter to ScriptClip) all its uses inside the run-time scripts can be replaced by last (or by nothing at all).

c = clip.isYV12() ? clip : clip.converttoyv12()

show ? scriptclip(c, "subtitle(string(separatefields().selectodd().averageluma() \
- separatefields().selecteven().averageluma()))") : \
scriptclip(c, "sep = separatefields()
... etc

[TheRyuu]

OK, your function probably works fine, but if you really must use a global variable inside a function, especially a function for public use, it's a good idea to use some sort of naming scheme to avoid clashes with variables elsewhere. In any case, it's best to avoid a name like c.

In fact here the global c is unnecessary and (since it is the parameter to ScriptClip) all its uses inside the run-time scripts can be replaced by last (or by nothing at all).

c = clip.isYV12() ? clip : clip.converttoyv12()

show ? scriptclip(c, "subtitle(string(separatefields().selectodd().averageluma() \
- separatefields().selecteven().averageluma()))") : \
scriptclip(c, "sep = separatefields()
... etc


Then change it?
I didn't write it :p

[egrimisu]

The source is 23,976 progesive, i don't think there is a need for deinterlancing!!! Atleast the new dgindex say that is progrsive and all the frame in the parse txt are marked with 2 ( * - 3:2, probably 2 progresive, i didn't find any manual but i bealive that is progresive). Just tell me if you need a sample.

[Gavino]

Then change it?
I didn't write it :p
I wasn't getting at you, it was a general point aimed at function writers everywhere.

The function as written will not work if you have a variable c (not even global) in your script.
Nor can it be called more than once. :(

And I already showed how to change it. :)

This message brought to you by the Campaign for Better Avisynth Functions...

[egrimisu]

So... how to get rid of those lines?

[martino]

So... how to get rid of those lines?
Have you read the replies? It should be pretty apparent. You even have scripts posted here rather than a "just use xxx filter".

[TheRyuu]

So... how to get rid of those lines?

Lets see what I wrote in my post:
Please read the last line of this post for what I think is best for you in this case.

Lets see what that last line is:
My suggestion for you would to be to just use smart fade. Apply it post telecide before decimate and see what kind of results you get with it.
If you have a progressive source or soft telecine (applied pulldown in dgindex/dvd2avi) then apply it post source. Any fades/crossfades will generally have the combing, smart fade just makes it easy.

Here is martino's modded version of smart fade, replaces that block of denoisers with vinverse+some AA/optional noise removal which is probably needed with vinverse (it can make noise):
###############################################################################################################
#
# Smartfade v0.2
#
# Aimed at removing interlaced fades in anime. Uses luma difference between two fields as activation threshold.
#
# Parameters:
#
# threshold (default: 0.4): Threshold for fade detection.
# dgm (default: false): Enables DeGrainMedian on fades for postprocessing.
# show (default: false): Displays luma difference between fields without processing anything.
#
###############################################################################################################

loadplugin("degrainmedian.dll")
loadplugin("vinverse.dll")
loadplugin("nnedi.dll")

function smartfade(clip clip, float "threshold", bool "dgm", bool "show") {

global threshold = default(threshold, 0.4)
dgm = default(dgm, false)
show = default(show, false)

c = clip.isYV12() ? clip : clip.converttoyv12()

show ? scriptclip(c, "subtitle(string(separatefields().selectodd().averageluma() \
- separatefields().selecteven().averageluma()))") : \
scriptclip(c, "sep = separatefields() \
avg = sep.selectodd().averageluma() - sep.selecteven().averageluma() \
abs(avg) > threshold ? interleave(sep.selecteven(),sep.selectodd()).weave().vinverse().lanczosresize(width*2,height*2).nnedi(field=0).lanczosresize(width,height) : last")

dgm ? degrainmedian(mode=3) : last

return last

}

And in case we are still blind, take the above function, throw it in an avsi file in your plugins directory, then in your script after source (or post telecide depending on if you needed it or not):
SmartFade(dgm=true)

[egrimisu]

Everything is fine but when a fade area appears seams like frames are droped(decimated).

[ChrisW77]

Do people READ ***ANYTHING*** on this forum, anymore ?

[martino]

Everything is fine but when a fade area appears seams like frames are droped(decimated).
Post your script. The smartfade script does not decimate anything, neither removes frames.

[egrimisu]

DGDecode_mpeg2source("D:\Work Hanbun no Tsuki ga Noboru Sora\Hanbun_no-Tsuki_ga_Noboru_Sora_01.DVD(MPEG22.d2v", cpu=0)
setmtmode(2)
SmartFade(dgm=true)
source = last
backward_vec1 = source.MVAnalyse(isb = true, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
backward_vec2 = source.MVAnalyse(isb = true, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec1 = source.MVAnalyse(isb = false, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec2 = source.MVAnalyse(isb = false, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
source.MVDegrain2(backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400,idx=1)
fft3dfilter(sigma=0.7, bt=1, bw=32, bh=32, ow=16, oh=16, plane=4, dehalo=0, ncpu=2)
ttempsmooth()
gradfun2db(thr=1.2)
crop (4,0,-4,0)
spline36resize(720,480)


The video is 29fps progresive, there's no interlanced frames

[TheRyuu]

Post a sample like 10 seconds of the part where you have the problem.

[Zep]

Post a sample like 10 seconds of the part where you have the problem.

you mean like neuron2 asked but you opened your mouth and side tracked him at the start of this thread and thus no sample yet 6 days later grrrrrr.......

[egrimisu]

http://www.yourfilehost.com/media.php?cat=other&file=Hanbun_no_Tsuki_ga_Noboru_Sora_01.DVD_MPEG2.WAV__Misu__Track1_cut.m2v

[martino]

That link isn't working for me, however I tried it today on the same source and couldn't notice anything wrong. I tried it with your exact chain, and nothing wrong. Framerate retuned is the same, as expected, framecount unchanged, frames matching. If you tried to play it in your media player then no wonder you noticed dropped frames. It's not a fact script, in fact it's pretty slow.

[egrimisu]

I'l post ny encode as soon as i get home tomorow night, i'm at my parents till then. Thanks

[egrimisu]

YES, i recompresed the clip and everything is OK. Probably i got something wrong using DUP in my previous encode. Thanks again

[Stephen R. Savage]

Apologies for the necropost, but I would like to point out that this doesn't even need global variables at all since you can do this with ConditionalFilter. Here is a proof of concept.

function SmartFade(clip input, float "thr")
{
fields = input.SeparateFields()
blend = Merge(nnedi2(input,field=0),nnedi2(input,field=1))
return ConditionalFilter(fields,blend,input,"Abs(SelectOdd().AverageLuma()-SelectEven().AverageLuma())",">",string(thr))
}

It doesn't process the same way as the original, but the general idea is here.

[egrimisu]

Hi and thanks, i solved the problem at that time using smartfade, thanks anyway :)

Apologies for the necropost, but I would like to point out that this doesn't even need global variables at all since you can do this with ConditionalFilter. Here is a proof of concept.

function SmartFade(clip input, float "thr")
{
fields = input.SeparateFields()
blend = Merge(nnedi2(input,field=0),nnedi2(input,field=1))
return ConditionalFilter(fields,blend,input,"Abs(SelectOdd().AverageLuma()-SelectEven().AverageLuma())",">",string(thr))
}

It doesn't process the same way as the original, but the general idea is here.