another restore function

[MOmonster]

Thanks for the fix. Everything works well.

I replaced the srestore.avs in the zip file (now 2.7d). A new version will maybe come later.

[AlanHK]

Thanks for the fix. Everything works well.

I replaced the srestore.avs in the zip file (now 2.7d). A new version will maybe come later.

Could I suggest you put a comment line in the AVS with the version number and date at least; if you're not intimately acqainted with the code there is no way to tell which version a particular "srestore.avs" file is.

[cretindesalpes]

Hello,

I have a telecined video with a weird kind of progressive blending between frames (I suppose it is intended to smooth transitions when displayed on cathodic screens). Is it possible to restore the original frames, and how ?

http://img386.imageshack.us/img386/7369/testp.th.jpg (http://img386.imageshack.us/img386/7369/testp.jpg)

[AlanHK]

A.avi and B.avi are 29.97 fps.

This script works:

AVISource("A.avi")
LanczosResize(720,576)
A=last

AVISource("B.avi")
LanczosResize(720,576)
B=last

A++B
Srestore(frate=25)


But I get an error with this script:

AVISource("A.avi")
Srestore(frate=25)
LanczosResize(720,576)
A=last

AVISource("B.avi")
Srestore(frate=25)
LanczosResize(720,576)
B=last

A++B
If A.avi and B.avi are different sizes, I get:
"ScriptClip: Function did not return a video clip with the same width as the source clip!"

But if they're the same size, it works fine.
Or if I replace "Srestore(frate=25)" with "ChangeFPS(25)", it also works.

So I guess a scoping error in SRestore.

[Gavino]

So I guess a scoping error in SRestore.
Because of its use of global variables, SRestore cannot be used more than once in a script.

[AlanHK]

Because of its use of global variables, SRestore cannot be used more than once in a script.

Well it can be used, and gives no obvious errors in some cases, presumably when the source videos are the same spec.

But is it actually giving correct results? It gives an output video of the right fps at least. It's worrying if it gives a wrong result without an error message.

[Gavino]

When I said it can't be used, I meant it cannot be relied on to work if called more than once. Even when it appears to work (ie no error is produced), the results are not necessarily correct, although you might get away with it under some circumstances.

Normally, I would be quite critical about a function (ab)using global variables (one of my pet peeves :)), but in the case of SRestore (and others of the same type), it is necessary for performance reasons.

Even so, the documentation should make clear that it should only be called once in a script.

[AlanHK]

Well, if it has to use global variables, could one be set so it knows if it has been called more than once, then exits with a error more informative than the one I mentioned earlier?

[Gavino]

Good idea.
How about it, MOmonster?

[MOmonster]

Is done!
Also version number is now documented inside the avs file.

Sorry, currently I really don't have much time for support.

@cretindesalpes
Yours source is theoretical restorable. Maybe srestore will have some problems with the particular blends.
After decimating (1of5) the pattern should be like this:
cbcbcbcbcbcb...
I don't see a reason reduce the framerate to ~12fps.

[SweetDream]

Thank you Momonster. :)

[Boulder]

What is the proper way to use SRestore for restoring DEFT conversions, i.e. is setting frate=25 enough? I'm working on the PAL Star Trek TNG DVDs.

[MOmonster]

Short answer: Yes, frate=25 is enough!
You can improve the quality of your bobbed source for example this way:
tdeint(mode=1,tryweave=true,full=false)
srestore(frate=25)

[Boulder]

I never got SRestore working well enough for that crappy TNG PAL conversion, but I suddenly remembered that I once asked for help when I used to capture the series when it was broadcast on TV.

Scharfis_brain eventually came up with this function which seems to work quite well:
function repairPAL_TNG_DVD(clip i, bool "show")
{

ord = getparity(i) ? 1 : 0

show = default(show,true)
global h=i.leakkernelbob(order=ord,threshold=5)

global he = show ? " even " : ""
global ho = show ? " odd " : ""
tele = show ? " fieldmatched" : ""
auto = show ? " deblended" : ""


# create analysis-clip for blend-detection

j=h.reduceby2()
# create edgemask of analysis-clip
global k=logic(j.DEdgemask(0,255,0,255,"5 0 -5 10 0 -10 5 0 -5",setdivisor=true,divisor=6,y=3,u=1,v=1),
\ j.DEdgemask(0,255,0,255,"5 10 5 0 0 0 -5 -10 -5",setdivisor=true,divisor=6,y=3,u=1,v=1),"max",y=3,u=1,v=1)
\ .greyscale().levels(0,0.5,160,0,255,false).bicubicresize(80,80)


# create some fwd & bwd clips. syntax:
# e=even | o=odd
# c=current | n=next | p=prev
# nn=next+1 | pp=prev-1

global kec=k.selecteven()
global ken=kec.deleteframe(0)
global kenn=kec.deleteframe(0).deleteframe(0)
global kep=kec.duplicateframe(0)
global kepp=kec.duplicateframe(0).duplicateframe(0)

global koc=k.selectodd()
global kon=koc.deleteframe(0)
global konn=koc.deleteframe(0).deleteframe(0)
global kop=koc.duplicateframe(0)
global kopp=koc.duplicateframe(0).duplicateframe(0)

global ko=h.selectodd() #.subtitle(ho)
global ke=h.selecteven()#.subtitle(he)
global kb=ko.mergeluma(ke,0.5).mergechroma(ke,0.5)

global th=1.02

# deblend the video
#k0 = ScriptClip(blankclip(ko), "(x>0) ? ko.subtitle(string(x)) : (x<0) ? ke.subtitle(string(x)) : kb.subtitle(string(x))")
k0 = ScriptClip(blankclip(ko), "(x>0) ? ko : (x<0) ? ke : kb")

k1 = FrameEvaluate(k0, "")


k2 = FrameEvaluate(k1, "x=(konl /th > kenl ) ? x+1 : (kenl /th > konl ) ? x-1 : x")
k3 = Frameevaluate(k2, "x=(kocl /th > kenl ) ? x+1 : (kenl /th > kocl ) ? x-1 : x")
k4 = Frameevaluate(k3, "x=(kocl /th > kecl ) ? x+1 : (kecl /th > kocl ) ? x-1 : x")
k5 = FrameEvaluate(k4, "x=(kopl /th > kecl ) ? x+1 : (kecl /th > kopl ) ? x-1 : x")
k6 = Frameevaluate(k5, "x=(kopl /th > kepl ) ? x+1 : (kepl /th > kopl ) ? x-1 : x")
k7 = FrameEvaluate(k6, "x=(koppl/th > kepl ) ? x+1 : (kepl /th > koppl) ? x-1 : x")
k8 = Frameevaluate(k7, "x=(koppl/th > keppl) ? x+1 : (keppl/th > koppl) ? x-1 : x")
k9 = FrameEvaluate(k8, "x=(konl /th > kennl) ? x+1 : (kennl/th > konl ) ? x-1 : x")
k10= Frameevaluate(k9, "x=(konnl/th > kennl) ? x+1 : (kennl/th > konnl) ? x-1 : x")

k17 = Frameevaluate(k10, "x=0")

k18 = FrameEvaluate(k17, "kopl=averageluma(kop)")
k19 = Frameevaluate(k18, "konl=averageluma(kon)")
k20 = Frameevaluate(k19, "kocl=averageluma(koc)")
k21 = FrameEvaluate(k20,"kepl=averageluma(kep)")
k22 = Frameevaluate(k21,"kenl=averageluma(ken)")
k23 = Frameevaluate(k22,"kecl=averageluma(kec)")
k24 = Frameevaluate(k23,"kennl=averageluma(kenn)")
k25 = Frameevaluate(k24,"keppl=averageluma(kepp)")
k26 = Frameevaluate(k25,"konnl=averageluma(konn)")
k27 = Frameevaluate(k26,"koppl=averageluma(kopp)")

result=k27


# try to telecide the video
tel=telecide(i,order=1,post=0)
tel0=tel.greyscale()
tel1=tel #.subtitle(tele)
tel2=result#.subtitle(" deblend")
flag1=blankclip(tel)
flag2=flag1.invert()

# if this fails (combing remains), then do fallback to deblend
flag=conditionalfilter(tel0,flag1,flag2,"iscombed(7)","==","false").crop(0,0,64,64)
flaga=flag.temporalsoften(2,255,255)
conditionalfilter(flaga,tel1,tel2,"averageluma()","<","62")
#conditionalfilter(flaga,last,last.subtitle("o"),"averageluma()","<","62")

}

The original thread is here: http://forum.doom9.org/showthread.php?t=82214. I don't know if the solution will work with all DEFT-conversions, but it is worth a try at least :) For my purposes, I did change the bobbing part of the function to YADIFMod + NNEDI2 compared to LeakKernelBob in that original version. Also note that the show parameter is "true" by default, you might want to change that to false.

[aand]

Some suggestions:
-change the name of the thread to "Srestore"
-adding a link to http://avisynth.org/mediawiki/Srestore in the first post

(haven't used the forum since 8 months ago, glad you're still here :D)

[osgZach]

Random question.

What does "DEFT" stand for?

[thetoof]

http://www.snellgroup.com/documents/brochures-conversion/alchemist/filmtoolsdata.pdf

[MOmonster]

@Boulder
Maybe you could upload some samples where srestore doesn't perform good. I could use them to improve srestore when I find the time for this.

@aand
I edited the first post. For the moment I'm happy with the title. Who knows when I will change the name again;).
The project srestore isn't really finished yet, but I just don't have time for this at the moment.

@thetoof
Nice PDF. If the DEFT-conversion would always work like it should do there would be no need for a restoring function.

[Boulder]

@Boulder
Maybe you could upload some samples where srestore doesn't perform good. I could use them to improve srestore when I find the time for this.Sure, I'll try to get a good sample for you when I have the time. It shouldn't be too hard as I got stuttering in the special FX parts and there's quite a few of them.

[Boulder]

@Boulder
Maybe you could upload some samples where srestore doesn't perform good. I could use them to improve srestore when I find the time for this.
It seems that I owe you an apology..in fact SRestore performs better than scharfis_brain's function. For some weird reason, the test scene I used when I compared the few DEFT repairing methods, the function performed much better than the others.

I think the problem was that I didn't set the speed parameter low enough. I set it now to -1 and the cgi scenes and the film scenes look quite good. In fact it is possible that they cannot be improved ;) Though, I only checked one episode of the first season so it might be that the later seasons have their own problems..

Here's three testclips to use. The one which has both cgi and film is an interesting one. See how the material changes between frames 806 and 807..

http://www.mediafire.com/?sharekey=9b19b691e7eb0ac9ab1eab3e9fa335ca04cb3d5388e2d538

[Boulder]

Also, should I exclude the SRestore part from multithreading (with SetMTMode) - if it uses global variables, there might be problems?

[thetoof]

Personally I'd multithread any decimation operation by splitting the clip to use multiple single threaded processes w/ trim + splice to encode.
and I don't think you'll get any problem with the global variables since they'd be called in different scripts/processes... but that'd have to be checked.

[crasus]

Any ideeas towards the removal of the ghosting in this sample? If anything can be done that is...

http://www.mediafire.com/?dyzymxmqinz

I tryied different settings and fps via srestore but no good. Main target is the original 20fps, and a simple tfm().tdecimate() should be enough if it wasn't for the damn ghosting.

Suggestions will be greatly appreciated. Thank you!

[manono]

I fooled around with it for awhile and tried a bunch of different things but couldn't come up with anything better than a basic:

Yadif(Order=1,Mode=1)
SRestore(FRate=20)

There's just too much ghosting in the source.

[jmac698]

Hello MO*,
I am trying to use srestore for ntsc->pal, to eliminate objectionable blending. Using the default works better than anything I tried before, and I even made my own filter based on a pattern I discovered.
However, there's some color artefacts left over and it's time to get into more advanced options.
However, I couldn't get it to work with the options and didn't know how to fix the results.
I am continuing this chore from some months ago, in case some bug was fixed.
I'll work on putting up a sample shortly.
If you have any parameter suggestions for this case, let me know in the meantime and I can try them.
thanks.

[jmac698]

ps
I had a thought, do you account for the different color subsampling when converting? ntsc 4:2:2 pal 4:2:0
I wonder if the conversion was done from 4:2:0 color. You should be able to do a better conversion from a better master if any pro formats use better color sampling? I can't remember right now.

[MOmonster]

@crasus
As manono said you can't do anything against the blending. You will get a pretty similar result with this:
Yadif(order=1,mode=1)
BlendBob(badFrames=0)
TDecimate(Mode=2,rate=20)

@jmac698
I really can't tell you the reason for this. Please upload a sample. I will have a look on it.

[Floatingshed]

Hello,

I'm getting an old problem rearing it's ugly head again.
"I don't know what d32 means".
It is difficult to work out why this is written across the video sometimes and not others. If it appears I simply abort virtualdub and save the file again and it will be fine. This means that I can never use fast recompress mode as I need to see the video!

Any thoughts most welcome...

[lokie538]

Not sure if this is the right place to post, but I am trying to get this script working as I have some weird source issues going on atm.

Im currently getting returned the error: "Script error: there is no function named "mt_makediff"" I presume masktools.dll is not being accessed properly?

Im trying with
LoadPlugin("C:\Program Files (x86)\megui\tools\avisynth_plugin\MaskTools.dll")
import("c:\srestore.avs")
mpeg2source("C:\tester1.hand.waving temp files\tester1.hand.waving.d2v",cpu=4,iPP=true)
srestore(frate=25)

I only added in the top line because I thought it wasn't loading properly.

Any ideas on how to get it functioning guys?

Cheers.

[Didée]

Get the up-to-date version of MaskTools: http://manao4.free.fr/masktools-v2.0a36.zip
From that package, import "mt_masktools-25.dll" (*not* the -26 one!).

[yantei]

Hi guys,

I'm looking for a suitable restoration function for my material:

progressive 23.976 fps material converted to progressive 25 fps PAL without any blends - 3 duplicate frames every second and one missing frame (?) between them

the pattern is something like this (duplicated frames):

1 3 5
26 28 30
51 53 55
76 78 80
99 101 103
124 126 128
149 151 153
172 174 176
197 199 201
222 224 226
247 249 251
270 272 274

...and so on...

the best would be to remove all duplicates and artificially generate this one missing frame every second - am I thinking right? is that possible? what would you suggest to try?

[manono]

Wrong thread for this, but how about a sample showing the problem?

[yantei]

okay - I've opened a new thread here: http://forum.doom9.org/showthread.php?p=1371516#post1371516 (the same post with a link to sample)

[Dogway]

I fooled around with it for awhile and tried a bunch of different things but couldn't come up with anything better than a basic:

Yadif(Order=1,Mode=1)
SRestore(FRate=20)

In this case, can't we use better deinterlacers? like McBob or TGMC? and then Decimate?

[lapinot]

Hi.
I use srestore to transcode&decimate from ntsc dvd to silent movie speeds, works not bad, but I get color artifacts with tinted prints on everything moving fast enough.
is there a way to "store" color info with avisynth to save before transcoding/re-apply after ?

edit : just found a way : invert().srestore().invert(). no more uninvited colors :)

[Dogway]

Well, here I come at last. I had hard time trying to figure out the problematic filter. I would like to know why this filter makes crash my encoding, strange thing is I can preview on Avsp:

tempgaussmc_beta2(2,2,3,EdiMode="NNEDI2")
SRestore(FRate=23.976)

tempgauss outputs 59.94fps right? cant I set srestore to 23.976? If i set it to 25fps or leave it blank (outputs 25fps) it works, so... how can I revert 29.970 material back to original 24 frames per second?
Is my script correct? If not the only thing left would be something with my filters in the plugin folder because I have heavily checked everything else.

[Didée]

Can't tell you about the crashing, but ... do you know at all what your source really is about? You can't just set the frate to your liking, it has to match the original framerate of the buried progressive content. In particular for blended 29.97 NTSC, destination 25.0 and destination 23.976 are two *completely different* scenarios (and should toggle different codepaths in Srestore!). 29.97->25.0 means restoring a *field*blended normconversion. 29.97->23.976 means restoring blended telecine, i.e. a 3/2 pulldown that has been *frame* blended by blur(0,1), or by progressive resize, or similar. Also, and perhaps most importantly ... if it really IS blended telecine (and 29.97->23.976 therefore is the right way), then you must not bob before Srestore ...

[Dogway]

Source is the anime piece I uploaded (http://forum.doom9.org/showthread.php?p=1375464#post1375464) a few days ago. I dont have a clue to identify what kind of source it is (would like tho), but manono said it was 23.976 so I trust. He recommended me yadif, which is in the same boat as TGMC right? Now he must have mistaken, and source is actually 25fps. I tested it and it didnt crash, but output was jerky with some dups.

When you say normconversion you mean PAL<->NTSC?
So always if it is blended telecine, like 3:2 pulldown, I would have to use TFM or... (dunno more)

Thanks for the help, it is very much appreciated. In the end if I could only know how to detect those scenarios...

[Boulder]

There is the function AnimeIVTC created especially for anime restoration. You might want to try that instead of SRestore.

[Didée]

Aaaahh ... THAT one, now I remember. Well, that is - animation ...

Quickly browsing through the long sample, it mostly looks like "progressive telecine with fieldblending interrupted by duplicates".

Impossible? No, not impossible. It's animation!

Sorry to have bothered, but from such weirdness I stay away as far as I only can.

[Dogway]

lol sorry Didee, Im a thick head but not because I want it so badly to restore, it has to be more the I want to learn no matter what (through this kind of projects).
About AnimeIVTC isnt that some kind of filters conglomeration? or does it have specific algorithms? I like kind of seeing what I do through steps...


btw. long story short "progressive telecine with fieldblending interrupted by duplicates"= blended telecine?

[Boulder]

About AnimeIVTC isnt that some kind of filters conglomeration? or does it have specific algorithms? I like kind of seeing what I do through steps...

http://forum.doom9.org/showthread.php?t=138305

[Dogway]

(...)It is not as versatile as doing everything externally(...)
.....
I will give it a try although I dont like installing 100+ filter for the sake of it

[Boulder]

You could extract all contents into a separate folder and load the plugins from there. Create a base script with the appropriate LoadPlugin lines and then just copy-paste the section whenever you need them.

[manono]

Now he must have mistaken, and source is actually 25fps. I tested it and it didnt crash, but output was jerky with some dups.
I wasn't mistaken. The 'base' framerate is 23.976fps. If you got missing frames and residual dupe frames then most likely you got the field order wrong (as I also said in that other thread).

And you can use whatever bobber you like. I used the bobbing form of Yadif for testing because it's fast, and also agreed that a better bobber was called for with that particular video.

AnimeIVTC can't do anything for that video (based on the samples provided) that a simple SRestore can't already do.

[Dogway]

Now this is funny. I encoded this script with default x264-scratchpad, and no-problemo.
I set some parameters and it crashes at second stage.

Makes me think srestore doesnt like some x264 parameters.
This is my preset file: http://www.sendspace.com/file/v2frao

edi=NNEDI2(field=-2)
YADIFMod(Order=1,mode=1, edeint=edi)
SRestore(FRate=23.976)

manono: Im with you, bobber+ 23.976 gives me best results(as preview), but it crashes like in this example. Order is 1, no mistake here. Although I read that when using it with nnedi2, you shouldnt really define it.

[Dogway]

maaaan, what stupid. I thought I was using Avisynth 2.5.7 and it actually was 2.5.8. That was messing up my MT. Now I could encode TGMC with srestore, no problem, so just forget about my last post.
I installed v.2.5.7 everything clean and neat, but the encoded video has a message hard coded in many frames saying:

Evaluate: System exception - Acces Violation (<¨o", line 96) ([ScriptClip], line 1)

Im not sure srestore is the cause, but looks like so.

tested with yadifmod and then this:

global MeGUI_darx = 257
global MeGUI_dary = 200

setmemorymax(768)
SetMTmode(5)
DGDecode_MPEG2Source("almostthere.d2v")
SetMTmode(2)
ColorMatrix(dest=2,hints=false,interlaced=true)

crop(10,0,-18,0)
tempgaussmc_beta2(2,2,3,EdiMode="NNEDI2")
SRestore(FRate=23.976)
Lanczos4Resize(704,height)
trim(10450,10575)

Distributor()

[Boulder]

Try this:

global MeGUI_darx = 257
global MeGUI_dary = 200

setmemorymax(768)
SetMTmode(5)
DGDecode_MPEG2Source("almostthere.d2v")
SetMTmode(2)
ColorMatrix(dest=2,hints=false,interlaced=true)

crop(10,0,-18,0)
tempgaussmc_beta2(2,2,3,EdiMode="NNEDI2")
SetMTMode(5)
SRestore(FRate=23.976)
SetMTMode(2)
Lanczos4Resize(704,height)
trim(10450,10575)

Distributor()


Also I'm not sure if you need Distributor. At least encoding with x264 using a command line does not require it..

[Dogway]

Yep, thank you guys, stephen "ghost" too. I remember a post talking to deal with srestore in mt (actually previous page), but I will fiddle with that in later episodes. I just want to encode this so badly lol
I use meGUI which Im not sure it needs Distributor, although HEnc seems to need it. I use it just in case.

EDIT: I made another test encode with the full script, and it appeared again in the first frames, so its not SetMode

[Boulder]

Adding Distributor "just in case" is very dangerous...