another restore function [Archive] - Page 14

wonkey_monkey

19th May 2012, 21:23

1) Identify the (bob) frames that are blends of prev+next

The next question is about 1) - how to figure that a frame is a blend of its neighbors?

I guess as a start, if a frame is a duplicate, it's pixel values will be partway between the values of its two neighbours, and that will be a constant ratio across the whole frame... so far my experiments aren't going so well though :)

David

wonkey_monkey

19th May 2012, 21:27

Hi there, anyone can help me with this clip : http://www.mediafire.com/?0qc3gvbat45tai6
The video was originally shot at 24p but it was then converted to 25i, plus some horrible blends. Could someone tell me how to figure out the suitable settings for srestore from the pattern? Thanks

That sample was fixed pretty well for me just by

bob
srestore

But use a better bobber :)

David

dragon_warrior

26th May 2012, 10:32

That sample was fixed pretty well for me just by

bob
srestore

But use a better bobber :)

David

Yeah, thanks!

Elvellon

1st June 2012, 08:54

I need to restore a PAL > NTSC field-blend, but the PAL was progressive since there are 5 motion phases in 12 fields.
(6 clean and 6 blended fields, c c b c b c b b c b c b)
Should I use a smart bobber or just Bob(0,1).SRestore()?
Thanks in advance!

EuropeanMan

22nd June 2012, 05:01

i've tried tvitc, m/srestore, tdeint...and for some reason, i don't think i'm deinterlacing this source "properly"...i still get either skipped frames, or blends still remaining...

now trying clense/knockout/tdeint (nnedi)/restore...will see what happens. what do you folks suggest, because i'm quite quite certain this goes to 25fps. thank you in advance!

http://www.sendspace.com/file/fwgd6u

EuropeanMan

22nd June 2012, 05:05

(well, with what i was trying above lastly...i get better scene changes; still 2 blends in first 5 frames, but clean the rest of the way)

atra dies

22nd June 2012, 05:48

LoadCPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\yadif.dll")

yadif(mode=1,order=0)
srestore(speed=-5)

This is what I used on a ntsc to pal field blend. You have to bob and get the field order right. On this one I was getting repeats so I checked the bob and there was motion judder so I changed the field order and all was well. The field order is the order parameter in yadif, so set it to 1 for top field first and check for smooth motion. For srestore I set speed to negative for better quality.

I am using the masktools for dither and it worked. Of course you need srestore.avsi too.

manono

22nd June 2012, 11:14

I get decent results with a simple:

Yadif(Mode=1)
SRestore(Frate=24.975)

What trouble are you having, EuropeanMan bhai? :)

EuropeanMan

23rd June 2012, 07:48

^ we'll find out in about 21 hours (left on my encode)...my problems were mainly stagnant blends here & there... :( will send you my script once i find success. additionally, manono BHAI :), i don't settle for DECENT...you should know this about me by now! hehehe. i want to make sure it's the best that i can do with all of your folks' help! lastly, why is it that i NEVER think of 24,975?!!!

@ atra - ty...will look into that once this encode finishes & fails :)

martin53

28th October 2013, 21:46

Updated v2.7e from AviSynth wiki to v2.7f, to allow multiple calls from the same script, especially to compare optimum parameter settings.
For the script design to allow that, see this thread (http://forum.doom9.org/showthread.php?p=1650250#post1650250).

EDIT Nov. 12, 2013: updated to v.27g which has a one line ScriptClip() call again. Initiated by Gavino's post that every line of a ScriptClip() string is re-evaluated for each frame and all identifiers fill the AviSynth heap space up until script crashes, if clip does not end before.
Tested by comparison of two johnmeyer clips v2.7e against v2.7g (settings: frate=23.976, omode=1). All >1200 frames processed same.

ChiDragon

8th December 2013, 21:22

Can someone add 2.7g from the post above to the Wiki (http://avisynth.nl/index.php/Srestore)? I would do it myself but I can't figure out how to become an Autoconfirmed User.

Reel.Deel

8th December 2013, 21:46

Can someone add 2.7g from the post above to the Wiki (http://avisynth.nl/index.php/Srestore)?.
Done. (Srestore page needs a makeover :/)

ChiDragon

9th December 2013, 00:16

Thank ya!

martin53

9th December 2013, 07:41

Done. (Srestore page needs a makeover :/)
Thank you from my side too - the Wiki didn't allow me to upload it at first, then I didn't find the time.

Cyn you point out please about the makeover? More than updating the version text? Aside from being multi-callable form the same script now, 2.7g should behave same as 2.7e

Boulder

9th December 2013, 08:09

After these changes, is it safe to use multithreading with SRestore?

Reel.Deel

9th December 2013, 15:07

Thank you from my side too - the Wiki didn't allow me to upload it at first, then I didn't find the time.

Cyn you point out please about the makeover? More than updating the version text? Aside from being multi-callable form the same script now, 2.7g should behave same as 2.7e

Regarding the "makeover" I simply just want to reformat the wiki page to something less cluttered. Something like this (http://avisynth.nl/index.php/BlindDeHalo3) or this (http://avisynth.nl/index.php/TComb).
I would like to add a changelog and add additional links (from d9) where Srestore has been successfully used in other ways not mentioned in the documentation. Along with other small things.

Anyways, thanks for the Srestore updates. :)

martin53

10th December 2013, 19:20

After these changes, is it safe to use multithreading with SRestore?
No more or less than before. I changed 'just' two things.

- GScriptClip(local=true) secludes all RTE from other RTE scripts. This is the first condition to be able to run multiple RTE scripts without mutual interference. However as a consequence, all local RTE script variables are now forgotten after a frame is processed.

- When the script graph is made, an individual instance counter is created in advance of inserting an instance of the RTE script into the graph. Then the script string gets a string replace treatment, so instance-specific scripts are made. All variables which are needed longer than the lifetime of one frame must be global variables, and they all have an instance-specific suffix. This is the second condition to have two or more instances work independently.

real.finder

23rd March 2014, 17:26

srestore v2.7h

update to use new Average http://github.com/AviSynth/Average

to make omode="pp1" work

pastebin (http://pastebin.com/sQw9CrMi) and attachment

Motenai Yoda

18th April 2014, 18:02

update to use new Average http://github.com/AviSynth/Average

https://github.com/tp7/Average/releases

btw on my setup (2.6 Set's MT) with more than 2 threads doesn't work well on vertical pan...
either 2.7 or 2.7g or 2.7h

Neilho777

2nd May 2014, 23:41

great thanks need to restore some videos !

Oline 61

23rd May 2014, 09:24

srestore v2.7h

update to use new Average http://github.com/AviSynth/Average

to make omode="pp1" work

pastebin (http://pastebin.com/sQw9CrMi) and attachment

Does this work for anyone?
Average refuses to load with error 0x7e for me.
Using SEt's 2.6 MT.

StainlessS

23rd May 2014, 11:40

Run this against Average.dll to see what it is dependant upon (probably C++ runtime).
http://www.dependencywalker.com/

MS usually supplies it with Visual Studio.

Oline 61

23rd May 2014, 19:38

Thanks StainlessS. I needed msvcr110.dll and msvcp110.dll. Everything is working now.

StainlessS

27th June 2014, 10:11

It should be noted that the script function StrReplace() used in srestore v2.7g is bugged under Avisynth v2.58 and v2.6Alpha3
(alpha4 and alpha5 OK).
There is a fixed script version (link below), fixed script uses RT_Stats RT_StrAddStr() function which fixes the string concatenation bug,
or better still use RT_Stats RT_StrReplace() which does not create any temporary (and un-freed) strings during string replacement.
You can just replace the 'StrReplace' function call with 'RT_StrReplace' in SRestore_27g.avsi, and delete the script version of StrReplace.

It is OK to use the Script version StrReplace on short strings, but anything approaching 4KB string length should avoid usage.

http://forum.doom9.org/showthread.php?p=1596053 (http://forum.doom9.org/showthread.php?p=1596053&highlight=strreplace#post1596053)

There are a number of other scripts using StrReplace, with the same problems.

EDIT: Also affects srestore v2.7h as in post #668 by Real.Finder.

EDIT: Martin53, uses RT_StrReplace() in his thread "Runtime variables scope and lifetime" but unfortunately
used StrReplace script function in the SRestore script:- http://forum.doom9.org/showthread.php?p=1650202#post1650202

EDIT: Thread:- Concatenation failure on 4096 byte boundries, http://forum.doom9.org/showthread.php?t=164212

wea00

8th July 2014, 22:58

Hi, I have some questions regarding srestore. I'm using a basic script like this:

MPEG2Source("VTS_01_1.d2v")
o=GetParity()?1:0
LeakKernelBob(o,4,true,true)
Crop(6,98,-6,-94)
srestore(dclip=last.Crop(4,4,-4,-4))

Is the positioning of Crop in this script correct?

When starting from a source running at 25fps srestore outputs 23.976fps. My feeling tells me that the original should not be NTSC but 24fps film. I realize I can set the desired framerate using the frate parameter, but is there a simple test I can perform to find out whether the true original framerate was 23.976fps or 24fps?

I don't understand the description of the speed parameter. What is the value to use for the highest output quality? Is it 25/-25? Or 1/-1? What is the real difference between positive and negative values?

I am also confused about what bobber to use? In general it seems that QTGMC is considered to be one of the highest quality bobbers available and I saw a few people in this thread talking about using it with srestore. On the other hand I have seen a statement by Didée that motion compensated filters are bad for this task. What's the deal here?

Continuing from the previous question: If I use QTGMC, what settings should I use? (I'm not asking about the general quality settings. Mostly I am interested in what to use for SourceMatch and Lossless.) If not, what would be a recommendation for a very high quality bobber to use?

When using a computationally expensive bobber it seems to be recommended to set up a separate detection clip. What to use as the basis for that? Plain old stupid bob? Or something that's a bit smarter but still reasonably fast?

foxyshadis

8th July 2014, 23:46

I'm assuming you have a source that's field-blended to hell and back. If you think about how smart bobbing works, it's obvious why it's a waste of time with field-blending: neighboring fields can't be matched up to each other, except near the end of the cycle when the fields are near-100%. Even the exact same object from two fields away will look different and be difficult to match and even more difficult to merge in, not to mention changing the blending and therefore wrecking the blend detection and smoothness of deblending.

If you're out for maximum quality, a dumb but high quality bobber like NNEDI3 is a decent choice, or a fairly smooth smart bobber like TDeint with edeint set to NNEDI3 and thresholds set conservatively.

Now, QTGMC does a fair amount of denoising as well, making it look a bit better, but it'd be better yet to run SMDegrain on it yourself after removing the blending.

manono

8th July 2014, 23:50

realize I can set the desired framerate using the frate parameter, but is there a simple test I can perform to find out whether the true original framerate was 23.976fps or 24fps?
No. About all I do is make it the default 23.976fps and only if I get too many blends in the output will I try 24fps to see if the result is any better.
If I use QTGMC, what settings should I use?
I usually just use everything at default unless, for some reason, I want it to go faster in which case I might change to a faster preset. I also like that it does some denoising by default and will sometimes even increase the denoising as many of my sources are noisy VHS tapes. Others will do things differently and I have no interest in the sourcematch and lossless settings. I'd even go further and say if you have a field-blended source there's no point in using those settings anyway as the DVD is already a disaster needing a whole lot of work. It's almost always okay to use QTGMC as the bobber. Otherwise you might try YadifMod with NNEDI3, something like:

YadifMod(mode=1, order=1, edeint=NNEDI3(field=-2))

hello_hello

8th October 2014, 02:04

I've only recently started playing around with srestore and it certainly seems like an effective script, but I have a couple of issues with artefacts. I'm just wanting to know if this sort of thing is normal. I'm trying to un-blend some film which was de-interlaced rather than ITVC'd (I assume). I'm using the following in a script.

srestore(mode=-4,omode="pp3")
tdecimate(cycleR=1)

99.99% of it looks pretty good. There's the odd blend which isn't undone (usually around a scene change) but that's not the end of the world. Every so often though, I notice artefacts. Mostly where there's fast motion. The artefacts in the first pic are easily reproducible each time. Most of the frame is nice and clean (the pic is only of around a quarter of the frame) but where there's fast movement you can see the artefacts.

http://s7.postimg.org/6e2w2t2nf/image.jpg

The second pic shows four frames in a row. That sort of problem is easily repeatable where there's a similar fade-in.
http://s15.postimg.org/z6ls21cx7/fade_in.jpg

Every so often I've seen an artefact, but after repeating the encode it's not happened a second time, but mostly if it's occurred once, it occurs if I encode the video again. I've tried different versions of all the plugins srestore requires, and I've tried srestore versions 2.7e, 2.7g and 2.7h, and the results are pretty much the same each time, so I figure the occasional artefact is just a necessary evil. Still I thought I'd ask, in case I'm missing something. Thanks....

DoctorM

8th October 2014, 02:49

@hello_hello- What you're trying to repair doesn't sound like something srestore is meant to handle.
If your source was really deinterlaced by removing comb artifacts, I'm not aware of much you can do. The original fields and frames are probably beyond restoration.

If, on the other hand, it was deinterlaced by blending fields, which it sounds like might be the case, try FixBlendIVTC(). http://avisynth.nl/index.php/FixBlendIVTC

hello_hello

8th October 2014, 04:12

DoctorM,
Thanks for the reply. I've tried FixBlendIVTC() and it produces pretty much the same result as I get from strestore, only it's much slower for some reason. Unfortunately though, it also produces the same artefacts on occasion.

Maybe I didn't describe it well, but if I open the video in question (29.970fps progressive) and step through the frames where there's motion (someone moving across the screen while the camera remains still makes it easy to see) there's a definite 3/2 pattern. Three "clean" frames, then two frames which appear to have frame blending. I live in PAL-land so NTSC isn't something I work with a lot, but I'd assumed it means the video was originally film with pulldown, but instead of the pulldown being removed it was de-interlaced. After adding srestore to a script like this:
srestore(mode=-4,omode="pp3")
I end up with one in five frames repeated which I then remove with tdecimate(cycleR=1) and the output is 23.976fps.

Mostly it looks pretty good. Motion is fairly smooth. There seems to be the odd frame on a scene change which isn't unblended and a very occasional "skip" in movement, which looks like a missing frame, but I guess it can't be perfect. On the whole though, it looks pretty good. If I'm doing any of that wrong, please let me know. It's the artefacts I'm still trying to fix, although I'm thinking they're just a necessary evil, so to speak.

hello_hello

8th October 2014, 04:48

What exactly is the definition of "double-blend-removal"? I ask because using srestore's double blend removal seems to give me the result I want, but I can't quite get my head around why it's called 'double blend".

While I'm thinking about it, there's an example of srestore usage on the srestore page here (http://avisynth.nl/index.php/Srestore).

srestore(dmode=-4,omode="pp0")
decimate(cycle=5,quality=0) #recommed decimating

Am I missing something or is "dmode' a typo? I think it should be mode=-4.
And recommend has a missing "n". :)

hello_hello

8th October 2014, 06:13

I'm still playing around a little, but it would appear when srestore does something odd to a frame, it's a frame which didn't need restoring. For example, here's a section of three frames in a row from the original video, then the same again with srestore(mode=-4,omode="pp3") in the script.

As you can see, the first frame of the original is a "blended" frame, while the next two are "clean".

For the srestore version, frames 1 and 2 are now identical. I assume one of them would be decimated. But frame 3 has lots of artefacts, despite it being a "clean" frame to begin with. I only notice that sort of thing very occasionally, but when I do it's quite repeatable. It occurs on the same frame every time.

http://s27.postimg.org/tnnd9677n/comparison.jpg

Once again the problem "fade in". The original frame is "clean".
srestore with pp0 and pp3.

http://s30.postimg.org/wxheqaan5/comparison2.jpg

The screenshots were taken while using srestore 2.7e, but the newer versions produced a similar result.

ChiDragon

8th October 2014, 16:37

You may want to try ExBlend (http://forum.doom9.org/showthread.php?t=157337), but I think a lot of the artifacts are inherent to subtracting and sharpening frames that have already been compressed.

StainlessS

8th October 2014, 18:42

For any real kind of assistance the obligatory sample clip might be a good idea.

foxyshadis

8th October 2014, 23:36

Artifacts usually appear on areas where there's less motion and editing has made the pattern drift, or it's overcompressed enough that the matching fails. Fades done after blending are rare but do happen, and that'll break the detection. It's not possible to tell if that's the case without a sample, but overall video quality with this kind of damage is always going to be pretty rough, so you can't hope for perfect.

hello_hello

9th October 2014, 07:42

Thanks guys. For the moment, I'm playing with ExBlend, which seems to be doing quite a good job (no artefacts), so I've migrated over to the ExBlend thread (http://forum.doom9.org/showthread.php?p=1696343#post1696343), as I have a few questions regarding it's usage. Depending how I go there, I'll make some small samples to upload. My internet connection is slow at the best of times and running particularly badly today, but I'll upload some later if need be.

Cheers.

wonkey_monkey

9th October 2014, 23:26

Every so often I've seen an artefact, but after repeating the encode it's not happened a second time

For what it's worth, srestore doesn't behave consistently if you jump around in the clip. Only after playing a number of sequential frames does it settle down to consistent results.

hello_hello

10th October 2014, 02:48

For what it's worth, srestore doesn't behave consistently if you jump around in the clip. Only after playing a number of sequential frames does it settle down to consistent results.

Yeah I worked that out eventually....... :)
Actually, initially I thought srestore wasn't working at all (I tried it a while back for a different type of unblending) because I didn't see an instant change when adding it to a script and reloading the preview. Now I realise it needs to "settle down" while the video runs. It's probably why occasionally I was getting artefacts I couldn't repeat, but I have found frames where they always happen (as per the screenshots in post #682).

At the moment I'm trying FixBlendIVTC again as I think maybe it's even less "jumping around" tolerant, so I'm trying a few encodes from beginning to end to see what happens.

Boulder

10th October 2014, 07:27

The cache parameter should help with nonlinear access, but nowadays I use RequestLinear(clim=100) right before the SRestore call instead (and never use any multithreading in scripts with SRestore).

Can anyone verify whether it makes sense to use speed=-1 instead of speed=1? And also can anyone explain the actual differences between the three modes (1, 2 and 3)? I'm having problems interpreting the code :)

Motenai Yoda

16th October 2014, 23:26

Can anyone verify whether it makes sense to use speed=-1 instead of speed=1? And also can anyone explain the actual differences between the three modes (1, 2 and 3)? I'm having problems interpreting the code :)
speed goes from -25 to 25 I think difference between -1 and 1 are negible.
this change only bsize to 16 if positive and to 32 if negative.