another restore function

[Didée]

Store that movie in a safe place.

Six feet under should be safe enough.

[EuropeanMan]

^ THAT is the answer I was waiting for :) LOL heheheheh

I love you man! YOU are great!

[bin_ch]

Hi,
http://www.mediafire.com/?7qbjsg4ccaq8ecx (~10MB, 10 seconds)

I see a pattern of 3 clean 2 blended, so probably telecined material blend-deinterlaced.
I used srestore(frate=23.976) but had no luck, maybe I did something wrong.

Any suggestions? Thanks.

[Didée]

To have Srestore doing double-blend removal, you need to specify something for "omode".

Though, I've tried it a few times in the past ... and everytime I reverted back to FixBlendIVTC. Dunno if that mode is half-broken in Srestore, or if I did something wrong (quite possible). In any case, FixBlendIVTC does exactly what it's supposed to do.

[manono]

I don't know. While I like FixBlendIVTC also, this seems to work OK on the sample after it locks on after the first few frames:

SRestore(omode="pp3")
TDecimate()

[bin_ch]

Thanks manono and Didée, both FixBlendIVTC and SRestore with omode work fine.

[DoctorM]

Does SRestore actually deblend frames like FixBlendIVTC? I thought it just discarded blends.

[dragon_warrior]

Hi, I need some help from you guys to restore this video

10-sec sample: http://www.multiupload.com/AR1CXA39BH

It was shot at 24p, then mastered to 25i before framerate converted again to 29.97i to be fit on the NTSC DVD. For it is BFF interlaced, I decided to give it a try with QTGMC, so this is the script I used

assumefps(30000/1000)
AssumeBFF().QTGMC( Preset="Slow")
Srestore(24)

Is there anything wrong with it? Thanks

[DoctorM]

Hi, I need some help from you guys to restore this video

10-sec sample: http://www.multiupload.com/AR1CXA39BH

It was shot at 24p, then mastered to 25i before framerate converted again to 29.97i to be fit on the NTSC DVD. For it is BFF interlaced, I decided to give it a try with QTGMC, so this is the script I used



Is there anything wrong with it? Thanks
Any reason to believe they used speed up from 24p to 25i?

Edit: Assuming there IS speed up, I get best results with this --
yadif(mode=1) #or favorite smartbobber
restorefps(50,-1) #-1 is true for this clip. 1 may be needed for the full video
srestore(frate=25)
assumefps(23.976) #I guessing there was speed up, but the audio might have been pitch corrected.

Anyone else?

[kws53]

Folks,

I'm recoding Madonna Sticky and Sweet Blu-Ray. From dga feed to avs with Yadif bobber thorugh VDmod, after careful analysis [rope jumping segment] I get the following field sequence indicating double blending:

F838 - EDIT ? ? ? ? AAB AAB B BC C CCD CDD D
F850 - E E E E F F G GH H H I I
F862 - J J JK K KKL KKL L LM M MMN MNN N
F874 - O O OP P PPQ PPQ Q Q R RRS RSS S
F886 - T T TU U UUV UUV V VW W WWX WXX X
F898 - Y Y YYZ YZZ ZZa ZZa a ab b b bcc ccd
F910 - d d dde dee eef eef f fg g g ghh hhi
F922 - i i iij ijj jjk jjk k kl l llm lmm m
F934 - n n no o oop oop p pq q qqr qrr r
F946 - s s st t ttu ttu u u v v w wwx
F958 - x x xy y yyz yyz EDIT ? ? ? ?

So 5 new frames in each 12 field segment except for Set5/6 with an overlap and hidden frame. What is the recommended treatment?

The problem includes a different start point at each edit.

Cheers,
Kurt

[manono]

Any reason to believe they used speed up from 24p to 25i?

Anyone else?
I saw no evidence it had been first converted to PAL before then converting it back to NTSC. It looks to me like it was field-blended from film directly to 29.97fps.

[dragon_warrior]

Any reason to believe they used speed up from 24p to 25i?


I got it from an encoder at mv-detox. He said most of Alexandra Burke's video were shot at 24p -> 25i -> 29.97i, the rest were 25p -> 25i -> 29.97i.
It is his encodes that make me believe him :)

[manono]

What is the recommended treatment?

None of what you wrote meant anything to me. Maybe it does to others. When asking for help the usual procedure is to post a short 10 second sample, not pictures, not a description.

[dragon_warrior]

I saw no evidence it had been first converted to PAL before then converting it back to NTSC. It looks to me like it was field-blended from film directly to 29.97fps.

Assuming what you said is correct, is this the right way to restore it
assumefps(30000/1000)
AssumeBFF().QTGMC( Preset="Slow")
Srestore(24)

Thanks

[manono]

Assuming what you said is correct, is this the right way to restore it
Sure, that's one way, although I don't quite understand the AssumeFPS(30000,1000) line as you'll then have to stretch the audio. I think I'd do it more simply:

QTGMC( Preset="Slow")#if you want to use QTGMC on it
Srestore(FRate=23.976)

Up to you, though. Maybe you have your reasons for changing the framerate a little bit.

[DoctorM]

As I said before, not knowing if there was any speed up or pitch correction of the audio, I stand by what I said before:
yadif(mode=1) #or favorite smartbobber (QTGMC isn't a bad idea)
restorefps(50,-1) #-1 is true for this clip. 1 may be needed for the full video
srestore(frate=25)
Although I've dropped the final assumefps line here. If the pitch is correct for any reason, be happy with 25p and add pulldown flags to 29.97.

Try it and see how you like the output.

[kws53]

None of what you wrote meant anything to me. Maybe it does to others. When asking for help the usual procedure is to post a short 10 second sample, not pictures, not a description.

Apologize for that - here is an link to the segment posted.
http://www.mediafire.com/file/g72m5tk4xekd9r4/madonna.demuxed.264

Cheers,
Kurt

[Didée]

Interesting. Can't remember to have seen a BR source f**ed-up like that.

What is the recommended treatment?
Return the disc and claim your money back. Point out that the content gives proof of ridiculous technical incompetence.

[kws53]

Interesting. Can't remember to have seen a BR source f**ed-up like that.

Return the disc and claim your money back. Point out that the content gives proof of ridiculous technical incompetence.

After bobbing [YADIF TFF]. it looks to me like it follows a repeatable [after the first 5 or so frames] sequence of 12 frames like this:

AAAA AAAA AABB BBBB BBBC BBBC CCCC CCDD DDDD DDDE DEEE EEEE [5 fields]

and then 2 middle sequences like:

AAAA AAAA AAAB ABBB BBBC BBBC CCCC CCDD DDDD DDDD DEEE EEEF [5 plus part of 6th field]

Wouldn't this indicate that we started at 25 [24.976]? And perhaps went through 24 on our way to 30? Any suggestions? Besides complaining to the purveyor of the disk, that is...

Kurt

[Vector13]

Hello everyone

I have looked through this thread the passed few days to try and find the best sollution to remove some ghosting from a clip. It was hard to figure out for me because i am new to this but i did my best anyway.

I became the best result with:

yadif(mode=1,order=1)
srestore()
TDecimate()

But i still get some additional "ghosting" or "dirty... things" in some frames. Some nasty leftovers from the previous frame in the next frame... I don't know how to call it. You can download a sample + script here: http://www62.zippyshare.com/v/57018655/file.html . I'm quite sure the source is bad ntsc => pal conversion as i read here a lot.

Is there any way i can get completely rid of the ghosting and/or the "dirty things" in the frames? Can anyone help me suggest anything? Thx in advance!

[manono]

I'm also seeing leftover blending and other 'dirty stuff'. I'm not sure you can entirely get rid of it. One thing though. By default SRestore gives you 23.976fps (for PAL sources) or 25fps (for NTSC sources). You don't need the 'TDecimate' at the end, and it will cause the framerate to become 19.181fps, which you don't want. Leave off that line.

And welcome to the forum!

[Vector13]

Thx for the welcome manono.

I put TDecimate() in at the end because i felt it removed more ghosting, but i didn't know it would affect framerate. Thx for that.

I'm hopeing there is something that can remove the blend leftovers and "dirty stuff", else this will have to do it i think.

[Overdrive80]

Hi, somebody help me with dvd source of this topic http://forum.doom9.org/showthread.php?t=161968

Thanks

[egrimisu]

hey guys,
i have a r2 japan anime dvd that have some blends i i try to remove it with srestore, ivtc was made with qtgmc because it has a more stable image than animeivtc.
how do i use srestore for deblend and bring the framerate back to 23.976? the script that i used and kept the framerate to 59.94 was:

qtgmc("fast")
srestore(frate=23.976,omode="pp3")

also in the srestore manuale dmode is not explained, what does dmode do?

Thanks.

[manono]

You bring it back to 23.976fps if the 'base' framerate really is 23.976fps. The field blended Japanese DVDs I've seen were made from PAL masters and the framerate was 24.975/25fps. You have to figure out the 'real' framerate first, before deciding what framerate to set SRestore for. You use SRestore's omode only if you have deinterlaced hard telecine, and not field blending. If you see interlacing, you don't want to use the omode.

For more advice a sample would be useful.

[egrimisu]

hi manono,
here is the sample http://www.megaupload.com/?d=CCB5L7J0

[manono]

You don't want to use omode with that one. I can't figure out what the framerate's supposed to be as there's not enough frames with movement every frame. I suppose you could make it 23.976fps, but to be sure you'd have to find a few seconds where every frame is unique and not a duplicate. Much of the sample is 8 or 12fps.

Oh, and I don't think you can unblend it completely as there are just too many blended fields.

[FlimsyFeet]

Hi,
Regarding this sample (http://www.sendspace.com/file/erjt15) I posted preiously in this thread (http://forum.doom9.org/showthread.php?t=161991), I found that
Bob()
Srestore()
was reasonably successful in restoring progressive 23.976fps, but if you have any recommendations for a different bob filter or tweaking the srestore parameters to get improved results I'd be interested to hear them.

[EuropeanMan]

i'm at wit's end with this source :(
here is a sample: http://www.sendspace.com/file/8pr90h

tried to restore to 23.976/25. with 23.976 i get tons of dupe frames, with 25 i get a dupe frame almost every 6/7 frames or so...
basic yadiff/srestore.

what else can you folks suggest? thanks in advance.

[manono]

It's definitely supposed to be 23.976fps. This works reasonably well:

Yadif(Mode=1)#or your favorite bobber
CDeblend()
BlendBob()
TDecimate(Mode=0,Cycle=15,CycleR=3)

[Didée]

Just tried a basic "bob(0,0.5).Srestore(frate=23.976,speed=-1)", and the result seems okay to me. I don't get anything like "tons of dup frames", the result is mostly smooth.

[EuropeanMan]

^ Didee, I had used yad/mod3 + sres/fr23 which gave me more dupes than sres/fr25...

Thanks manono...will check it out. Also will have to read on what speed=-1 would accomplish in the above setting.

edit: detail function for speed. thanks Didee :)

[EuropeanMan]

Ran both of the offered suggestions...thanks for that. I find that Didee's solution works better, especially where scene changes occur. However, manono's is QUITE fast, encoding wise :) Ran a lag w/ both against 2,001 frames and D's came out with one extra frame (1601 v 1600). OFF the top of your heads, 'why' could/did this happen? Would it be due to the bobbers used? or...

[EuropeanMan]

Off the top of my head, could something like this ALSO work?

tdeint(mode=1)
srestore()
tdecimate()
assumefps(23.976)

[manono]

Off the top of my head, could something like this ALSO work?
No. Your script after the third line gives you 20fps, and then you speeded it up to 23.976fps. The result will be a jerky video that plays much too fast. Not to mention the out-of-synch audio.

When using standard SRestore, even with Speed=-1 added, I was getting duplicate frames and missing frames about 11 seconds in with the two men in the scene. That's why I suggested what I did, something that sometimes works when the field-blending is in an unusual pattern.

The one extra frame isn't anything to be concerned over. Even for the entire movie, at most there will be a single frame difference in the final framecount, nothing to worry about.

[kws53]

I have the Jimi Hendrix Live at Monterey documentary. And like virtually all documentaries that use old video or TV footage, I have a blend of scenes with telecined material and other scenes with simple interlaced.

For example after bobbing with Yadif, the historical scenes will be the AABBBCCDDD scenario of telecining and the new interivews will be ABCD.

Is there some "easy" way of handling this? Or do I need to do a scene by scene conversion?

Thanks in advance.

Kurt

[StainlessS]

FixBlendIVTC_v091a.avs supplied in R_Pack (18/11/2009) is broken:-

http://forum.doom9.org/showthread.php?t=95924

Seems that it has been left in editing/testing mode, does not work and leaves a metric
on the clip. v0.9b in the wiki works fine.

http://avisynth.org/mediawiki/FixBlendIVTC

EDIT: the problem is in lines 135,36
ScriptClip(source, " source.subtitle(string(rvc0)) ")
#ScriptClip(source, " counter==0 ? final : counter==1 ? source.trim(1,0) : source ")
Commenting out the first line and uncommenting the second will fix the problem.

[cobo]

Is there a way to set srestore or adapt it so that it recognizes and replaces blends that use a unique frame - that is where one of the frames making up the blended frame is neither the previous or next? That's where it seems to be failing for me no matter what settings I try.

[Zep]

Normally I just throw srestore or FixBlendIVTC at this stuff and it works well with defaults but not this time. It appears the video is 1p5i/b. All i have ever seen before is 3p2i/b and 4p2i/b. Could some else please try their hand at it? I made a clip of three different areas of the video that clearly show the crappy blends.

here is the link. it is a 11 megabyte file.

http://www.fileserve.com/file/s8yjuBD/blended.avi



thanks :)


.

[Didée]

That's a 25p->30p normconversion in convertFPS()-style. Use RestoreFPS (http://avisynth.org/mediawiki/RestoreFPS) for this. Also, see here (http://forum.doom9.org/showthread.php?p=1534977#post1534977).

[Zep]

That's a 25p->30p normconversion in convertFPS()-style. Use RestoreFPS (http://avisynth.org/mediawiki/RestoreFPS) for this. Also, see here (http://forum.doom9.org/showthread.php?p=1534977#post1534977).

really I always thought the 4p2i/b was 25p FPS. This 1p5i/b didn't come from 25i? interesting...

Really really sad at how bad they convert stuff.

Anyway, thanks I will check it out.

[Didée]

The pattern is 1 clean, 5 blends, aso. The stretch is +20%, i.e. 5 input frames have become 6 output frames. 25*1.2 = 30.

Take some random 25p source, apply "ConvertFPS(30)", and compare the result to this source here. You'll see it's the very same pattern.


Just because it's HD/BluRay it isn't necessarily good quality. Over time, I've seen numerous documentations on BD that just have been blend-normconverted.
Embarrassing? Yes, embarrassing.

[Zep]

The pattern is 1 clean, 5 blends, aso. The stretch is +20%, i.e. 5 input frames have become 6 output frames. 25*1.2 = 30.

Take some random 25p source, apply "ConvertFPS(30)", and compare the result to this source here. You'll see it's the very same pattern.


Just because it's HD/BluRay it isn't necessarily good quality. Over time, I've seen numerous documentations on BD that just have been blend-normconverted.
Embarrassing? Yes, embarrassing.


It is as if they do it on purpose it is so bad.

Anyway the hard part here is gonna be locking the phase down since it am getting drift. Too bad restroreFPS doesn't have a parm for that. i.e. an amount to gradually shift by the time you get to the last frame. it appears i will need .68 at start, about .74 in the middle and about .80 near the end.


again thanks

[looney]

I try to make 50.000->23.976fps conversion. If i left sRestore defaults (in fact explicitly say frate=23.976, mode=6) everything works perfectly.
But when I try to use post-processing omodes weirdness begun:
a) "pp0" : if i use Trim() on a clip it doesnt start at actual let say 9000th frame as it did when omode is omitted
b) "pp1" : nags for RemoveGrain.dll (which is required) i had v.0.9 (http://home.pages.at/kassandro/RemoveGrain/RemoveGrain.zip) which doesnt have required mode=19, so i replace it with v.1.0 prerelease (http://home.arcor.de/kassandro/prerelease/RemoveGrain-1.0.rar). After that nags stop. But as i don't know which of RemoveGrain is required i start to renaming them but they still loads?
Later I tried different approach, by removing all RG.dlls and adding and removing one-by-one into plugins starting with SSE3 to the plain one and all behaves same as bad just plain and SSE introduces more luma highlight artifacts in some parts of frame. Plus motion shimmering is unavoidable.
c) "pp1"/"pp3" ("pp2" i didn't test) gave me some weird shimmering artifacts, or even nice chroma removal

Q: How to know which RemoveGrain is used or sRestore always uses plain nonSSE version?
A: sRestore w/ RemoveGrain call uses whatever RemoveGrain.dll it finds in plugins. But i still question how it prioritize them when you have them all into plugins

I use K10 processor, so is it maybe some of RemoveGrainSSEx.dlls broken for them?

Why when omode="pp0" is enabled Avisynth Trim() function doesn't work properly?

[Jawed]

For the sake of your sanity I recommend processing this clip in two passes:

1. srestore and nothing else
2. everything else

You can create an ultra-high quality x264 encode setting for the intermediate encoding. Just create an x264 profile with --qp 0 or --qp 1. Truthfully you would hardly see the difference if you used --qp 10, but I don't want to start an argument on that subject. Basically you want to use a fast encode setting here, you're not too worried about the file size.

Then use this intermediate encoding as the input for your second script. You should use FFMS2:

http://code.google.com/p/ffmpegsource/

to load the high quality srestored clip into the avisynth script for the 2nd pass.

This is what I normally do for srestore. It saves a lot of grief.

[Didée]

Also, note that pp=[undefined] and pp="something" will cause two completely different processing chains. The former does (single) blend removal including framerate change. This is the "NTSC<>PAL normconversion" mode. As soon as pp is used, Srestore switches to "double blend removal" mode with no framerate change, for undoing blended telecine. (Which needs to be followed by an additional decimator, for that purpose.)

Srestore is not "throw clip at it and see what happens". Srestore is "you need to know what you are doing".

[looney]

Srestore is not "throw clip at it and see what happens". Srestore is "you need to know what you are doing".

I hoped for second but not in those words :grin: But in fact it seems that i unconsciously followed them myself.

You're right i still cannot differentiate real meaning between one blend and double blend but that's probably for some deeper research (with all the links i should learn). But still what is double blend and when it occurs in practice?

So as far as i could get if I use pp0-pp3 omodes then i suppose to use mode 1-5, and not default mode 6 which is used video decimation?


For the sake of your sanity I recommend processing this clip in two passes:
1. sRestore and nothing else
2. everything else

Thx it's appreciated. But the bad thing is that in fact i don't use nothing else, well except my favorite fft3dgpu, but even with it disabled i get same chroma artifacts/shimmering.

If you refer to AviSynth builtin Trim() i thought that shouldn't collide with anything because after all it's on source.output and it's just there so i could test behave of sRestore (SR) on particular part of video clip but seems that ain't possible when SR is bothered with.

At least I learnt that fft3dgpu should be down the chain after SR if i want to avoid random implanting of subliminal frames taken from same video being processed :D Which i believe is side effect of non-linear frameserving to SR which is pretty sensitive to it (not even max cache=10 option doesn't help). So with that filtering order everything works flawlessly. Only artifact i have are from original stream, which resembles to some weird memory/buffering issues original stream had. ntm, I've resolved the same thing some few years back but i simply forget it.

With mentioning .mkv intermediate file you raised an issue bothering me and was thinking about lately for quite some time. Well, in fact in my mind there's an obstacle when i need to create large raw x264 file which afterwards needs to be muxed into .mkv stream for further processing and that requires secondary hdd to get reasonably sane muxing time of large raw stream. And that impose additional obstacle for hobbyist use like myself in this case. So some x264_cli build (which i still seek) which could directly mux raw x264 stream into .mkv during x264 encoding would be of much help So it could be avoided time-hardware consuming job that should be done afterwards.

I'd rather use lossless x264 mode than lossless huffyuv (which require avi multiple containers) if encoding+muxing could be done at the same time but i didn't found anywhere that this is possible and how. I believe you when you said that -qp 10 shouldn't show any difference from -qp 0, especially with 10bit encoder, but i'd rather use lossless x264 which is still quite computing expensive as i read about. And another bug. I shouldn't use 10bit but 8bit because i dont know how to open it for further editing later (i use ffdshow).

I'm using ffms2 (http://code.google.com/p/ffmpegsource/) (FFVS tbe) to open my huffy_intermediate.avis (though i'm well aware how old-school that is :D) as it's way more memory friendly than AviSynth's DDS. But could it open raw x264 also when you mentioning it?

I've problem comprehend all the features ffms2 gives to me thru FFVS thou i read ffms2 readme file (ffms2-avisynth.html) quite few times but don't recollect noticing any point out how to open raw stream with FFVS. Although I think I did read, few weeks back, a warning for inexperienced users somewhere that opening .mkv thru FFVS should be done only as a last resort because something is broken. So for that reason i still use DDS. I read it again and in Video support there still no mention of any raw file support nor how to open it. (I'm still using obsolete 2.16 version even thou i downloaded 2.17 more than month ago). I know that libav is contemporary holy grail for video encoding and decoding.

But to avoid further confusion: How to open raw x264 thru ffms2-avisynth? Because answer is eluding me.

And that intermediate encodes should be 8bit? Because ffdshow that i used to open x264/mkv files still doesn't support 10bit encodes? (I know i still use obsolete r3984 official but i dont like to upgrade to newer just to find that it has some major bug that affect decoding i'm using it for)

[dragon_warrior]

Hi there, anyone can help me with this clip : http://www.mediafire.com/?0qc3gvbat45tai6
The video was originally shot at 24p but it was then converted to 25i, plus some horrible blends. Could someone tell me how to figure out the suitable settings for srestore from the pattern? Thanks

[wonkey_monkey]

I've just used srestore for the first time on a 60i blu-ray that was converted from 25p, and wow - it's an almost perfect restoration without even tweaking any parameters. All I'd like to know is... how the heck does it work?! :D I had a look at the .avsi but I can't make head or tail of it...

David

[Didée]

All I'd like to know is... how the heck does it work?!
1) Identify the (bob) frames that are blends of prev+next
2) Replace the blended bob frames with a duplicate of one neighbor
3) Decimate

That simple.:)

The next question is about 1) - how to figure that a frame is a blend of its neighbors? This brings up ...
I had a look at the .avsi but I can't make head or tail of it...
Since the time of making Restore24, I have an idea and a few how such identification can be done. But I have no idea how Momonster did it in Srestore. The code is ... well, it's hard to follow, to say the least. A dozen of nameless 2-letter-variables, and miles and miles of and-and-or-or-or-and-or- logic-juggling and compairing with arbitrary thresholds. You want to reverse engineer? Me not.

It works, and that's about it. :)