another restore function

[Monkeylord]

I'm having a hell of a time with my PAL [Adult Swim] DVD's, in particular Metalocalypse and the Aqua Teen Hunger Force movie. Both of which end up as incredibly blurry messes when I try and use AnimeIVTC on them.

Wonder if anyone in this thread could give advice.

I'll post a sample once I'm home from work (can anyone recommend a file hosting site?), but in the mean time if anyone's dealt with these before, tips would be appreciated.

[Usedocne]

File hosting site, try: mediafire.com

[Monkeylord]

File hosting site, try: mediafire.com

Ta.

I'll be home in a couple of hours and will try to get a sample up ASAP. 10-15secs worth is the norm, right?

[Usedocne]

Ta.

I'll be home in a couple of hours and will try to get a sample up ASAP. 10-15secs worth is the norm, right?

About rite, yeah. Try to find a scene w/ motion though.

[Monkeylord]

As promised, here's the samples. Would've been upped sooner, but I had to work late :(

ATHF (15secs)
http://www.mediafire.com/?beubhcv1gcfrd5i

Metalocalypse (25secs)
http://www.mediafire.com/?355rzjbo2bo9ejz

Sorry about the lengths, but I wanted to get a reasonable range of motion.

[Usedocne]

Had a quick look at your sources, seems that there field-blended, as is a lot of PAL stuff. Just so you know, I don't think you can undo all of the damage done from the NTSC>PAL conversion.

Here's all I could come up with:
interp=nnedi3(-2)
yadifmod(order=-1, mode=1, edeint=interp)
srestore(frate=23.976)
Maybe someone a bit more in the know could do better?

[Monkeylord]

I appreciate the help. It's certainly a damn sight better than what I've managed to achieve so far.

[txporter]

Had a quick look at your sources, seems that there field-blended, as is a lot of PAL stuff. Just so you know, I don't think you can undo all of the damage done from the NTSC>PAL conversion.

Here's all I could come up with:
interp=nnedi3(-2)
yadifmod(order=-1, mode=1, edeint=interp)
srestore(frate=23.976)
Maybe someone a bit more in the know could do better?

I haven't looked at the clip (host sites blocked at work), but was this originally PAL material? You should srestore back to 25fps if that is the case.

I use something like this to unblend PAL material that has been converted to 29.97fps NTSC.
o = MPEG2Source("source.d2v")
y = o.yadif(mode=1,order=1)
qtgmc = o.QTGMC(Preset="Ultra Fast",rep2=4,SLmode=1,SLrad=1)
qtgmc.Srestore(frate=25,dclip=y)

[Monkeylord]

I haven't looked at the clip (host sites blocked at work), but was this originally PAL material? You should srestore back to 25fps if that is the case.

I use something like this to unblend PAL material that has been converted to 29.97fps NTSC.
o = MPEG2Source("source.d2v")
y = o.yadif(mode=1,order=1)
qtgmc = o.QTGMC(Preset="Ultra Fast",rep2=4,SLmode=1,SLrad=1)
qtgmc.Srestore(frate=25,dclip=y)

Nope. Both NTSC originals converted to PAL... by all accounts, badly :(

[txporter]

Ok. You should be able to use the script that I linked above, but change the frate on Srestore back to 23.976 like Usedocne had. You might like the results of QTGMC a little better than nnedi3/yadifmod as the bob flicker is eliminated.

[dansrfe]

So, just to make sure I'm clear on what's happening: yadif is bobbing, QTGMC is interpolating and srestore is droping frames etc to "restore it" based on the frate and pattern detected?

[txporter]

correct

[Usedocne]

I already tried QTMC(w/ yadifmod+srestore) when I first looked @ ML's vids, but the writing on the fire extinguisher in ATHF looked messed up, and put me off recommending it.

txporter's script looks a bit different than mine though, so maybe that doesn't happen w/ his?

[Monkeylord]

Ok. You should be able to use the script that I linked above, but change the frate on Srestore back to 23.976 like Usedocne had. You might like the results of QTGMC a little better than nnedi3/yadifmod as the bob flicker is eliminated.

This showed even more improvement. Thanks. There's still a lot of blended frames, but unless you're some kind of superhuman they're barely noticible during playback.

Thanks again!

[Usedocne]

Monkeylord

I think txporter's suggestion is about as good as you'll likely get. So you can probably move on to your next step now, whatever that may be.

[DoctorM]

I've run into an interesting issue and I'd like to pick your brains. My source has fair to moderate quantities of compression artifacts. I did tests with both Srestore (with Yadifmod and NNEDI2) and Restore24.

I found that Restore24's internal deinterlacing is somewhat soft in its results. Yadifmod and NNEDI2 produced a much sharper picture. It seems because of the better deinterlacers, more of the original compression artifacts remained and caused SRestore to select more blended frames than Restore24.

Testing Srestore with original Yadif, a faster/softer deinterlacer, clean frame selection was more accurate.
It would appear that a heavily scrubbed source video allows Srestore to be more accurate.

Has anyone else come across this?
I was wondering if there is a way to get SRestore (or even Restore24), to use a strongly filtered source clip as a guide to select which fields to use, but then pull the actual fields from a clip that uses a better deinterlacer and less filtering.

[Didée]

The inbuild bob-deinterlacer of Restore24 is one big quirk.

For SRestore, there is the "dclip" parameter to feed a separate analysis clip:

a = source.sharpbob
b = source.softbob
a.Srestore(dclip=b)

Could be that Restore24 had a similar option, but .... seriously, I don't remember. :)

[dansrfe]

so as per above something like:

a = source.qtgmc([bob-double framerate])
b = source.yadif(mode=1, order=1)
a.srestore(dclip=b)

?

Also what parameters should I put in qtgmc for a field-blended PAL -> NTSC type of source to double the framerate for successful pass on to srestore?

[DoctorM]

The inbuild bob-deinterlacer of Restore24 is one big quirk.

For SRestore, there is the "dclip" parameter to feed a separate analysis clip:

a = source.sharpbob
b = source.softbob
a.Srestore(dclip=b)

Could be that Restore24 had a similar option, but .... seriously, I don't remember. :)

Ooh, sounds good, I'll definitely give that a go.

[egrimisu]

Back to chrfix.
Can chrfix called externally from srestore or within srestore?
ex animeivtc(mode=2).srestore(chrfix option if available) i checked the srestore function but found nothing
animeivtc(mode2=2).chrfix function()



WRONG THREAD, ADMIN, PLEASE MOVE THIS TO THE ANIMEIVTC THREAD. THANKS IN ADVANCE

[Boulder]

Regarding the "speed" parameter: how does it affect blended frame detection? That is, if I want to do an exhaustive search for blend artifacts in each frame, I have to use speed=-1. What are the cons of using it?

[EuropeanMan]

Quick question.

I have progressive content. However, the pattern seems to be 4P2B...B for BLENDS. not interlaced. seems as there would be ONE clean frame from those 2. would srestore be the right function or should be looking at something like cdeblend/unblend filters? it's been ages since i've seen this sort of content. let's say you see a person walking. in the last 2 frames, you would see the same guy twice, just a few millimeters apart in outline edges...same for the next frame...then to start the next 6 frame cycle, it's just one person again for 4 frames...etc etc.

here's an example...
http://i56.tinypic.com/bis5dk.jpg

[manono]

However, the pattern seems to be 4P2B...B for BLENDS. not interlaced.
SRestore doesn't handle that kind, although I don't see any reason it couldn't be modified for it. StainlessS's ExBlend is supposed to be able to do it, but I haven't had much luck. I must be doing something wrong.
The filter will also deal with similarly blended 25 FPS to 30 FPS video in which case
the blending would be as below:-

A B C D E # 25 FPS Source (5 blocks of 5 frames = 25 FPS)

A B cb cd D E # Blended to 30 FPS (5 blocks of 6 frames = 30 FPS)
http://forum.doom9.org/showthread.php?t=157337

[Didée]

For those I still use FixBlendIVTC+TDecimate, usually with decent results.

[Boulder]

While you pros hang around here, can either of you help with this one: http://www.mediafire.com/download.php?myohy405zyg ? I've not been able to get a clean result, probably because there's two different cycles depending on whether it's an outdoor or indoor shot. In bobbed stream, outdoor repeats CBCD and indoor CBCBC (C for clean, B for blend and D for duplicate of previous frame). It's a long project of mine to get that series cleaned :)

[manono]

For those I still use FixBlendIVTC+TDecimate, usually with decent results.
Quoting from the FixBlendIVTC function itself:

## FixBlendIVTC is a blend replacing / frame restoring function for doubleblends, caused by blenddeinterlacing
## of telecined sources.

But something like what EuropeanMan has wasn't telecined, at least not in the usually accepted manner, and is PAL2NTSC followed by a blend deinterlace resulting in the 4/2 pattern he described, rather than the 3/2 pattern for which FixBlendIVTC (and SRestore) work well. Did you modify it somehow to work with deinterlaced PAL2NTSC stuff?

[manono]

In bobbed stream, outdoor repeats CBCD and indoor CBCBC (C for clean, B for blend and D for duplicate of previous frame).
I don't quite see that pattern for the outdoor shot, but that car's so far in the distance it's hard to tell. The only real problem I found with the sample is that I had to set the field order or else AviSynth got it wrong and the unblending screwed up. This seemed to work OK for me:

Yadif(Mode=1,Order=1)
SRestore()

But maybe a better outdoor shot might be useful.

[Didée]

Well, what I understood is that the pattern is - 4 clean - 2 blends - 4 clean - 2 blends - etc, with one clean progressive frame "hidden" in the 2 blends? Perhaps I misunderstood?
Coincidentially, this very pattern I've seen just a few days back, in some Indian movie samples. FixBlendIVTC worked quite OK there.

[manono]

Well, what I understood is that the pattern is - 4 clean - 2 blends - 4 clean - 2 blends - etc, with one clean progressive frame "hidden" in the 2 blends? Perhaps I misunderstood?
No, you didn't misunderstand. Originally it would have been 4 clean/2 interlaced, but after the blend deinterlace it results in 4 clean/2 blended. I remember trying it a while back and it seems to work on a 5-frame cycle, and left behind a whole lot of blends. I ran it twice and it worked somewhat. Maybe I should dig out my DVD with a whole movie like that and try again.

[Boulder]

I don't quite see that pattern for the outdoor shot, but that car's so far in the distance it's hard to tell. The only real problem I found with the sample is that I had to set the field order or else AviSynth got it wrong and the unblending screwed up. This seemed to work OK for me:

Yadif(Mode=1,Order=1)
SRestore()

But maybe a better outdoor shot might be useful.

How about this one?

http://www.mediafire.com/?6qotze4u2v4l21o

I did try that earlier on, but there's quite a bit of jerky motion in the output. It is possible that the outdoor car scene is actually the same footage that was used for the first season.

I found out via Wikipedia that the second season was shot on 16 mm film, if that is of any help.

[manono]

I see now. The new outdoor scene was shot on video and converted to PAL. Try this:

Yadif(Mode=1,Order=1)
SRestore(Frate=29.97)

So, you have problems. What's the output format? DVD? Can you make an NTSC DVD, or does it have to be PAL? If for PAL DVD you can prepare the different parts separately using trims, reinterlacing the 29.97fps stuff to interlaced 25fps.

[Boulder]

My output format is MPEG-4 AVC (encode with x264) so it doesn't matter which framerate I output. So you expect that I should be able to use 29.97fps throughout the whole material? At some point I thought that I would have to go the VFR route which can be a pain since there's 19 episodes to process :(

[manono]

Well, VFR is one possibility. I don't guess you can mix progressive and interlaced the way you can with DVD, so I don't know what to suggest as I don't know much about x264. It seems a shame to turn the 23.976fps parts into 29.97fps, either by hard telecining or by adding duplicate frames. Sorry, but I'm not much help there.

[EuropeanMan]

Well, what I understood is that the pattern is - 4 clean - 2 blends - 4 clean - 2 blends - etc, with one clean progressive frame "hidden" in the 2 blends? Perhaps I misunderstood?
Coincidentially, this very pattern I've seen just a few days back, in some Indian movie samples. FixBlendIVTC worked quite OK there.

Was there a link/thread to those samples on this forum? Thanks again. I'd like to follow that thread and see what I can do.

Apparently FBIVTC/Tdecimate still resulted in residual blends here & there...

And from what I remember...perhaps I'm wrong (again Lol). 4/2 would go to 23.976, no? Or is always 25? Because 3p2i goes to 23.976...and 5i1p is 25...I'm so confused

[Didée]

3p2i is the result of field pulldown from 24p to 30i.

4p2i is the result of field pulldown from 25p to 30i.

1p5i is the result of field pulldown from 25i to 30i.


<source sample (http://www.mediafire.com/?kdaqdfsktx7otmr)> -- <Script (http://www.mediafire.com/?6g6aw5vtga3i81g)> -- <result (http://www.mediafire.com/?e2pgwj4a1n72n8c)>


BTW, you can easily make your own test footage. Take some 25p material, and do

AssumeFPS(25000,1001)
bob()
ChangeFPS(60000,1001)
AssumeTFF().SeparateFields().SelectEvery(4,0,3).Weave()

# Now perform the "BlendIVTC":
blur(0,1)

Et voilà - there's your test footage with which you can try the cleaning process.

(To make the scenario even more realistic, perform a poor-quality mpeg2-encoding of the script. The poorer the mpeg, the more realistic the scenario.) :D

[Gser]

You know you've been here too long when you start laughing at all of Didee's jokes. :D

[EuropeanMan]

3p2i is the result of field pulldown from 24p to 30i.

4p2i is the result of field pulldown from 25p to 30i.

1p5i is the result of field pulldown from 25i to 30i.


<source sample (http://www.mediafire.com/?kdaqdfsktx7otmr)> -- <Script (http://www.mediafire.com/?6g6aw5vtga3i81g)> -- <result (http://www.mediafire.com/?e2pgwj4a1n72n8c)>


BTW, you can easily make your own test footage. Take some 25p material, and do

AssumeFPS(25000,1001)
bob()
ChangeFPS(60000,1001)
AssumeTFF().SeparateFields().SelectEvery(4,0,3).Weave()

# Now perform the "BlendIVTC":
blur(0,1)

Et voilà - there's your test footage with which you can try the cleaning process.

(To make the scenario even more realistic, perform a poor-quality mpeg2-encoding of the script. The poorer the mpeg, the more realistic the scenario.) :D

Didee...thanks boss!

Next ? - IF film is shot @ 24fps, then why would the REAL SOURCE be 25fps for the above 2 examples you gave for pulldowns?

[EuropeanMan]

Didee...I ran your script on the sample you have provided. There are still blends left in the output. I didn't change anything at all. I will then assume this is the 'best' that can be done with this particular source? Will wait to hear from you tomorrow...thanks again.

[Boulder]

Well, VFR is one possibility. I don't guess you can mix progressive and interlaced the way you can with DVD, so I don't know what to suggest as I don't know much about x264. It seems a shame to turn the 23.976fps parts into 29.97fps, either by hard telecining or by adding duplicate frames. Sorry, but I'm not much help there.I could always bob the whole stream and encode as progressive. In fact, that's what is recommended if the source is interlaced.

[Didée]

IF film is shot @ 24fps, then why would the REAL SOURCE be 25fps for the above 2 examples you gave for pulldowns?
It's an Indian movie. India is PAL country with 50Hz electricity. Therefore it's no surprise if their movies start out being 25p.


I ran your script on the sample you have provided. There are still blends left in the output. I didn't change anything at all. I will then assume this is the 'best' that can be done with this particular source?
That's to be expected. When blend-removal is involved, better don't do [too much of] single-frame examination. There will always be errors and misses ... simply because real-world scenarios usually don't strictly follow the theoretical models on which the deblending-scripts are based on. Theory is clean, practice usually is polluted with all kinds of additional oddities.

The top question is whether the result is sufficiently free of judder during playback. If playback is "smooth", then it's OK. The occasional blending here and there should be accepted.

For the perfection nerds, there's always the possibility to put several days/weeks/months into manual single-frame-editing of such a mistreated source. Do so if you've no better use for your time. :p

[EuropeanMan]

I could have sworn up and down and sideways that all film was shot @ 24fps irrespective of geography :( Thanks for your help...much appreciated.

[Mark Regalo]

Yeah, I thought it was this way as well.

[Didée]

Depends on the camera with which it's shot. But in PAL countries, speedup is as common as telecine is in NTSC regions. You can't broadcast 24p on TV (traditionally, from pre-HDTV era), only 25p/50i. Same for DVD releases, which obviously are 50Hz based in PAL regions. Playing a 24p movie on a 50Hz analog TV is not funny, but true horror.

Hence, the question is not so much whether the material originally was shot at 24 or 25 fps. For what is relevant here (i.e. this thread), the only important thing is: how many unique pictures-per-second have there been before the blending-process did its work. And if it's a Bollywood movie, you can be pretty sure that it's 25fps.

[EuropeanMan]

Would it make sense then to get back to 25fps for said sources, then keep going to 24fps? Just to get BACK the original?

[manono]

I could have sworn up and down and sideways that all film was shot @ 24fps irrespective of geography.
It is. Film, as shown in a movie theater, runs at 24fps. (actually it usually runs at 48fps, with each frame duplicated). The problem with Indian films on NTSC DVD is that a 25fps master was prepared first, perhaps for broadcast, maybe for PAL DVD. And rather than prepare a new NTSC master for NTSC DVD (which costs money), they run the 25fps PAL master through a cheap standards conversion box which just ruins it.

This is also the reason why maybe 99% of Indian classic films are fieldblended on NTSC DVD - the original 15-20 year old 25fps PAL VHS tape (or even worse, the 25fps PAL VCD) was used as a source.

This isn't to say that there aren't progressive 25fps cameras out there, just that the Indian film with that ugly 4/2 pattern you described started out life as 24fps before being speeded up to 25fps, that 25fps master being used as the source for that ugly DVD of yours.

[EuropeanMan]

I hate Bollywood...simple.

[Didée]

The funniest thing is when you run across a PAL version which was blendconverted from a NTSC version. :rolleyes:

[EuropeanMan]

Please help!!!

http://www.sendspace.com/file/7s9z85
Sample above

I've done anything I can using yadif & srestore to all sorts of framerates..and still I get the blends!!! 25, 23.976, 24
What more can I do?

Didee master & others...PLEEEEEEEEEEASE. Thanks :)

[manono]

Get a different version. Mine only needed an IVTC, but I don't know who made it now. Here's a review with pics:

http://memsaabstory.wordpress.com/2008/05/16/jheel-ke-us-paar-1973/

And here's a song:

http://www.youtube.com/watch?v=f1GaH0ycAuc

I can probably find out from memsaab who made her DVD if you like. Or you can ask her yourself in the comments section.

To answer your question, though, it's 24.975/25fps and there's no way it can be properly unblended. Just count the percentage of blended fields. It looks doubly blended to me, NTSC2PAL followed by PAL2NTSC.

[EuropeanMan]

Well, I had already done the lag for restoration @ 25fps and will just work with that. Thanks Manono...i'm sure you know these DVDs don't come cheap. We didn't even have this movie in our area stores...I had to special order this movie from India...and this is what I got. Just have to deal with it :(

I'm hoping Didee will have an opportunity to check this thread out as well and give his input...

(just glad I didn't delete my lag files!)


Hmmm...wait a second. Mine is progressive blended material now...wouldn't his script above now take care of the blends I have left over???