Can this horribly field-blended PAL movie be saved?

[tekNetium]

I apologize in advance for my possibly bad English.

The video in question is a Korean movie called "Natural City" and part of this "Asian Vision"-franchise/series. I own the DVD and I am trying to make a 1CD backup of it, and this is not against Norwegian law, thereby avoiding breakage of this forums rule number 6.

Workflow (please comment if I am doing this inefficiently):

DVDDecrypter to ISO file.
Mount the ISO in Daemon Tools.
Use DGIndex to make a .d2v with no changes to the video (demux audio to be processed separately, and "Honor pulldown flags"). It's a fully interlaced PAL movie without any progressive frames what so ever.
Make an AVISynth script using the DGDecode-plugin, LeakKernelDeint, apply sharpening/denoising as needed, adjust brightness levels as I see fit, and crop away the borders. I am trying to make an anamorphic encode using the videos full remaining resolution (frame minus the borders). Thank you XVID developers for making this possible; i can retain maximum detail and see no blocks in my encodes, and thank you Bond (I think?) for that awesome custom matrice that further enhances the quality/file size-factor.
Calculate the avarage bitrate needed to meet the required final file size.
Open the .avs in Virtual Dub and "fast recompress" with XVID in two passes and no audio. The video seems to be mastered in YV12 colorspace, and it's kept that way all the way to the end (some movies I might have to convert to YUY2 depending on the AVISynth plugins used).
Downmix/compress the AC3 5.1-channel audio track into 192 Kbps stereo using BeSweet and the LAME MP3 encoder.
Mux the audio and video files into the .mp4 container format using Yamb/mp4box.
The usage of the now, hopefully perfect, .mp4-file is storage on my hard drive, playing it on my computer via MPC and FFDShow and possibly to be played on a CRT television set via my HTPC's S-video output, but that is secondary. Primary goal is sharp, seamless and blockless 1600*1200-Lanczos-upsampled playback on my computers 21-inch CRT monitor.

All things said in this post might have been off-topic, so without further adue (spelling?) I present you with the horrible DVD-mastering I now have to combat:
Extracted VOB-fragment, 12.3 MB sample clip (http://oyvindryeng.net/temp/nc.m2v).

Framegrab of said clip in uncompressed PNG-format:
Here's a link in case the forum fails to display the picture in this format. (http://oyvindryeng.net/temp/nc.png)
http://oyvindryeng.net/temp/nc.png

Now, getting to my question, and reason I made this post;
Does there exist, an AVISynth plugin that will remove the field-blending you can see in this clip (it's like this from start to finish)? I have sniffed on the RestoreFPS plugin, thinking that the PAL DVD I have in my hand was made from an NTSC DVD, but it either crashed my computer or had no noticeable effect on the clip. Hey, you never know what infinite stupidity those mastering-guys around the world are capable of - I have seen my share of badly made DVD's, but this one takes the cake. I am at a loss here, and I can not and will not accept having to make a 2CD backup of this movie due to the hard-to-compress-efficiently field-blending.

[scharfis_brain]

do not post reencoded videos.
post a unprocessed VOB or M2V sample instead.

you can try Restore24() to remove the fieldblending.

[tekNetium]

I am sorry I have failed you sir. To further embarrass myself I find myself unable to extract the section in question from the VOB files I have - I require a tool I am unable to locate in order to be able to proceed. But the tragedy does not stop there - I am also unable to find the reccomended Restore24-plugin of wich you speak, rendering me unable to attempt repair on the video clip in question.

[Guest]

Just load the VOB in DGIndex, set a range, and then save project and demux video. Post a link to the demuxed video file. We need at least several seconds of video with motion from a representative part of the VOB.

[tekNetium]

All right, here's the clip (http://oyvindryeng.net/temp/nc.m2v), in m2v-format. Please have a look at it and see if you are able to point me in the right direction. I really appreciate your help and efforts.

[scharfis_brain]

http://home.arcor.de/dhanselmann/_stuff/LostSwordship_test_scripts.rar
inside of this you'll find the AVS-function Restore24().

Your restoration script should look like this:
import("X:\Pfad\restore24_RC1_WIP.avs")

loadplugin("X:\Pfad\dgdecode.dll")
loadplugin("X:\Pfad\TDeint.dll")
loadplugin("X:\Pfad\leakkerneldeint.dll")
loadplugin("X:\Pfad\tivtc.dll")
loadplugin("X:\Pfad\masktools.dll")
loadplugin("X:\Pfad\avisynth_c.dll")
loadCplugin("X:\Pfad\smartdecimate.dll")

mpeg2source("thecage.d2v")
restore24()
crop(...)
resize(...)
other_filters_except_deinterlacers(...)

the final result will have 23.976 fps and be in sync with the audio of the DVD.

[bkman]

There were a number of plugins which you failed to mention, sharfis, like RemoveGrain() and ReduceFlicker(). And even then, it still doesn't work, with the error: RemoveGrain: Invalid mode 19 (Restore24_RC1_WIP.avs , line 181)

I'm guessing that it doesn't like the recent versions of RemoveGrain.

[Didée]

I'm guessing that it doesn't like the recent versions of RemoveGrain.
The other way round ... you're probably using the latest official version, which is not late enough. ;)
Needed is the latest v1.0 pre-release (http://home.arcor.de/kassandro/RemoveGrain/RemoveGrain.rar) of RG.

edit: BTW, once it's running, results might be slightly better when the numr/deno parameters are explicitely given, to use SmartDecimate. TDecimate's mode=7 does a darn good job, but for R24 SmartDecimate still has the edge, somehow .

[bkman]

Jeez, the plugin situation here (avisynth land) is a mess. Prerequisites everywhere that must be tracked down for everything. :rolleyes:
Anyway, I tracked down everything for Restore24, and put it up here: (down)

Just extract AvsRecursion.dll to your System32 dir, and use import("...\Restore24\Restore24.avs") in your scripts after extracting the rest somewhere.

Hopefully, the topic creator and others will find it useful.

Edit: Above host failed, so people can get it >here (http://members.optusnet.com.au/bdkon01/Restore24.7z)< instead.

[Mug Funky]

yeah, plugins can be a pain. i usually just get all of them and make them autoload, but that can be a pain with some plugins (warpsharp.dll i think).

my big pipedream project is to build a fully-fledged NLE app around avisynth, and include all plugins that get used... it'd require constant updating though. and the project is hindered by the fact that i don't know programming and haven't the time to learn.

[Didée]

Jeez, the plugin situation here (avisynth land) is a mess. Prerequisites everywhere that must be tracked down for everything.
Hey, that's part of the fun! :)


Anyway, I tracked down everything for Restore24, and put it up here: [url /url]

Just extract AvsRecursion.dll to your System32 dir, and use import("...\Restore24\Restore24.avs") in your scripts after extracting the rest somewhere.
I can NOT GUARANTEE that the script is working with these. I never used any of kassandro's filters as soon as he started to turn everything inside-out and upside-down with the new recursion idea and changed his filters to require avsrecursion.dll.
(He might have valid points why he did it, however from a user's point of view the changes in how scripts have to be written is not to my liking.)

Maybe it's okay and just works. If not, don't blame me. The script was designed using the filters of pre-avsrecursiondll era, anything newer was not tested.
(And at times Restore24 is *very* picky.)

[bkman]

Obviously people shouldn't blame you if the package I put together doesn't work, but it WorksForMe (tm) and I thought people might appreciate the leg up.

You could put a pack together yourself with the preferred versions of all the required filters, however. I'm sure that the original authors wouldn't mind as long as the proper accreditations/help files are included, and it would let a lot more people make use of your fine script (without turning the Internet upsidedown looking for the proper components). ;)

[Arlong]

I've got a problem with this function. My script is the following:

LoadPlugin("D:\PROGRA~1\GORDIA~1\DGMPGDec\DGDecode.dll")

SetWorkingDir("D:\Programmi\AviSynth 2.5\plugins\Restore24")

Import("D:\Programmi\AviSynth 2.5\plugins\Restore24\Restore24.avs")

mpeg2source("D:\Lum\VIDEO_TS\lum1.d2v")

Restore24()

crop(14,4,686,568)

BicubicResize(592,448,0,0.5)

and I get this error from VDub:

http://img4.imageshack.us/img4/2030/cattura7ew.jpg

Something involved with ReduceFlicker: even trying to use it "alone", it still refuses to load! What I'm doing wrong? :confused:

Thanks!

[Didée]

Read here (http://forum.doom9.org/showthread.php?p=708243#post708243) why I dislike "plugin packages".

In short: There are quite a few script functions from me, where people often beg "please make a package". But if I would make this for, say, 10 functions, then there will be 10 different "plugin packages", and in each of 'em the very same plugins will occur over and over again. And making "one big package" with all plugins I'll ever happen to use will become just that: a *big* package ... plusit would be annoying for those who have already most of them ... plus it would need constant updating, too, requiring re-downloading of the package all the time.

Uh, that was not exactly short. ;) But really short:

I dislike plugin packaging, period.

Or:

"The truth is out there, but you can't eat wisdom with a spoon."



@ Arlong:

My quick guess is that it's related to avsrecursion. :)

[Arlong]

@ Arlong:

My quick guess is that it's related to avsrecursion. :)

Now it works! Thanks Didée! ^__^

[bkman]

Read here why I dislike "plugin packages".

In short: There are quite a few script functions from me, where people often beg "please make a package". But if I would make this for, say, 10 functions, then there will be 10 different "plugin packages", and in each of 'em the very same plugins will occur over and over again. And making "one big package" with all plugins I'll ever happen to use will become just that: a *big* package ... plusit would be annoying for those who have already most of them ... plus it would need constant updating, too, requiring re-downloading of the package all the time.

Most people have broadband now and large hard-drives, so they don't mind some redundancy -- especially if it gives them a plugin that will work straight away without having to scour the web for the correct version of plugins by different authors. But its your choice after all...

[tekNetium]

bkman: Thank you for the file, but even with 7-Zip installed I can't open the file. I probably would have worked perfectly if it had been compressed with regular WinZip / WinRAR, but I'll never know.

Anyway, I give up. I can't even find all the required extra plugins I need to make Restore24 run. This is way over my head. I'll just make a 2CD-backup and be done with this atrocity and insult to DVD discs all over the world. I don't know if you skilled AVISynth plugin-makers want my input, but I'll give you a piece of my mind; AVISynth is obviously a powerful tool, and frameserving is the way to go when trying to edit / encode large files. Simple extra plugins like sharpeners and denoisers are a great help and of great value because they immidiately work and are simple to understand. But the large Restore24 plugin that rely on lots of other plugins is out of my scope; not because it doesen't work (it obviously do for you guys), but because i can't get it to work. There are no help-files what so ever and I have to manually track down and maybe try to understand a heap of other obcene plugins just to get it to run and I just don't have the time or sadly, the interreset to try and make it work.

I appreciate your efforts, I really do, but this is just too much. I probably could have made it work if the required plugins (and the correct version of them!) were available for download in a common .zip-format and if there had been an included helpfile that could point me in the right direction.

Best regards.

[bkman]

Zip file: http://www.insightz.net/QpIUQqWOdwxVb3xIcdS1/Restore24.zip

I understand your frustration, tekNetium, which is why I made this pack. Using 7-zip is not so hard, btw, the program just foolishly doesn't associate with .7z by default. You can enable it in the options. Newer versions of Winrar can also read 7z files.

I use 7-zip because its free, compresses better, and is now relatively widespread.

[Didée]

I just don't have the time or sadly, the interreset to try and make it work.
Well, pity.

Putting aside that *this* version of Restore24 is unofficial anyways (it's unfinished, and wasn't meant to sneak into public before it will be ... that's why support for this version is nonexistant yet), let me tell you this:

The numbers of hours I have struggled with building, improving and making Restore24 as versatile and failsafe as possible counts in thousands (no joke), for nothing but love.

Considering this, if a user states he is "not interested enough", guess how much I care? If the user only wants it for just another "eat-it-then-forget-it" flic, he should eat the flic on a TV (where fielblending is almost invisible), and then forget it. For this, Restore24 is not needed at all.

Enjoy your meal.

[tekNetium]

bkman:
Thank you for the .zip-archive, it uncompressed flawlessly.

Didée:
I am very sorry if I have angered and/or insulted you or your work; that was not my intention, and I apologize for my the Norwegianized English i have to use to articulate myself - it might not be clear-cut. I have nothing but respect for your work and efforts, I am just frustrated about all the extra bells and whistles needed to make it work. However, now that I have the files that I need I can give it a go. I will report back with my results.

EDIT:
Restore24 runs, and the results simply defies all common sense. There are practically no blends left what so ever. This result is really above and beyond all expectation.

[mg262]

I have to agree. scharfis_brain put together a package containing everything needed to run this on the first page of the Restore24 revisited thread... it's practically the first thing you find if you use search. When I tried this (many months ago) everything worked perfectly first time.

Time and again on this forum I see people simultaneously wanting to use cutting-edge tools that are still in development, and complaining that they aren't being spoonfed. I ended up putting down my own blend reversal work for this exact reason: I knew how to get it all working, with fieldblending or frameblending and treating any number of consecutive blends, but it required intelligent analysis from the user... and reading these forums tells me that is just a recipe for getting complaints and insults. So I gave up releasing plug-ins for a general audience, and now I just release things for the scripters.

Edit: tekNetium, if you get that to work, great. If not, try a non-experimental version:
http://forum.doom9.org/showthread.php?t=75432&highlight=restore24

[Mug Funky]

If the user only wants it for just another "eat-it-then-forget-it" flic, he should eat the flic on a TV (where fielblending is almost invisible), and then forget it. For this, Restore24 is not needed at all.

:goodpost:

people need to realise that there's absolutely no reason that a video should be pure progressive unless you've got broken equipment. video has been interlaced since it's inception, and will continue to be even if backerupperers don't like it. of course in the digital domain there are advantages to progressive, and film sources are better this way (if you don't mind it being a little "up-tempo").

good to hear new r24's working well though.

@ clouded:

i'd very much like to see those updated plugins that require heavy user intervention :) i might even find a way to automate them as much as possible (i work with DVD authors, not video geeks, meaning things have to be quick and simple. usually i just do the hard stuff myself, but it can get pretty hectic if you do everything yourself). the situation where blends need reversing doesn't come up often, but it can be a real pain when it does... like Bollywood films for instance - i have no idea what goes on there, but we get given NTSC masters with blends all over them and have to convert these to PAL...

[mg262]

T' isn't written yet -- or I would have PMed you about it. I'm generally better at working on things when there's a concrete example needing it... next time you run across some animated blends (assuming it's not this ecchi/hentai ;)) -- can you drop me a long sample? (Lack of long samples... 3000+ frames... is the other thing holding me up.)

[Mug Funky]

hmm. bollywood is live action... but i can send you standards-converted PAL animation by the bucketload :). i assume you like european animation more than japanese?

[Alain2]

I ended up putting down my own blend reversal work for this exact reason: I knew how to get it all working, with fieldblending or frameblending and treating any number of consecutive blends, but it required intelligent analysis from the user... and reading these forums tells me that is just a recipe for getting complaints and insults. So I gave up releasing plug-ins for a general audience, and now I just release things for the scripters.
[off topic] That's really sad, I liked your restorefps and its potentiotential; I must confess I never really took enough time to fully understand your explanations (they are not that straigthforward to me), but your work on this was appreciate :) I thought you'd stop after seeing momonster / foxy doing another deblend filter and knowing Didee will update (and publish) its r24 someday ^^