New IVTC for Avisynth

[neuron2]

I have just released the first really good version of my new IVTC for Avisynth. It performs extremely well with default parameters, even on difficult anime such as "Cowboy Bebop". Please be sure to get Decomb Version 1.5 if you have already obtained an earlier version.

Here is a filter chain to use for tough anime:

LoadPlugin("decomb.dll")
AVISource("bebop1.avi") [or mpeg2source]
ConvertToYUY2()
Telecide
Decimate15
FieldDeinterlace

Note that the last step just catches any stray combed fields that might stray through due to input clip problems. It DOES NOT TOUCH clean progressive frames and can be omitted for clean source material (Cowboy bebop is not clean!). The help file in the distribution gives extensive instructions and tips. I think you will find the results to be truly stunning.

MMX/SSE optimizations are under way.

Tom Daniel (manono) assisted extensively with testing.

Get the software at http://sauron.mordor.net/dgraft/

Feedback will be gratefully received.

[Nic]

Im going to try & add this to DVD2AVI if thats ok...As with everything you create Donald....Very Impressive :)

Cheers,
-Nic

[neuron2]

@Nic

What would adding it to DVD2AVI entail? Do you just invoke the DLL?

My only concern is if there would be proper credit for my work and that of Tom Daniel who was an important contributor.

Also, the optimized version is under development. Would you be able to upgrade to that easily?

[trbarry]

Originally posted by neuron2
@Nic
What would adding it to DVD2AVI entail? Do you just invoke the DLL?


Donald -

Congrats, I'll check that out.

If you are branching out into Avisynth filters you might want to consider the leverage of also adding the DVD2AVI versions and it could certainly use your talent. DVD2AVI may be going to CVS/Sourceforge. See the DVD2AVI CVS (http://rilanparty.com/vbb/showthread.php?s=&threadid=11442) thread.

- Tom

[TactX]

Thanks Donald.

I've tested Decomp and the result was better than any other IVTC-filter I've tested so far. Great !

Thanks once again :)

[Tri]

Impressive! I tested it on a short part of Gasaraki and it performes extremely well, the image quality is higher than with GreedyHMA! Decomb also doesn't produce choppy output, playback is very smooth.

Good work, I'm waiting for MMX and SSE optimizations :)

[daxab]

Good stuff -- Decomb 1.5 seems to handle some cases that trip up my current favorite, GreedyHMA. But it does seem to have trouble with mostly static scenes where the only movement is an actors lips, during dialog. GreedyHMA is really good with that.

[neuron2]

I'm glad to hear that this early version is producing good results. But there is still a lot of room for improvement. So, daxab, can you please make a short clip available to me that shows the problem?

[daxab]

I trimmed things down and recompressed using HuffyYUV. You can get it here:

(=EDIT: Dead link removed=)

(Please let me know when you have the clip so I can free up the space.)

I hope this helps. Also if you want me to do any beta testing for you I'd be happy to -- I am IVTC'ing noisy source material (you'll see what I mean) every week.

[b0b0b0b]

Thanks for your work, I'm really looking forward to trying it out!

[neuron2]

I've released a new version of Decomb that does better with small moving mouths, etc. Get Version 1.6 at
http://sauron.mordor.net/dgraft/

[Tri]

OK, there is a problem. After some time of encoding, Windows closes all applications reporting there is too little virtual memory. This happened a several times.

Having a look into the task manager, I see the swapfile is constantly increasing. I have made a swapfile of 700 MB now (with 512 MB RAM) but this still happens.

This is definitely related to Decomb, because it didn't occur with GreedyHMA. It would be great if you could fix that, Donald, as the quality of Decomb is very impressive.

[Nic]

Sounds like a memory leak, but I'd never accuse Donald of anything so scandalous :)

Erm..I'd add DVD2AVI to allow DVD2AVI to have some proper IVTC code. I don't use it personally (I live in UK & only rip PAL stuff) but many people have asked for IVTC in DVD2AVI. You could have any sort of recognition you wanted (Splash Screen, MessageBox, something in the about, etc)

Unfortunatley it wouldn't be easy...id have to do a lot of fiddling with the code (& I know the source is pending so I guess i'd have to wait :) (there isn't a nice easy ->GetFrame(n) in DVD2AVI that I can use, (but im trying to make DVD2AVI more AVISynth friendly)

But if you'd really rather that I didn't try to add it to DVD2AVI than I more than respect that...

Take Care,
-Nic

[neuron2]

Version 1.7 with the fix for the memory leak can be obtained from my web site http://sauron.mordor.net/dgraft/

Sorry for the inconvenience. I'm normally not like this. :-)

[dvd2svcd]

Hi neuron2 aka Donald,

It seems to work very well indeed, and I have added support for it in dvd2svcd. I wanted to ask you, if it was ok with you, if I add Decomb to my software bundle. You will ofcourse be given full credits for the software and I will include the entire Decomb package.

[DDogg]

"OK, like, gag me, where's the attachment? I feel oh so newbie. :-("

Not your problem. Attachments have to be approved due to the the problems other boards had with cracks and such. Sorry for the cramp. :)

[Tri]

It works now! Great job! :)

[neuron2]

DON'T USE THE ATTACHED ONE!!! It is bad, bad, bad.

Use the one from my web site. Thank you.

[daxab]

Question: What will Decomb do in NTSC IVTC mode (Telecide.Decimate15 and possibly FieldDeinterlace) if given interlaced source? In other words, if I have some source that is 90% real film, but 10% video or whatever, what will Decomb do with that 10%?

[neuron2]

@dvd2svcd

That is OK. But realize the package is evolving fast. Keep an eye on it so you always have the latest and greatest. :-)

[neuron2]

@daxab

It will detect those frames as interlaced and then perform a smart deinterlace on just those frames. Please let me know if this is not working as advertised or if you want something different.

[daxab]

It will detect those frames as interlaced and then perform a smart deinterlace on just those frames.Cool. Will the Decimate15 then pick 'the two most similar' frames in a group of 5, and delete one of them? I know it won't be perfect -- you can't go from 60 fields/s to 24 frames/s without some consequence -- I'm just curious.

[daxab]

Wow, I just checked out version 1.7 and it catches all of the stray interlaced frames that were sneaking through with version 1.5.

My source material is noisy NTSC TV capture, not the easiest stuff to handle, and Decomb does a great job -- perhaps better than GreedyHMA now. When I combine this with a low-threshold deinterlace -- FieldDeinterlace(false, 3) -- I get very nice results.

[neuron2]

@daxab

I don't recommend the low threshold on FieldDeinterlace after
Telecide. You'll touch clean frames doing that. The default of
20 should be OK. You might lower or raise that a bit, but 3 is going to touch most of your good frames. Use the debug option on FieldDeinterlace together with the DebugView utility to see the effect of setting such a low threhsold. It'll tell you that most of the frames are combed when they are not.

In fact, manono found that with Cowboy Bebop he had to raise it to 30.

Yes to your question about Decimate.

Tonight or tomorrow I'll release a version that integrates Telecide and FieldDeinterace, improving peformance and making things a little cleaner for the user.

[neuron2]

I've released a new version of Decomb, version 2.0, that integrates the deinterlace postprocessing into Telecide. Forget about FieldDeinterlace; it is gone. Please refer to the help file for the new usage syntax and instructions. Get the filter at:
http://sauron.mordor.net/dgraft/

Note that the threshold is no longer problematic. It is treated exactly the same as the threshold in Smart Deinterlacer.

Your script for dirty source will now look like this:

Telecide(false,true)
Decimate15

[neuron2]

Sorry, people, Version 2.0 was bad. It had a blurry look. Get Version 2.1. That is a good release. The default threshold is a bit high, though. If you see interlacing come through, reduce the threshold to 15. I'll make that the default in the next release. I'll let this one sit for a while and get feedback.

[dvd2svcd]

Hi neuron2,

In the past you have made some extremely good VirtualDUB plugins, and I have used some of them quite a lot in the past. It now seems you're moving on to making Avinsynth Plugins. I for one would like to thank you for this. I have never had any problems with your VirtualDUB plugins, and I know that the Avisynth Plugins will be great (as your Decomb already has turned out to be).

I just had a quick look on your homepage, I was searching for a way to donate some money to you, as I find your software developments really deserves it (and since I have used it so much too). Is this possible?

[neuron2]

@dvd2svcd

Hi. Please use version 2.1 and threshold 15 for your DVD2SVCD package.

Regarding donations: I appreciate the thought but I do this stuff for my own pleasure and seeking after truth. :-) I am not in need of any funds to continue the efforts.

You can help by sending your feedback and ideas. Thank you.

[jarthel]

Is Convert2YUV line needed if I use a .d2v file? (.d2v file is already YUV)

[neuron2]

@jarthel

You can omit the ConvertToYUY2 call in that situation, yes.

[neuron2]

I've made my last release for this weekend. :-)

Version 2.2 enables postprocessing by default, changes the default threshold to 15, and adds an option for cubic interpolating the first and last fields if needed. Note that the Telecide parameters have changed, so edit your script files appropriately.

[yangus]

ok this is a really stupid question but how do i use your plugin? i never used avisynth before and so i'm kinda stuck. please give a newbe a helping hand :) and btw is this way better than tmpenc?

[Mentar]

Yangus: Learn to use Gordian Knot and read the HTML helpfile included in the decomb package, it's fairly easy then. And yes, it's way better than having it done by TMPGenc.

Donald, AMAZING filter. Just plain amazing. My kudos, I'm thoroughly impressed. Please keep it up and good luck with the speed optimizations ;)

[neuron2]

@yangus

It's really very simple. Download the version of Avisynth linked from my Decomb page at my web site. Put the avisynth DLL in windows\system (or equivalent). Then right click on the .reg file that came with Avisynth and select Install (or Merge). Now Avisynth is installed and
ready to go. Now make a file called test.avs as follows:

LoadPlugin("your_path\Decomb.dll")
AVISource("your_path\video.avi")
ConvertToYUY2()
Telecide
Decimate15

Now simply open that file with VirtualDub. Voila, you have the filtered output of video.avi.

[Blight]

And the last step should be moving decimate into telecide ;)

Save the re-loading of the frame buffer by doing the decimate checksum within telecide and save a few more CPU cycles.

[Taranli Maren]

Excuse my ignorance if this is a dumb question...

If you use this on a pure ntsc source, does it deinterlace each frame seperately, or will it actually reconstruct the film with duplicate frames. My confusion is whether or not it would be a good idea to use the decimate15 to drop it to 24 fps. if it deinterlaces each frame seperately, then each frame would end up different, with no duplicate frames. It seems that it might look smoother (and not give too much of a compression loss) to leave it at 30 fps instead of dropping a random frame every 5. But if there are actually duplicate frames, it seems that it would be better to just get rid of them. Also, if there are duplicates, does the decimate drop the duplicate, or does it just drop an arbitrary frame.

I do recall that the web site said not to use this for pure ntsc video, but I want to see if its possible to use it for this anyway, since its supposed to work so well.

Taran'li Maren

[yangus]

hey yall, thanks for answering the question. btw Mentar, u said learn 2 use gnot and i already do! so are u implying that i can use this in gnot? i thought it got its own seperate ivtc deal. and also neuron2, correct me if i'm wrong, but after installing avisynth, do i make a text file with the script u perscribed ("load dll, etc)? i read this on nicky's guide and i thought this might be what u're refering 2. and one last thing, what do u mean move decimate into telecide? okey-dokey, that's about all. later!

[Mentar]

Hi yangus, please don't take my comment as degrading, it honestly wasn't intended this way :) ... if you're using GK already, here is how to use decomb in GK:

After you hit "Save and Encode" you get the option to apply different filters like resize, noise filter etc. On the bottom of this window there's a "Edit" button. When the rest of your choices are made, hit "Edit", and it will open up a small editor with the avisynth script to edit.

First thing to do here is to comment out any ivtc commands you might have accidentally selected. Then, insert these lines (assuming that you use decomb2.2)

LoadPlugin("your_path\Decomb.dll")
Telecide
Decimate15

To verify that everything is fine, hit "Preview". If it shows clean frames, you're done. Just click on Save&Encode and everything is fine.

Good luck!

PS: Yes, in my humble opinion you should prefer decomb to the standard ivtc. It's slower, but the results tend to be significantly superior.

[dragonlz]

Has anybody tried decomb with anime? It's usually the hardest type of source to IVTC. From what I read about decomb so far, it sounds like it'll handle anime better than anything else.

[Mentar]

Yes, I did. Ran it over BGC2033 first episode and Video Girl Ai for 2 tough nuts to crack. Based on these experiences, it seems that decomb retains at least the sharpness of Greedy HMA without the occasional interlace errors which occur in it. Decomb is slow yet, but it definitely produced the best output of all three.

Everything IMHO of course.

[manono]

Hi boys and girls-

yangus- Do what Mentar said. I threw Decomb.dll in with the other .dlls in the GKnot file. Then you can just cut and paste the correct path for the LoadPlugin("......)Decomb.dll") from the GreedyHMA.dll or Wizard's IVTC or MPEG.dll. Skip that AVI Source and Convert To YUV stuff-that doesn't apply to us DVD conversion-GKnot users. AviSynth was installed when you installed the GKnot Rippack. Then put:

Telecide
Decimate15 (if Region 1 or some other NTSC area)

down in the IVTC area.

Decimate into Telecide-It's not implemented yet (as Blight said, it'll speed things up), but should be before too long (I believe).

dragonlz-I've tried it a lot. It's the best so far (not perfect)-unless the source is simple to IVTC, you'll almost certainly have to keep the deinterlacer(default in Telecide), but its beauty is that it won't screw up good frames while deinterlacing the few interlaced frames that slipped through Telecide.

Mentar-yes it's still a bit slow. It's way slower than GreedyHMA, and about as fast as the default IVTC in GKnot (since you'll need to add a deinterlacer because so many interlaced frames slip through). But if you've looked closely, you've noticed that panning and scrolling scenes are way smoother than the default IVTC, as well as allowing fewer interlaced frames through. And neuron2 hasn't begun work on the speed optimizations yet.

[triffid]

wooohoo! ive been looking forward to this!

i havent tried it yet but this looks like it may finally be the way to knock out IVTC and stray interlaced scenes at once. Like i'm sure many anime encoders are, I am tired of having to both IVTC in TMPG (averaging about 1:30 for every 1hr of video) and then run a deinterlace filter that halves the encode speed.

I haven't used Avisynth yet, as currently i use the standard Nandub method according to the official guide with a few anime-related changes. However, I'm always looking for a better and/or faster way to do things.

Mr. Graft, your v-dub filters are essential to my encode method...please make your avisynth filters essential too. :D

-triffid of DALnet, who is happy today

[ppera2]

I downloaded latest version (2.2) and recommended AVISynth 1.05b.
But I can't use plugin. It always causes access violation.

Here is AVISynth script:

LoadPlugin("F:\WinUtils\AVISynth\MPEG2DEC.DLL")
LoadPlugin("F:\WinUtils\AVISynth\Decomb.dll")
mpeg2source("F:\Tar.d2v")
Crop(24,62,672,360)
Telecide
Decimate15

Machine is OK, everything other work fine...

[dragonlz]

thanks for the info manono & Mentar. I'm encoding some anime right now using decomb to ivtc. From what I can tell by watching the .avs, it produces sharp-looking results and no interlaced artifacts have been left.

This is a really promising program, good work Donald!!

[LotharZ]

Originally posted by ppera2
I downloaded latest version (2.2) and recommended AVISynth 1.05b.
But I can't use plugin. It always causes access violation.

Here is AVISynth script:

LoadPlugin("F:\WinUtils\AVISynth\MPEG2DEC.DLL")
LoadPlugin("F:\WinUtils\AVISynth\Decomb.dll")
mpeg2source("F:\Tar.d2v")
Crop(24,62,672,360)
Telecide
Decimate15

Machine is OK, everything other work fine...
This is the correct format:
--------------
LoadPlugin("F:\WinUtils\AVISynth\MPEG2DEC.DLL")
LoadPlugin("F:\WinUtils\AVISynth\Decomb.dll")
mpeg2source("F:\Tar.d2v")
Telecide
Decimate15
Crop(24,62,672,360)
------------------

Crop after of Telecide and Decimate.

[ooze_21]

So, I live in PAL country. Is there any way of using this plugin to make 25 fps PAL (film) go back to it's original 24 fps? Look at this topic if you don't know what I mean --> http://forum.doom9.org/showthread.php?threadid=14391

Thanks
HJ

[manono]

Hi ppera2-
I don't know where Land Of Confusion is (I'm weak on Geography-near to Germany?), but for the European readers, of course you don't need the Decimate15 line, as it looks for and drops the 1 frame in 5 looking most like a duplicate. But this thing will rebuild the original frames (the way GreedyHMA can) and therefore is way better than a straight deinterlacer.

Also-do you have a Resize line in there somewhere after Crop?

As for going back from 25fps to 24fps-I don't know-maybe neuron2 can answer that one, or modify it to handle what you want, but have you tried Decimate125? I don't know if that'll work, though.

Edit: I just tried that-Access violation. So, as of now, you can't go from 25fps to 24fps. I'm not sure you want to anyway, as you'll be dropping an original frame out of every 25, and it may become a bit jerky (I think).

[neuron2]

First, the entire 2.x line of revisions was simply a bad idea and I withdraw them! Please get version 1.75 from my web site. The quality on the 2.x line was simply not up to snuff. I have to confess that I implemented a new and improved algorithm for 2.x that was in fact new and degraded.

An important note on color spaces: YUV and YUY2 are not the same! The YUV you get from DVDs and the default mode of HUFYUV has the following byte ordering:

U Y1 V Y2

whereas YUY2 has:

Y1 U Y2 V

This is really unfortunate for the current code. If you feed your YUV into Decomb, it will start working on color when it should be working on luma and vide versa. Therefore, 1.75 enforces YUY2, and this will continue until I recode for native YUV and YUY2.

Finally, I plan to integrate the functions (as I had started in the 2.x line), but using the better 1.7 algorithm.

Sorry for leading people astray with 2.x, but if you think *you* went astray, think about me! For days I've been unaware of the color space issue and you can imagine what that did to my debugging.

[ooze_21]

Well, The reason I want to go from 25-24 is that I'm not happy with the quality of current deinterlacers. Ghosting and artifacts. The picture gets jerky when I use them. Don't know if converting the frame rate to it's original would help much but...

HJ

[neuron2]

@ooze_21

Yes, I know about the requirement as I do support it in the VirtualDub Decimate filter. There are two ways to show 25 fps from 24 fps: you can just speed it up or you can add two extra fields per 24 frames. The latter requires a 1-in-25 frame decimation.

The problem in doing this for the Avisynth version is that I would have to look ahead 25 frames, which is rather a lot. (The VirtualDub filter does two passes and so only needs too look ahead one frame, storing all the comparison results and then deciding every 25th frame what to output to the script.) I am thinking about possibilites here. But for now it is not supported.

Changing the frame rate to 24 would not remove the stutter.

[neuron2]

Originally posted by Taranli Maren
Excuse my ignorance if this is a dumb question...

If you use this on a pure ntsc source, does it deinterlace each frame seperately, or will it actually reconstruct the film with duplicate frames. My confusion is whether or not it would be a good idea to use the decimate15 to drop it to 24 fps. if it deinterlaces each frame seperately, then each frame would end up different, with no duplicate frames. It seems that it might look smoother (and not give too much of a compression loss) to leave it at 30 fps instead of dropping a random frame every 5. But if there are actually duplicate frames, it seems that it would be better to just get rid of them. Also, if there are duplicates, does the decimate drop the duplicate, or does it just drop an arbitrary frame.

I do recall that the web site said not to use this for pure ntsc video, but I want to see if its possible to use it for this anyway, since its supposed to work so well.

Taran'li Maren

"Pure NTSC source" is ambiguous as it doesn't tell me whether the source is telecined progressive, simple interlaced, or a hybrid thereof. I assume you mean simple interlaced video. If so, then there are no progressive frames to recover. Using Decomb rather than Smart Deinterlacer buys you nothing. Only when there are progressive frames in the stream should you use Decomb. If I haven't understood you properly, please clarify your question.

[ppera2]

Originally posted by manono
Hi ppera2-
I don't know where Land Of Confusion is (I'm weak on Geography-near to Germany?), but for the European readers, of course you don't need the Decimate15 line, as it looks for and drops the 1 frame in 5 looking most like a duplicate. But this thing will rebuild the original frames (the way GreedyHMA can) and therefore is way better than a straight deinterlacer.


Is it so strange that someone in Europe performs IVTC on R1 DVD's ?

[manono]

Hi-sorry-I didn't know they were readily available. I'm just a provincial American.

[yangus]

ok people, i tried decomb out using the gnot way Mentar provided. however when i tried to preview it, all it did was give me an error message. first time, it played a short avi file that said "... command not valid" or something like that. then the second time it just gave me an error. this is basically what it said when i clicked edit: (sorry for the long post)
#
# Created with Gordian Knot
#
# http://thewef.nav.to
#
# PLUGINS
# get them from http://users.win.be/dividee
LoadPlugin("C:\DOCUME~1\Angus\Desktop\DVDRIP~1\GNOTON~1\GORDIA~1\mpeg2dec.dll")
#LoadPlugin("C:\DOCUME~1\Angus\Desktop\DVDRIP~1\GNOTON~1\GORDIA~1\InverseTelecine.dll")
#LoadPlugin("C:\DOCUME~1\Angus\Desktop\DVDRIP~1\GNOTON~1\GORDIA~1\Avisynth_Spatial.dll")
#LoadPlugin("C:\DOCUME~1\Angus\Desktop\DVDRIP~1\GNOTON~1\GORDIA~1\GreedyHMA.dll")
#LoadPlugin("C:\WINDOWS\System32\vobsub.dll")
#
# SOURCE
mpeg2source("C:\Documents and Settings\Angus\Desktop\Dvd Rip\Emperor and Assasin\emperor.d2v")
#
# TRIM
#trim(startframe,endframe)
#
# IVTC
#InverseTelecine(40,10,15)
# or use
#GreedyHMA(1,0,4,0,0,0,0,0)
#
# CROPPING
crop(0,8,720,461)
#
# DEINTERLACING
#SmartDeinterlace(2,15,true,true,true)
# or use
#VerticalReduceBy2
# or maybe
#GreedyHMA(1,0,0,0,0,0,0,0)
#
# SUBTITLES
#VobSub("FileName")
#
# RESIZING
BicubicResize(544,288,0,0.75)
#
# DENOISING: choose one combination (or none)
# 1) little noise (fast)
#TemporalSmoother(2,1)
#
# 2) medium noise (slow)
#SpatialSoftenMMX(1,4,6,false,false,4,4,6,8)
#TemporalSmoother(2)
#
# 3) heavy noise (very slow, you have been warned)
#SpatialSoftenMMX(2,4,6,false,false,4,4,6,8)
#TemporalSmoother(3)
#SpatialSoftenMMX(1,4,6,false,false,4,4,6,8)
#
# BORDERS
#AddBorders(left,top,right,bottom)
#
# COMPRESSIBILITY CHECK
# !Snip Size has to be 13 for use in GKnot!
#SelectRangeEvery(260,13)
#
# FOOL CCEnc
#ResampleAudio(44100)

so my question is do i delete the line about loading the greedhma plugin and replace that with decomb? and under ivtc do i delete that line altogether? and where do i insert telecide and decimate? i'm sorry, but i'm kinda slow at this. and just to double check, i tried the vdub way by loading the test.avs file. it also gave me an error message! what to do, what to do... oh one last thing, is it possible to perform decomb as a seperate job instead of part of the encoding. the reason i ask this is b/c i usually do audio and ivtc (using tmpenc) on one day and encoding while i'm gone at school. this is a lot better than doing this all at once since i usually need to use my computer after school and encoding on a duron650 takes hours. u get my drift? it would be good if i can do ivtc seperatly and allowed to resume (tmpenc once again) anyway sorry for the long post but i really needed to get this answered!!:confused:

[hakko504]

It should look like this:

#
# Created with Gordian Knot
#
# http://thewef.nav.to
#
# PLUGINS
# get them from http://users.win.be/dividee
LoadPlugin("C:\DOCUME~1\Angus\Desktop\DVDRIP~1\GNOTON~1\GORDIA~1\mpeg2dec.dll")
#LoadPlugin("C:\DOCUME~1\Angus\Desktop\DVDRIP~1\GNOTON~1\GORDIA~1\InverseTelecine.dll")
#LoadPlugin("C:\DOCUME~1\Angus\Desktop\DVDRIP~1\GNOTON~1\GORDIA~1\Avisynth_Spatial.dll")
#LoadPlugin("C:\DOCUME~1\Angus\Desktop\DVDRIP~1\GNOTON~1\GORDIA~1\GreedyHMA.dll")
#LoadPlugin("C:\WINDOWS\System32\vobsub.dll")
#Put the Decomb filter in the same directory as the other plugins
LoadPlugin("C:\DOCUME~1\Angus\Desktop\DVDRIP~1\GNOTON~1\GORDIA~1\Decomb.dll")
#
# SOURCE
mpeg2source("C:\Documents and Settings\Angus\Desktop\Dvd Rip\Emperor and Assasin\emperor.d2v")
#
# TRIM
#trim(startframe,endframe)
#
# IVTC
#InverseTelecine(40,10,15)
# or use
#GreedyHMA(1,0,4,0,0,0,0,0)
# or in this case
Telecide
Decimate15
#
# CROPPING
crop(0,8,720,461)
#
# DEINTERLACING
#SmartDeinterlace(2,15,true,true,true)
# or use
#VerticalReduceBy2
# or maybe
#GreedyHMA(1,0,0,0,0,0,0,0)
#
# SUBTITLES
#VobSub("FileName")
#
# RESIZING
BicubicResize(544,288,0,0.75)
#
# DENOISING: choose one combination (or none)
# 1) little noise (fast)
#TemporalSmoother(2,1)
#
# 2) medium noise (slow)
#SpatialSoftenMMX(1,4,6,false,false,4,4,6,8)
#TemporalSmoother(2)
#
# 3) heavy noise (very slow, you have been warned)
#SpatialSoftenMMX(2,4,6,false,false,4,4,6,8)
#TemporalSmoother(3)
#SpatialSoftenMMX(1,4,6,false,false,4,4,6,8)
#
# BORDERS
#AddBorders(left,top,right,bottom)
#
# COMPRESSIBILITY CHECK
# !Snip Size has to be 13 for use in GKnot!
#SelectRangeEvery(260,13)
#
# FOOL CCEnc
#ResampleAudio(44100)

[manono]

If for some reason you can't do it in Edit while still in GKnot, then you can open the .avs later with notepad or something, and insert the lines hakko504 provided you (I guess you saw the note from neuron2 above that says to go back to an earlier version, so you'll need the FieldDeinterlace command if you want to deinterlace (after putting the v 1.75 .dll in GKnot)).

Funny-I just encoded The Emperor And The Assassin a week ago. A wonderful movie (Gong Li should have had more on screen time, though).
I'm in love with Gong Li.

As for doing it separately, you have to do it while encoding (not like TMPGEnc). I might suggest doing the first pass one one day, and the second pass on the next day, or overnight. That's such a long movie that it'll take longer than usual.

[ppera2]

Hi manono,
actually we have here lot of Chinese DVD. It's NTSC, and force film works in rare cases.

I moved crop line after decomb ones, and now it works...

However I don't like it. It should be faster when work with smaller size. IVTC plugin work fine when crop is before.

[neuron2]

Version 1.8 has been released. It greatly improves deinterlacing of frames detected as combed, and offers an option for forcing interpolation of the first and last frames.

NOTE: PLEASE DISCARD VERSION 2.x. IT HAS BAD QUALITY!!!
Version 1.8 is the cat's meeow for output quality and faithfulness to the original progressive frames.

@ppera2

What is the size of your cropped frame?

[ppera2]

neuron2:

Size after crop is 672x360. I tried also with 352 vertical (div. by 32), but got same access violation error.

[Krajensky]

Donald,

Wonderful job...I have tried every IVTC out there today (on TV episodes...Star Trek: Voyager to be precise), and yours is the first to do the job without an explicit deinterlace...they change the IVTC pattern on every single scene change and it gets to be rather annoying when there's one that sneaks through as interlaced. I have yet to notice stray frames getting through (I was using 2.2, just now upgraded to 1.8 ), which easily compensates for the speed. The only question I have is this: With other filters, you could seek over a given area of video, to make it cache it, then you could play that segment back very quickly...your filter seems to not cache its decoded frames...is this on purpose?

Krajensky

[TactX]

I got an access violation too when cropping before telecide.

I thought this was caused by cropping odd values.

btw: Windows 2000, Avisynth 1.05...

[b0b0b0b]

I have 2 questions for neuron2:

1) decomb is faster than smartdeinterlace because it works in the YUV colorspace, right? Thus, when encoding a dvd through avisynth there's no need to go to RGB and back at any stage.

2) when deinterlacing, suppose that at a scene change, the frame is composed of fields from disparate scenes. For example, the top field is from scene a but the bottom field is from scene b. Does smartdeinterlace simply merge the two? Doesn't this force the divx codec, which is downstream, to mark this as a keyframe? Isn't this a bad thing because either a) this is the keyframe for scene b, meaning that the next delta frame will need to make up for the munged information from scene a, or b) this frame is a keyframe, but so is the subsequent frame which is composed entirely of fields from scene b.

Thanks very much!

[neuron2]

@ppera2

I am investigating this issue. Please standby...

@Krajensky

The frame cache is maintained by Avisynth. It is a limited size. If you scan around within a reasonable range of your current frame, then you'll get cached frames. You can see the cache behavior by enabling debug in Telecide and using the DebugView utility (from my web site). Start the video and then stop. Backup one frame and you won't see Telecide recalculate. Keeping backing up and you'll run out of cache and Telecide will recalculate. The filters don't do any caching; it is all the Avisynth kernel.

@b0b0b0b

I'm still investigating color issues. But yes, Decomb operates in YUY2 space. The VirtualDub Smart Deinterlacer runs in RGB32 because all VirtualDub filters must be RGB32. So Decomb will be faster (even though it is doing a little more work to decide whether frames are combed before deinterlacing them). It appears that in the VOB files, the coding is UYVY and that mpeg2source() converts to YUY2. That means you don't need a ConvertToYUY2 when doing rips (though it will be ignored and has no effect on speed). The only time you need ConvertToYUY2 is when the source is in RGB32 or RGB24 format. When encoding a DVD from a VOB, there is no RGB conversion. But NEVER use DVD2AVI to save the VOB content as an RGB AVI file, because then you will have the conversion to YUY2 to do. Clear?

When there is a field that can't be matched, then the frame is smart deinterlaced (either by blend or interpolate). I do not know anything about the keyframes issue. But I don't think that the YUY2 coming out of Decomb has keyframes. Won't DivX decide what to make keyframes? Pease investigate and tell me. Thank you. :-)

[Krajensky]

@neuron2:
I tried debugview on 1.8: I get about 37 frames cached, when going forward then seeking back until I see a new request for a set of frames. Using the InverseTelecine v2.101 plugin, I get 62-ish frames cached...any ideas?

Here's my script for decomb
LoadPlugin("c:\avisynth\decomb18.dll")
SegmentedAviSource("voy_3.avi")
Telecide(false, false, true)
Decimate15

and for inversetelecine
LoadPlugin("c:\avisynth\inversetelecine2101.dll")
SegmentedAviSource("voy_3.avi")
InverseTelecine(40,10,15)

If Avisynth is keeping all the necessary frames in the cache, why is it so much smaller? Is this due to the fact that your plugin uses two commands, and the other only uses one? Wouldn't it still cache either the input or output frames, which are of the same number?

Krajensky

[neuron2]

I've released version 1.81 at my web site:
http://sauron.mordor.net/dgraft/

This version fixes ppera2's Crop bug and is about 10% faster.

When a source frame is cropped, the Crop filter simply reduces the row size but leaves the row pitch as is! That means when Telecide gets a source frame from Crop it has the pitch of the original uncropped frame. But frames allocated within Telecide get a pitch equal to the cropped size. So my loops had to use separate pitches and the previous code just assumed that they would both have the same pitch. Something for you Avisynth coders to watch out for.

Some simple changes gained me 10% in performance. I haven't really tried to optimize seriously yet, because I think it's a mistake to paint oneself into a corner too early.

I'm really happy with this version.

[neuron2]

@Krajensky

I believe the frame cache difference is explained by the fact that Telecide opens three input frames from its source whereas IVTC and Greedy open only 2. The ratio of 37/62 is not too far from 40/60, which would be 2/3. The reason Telecide opens three frames is because he wants to match against the previous, current, and next frames. This allows the field matching to live through many strange things at cuts and scene changes, as well as eliminating the need for specifying top first/bottom first. This is one of my little secrets.

I haven't looked at the Avisynth code to verify this theory, but it seems reasonable.

Greedy came from the real-time DScaler world, where the extra field matching would be serious for sustaining the frame rate. My perspective is to seek the highest quality encodes, so the extra time is acceptable. As it is I am currently at about 80% of the speed of Greedy and I haven't applied a few obvious code rearrangements and MMX/SSE optimization yet, so things look promising on the speed front.

[samfchen]

Hello,

I read the past posts pertaining to getting the Decomb filter to work with Gknot, but I am still a little confused. Source is of Asian origin (Anime'ish), NTSC, Interlaced, 29.xxxfps. ForcedFilm under Dvd2Avi does not produce good results.

Thus to use Decomb for DVD2AVI (to feed into Gknot), do I simply leave the video field operation to be none and create the .d2v, then feed into Gknot. And in Gknot, do I select 23.976 (though it is shown as 29.xxx)? Do I need to (can I) use any resize filters or does the Decomb package take care of that with Telecide and Decimate15?

Sorry for the questions, but help is greatly appreciated.

Thank you,
Sam:)

[DJ Bobo]

1) When you have NTSC Interlaced 29,97fps showed in DVD2AVI, you CAN'T use Forced Film, so leave it unchecked!

2) You should select 23,976fps in GKnot (but it's not that important)

3) Decomb doesn't resize, so you must use resize filters.
NB: you should crop and/or resize after Decomb

[samfchen]

Thanx for the help, bobotns.
Greatly appreciate it.

you mentioned at the end Crop and Resize AFTER Decomb (when used with Gknot + Decomb), why? Why can't I simply resize it with the Gknot interface and simply apply Decomb via editing the Gknot file? And if I don't resize via Gknot, doesn't that mess up the filesize approximation built into Gknot? Assuming I didn't resize via Gknot, what program do I resize the Decomb'ed Divx avi with?

Thanx in advance,
Sam:)

[Blight]

I would like to make a feature suggestion.

Some sources have hybrid 24/30fps sources. At times even progressive 30fps (CGI Video like in Babylon 5).

Now, the problem is thus, if you decimate 1:5, you may lose actual progressive frames from the 30fps portion and the video will become jerky. If you don't decimate, the fields on the 24fps portion will match and then you'll have a duplicate frame and now your 30fps video will be jumpy since you'll have duplicate frames on the 24fps portion.

The solution ... add a switch that once a duplicate frame is detected in a 5 frame segment, instead of keeping 2 duplicated frames, you dump one of the duplicated frames and cubic interpolate between two frames so that motion will maintain fluidity.

[DJ Bobo]

@ samfchen
You better use Decomb before resizing for optimal quality.
You can edit your AVS file so that the crop and/or resize commands come after Decomb. For example:

LoadPlugin("path\mpeg2dec.dll")
LoadPlugin("path\decomb.dll")
mpeg2source("path\project.d2v")
Telecide
Decimate15
crop(8,0,704,480)
BilinearResize(640,480)

[neuron2]

@samfchen

You can crop before Decomb but you can't resize. Decomb needs the original line spacing to detect the combing artifacts.

@Blight
If you mean make a temporally interpolated frame, that opens a giant can of worms. It is a really expensive computation. And from what I have seen of such interpolators, they are not good enough, especially with 1 in 5 frames being done; it would be very noticable. It's a great idea in theory, but the practice is sobering.

[western shinma]

Having some problems with 1.81. It always gets an access violation after 8000 frames or so, encoding in VirtualDub. Here's the avs:
LoadPlugin("MPEG2DEC.dll")
LoadPlugin("Decomb.dll")
MPEG2Source("temp.d2v")
Crop(0,57,720,366)
Telecide
FieldDeinterlace(false)
Decimate15
BicubicResize(640,272,0,0.5)
2.x worked fine in this case, although the new version is noticeably faster.

[ppera2]

I tried Ver. 1.81 . IVTC with cropping before. It worked fine with sample of ~11000 frames.
I will try whole movie 2 pass during night...

[daxab]

Did a 5-pass with 1.81, no problems.

[western shinma]

Yes, it works fine if I don't crop first. It is a bit slower this way however.

[neuron2]

@western shinma

I duplicated your problem and fixed it, I believe. Also, I added a decimate 1-in-25 function.

Please get Version 1.82 from http://sauron.mordor.net/dgraft/

Thank you for your feedback and sorry for the trouble it has caused you. We are converging on a clean, stable release, methinks.

[western shinma]

No trouble really :D, just testing some things out. I'll give it a try tonight.

[Blight]

neuron2, even doing a 50% blend between the two frames would be better than a duplicated frame (as far as fluid motion goes).

[neuron2]

@Blight

OK, I'll run an experiment and see how it looks.

[neuron2]

I had a user submit a clip that had been captured with a bad capture card that was delivering many out-of-sequence fields, but fortunately always in the top field. Decomb in its default behavior vomited on it because the field matching failed. FieldDeinterlace could have been used, but there was a better way! Following is my response to the user. I will add this situation and resolution to the help file.

----------
Hi Jussi,

I got your file and diagnosed the problem. Your capture card or software is broken. But you are in luck as Decomb can work around the problem! First the analysis...

The card is not delivering fields in correct and reliable order. You can see that from separating fields and examining them.

Here is an analysis of a short section I looked at. The number 1 refers to a picture where a man sits at his desk. 2 refers to the next picture which is a closeup of him. b and t refer to top and bottom fields. These are the five frames:

b1/t1 b1/t1 b2/t2 b2/t1 b2/t2

The field t1 is wrong in the fourth frame. It should be t2. How t1 got there is anybody's guess, but most likely a capture card problem, unless you've processed this with some other software. It so happens that all the bad fields are in the top field, so you can trick Telecide into matching on the bottom instead of the top, and that makes your clip play correctly. Here is the script:

LoadPlugin("d:\don\programming\c++\avisynth\mpeg2dec.dll")
LoadPlugin("d:\don\programming\c++\avisynth\decomb\release\decomb.dll")
mpeg2source( "decomb_problem.d2v" )
ConvertToYUY2
SwapFields
Telecide(true)
Decimate15
##FieldDeinterlace(false,15,false)

Thank you for submitting this interesting problem. It extends my knowledge and experience. :-)

best regards,
Don

[DDogg]

We are going to have to start calling you, "Gentleman DG" :)

BTW, that is a compliment lol

[neuron2]

@DDogg

My mommy taught me to treat others as I would like to be treated. LOL

Actually, it is just a result of the awe and deference that I feel is merited by such an august group of people as yourselves.

[daxab]

If you have 24fps source material that has been telecined and then run through telecide, you get 30fps material with one duplicated frame out of every 5. The frames, in time, look like this:

|----|----|----|----
0....1....2...3,4...

3 and 4 are the duplicates, in this example.

Decimate15 deletes the duplicate and gives you this:

|----|----|----|----
0....1....2....3....

The frames are spaced evenly in time.

If, instead, the duplicate frame is deleted and a blended (or interpolated) frame is inserted, you get this:

|----|----|----|--|--
0....1....2....3..x..

Frame x, the blended frame, is not evenly spaced in time. When you play these frames back at 30fps, you will get a glitch every 5 frames.

[DDogg]

"Actually, it is just a result of the awe and deference that I feel is merited by such an august group of people as yourselves."

I could never spell galoshes :D

[neuron2]

@daxab

That doesn't look right to me. Would you not get from Telecide:

|----|----|----|----|----|----
0....1....2....3....3....4....

and then if you decimate normally:

|----|----|----|----|----
0....1....2....3....4....

if you "decimate" with blend:

|----|----|----|----|----|----
0....1....2....3....3/4....4....

[daxab]

@neuron2

Yes, I agree -- I tried to say that but my notation was confusing. I am a poor ASCII artist.

I think the problem with the decimate+blend idea is that the timestamps will be off.

Starting with 24fps film, we have:

0-000ms @ 000ms 24fps
1-042ms @ 042ms
2-083ms @ 083ms
3-125ms @ 125ms
4-167ms @ 167ms
...

On the left is the timestamp of each frame when it is supposed to play, to recover the original motion. On the right is the timestamp when it will play. Since we haven't don anything yet everything matches up. During telecining, this is split into fields, recombined , and comes back out as 30fps. I won't draw this but most people can picture this in their heads.

Telecide reconstructs the original frames and yields a duplicate. So we have:

0-000ms @ 000ms 30fps
1-042ms @ 033ms
2-083ms @ 067ms
3-125ms @ 100ms
3-125ms @ 133ms
4-167ms @ 167ms
...

This won't play right, as is, because the timestamps are wrong -- they don't match our original film frame timestamps.

Decimate15 deletes one of the extra frames numbered 3 and adjusts us back to 24fps, so we get the original picture again, which does play correctly.

0-000ms @ 000ms 24fps
1-042ms @ 042ms
2-083ms @ 083ms
3-125ms @ 125ms
4-167ms @ 167ms
...

But if, instead, we do a decimate with blend, we'd get something like this:

0-000ms @ 000ms 30fps
1-042ms @ 033ms
2-083ms @ 067ms
3-125ms @ 100ms
X-146ms @ 133ms
4-167ms @ 167ms
...

X is the new frame, interpolated between 3 and 4, so it has a timestamp between that of 3 and 4.

This might be marginally better than using the result of Telecide (with no Decimate) but not much -- all frames still are playing at the wrong time. I think the result would be very similar to viewing the output of Telecide, but the periodic hitch in the motion would look different (but still be there).

[yangus]

hey yo thanks for all your help! sorry for the late reply, since i just finished finals today and i'm dead tired (yea i pulled a couple of nighters...) anyhow i'm gonna try out the suggestion tomorrow on this particular anime title (anyone heard of kenshine?) :D btw someone mentioned that chinese films rarely looked good with force film but isn't that not true? when i did preview under dvd2avi, it was 95% film most of the time, so doesn't that mean ivtc's not necessary? well anyway i'll let u guys know how it turns out. later!

[Tri]

I have a problem... When I use 'FieldDeinterlace' I get some frames with strange lines. This only happens with decomb 1.8 and later, but 1.7 works fine.

Also, it doesn't happen without the 'FieldDeinterlace'-command.

[neuron2]

@tri

You're not giving me enough data to help you. "Strange lines" tells me nothing. You need to mail me a short clip that demonstrates the problem, including the script you are running. Make sure you send the input file, not the processed file. Better yet, make it available for download and post the URL here.

[Tri]

Ah sorry, it's nothing to do with decomb, it's just that avisynth suddenly doesn't want to resize anymore and the spatialsmoother doesn't work anymore... :angry::confused:

[neuron2]

There is a new release of Decomb. Version 1.9 combines Telecide and FieldDeinterlace to give a small speed increase and prepare the ground for optimizing. Also, Decimate now supports 1-in-N, where N can be set from 2-25. Please note that the syntax changes, so read the new help file carefully and revise your scripts accordingly. You will no longer need to follow Telecide with FieldDeinterlace, and Decimate15 will become simply Decimate (add a parameter if your decimation differs from 1-in-5).

Get the new release from http://sauron.mordor.net/dgraft/

A few speed tips: 1) If not required, omit ConvertToYUY2 from your scripts. If it is needed, Telecide will tell you. Read the help file for more details, and for how to make HUFYUV convert to YUY2. 2) When serving to VirtualDub (or similar applications), disable display of the input and output frames during processing. You can do this via the Options pulldown menu.

[hakko504]

@neuron2

Would it be possible to extend Decimate to also handle NTSC <=> PAL conversions? I'm thinking 3 distinct options:

1 NTSC interlaced => PAL progressive

Bob(height=576)
Decimate(NTSC2PAL,i)

or maybe

Bob(height=576)
Decimate(NTSC2PAL)
Decimate(2)


2 NTSC progressive => PAL progressive

Decimate(NTSC2PAL)
BicubicResize(720,576)


3 PAL interlaced => NTSC progressive

Bob(height=480)
Decimate(PAL2NTSC)


For the last conversion, PAL progressive => NTSC progressive, I think it should be possible to just use ChangeFPS(29.97) to gain speed.
So what do you think, would this be possible?

[Blight]

hakko: that has nothing to do with decimate.

Once you get a progressive NTSC, you just speed up the video to 25fps (without adding any frames) and then run a pitch correction algo on the audio to make it match the new video length without making people sound like chipmunks.

[hakko504]

@blight

First note that I do not talk about film (23.976fps/24fps) content here. I mean genuine interlaced (or progressive) NTSC 29.97 content.

Secondly, Decimate is just a filter that removes the frames that will yield the most jerky video. NTSC2PAL could be done like this: ChangeFPS(30)
Decimate(6)but I've had some trouble with ChangeFPS and it should be faster and more reliable to do the decimation on the original content.

Thirdly, I don't like speeding up video's and messing with pitch correction. It just doesn't sound good.

[ppera2]

hakko504

ChangeFPS is not good idea - it inserts or drops frames to get given value, what will cause choppy movement.
For example when I used it to get 25 fps from 23.976 it caused every second short stop - because of repeating aprox. one frame per second.
In your example it will be much rarer, but will happen.

Better is AssumeFPS, but it changes duration - nothing is perfect.

And as I observed there is bug - ChangeFPS doesn't change framecount but duration, at least not in file information of VDub, but same error is visible in TMPGenc.

Speeding up is used by Film to PAL conversion. It is 4% what is not too noticable. Of course speeding from 25 to 30 is too much.

And I think that in AVISynth it should be simple (NTSC to PAL) -
ChangeFPS(25). Other thing is how this function is realized.
It should drop out every sixth frame, and ocassionally seventh to maintain 29.97 fps rythm.
Even better would be if it worked field, not frame based.

Anyway, there is a way to make such conversion good. Perhaps we expect too much from programmers here.

[Mental]

One thing that would be really useful is a feature to restore a progressive NTSC stream that has been converted to PAL.
Because a lot of material that is shot for TV is recorded on FILM at 30fps. This makes for easy NTSC convertion, but introduces some nasty interlacing when converted to PAL.

I myself have no idea on how this should be accomplished, I've read some discussions on other forums, but noone seemed have a good method for dealing with these kind of streams.

[Blight]

I did notice that a lot of music videos originating from the U.S. and aired in PAL have blended fields. They basically do 30fps -> 25fps by blending the fields.

I doubt there's a way to restore a good signal other than doing smart deinterlace.

BTW neuron2, have you had a chance to test my suggesting on blending the duplicate frame to mainain a smoother 30fps video with progressive reconstructed fields?

[wmansir]

@Mental
The NTSC(telecided material) -> PAL -> Original Film conversion is tricky. If the production company did it the right way they would IVTC the telecided material back to 24fps progressive frames and then speed it up to 25fps for PAL systems, but often they take the cheap route and just blend the fields of the 29.97fps telecided material when converting to PAL by running it thru a quick and dirty NTSC->PAL converter. Since the result is blended from two frames it's just about impossible to recover the original frames.

Season five of Buffy was subjected to this treatment on Sky One and after receiving some negative feedback promised to do it right for season six. (unfortunately, I hear they are now editing content out of the season six eps, but that's another issue.)

@neuron2
First, let me say great work, on both this plugin and your vdub filters, which I use often.

I would like to nominate your Antiflicker filter for consideration to be ported to Avisynth. I believe it makes a good canidate because it's usablilty would be greatly increased over it's current vdub implementation, as would it's speed when operating in the YUV colorspace. Of course, you are working on this project now, but if you find yourself looking for something to do I just wanted to throw in this suggestion.

Again, thanks for your work.
wes

[jugm]

Not sure if it is a problem at all and if it is in Decomb or somewhere else... I'm doing NTSC interlaced rather old movie with VOB -> DVD2AVI -> Avisynth -> NanDub. Avisynth does just a standard deinterlace (there are lots of examples in this thread how to do it right). Somewhere close to the end Avisynth produces (in NanDub status line) error msg like "error read 0x..." I dont have exact text now, but I will reproduce it tomorrow. So I had to insert DeleteFrame statements to delete two frames at the very beginning of my .avs file (before Decomb lines). This fixed the error.
Now I dont know how to give you example of my frames so you could reproduce it (its ~4 Gb of VOB files). I tried to use DVD2AVI to feed frames to NanDub and save problematic frames but it didn't work - I can't reproduce my error with result AVI.
So question is - is it something worth to mention like an error or DeleteFrames is good enough fix ? Do you want me to send those "bad" frames and what is better way to do it ?

Yes I tried both 1.82 and 1.9 - same result.

[neuron2]

@jugm

What happens if you leave everything else the same but just comment out the Telecide and Decimate calls in your script? If the problem stops then what if you comment out just the Decimate call?

Also, please post your exact Avisynth script here. Do not omit anything. Thank you.

[jugm]

I will post more details. Now I'm holding my breath and don't touch that computer until it finishes at least 1-st pass because it takes about one day.

My script is :

LoadPlugin("c:\divx\dll\MPEG2DEC.dll")
LoadPlugin("c:\divx\dll\Decomb.dll")
MPEG2Source("c:\divx\my.d2v")
Crop(9,0,705,478)
Telecide
Decimate15
FieldDeinterlace


Error won't come up if I keep everything as above but comment out FieldDeinterlace. When FieldDeinterlace is active I had to put DeleteFrame before Crop statement. This is from my head. Can't verify this now sorry.

And if it helps... right after that "problem" frame I had to skip, whole screen is getting full of big blue (like sky) squares for a few seconds.

[neuron2]

@jugm

You horizontal width must be a multiple of 4. You have 705. If it doesn't crash, consider yourself lucky! If it doesn't crash *and* it gives good output, consider yourself very very lucky. Change the width to 704 and let me know the result.

[jugm]

Ahh good to feel lucky for few minutes ! Now I'm not lucky again. Setting X size to 704 didn't help. So now it's version 1.9 only. And Avisynth 1.0 beta 3. First I got this error with 1.0 beta5. Then thought that 1.0 beta 3 could be more stable. Avisynth script is :

LoadPlugin("c:\divx\dll\MPEG2DEC.dll")
LoadPlugin("c:\divx\dll\Decomb.dll")
MPEG2Source("c:\my divx\my.d2v")
DeleteFrame(201504)
DeleteFrame(201505)
Crop(9,0,704,478)
Telecide
Decimate(5)
#FieldDeinterlace(false,12)


Error (without DeleteFrame lines) says :

Error fetching frame 161208: Avisynth read error: Avisynth: caught an access violation at 0x011f209e, attempting to read from 0


Same error if I comment out Telecide or Decimate. No error if both are commented out.

[neuron2]

How did you know to delete frames 201504 and 201505 and how many total frames are there? What happens if you delete the Crop?

[neuron2]

Maintenance release of Decomb. Version 1.91 makes the following fixes:

Symbolic parameters are now supported. For example, you can say Telecide(blend=false,debug=true).

The debug mode of Decimate was fixed; previously no output was generated.

Fixed erroneous blend of the top and bottom lines in frames detected as combed.

Fixed erroneous printing of the frame number in the debug output of Telecide.

Telecide now throws an error if the input width is not a multiple of 4.

Speed tips and other miscellaneous fixes added to help file.

[neuron2]

@wmansir

Thank you for the suggestion about porting Antiflicker to Avisynth. That sounds like a good idea. When I finish up the optimized version of Decomb, I will tackle it. I have some ideas for improving it as well (e.g., reset moving luminance average on scene change).

@all
BTW, coding of the optimized version of Decomb is complete and there are significant gains (20%). I have a problem, however. When I turn on the release version (debug off) in VC++, the optimizer ruins things. I'm trying to find a way to avoid this. The debug version cannot be released as it is slower than the release version. Does anyone have any experience with this issue?

[jugm]

@neuron2

Original material has 228494 frames. After deinterlace there are 182793 frames. When I found that error message I opened NanDub stats reader, found where it crashed. Checked time stamp of that frame. Opened original file, found same time in there. And then just matched picture frame by frame to about same moment in interlaced original. To find exact two frames I put like 10 DeleteFrame first to stop crashes. Then removed them one by one until I figured out what exact frames I need to skip.

Deleting Crop doesn't affect anything. Same error if I do not delete those two frames.

[neuron2]

@jugm

My best guess is that since it is consistently the same frames, that you have a corrupted VOB. There have been threads describing similar problems that turned out to be that. This kind of corruption does not always have reliable, predictable symptomology.

@blight

Haven't specifically coded your idea, but you can get an idea of it by serving a 3:2 movie into this script (don't apply Telecide and Decimate):

FieldDeinterlace(blend=true)

The two frames out of five that are combed will now be blends. Watch the result at normal speed and decide if you can tolerate it. Yes, there are two blends in 5 instead of one, I understand that. Tell me what you think of the result and we'll go from there.

[Leica Man]

Please help me. I am having problems.

I create the script AVS file, which I then open in VirtualDub. I can see the filter doing it's work as I go frame by frame and play the file in VirtualDub. AND IT IS EXCELLLENT! Finally a true inverse telecine filter! BRAVO MR GRAFT!

Then I save the file in VirtualDub (I tried both uncompressed and HUFFYUV compression), using no filters or alterations and then when the VirtualDub job gets to 99.99%, i.e. almost finished the whole program (VirtualDub) crashes!

Then I open VirtualDub again and the file. It then "builds" that file and I get the first 2GB of that file, which is a beautifully done 23.97fps movie without any lines or duplicate frames, again showing the masterpiece that decomb is. Of course I only get the first 2GB of it and so it is useless other than to admire Mr Graft's true genius.

Then I saw that this was a know bug in VirtualDub 1.4.7. and so I upgraded it to 1.4.8. and now if I open the file I get "aggressive" reindexing, which takes a very, very long time and then when that I is done I get an unplayable AVI file!

I am using the 1.91 version of Decomb and my AVISynth is Verion 1 Beta 5. I tried version 0.3 but I get a message saying no support for plugins.

So please help me use this amazing masterpiece by Donald Graft!

Thanks!!!!

P.S. I know the Mr Graft refuses donations. But if there was a package that combined VirtualDub, with all the filters of Donald Graft and was very bug free and very stable, I would happily pay around $200 for it.

[neuron2]

@Leica Man

Thank you for your kind words.

Let's get the obvious stuff out of the way first. What OS are you using and are you not just running into the 2GB limitation on standard AVI files? Try saving it in DivX or something that will not exceed 2GB and see what happens.

You say there is a known bug in VirtualDub 1.4.7. Do you mean one that explains the crashing or one that explains the failure to recover the AVI file? If you try processing with version 1.4.8 does that crash at the end too? Or are you just trying to recover this first AVI? If 1.4.8 crashes too on a new attempt at processing, what is the crash data that it provides?

[vinetu]

@Leica Man
Before You pay $200 to Mr. Graft and Avery Lee You beter check
the prises of "discreet logic Edit 5" video editing software.
For more than 2 years of fighting with this "editing" software
I can tell You that edit 5 is a rich of bugs,VERY slow,
unstable,low quality render operations ... and support of
sort of a "...yes we work at that problem..."
Generaly edit5 and VirtualDub are different applications,
but operations like :
Slow Motion at 50%,deinterlacing(lack in edit5),brigthnes-contrast-gamma,collor
corections,resizing,reencoding(in same or different codec format),
import-export of avi files,open large files!!! ... all this simple,but frequently used (by video editors)
actions can be done by VirtualDub(+plugins) MANY times more
precisely,speedily,painless ... so generaly - YOU CAN DO THE JOB
if You use VirtualDub,but if You use Edit 5,that is not so sure ...
BTW I started with Adobe Premiere ver. 1.0 about 10 years ago ...
... so check the prises ...

[jugm]

@neuron2

Thank you for all efforts to help me and of course for your software. I will play with that some more. Strange thing is that I just made one pass WITH those "bad" frames deleted and got same error and in the same place. Have not checked details yet... may be it crashed few frames later.

Are there any way to check VOB integrity ? Err hmm wrong forum sorry.

[neuron2]

@vinetu

What is the relevance of "discreet logic Edit 5" to either me, Avery, or this thread? What exactly is your point here?

[Leica Man]

@neuron2

I feel like an idiot (and happy at the same time). I tried everything again and now it works like a charm! So I don't know what was wrong before!

Sorry to have wasted your time and caused you conern.

I am using Windows XP by the way. So there is no 2GB limit, my partitions are NTFS.

Sorry about the confusion of VirtualDub 1.4.7. The bug I believe is that when it repairs an AVI file it only repairs the first 2GBs of it. This is supposidly fixed in 1.4.8.

Can I ask you one thing, please. Prior to this I had been using your Telecide filter in VirtualDub. That filter shifted the sound by one frame, as documented in your help file. To compensate for that I adjusted the sound in VirtualDub by 33ms.

Do I need to do the same in this new Decomb filter? Or it the sound not desyncronized?

Thanks for all your help!

P.S. I too don't understand what point vinetu is trying to make. Is he praising your work or slandering it???

[neuron2]

@Leica Man

You're into classic cameras, I assume.

I'm glad things are working for you now. Thanks for updating us on the problem.

The Avisynth version of Telecide does not introduce any delay. That is possible because you can easily look ahead at future frames when coding Avisynth filters. In VirtualDub I have to buffer up a frame and lag one behind to get the same effect. I've gained great respect for the basic Avisynth philosophy and implementation since working on this filter. There is great stuff in the Avisynth kernel and it is a real shame that Ben Rudiak-Gould is no longer active. He is a genius extraordinaire and we all miss him.

Vinetu must be confused. :-)

[Blight]

neuron2:

using smart deinterlaced on a 3:2 film gives a smooth picture, although with some ghosting (since fields are being blended). However, this is a bit more complex here since you're only doing 1 frame. I think a 50% blend between the frames is something that should be relatively easy to insert into the decimate filter itself. Instead of cutting the frame, it would just blend it with the next non-duplicate frame.

[manono]

Hi neuron2-

I too had difficulty making sense of vinetu's statement. But I'm pretty sure he's paying you and Avery Lee a backhanded compliment by saying, that compared to the high retail prices of Adobe Premiere and Discreet Logic Edit 5, that the $200 Leica Man was prepared to pay for an integrated bug free version of VDub-DGraft Filters wasn't near enough.

[b0b0b0b]

Supposing I had a source that has telecined and regular interlaced material intermingled.

Is there a way to deinterlace the regular stuff but ivtc the telecined stuff but insert duplicate frames to get that part up to 30fps? This would be so there would be no blending or ghosting.

Thanks!

[neuron2]

@manono

Aloha! Hmmm, maybe you're right. If so, that would make me the confused one. (Man, that Kona coffee is good!)

@b0b0b0b

Your question is similar to Blight's, except he wants a blended frame instead of a duplicate. Seems to me that your idea can simply be achieved by omitting the Decimate call. Telecide alone will recover progressives, making duplicate frames out of the extra fields, and its default postprocessing will deinterlace any nonprogressive content. To totally avoid blends, you'd select blend=false, then field match failures would not result in blended frames. Of course, if your fields are blended, you're up the river without a paddle.

@blight

When I complete the current optimization process for Decomb I'll code your idea. I've solved my compiler optimizer problems, BTW.

[vinetu]

manono,

Thanks for explanation !
(the software "Discreet Logic Edit 5" price is ~ $16 000)
... yes neuron2,this is not the thread for such a thoughts ...

[BBWoof]

I recently switched from decomb 1.82 to 1.91 and instead of getting a 20% speed increase, I've experienced a drastic decrease.

The 1.82 values I used were:

Telecine
FieldDeinterlace(false)
Decimate15

For 1.91:

Telicine
Decimate(5)

With 1.82 I was encoding with cce 2.50 at a speed of close to .500,
with 1.91 it's dropped to < .350.

Is there something I'm leaving out of the script.

[Blight]

b0b0b0b:
You're barking up the wrong alley. using a duplicated frame instead of a blended one produces a rather jumpy image, especially on pans. My point was to blend that duplicate frame with the next frame in order to have smoother motion. It mays also be able to interpolate the 2 frames instead, but I'm not sure how good that would be, maybe making a smaller jump but interpolation has more error artifacts and is harder to do, this is why I suggesting 50% blending instead (very fast and easy to do).

[daxab]

Has anyone actually tried this? I've been playing around with it and I find the results quite jumpy. You can't get smooth motion just by inserting one blended frame. You need to do more work than that.

What is happening is that you have 4 frames (from you 24fps source) and you want 5 in the same time interval. Adding one blended frame "X" gives you:

0-000ms @ 000ms 30fps
1-042ms @ 033ms
2-083ms @ 067ms
3-125ms @ 100ms
X-146ms @ 133ms
4-167ms @ 167ms
...

This is jumpy. You want to generate 4 new frames, A, B, C, and D, replacing 1, 2, and 3, like so:

0-000ms @ 000ms 30fps
A~033ms @ 033ms
B~067ms @ 067ms
C~100ms @ 100ms
D~133ms @ 133ms
4-167ms @ 167ms
...

This will be perfectly smooth because the time stamps match exactly. It's more compute intensive but we want quality, right :)? Assuming we have some high quality frame blending code, all we have to do is 4 partial blends:

A = 0.8 x frame 0 blended with 0.2 x frame 1
B = 0.6 x frame 1 blended with 0.4 x frame 2
C = 0.4 x frame 2 blended with 0.6 x frame 3
D = 0.2 x frame 3 blended with 0.8 x frame 4

(Ratios are computed from time stamps, so for frame A, we have 0.033333... / 0.041666... = 0.8)

[Blight]

that will cause quite a bit of ghosting ... I think one blended frame should be the best here, but there needs to be some actual code to test this.

[trbarry]

Blight -

I didn't make the option available in GreedyHMA but if you use DScaler you can try the Greedy/HM deinterlace method and turn on In-Between-Frames to see the results of trying to 50% blend every 5'th frame to reduce judder.

It was just experimental and I never got back to it because it caused too much ghosting and people didn't like it that much. But I decided that if I ever really did try to fix it then it should probably only try to do if for slowly changing scenes where the judder is noticeable but the ghosting is not. So maybe you could compare the candidate two frames that were to be merged and only do it under some arbitrary motion threshold.

But I think even that would cause annoying ghosting when, say, someone moves their arm rapidly in an otherwise still scene. I've seen that effect in DScaler and am not sure what to do with it. I pretty much dropped the idea but maybe Don or someone can make it work better.

But simple temporal interpolation is never what I expected. When scenes change, pixels don't really change through a range of values. Instead they tend to be replaced by other pixels from different locations on the screen. So without good (and expensive) motion compensation you are often blending two things that have no real relationship. But machines are getting faster and I guess someday we can get away with it.

Or we can all go buy one of those Teranex things. ;)

- Tom

[neuron2]

@BBWoof

My testing does not confirm your timing result and I don't see any theoretical reason for it either. Are you sure nothing else has changed? For example, a clip that has more combed frames will spend more time being deinterlaced. I suggest you test a single file using Decomb standalone. You should measure each run several times to try to average out disk caching effects, etc. If your results still show 1.9.1 being slower please let me know.

BTW, the first cut at an MMX optimized version will be released within a few days. The new version will also add an option that makes the deinterlacing use chroma as well as luma. With some clips, especially animation, you can have areas that are different colors but have close luminance, causing the combing detection to fail. Including chroma solves the problem but takes a little longer to process. Most clips seem to do just fine with luma only, but for those tricky clips, you'll be able to turn on chroma. This option is equivalent to "compare color channels" in Smart Deinterlacer.

[BBWoof]

Thanks very much for the response. And you're right. I should have done more research. I will try to do that this evening on several different clips with both versions of the plugin.

[Blight]

trbarry:
My reception is PAL, so I don't have this issue myself. I'm trying to resolve it for others as I had people contact me about issues with hybrid 24/30fps content. Trying to force that content to 24fps make for a very jumpy video on the 30fps content. That's why I think blending the duplicate frame should look better.

[neuron2]

@daxab

Your point about the temporal inaccuracies is well taken. Interesting of course, in that regard, is the fact that even a straight 3:2 pulldown of 24fps content will produce such inaccuracies, yet we do not hear complaints about 3:2 material looking bad. Is the issue then how great the inaccuracy can be before it becomes objectionable to the viewer?

[ppera2]

neuron2:

Unfortunatelly I must confirm that IVTC with Ver. 1.91 is much slower than with 1.81.

I have about 10-11 fps with 1.91 and 14-15 fps with 1.81.

All with same settings, video is Tom and Jerry cartoon, pretty interlaced NTSC from Cinese DVD, but decomb does IVTC very well, output is perfect. If you need more details - tell me.

[neuron2]

@ppera2

Hmmmm. Very strange. Let me investigate.

[neuron2]

@ppera2

I'm away from home and have only a small test file. But my retest just showed 1.91 to be running slightly faster than 1.81, as expected! Can you please do a few things for me? First, use the debug mode of Telecide to be absolutely sure that the versions are what you think they are. The filter will print a version number on startup. Second, please give me the scripts you are using. Finally, what are you serving it into? Thanks.

@BBWoof

Please, also verify the version numbers using debug and let me know what you find.

[ppera2]

@neuron2

Script is:

LoadPlugin("F:\WinUtils\AVISynth\MPEG2DEC.DLL")
mpeg2source("F:\Tom.d2v")
Crop(16,16,704,464)
LoadPlugin("F:\WinUtils\AVISynth\Decomb.dll")
Telecide
Decimate(5,true)
BilinearResize(640,480)

For Vers 1.81 stays Decimate15 . AVISynth is 1.0 beta 5.
I make fast recompress of it in Virtual Dub with DivX 4.12.

[neuron2]

@ppera2

I wish you'd have given both scripts! You leave me wondering if you maybe run the 1.81 script without FieldDeinterlace, since the only change you mention is the Decimate call. If you omit FieldDeinterlace with 1.81, the equivalent for 1.91 would be Telecide(postprocess=false), which runs a whole lot faster. Please clarify.

[ppera2]

Well, that's right. I didn't use FieldDeinterlace in 1.81.
Didn't read whole documentation carefully. It's a bit confusing that in one version it is default in other isn't (at least in documentation itself).
Anyway, without postprocessing 1.91 is really little faster :)

[Blight]

Em, I'm not sure if anyone is aware of this, but if you're encoding AVI using VirtualDub, make sure to set it to "fast recompress" and have all the processing within the AVS file.

This makes sure the video is not converted to RGB and you gain about a 10-20% speedup.

[neuron2]

I have released version 3.0 of Decomb. Version 3.0 contains coding optimizations and new information (see the help file) that lead to large speed gains. Also, a new parameter has been added to Telecide and FieldDeinterlace that optionally enables chroma comparisons in the deinterlacing step. To obtain the maximum speed gains possible with this version, be sure to read and apply the tips in the help file, especially the one about using null parentheses!

Version 3.0 requires an MMX-enabled processor.

I'll tell you about the null parentheses issue here because it is very important and very interesting. If you have a script like this where the parentheses are omitted:

Telecide
Decimate

...you can speed it up by as much as 25% simply by changing it to:

Telecide()
Decimate()

Apparently there is a quirk in Avisynth's parameter processing that causes it to waste large amounts of time in the first case. This works with previous versions as well. Believe me, I was blown away when I discovered this!

If you apply this and other tips described in the help file, you will find that Decomb is now as fast as GreedyHMA.

The new version can be obtained at http://sauron.mordor.net/dgraft/decomb.html

[SleepEXE]

:( Haven't been able to reach your site, Donald. Seems you mentioned a changeover in internet service so hopefully it will be fixed soon. Anyone happen to grab v3.0 before the site went down and willing to post it as an attachment?

Best regards,
SleepEXE

[DDogg]

I'll attempt to do the attachment.

Edit: Sorry, I could not get it to work

[b0b0b0b]

attachment came through as 0 bytes for me.

[Blight]

Anyone done a speed comparison between GreedyHMA (Setting 4, IVTC adaptive deinterlacing+field reconstruction) against Decomb 3.0 with comparable setting?

[neuron2]

@Blight

Yes, I have. They are comparable. :-)

@all

Sorry, but the filter site server main disk died. Must have been an IBM. It probably won't be back online until tomorrow.

When I get home, I'll put the zip onto an alternative site and post the URL here.

[Blight]

your site is back up ;)

Several things, reading the docs for 3.0 ...

I'm not aware of any setting in HUFFYUV to force YUY2. There is a setting to force RGB and by default it suggests YUY2.

There is a setting in PicVideo to force YUY2.
Maybe someone should add one to huffyuv as well.

I've tested the following scripts:


LoadPlugin("decomb.dll")
AVISource("test-huffy.01.avi")
#
# (swapfields, blend first/last frames, deinterlace non-matching frames, threashold, blend instead of interpolate, use chroma search, use debug info)
Telecide(false , false , true , 15 , true , false , false)
#
# (Remove 1 in [n] Frames, 5 = [1 in 5] = IVTC)
Decimate(5)


and ...


LoadPlugIn("GreedyHMA.dll")
AVISource("test-huffy.01.avi")
GreedyHMA(0,0,4,0,0,100,0,40)


It's a 123 frame 720x480 clip that has about 3 missing fields on scene changes with the rest of the scene being proper 3:2 material.

Both encoded in 20 seconds, doh. Guess I'll have to look for a longer clip.

[BBWoof]

Sorry I didn't get back to you. I've been somewhat overwhelmed at work.

Version number was confirmed as 1.91 with the debugview utility.

Here's the script I used.

LoadPlugin("d:\program files\avisynth\MPEG2DEC.dll")
LoadPlugin("d:\program files\avisynth\decomb.dll")
LoadPlugin("d:\program files\avisynth\SimpleResize.dll")
video = mpeg2source("h:\blade\blade.d2v")
audio = wavsource("h:\blade\blade.wav")
audiodub(video,audio)
Telecide
Decimate(5)
Crop(4,0,716,480)
SimpleResize(480,360)
AddBorders(0,60,0,60)

And I'm frameserving to cce 2.50

As you can see, the telecide doesn't have null (). And maybe that was my problem. I'll give it a try tonight.

[neuron2]

@Blight

20 seconds for 123 frames? Time for a new computer my friend! My XP Palomino 1900+ with Soyo Dragon Plus and 512K DDR RAM arrives tomorrow. Should keep me going for about 6 months.

On the YUY2 thing: If you apply some filters in VirtualDub and then encode to HUFFYUV without changing the defaults and save to AVI and then send the AVI into Telecide, it declares it as RGB and needing ConvertToYUY2. If you select the HUFFYUV right-hand pulldown menu and select that arrow thing that says "Convert to YUY2" then the resulting AVI does not require ConvertTOYUY2. This is what I mean. I assumed that the intermediate format offered to HUFFYUV by VirtualDub is RGB and if you don't select the convert option, it leaves it that way. This would not apply for Fast Recompress. Am I deranged or just confused?

Um, how do you make smilies?

[DDogg]

off topic: You are going to love the dragon plus. Pop the fsb to 142. That should be rock solid and push you to xp2000. You might have to upgrade the bios depending on when the mb was made. Excellent mb.

[neuron2]

@BBWoof

>Sorry I didn't get back to you.

Don't be sorry, you just did. Thanks for the update. Yes, the parentheses thing could well account for everything. It is a really strange result but easily duplicatable. When I get a little disposable time, I'm going to dig into Avisynth and find out why this happens. Just think if it is affecting some other common scenarios and people have been needlessly wasting time!

I was scrounging for 8-10% with MMX code and then just offhand I noticed when doing some timings that "Telecide(postprocess=true)" was much faster than "Telecide". But they are supposed to be functionally identical! I thought maybe I had the sense of the postprocess option reversed but it wasn't. Then I concluded you had to have at least one option there and then I thought why not try just empty parentheses. Probably this is invoking some kind of C++ virtual function craziness. I hate C++. You never know for sure what's going on. I don't go as far as Avery; he says he feels most at home just coding in MMX! C is just fine for me.

Oops, I'm starting to ramble.

[trbarry]

I don't go as far as Avery; he says he feels most at home just coding in MMX! C is just fine for me.

Listen to Avery.

Continuing the ramble ... I used to hang out on an assembler language board. Don't remember who, but on of the guys had the sig "Assembler means never having to say you can't".

The problem is that assembler is so politically incorrect these days that no one will put up with it unless they need fast video. ;)

I'll go away now.

- Tom

[neuron2]

@trbarry

Don't go, come back!

Watch out for those symbolic, optional parameters I told you about. If users start leaving off the parens, etc., they might start complaining that your filter has become very slow!

I thought I was really clever because I figured out how to do abs() in MMX code. Then I found out later instruction set extensions add intrinsic abs instructions. Dilettante alert!

[Blight]

Yeah, I'm still on a p3/733. I keep delaying the purchase. I feel the athlons at their current mhz and die size are just too hot. I think i'll wait for the 0.13m shrink which is due in the next 2-3 months.

Might grab some GF4 while I'm at it.

Also, what do you think of SSE. Anywhere to implement it to improve speed? I know most of the code is probably integer so SSE won't be as useful, but maybe in some places... Also, SSE2 should be useful as well, although you probably lack the P4 to do it.

BTW, Just for kicks I encoded some episode of Xena with v1.91 last week. Even though it was mostly blended fields (I hate those 30->25 fps blend conversion) the resulting image was very nice...

Oh, and regarding the comparison I ran last night. Here's a weird thing. Both videos were encoded using DivX 4.12 1-pass quality based (100 quality). The Greedy file came out at 3mb while the Decomb file came out at 2mb. I think this is due to greedy's deinterlacing not blending on missed fields. Not entirely sure though.

I'll try running longer comparisons this weekend.

P.S.
I still think adding decimate into the telecide filter would speed things up even further ...

[western shinma]

I've only done one such comparison but with Nandub at 3x constant quantizer. Decomb clocked in at 25 megs, and Greedy was only 20. This was with default postprocessing in Decomb and force film with vertical filter enabled for Greedy. With Decomb's postprocessing disabled and Greedy's vertical filter deactivated, both files turned out almost exactly the same size, but there was a considerable amount of interlacing left over.

[Blight]

Ok, took another try at benchmarking...

Same parameters as last test. Used a 1 minute clip of high-motion from Andromeda.

I did multiple encodes to rule out caching improving speeds.
Encoded (video only) DivX 4.12 one-pass quality at 100%

GreedyHMA = 3:09 - 77.0mb
Decomb 3.0 (blend:on) = 2:54 - 50.0mb
Decomb 3.0 (blend:off) = 2:54 - 51.5mb

Speed difference = 8-9% in favor of Decomb 3.0

I'm not quite sure how to explain the differences in size, My guess would once again be that the field blending in Decomb compresses better than the Interpolation in GreedyHMA. It's very hard to know which gives the better quality.

Also, blending on/off didn't seem to have any effects on speed, at least not on this one minute clip.

[neuron2]

Thank you, Blight, for the results. That is good news!

One little point. Decomb will do either blend or interpolate. You obviously know that but for others I point it out in case they think they need to use GreedyHMA to use interpolation.

On the file size difference: yes, the blending may compress slightly better, but it is unlikely to be responsible for the results you see, because you'd expect a big difference between the blend on and blend off cases for Decomb. I wonder if GreedyHMA is letting any combed frames go through to the output, as its combing detection is not as effective as Decomb's and it takes a lot of bits to encode combed areas.

I'm working on the Decimate-Blend experiments now.

[neuron2]

@Blight

Just had another thought. Since the times for Decomb blend on and blend off are so similar, it probably means that Decomb did not detect very many frames as combed and so did not deinterlace many frames. If GreedyHMA is deinterlacing a lot more frames, then this could salvage your theory about blending versus interpolation being responsible for the size difference. I do know that GreedyHMA tends to deinterlace more frames than it should, because the combing detection is not as effective as that of Decomb.

You could test your theory by just running FieldDeinterlace(full=true) with blend on and blend off. If there is a big file size difference then your theory would stand up.

[trbarry]

I wonder if GreedyHMA is letting any combed frames go through to the output, as its combing detection is not as effective as Decomb's and it takes a lot of bits to encode combed areas.

That was also the first think I thought of when reading this. If so there are probably some obviously combed frames to be found in the output. But it may also be a question of filtering. Softer material always compresses better, so adding any filter including just turning on Greedy's vertical filter options would correct this.

From my DScaler background I personally like large resolutions and very crisp detail and much of my programming reflects it. But obviously that does not make material that compresses very well, at least without some additional filtering. That should be moot if I ever release the version with variable sharpen/soften.

Of course now after Blight's results I'm going to have to go back and tune the darn thing to go faster so that might take longer, even when I do get back to GreedyHMA. ;)

On another topic, I believe that GreedyHMA and Wiz IVTC decimation is broken towards the end of the clip (when the filter requests frame numbers greater than the decimated number, which it must do to determine which frames to decimate).

I thought I had a check in the decimation code to not read past the end but just not try do drop non-existent frames. Don't I? Or do you have reason to believe I forgot that (easily possible)? Have you seen it make blank/garbage frames at the end?

The decimation code was actually added by request for Avisynth only. There is no equivalent in GreedyHM/DScaler, so it probably hasn't been tested as much. But I wasn't aware of any other reported problems there.

- Tom

[ssjbrolly]

I mainly encode from PAL DVD sources using GKnot, I tryed some time ago to encode a NTSC source and got bad results with GKnot.
So I was curious on trying Decomb as it seems to be working preety well, I have a couple of questions tough...

If I have a NTSC source (From DVD2AVI), I should use Telecide and then Decimate15 to perform IVTC right? But should I still use FieldDeinterlace after that or wouldn't this be necessary?

If I have a FILM > 95% (From DVD2AVI), should I even bother to use Decomb or a simple Forced FILM on DVD2AVI and a deinterlace method of GKnot would be enough? Or even only using FieldDeinterlace method of Decomb(If I do this should I still use Force Film on DVD2AVI?)

I don't have many experience with IVTC since as I said I usually don't encode from NTSC sources so if you could give some hints I'd appreciate it.

[neuron2]

@ssjbrolly


If I have a NTSC source (From DVD2AVI), I should use Telecide and then Decimate15 to perform IVTC right? But should I still use FieldDeinterlace after that or wouldn't this be necessary?
Just because the source is NTSC, that doesn't necessarily mean that you need to apply IVTC. What matters is whether or not there are telecined progressive frames in the stream. If you don't know what that means, you should find out before attempting to deal properly with your clips. Otherwise, it's like trying to do medicine without an MD degree!

Here's how you can tell if you have progressive frames. Find a high-motion part of your clip (look at the clip without any processing!). Step frame by frame. If you see *some* frames that are moving relative to their previous frame, AND they do not show combing, then you most likely have a telecined stream. Once you have established that the stream has progressive frames, you can apply Telecide() to recover as many of them as possible. You apply Decimate() if the original material was 25fps and you want to return to that frame rate. You should never use FieldDeinterlace after Telecide because the exact same thing is achieved through Telecide's default behavior (postprocess=true).
If I have a FILM > 95% (From DVD2AVI), should I even bother to use Decomb or a simple Forced FILM on DVD2AVI and a deinterlace method of GKnot would be enough? Or even only using FieldDeinterlace method of Decomb(If I do this should I still use Force Film on DVD2AVI?)
Assuming you do in fact have progressive source (!), I would try Forced Film and then look at the result of that alone. If you see no combed frames, then do nothing else. If you see some but not a lot of combed frames, you could apply FieldDeinterlace(full=false). If you see a lot of combed frames, then you might want to turn off Forced Film and use Telecide(postprocess=true).

If your source is not in fact telecined progressive source, then just apply FieldDeinterlace(full=true).

I don't know whether the Gordian Knot deinterlacer is adaptive so I cannot advise you about it.

Decimate15() is from an earlier version of Decomb. You should upgrade to Version 3.0. You won't regret it. :-)

Looks like I need to add a Wizard (decision flowchart) to my help file.

[neuron2]

@trbarry

Hi Tom. I'm looking at your code and behavior now and will report back soon. Quite possibly you are doing something different from the vi.num_frames trick that solves the problem I mentioned. I should have checked before commenting. Sorry.

[neuron2]

@trbarry

I thought I had a check in the decimation code to not read past the end but just not try to drop non-existent frames. Don't I? Or do you have reason to believe I forgot that (easily possible)? Have you seen it make blank/garbage frames at the end?
Tom, your code is just fine! It is all my fault and I am sorry for making that comment (I've edited to remove it from the post). The thing is I had a miscoding in one part of my code and the little trick I described canceled it out, but I attributed it to an inability of Avisynth to read past the declared num_frames. But as you so cogently observed [:-)], that would mean that the frames would appear blank or garbage, and they don't! So hearing you LOUD AND CLEAR on that I inspected *my* code and found the problem. There is no misbehavior in anyone's released code; that is the bottom line. Again, so sorry.

To possibly regain some respect... :-) I suggest that your decimation code appears needlessly complex, with the tests for out-of-sync and random access, etc. Can't you just do this:


PVideoFrame __stdcall Decimate::GetFrame(int inframe,
IScriptEnvironment* env)
{
int dropframe, useframe;

/* Determine the correct frame to use and get it. */
useframe = inframe + inframe / (cycle - 1);
dropframe = FindDuplicate((useframe / cycle) * cycle, env);
if (useframe >= dropframe) useframe++;
PVideoFrame src = child->GetFrame(useframe, env);

return src;
}

where FindDuplicate(frame, env) inspects 6 frames, (frame - 1) through (frame + cycle - 1), and returns the right frame to remove. Of course, it caches the result and only recalculates for each new cycle.

[neuron2]

@Blight

I have coded your idea about not removing duplicates coming out of Telecide, but rather replacing them with an interpolated (blended) frame as discussed earlier in this thread. It doesn't look so bad! In fact, it looks surprisingly good.

You can download it at http://sauron.mordor.net/dgraft/decomb301.zip

I have not officially released it so if anyone gets it to play with, please do not re-distribute it or otherwise publish it. If people think it is worth including in the official release I will do so. Please comment here on what you think of this feature. Thank you.

[trbarry]

To possibly regain some respect... :-) I suggest that your decimation code appears needlessly complex, with the tests for out-of-sync and random access, etc. Can't you just do this: ...

Yes, that might be better but I actually added the Decimate code with the intention it would be temporary. It would be much more efficient embedded into the Update_Fieldstore function deep inside the Greedy code. But I wanted a quick and simple add-on for the moment that didn't require me to regression test everthing else that Greedy does.

And besides, I am under an ancient curse. No matter what language I program in, it is destined to come out looking like I wrote it in assembler. I've learned to live with that and try to make up for it with lots of comments. ;)

- Tom

[ssjbrolly]

Thanks for the reply, it really helped to clarify somethings, this might sound dumb, but I still have a doubt regarding progressive/telecined streams...

Originally posted by neuron2
Here's how you can tell if you have progressive frames. Find a high-motion part of your clip (look at the clip without any processing!). Step frame by frame. If you see *some* frames that are moving relative to their previous frame, AND they do not show combing, then you most likely have a telecined stream.


I usually use DVD2AVI to check for interlaced frames, so if I see any combing artifacts on the clip that stream is interlaced and theres no need to IVTC right? But what exactly do you mean by a frame moving relative to the previous one? (I think I'm totally missing the point of what's a progressive and telecined progressive frame...)

Originally posted by neuron2
You apply Decimate() if the original material was 25fps and you want to return to that frame rate.


Wouldn't this be 23,976 fps if I have a 29,970 fps NTSC source?

Originally posted by neuron2
Looks like I need to add a Wizard (decision flowchart) to my help file.


That might be useful :D

I've downloaded Decomb 3.0 now and have been reading the help file, wich brings me to another question... what exactly are the advantages of using Progressive Frame Recovery instead of IVTC? I thoght that when there are progressive frames we should allways IVTC...

Thanks again for the help.

[Blight]

neuron2:
You already coded it, so keep it in ;)

[neuron2]

@Blight

OK.

@ssjbrolly

I usually use DVD2AVI to check for interlaced frames, so if I see any combing artifacts on the clip that stream is interlaced and theres no need to IVTC right? But what exactly do you mean by a frame moving relative to the previous one? (I think I'm totally missing the point of what's a progressive and telecined progressive frame...)
I don't know what you are setting for DVD2AVI. You need to look at the raw clip as