Automatic interlace detection

[berrinam]

EDIT: Another new version (10th Jan) here (http://forum.doom9.org/showthread.php?p=765485#post765485)

While this isn't actually relevant to MPEG-4 AVC, I'm posting this here because it seems relevant to some of the programs which reside in this subforum, like MeGUI, RealAnime and staxrip. If a mod thinks that this is the wrong forum, then please move it.

This post is to share my results with detection of content type (ie progressive, interlaced, film, or a hybrid of those). I would be pleased if you could test the program I've uploaded, and give feedback. I haven't been able to test much, as I have very few sources to work with.

Algorithm
The algorithm I'm using is a combination of what I could pick up of the AGK algorithm and what I worked out by myself. Given an avs script, my program does a selectrangeevery on it, so it doesn't have to analyse the whole video, and then adds code which is copied straight from AGK's avs scripts, which checks whether the frames are (a) moving; and (b) combed (using Decomb's IsCombed function). These results are written to a file called interlace.log, in the directory of the avs script.

My program analyses this file in 5-frame chunks (as len0x said he did. Five is chosen, because telecining is most commonly in a 5-frame cycle). It ignores the chunks in which not every frame is moving, and then declares the chunks in which there is no combing as progressive, and declares the chunks in which there are exactly two consecutive combed frames as telecined, and declares everything else as interlaced. At the end, it counts the number of each type of chunk, and compares the most common one with the second-most-common one. If
mostcommon > hybridthreshold * secondmostcommon
then the source is declared as wholly of the type of mostcommon. Otherwise, it is declared as a hybrid of the two more common ones.

GUI parameters
My program comes with two things that can be tweaked in the GUI: the hybrid threshold, and the selectrangeevery parameters for the avisynth script. I've explained the hybridthreshold briefly above, and the selectrangeevery parameters are there to control how much of the video is analysed. The length parameter should be kept as a multiple of 5, because the output file is analysed in 5-frame chunks.

Of course, my program is only intended to test the algorithm at this stage in time, and not for any practical purposes until the algorithm is either more reliable, or a better algorithm is found.

[berrinam]

I suspect I chose wrongly when posting this in the AVC forum. Could someone please move this to the AviSynth forum?

[charleski]

I'm so glad you're doing this - I was thinking of reverse-engineering AGK for MeGUI as well, but had been putting it off until the New Year. Now I don't need to bother :).

[Dark-Cracker]

Perhaps could you ask Neuron2 if it kown some usefull tips to improve your algo.

[neuron2]

tritical and scharfis_brain, too, should be consulted. Unfortunately I cannot even download the files until I return from vacation.

[Dark-Cracker]

i have try to work on such tool :

http://forum.doom9.org/showthread.php?s=&threadid=68024

and i should still have a few hybrid sample, i will try to find the time to test it once i come back in my house.

Bye and happy new year :)

[berrinam]

Here's a new version.

New features
Field order detection (there you go, Mug Funky ;) )
Updated thresholds
An analysis % way of controlling the SelectRangeEvery parameters.

Ok, so the feature-list isn't very big. But it seems to be giving very similar results to AutoGK, and it also has a field order detection algorithm.

Getting my program
On rapidshare, you can get the
sources: http://rapidshare.de/files/10428446/interlace_detection-src.zip.html
binaries: http://rapidshare.de/files/10427282/interlace_detection-bin.zip.html

I have also attached them, and they will be downloadable from the end of this post, pending approval of a moderator.

Algorithm
The source detection algorithm remains (from memory) entirely the same as described in my first post.

The field order algorithm is a pretty intuitive algorithm I came up with, which works the same way as humans do it: try separating fields with both assumetff and assumebff, and see which one is jerkier (in more concerete terms, work out which one gives more motion, as defined by Y/U/VDifferenceFromPrevious). This algorithm seems to work very well. I've put a threshold in for saying that the field order changes (ie if the amount of the wrong field order detected is more than threshold, declare it as a changing field order). Once again, from lack of sources, I have not really had any way of tuning this threshold, so I pulled the value from thin air.

My testing results
For pure sources, the results are accurate. For hybrid sources, I believe it is just a matter of finding the thresholds.

I've done some testing of my results against AutoGK's results. With the current default thresholds, my results seem quite similar to AutoGK's results.

AutoGK is different, because
It seems to take into account the input framerate, whereas my program only looks at the statistics output by AviSynth.
I call my clip a hybrid if the ratio between the first and second most common combing types is less than hybrid threshold. AutoGK calls it hybrid if the ratio between telecine and interlaced patterns is less than some threshold. I'm not sure how much difference this makes.

I presume that (a) taking into account the framerate, and (b) taking into account what DGIndex reports as the film % (assuming mpeg2 input) could make this algorithm more accurate, but at the moment, I am trying to see how much can be done without this extra information.

What you can do to help me (please do ;) )

Download my program
Run it on your sources
Tell me if it works.


I would especially like people to run it on hybrid clips, because these are the most problematic.

[berrinam]

i have try to work on such tool :

http://forum.doom9.org/showthread.php?s=&threadid=68024I had a look through that thread, but it seems to have ended rather abruptly. Wilbert said on the AviSynth forum that you implemented that in AutoRV. Is this true, and how good are your results (if you don't mind sharing, of course)?

and i should still have a few hybrid sample, i will try to find the time to test it once i come back in my house.Are you still offering to do this?

[stax76]

Great work, for average Joe deinterlacing is far to complicated. Besides encoding frondends it makes also sense for any AviSynth user having a standalone application. Instead of mplayer maybe the avifile API could be used, I would be willing to contribute such things, I have a very fast .NET avifile wrapper that might be easy to use for this.

[berrinam]

@stax: No problem, I would love it if you contributed. You might find the code disorganised, however -- I wasn't expecting anyone else to work on it.

@edit: The code is under GPL.

[charleski]

Using the latest version, it crashes with the error
Could not find file 'C:\MyVideos\interlace.log'

************** Exception Text **************
System.IO.FileNotFoundException: Could not find file 'C:\MyVideos\interlace.log'.
File name: 'C:\MyVideos\interlace.log'
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options)
at System.IO.StreamReader..ctor(String path, Encoding encoding, Boolean detectEncodingFromByteOrderMarks, Int32 bufferSize)
at System.IO.StreamReader..ctor(String path)
at WindowsApplication3.Form1.analyse(String filename, String inFileName)
at WindowsApplication3.Form1.analyseSourceButton_Click(Object sender, EventArgs e)
at System.Windows.Forms.Control.OnClick(EventArgs e)
at System.Windows.Forms.Button.OnClick(EventArgs e)
at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent)
at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.ButtonBase.WndProc(Message& m)
at System.Windows.Forms.Button.WndProc(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

[berrinam]

Well, the error is pretty simple. Anyway, AviSynth is supposed to be generating that file, so I would guess that the problem is that, for some reason, AviSynth is not running properly. Can you see what happens when you try loading this script in VDub?

EDIT: Just in case it isn't clear, you need to specify where mplayer is (and it needs to be able to play avs, which means a more recent build than about half a year ago), and you need Decomb.dll in your AviSynth plugins folder. Also, just for safety's sake, make sure your script ends in a newline. I think that's all you need.

Looking at it in VDub should hopefully solve it, but I did get one problem which VDub managed fine, but mplayer crashed on, which had problems.

[stax76]

I've removed the mplayer dependency: http://rapidshare.de/files/10522699/interlace_detection-src.zip.html

[berrinam]

Thanks to stax for providing a version which uses AVIFIle.

Ok, here is what I hope could be a final version before it is integrated into MeGUI (and staxrip, if stax wants to take the time to wade through my code again). It does better field order detection, it recommends a script, and it adds 'portion management', which is ideally for progressive content with interlaced sections in it, like the credits or something, in which case it would only deinterlace the credits. I've put a fair bit of work into that, and I'm interested in the results of that, because it often kicked in when it shouldn't.

I'm too lazy at this point in time to write a summary of the algorithm or any of the other sections I did before, so here are the links:

Bins on rapidshare (required .NET 2.0): http://rapidshare.de/files/10771821/source-detection_bin.zip.html
Sources on rapidshare: http://rapidshare.de/files/10771626/source-detection_src.zip.html

I've once again attached the files here as well.

[berrinam]

@stax: Do you mind if I use the extra AVIFile code you added in MeGUI?

[stax76]

No but since you port the entire thing to MeGUI maybe it's better to use the code that is already there because it's probably derived from the same source and identical. If you need very fast avifile drawing btw. check out my source code.

Dimzon mentioned there is a lot duplicated code and I noticed this as well e.g. each and every codec dialog has the same zones code, I think that's really not good and belongs in a zones user control.

http://c2.com/cgi/wiki?DontRepeatYourself

[stax76]

Ok, here is what I hope could be a final version before it is integrated into MeGUI (and staxrip, if stax wants to take the time to wade through my code again).


Thanks for offerring me using the code, of course I'm much interested in having such a feature available in StaxRip. Maybe I can help you as well somehow with something I did for StaxRip or AVSEdit?

[berrinam]

Thanks for offerring me using the code, of course I'm much interested in having such a feature available in StaxRip. Maybe I can help you as well somehow with something I did for StaxRip or AVSEdit?
Well, of course, as all of these projects are GPL'd anyone could use the code without asking if they wanted to, but it's nice to be polite.:D Also, I want to make sure that the algorithm doesn't disappear even if MeGUI does.

If you plan to add this to StaxRip, then I would actually recommend using the MeGUI code. It went through considerable alterations when being put into MeGUI, which should make it interface better with a program.

As regards to your offer, nothing springs to mind at the moment, but thanks anyway. I'll keep it in mind in case I want anything in the future:D

[stax76]

Thanks, since you've helped me twice maybe I can spot a bug, sent a patch or share some experience here and there for MeGUI as well. Is there a source release or do I have to make a anonymous cvs checkout?

[berrinam]

Sharktooth puts the sources up here, but they tend to become obsolete quite quickly. CVS is your best option, although SF's CVS is a bit unreliable.

Sharktooth's sources: http://files.x264.nl/?dir=./Sharktooth/megui/Sources

[Revgen]

@Berrinam

I just tried your program, and it kinda works and kinda doesn't.

Here is my script:

avisource("H:\ds002.avi")
assumetff()


What works:

1) It reported that my video capture clip was a Film/NTSC hybrid. Which is true.


What doesn't work:

1)It reports that the majority of my clip is film. This is not true.

2)It recommends that I use tfm(order=0).tdecimate(hybrid=1) even though my clip is TFF, which I specified in my original .avs script.


I hope this helps.

[berrinam]

Ok. Thanks for testing, Revgen. I haven't really updated this program recently, since I have integrated it into MeGUI. However, I am interested in your results, especially concerning hybrids.


1)It reports that the majority of my clip is film. This is not true.Can you tell me the numbers it reports as numTC, numInt, numProg and numUseless?

2)It recommends that I use tfm(order=0).tdecimate(hybrid=1) even though my clip is TFF, which I specified in my original .avs script.I presume you don't simply mean the assumetff when you talk about your original script? On that topic, why is that line even there? AssumeTFF doesn't force the field order, it just overrides AviSynth's field order value, and since there are no filters after that, it's not needed for anything.

Anyway, in this case, can you also give me the amount that it was declared BFF by (it gives two numbers).



EDIT: Final question. Would it kill you to try your same source with AutoGK, to see whether it does it better?

[Revgen]

Can you tell me the numbers it reports as numTC, numInt, numProg and numUseless?

numTC=0, numInt=0, numProg=0, and numUseless=201


Anyway, in this case, can you also give me the amount that it was declared BFF by (it gives two numbers).


Where are these numbers located?


EDIT: Final question. Would it kill you to try your same source with AutoGK, to see whether it does it better?

I'll take a look.:)

EDIT: Oh, I forgot to mention this, but my capture device captures NTSC video at 30i instead of 29.97i like other devices might do. Would this affect your algorithim. Sorry for not mentioning this earlier.

[berrinam]

numTC=0, numInt=0, numProg=0, and numUseless=201
Well herein lies the problem. Useless sections are labelled just so -- because they are useless to the program. The program ignores them, because they don't have movement there. (EDIT: something else occurred to me: is your source anime? I haven't tested with any cartoon type sources, so this may also be the problem) Since all the sections are useless, there is nothing to analyse. It sounds as if the log file generated has something wrong with it (which means the avisynth script also has something wrong with it). My program writes a file called autodeint_yourscript.avs. Can you check whether that file opens fine in VDub please? I suspect that may be the problem.

EDIT: Oh, I forgot to mention this, but my capture device captures NTSC video at 30i instead of 29.97i like other devices might do. Would this affect your algorithim. Sorry for not mentioning this earlier.
That is irrelevant. My program tries to make its decision solely on the analysis of the frames.

Where are these numbers located?
After it has found the field order, it should say something like: source was declared TFF by a margin of ??? to ???. Those are the numbers I want. It could also say: source has a hybrid field order of ??? and ???.

[Revgen]

BTW, I tried AutoGK and it reported that my source was a hybrid and that it was mostly NTSC. Which is correct.

Well herein lies the problem. Useless sections are labelled just so -- because they are useless to the program. The program ignores them, because they don't have movement there. (EDIT: something else occurred to me: is your source anime? I haven't tested with any cartoon type sources, so this may also be the problem)

I've now tried this on 2 new and different video captures and it gives me the same numTC=0, numInt=0, numProg=0, and sometimes the numUseless is a different number. It also says that these other video captures are hybrids despite the fact that they contain no film material whatsover. It also reports that they are mostly film. I think this problem may have something to do with the program and not the source itself.



Since all the sections are useless, there is nothing to analyse. It sounds as if the log file generated has something wrong with it (which means the avisynth script also has something wrong with it). My program writes a file called autodeint_yourscript.avs. Can you check whether that file opens fine in VDub please? I suspect that may be the problem.

I opened them in Vdub and they are very small boxes that show only a fraction of the image. And the audio skips to different portions of the clip. Other than that it worked fine.



After it has found the field order, it should say something like: source was declared TFF by a margin of ??? to ???. Those are the numbers I want. It could also say: source has a hybrid field order of ??? and ???.

It doesn't say that anywhere that I've found. Is this supposed to happen before or after it gives me a suggestion to use TFM? After it gives me suggestions for TFM it analyses the video and creates the autodeint_yourscript.avs and the .log files then shuts down. Is this the way it's supposed to happen?

[berrinam]

I've now tried this on 2 new and different video captures and it gives me the same numTC=0, numInt=0, numProg=0, and sometimes the numUseless is a different number. It also says that these other video captures are hybrids despite the fact that they contain no film material whatsover. It also reports that they are mostly film. I think this problem may have something to do with the program and not the source itself.

Yes, it may well be a program bug. If it always reports 0 for numTC, numINt and numProg, it will always claim it is hybrid film/ntsc, mostly film, because these are the numbers which are used to determine the portions.

I am somewhat disappointed that my program fails so miserably (ie, it can't find ANY information to work with), and as I can't reproduce it I am stuck. Could you send me a sample of one of these sources please?

I should also make some initial checks: Do you have Decomb.dll in your AviSynth plugins dir? Do you have have a newish version of AviSynth which supports frameevaluate etc? Perhaps you could also try MeGUI full version (http://sourceforge.net/projects/megui) to see if it catches any errors that my program does not, as MeGUI contains a newer version of the interlace detection code.

PS: You didn't answer my question: Is your source cartoon/anime?

[Revgen]

I am somewhat disappointed that my program fails so miserably (ie, it can't find ANY information to work with), and as I can't reproduce it I am stuck. Could you send me a sample of one of these sources please?


Well, they are multi-GB Huffyuv caps, so I'll have to shorten them in Vdub so I can send 'em. I'll have to check to see if this will affect how your program works with them.

Can you PM me your email. That way I can send you a link via http://www.yousendit.com



I should also make some initial checks: Do you have Decomb.dll in your AviSynth plugins dir? Do you have have a newish version of AviSynth which supports frameevaluate etc? Perhaps you could also try MeGUI full version (http://sourceforge.net/projects/megui) to see if it catches any errors that my program does not, as MeGUI contains a newer version of the interlace detection code.

1) I do have the Decomb.dll in the AviSynth plugins directory.

2) I have a custom version of Avisynth 2.56 that was compiled by TSP to take advantage of multii-threaded cpu's or dual-cores. I didn't use any of these muti-threading commands while using your program, so it couldn't be anything related to these changes. You can get more info about this version here. (http://forum.doom9.org/showthread.php?t=94996) I'm not sure if it supports frameevaluate or not. You could ask TSP about it.

3) I'll try the new version of MeGUI and report back.

PS: You didn't answer my question: Is your source cartoon/anime?

Oops, sorry:o

My original source was a basketball game shot purely interlaced with commercial advertisements that are film based.

The other 2 were a school play shot in pure interlaced and recorded from an VHS tape and an 8 second sample from a football game that is purely interlaced.

[berrinam]

Well, they are multi-GB Huffyuv caps, so I'll have to shorten them in Vdub so I can send 'em. I'll have to check to see if this will affect how your program works with them.

Can you PM me your email. That way I can send you a link via http://www.yousendit.comThanks.

1) I do have the Decomb.dll in the AviSynth plugins directory.I thought you would have, but it's good to be sure.

I'm not sure if it supports frameevaluate or not. You could ask TSP about it.You should be able to find out by opening interlace.log. It should have a bunch of lines which are all true-true, true-false, false-false, or false-true. If you have something like "I don't know what "a" means", then there's some avisynth problem.

3) I'll try the new version of MeGUI and report back.Ok thanks. That has error catching for avisynth errors, if I'm _very_ lucky, then it will turn out to be an AviSynth error after all. It's probably too much to hope for.


Oops, sorry:o

My original source was a basketball game shot purely interlaced with commercial advertisements that are film based.

The other 2 were a school play shot in pure interlaced and recorded from an VHS tape and an 8 second sample from a football game that is purely interlaced.
Ok, thanks. I was just wondering, because anime has lots of non-moving frames, and if my program detects non-moving frames, it flags that group of 5 as useless. You can see why I thought that might be causing your problems.

[Revgen]

Okay I've got some good news.:)

I decided to reencode my video with FFDShow's version of Huffyuv to try to reduce the size so I could post it. After I did that I tried your program again.

It now works. It recognizes the video as a hybrid TFF source that is mostly NTSC. Which is correct.

This is interesting, because I use the non-ffdshow Huffyuv 2.1.1-CCESP Patch v0.2.5 version of Huffyuv to capture my videos. All of the other captures also used this version of the codec. Perhaps your program has a problem with this codec.

Anyways I'll load a section of my original capture tommorow. It'll be larger file than I orginally wanted to upload since it appears that it is the codec itself that is the problem.

[berrinam]

Okay I've got some good news.:)

I decided to reencode my video with FFDShow's version of Huffyuv to try to reduce the size so I could post it. After I did that I tried your program again.

It now works. It recognizes the video as a hybrid TFF source that is mostly NTSC. Which is correct.
Ah well. I am very relieved. Thank you.
This is interesting, because I use the non-ffdshow Huffyuv 2.1.1-CCESP Patch v0.2.5 version of Huffyuv to capture my videos. All of the other captures also used this version of the codec. Perhaps your program has a problem with this codec.This could be. It could also be a colour-space problem. However, if there is interest in this program, I will endeavour to fix such problems. Now's the time to be honest (don't worry, I won't be offended): were you trying this out just to be nice, or do you actually plan on using it?

Anyways I'll load a section of my original capture tommorow. It'll be larger file than I orginally wanted to upload since it appears that it is the codec itself that is the problem.Is there any reason to upload it now that it works?

[berrinam]

I decided to reencode my video with FFDShow's version of Huffyuv to try to reduce the size so I could post it. After I did that I tried your program again.Perhaps this is a discrepancy between AviSource and DirectShowSource, ie the other version of huffyuv it was encoded with didn't have a VfW decoder on your system, so it was returning blank frames, whereas AutoGK, which uses DirectShowSource, didn't require the VfW decoder, and so there was no problem.

[Revgen]

Ah well. I am very relieved. Thank you.
This could be. It could also be a colour-space problem. However, if there is interest in this program, I will endeavour to fix such problems. Now's the time to be honest (don't worry, I won't be offended): were you trying this out just to be nice, or do you actually plan on using it?

Yes, I do plan on using it, I capture alot of TV programs that usually tend to be hybrids. Like cartoons with CGI and other stuff.

The reason why I have to capture with the Huffyuv 2.1.1 CCESP version instead of the ffdshow version is because I use Dscaler to record with my capture card because it records faster than any other software for my card. The problem is that Dscaler doesn't capture well with FFDshow integrated codecs for some reason. The devs are working on a fix, but they haven't fixed it yet. The only way I can record in Huffyuv is to use this older version.

Anyhoo, I know you work hard on MeGUI and other apps, so you don't have to work on it if you don't want to.

I hope I didn't give you a heart attack.:D

BTW- Is there a reason why AutoGK recognized it and this one didn't?

[berrinam]

BTW- Is there a reason why AutoGK recognized it and this one didn't?See my second post: it's about using DirectShowSource instead of AviSource (IMO).

Well, I'm glad that someone is interested in this program. As I said, I ported this into the MeGUI AviSynth Script Creator, and the changes there mean it is less error-prone, and easier to use. MeGUI also uses DirectShowSource for the avi loading. The code structure is also better in MeGUI, so it should be quite easy to export the changes, if a standalone app is really wanted.

[Sirber]

Thanks for offerring me using the code, of course I'm much interested in having such a feature available in StaxRip. Maybe I can help you as well somehow with something I did for StaxRip or AVSEdit?I would be interrested too :)

[berrinam]

Well, it's GPL'd, so feel free to look around. I'm not sure how well it will work with anime (I think I explained my concerns briefly above: it ignores any section of 5 frames in which some frames have no movement. It seems to me that in anime, which has lots of duplicates, it will be ignoring a lot of sections, and therefore not very well able to do its analysis, as it has very little information. However, that's all theory, and I have not actually tested it on anime-type sources, so you should test it and see whether I am right).

This algorithm has been integrated into MeGUI, and the code there is really better structured and more stable, so if you want to use it, look at the MeGUI code: SourceDetector and ScriptServer.

Since you both seem to be interested in this, I will (sometime) rewrite the Algorithm summary, so that it actually gives a description of what it now does.

[Sirber]

I plan to add a "DVD import" section in RealAnime, so it's not only anime. I will have a look at MeGUI later. Thanks!

[iNFO-DVD]

Without trying to sound too stupid..... what am I missing? Doesn't seem to work for me... What am I doing wrong?

It seems the script doesn't have 'a' ?

c = WriteFile(c, file, "a", "sep", "b")

I can't see what 'a' equals?

My interlace.log file always says:

I don't know what "a" means-false
I don't know what "a" means-false
I don't know what "a" means-false
I don't know what "a" means-false
I don't know what "a" means-false
I don't know what "a" means-true
I don't know what "a" means-false
I don't know what "a" means-false
I don't know what "a" means-false

I have Decomb.dll in AVISynth folder, even copied that same file to movie/avs folder and program folder, still the same, using Decomb.dll from this site.

I know I'm doing something stupid, that's what I get for staying up all night, can someone be a good chappy and tell me what I'm missing.

TIA