Decomb 5.0.0 beta 15: Fix for crashing problem

[gizmotech]

No, it had the subject line "huffy's" iirc.

Actually there should have been 2 emails, as the first one I got to typing, sent it, then realized I forgot to attach things to it :)

Gizmotech

[Guest]

Originally posted by gizmotech
@neuron2
This is gonna sound like a lame request but is there any way for you to occupy a few less lines with the show=true output? Pretty lame, I agree.

Use debug=true instead of show=true and you'll be able to see all the video.

[gizmotech]

Hmm... I've encounterd a very strange problem.

I'm currently attempting to use decimate mode=1 however I can't get it to deciamte at this point. I've tried 0,2,3 and they all decimate fine down to around 36000 frames, however when activating mode=1 it doesn't perform any decimation (final frame count is the original 45000). Now I've tried commenting out every line from my AVS so that is's a minimal avs w/o a telecide line, just a decimate line and it still will not decimate in mode=1.


LoadPlugin("C:\newfilters\mpeg2dec3.dll") (version 1.08)
LoadPlugin("C:\newfilters\decomb5b9.dll") (I rename for clarity)
mpeg2source(d2v="C:\bloodylongpath\episode5.d2v")
YV12toYUY2(interlaced=true)
#Telecide(line w/ lots of options)
Decimate(cycle=5,quality=3,mode=1)


Now as is, it should just perform a mode 1 decimation, however I cannot get it to do this. I've tried it on 2 d2v files, w/ 2 version of mpeg2dec3, however I cannot get it to activate.

Many thanks,
gizmotech.

[InfoCynic]

Gizmo? Have you read the manual? :)

Mode 1 should be used for hybrid material that's mostly video, not film. The FILM portions are ramped up to 29.970 by inserting an extra frame after deleting the dupe (manual explains better), the VIDEO portions are passed through untouched. The final frame count WILL equal the original, because that's how it works!

Next time, someone's going to be a lot ruder and just tell you to RTFM... :)

P.S. Isn't it better just to use mpeg2dec.dll and not do the extra YV12toYUV2?

[gizmotech]

/me slaps forhead.

Ignore my foolishness.

Wait, Ok now it all makes sense.

Probably, but it doesn't seem to be any faster then dec3 w/ the convert statement.

[Guest]

It may not seem faster, because mpeg2dec has to convert the color space anyway. But there is some small overhead from having another filter in the chain, as well as an extra unnecessary load on the Avisynth frame cache.

[gizmotech]

@neuron2

Overriding gthresh. As we discussed earlier, gthresh is a limiter, mean when I receive a mismatch of 8.25% and set the gthresh to 8 it "should" use the appropriate guide value instead of the mismatched frame? Am I understanding this part correctly? (if not ignore next part)

I'm wondering then 2 things:
1) can it be made to accomodate decimal values? currently I receive an error when I enter in 8.2.
2) If I set it to 8 it should stay in pattern, however currently it doesn't, however if I set it to 7 is stays in patern. (the same applies to 0.25% and gthresh=0)

I'm wondering if I understand this correctly, and if so is my second question reproducable in your tests?

Thanks,
GizmoTech.

[Myrsloik]

Would it be possible to have an option added to decomb that displays the vmetric for all possible matches and not only the current one? This would really help if you attempt to do something like the decomb log parser by cuisinart since it then would be possible to calculate how the matches would have been with a different vthresh after the log is made and lots of time could be saved that way.

[Guest]

I'll be coming back to Decomb 5 after I finish off DGBob(). I will add the display of all vmetrics and consider the other requests as well. When I resume the development, we will shake down pattern guidance. I'm really happy with the blind field matching right now. I don't see any good way to improve it, and it does very well on many of my torture clips that previously required pattern guidance to be rendered correctly, and which now match correctly with just the blind matching.

[Milkman Dan]

That's the error I get when I try to use TMCpost instead of the built in postprocessor. I know I'm doing something stupid, but I can't figure out what.

Here's my script:

MPEG2Source("D:\vobSource\testFragments\KareKano.d2v").ConverttoYUY2(interlaced=true)
Telecide(order=1,guide=0,post=1,nt=10,hints=true).TMCPost(tff=true,search=5,vf=0)

So what stupid thing am I doing?

EDIT: I'm using Beta 9, if that makes a difference.

[Guest]

Use vf=false or vf=true.

My bad, the help file is ambiguous. Sorry for the inconvenience.

[gizmotech]

@neuron2

Just a quick idea for when you come back to decomb, would it be possible to add adjustable triple=true variable in the override?
As you've noted, triple=true will cause all types of evul on pans and similar slow motion movements, however on "flapping Mouth's" it works wonders. As such, if I could turn on and off triple=true via override it would save me alot of time. (identify a pan is far quicker then identifying mouth combes for previous matches)
Though this doesn't actually fix the problem of the jerkiness in those types of motion scenes it would make the overrides easier.

Thanks GizmoTech

[Guest]

Originally posted by gizmotech
Just a quick idea for when you come back to decomb, would it be possible to add adjustable triple=true variable in the override? Yes. Thank you for the suggestion.

[Milkman Dan]

Originally posted by neuron2
Use vf=false or vf=true.

My bad, the help file is ambiguous. Sorry for the inconvenience.

No problem. Thanks for the response, and all the great work. I can't wait until you wrap this baby up with some nice YV12 action.

[Guest]

OK, after my little sabbatical, I am back to work on Decomb 5, attacking pattern guidance, which, I confess, currently is poor in Decomb 5.

But don't worry! I have stumbled onto a really effective strategy and tested it successfully for 3:2 pulldown guidance. It hardly ever falls out of pattern except at some scene changes, where it is expected.

I just have to add support for 3:2:2:3:2 and 2:2 and then I can release a new beta. I also want to write it up in my journal. Assuming everyone is happy with it, then we move on to postprocessing and additional color spaces.

[Obi Wan Celeri]

Wow, thanks for Decomb and now for DGBOB :)

I don't know about other places in the world but up here in Quebec a lot of local titles are interlaced instead of progressive and the classic solution has always been to blend the whole thing together (giving some rather horrible conversions). The same thing happens to anything being brought in from europe (conversion from PAL to NTSC). Anyways DVD2AVI is conviced those movies are interlaced. But now with DGBOB the results are just FANTASTIC :)

Excellent work Neuron2 :)

Now here's a little suggestion.

There are quite a few decomb plugins lying around on my computer ... and there's no real (easy) way to identify what version they are ... Do you think it would be possible to add a version number that could be read by checking on the DLL's properties?

Oh and does NTSC really stand for Never The Same Color? ;)

[cipher]

There are quite a few decomb plugins lying around on my computer ... and there's no real (easy) way to identify what version they are ... Do you think it would be possible to add a version number that could be read by checking on the DLL's properties?

Obi, enabling "show=true" and previewing in vd/vdmod, you will see the version numbers.:)

[gizmotech]

@Obi, Alternatively you could just rename the files themselves.

I have 11 different versions of Decomb sitting in my filters folder, all the 5 betas, 4.1b2 for 2.5, and the old 4 for 2.0x

@neuron2

I look forward to reading the journal entry. This current method for patern guidance in 3:2 pulldown, will it be able to "intelligently" switch to between ccppc and ccnnc guidance? Currently the only solution to source swich essentially perfrom this switch is to enable triple=true, and compensate for pans/motion sequences manually. Not that this is hard to do, as I have to make an override for those sources anyways ... **Kicks the EVUL COTS Source** nich :)

At any rate, I look forward to seeing this in action.

GizmoTech

**Also going to start some experimentation w/ decomb 5 on *Almost* FILM source. You know those evul film sources that are 98% and come out after a forced film with a few interlaced scenes. Assuming there isn't noise, the guide=0 w/ triple=true could work wonders on these sources for cleaning up the few remaining 100 matchable frames.

[Obi Wan Celeri]

Hey thanks @Cipher and @Gizmotech!

Renaming DECOMB.DLL is a great idea, I should do that :)
And using "show=true" is equally good ...

But since @NEURON2 is such a compulsive programmer (eheheheh) I think DECOMB is a good candidate for proper identification (if it's not too much of a hassle and it can be done easily).

Oh by the way, "show=true" does not always show the proper version ... 5b6 had that problem (if my memory serves me right).

@Neuron2:
Sorry I could not come up with a better suggestion :)
Some of you guys out there are seriously smarter than I :)
... and just reading your journal overloads my CPU ehehehe

[Guest]

Decomb 5.0.0 beta 10 is now available:

http://shelob.mordor.net/dgraft/decomb/decomb500b10.zip

This version improves pattern guidance. Please read the following notes carefully:

1. If you have a non-hybrid, you'll have post=2/3. When a frame is declared combed with post=2/3, pattern is NOT declared lost, unless no prediction can be made (mismatch = 100%) or mismatch exceeds gthresh.

2. If you have a hybrid, you'll have post=4/5. When a frame is declared combed with post=4/5, pattern IS declared lost.

3. In any post mode, if a backward match is chosen, pattern IS declared lost.

4. There is no guidance for patterns with backward matches. They should occur only at edit cuts or other pathological cases, where clearly pattern is lost.

If this new handling holds up, we'll move on to fix up postprocessing and then do the YV12 support.

[HomiE FR]

Thanks again for this new build! :) It was a "long" time (for me I mean) since the previous one.

Sorry to bother you, but didn't you say that you would give the possibility to see the vmetric for each possible field matching instead of only the chosen one? It's no real problem if we only have the chosen field match vmetric, but it could be useful to choose the right field matching when a bad one is done (very rare, but I like perfection :p )

Thanks again for your work.

[Guest]

Let me concentrate on the major functionality right now. I'll clean up the other stuff at the end.

How is the new pattern guidance working for you?

BTW, I have the new postprocessing done and a new beta will be along shortly.

[gizmotech]

Hi neuron2.

Before you release beta 11 can you please add the ability for patern locking? I'm currently applying both beta 9 and 10 to a source with (essentially) perfect ccnnc source, and 98% of the time decomb is correctly patern matching, however it will also go for long spells of incorrect patern matching based on 0.05-0.25% patern mismatch causing all types of combe errors. Setting gthresh=0 is currently not stopping this.

So far it looks good, but because I can't lock the patern or put gthresh=0 low enough to stop the overrides I'm still having to override large chunks of the source.

Another quick question, directly concerning override files. Would it be possible to evaluate the dicision in the override file based on the patern instead of just automatically declaring the overriden match "out of patern"? A number of overrides I perform are to get it back into patern, which is related to my previous point, and the frames are declared out of patern.

otherwise looks good. looking forward to yv12 :)

GizmoTech

[Cyberia]

@Neuron2:

Could you explain what this means? I am confused by the "...instead of using the best field match for the frame, the original frame is deinterlaced" part. poof. HOW is the original frame deinterlaced?

post=4: This is the same as post=2 except that instead of using the best field match for the frame, the original frame is deinterlaced. You would use this to pass video sequences through when you have a hybrid clip.

I'm sure I am missing the obvious, but I *figured* it would be deinterlaced. (thats why I'm using Telecide) It's the 'HOW' more than the 'WHAT' that is important.

Fantastic work on this latest Decomb!

[bilu]

I think it goes like this:

Imagine a telecined movie field sequence.

a b b c d
a b c d d

Telecide does this in forward matching ( I think):

a b c d d
a b c d d

post=4 should do this:

a b b* c* d
a b c* d* d

frames b*c* and c*d* were deinterlaced, not matched.

Maybe I'm completely wrong as I don't know how does it remove combing without match though :rolleyes:


Bilu

[Guest]

Originally posted by gizmotech
Before you release beta 11 can you please add the ability for pattern locking?It's already there. Use this kind of override:

200,500 ccnnc

If you are asking for something else, spell it out. If you are reporting that pattern matching is failing, then you know the drill: provide a source clip and script!

I can fix the <= test to < so that gthresh=0 stops all overrides, but then that would be the same as setting guide=0. That won't be of any use.

Another quick question, directly concerning override files. Would it be possible to evaluate the dicision in the override file based on the patern instead of just automatically declaring the overriden match "out of patern"? A number of overrides I perform are to get it back into patern, which is related to my previous point, and the frames are declared out of patern.I'll think about it. It's only a display issue, isn't it?

[Guest]

Originally posted by Cyberia
Could you explain what this means? I am confused by the "...instead of using the best field match for the frame, the original frame is deinterlaced" part. poof. HOW is the original frame deinterlaced?

I'm sure I am missing the obvious, but I *figured* it would be deinterlaced. (thats why I'm using Telecide) It's the 'HOW' more than the 'WHAT' that is important. Well, bilu is a little confused so I'll describe the what and the how. :)

Suppose we make a field match and the resulting frame is detected to be still combed. As long as post is greater than 0, we are going to deinterlace that frame, no matter what. If post=2, we take the frame as matched and deinterlace that. If post=4, we ignore the match and just use the incoming frame as is, without matching, and deinterlace that.

How do we do deinterlacing? Well, we just apply FieldDeinterlace() to it. So, decombing can be achieved either through field matching, or, when that fails, through normal video deinterlacing methods, such as FieldDeinterlace().

[Guest]

@all

The pattern guidance algorithm has just been published in my website journal.

[Cyberia]

OK, if the composite frame conjured up by Telecide is detected as (still) combed then look at the POST value. If POST=2, deinterlace the Telecide frame. If POST=4, deinterlace the original frame.

If the Telecide frame is not detected as combed, keep it.

In either POST case, 'deinterlacing' is either Interpolating or Blending the fields depending on the value of Telecide's BLEND parameter.

Gotcha. Thanks.

[Guest]

@Cyberia

You are spot on with one refinement: the deinterlacing is spatially adaptive, so that only combed areas of the picture are interpolated/blended.

[Boulder]

Donald,

If I've understood correctly, the basic functions of Telecide will still be the same (with your latest improvements and inventions) as in Decomb v4 and us PAL people can still use the plugin as we used to after finding out the correct field order? I use simply Telecide() for sources that I know to be originally film (Star Trek TOS, for example).

Eagerly waiting for the YV12 version here:)

[Guest]

Originally posted by Boulder
If I've understood correctly, the basic functions of Telecide will still be the same (with your latest improvements and inventions) as in Decomb v4 and us PAL people can still use the plugin as we used to after finding out the correct field order? I use simply Telecide() for sources that I know to be originally film (Star Trek TOS, for example). Yes, of course. I wouldn't leave you PAL guys out in the cold!

[Milkman Dan]

I had a question about the eventual "1.0" release of Decomb.

You've already mentioned that you are going to work a bit on post, which you've already done, I believe. And then adapt for YV12.

However, are you planning to do any low-level ASM optimizations with the code, or have you been doing that all along?

I ask, because I'm learning ASM (MASM) and looking to cut my teeth a bit on some code before moving along to something really complex. So if you've optimized it already, I can study that, and if you haven't, I can see if I can figure out what would be improved.

Just so that this isn't totally off-topic, I've seen good improvements with the pattern matching so far. I feel safe turning it on now, instead of leaving it on guide=0 just to be safe.

Unfortunately, it doesn't help much on Kare Kano (at least so far as I've tinkered), but it's a particularly subborn source anyway. Too much pure interlaced material, I think. DGBob does a good job though!

[Guest]

Originally posted by Milkman Dan
However, are you planning to do any low-level ASM optimizations with the code, or have you been doing that all along?

I ask, because I'm learning ASM (MASM) and looking to cut my teeth a bit on some code before moving along to something really complex. So if you've optimized it already, I can study that, and if you haven't, I can see if I can figure out what would be improved.It's planned but not yet implemented. You can find some low-level code in Decomb 4 for study. Also, it would pay to study the work of others, such as trbarry and sh0dan, just to mention a few. And, of course, Avisynth itself is chock full of low-level code.

Just so that this isn't totally off-topic, I've seen good improvements with the pattern matching so far. I feel safe turning it on now, instead of leaving it on guide=0 just to be safe.That is good to know. Thank you for reporting on your results.

Unfortunately, it doesn't help much on Kare Kano (at least so far as I've tinkered), but it's a particularly subborn source anyway. Too much pure interlaced material, I think. DGBob does a good job though! Make sure you get version 1.5.1 of DGBob(). Double rate output is broken in 1.5.0.

[bilu]

@neuron2

From DecombReferenceManual.html v.5.0b10:

On triple explanation:"By default, only two-way match testing is used, which tests for matching with the current and next frames."

On Appendix A. Telecide() Theory of Operation: "When it receives a request for a frame it gets access to the previous frame and the requested one (called the current frame)"

What's the default behaviour, matching previous/current or current/next?


Bilu

[Guest]

current/next

Thank you for pointing out the documentation error.

[Guest]

See the first post for the link to Decomb 5.0.0 beta 11. This is the last release (barring bugs) before the YV12 support is added.

Here are the changes (refer to docs for details):

1. Fast deinterlacing based on metricX is now used for deinterlacing frames still combed after field matching. Use FieldDeinterlace() externally with post=0/1 to Telecide() if you want the old way. Note: there is no blend mode in Telecide() now, so I'll never hear another complaint about ghosting. :)

2. DGBob() is now included in the Decomb dll.

3. A pattern override does not automatically declare out of pattern. It is now properly tested.

4. vmetrics are shown for all matches.

5. The shipped DLL is named by version.

6. The triple parameter can now be overridden.

7. gthresh, vthresh, and dthresh are now all floating point.

8. Docs updated.

I'll especially appreciate an audit of the docs against the shipped functionality. All feedback will be appreciated. YV12 is getting close!

[HomiE FR]

Thanks thanks thanks thanks! :D

I have some exams right now, but be sure that I'll do some proper tests with b11 before sunday. :) But with b10 I already noticed some good improvements with pattern guidance (with some mouth combing for instance, but I guess it'a a "side effect" of better pattern guidance).

By the way, thanks for the full vmetric information!

I'll be back. ;)

[Cyberia]

So Interpolation is now done *always* to remove leftover combing artifacts? Why not just default Blend to false, and leave the option?

[bilu]

@neuron2

Using Telecide(order=1,guide=1,post=4,debug=true):

A Monty Python's Telecined tree panning scene
=============================================
!Telecide: frame 7: matches: 98828 198989 97486
!Telecide: frame 7: vmetrics: 18 34 19 19 [chosen=9333040]
!Telecide: frame 7: [using n] [progressive] [in-pattern]
!Telecide: frame 8: matches: 97486 183712 106774
!Telecide: frame 8: vmetrics: 19 24 20 20 [chosen=9333040]
!Telecide: frame 8: [using n] [progressive] [in-pattern]
!Telecide: frame 9: matches: 106774 106774 240076
!Telecide: frame 9: vmetrics: 20 20 36 20 [chosen=9333040]
!Telecide: frame 9: [using c] [progressive] [in-pattern]

Your Star Trek hybrid challenge stream
=======================================
!Telecide: frame 7: matches: 210930 109769 210892
!Telecide: frame 7: vmetrics: 75 24 68 24 [chosen=9333040]
!Telecide: frame 7: [using c] [progressive] [out-of-pattern]
!Telecide: frame 8: matches: 210892 105463 210081
!Telecide: frame 8: vmetrics: 68 22 71 22 [chosen=9333040]
!Telecide: frame 8: [using c] [progressive] [out-of-pattern]
!Telecide: frame 9: matches: 210081 105268 215254
!Telecide: frame 9: vmetrics: 71 19 74 19 [chosen=9333040]
!Telecide: frame 9: [using c] [progressive] [out-of-pattern]


I've noticed that [chosen=9333040] appears everywhere during the whole movies. Tried on another movie with the same result.What is it? :confused:


Bilu

[Mentar]

I would also politely request to keep the blend option in Telecide, because interpolation is extremely rude to the eye. It catches the attention of the viewer much more than a ghost ever would. Please reconsider and enable blend at least as a toggle, because ALL my encodes are blend. For anime I consider interpolation as barely usable.

Otherwise, thanks very much for your efforts :)

[HomiE FR]

Mentar : I totally agree with Mentar on that point. I would like not to lose the blend switch which I used all the time with my anime source (I gave you some parts of it for tests). But I'm doing some more tests...

[Guest]

The metricX-based fast deinterlacing doesn't easily allow for blend deinterlacing. I will see if I can add it without too much hassle, otherwise I'll make FieldDeinterlace() observe hints and you can use that to get your blending:

Telecide(post=1,hints=true)
FieldDeinterlace(full=false)

My intent is to have a suite of possible postprocessing filters, so I'm not too bothered about just having a simple fast one built-in to Telecide. Thoughts?

BTW, YV12 is done, but now you've delayed the release. :)

[Guest]

@bilu

It's a bug in debug=true display. The 4th vmetric value shown is the chosen match's vmetric, and the one shown in brackets is garbage. It will be fixed in beta 12. Thank you for pointing it out.

[Guest]

The long-awaited YV12 support has arrived, coincidentally in beta 12! I also managed to put back the blend mode for deinterlacing. I hate to make my users unhappy. :)

Also, FieldDeinterlace() will now honor hints from Telecide() about interlaced/progressive frames. Hints are now on by default for Telecide().

Please report any bugs in the DLL or in the documentation here. Get the new beta from the link in the first post of this thread.

[HomiE FR]

Just one word can sum up what I think right now : THANKS! :)

I'm going to test it right now, I'll check the documentation too because I know it (too?) well.

Just a little mistake in the documentation according to your latest post :
hints (true/false, default false) enables Telecide() to pass hints to Decimate().
default should be true I think.

[bouis]

Decomb 5.0 Beta 11 and Beta 12 consistently crash on a certain scene from the movie "The Comancheros." The error message is "Avisynth: caught access violation at 0x00c5b2bc, attempting to read from 0x0284e44". Beta 9 does not crash and I haven't tried beta 10.

However, when I create a huffyuv2 of the scene, no crash occurs. I'd like to help, but what should I do?

I'm using this script:
loadplugin("E:\Decomb500b12.dll")

mpeg2source("coman.d2v")

#AssumeTFF().SeparateFields()

Telecide(guide=1,order=1,nt=10,post=0)
Decimate(cycle=5)

FieldDeinterlace(full=false,debug=true,dthreshold=255)
## FieldDeinterlace(full=false,threshold=255,dthreshold=8,debug=true,ovr="over.txt")

AddAudio()

[Guest]

@bouis

Please use a VOB splitter from the doom9 download page to make a small VOB containing the frame that cause the crash. Then put the VOB somewhere I can download it. Thank you.

Also, add Telecide, Decimate, and FieldDeinterlace one-by-one and see when it crashes. That will tell me which filter is having a problem.

Also, are you running YUY2 or YV12?

Is there anything funny about the frame? All dark, all bright, etc.?

BTW, thresholds of 255 to FieldDeinterlace defeat its purpose. I assume you are doing that for debugging.

[Guest]

Beta 12 is temporarily withdrawn. It has some serious problems.

[Guest]

Beta 13 fixes hard pattern guidance and makes backward matching more intelligent.

Is beta 13 the lucky charm? :p