DGDecIM: Frame-accurate Avisynth source filter for QuickSync

[Guest]

It's hard to help when you disregard my questions and requests. All I can say is there appears to be some problems with your system or environment.

[MeteorRain]

It happens to all the files I have.

I ran a few test with different file path,

This one returns "there' is no function name "" ":
this one returns "there is no function named "gi" ":
this one returns "there is no function named "" ":
this one returns "there is no function named "gi" ":


Do you have a corrupted file system?

Or are you using a mapped folder from another system / through virtual machine?

I have observed similar file corruption under vmware / virtualbox shared folders mounted on a Linux guest. Not sure if it's a similar problem but there's a big chance the avs file is not written properly or not read by other application properly.

[RedDwarf1]

I can't duplicate it. How are you opening the script? Just playing in VirtualDub? What version of Avisynth?

Post your DGI file.

Anybody else seeing this?

Yes I am getting a similar thing.

this works

LoadCplugin (AviSynthPluginsDir + "CPlugins\yadif.dll")
DGSourceIM(engine = 1, deinterlace=1, debug=true, "M:\Video Record\DVBViewer\test\2014-03-16_21-40-44_BBC Red Button HD_.ts")
#FlipVertical()
#yadif(order=1)

either of the following gives> script error: there is no function named ""

LoadCplugin (AviSynthPluginsDir + "CPlugins\yadif.dll")
DGSourceIM(engine = 1, deinterlace=0, debug=true, "M:\Video Record\DVBViewer\test\2014-03-16_21-40-44_BBC Red Button HD_.ts")
#FlipVertical()
yadif(order=1)

LoadCplugin (AviSynthPluginsDir + "CPlugins\yadif.dll")
DGSourceIM(engine = 1, deinterlace=1, debug=true, "M:\Video Record\DVBViewer\test\2014-03-16_21-40-44_BBC Red Button HD_.ts")
FlipVertical()
#yadif(order=1)

Opening a dgi works as it should.

[Guest]

@RedDwarf1

The filename should be the first parameter! I give clear examples in the Notes.txt file.

DGSourceIM("M:\Video Record\DVBViewer\test\2014-03-16_21-40-44_BBC Red Button HD_.ts", engine=1, deinterlace=1, debug=true)
FlipVertical()

I did duplicate the issue with your parameter order. It seems to be an idiosyncracy of Avisynth script parsing. Probably Gavino could explain what is going on. I will ping him. It's not something I'm bothered about, as things work fine with the input file given first (at least for me).

[RedDwarf1]

The auto created dgi file

Ah okay I will try that

[RedDwarf1]

That seems to of stopped the error

[Guest]

OK, good, and thank you for the DGI file. I PM'ed Gavino about this as I see it as an Avisynth issue.

It's not the same as lansing's case, unless he has not given the exact scripts.

[Guest]

Here is beta 17 with three bug fixes:

* Error messages and graceful exits were added for 10-bit and chroma formats other than 4:2:0. These checks are in the indexer, so don't try to use a DGI that you made previously! Now you will be unable to create the DGI unless you cheat and use the DGNV indexer. I will fix the latter at some point also.

* Some failures with DecodeHeader() were fixed. Some files that hung on open now work properly.

* When a pipeline init failure occurred a crash would occur. Now a proper ThrowError() is executed.

http://neuron2.net/misc/dgdecim_b17.zip

[RedDwarf1]

@RedDwarf1

The filename should be the first parameter! I give clear examples in the Notes.txt file.
Some people do read before bed, just not me. lol It was nearly 12:30am and I should of been in bed but I was attempting to reproduce the errors that lansing had posted in order to help out. It's nice to have deinterlacing available and I did not want some unresolved error holding things up.

Initial testing shows that it seems to work fine with BBC MBAFF video. All the MBAFF video that I have seen previously looked like progressive video and only the end credits were interlaced. However this Red Button Click programme does have interlaced video, the news ticker is interlaced and so are some other areas of the video. DGDecIM seems to be handling it fine.

DGSourceIM("M:\Video Record\DVBViewer\test\2014-03-16_21-40-44_BBC Red Button HD_.ts", engine=1, deinterlace=1, debug=true)
FlipVertical()

I did duplicate the issue with your parameter order. It seems to be an idiosyncracy of Avisynth script parsing. Probably Gavino could explain what is going on. I will ping him. It's not something I'm bothered about, as things work fine with the input file given first (at least for me).
Thanks for explaining things. It seems to work if a function is not used after the source filter. If a variable is declared before any function is used then it works fine. Even with the filename after the parameters or with a video filename.

I had not used any parameters before this because deinterlacing was not available when I last tried DGDecIM. I just stuck them in to test it without reading the notes ;)

this works
LoadCplugin (AviSynthPluginsDir + "CPlugins\yadif.dll")
DGSourceIM( "M:\Video Record\DVBViewer\test\2014-03-16_21-40-44_BBC Red Button HD_.ts")
#FlipVertical()
dummy=0
yadif(order=1)
return last
and this, I did purposely put the parameters before the filename and it works!
LoadCplugin (AviSynthPluginsDir + "CPlugins\yadif.dll")
DGSourceIM( engine = 1, deinterlace=1, debug=true, "M:\Video Record\DVBViewer\test\2014-03-16_21-40-44_BBC Red Button HD_.ts")
#FlipVertical()
dummy=0
#yadif(order=1)
return last

but not this
LoadCplugin (AviSynthPluginsDir + "CPlugins\yadif.dll")
DGSourceIM( "M:\Video Record\DVBViewer\test\2014-03-16_21-40-44_BBC Red Button HD_.ts")
FlipVertical()
dummy=0
yadif(order=1)
return last

or this
LoadCplugin (AviSynthPluginsDir + "CPlugins\yadif.dll")
DGSourceIM( "M:\Video Record\DVBViewer\test\2014-03-16_21-40-44_BBC Red Button HD_.ts")
#FlipVertical()
#dummy=0
yadif(order=1)
return last

OK, good, and thank you for the DGI file. I PM'ed Gavino about this as I see it as an Avisynth issue.

It's not the same as lansing's case, unless he has not given the exact scripts.

I was getting the same errors because I did get avisynth reporting "I don't know what "dgi" means" as well as 'script error: there is no function named ""'. That was why I thought that it was related to the same issue. I cannot get it to do that now for some reason - a nights sleep and no longer being so tired.

[Guest]

Are you saying something is still not working? I am having trouble getting your point.

I don't have yadif installed but this works fine for me:

loadplugin("dgdecodeim.dll")
dgsourceim("00000.mts",deinterlace=1,engine=1)
FlipVertical()
dummy=0
FlipHorizontal()
return last

If something like that fails for you, can you try it without C plugin stuff?

[RedDwarf1]

Are you saying something is still not working? I am having trouble getting your point.

I don't have yadif installed but this works fine for me:

loadplugin("dgdecodeim.dll")
dgsourceim("00000.mts",deinterlace=1,engine=1)
FlipVertical()
dummy=0
FlipHorizontal()
return last

If something like that fails for you, can you try it without C plugin stuff?

Sorry for complicating things.

Something, maybe the Avisynth script parsing as you say, causes an error when an internal or external filter is called immediately following the source filter, unless a variable declaration is placed immediately after the source filter.

For example, this gives an error 'script error: there is no function named ""' line 2
DGSourceIM( "M:\Video Record\DVBViewer\test\2014-03-16_21-40-44_BBC Red Button HD_.ts")
FlipVertical()
#dummy=0
return last

this works:
DGSourceIM( "M:\Video Record\DVBViewer\test\2014-03-16_21-40-44_BBC Red Button HD_.ts")
dummy=0
FlipVertical()
return last

because there is no internal or external filter following the source media file filter, there is a "dummy" variable declaration which prevents the error from occurring.

The question is, why is this occurring? Is it some anomaly with avisynth script parsing or is there a problem with DGDecIM?

[Groucho2004]

For example, this gives an error 'script error: there is no function named ""' line 2
DGSourceIM( "M:\Video Record\DVBViewer\test\2014-03-16_21-40-44_BBC Red Button HD_.ts")
FlipVertical()
#dummy=0
return last

Which version of Avisynth are you using?
Have you tried other versions?

[Guest]

Hmm, very strange, this is failing for me:

loadplugin("H:\Don\Programming\C++\DGDecIM\DGDecodeIM\x32\release\dgdecodeim.dll")
dgsourceim("00000.mts")
FlipVertical()

But it works if I specify the DGI file. I'm going to do some tracing now. BTW, I run Avisynth 2.5.8.

[Guest]

Oops, I made a boo-boo. Fix coming shortly.

:stupid:

[Guest]

Please re-download beta 17 and everything will be milk and honey again.

The technical explanation for the curious: Avisynth had set aside space for the argument holding the filename array, but that space only has room for the string actually given. Then when I mapped from the source file to the DGI file name I overwrote that Avisynth array space with the new longer filename, not realizing the array might be too small (actually I didn't think about it at all and just typed strcpy(dgi, ifile) where dgi was the first parameter to the create function, a pointer to the first argument space). I had to make a local array with proper space and then copied the first parameter to it, after which I could append ".dgi" without hosing up Avisynth. Moral of the story: do not modify Avisynth arguments.

[RedDwarf1]

Thank you neuron2.

I had just tested using 2 different source filters and they worked regardless of any workarounds. It's good to see that you worked it out and splatted another bug.

I'm using 2.6 a5 MT build

[Guest]

You're welcome, RedDwarf1, and thank you for your testing and reports.

[RedDwarf1]

I have just done a speed test and I got very low speed :( Much slower than DGDecNV, barely averaging above realtime! DGDecIM is deinterlacing the same BBC Red Button MBAFF video that I have been testing but this is the first speed test that I have done for deinterlacing.

M:\Video Record\DVBViewer\test>avsmeter "red button.avs"

AVSMeter 1.7.6 (AVS 2.6, x86) by Groucho2004
AviSynth 2.60, build:Mar 9 2013 [13:28:27]
Active MT Mode: 0

Number of frames: 3476
Length (hhh:mm:ss.ms): 000:02:19.040
Frame width: 1920
Frame height: 1080
Framerate: 25.000 (25000/1000)
Interlaced: No
Colorspace: YV12

Frames processed: 3476 (0 - 3475)
FPS (min | max | average): 9.121 | 131.2 | 26.59
CPU usage (average): 3%
Thread count: 19
Physical Memory usage (peak): 332 MB
Virtual Memory usage (peak): 589 MB
Time (elapsed): 000:02:10.715

#LoadCplugin (AviSynthPluginsDir + "CPlugins\yadif.dll")
DGSourceIM("M:\Video Record\DVBViewer\test\Red Button HD.dgi", engine = 1, deinterlace=1)

What has slowed it down?

[Guest]

Apparently the deinterlacer has slowed it down. ;)

I need your stream please.

I too noticed that the deinterlacer seems slow. I'm not doing anything different or weird, just following their sample code. I will try to look into it. Still, I reckon, real-time deinterlacing of HD is tolerable, especially when transcoding, compared to a SW solution.

Have you compared it to DXVA via QS-enabled DirectShow filters?

[RedDwarf1]

Apparently the deinterlacer has slowed it down. ;)

I need your stream please.
uploading to mediafire

I too noticed that the deinterlacer seems slow. I'm not doing anything different or weird, just following their sample code. I will try to look into it, but if it becomes an Intel support issue, forget it, their support is virtually nonexistent. Still, I reckon, real-time deinterlacing of HD is tolerable, especially when transcoding, compared to a SW solution.

Have you compared it to DXVA via QS-enabled DirectShow filters?
Not yet but I will later. I'm heading to bed ASAP.

[RedDwarf1]

FullHD transport stream (https://www.mediafire.com/?cfid42cpl4z4ldi)

[yup]

Hi neuron2!
I want testing beta 17 and get error about license. I am replace only dll and exe file in my directory. DGIndexNV also get the same error.
My PC Windows 7 64 professional, without connection to network (may be it is reason).

yup.

[aegisofrime]

Apparently the deinterlacer has slowed it down. ;)

I need your stream please.

I too noticed that the deinterlacer seems slow. I'm not doing anything different or weird, just following their sample code. I will try to look into it, but if it becomes an Intel support issue, forget it, their support is virtually nonexistent for end users like us. Still, I reckon, real-time deinterlacing of HD is tolerable, especially when transcoding, compared to a SW solution.

Have you compared it to DXVA via QS-enabled DirectShow filters?

I haven't had a chance to try the deinterlacer (currently overseas away from home), but I'm curious as to how's the quality? Is it to much to hope that it might be as good as QTGMC?

[Selur]

Is it to much to hope that it might be as good as QTGMC?
yes, it definitely is, you should hope for more like Yadif-level (or a bit worse) quality deinterlacing.

[Guest]

I don't find it to be worse than yadif! I find it closer to qtgmc quality. I'll post some screenshots when I get a chance.

[Guest]

Hi neuron2!
I want testing beta 17 and get error about license. I am replace only dll and exe file in my directory. DGIndexNV also get the same error.
My PC Windows 7 64 professional, without connection to network (may be it is reason). When you disconnect your internet connection, your machine ID changes. Make a new license for that configuration and *add* it to your existing license file. Please follow up in email if needed as I don't want to clutter this thread with licensing issues.

[Guest]

FullHD transport stream (https://www.mediafire.com/?cfid42cpl4z4ldi) Thank you.

I am seeing better performance than what you reported (see below, second is with deinterlacer). What is your processor? Mine is an i7 4770.

C:\Users\Don\Downloads>avsmeter "2014-03-16_21-40-44_BBC Red Button HD_.avs"

AVSMeter 1.7.5 (AVS 2.5, x86) by Groucho2004
AviSynth 2.58, build Dec 22 2008 [08:46:51]

Number of frames: 3476
Length (hhh:mm:ss.ms): 000:02:19.040
Frame width: 1920
Frame height: 1080
Framerate: 25.000 (25000/1000)
Interlaced: No
Colorspace: YV12

Frames processed: 3476 (0 - 3475)
FPS (min | max | average): 35.33 | 449.2 | 320.0
CPU usage (average): 6%
Thread count: 12
Physical Memory usage (peak): 346 MB
Virtual Memory usage (peak): 444 MB
Time (elapsed): 000:00:10.864

C:\Users\Don\Downloads>avsmeter "2014-03-16_21-40-44_BBC Red Button HD_.avs"

AVSMeter 1.7.5 (AVS 2.5, x86) by Groucho2004
AviSynth 2.58, build Dec 22 2008 [08:46:51]

Number of frames: 3476
Length (hhh:mm:ss.ms): 000:02:19.040
Frame width: 1920
Frame height: 1080
Framerate: 25.000 (25000/1000)
Interlaced: No
Colorspace: YV12

Frames processed: 3476 (0 - 3475)
FPS (min | max | average): 8.340 | 96.51 | 63.45
CPU usage (average): 9%
Thread count: 13
Physical Memory usage (peak): 349 MB
Virtual Memory usage (peak): 571 MB
Time (elapsed): 000:00:54.786

[Guest]

Here is a deinterlacer comparison. The first is QTGMC and the second is QS. This is from one of my deinterlacer torture clips, so neither looks very good, but they appear comparable to me, although QTGMC clearly looks sharper. Both handle the diagonal lines well without aliasing. Given the relative slowness of QS deinterlacing compared to NV, I'd have to expect that they are doing something useful with the processing time. ;)

[kalehrl]

QS is definitively better than Yadif but worse than QTGMC even at Super Fast preset.

[Guest]

That pretty much sums it up. One must decide if the extra quality of QTGMC is worth the CPU processing time. IMHO, a lot depends on the quality of the input stream. If it is typical broadcasted stuff, one might argue that QTGMC is overkill. For bluray streams and other high-quality sources, one can make a case for QTGMC. It is good to have options.

Remember too that QS is real-time so it can be used in players, etc.

[Guest]

Intel beta driver 3496 fixes the issue with HW decode of VC1, but I have to release a new beta that doesn't force SW for VC1 before you can take advantage of it. I want to investigate one more issue I am aware of for MPEG2 streams before I do that.

[Guest]

Here is beta 18:

* VC1 HW decoding is now supported. Use it only if you have Intel driver 3496 beta or greater installed, otherwise, use SW decoding (engine=2).

* Seeking to the last GOP in MPEG2 files now works.

* Fixes bad frames at the beginning of the stream for some MPEG2 streams.

http://neuron2.net/misc/dgdecim_b18.zip

My priorities now are:

1. Double rate deinterlacing.

2. Crop/resize on iGPU.

3. Create documentation and make first formal release.

4. MVC support.

5. Performance tweaking.

6. IVTC.

Did I miss anything?

[lansing]

beta17 fixed my problem with the "there' is no function name "" " issue, thanks RedDwarf1 and neuron2 for the follow up.

And where can I get the beta Intel driver? I used their site's auto detect and they only gave me 3412, which I already have.

]
Did I miss anything?

HW IVTC?

[Guest]

Don't do auto detect. Do a search for 3496 using the browse function.

https://downloadcenter.intel.com/Detail_Desc.aspx?agr=Y&DwnldID=23644&keyword=3496&lang=eng

IVTC will be worth considering if it is properly adaptive, reliable, etc. I have to do some testing. For example, if there is no postprocessor to catch bad matches then one has to run a SW external postprocessor, which negates much of the value of doing IVTC on the iGPU.

[WorBry]

That pretty much sums it up. One must decide if the extra quality of QTGMC is worth the CPU processing time. IMHO, a lot depends on the quality of the input stream. If it is typical broadcasted stuff, one might argue that QTGMC is overkill. For bluray streams and other high-quality sources, one can make a case for QTGMC. It is good to have options.

Remember too that QS is real-time so it can be used in players, etc.

Another point to consider is that QTGMC, in default mode, doesn't preserve the original fields (requires highly CPU-intensive SourceMatch/Lossless modes to do so) and, I assume, the QS 'double-rate' deinterlacer will, in order that the original fields can be recovered and re-interlaced downstream, if desired ? So in comparing quality, perhaps YadifMod-NNEDI3 would be a more valid reference as a high-end 'pure' (in the classic sense) deinterlacer?

[Guest]

Another point to consider is that QTGMC, in default mode, doesn't preserve the original fields Can you elaborate a little on this please? I wasn't aware of that.

BTW, my deinterlacer comparison is a bit off because the QTGMC is doing double rate and the QS is doing single rate. When I complete the QS double rate support, we can compare them fully correctly.

[bxyhxyh]

In index file it says SIZ 1920 x 1088
Source is 1920x1080.
Is it ok?

[Guest]

Yes, it shows the coded size not the display size. Don't worry, though, it will be cropped for output.

[WorBry]

Can you elaborate a little on this please? I wasn't aware of that.


The English Instructions included in the QTGMC package download give a more complete explanation:

http://forum.doom9.org/showthread.php?p=1423459#post1423459

But basically, in default mode, the temporal gaussian blur and resharpening functions that are applied to the core bob (to eliminate interline shimmer/twitter) alter both the interpolated and original field pixels. In fact, if you re-interlace the output, what you see on playback (in a media player with bob-deinterlacer) is a kind of undulating distortion of fine line detail and high-contrast edges where shimmer would otherwise have appeared.

There are two optional 'lossless' modes, one of which does restore the 'exact pixels' of the source fields, but possibly at the expense of some residual shimmer, minor combing and noise. Like I said, the TQGMC 'instructions' explain it more fully.

[Guest]

Thank you for the useful explanation and link.

Maybe QS doesn't preserve them too? We'll have to see when I get double rate implemented.

[Mixer73]

Subbed, I do a bit of encoding on a mobile machine with QS so I wonder what benefit using this would bring for MPEG2->264?

[Guest]

It serves decoded video via Avisynth at high speed and with low CPU usage in a frame-accurate manner. It has all the same features as DGDecNV.

[MeteorRain]

I just experienced a deadlock(?) in DGIM when encoding a tv cap. DGIM was used with deinterlace=1.

It was a 2 hours long tv capture video file and I just left it encoding and when I came back it halted with DGIM consuming 100% of a single core.

I reproduced the problem with exact the same process. It halted at (in my case) frame #53922 when encoding, in both try.

However if I open the avs inside VDM, jump to #53500 and then play it for about 1000 frames, it works perfectly without any problems.

I'm still trying to find out the minimum condition to reproduce the issue, so just a FYI for now.

[MeteorRain]

OK I now can reproduce the issue if I started from #40010 and DGIM halts at #13925(+40010) when encoding, with or without deinterlace.

I come up with a TS file of 931MB which can reproduce the issue. Further trim the file and the issue will not happen.

Do you want me to upload the file somewhere?

dgsourceim("testc.dgi")
crop(0,0,16,16)

[MeteorRain]

FPS (min | max | average): 35.33 | 449.2 | 320.0
CPU usage (average): 6%

FPS (min | max | average): 8.340 | 96.51 | 63.45
CPU usage (average): 9%


I'm also curious about the CPU usage. By using process explorer I observed very high usage from libmfxhw32.dll. Expected behavior?

EDIT: Maybe It wasn't clear. I meant the deinterlace function inside DGIM/QS causes high CPU usage on libmfxhw32.dll even in hardware mode. Is the deinterlacer working under SW mode?

[bxyhxyh]

I just experienced a deadlock(?) in DGIM when encoding a tv cap. DGIM was used with deinterlace=1.

It was a 2 hours long tv capture video file and I just left it encoding and when I came back it halted with DGIM consuming 100% of a single core.

I reproduced the problem with exact the same process. It halted at (in my case) frame #53922 when encoding, in both try.

However if I open the avs inside VDM, jump to #53500 and then play it for about 1000 frames, it works perfectly without any problems.

I'm still trying to find out the minimum condition to reproduce the issue, so just a FYI for now.
I have same issue with bluray source.
I didn't use deinterlace or any other parameters. I can't upload sample now because of internet problems.

[Guest]

Do you want me to upload the file somewhere?

dgsourceim("testc.dgi")
crop(0,0,16,16) Yes please. Also I need your encoder version and command line. Thanks.

[MeteorRain]

Yes please. Also I need your encoder version and command line. Thanks.

avs2avi dead.avs -c null -o n

avs2avi alone can reproduce the issue.

I'll find a place to upload the huge file. Hold on....

https://copy.com/0LObIiiRjWbp Be sure to remove the link if it is inappropriate for this forum.

Since it hangs when decoding a specific GOP, I cut the file one GOP after. So you should be able to hit the issue on the last GOP of this file.

[Groucho2004]

avs2avi dead.avs -c null -o n

avs2avi alone can reproduce the issue.
Try VirtualDub analysis pass or avsmeter for this purpose.

[MeteorRain]

Try VirtualDub analysis pass or avsmeter for this purpose.

AVSMeter 1.7.3 [AVS2.6] by Groucho2004
AviSynth+ 0.1 (r1576, x86)

Number of frames: 11790
Length (hhh:mm:ss.ms): 000:06:33.393
Frame width: 16
Frame height: 16
Framerate: 29.970 (30000/1001)
Interlaced: No
Colorspace: YV12

Frame (current | last): 11739 | 11789
FPS (cur | min | max | avg): 183.00 | 35.72 | 265.26 | 182.36
CPU usage (current | average): 11% | 12%
Thread count: 28
Physical Memory usage: 125 MB
Virtual Memory usage: 252 MB
Time (elapsed | estimated): 000:01:04.378 | 000:01:04.652

Press ESC to stop...


I didn't think it's avs2avi's fault because I tried different x264, tried avs4x264mod piping, tried wrap it inside MP_Pipeline, etc., and they all fail the same. Process explorer clearly shows the high cpu usage inside the dgim library dll thread.