DGDecIM: Frame-accurate Avisynth source filter for QuickSync

[Groucho2004]

I tried different x264, tried avs4x264mod piping, tried wrap it inside MP_Pipeline, etc., and they all fail the same.
I assume you tried different Avisynth versions as well, right?

[MeteorRain]

I assume you tried different Avisynth versions as well, right?

Not yet. This is a good point but I'm still doing multiple parallel encoding and can't replace the dll in the middle.

Also from experience if something was wrong inside the dgim dll thread, and switching from dgim to dgnv will solve the problem, I would first suspect the issue staying inside dgim, unless avs+ was doing something really strange and no one has noticed that before.

Anyway thanks for pointing that out. I'll try go back to avs and see what happens. (Of course after I wake up)

[bxyhxyh]

Mine is avisynth 2.6 icl10 build by Groucho2004, x264 is 2409.

When I open source file with LWLibavVideoSource, it was ok, dss2 was ok, ffvideosource was ok.

It is stopping like dss2, which sometimes stuck at 99%.

[Guest]

Please report your Intel driver version. We need to try with 3496 beta. I am away from my PC until later today, then I will run some longer tests.

[MeteorRain]

libmfxhw32.dll = 4.0.1533434.80609, 4.14.1.9
Intel driver = 10.18.10.3412 on 2014-1-22

Also tested on AVS 2.6

AVSMeter 1.7.3 [AVS2.6] by Groucho2004
AviSynth 2.60, build:Mar 9 2013 [13:28:27]
Active MT Mode: 0

Number of frames: 11790
......

Frame (current | last): 11749 | 11789
......

[bxyhxyh]

Intel driver 10.18.10.3412

[Guest]

Can you gents please try 3496 beta as I suggested?

I'm downloading MeteorRain's sample now for investigation.

@bxyhxyh

Please give me the bluray title and tell me if it is AVC, MPEG2, or VC1. Thank you.

[MeteorRain]

Can you gents please try 3496 beta as I suggested?
The previous link is dead. I assume you are talking about Intel® Iris™ and HD Graphics Driver for Windows* 7/8/8.1 64-bit
Version: 15.33.18.64.3496 (https://downloadcenter.intel.com/Detail_Desc.aspx?agr=Y&DwnldID=23714&lang=eng), right?

Will give it a shot.

No it still doesn't work

[bxyhxyh]

@neuron
It is anime called Paprika, mpeg-2. I'll try to upload sample.

[Guest]

Ok, thanks (Paprika is a cool anime). I have duplicated the issue with MeteorRain's stream. It seems to affect MPEG2 only. 3496 beta will not help you.

Initial indications point to an issue in the Intel SDK. Investigating...

[Guest]

OK, I have slipstreamed a possible fix into beta 18. It fixes MeteorRain's stream. Re-download beta 18 and update DGDecodeIM.dll. Please report your results.

MPEG2 is hosed in the Intel SDK and I am having to code workarounds and hope the house of cards remains standing. Meanwhile my post about it at the Intel forum remains waiting for support.

[MeteorRain]

So far so good for this particular file. I'll be doing several long time encoding recently, and will report if I find issues later.

Thanks for the quick fix!

[bxyhxyh]

Sample - https://www.dropbox.com/s/9bqh4syiu6zotet/00000.ts

[Guest]

I meant the deinterlace function inside DGIM/QS causes high CPU usage on libmfxhw32.dll even in hardware mode. What do you mean by high? Action in the hw dll should not be surprising when the hw engine is used.

Is the deinterlacer working under SW mode? You can easily show that it does not work. That's why it is mentioned in the Notes.txt file.

[Guest]

Sample - https://www.dropbox.com/s/9bqh4syiu6zotet/00000.ts Thanks for your sample. It's fixed by the latest beta 18.

[MeteorRain]

What do you mean by high? Action in the hw dll should not be surprising when the hw engine is used.

I was concerning about a higher CPU usage by the dll under de-interlaced mode.

As a general thinking, if the deinterlacer is working purely on hardware, there shouldn't be that much CPU usage inside the dll compared to non-deinterlaced mode. It looks like its speed is limited due to the single core performance instead of the iGPU chip.

[Guest]

I can't understand meaningless terms like "that much".

In my earlier posted results I had 6% CPU for deinterlace=0 and 9% for deinterlace=1. Do you find this to be problematic?

[MeteorRain]

In my earlier posted results I had 6% CPU for deinterlace=0 and 9% for deinterlace=1. Do you find this to be problematic?

I had process explorer (the sysinternal one) monitoring the CPU usage of all threads inside the process when I'm running avsmeter.

I observe the fact that a single thread inside libmfxhw32.dll consume 10.5% when deinterlacing. Considering it's a 4c8t processor, 10.5% equals to 84% usage of a single physical core, which I think is a rather high usage.

If it's multi-threaded decoding, at least its performance can be scaled on a multi-core processor. But consider that it's a single thread that takes that much resources, the performance is strictly limited to the single core performance.

Please correct me if I was wrong. Thanks.

========== EDIT

Also if you consider the decoding speed, the performance is more "problematic".

Without deinterlacing, it works at 250fps+ when consuming 6.5% and AVSMeter and DGIM consume the rest 6%.
With deinterlacing, it works at 70fps and consumes 10.5% and AVSMeter and DGIM consume 2% combined.

This result comes out under both circumstances that I did or did not limit the process to use only one core.

[Groucho2004]

I observe the fact that a single thread inside libmfxhw32.dll consume 10.5% when deinterlacing. Considering it's a 4c8t processor, 10.5% equals to 84% usage of a single physical core, which I think is a rather high usage.
That's a weird logical fallacy.
If you consider a real 8 core CPU where 1 thread is at 10.5% usage, the total CPU usage (which is by the way what AVSMeter measures) would be 1.3%.

The load for a single core CPU in this scenario would be 10.5%.

[MeteorRain]

That's a weird logical fallacy.
If you consider a real 8 core CPU where 1 thread is at 10.5% usage, the total CPU usage (which is by the way what AVSMeter measures) would be 1.3%.

The load for a single core CPU in this scenario would be 10.5%.

Apologize if I didn't state clearly. 10.5% usage I mentioned was against the total CPU usage. I also test it with 2 physical core given (Core 1 and Core 5) and it remains the same.

If this is not that clear, here is the screenshot.

http://i.imgur.com/npeeSA7.png

EDIT: If you see my edit section you should have some idea when I said 6.5% plus the rest 6% which is 12.5%, one-eighth of the whole processor.

[Groucho2004]

OK, I see what you mean. So, how is the usage distribution when you use less cores? I'm not sure what you mean by "I also test it with 2 physical core given (Core 1 and Core 5) and it remains the same".

[MeteorRain]

OK, I see what you mean. So, how is the usage distribution when you use less cores? I'm not sure what you mean by "I also test it with 2 physical core given (Core 1 and Core 5) and it remains the same".

I tested with 1 core assigned (by setting affinity), and with 2 cores assigned, and I observed the same usage. That is to say, it is single threaded and consumes as much as a single core and this becomes its bottleneck.

And I also concerned about the high CPU usage (and the much difference between deinterlace is on or off).
If the deinterlacing is done purely on hardware, there shouldn't be that large difference on the CPU usage.

I suppose the dll is just pulling frames from the GPU chip, right?
Please correct me if I'm wrong.

[Groucho2004]

Setting affinity to one core is not the same as having only one core available. You have to disable cores in the BIOS to simulate this.

[Guest]

And I also concerned about the high CPU usage (and the much difference between deinterlace is on or off).
If the deinterlacing is done purely on hardware, there shouldn't be that large difference on the CPU usage. Who says it's done purely in hardware? The Intel sample code also shows a similar impact for using the deinterlacer. And have you compared performance of (say) DXVA with the deinterlacer on and off? Don't be so fast to pin what you see as poor performance on DGDecodeIM. As I said earlier, I am not doing anything special, just following the Intel sample code.

I suppose the dll is just pulling frames from the GPU chip, right? Who knows? I don't have access to the details of the implementation.

Performance tweaking is on my priority list, and some improvement may be possible. I just don't know yet.

[nevcairiel]

I get similar results in my MediaSDK decoder implementation with deinterlacing. Around ~450-470 fps raw decoding on this particular 1080p H264 interlaced clip, and only ~140 fps with double-rate deinterlacing (or 70-80 with half rate, but really, who uses that).
Its the one part where NVIDIAs CUDA/CUVID decoder is faster, even before Maxwell, as it can decode at ~140 fps and then double-rate deint to 280 fps.

[Guest]

If it's multi-threaded decoding, at least its performance can be scaled on a multi-core processor. But consider that it's a single thread that takes that much resources, the performance is strictly limited to the single core performance.

Please correct me if I was wrong. There is only one iGPU. How do you intend to negate this fact through multithreading?

Really, if you think you can improve the architecture or implementation, you should contact Intel about it. I suspect that they have thought it through carefully, however.

[MeteorRain]

Who says it's done purely in hardware? The Intel sample code also shows a similar impact for using the deinterlacer. And have you compared performance of (say) DXVA with the deinterlacer on and off? Don't be so fast to pin what you see as poor performance on DGDecodeIM. As I said earlier, I am not doing anything special, just following the Intel sample code.

Who knows? I don't have access to the details of the implementation.

Performance tweaking is on my priority list, and some improvement may be possible. I just don't know yet. If you find it unusable as is, then my advice would be to not use it.

I never want to blame DGIM. The high usage comes from the Intel SDK and I was just curious about it and wondering if you have some idea. Well anyway it seems to be off-topic so I'll stop here.

Thanks for various ideas.

[Guest]

it seems to be off-topic It's not off-topic, just very low on the priority list right now. Pressing about it is distracting from the directions I want to go in right now. If you have some technical ideas about improving performance feel free to post about it.

All of which does not detract from my appreciation for your contributions to the thread (and the forum), for which thank you. :thanks:

[Guest]

I got double-rate deinterlacing working in the Intel sample (it's not super obvious). Look for a new beta tonight or tomorrow. I will also remove the time limit on the beta.

Also, at this point I politely decline further free beta licenses, as I plan to make a formal first release tomorrow.

[asteri]

Hello,

I cant make DGDecIM work in HW mode on laptop with Intel i7-2630QM. Of course it has QuickSync http://ark.intel.com/products/52219/Intel-Core-i7-2630QM-Processor-6M-Cache-up-to-2_90-GHz
I cant even install Intel GPU drivers and in Device manager I dont see iGPU just only the Quadro1000. In BIOS I dont see any option to enable iGPU.

Any advice welcome!

@neuron2
Can you make also x64 version with initial release?

[Video Dude]

I cant make DGDecIM work in HW mode on laptop with Intel i7-2630QM. Of course it has QuickSync http://ark.intel.com/products/52219/Intel-Core-i7-2630QM-Processor-6M-Cache-up-to-2_90-GHz
I cant even install Intel GPU drivers and in Device manager I dont see iGPU just only the Quadro1000. In BIOS I dont see any option to enable iGPU.



If you can't find the iGPU in the BIOS, look in the user manual or pdf for changing the graphics mode. It may be listed in a different section or labeled different than what you were expecting. Do a Google search with the laptop model and how to enable the graphics in the BIOS.

Your laptop likely has switchable graphics (Nvidia Optimus). There should be a graphics control panel that lets you specify which graphics to use per installed program. If you don't see it, make sure you have both the Intel and Nvidia drivers installed correctly. For some laptops, installing the Nvidia drivers found on the Nvidia site will disable Nvidia Optimus. Nvidia mentions this on their driver download page and you must use the Nvidia OEM drivers distributed directly from the laptop manufacturer instead.

[asteri]

Thanks for tip, but I cant find any Intel graphics related drivers on vendor website. The model is HP EliteBook 8560w, all drivers for this model should be here:
LINK (http://h20565.www2.hp.com/portal/site/hpsc/template.PAGE/public/psi/swdHome/?sp4ts.oid=5071173&spf_p.tpst=swdMain&spf_p.prp_swdMain=wsrp-navigationalState%3DswEnvOID%253D4059%257CswLang%253D%257Caction%253DlistDriver&javax.portlet.begCacheTok=com.vignette.cachetoken&javax.portlet.endCacheTok=com.vignette.cachetoken)
All drivers are nVidia related :/ (or ATI). This laptop is aimed to CAD, CAM application using certified graphics card.

Any other suggestions?

[Guest]

You need to go to your support site for your PC and ask them. I cannot do that support in this thread.

[Nico8583]

Hi, a little question : with your DGDecIM (or DGMVCSource), is it necessary, recommended or useless to install a codec pack (like ffdshow) in order to encode avs to x264 ?

Also, I've seen on an earlier post you have changed DGI name but I don't see anything on release note ? And you write "This beta expires on April 1, 2014.", what happens after ?

[CarlPig]

I think DGDecIM goes Final April 1, 2014.

[Groucho2004]

Hi, a little question : with your DGDecIM (or DGMVCSource), is it necessary, recommended or useless to install a codec pack (like ffdshow) in order to encode avs to x264 ?
Necessary - no
Recommended - no
Useless - Yes

:D

[Nico8583]

Necessary - no
Recommended - no
Useless - Yes

:D

Thanks, it's simple and clear :D so I need to install only Avisynth, not ffdshow or matroska :)

[Guest]

As I mentioned in #379 I will remove the time limit before it affects anyone.

I've run into a big problem with the VPP in double-rate deinterlacing mode. On a random access I have to reset the VPP. In single-rate mode it works fine, but in double-rate mode decoding after a VPP reset results in a crash in libmfxhw32.dll. All attempts to work around it have failed so far. I suppose I could try to completely tear down the pipeline and re-init it, but that will change the design radically and why should I have to do that when the documentation says I only have to reset the engine?

There's no way to get support because my posts at the Intel forum don't get answered. Simple things sometimes get answered, sure, but it seems they are unwilling to duplicate reported problems and the actual developers aren't on the forum, and nobody really seems to care. I will likely just release what I have and suspend further development until there is some improvement in support. Anyway, Nvidia's Maxwell and Pascal are looking really exciting and there is great support, so why should I continue banging my head against a wall with an SDK that is hosed up in many ways and isn't properly supported? I don't want to spend my time getting frustrated randomly trying things and building a house of cards. To be honest, I rue the day I even started this project.

[aegisofrime]

As I mentioned in #379 I will remove the time limit before it affects anyone.

I've run into a big problem with the VPP in double-rate deinterlacing mode. On a random access I have to reset the VPP. In single-rate mode it works fine, but in double-rate mode decoding after a VPP reset results in a crash in libmfxhw32.dll. All attempts to work around it have failed so far. I suppose I could try to completely tear down the pipeline and re-init it, but that will change the design radically and why should I have to do that when the documentation says I only have to reset the engine?

There's no way to get support because my posts at the Intel forum don't get answered. Simple things sometimes get answered, sure, but it seems they are unwilling to duplicate reported problems and the actual developers aren't on the forum, and nobody really seems to care. I will likely just release what I have and suspend further development until there is some improvement in support. Anyway, Nvidia's Maxwell and Pascal are looking really exciting and there is great support, so why should I continue banging my head against a wall with an SDK that is hosed up in many ways and isn't properly supported? I don't want to spend my time getting frustrated randomly trying things and building a house of cards. To be honest, I rue the day I even started this project.

That is disappointing on Intel's part. I do want to get an nvidia GPU, but the 750Ti isn't fast enough for gaming, and I don't want to get a GTX 770 when the 870 is likely around the corner. :(

Have you tried asking egur for help?

[Guest]

Here is beta 19 with a few small fixes and the time limit is removed.

http://neuron2.net/misc/dgdecim_b19.zip

[Sharc]

Thanks for all your efforts.
Same destiny for DGMVCsource?

[Guest]

Same destiny for DGMVCsource? We'll see. It may not be too hard to merge the two.

[Nico8583]

We'll see. It may not be too hard to merge the two.

Good news :) and yes, thanks for all your work !!

[asteri]

Here is beta 19 with a few small fixes and the time limit is removed.

http://neuron2.net/misc/dgdecim_b19.zip

Remove line from Notes.txt ;)
9. This beta expires on April 1, 2014.

[Guest]

Thanks for pointing it out.

[aegisofrime]

Out of curiosity, after the recent setbacks due to Intel's lack of cooperation, is it safe to conclude that this project is dead in the water? After reading rumors about Maxwell high end parts being delayed to next year, I'm thinking about finally getting that GTX 750Ti :(

[CarlPig]

Out of curiosity, after the recent setbacks due to Intel's lack of cooperation, is it safe to conclude that this project is dead in the water? After reading rumors about Maxwell high end parts being delayed to next year, I'm thinking about finally getting that GTX 750Ti :(

Here is a link for maxwell support: http://forum.doom9.org/showthread.php?t=170349

[Guest]

Out of curiosity, after the recent setbacks due to Intel's lack of cooperation, is it safe to conclude that this project is dead in the water? I cannot proceed further without support and at this time it is not forthcoming, although that can change. So draw your own conclusions. Whatever happens, it still works pretty good as is but won't get the bells and whistles.

[WorBry]

.... it still works pretty good as is....

Indeed it does, and if double-rate deinterlacing is the only bell/whistle that is not possible just now, folks can just as easily use a deinterlacer of their choice. Any possibility of adding audio demux though?

[Nico8583]

MVC support would be marvelous before to add new functionality :)