DGDecIM: Frame-accurate Avisynth source filter for QuickSync

[Guest]

I've made enough progress on supporting the Intel Media SDK in DGDec to start a new thread. I plan to have DGDec support both nVidia and Intel decode engines but for the purposes of this thread I will call it DGDecIM.

The chick has poked its head through the eggshell but still has to wriggle its whole body out. What I have done is this: I made a DGI for an AVC stream (panama.264 famous in another thread) using DGIndexNV in the normal way. Then I modified DGDecodeNV to support the Intel decode engine using the same DGI file. Random access is working fine and I get over 200fps for a 1080p stream with 8% CPU usage in QuickSync mode. Pretty fine! The implementation uses what I called a potential simplified design in the DGMVCSource thread, so that one thread and consequent synchronization (with large Windows scheduling delays) is eliminated, significantly increasing the decode frame rate.

Before releasing a beta, I still need to add VC1 and MPEG2 and generally check robustness and end of stream handling. Also, I can consider adding support for Intel HW cropping/resizing, deinterlacing, IVTC, etc., and add MVC support. But things are looking good!

-----
Current version:

http://neuron2.net/misc/dgdecim_b20.zip
-----

NOTES ON POSTING IN THIS THREAD:

Rules 3 and 11 (rules) are enforced broadly in this thread and posters are asked to make sure all posts have useful technical content and/or report defects. Requests about status of developments, expressions of approval or disapproval thereabout, chit-chat, and other random nontechnical posts are eschewed and may be deleted. I believe this policy results in a more useful and readable thread. Thank you for your understanding and cooperation.

Posts reporting defects should provide a source stream, the Avisynth script, and instructions for demonstrating the defect with that same source stream and script.

[mastrboy]

It has begun

Will it be licensed as DGIndexNV is? and if so, would existing DGIndexNV users be able to reuse their licenses or would it require a new one?

[Guest]

I haven't decided about that stuff yet.

[Guest]

Status update: I have implemented the EOS stream handling and it is working fine. I have added support for MPEG2 and VC1. Unfortunately, the current HW support for VC1 is broken (http://software.intel.com/en-us/forums/topic/494916) and a fix is expected possibly at the end of February. The VC1 support works fine in SW mode (which is way slower of course) so I will release a beta that forces SW mode for VC1 input. I want to do a little more testing and then I'll give you guys a beta, possibly as early as tomorrow.

[Guest]

Here is beta 1:

http://neuron2.net/misc/dgdecodeim_b1.zip

As mentioned earlier, this requires you to use DGIndexNV to create your index file. I am working on an indexer that can be used without Nvidia.

Please read the Notes.txt document for usage information.

Your feedback will be appreciated.

EDIT: In some initial tests, QuickSync appears about 20-25% faster than CUVID.

[Stereodude]

1080i MPEG2 Intel engine:

Number of frames:                 10001
Length (hhh:mm:ss.ms):    000:05:33.700
Frame width:                       1920
Frame height:                      1080
Framerate:                       29.970 (30000/1001)
Colorspace:                        YV12

Frames processed:                10001 (0 - 10000)
FPS (min | max | average):       79.22 | 320.96 | 191.33
CPU usage (average):             11%
Thread count:                    31
Physical Memory usage (peak):    565 MB
Virtual Memory usage (peak):     715 MB
Time (elapsed):                  000:00:52.272

1080i MPEG2 Nvidia engine:

Number of frames:                 10001
Length (hhh:mm:ss.ms):    000:05:33.700
Frame width:                       1920
Frame height:                      1080
Framerate:                       29.970 (30000/1001)
Colorspace:                        YV12

Frames processed:                10001 (0 - 10000)
FPS (min | max | average):       68.01 | 261.23 | 141.07
CPU usage (average):             13%
Thread count:                    6
Physical Memory usage (peak):    526 MB
Virtual Memory usage (peak):     625 MB
Time (elapsed):                  000:01:10.896

[Guest]

Thanks, interesting! That's about 35% faster with 2% less CPU using QuickSync.

Now, if Intel can fix the VC1 HW support, we will be golden.

[Stereodude]

1080p H264 (Blu-Ray) Intel engine:

Number of frames:                 10000
Length (hhh:mm:ss.ms):    000:06:57.083
Frame width:                       1920
Frame height:                      1080
Framerate:                       23.976 (24000/1001)
Colorspace:                        YV12

Frames processed:                10000 (0 - 9999)
FPS (min | max | average):       50.13 | 338.37 | 170.68
CPU usage (average):             4%
Thread count:                    31
Physical Memory usage (peak):    571 MB
Virtual Memory usage (peak):     691 MB
Time (elapsed):                  000:00:58.591

1080p H264 (Blu-Ray) Nvidia engine:

Number of frames:                 10000
Length (hhh:mm:ss.ms):    000:06:57.083
Frame width:                       1920
Frame height:                      1080
Framerate:                       23.976 (24000/1001)
Colorspace:                        YV12

Frames processed:                10000 (0 - 9999)
FPS (min | max | average):       50.31 | 172.54 | 89.84
CPU usage (average):             3%
Thread count:                    4
Physical Memory usage (peak):    475 MB
Virtual Memory usage (peak):     556 MB
Time (elapsed):                  000:01:51.307

[Guest]

Wow!

Don't bother posting VC1 as HW acceleration is not yet implemented for Intel.

[Stereodude]

I have a troublesome 1080i MPEG-2 TS sample for you.

AVS script:

LoadPlugin("C:\HDTV Tools\DGDecIM\DGDecodeIM.dll")

DGSource("test_clip2.dgi",engine=0)

My findings:

• The Intel engine will lock VD up when trying to step backward through the clip. To see this open the script in VD, jump to frame 125 (CTRL+G), and then press and hold down the left arrow on the keyboard.
• The Nvidia engine doesn't lock up, but won't decode the clip correctly when stepping backward either (DGDecNV does the same thing also). To see this change the engine in the script, open the script in VD, jump to frame 125 (CTRL+G), and then step backward to frame 103 (or earlier).
• DGMPGDec has no problem stepping backward with the clip.
• The Intel engine and Nvidia engine don't return the same video. You can see this with the following script. Play/step to frame 24&25 and you'll see they're offset 1 frame from each other, yet both return 242 frames. FWIW, the Nvidia engine matches DGMPGDec with this test.

  LoadPlugin("C:\HDTV Tools\DGDecIM\DGDecodeIM.dll")
  
  source1=DGSource("test_clip2.dgi",engine=0)
  source2=DGSource("test_clip2.dgi",engine=1)
  
  stackvertical(source1,source2)

[Sharc]

ok, I have prepared the .dgi with DGIndexNV.
How do I use DGDecodeIM now? DGsource(.....dgi, engine=1)? Can you perhaps post an example for a basic script?

Edit: I just got it....

[Stereodude]

Originally Posted by neuron2

Wow!

I suppose I should mention I'm using a i7-4770k w/ HD 4600 integrated graphics and a GeForce GT440 (VP5).

[Sharc]

Here a problematic VC-1 interlaced source.
SW decoding (engine=1); NVIDIA GForce 9600GS.

[Reel.Deel]

Hi neuron2,

Tried various Blu-ray/DVD sources and they all work with the Nvidia engine, but with the Intel engine VirtualDub instantly just hangs and no error message.
In AVSMeter I get the following message:

AVSMeter 1.7.6 (AVS 2.6, x86) by Groucho2004
Analysing script, please wait...
Return on error: error code -3, pipeline_decode.cpp 316

Don't know if this is any help but my CPU is a i7-4930k (no integrated graphics).

[Guest]

Originally Posted by Sharc

ok, I have prepared the .dgi with DGIndexNV.
How do I use DGDecodeIM now? DGsource(.....dgi, engine=1)? Can you perhaps post an example for a basic script?

Edit: I just got it....

Just for others then:

loadplugin("dgdecodeim.dll")
dgsource("file.dgi") #uses Intel
dgsource("file.dgi", engine=0) # uses Intel
dgsource("file.dgi", engine=1) #uses CUVID

[Guest]

Originally Posted by Sharc

Here a problematic VC-1 interlaced source.
SW decoding (engine=1); NVIDIA GForce 9600GS.

In what way is it problematic? Does it work in plain DGDecNV?

[Guest]

Originally Posted by Reel.Deel

Hi neuron2,

Tried various Blu-ray/DVD sources and they all work with the Nvidia engine, but with the Intel engine VirtualDub instantly just hangs and no error message.
In AVSMeter I get the following message:

Don't know if this is any help but my CPU is a i7-4930k (no integrated graphics).

I'll release a new beta this evening with some more tracing to see what is happening. I simply do not know if the Intel Media SDK will work in SW mode without integrated graphics. Further tracing will reveal the answer.

[Stereodude]

Originally Posted by Reel.Deel

Don't know if this is any help but my CPU is a i7-4930k (no integrated graphics).

If you don't have an iGPU do you have the QuickSync engine?

[Sharc]

Originally Posted by neuron2

In what way is it problematic? Does it work in plain DGDecNV?

It produces jerky playback and blended frames.
It works in plain DGDecNV, however there are few duplicated frames (hard telecined?) which cause some jerkyness, but no blended frames in DGDecNV.

[Guest]

Originally Posted by Stereodude

If you don't have an iGPU do you have the QuickSync engine?

Intel supports SW decoding also, so intuitively you wouldn't need an iGPU for that. Unless...even the SW support requires the appropriate Intel driver, which would not be installed on a system without an iGPU. We can do some tracing to see what is really true.