DGDecIM: Frame-accurate Avisynth source filter for QuickSync

[CarlPig]

I personally voted on HW deinterlace, because i don't own a 3d device, so i do not need MVC.

[Sharc]

More than HW deinterlace, ivtc, and crop/resize?
Do you expect better deinterlacing quality compared to decomb, tdeint or similar? Something like a speedy QTGMC perhaps?
When I go for deinterlacing at all it is always for subsequent re-encoding which tends to mask the HW decoding speed benefit -- with the exception of QTGMC. Similar applies for HW crop & resize I think.
But mainly: my current HW does not support the new Intel functions, hence my selfish preference for MVC ;-)

[lansing]

Have you tried different decoders? I tried DGDecodeNV and LWLibavVideoSource. No problem.
I can't use DGDecIM, don't have the required hardware.

Just tried, LWLibavVideoSource and dgnv engine were both fine.

[Guest]

Have you tried different decoders? I tried DGDecodeNV and LWLibavVideoSource. No problem.
I can't use DGDecIM, don't have the required hardware. Hi Groucho2004. You are right, it's an issue with my filter not AVSMeter. I'm working on it.

[Guest]

Do you expect better deinterlacing quality compared to decomb, tdeint or similar? Something like a speedy QTGMC perhaps? Certainly better than decomb and tdeint! I suppose it will be comparable to the Nvidia deinterlacing we already have, but I haven't tested anything yet.

[tormento]

it fixes tormento's "bad" MKV stream handling.

Thanks.

Have you tried some MEGui testing with screen preview enabled and AVS improved opening?

[Sharc]

Certainly better than decomb and tdeint! ......
Interesting. Is there a principal advantage with HW deinterlacing over SW, other than speed? Are their algos more sophisticated, or something which can't be done similarly in SW (apart from speed of course)?

[Guest]

Yes, the HW speedup allows for more sophisticated algorithms. And of course one goal is to recover CPU for encoding, so just the fact that the deinterlace is removed from the CPU is an advantage.

[Guest]

Have you tried some MEGui testing with screen preview enabled and AVS improved opening? Yes, I have no issues with MEGUI. Do you have any issues with it?

[Guest]

Here is beta 13. For me it fixes the problem with AVSMeter hanging. lansing, please report your result.

There may be others issues, I am still working on it.

http://neuron2.net/misc/dgdecim_b13.zip

[tormento]

Yes, I have no issues with MEGUI. Do you have any issues with it?

Tried with MT enabled?

[Guest]

Tried with MT enabled? No. I don't consider MT important for a fast source filter and anybody that wants to try to multithread it is on their own. Didn't we go through this once already?

[lansing]

Here is beta 13. For me it fixes the problem with AVSMeter hanging. lansing, please report your result.

There may be others issues, I am still working on it.

http://neuron2.net/misc/dgdecim_b13.zip

still hangs in my machine

[MeteorRain]

Here is beta 12. It changes the source filter name to dgsourceim()

Have been afk for a few days. Update to b13 and failed to load old scripts. Finally find this.

Changing the name of a filter breaks all old (used, prepared, or as template) scripts. :scared:

No one remembers the avisynth way of distinguishing filters with same name?

DGDecodeIM_DGSource("clip.dgi") # DGIM will be used
DGDecodeNV_DGSource("clip.dgi") # DGNV will be used

BTW, I can confirm the hanging problem got fixed on b8. Good job. Thanks.

EDIT: I also occasionally noticed that the FPS fraction line inside dgi (by dgindexim) is not reduced. It writes 120000/5005 instead of 24000/1001.

In the dgi for the same file by dgindexnv, it writes 24000/1001.

Not a big deal though, just FYI.

[lansing]

Found a bug on filter ignoring file extension.

I have a test1.ts, and a test1.dgi that was pointing to it. And in my avs script I was calling test1.mkv, but the script still works.

[Guest]

Currently, if the DGI exists, it is used. I could enforce that the referenced file matches the file given in the DGI. Thanks for pointing it out.

[yup]

neuron2!

I try last beta13, the same issue.
See uploaded files
source mts
https://www.dropbox.com/s/2l8ttetg8uh73xb/00000.MTS
and converted h264
https://www.dropbox.com/s/o8vhciux96xmp47/test.h264

yup.

[kypec]

Do you folks out there want me to have separate filter names:

dgsourceim()
dgsourcenv()

This affects only people who want to be able run both of them.

I don't quite understand why are you trying to reinvent the wheel. AviSynth provides very elegant method for resolving function name ambiguities, please look at the bottom of AVS wiki (http://avisynth.nl/index.php/Plugins) page to see how it works:# using methodX from pluginA.dll
pluginA_methodX()
# using methodX from pluginB.dll
pluginB_methodX()

[MeteorRain]

I don't quite understand why are you trying to reinvent the wheel.Which I already said on #219;)

[Guest]

I prefer my method. It's less typing. And with the "standard" method users that don't know about it can still go wrong, because the prefix is optional.

[RedDwarf1]

The small amount of testing that I have done shows that the speed seems to be influenced by the iGPU core clock which tends to ramp up and down quite a lot while the tests are running. While at 1150Mhz the decode speed seems higher than when it's clocked at 350Mhz. It might be worth seeing if that can be fixed at full speed while the test is running. Has anyone else found the decode speed fluctuating depending upon the core speed ramping?

Is there a test sample that can be used to compare speeds? It would be nice to see any differences between the 4th gen and 3rd gen CPU's to compare how any optimizations affect different CPU's. I realise that this is only a beta and your trying to iron out decoding errors etc but it would be nice to have a glimpse of any speed differences between the CPU's. Mine is a 3770k.

I have limited access to 1080p except for satellite broadcasts and a few demo videos such as rat and pigeon impossible.

Pigeon impossible on Win 7 x64 with 3770k
AVSMeter 1.7.6 (AVS 2.6, x86) by Groucho2004
AviSynth 2.60, build:Mar 9 2013 [13:28:27]
Active MT Mode: 0

Number of frames: 8968
Length (hhh:mm:ss.ms): 000:05:58.720
Frame width: 1920
Frame height: 1080
Framerate: 25.000 (25000/1000)
Interlaced: No
Colorspace: YV12

Frames processed: 8968 (0 - 8967)
FPS (min | max | average): 1.732 | 557.7 | 166.5
CPU usage (average): 9%
Thread count: 19
Physical Memory usage (peak): 494 MB
Virtual Memory usage (peak): 738 MB
Time (elapsed): 000:00:53.869

[Guest]

Thanks, that's certainly an important line for us to follow. I won't be able to do much on it personally though until any remaining decoding issues are resolved, as you mentioned. But I welcome you and others getting a start on it and posting here. I'll be able to engage with you on it hopefully in not too long.

Right now, lansing's problem stream is being highly recalcitrant. I have some theories about it, however, and am pursuing it as the top priority. I'd also welcome yup to post his problem stream.

[yup]

I'd also welcome yup to post his problem stream.
Hi neuron2!
Please see post #222 this thread.
yup.

[MeteorRain]

A MKV file that causes DGIM to halt on opening.
DGNV works fine.
Extract h264 from MKV and index as dgi, works fine.
I'm on ver. b13.

https://copy.com/Ew9GR4nC6Yrs

[Guest]

I'm on ver. b15.
Did you mean b13? Thanks for the stream. Investigating...

@yup

Thank you!

[Guest]

A MKV file that causes DGIM to halt on opening. DGNV works fine. This is fixed in beta 14:

http://neuron2.net/misc/dgdecim_b14.zip

I wasn't injecting SPS/PPS from CodecPrivate when packaging data for the SDK DecodeHeader() call. Thank you for pointing it out.

[Guest]

Please see post #222 this thread.
yup. Thanks, yup. I ran your sample with beta 14 and didn't see any issues. I made the minimal script and opened it in VirtualDub and played it through with no errors. I even added reverse() to the script to force random access on every frame and that ran fine too. So, if you still have an issue, please provide your script and x264.exe command line using your sample. Also please tell me which version of x264.exe you use. Are you using the Intel HW or SW decoding?

I have tested a lot of MTS files and don't find any issues.

[Guest]

I also occasionally noticed that the FPS fraction line inside dgi (by dgindexim) is not reduced. It writes 120000/5005 instead of 24000/1001.

In the dgi for the same file by dgindexnv, it writes 24000/1001.

Not a big deal though, just FYI. Interesting. I don't do any reduction; rather, the values received from the decoder are used, in this case, the Intel SDK DecodeHeader() call. We could look to see what the ES actually specifies, but it seems unimportant in practice.

[Guest]

lansing's bad stream misbehaves with the Intel decode sample also, so I have posted about it at the Intel forum:

http://software.intel.com/en-us/forums/topic/506806

[yup]

Hi neuron2!

Now i am out my work horse (short vocation).
I am using simple script
LoadPlugin("DGIndexIM.dll")
DGSourceIM("t.dgi")
I am not set engine, my PC have Intel Graphic.
You advice manual set engine?
When I back I continue testing.
yup.

[Guest]

@yup

Thanks yup, but the stream you gave me is not called t.mts, so it seems you have not given me a stream that I can use to duplicate your issue. I will ask again for what I need. You need to give me these if you want support.

1. A stream that I can use to duplicate your issue. Don't send me some random stream, send me the one you have personally experienced issues with.
2. The corresponding Avisynth script.
3. A description of how to make the problem happen. If you are transcoding then I need the encode command line and encoder version number.

Thank you for your understanding. I can fix things only when I can duplicate them.

You do not need to set the engine manually.

[CarlPig]

Do you have a status for supporting HW deinterlace??

[Guest]

I report all my activities in real time, and have made my priorities clear. Pinging me about it is a NOP and risks annoying me. ;)

[Guest]

Here is beta 15. It seems to fix MPEG2 streams, including lansing's stream that hangs AVSMeter. (Nothing at all wrong with the great AVSMeter!)

http://neuron2.net/misc/dgdecim_b15.zip

I found two problems:

1. The Intel decode sample has a bug that causes it to exit the main decode loop too soon in some cases. DGDecIM inherited the bug.

2. The Intel MPEG2 decoder discards leading B frames in display order! The AVC and VC1 decoders just deliver (corrupted) frames as I would expect. My existing random access relies on the fact that pushing a complete frame to the decoder always results in a delivered frame, and CUVID does so consistently for all the video types. Certainly it is perverse to have MPEG2 handled differently from AVC/VC1 by a decoder in this regard. I was able to make a workaround for this wart. Please advise if you find any issues with MPEG2 streams.

Now if we can just get Intel to fix HW decode for VC1...

[Guest]

Please re-download beta 15. I slipstreamed an additional fix needed as a result of the workaround.

[lansing]

confirm beta 15 fixed the mpeg2 hang problem, thanks for the work.

[Guest]

Thanks, lansing.

I have slipstreamed some more fixes into beta 15. Please redownload beta 15 to get these fixes. The leading B frame handling was broken for MPEG2 and there was a regression for some VC1 and AVC streams, so it's important to get this updated beta 15.

This version also fixes an issue raised earlier by lansing. When a mismatch is seen between the filename in the script and the filename in the existing DGI, the user is offered the choice of ignoring the mismatch and using the existing DGI file or regenerating the DGI using the filename in the script.

http://neuron2.net/misc/dgdecim_b15.zip

[mastrboy]

I finally have a weekend coming without any work, so was considering testing out DGDecIM on some sources, but do I have to have a monitor connected to the intel GPU, or is enabling it in bios enough?
(I currently have it disabled in bios as I use a nvidia 660 card instead)

[Guest]

It depends on which OS you run: Win7 versus Win8.

I have Win8 and all I had to do was enable the iGPU in the BIOS and install the Intel driver. My Nvidia card remains primary and there is no monitor or dummy connected to the iGPU spigot.

More info:

http://software.intel.com/en-us/forums/topic/311872

On Win7 you have 3 options it seems:

1. Connect a spare monitor to the iGPU.
2. Connect a VGA-dummy to the iGPU.
3. Use Eric's workaround given in the link above.

[aegisofrime]

It depends on which OS you run: Win7 versus Win8.

I have Win8 and all I had to do was enable the iGPU in the BIOS and install the Intel driver. My Nvidia card remains primary.

More info:

http://software.intel.com/en-us/forums/topic/311872

On Win7 you have 3 options it seems:

1. Connect a spare monitor to the iGPU.
2. Connect a VGA-dummy to the iGPU.
3. Use Eric's workaround given in the link above.

I'm on Windows 7 and I didn't have to do anything, my monitor remains connected to my AMD Radeon 7850. My situation is exactly like yours. FWIW I'm using a MSI Z87-GD45 Gaming and my CPU is a i7-4770. Not sure if my particular hardware supports not needing the iGPU connected.

[Guest]

That's great news, thanks for reporting it. Maybe things have changed since Eric's older post. Let's see what happens for mastrboy.

[lansing]

I finally have a weekend coming without any work, so was considering testing out DGDecIM on some sources, but do I have to have a monitor connected to the intel GPU, or is enabling it in bios enough?
(I currently have it disabled in bios as use a nvidia 660 card instead)

If you're using Window 8, then there's nothing to worry about, as you can simply enable iGPU and discrete GPU in the BIOS and it'll work.


If you're using Windows 7 desktop, watch the link in here about creating a fake display:
http://forum.doom9.org/showthread.php?p=1652628#post1652628

It doesn't matter which port you connect your display to. I connected my monitor through my video card and it still works. Also remember to position your display appearance like vivan said in the post.


If you're using Windows 7 laptop with a Nvidia card built in, you can probably able to set it in Nvidia's control panel to use iGPU for a particular program (I forgot the general name for those laptop model). It's so stupid for Nvidia to enabled this in laptop but not in desktop.

[Guest]

It's so stupid for Nvidia to enabled this in laptop but not in desktop. Can I ask you to please refrain from insults like this, at least in this thread? Thank you. If there is one thing that Nvidia is not, it is not stupid.

[lansing]

This version also fixes an issue raised earlier by lansing. When a mismatch is seen between the filename in the script and the filename in the existing DGI, the user is offered the choice of ignoring the mismatch and using the existing DGI file or regenerating the DGI using the filename in the script.


This doesn't make much sense to me. The filename of the script could had no relationship to the filename of the dgi. For example, if I loaded two dgi in a script, test1.dgi and test2.dgi, and appended them together, it's highly unlikely that I'm going to name my script test1.avs.

A reasonable way to do this is to generate the dgi with the video file extension added to the dgi filename from the start, like test.mkv.dgi, test.avi.dgi instead of test.dgi causing the confusion, then users don't have to make any choice.

[Guest]

It's worth considering, maybe later. Thanks.

[mastrboy]

I'm running Win 8.0 on the machine that was in question. So thanks for all the info, I will do some test tomorrow when various sources since it doesn't seem I need to have a monitor connected.
(I was already looking into making a fake plug, but couldn't find any resistors with 68-75 ohm resistors laying around to plug into a dvi-vga adapter... (ref: http://www.overclock.net/t/384733/the-30-second-dummy-plug)

[asteri]

I'm running Win 8.0 on the machine that was in question. So thanks for all the info, I will do some test tomorrow when various sources since it doesn't seem I need to have a monitor connected.
(I was already looking into making a fake plug, but couldn't find any resistors with 68-75 ohm resistors laying around to plug into a dvi-vga adapter... (ref: http://www.overclock.net/t/384733/the-30-second-dummy-plug)

Thanks it looks pretty easy and usefull! Can you please write exact pins which are connected? I am not sure, the picture is not accurate.

Thanks

[Guest]

Did you check this link?

http://http://soerennielsen.dk/mod/VGAdummy/index_en.php

[Guest]

I did some experiments with the QuickSync deinterlacer (using the VPP sample) in both single and double rate. It works fine and doesn't look half bad, as the English would say. Now on to integrating it into DGDecodeIM...

Also, please do let me know if you have any decoding issues. Things have gone quiet on that front so either things are looking good or no one is using it. ;) After I get the deinterlacing working I'd like to make a first formal release. Then I would add cropping/resizing.

[kalehrl]

Would you be so kind as to have a look at a sample posted here:
http://forum.doom9.org/showthread.php?p=1672767#post1672767
The sample was a h264 SD .ts capture turned into a .mkv with MeGUI's HD streams extractor.
I can also post untouched .ts sample if you need it.
Thank you