DG NV tools

[crusher497]

neuron2,

Any plans to be able to index h264 FLV files?

[linyx]

@linyx
Need your stream and your diff results, please.
As for timing, I got:
DGIndexNV 2018
15:17
12:22

then with DGIndexNV 2018 Perftest
14:15
11:50

on the same files (Windows 7 x64 here).

With the corrupted video, it only appears in the DGIndexNV window, the actual decoded video appears to be fine (albeit, it cannot demux it properly from a few tests). It appears to only happen with MKVs, too.

Here are three clips which each produce the issue.
http://www.mediafire.com/download.php?n403xuvkaqy
BTW-Thanks for the work on speeding up indexing!

[RedDwarf1]

I use Win XP Pro x86 and would help test but I don't have a Blu Ray drive so I could only do testing from either DVD or hard drive.

I only have BBC and ITV HD broadcasts which are 1440x1088i@around 9Mbits. So unless there is a source to test with, I cannot really help.

I do have a GT 240 1GB which gives around 100fps, typically 95 to 105fps, while previewing BBC HD TS's.

Core 2 Q9450@2.66GHz
Win XP x86
4GB DDR2
5 SATA hard drives

[Guest]

You can test on anything as long as it takes a reasonable amount of time to index. If the stream indexes in 1 second, it's hard to discern any change. :)

Load the same file 10 times, whatever it takes to make indexing take minutes.

[Guest]

With the corrupted video, it only appears in the DGIndexNV window, the actual decoded video appears to be fine (albeit, it cannot demux it properly from a few tests). It appears to only happen with MKVs, too. Very strange MKVs. Please tell me what the source material was and how you generated these MKVs, in detail. Thank you.

EDIT: It appears that the stream has multiple different PPSs but only one is stored in the MKV's CodecPrivate. Here's what would help me to figure this out. Post a link to the source material used to make the MKV and explain exactly how you create the MKV from the that source material.

[laserfan]

My results on my i7-980X Win7 64bit with VelociRaptor (same project as my WinXP results).

2018 - 3:16
2018 perf - 3:12

This supports my WinXP versus Vista/Win7 theory. I'm tending towards going with the perf version due to the big gains it gives XP users. I don't know why some of you see drops on Vista/Win7; could it be the RAID 0?I tried DGIndexNVperf for grins, following 2016 and then 2018 (32-bit versions, but I'm a W7 x64 quadcore PC). All three were around 6 1/2 minutes, with the "performance" version actually a few seconds slower. FWIW.

BTW because you always write-out the log as "inputfilename.log" each run will overwrite the previous. Seems to me you should name the .log file with the Save as Project/Output file e.g. then I'd have 3 easy logs: videoIN2016.log, videoIN2018.log, videoINperf.log to go along w/my 3 .dgis?

[Guest]

I tried DGIndexNVperf for grins, following 2016 and then 2018 (32-bit versions, but I'm a W7 x64 quadcore PC). All three were around 6 1/2 minutes, with the "performance" version actually a few seconds slower. FWIW. Further support for the WinXP vs. Vista/Win7 theory. Thank you.

[RedDwarf1]

I've just had an error with 2018. A TS was open which I attempted to move to another folder to use for testing. Unlocker opened so I unlocked it and moved the file. I then closed the file in DGIndexNV but when I attempted to open the file in the new folder I got the message GPU decoder: failed to create video decoder[1]. Closing DGIndex and re-opening and it worked fine.

This is what happened with an invalid license file after I moved from 2013 first version to 2018. Not really a big issue, more about error handling.

Indexing tests done with a 1 hour 4 minute BBC HD Transport stream

Indexed to the same folder:
2018 took 3 mins 21 secs
The test version indexing the same file took 2 mins 15 secs

Indexed to a different Drive:

2018=3 mins 20 secs
Test version=2 mins 51 secs

Strangely, the 2018 version took the same amount of time whereas the test version took quite a bit longer than when indexing to the same drive.

I've moved to driver 197.45 after getting a BSOD specifying a nVidia driver file with the latest driver.

You could provide an option so that people could use whichever provides the best performance for them rather than optimizing it for one version of windows or one type of hardware.

[Guest]

You could provide an option so that people could use whichever provides the best performance for them rather than optimizing it for one version of windows or one type of hardware. That's easy to do for buffer size but it's a nightmare to select low-level IO versus C runtime IO.

[woah!]

Looks like it might be a Win7 versus WinXP issue.

@woah!

Please give your system details.

amd thuban x6 1055T
winxp sp3 32bit
mech hard drives for files encoding, except the OS drive is ssd.

[linyx]

Very strange MKVs. Please tell me what the source material was and how you generated these MKVs, in detail. Thank you.

EDIT: It appears that the stream has multiple different PPSs but only one is stored in the MKV's CodecPrivate. Here's what would help me to figure this out. Post a link to the source material used to make the MKV and explain exactly how you create the MKV from the that source material.
The source material was the respective Blu-Ray disc for each title. I used eac3to like this for each:
eac3to.exe "Source" 1) 2: B:\Paycheck\Video.mkv
I assume it is Haali's Matroska Muxer (2009-11-14) that was used by eac3to, then those I remuxed (to cut down on the size) with MKVMerge GUI (4.0.0.0), and finally split them to 50 meg pieces (uploaded the first piece) with DGSplit.

What do you need from the source material? Seeing as it is a full Blu-ray, uploading the whole thing could take a few months and would probably be considered... less than legal.:eek:

[Guest]

Never mind. I used mkvextract to get the ES. I think I know what's wrong. Stay tuned.

[hydra3333]

I don't understand ... just copied 2018 over and set the license to #2 (it could have been the wrong one), and used commandline
"C:\software\DGindex\DGIndexNV.exe" -i "G:\HDTV\Q1.mpg" -a -o "G:\HDTV\Q1.dgi" -e a few times. From a non-priv account (the .ini is in a read-only folder). And it popped up with could not create decoder. Logged into a priv account, set the license to #3 and it didn't throw that error. Logged back into user account and ran the command line and it just sits there not opening the file. Same interactively.

Now doesn't open any pal HD .mpg from either account. Can swap amongst the account and licenses but nothing opens at all. Rebooted same issue stays. What should I do ?

edit: XP-sp3 32bit latest nvidia driver 8600GT q6600
edit2: no, I think license 2 was the right one

[Guest]

I don't know anything about "priv" and "non-priv" accounts. You have to be able to read the INI file and it must be set to the right license before opening your file.

If it pops up "could not create decoder" then you are pointing to an old obsolete license. You need to use the regenerated one.

[Guest]

Regarding linyx's issue...

There are two things going on.

1. My support for MKV works by reading an entire timecode's worth of data into the buffer. That may include an SPS, PPS, SEIs and the slices of the frame at that timecode. If it is an IDR, it can be quite large. So when I reduced the buffer it became insufficient to hold the MKV data for one timecode. I can fix that by setting a large buffer when an MKV file is loaded. That explains the bad demux and general crappiness.

2. The earlier issue I fixed by revising the strategy for PPS injection needs to be carried over to MKV. But that is broken even in vanilla 2018, as you can see by loading Paycheck.mkv and GOP stepping forward.

I can fix 1 easily. For 2 I am putting on my thinking cap. In the meanwhile, do not use 2018 perf test version for MKV.

[hydra3333]

Had just installed a brand-new fresh squeaky clean XP-sp3 with latest drivers, q9450, 8800GT, fully patched, dual-nic gigabyte motherboard on a different machine (same motherboard type).

Checked the right license file and license content (correctly pasted) into license.txt in the same folder as the .exe.

Ran it from an admin account, so no likely issues there. Tried to open a .mpg, it threw the error
GPU decoder: failed to create video decoder [1]

Still at a loss.


edit: ugh, just went back to the license generator and it gave me different long strings than last time. Works now :) Must have missed reading some posts on re-doing the licenses.


EDIT2: What ?? ? I just did 4 of these by dragging and dropping the .mpgs onto a .bat file which had that command line and they all worked. Then I tried a 5th and no it only sits there with a blank black area. So I rebooted and re-tried one of those that did work and now same result again no matter what. Help !!!

EDIT3: Oh. One of my NICs is connected to another box which I use for other things, and when that box is down the NIC loses connectivity and DGindexNV doesn't work. When it's up then the NIC connection is restored and DG does work ... Is it possible for DG not to depend on the boot state of another machine ?

[MrVideo]

AMD Quad core 3.2 GHz
SATA HDD (separate from C:)
GT240
Latest NVidia driver

Don't know about the cache issue, but for me there was definitely a difference.

The source is a MPEG-2 elementary stream (don't do audio with my jobs, done separately), 11.6 GB.

2018 - 2:41
2018p - 1:43

I think the big reason for the time difference is the lack of waiting for the buffer. The hesitation is gone. There is now a consistent reading of the source file.

Based on test results from Win7 users, it is another reason I'm not leaving XP. :D

[tormento]

The test I promised you.

Windows 7 x64 - 3 x Raptor 300GB Raid 0 ~ 300 MB/s average read :devil:

Rocky Balboa BD :p

DGIndex x86 standard - dgi only: 06m36s
DGIndex x64 standard - dgi only: 05m15s :eek:
DGIndex x86 optimized - dgi only: 04m27s

DGIndex x86 standard - dgi with audio: 10m32s
DGIndex x64 standard - dgi with audio: 10m30s
DGIndex x86 optimized - dgi with audio: 06m00s

DGIndex x86 standard - dgi with audio + video: 12m30s
DGIndex x64 standard - dgi with audio + video: 14m05s :confused:
DGIndex x86 optimized - dgi with audio + video: 10m57s

P.S: When reading mpls it could be nice to have language instead of PID XXXX in demuxed audio. ;)
P.P.S: Uh and an audio *beep* when finished should be nice too.

[Guest]

EDIT3: Oh. One of my NICs is connected to another box which I use for other things, and when that box is down the NIC loses connectivity and DGindexNV doesn't work. When it's up then the NIC connection is restored and DG does work ... Is it possible for DG not to depend on the boot state of another machine ? No guarantees, but I'll look into the feasibility of that after I get the performance release out. There are technical issues with having multiple keys in the license file with my current architecture.

[tormento]

There are technical issues with having multiple keys in the license file with my current architecture.
It's a pity.

Moreover: I use x64 and x86, could you please make x64 version look for license file in parent directory too?

[Guest]

It's a pity. I just got done saying I'd look into it.

Moreover: I use x64 and x86, could you please make x64 version look for license file in parent directory too? No, it's very easy to copy a file.

[Guest]

and finally split them to 50 meg pieces (uploaded the first piece) with DGSplit. Ouch, I just noticed this. You cannot cut MKVs with DGSplit!

So I cannot use those MKVs. Please wait for my next test version and do not test on MKVs splitted with DGSplit.

[Guest]

OK, I have refreshed the perftest version with the fixes for MKV. Please let me know if MKVs are not working properly in any way.

http://neuron2.net/misc/DGIndexNV_perftest.zip

[um3k]

Hi Neuron2, I just wanted to make sure you are aware of my previous post (http://forum.doom9.org/showthread.php?p=1413855#post1413855), as you seem to have overlooked it. Thank you.

[Guest]

Hi Neuron2, I just wanted to make sure you are aware of my previous post (http://forum.doom9.org/showthread.php?p=1413855#post1413855), as you seem to have overlooked it. Thank you. Investigating.

[linyx]

Ouch, I just noticed this. You cannot cut MKVs with DGSplit!
Oops, sorry.:stupid:

OK, I have refreshed the perftest version with the fixes for MKV. Please let me know if MKVs are not working properly in any way.

http://neuron2.net/misc/DGIndexNV_perftest.zip
Very nice! Everything I have tested works beautifully!
Thank you very much for the quick fix:thanks:

[Guest]

Hi Neuron2, I just wanted to make sure you are aware of my previous post (http://forum.doom9.org/showthread.php?p=1413855#post1413855), as you seem to have overlooked it. Thank you. I will add a new INI file value to not pop up the dialog and assume the response Cancel.

@linyx

Thanks for your test results on MKV streams.

[Guest]

Hi Neuron2, I just wanted to make sure you are aware of my previous post (http://forum.doom9.org/showthread.php?p=1413855#post1413855), as you seem to have overlooked it. Thank you. OK, I have updated the perf test version to add this. Re-download that version. Then start DGIndexNV and close it right away (to remake the INI file). Then open the INI file and change the last line to be 1 instead of 0. Then you should not see those popups anymore and the misdetected streams will be ignored.

[um3k]

Thank you neuron2, that works like a charm.

[LigH]

Late reply, and possibly not really necessary... - I had similar issues with Delphi, looking for a system to access bigger files fast I found mmio and Win32API functions (specifically the GpHugeFile unit). It was an impressive difference too. Especially the "buffered" vs. "unbuffered" Win32API file access made much difference under NT Windows versions.

FileBench.exe (http://www.ligh.de/software/FileBench.exe) (220 KB) -- not even close to a "real benchmark"; warning, you may not be able to cancel it even though I tried to ensure it, might require a threaded design instead of just a few Application.ProcessMessages injections.

[mikeathome]

I am having issues with TSMuxer .ts containers (Error message: not a Transport Container). Had to switch back to an older version.

mike

[Guest]

I am having issues with TSMuxer .ts containers (Error message: not a Transport Container). Had to switch back to an older version.
You need to regenerate your license as explained numerous times. If that doesn't help, then post a sample stream that I can use to duplicate the issue. I've just tested a TS file out of TSMuxer and it worked fine so I think a license refresh will fix things up for you.

[Guest]

* Made license switching a bit more convenient. Previously if you opened
a stream with the wrong license selected, then set the correct license, and then
did File/Open followed by OK with the intent of reopening the file, the file would
not be reopened because no change was made to the file list. Now any change in
the selected license will be treated as if the file list was changed, and the
input files will be reopened.

* The initial window position for DGIndexNV stored in the INI file was not being
honored. Fixed.

* Added an auxiliary INI file option to avoid a popup when there is a mismatch
between the PAT/PMT audio type and the actual detected audio type. Delete your
old INI file to allow DGIndexNV to create a new one with the option.

* Improved the heuristics for SPS/PPS injection for MKV streams.

* Performance improvements for indexing on many systems. The improvement is
substantial on most WinXP systems. The large pauses during playback in DGIndexNV
also now do not occur.

* Fixed the remaining time display in the Info dialog so that it properly
honors the configured project range.

* Added a timestamps dump function under the Options menu. This will dump the
STC/PCR, PTS, and DTS timestamps for the video and audio streams in a transport
or program stream.

* Fixed a bug that manifested as follows for some MKV files: Open the MKV file.
Hit preview/play. Hit stop. Hit preview/play again. DGIndexNV hangs.

http://neuron2.net/dgdecnv/dgdecnv.html

[Guest]

I just slipstreamed a fix into 2019 for the timestamps dump (worked only for MPG, now works for all). So re-download it if you previously downloaded it.

[kebulek]

I just downloaded v2019 (second version i guess - after neuron's #1934 post) and indexed h264 hdtv stream. But when i load the avs to the mpc/vd/megui etc... there are few green frames at the end. :(

Simple AVS script:
LoadPlugin("C:\Program Files\Avisynth\plugins\DGDecodeNV.dll")
DGSource("video.dgi", deinterlace=1).Spline36Resize(1280,720)

(DGDecodeNV.dll in avs directory is replaced with new one, of course)

.TS SAMPLE (http://www.mediafire.com/file/fomnzzyykmw/sample.ts) (39MB) - it's the end of the original .ts cutted with TSPE.

My system: WinXP SP3, GF 9600GT with 191.07 drivers

[Guest]

I cannot duplicate it with the sample you uploaded. Have you tried with that sample?

[LaTo]

* Made license switching a bit more convenient. Previously if you opened
a stream with the wrong license selected, then set the correct license, and then
did File/Open followed by OK with the intent of reopening the file, the file would
not be reopened because no change was made to the file list. Now any change in
the selected license will be treated as if the file list was changed, and the
input files will be reopened.

* The initial window position for DGIndexNV stored in the INI file was not being
honored. Fixed.

* Added an auxiliary INI file option to avoid a popup when there is a mismatch
between the PAT/PMT audio type and the actual detected audio type. Delete your
old INI file to allow DGIndexNV to create a new one with the option.

* Improved the heuristics for SPS/PPS injection for MKV streams.

* Performance improvements for indexing on many systems. The improvement is
substantial on most WinXP systems. The large pauses during playback in DGIndexNV
also now do not occur.

* Fixed the remaining time display in the Info dialog so that it properly
honors the configured project range.

* Added a timestamps dump function under the Options menu. This will dump the
STC/PCR, PTS, and DTS timestamps for the video and audio streams in a transport
or program stream.

* Fixed a bug that manifested as follows for some MKV files: Open the MKV file.
Hit preview/play. Hit stop. Hit preview/play again. DGIndexNV hangs.

http://neuron2.net/dgdecnv/dgdecnv.html

Is there a trial version somewhere?
Sorry but I haven't found it... Thanks!

[Guest]

Is there a trial version somewhere? There is no trial version.

[kebulek]

I cannot duplicate it with the sample you uploaded. Have you tried with that sample?
Yes I've tried. :confused: I'll try it again tomorrow after restart.

[asarian]

Neuron, I seem to be having an issue (http://forum.doom9.org/showthread.php?p=1415035#post1415035) with FFVideoSource. Are your DG NV tools frame consistent, like FFVideoSource? (absolute requirement, as I do slow stuff, for which DirectShowSource gets out of sync).

Thanks.

[Guest]

Are your DG NV tools frame consistent, like FFVideoSource? Frame accuracy for random access is the raison d'etre of DGDecNV.

[asarian]

Frame accuracy for random access is the raison d'etre of DGDecNV.
Cool. But you seem to have edited out my other question (?). Well, I'd still like to know, Do your tools also support decoding 1080p VC1 streams on 'Feature A' cards? (like my GTX260).

[Audionut]

Well, if you did a little search you would find your answer. DgindexNV is limited by the CUDA capabilities of your card. So hit up the nvidia site and see what your card is capable of.

[Guest]

It will work with partial acceleration but it *will* work.

[rco133]

Hi.

I have tried the Output Trimmed TS function, but for some reason I can't get it to work.

If I only cut small sections it seems to work fine, but if I try to cut 50 minutes out from a 105 minute cut, then after a while the % number just starts flickering, and the filesize of the output file no longer increases.

The source TS file is a 15 GB big file.

I would like to buy dgindexnv, but wanted to make sure that this function worked first, because I want to cut out commercials. DGindexNV should work without a license shouldn't it?

The above happens with version 2012. With version 2019 I can't even open the TS file.

rco133

[Alf Bundy]

Hi neuron2,

In build 2019, while indexing, infos about bitrate (avg & max) are not shown anymore.
I tried with a .264 ES, a .MKV and a .mts from my camera.

It's not a life threatening situation, but I would like to see that info again ... Please.

Other than that, it's working very well. Thanks !

[Guest]

I would like to buy dgindexnv, but wanted to make sure that this function worked first, because I want to cut out commercials. DGindexNV should work without a license shouldn't it?

The above happens with version 2012. How did you run 2012 without a license?

[Guest]

It's not a life threatening situation, but I would like to see that info again ... Please.
Removed for performance reasons. I didn't actually measure it but thought it was not needed. If it does not impact performance adversely I will put it back.

[rco133]

How did you run 2012 without a license?

I just downloaded 2012 from this page and ran dgindex.

If dgindex is not supposed to work without a license, then I am not sure whats going on.

Does Output trimmed TS work OK on large files?

rco133

[kebulek]

I just downloaded v2019 (second version i guess - after neuron's #1934 post) and indexed h264 hdtv stream. But when i load the avs to the mpc/vd/megui etc... there are few green frames at the end. :(

Simple AVS script:
LoadPlugin("C:\Program Files\Avisynth\plugins\DGDecodeNV.dll")
DGSource("video.dgi", deinterlace=1).Spline36Resize(1280,720)

(DGDecodeNV.dll in avs directory is replaced with new one, of course)

.TS SAMPLE (http://www.mediafire.com/file/fomnzzyykmw/sample.ts) (39MB) - it's the end of the original .ts cutted with TSPE.

My system: WinXP SP3, GF 9600GT with 191.07 drivers

I've done some testing today and that bug occurs only with DGSource(...) and CUVIDserver. With DGMultiSource is everything OK.

Green frames show up after frame 744+.