Release of DGDecode 1.1.0+ by NaN

[NaN]

This version is only compatible with DGDecode/DGMPGDec 1.1.0! Not with current versions of neuron2's DGMPGDec package! Use my version with DVD-rebuilder!

DGDecode 1.1.0+ is an overall enhanced version that is fully compatible with the original release. Intended for the use with DVD-Rebuilder, it offers small additions in both terms, features and performance.

The original code was already highly optimized, however this version needs approx. 1/3 less cpu-time and accelerates encoding times - tested on a P4 with CCE - by roughly 10%.

New (main) features:
10% more encoding speed without quality loss
broken "info" and "upConv" parameters are now fixed
backported the better slice decoding from DGDecode 1.4.3 (better resilience to video errors)
added exception handling (was introduced in DGDecode 1.3.0)

I'm using my dev versions for months now without any problems, a comparison showed identical output to the original build.

A big "thank you" to all opensource developers! It's a great way to learn a lot, even if the code is a mess sometimes.

Maybe somebody is interested in using it, I thought I would share it. So if somebody experiences bugs, odds, whatever, please drop me a note. Unfortunatly I'm not able to dedicate that much time as Don, but I will try and see what I can do!

Enjoy! NaN

PS: If somebody wants to put them on a homepage (maybe together with the release or just the release - whatever), that would be great, since I don't have one.

Edit: source attached.
Edit: added readme to release
Edit: SSE build attached.
Edit: SSE build removed, lookin into it
Edit: new SSE build up. Thanks to the testers!

I'm not sure if it makes sense to release the SSE build, since (only?) P4s show a benefit. For sake of completeness (and for the brave ;-)) I attach it anyway.

If in doubt, choose the SSE1 build!
It uses all optimizations by itself - due to different scheduling and threading it's maybe a tiny amount slower.

[Guest]

While I appreciate your work, I have two problems with this. First, you are going to bring back the versioning nightmare that we had with DVD2AVI. Please rename your version to avoid this, e.g., NaNDecode.dll or something like that. Second, you have to release source code to comply with the GPL. I will delay approving your attachment to hear your response.

[NaN]

Seems we're in trouble.

I respect the GPL, but I don't have a homepage and I don't want one. If somebody wants to host the sources, I will gladly accept that offer. If attaching the sources to the post is ok, I will do that (but I think doom9 won't be very happy about that). If somebody points out another simple possibility I'll do that.

To the other point: my build is a drop-in replacement for your original build. The dvd-rebuilder only accepts a "DGDecode.dll", no other names allowed - so the very first step of every user is renaming my build to that name.

There are at least 3 possibilities to identify my build: size, date and of course the info parameter that displays the version.

All my changes are under the hood and don't change the api or file format in any way.

Cheers, NaN

[Guest]

Not following GPL is not an option. Maybe jdobbs would be willing to host it for you. If you attach a zip with source, I'll approve it. I doubt that doom9 would have any objection to that.

Regarding the name, I see the problem and I suppose it's just something we'll have to live with.

[alfixdvd]

Hi NaN

Can you post your files at forum.digital-digest.com ?

[Guest]

Attachments approved. Thank you for your contribution, NaN!

Anybody that plans to host this, please observe GPL and also host the source code. Thank you.

[Pasqui]

Hi NaN,
a SSE version would be greatly appreciated :D

[Tima]

Donald, will you merge these optimizations into DGMPGDec? :)

[Guest]

Donald, will you merge these optimizations into DGMPGDec? I'll have a look at it as time permits.

[Prodater64]

@NaN: While your dll must to have dgdecode.dll name, you could change your zip file name to NaNDecode.zip or something like this to avoid confusions as Donald said you previously. Also a txt file inside the zip would be appreciated.
Thanks.

[NaN]

@Prodater64: thanks for your suggestion! A readme is added.

@Pasqui: sorry that I cannot fulfill your wish immediatly! But I need to check 1 or 2 things to be sure that the SSE build is ok!

@alfixdvd: unfortunatly I don't have a registration there - feel free to post the files over there, if you wish. Please add in that case that comments shall be posted here at doom9.

Cheers, NaN

[Amnon82]

@NaN: I've troubles to use your dgdecode.dll-build. I'm using the newest DGDecode 1.4.5 which produces 'DGIndexProjectFile11'. I always got an error that the d2v can't be opened. With donalds build it works ... I'm using an AMD 64 3.500+.

[Guest]

DGIndex 1.4.5 should have D2V file version 12. It was erroneously released as 11 and was corrected in 1.4.6b1.

[NaN]

Maybe I wasn't clear enough - my build is file format compatible only with DGMPGDec 1.1.0! Not with current versions! You have to use DGIndex 1.1.0, since you can only use matching pairs together!

Sorry for causing troubles! NaN

[FredThompson]

I'll be more than happy to provide a "home" for derivative works and other "homeless" filters. There's a major update coming to the A/V links list on neuron2's site (really, I promise...Beuler? Beuler?) to which I plan to put links to things like an archive fo the long-gone Sub Stream Alpha site and others which people request on a fairly regular basis.

[FredThompson]

Please post a non-SSE2 version. Paupers like me would appreciate this.

[Malphas]

Couldn't you just host it at SourceForge?

[NaN]

@Malphas: yeah sure - however I don't plan to release a new build every week (actually I'm not perfectly sure that there will be another release - besides bug fixing versions of course), so attaching to the release post is sufficient for me.

@FredThompson: thanks a lot for your offer - and I would gladly accept that, if you really want to put the files on your server.

SSE build needs some days (I've to look at 1 or 2 things).

Have a nice evening, NaN

[FredThompson]

I'll host it for you. You might also want to post it to neuron2's discussion board. There are some other derivatives of his filters there.

[Amnon82]

@neuron2: thx for the info.
@NaN: thx to u 2

[FredThompson]

Oh, that was quick. Thanks for the SSE version.

[Nic]

Nice work :)

Could someone, who is so inclined, benchmark speed difference when decoding between the current version of DGDecode and NaN's version? I think that would be useful to see if someone's time and effort would be worth porting the changes over quickly (as not to disturb DG's development).

@NaN: How complete is changelog.txt? Do you think that covers most of your changes? If so it's a good easy reference to help in porting the changes, in case you don't want to do it yourself.

"changed all dec eax to sub eax, 1 in mcsse.asm (inc/dec is depreciated nowadays)" - I did not know that. Even had to look it up to check.
:)

-Nic

[Cyberia]

I am conducting a speed test right now. I'll have results in the morning.

[Rippraff]

Hi NaN,

thank you very much for the SSE version! :) :thanks:
I will give it a try tonight with my XP-M and announced both versions at german doom9 forum.

Edit: Sorry to say that SSE version didn't work in my case. Tried an OPV test with a PAL movie and RB 1.01, CCE SP 2.70.2.1, Avisynth 2.55, it resulted in a green screen movie. :(

Cu Rippraff

[int 21h]

Not following GPL is not an option. Maybe jdobbs would be willing to host it for you. If you attach a zip with source, I'll approve it. I doubt that doom9 would have any objection to that.

Regarding the name, I see the problem and I suppose it's just something we'll have to live with.

He only has to offer a copy of the sourcecode through some sort of a 'medium customarily used for software interchange' to comply, or am I missing something here?

[Guest]

All I said was that he needed to follow GPL. I suggested one way to do that. Are you reading more into it than what I said, or are you just being ornery?

[int 21h]

All I said was that he needed to follow GPL. I suggested one way to do that. Are you reading more into it than what I said, or are you just being ornery?

I think I lost some of the reply's context by not following the thread before his edit and what not, sorry. Not ornery, just curious.

Cleaned out my OP to reflect this.

[NaN]

Please post your experiences with the SSE build!

I got a crash report and I'm not sure that this user used the correct version (using the SSE2 version on a SSE machine causes the (compiler inserted) cpu-check to kick in and causes a shut down of the DGDecode.dll - a green image is displayed).

Cheers, NaN

[Carpo]

front page news :D

[NaN]

@Nic: thanks! The changelog is quite complete (maybe some early points missing). I think porting the function pointer code can be done in extremely short time, is useful in every case, since the if cpu.xxx branches kill a lot from the speed the optimization brings and additionally the code becomes easier to read. Unfortunately the static member function pointers are called via the this-pointer which takes a register, so using these creatures excessively is not always a good idea.

The dec/inc replacement is faster on P4s, on Athlons (only older ones?) the dec/inc and jz/jnz are paired in the pipeline and should be faster. In the end it's the decision of the coder...who has a P4 ;-)

However I think I will rewrite the MC_mmxext code some day, since it isn't very beautifull code.

Have a nice day, NaN

[FredThompson]

Yup, SSE version yields green screen on my Athlon XP 2200, too.

[Grumpy Troll]

I tried the SSE version on my "homemade" with an Athlon XP 1400 cpu using CCE SP 2.50, didn't get a blue screen but CCE would only do the first pass then give a screen saying checksum error, first pass isn't equal to second pass then start running first pass on the next file. Sorry if i'm not to clear as I'm a "User". Does that mean I'm an idiot? It takes one to know one! ;) ;)

I hope this helps.


Grumpy Troll

[NaN]

Update on the SSE-build:

I found the problem and - if no additional troubles show up - the new build will be released. Currently 2 nice testers are looking at it.

Cheers, NaN

[Rippraff]

Yes it works like a charm! :thanks:

By the way
All I said was that he needed to follow GPL. I suggested one way to do that. Are you reading more into it than what I said, or are you just being ornery?
Are you talking to me? If so I didn't get you, sorry. :confused:

@int 21h
I didn't delete anything of my post, I gave just an additional information about my test result.

Cu Rippraff

[Guest]

Cleaned out my OP to reflect this. Tried to send you a PM but your box was full.

Just to say that I deleted my message too. If you want to delete your last I can make it, and this one, disappear completely.

Sorry for any confusion.

[Carpo]

I tried the SSE version on my "homemade" with an Athlon XP 1400 cpu using CCE SP 2.50, didn't get a blue screen but CCE would only do the first pass then give a screen saying checksum error, first pass isn't equal to second pass then start running first pass on the next file. Sorry if i'm not to clear as I'm a "User". Does that mean I'm an idiot? It takes one to know one! ;) ;)

I hope this helps.


Grumpy Troll

means you need the checksum patch - google and you should fine it

[Guest]

I suppose it would make the most sense to host NaN's builds with mine, so when they are stable I'll do that.

[NaN]

@neuron2: thanks a lot! I gladly accept that!

@Carpo & Grumpy Troll: has nothing todo with CCE. I'm sure when viewing the avs files, you only see (maybe flickering) green. It was caused by an uninitialized cpu struct that was so nice to select the (new) SSE2 code even for MMX machines.

1 tester already reported success, looking forward to the 2nd!

Cheers, NaN

[MarkP]

Ok now i am lost between the 2 threads on this subject, is there a non SSE2 version posted anywhere, cos I cannnot find it anywhere yet people seem to be using it

Thanks

[Rippraff]

@MarkP

NaN deleted the SSE-Version because it didn't work on Athlon XPs. Me I'm one of the two beta testers who NaN mentioned above.
I'm pretty sure there'll be a new version soon. ;)

Cu Rippraff

[MarkP]

OK Thanks

Hows the testing going? Looking better?

[Cyberia]

I tested this last night on my Athlon 3200 and everything worked fine. However, there was little if any improvement speedwise.

I used the SSE version, and did a two pass encode of a 20 min vob, with no postprocessing or any extras.

52 minutes using DGDecode (1.4.6 b1)
51 minutes using NANdecode

I can try again tonight with a much longer encode and will probably use standard postprocessing so it's a real world test.

I didn't have any crashes or green screens.

[Rippraff]

Here are my first results:
Original DGDecode 1.1.0
-----------------
[02:21:31] Phase I, PREPARATION started.
- CCE SP 2.70.2.1 encoder selected.
- "One Pass VBR (w/analysis)" mode is enabled.
- VTS_08: 2.957.614 sectors.
-- Scanning and writing .D2V & .AVS files
-- Processed 185.274 frames.
-- Building .AVS and .ECL files
- Reduction Level for DVD-5: 70,9%
- Overall Bitrate : 4.002Kbs
- Space for Video : 3.620.562KB
- Analyzing VTS_08 for optimal Q factor.
-- TargetSize (sectors):1.837.435
-- Sampling 5568 of 185274 frames.
-- Predicted size (sectors) at Q=26: 1.816.183
-- Predicted size (sectors) at Q=25: 1.854.306
- Q Value selected: 26
- HIGH/LOW/TYPICAL Bitrates: 4.848/3.128/4.002 Kbs
[02:29:48] Phase I, PREPARATION completed in 8 minutes.
[02:31:25] Phase II ENCODING started
- Creating M2V for VTS_08 segment 0
- Creating M2V for VTS_08 segment 1
- Creating M2V for VTS_08 segment 2
- Creating M2V for VTS_08 segment 3
- Creating M2V for VTS_08 segment 4
- Creating M2V for VTS_08 segment 5
- Creating M2V for VTS_08 segment 6
- Creating M2V for VTS_08 segment 7
- Creating M2V for VTS_08 segment 8
- Creating M2V for VTS_08 segment 9
- Creating M2V for VTS_08 segment 10
- Creating M2V for VTS_08 segment 11
- Creating M2V for VTS_08 segment 12
- Creating M2V for VTS_08 segment 13
- Creating M2V for VTS_08 segment 14
- Creating M2V for VTS_08 segment 15
- Creating M2V for VTS_08 segment 16
- Creating M2V for VTS_08 segment 17
- Creating M2V for VTS_08 segment 18
- Creating M2V for VTS_08 segment 19
- Creating M2V for VTS_08 segment 20
- Creating M2V for VTS_08 segment 21
- Creating M2V for VTS_08 segment 22
- Creating M2V for VTS_08 segment 23
- Creating M2V for VTS_08 segment 24
- Creating M2V for VTS_08 segment 25
- Creating M2V for VTS_08 segment 26
- Creating M2V for VTS_08 segment 27
- Creating M2V for VTS_08 segment 28
- Creating M2V for VTS_08 segment 29
- Creating M2V for VTS_08 segment 30
- Creating M2V for VTS_08 segment 31
[03:22:02] Phase II ENCODING completed in 51 minutes.


NaN's DGDecode 1.1.0+
-----------------
[15:23:51] Phase I, PREPARATION started.
- CCE SP 2.70.2.1 encoder selected.
- "One Pass VBR (w/analysis)" mode is enabled.
- VTS_08: 2.957.614 sectors.
-- Scanning and writing .D2V & .AVS files
-- Processed 185.274 frames.
-- Building .AVS and .ECL files
- Reduction Level for DVD-5: 70,9%
- Overall Bitrate : 4.002Kbs
- Space for Video : 3.620.562KB
- Analyzing VTS_08 for optimal Q factor.
-- TargetSize (sectors):1.837.435
-- Sampling 5568 of 185274 frames.
-- Predicted size (sectors) at Q=26: 1.816.183
-- Predicted size (sectors) at Q=25: 1.854.306
- Q Value selected: 26
- HIGH/LOW/TYPICAL Bitrates: 4.848/3.128/4.002 Kbs
[15:32:01] Phase I, PREPARATION completed in 9 minutes.
[15:32:01] Phase II ENCODING started
- Creating M2V for VTS_08 segment 0
- Creating M2V for VTS_08 segment 1
- Creating M2V for VTS_08 segment 2
- Creating M2V for VTS_08 segment 3
- Creating M2V for VTS_08 segment 4
- Creating M2V for VTS_08 segment 5
- Creating M2V for VTS_08 segment 6
- Creating M2V for VTS_08 segment 7
- Creating M2V for VTS_08 segment 8
- Creating M2V for VTS_08 segment 9
- Creating M2V for VTS_08 segment 10
- Creating M2V for VTS_08 segment 11
- Creating M2V for VTS_08 segment 12
- Creating M2V for VTS_08 segment 13
- Creating M2V for VTS_08 segment 14
- Creating M2V for VTS_08 segment 15
- Creating M2V for VTS_08 segment 16
- Creating M2V for VTS_08 segment 17
- Creating M2V for VTS_08 segment 18
- Creating M2V for VTS_08 segment 19
- Creating M2V for VTS_08 segment 20
- Creating M2V for VTS_08 segment 21
- Creating M2V for VTS_08 segment 22
- Creating M2V for VTS_08 segment 23
- Creating M2V for VTS_08 segment 24
- Creating M2V for VTS_08 segment 25
- Creating M2V for VTS_08 segment 26
- Creating M2V for VTS_08 segment 27
- Creating M2V for VTS_08 segment 28
- Creating M2V for VTS_08 segment 29
- Creating M2V for VTS_08 segment 30
- Creating M2V for VTS_08 segment 31
[16:23:46] Phase II ENCODING completed in 51 minutes.

Movie plays fine but I couldn't see any speed increase on my XP-M.
But to be fair, first one runs during the night with system untouched, while I was working during second test.
So I will rerun the test tonight and post the result tomorrow.

Cu Rippraff

[Carpo]

I tried the SSE version on my "homemade" with an Athlon XP 1400 cpu using CCE SP 2.50, didn't get a blue screen but CCE would only do the first pass then give a screen saying checksum error
Grumpy Troll

@NaN thats what i was on about - its a known prob with cce on athlons :)

[Manao]

Cyberia : mpeg2 decoding on your computer should be @ 300 fps. which means that 20 minutes ( x2 passes ) gets decoded in 200 seconds. Now, you gain 1 minute between the NaN build and the old one, on these 200 seconds, you're gaining 60 seconds, which means a 60/200 = 30% speed up of the decoding process. Of course, timing are not precise enough, and the real gain should be ~10%, as NaN said, but all that is masked by all the processing you doing alongside ( the mpeg2 decoding is after all taking only 200/2400 = 8.3 % of the overall encoding time ).

[Cyberia]

@Manao: I don't understand, please explain a bit more slowly for a dumb person like me. :)

I understand that the encoding process is seriously distorting the test, but I don't know of any way to test the decoding portion in isolation.

I don't follow where you are getting the 300fps and 200s figures.

Also, a correction on my part. Here are the exact video properties:

15969 frames
8 minutes, 52.83 seconds
29.970 fps

Pass 1 took 9 minutes for both cases.
Pass 2 took 43 minutes for DG, and 42 minutes for NAN

VirtualDub's Batch Controller doesn't list seconds.

Encoder: Vdub 1.6.10, Fast Recompress, no filters, no resizing
Codec: XviD, 2-pass, 1200 bitrate

AviSynth scripts were as follows:DG Version
LoadPlugin("D:\temp\DGMPGDEC\nantest\DGDecode.DLL")
mpeg2source("D:\temp\DGMPGDEC\nantest\DGd2v.d2v")

NAN Version
LoadPlugin("D:\temp\DGMPGDEC\nantest\DGDecode.DLL")
mpeg2source("D:\temp\DGMPGDEC\nantest\DGd2v.d2v")

[Cyberia]

Please allow me to retest, since I clearly screwed up the scripts. ie: They are identical.

My Apologies.

[tritical]

I understand that the encoding process is seriously distorting the test, but I don't know of any way to test the decoding portion in isolation.
How about using avstimer()?

LoadPlugin("D:\temp\DGMPGDEC\nantest\avstimer.DLL")
LoadPlugin("D:\temp\DGMPGDEC\nantest\DGDecode.DLL")
mpeg2source("D:\temp\DGMPGDEC\nantest\DGd2v.d2v")
avstimer(name="dgdecode-orig",frames=2000)
GetAll(frames=14000)

and then change as needed to test the other dll. That should give a pretty good idea of the speed difference.

[foxyshadis]

Best way to test decoding: Create a script with nothing but mpeg2dec("file") (optionally with other options that have been optimized), crop(0,0,4,4), load into vdub, set to direct stream copy, run video analysis pass. Now you've removed filtering, encoding, and hard drive writing from the chain and minimized memory transfer speed; only reading and decoding should change the results.

This is much more useful for fast codecs and quick filtering, such as huffyuv; speeding up decoding is going to be lost in the noise of filter chains and mpeg encoding.

Edit: Good call on the avstimer.

[Rippraff]

Me again, I've rerun the RB encoding, and both versions are equal in encoding time (51 min see above).
So maybe the speedup is only with SSE2 version and it seems to be bigger on a P4 compare to an AMD64.

Cu Rippraff