MPEGDecoder.dll (Direct VOB/M2v -> AviSynth)

[Nic]

http://nic.dnsalias.com/MPEGDecoder.zip

This is a beta I wrote for direct MPEG-2 into AviSynth...

sample script looks like:
LoadPlugin("MPEGDecoder.dll")
MPEGSource("d:\mp2transcode\mya.m2v")

It decodes quicker than MPEG2Source (MPEG2DEC.DLL) & therefore speeds up encoding...also you dont need to make a d2v file.

It can also read MPEG-2 Transport Streams :)

Ill be modifiying over the next few days, but I thought people could try it out... :)

Cheers,
-Nic

ps
Read the Readme.txt b4 use!

[kyousuke]

i'll try this right now!

thanks you ! :D

[Koepi]

Ahoy Nic,

how _dare_ you...?

Now I'm busy somehow extracting the m2v from my matrix VOBs which are unencrypted on HD (the DVD is gone somehow *whistle*), which is a major pain in the a..iroplane.

Would something like

s1=MPEGSource("vob_01_01.m2v")
s2=MPEGSource("vob_01_02.m2v")
s3=MPEGSource("vob_01_03.m2v")

Source=s1+s2+s3

work?

Another idea is
MPEGSource("D:\Video_TS\TheMatrix\vob_01_01.m2v"+
"D:\Video_TS\TheMatrix\vob_01_02.m2v"+
"D:\Video_TS\TheMatrix\vob_01_03.m2v"...)

Unsure if this works either...

(I can't simply join all MPEGs as FAT32 has a well known 4GB barrier :-/ )

I'm not sure as new vobs usually can start with a p-frame...

Anyways, great work, and hope fully I can report some success about the speed impact :)

Thanks,

best regards,
Koepi

EDIT:

ARGS! Dumb I am! Read the readme. Should do that before I start creating a plain MPEG2 file *lol*

[Nic]

s1=MPEGSource("vob_01_01.m2v")
s2=MPEGSource("vob_01_02.m2v")
s3=MPEGSource("vob_01_03.m2v")

Source=s1+s2+s3

doesnt seem to work (grrrr...) ill fix that now :)

Cheers,
-Nic

[Koepi]

Damn. I didn't want to cause more work...

Btw., nice credits in the readme ;)

I'm looking forward to the seeking support!

Best regards,
Koepi

[ACClarke]

A real speed improvement, very good work !!!
But I can just load 1 vob file....

This script crashes in virtualdub

LoadPlugin("C:\DIVX\avisynth\MPEGDecoder.dll")
mpegsource("F:\VTS_01_PGC_04_1.VOB")+mpegsource("F:\VTS_01_PGC_04_2.VOB")

But it's really a very great job.

[Koepi]

That script is technically the same as that one which Nic said it doesn't work and that he's fixing it ATM.

Regards,
Koepi

[ACClarke]

I didn't refresh the page before posting :)
so i don't see his answer.......
I'm sorry

[Koepi]

No need to be sorry, I just wanted to clear that up, some people don't know weather it's different or not. It wasn't meant offensive or anything in that direction.

Best regards,
Koepi

[Nic]

Hmmm, ive fixed it, ill edit this post & say when the latest version is up in a minute (im just testing at present)

Thinking about it though, using multiple vob files could cause problems because if the code guesses slightly wrongly about how many frames there are, you could miss a couple at the end of each vob, thats something that needs to be worked on...

Cheers,
-Nic

[Dreassica]

Originally posted by ACClarke
A real speed improvement, very good work !!!
But I can just load 1 vob file....

This script crashes in virtualdub

LoadPlugin("C:\DIVX\avisynth\MPEGDecoder.dll")
mpegsource("F:\VTS_01_PGC_04_1.VOB")+mpegsource("F:\VTS_01_PGC_04_2.VOB")

But it's really a very great job.

If im not mistaken u can merge the VOB's when riping them with Smartripper!! Would that work or is there some hidden prob with it?
I tried it and although it seemed slightly faster!

[hakko504]

If I recall correctly, one of the problems with using DirectShowSource on vobs are that unless you start each vob with a (I-)key-frame there would be problems with dropped frames when changing vob. I suspect it is the same problem you experience here. I think you should try to create something like SegmentedAVISource, where you give the filename as "filename.avi" but in reality loads filename.00.avi + filename.01.avi + ... Then you could have SegmentedMPEGSource("filename.vob") that automatically loads filename00.vob + filename01.vob + ...

[ACClarke]

If im not mistaken u can merge the VOB's when riping them with Smartripper!! Would that work or is there some hidden prob with it?
I have some problems with files bigger than 4GB
and i already have the vobs files on my hard disk

[WarpEnterprises]

1. Looks VERY VERY GOOD!!!
2. What's the reason (beside not having some time) that such a plugin doesn't already exist? (Was it difficult to make, or some drawbacks not clearly visible or mentioned?)

[Dreassica]

Originally posted by ACClarke

I have some problems with files bigger than 4GB
and i already have the vobs files on my hard disk
That only applies to AVI files, not MPEG

[ACClarke]

That only applies to AVI files, not MPEG
There is no limit for the file size under w98 ????
I'm sure that FAT32 file size limit is 4GB....

[Koepi]

I like the idea of SegmentedMPEGSource(). It should work a little different as usual though, because else we'd need to rename our vobs ;)

It wouldn't make a big differnce to do that, but while we're at it, it would be nice if that algo searches for "vts_02_X.vob", so the last number before the suffix should be incremented. This would make the frame-number problem less painfully I think, as that way you could treat the vobs as a virtual, huge single vob ;) (dunno if that's possible, but this is the idea that came to my mind).

Best regards,
Koepi

[Nic]

Hmmm...That does sound the right idea. If I treat it as one vob (i.e. make the read commands span over all the vobs, that could solve easily :) )

"I'm sure that FAT32 file size limit is 4GB" that is correct, (BTW standard (not DML) AVI files tend to have problems over 2GB (in size))

Ill work on that tonight :)

Cheers,
-Nic

[easyfab]

First at all, thanks Nic for this great work. For me, it give 5% acceleration.
I've a little question about iDCT, in the readme you say it's the same as in dvd2avi but 32 bit, 64 bit or IEEE-1180 reference ?

[Nic]

Its the SSE MMX idct (i.e. the 32-bit one)...Do you want others? (for me its always been the fastest (apart from SSE2 ones) & the accuracy very good)

-Nic

[Koepi]

Hehehe...

work on the 4GB limit of FAT32? ;)

Damn. Then I've now the time to fix OggMux, and I just looked at my sources and they're in such a mess that I really dislike doing that ;) Nic, c'mon, code now ;)

Btw., could you try and compile the filter with the intel compiler and with the options

-Ot -G6 -Qunroll -Qvec_report1 -QxiM -Qip -Gf -Qopenmp -openmp_report1

(removing other /O2 and /G* references)? This could give another speed boost I think :) (well, and it doesn't run on k6/k6-2 anymore... maybe we need two versions of that dll ;) )


Hm.

Thanks for your work Nic, I really appreciate it!

Best regards,
Koepi

[Nic]

Yep ill try with those Params...I normally just use /O3. Doesn't that use full optimisation?

There is too much SSE code in MPEGDecoder.dll so no k6-2/k6-3 support anyway :)
(I could use 3D-Now Motion Compensation for Athlon's etc, but im unsure that it would be better/quicker than SSE MMX.... )

Cant code now :) will later :)

-Nic

[easyfab]

Nic says :
Its the SSE MMX idct (i.e. the 32-bit one)...Do you want others? (for me its always been the fastest (apart from SSE2 ones) & the accuracy very good)

For me SSE MMX idct is good, actually just wanted to know if some speed optimisation could be done with using it.

[CruNcher]

Nic could u intergrate Dmitry and Frankensteins SSE2 Code Optimizations to get a little more speed on P4 systems that would be fine :)

[Nic]

Ive already thought about that :) I will do...
Do you have a link to Frankenstein's source? I know it must be in the DVD2AVI forum somewhere... :)

-Nic

[trbarry]

Nic -

Great work!

Much of DmitryR & Dr. Frankensteins source is already in the save-oe project, but not the GetBytes or color conversion code. But I have all of it from a few months ago. Email me if you need it.

But the performance important parts are probaby SSE2 motion estimation and IDCT, and they are already in there.


Thinking about it though, using multiple vob files could cause problems because if the code guesses slightly wrongly about how many frames there are, you could miss a couple at the end of each vob, thats something that needs to be worked on...

A wild idea ... have you considered taking your best shot and then sticking to it. ;)

That is, estimate the number of frames as good as you can and then returning exactly that many. Stop if long or pad black frames if needed.

Otherwise there may sometimes be Vdub like problems (used to be) where when you used DirectShowSource it would cause an error at the end of the file.

Maybe even stopping isn't necessary but I'll bet padding is. If you report 10000 frames then maybe you should return that many, even if they are not really there. ???

I don't know what the effect of all this would be on audio sync.

- Tom

[dividee]

Very nice Nic ! :)

I'm curious: Wouldn't it be difficult to implement random access without a d2v ?

What's going to be next? Azid-based AC3 decoding ? :p

[Nic]

:) Yup very difficult without a d2v file (well not difficult, but Id have to make an educated guess, seek to there in the file, scan for the correct frame, etc etc :) )

No random access with this filter, because for encoding, we dont use it :)

What's going to be next? Azid-based AC3 decoding ?
:) Now thats not a bad idea :) lol

-Nic

[Koepi]

Niiiiic!

Stop posting, add the multisegment reading, I want to do a testrun for the matrix! ;)

Best regards,
Koepi

[dvman]

I've used your sample script and only modified it to reflect my local paths. And guess what! I have no speed increase over the "old" dvd2avi(1.76)+avisynth(2.04)+mpeg2dec.dll. But thanks for your work anyway... :)

My core specs: AthlonTB 1400, 1Gig of Ram...

[Nic]

Strange ? Well it was definitely faster for me...but ive now added a MMX version of memcpy & restructured the decoding (I directly decode into AviSynth's frame buffer)

Dont expect a big increase though, its just that little bit quicker :)
(best to look at the estimated end time in VDub & compare)

@Koepi: LoL :) Ive just finished it (not yet for transport streams though), testing it now...

It now works so a sample script would look like:
LoadPlugin("MPEGDecoder.dll")
MPEGSource("e:\video_ts\vts_05_01.vob + e:\video_ts\vts_05_02.vob")

You can add upto 30 file names like that...does anyone know if there is a limit on the string size of AVISynth parameters??

Cheers,
-Nic

[Koepi]

Heh :)

Nice Nic!

If I remember correctly, a statement can only have 255 characters (somewhere deep hidden in the original aviysnth documentation).

Maybe it helps if you rename your files to something short and put the avisynth script into the same dir, then it shouldn't be a problem.

For now, it's beta, so such limitations are still very comfortable to live with, I think :)

Best regards,

Cheers *hicks*, :)
Koepi

[Nic]

Hmmm... 255 Thats even less than MAX_PATH. I guess I could also add support for .lst files then :)

-Nic

[dividee]

By looking at the source, I think there isn't any length limitation on strings. If there is one, it should be more than 4096 chars.

But it would be nice to have a SegmentedMPEG2Source anyway :p

For memcpy, you might want to use the env->BitBlt function provided by avisynth. For now it uses the stock C lib memcpy, but soon it will be replaced by highly optimized processor-dependent code.

[Moat]

Hmm, It doesn't seem to like my .m2v files. Mine are created from my tivo using extraction methods.

-Moat

[Nic]

@dividee:
Thats good news, ill add a segmented mpeg source too :)
(The memcpy im using at present is the mmx one supplied by AMD, Ive tried to cut the number of memcpy's now as well :) )

@moat:
Could you upload a sample somewhere of one of your small m2v files?

-Nic

[jonas]

sadly vdub crashes when I try to load my test.avs:
LoadPlugin("C:\...\MPEGDecoder.dll")
MPEGSource("C:\...\test.mpg")

Am I doing something wrong?
I'll add vdubs crashinfo as attachment.
The test.mpg is an mpeg2 file, 4.5 Mb in size, created with power vcrIII and works fine with .dv2+mpeg2dec.dll.

jonas

Edit:
Forgot to mention. I'm using the 2.04 avisynth at the moment. so might be that?

[Nic]

Hi Jonas,

is there anyway you can upload/send me a bit of that file.
I have a feeling it might be a raw mpeg2 file (ES stream). My new version has support for that....(i.e. you type "raw" in the PID field to set the decoder in raw mode)

I should release it soon :)

-Nic

ps
Whoever, came up with state sequence codes & program end codes for MPEG-2 should be taken outside & shot :)

[jonas]

Hi Nic,
I don't have much time at the momet, so I'll just add the mpeg2 file as attachment.
Would be great if it's allready solved.

jonas

[jonas]

the hurry made me forget to add the addachment

[Koepi]

Originally posted by Nic
I should release it soon :)


I'm sure plenty people will appreciate that! :)

Regards,
Koepi

[sh0dan]

Originally posted by jonas
sadly vdub crashes when I try to load my test.avs:
LoadPlugin("C:\...\MPEGDecoder.dll")
MPEGSource("C:\...\test.mpg")

[...]
The filter seems to require integer SSE - If you haven't got a P3, P4 or K7 this will happend.

Maybe there should be fallback to slower (pure MMX?) code?

(it's an illegal instruction with a pshufw)

[Nic]

Well spotted sh0dan, I just assumed that he had read the thread & seen & understood that it needs SSE...I should have made it more clear.

The original C code is still quite quick, so Ill add that back in later, but most probably have a SSE computer (even this ancient Celeron im using does) :)

Just added a new function called NicCrop...same as AviSynth's Crop but takes more obvious parameters...
i.e.
NicCrop(8,10,64,68)
will crop 8 off the left handside
10 off the right handside
64 off the top &
68 off the button

(well, it will be useful for me at least...just testing to do now :) )

-Nic

[Koepi]

Nic,

you certainly rule! I like that crop version, it's easier to take over the values you found in vdub, thus more intuitive (no more switching around between the filter chain view and the cropping window! YES!).

Thanks for your nice work!

Btw., could you upload a current snapshot? I'm still waiting to encode the matrix again ;)

Best regards,
Koepi

[Nic]

Well just to please Koepi: :D

http://nic.dnsalias.com/MPEGDecoder.zip
version 1.0 beta (beta because I havent tested thoroughly yet)

The documentation has been hurriedly updated so please re-read for new features, ill update it properly soon :)

This version is definitely faster, supports multiple files, list files, NicCrop & more stable (hopefully)

Cheers,
-Nic

[hakko504]

Just nitpicking a little, but how would NicCrop(8,10,64,68) differ from the regular (A2.03+) Crop(8,10,-64,-68) ?

[WarpEnterprises]

It seems not at all.
We will have to mark the new functions very clearly in the docs.
(There isn't only this example!)

[Nic]

:D LoL Didn't know you could do that with the negative numbers....oh well, its there anyway for people to use :)

Cheers,
-Nic

ps
Oh I didnt realise its a recent addition (Avisynth 2.03+), Im glad I did it then, because im still using beta 5 :)

[hakko504]

Originally posted by Nic
Oh I didnt realise its a recent addition (Avisynth 2.03+), Im glad I did it then, because im still using beta 5 :)
I thought so. :D


Keep up the good work: I haven't had time to test the new decoder properly yet, but I will do so in the near future. It looks very promising, to say the least. Just adding a few fps will be very useful :)

[ACClarke]

multi vobs is OK
but there is a problem with frames estimation
for exemple
Lord of the ring (the anime not the movie)
real frame number(dvd2avi) -> 192008 frames
when I use a lst file -> 221898 frames
with vob1 + vob2 + ...... -> 285598 frames

for me it's the last bad point