DVR-MS in DGIndex

[Carraway]

This post is sort of a test balloon. I know virtually nothing about programming and very little about how DVD2AVI & DGIndex work. I figured I'd post this on the very off-chance that someone in here is familiar with the dvr-ms container and can give me a little a little bit of info.

Basically, DVR-MS files are the container that Microsoft's Media Center program uses to store the television its recorded. I don't know everything about this format, but I can tell you with 100% certainty that they are MPEG-2 and some kind of MPEG audio and DVR-MS. The graph looks like this:

http://home.columbus.rr.com/carraway6/dvrms01.gif

So if they were in the MPEG container, DGIndex would presumably have no problem with them. But they're in the DVR-MS container, and this happens. Bah:

http://home.columbus.rr.com/carraway6/dvrms02.gif

The stats looks like this (took a screenshot in preview mode):

http://home.columbus.rr.com/carraway6/dvrms03.gif

I'm guessing V.E.! means video error or something similar. Again, I really know not of what I speak, but there doesn't seem to be very much in the way of DVR-MS files being treated like normal MPEG-2 files within DGIndex. Is there anyone at all familiar with the ins and outs of DVR-MS files who might know what causes this, whether its can be fixed (aside from simply converting it to the MPEG container, which is no picnic), or have any other info that might make me go "ooh"?

[bond]

read more about dvr-ms files here (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnwmt/html/dvrfilechanges.asp)

[Carraway]

Originally posted by bond
read more about dvr-ms files here (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnwmt/html/dvrfilechanges.asp)

:thanks:

So, ok, I have a bit of understanding about how DVR-MS files work. Now my question isn't so much about the file format itself as much as with how it interacts with DVD2AVI. I literally know nothing about how DVD2AVI really works (I just accept that it does, its like magic!), so I don't know if creating working D2V files from DVR-MS mpeg2 files constitutes a small or large problem, i.e. is there a fix I can do myself? Would this require a very small addition to DGDecode.dll? Would this require a ridiculously large re-write of DGDecode.dll? Is this a request I could make to Donald or would this addition to DGIndex be such a difficult undertaking that it'd be easier for everyone if I just sat down and stayed quiet? :)

[Guest]

Microsoft would crush my balls in a vise if I added decoding of DVR-MS. I like my balls.

[Cyberia]

Originally posted by neuron2
Microsoft would crush my balls in a vise if I added decoding of DVR-MS. I like my balls.
Don, you crack me up!

[DDogg]

Microsoft would crush my balls in a vise if I added decoding of DVR-MS. I like my balls.Don, do you still feel this is true? There are a few commercial products like VideoReDo that read and rewrite these files (the unprotected ones). The ability for dgindex to open these unprotected files would be very useful. If you do become interested, I can make samples available and do any other grunt work you need done.

I also noticed this post (http://forum.doom9.org/showthread.php?t=105013&highlight=center) which went dead.

[zambelli]

Microsoft would crush my balls in a vise if I added decoding of DVR-MS.
Says who? Sheesh, it's not illegal to implement ASF. :)

http://www.microsoft.com/windows/windowsmedia/forpros/format/asfspec.aspx

[Guest]

I wonder then why Avery Lee was forced to remove ASF support from VirtualDub.

In any case, I have started some work on this for DGMPGDec, but it is going slower than I like due to personal issues leaving me with little time for coding right now.

[DDogg]

I wonder then why Avery Lee was forced to remove ASF support from VirtualDub.

In any case, I have started some work on this for DGMPGDec, but it is going slower than I like due to personal issues leaving me with little time for coding right now.Well that was in the dark ages. Probably some over zealous MS legal department at the time. I'm thinking stopping VDub asf was an ill considered move that hurt the format considerably with the video community. Seems a different climate now.

If/when you do find time, let me know if I can help in any way - samples, testing, etc.

btw, AutoMKV has new support for DVR-MS via directshowsource (convertfps = true). Still a few ar problems with the mux, but they should be fixed soon.

[zambelli]

Well that was in the dark ages. Probably some over zealous MS legal department at the time.
Actually, it wasn't even that. From talking to Avery, it turns out that somebody just acted on their own accord. Avery was never contacted by a Microsoft attorney or any attorney associated with Microsoft. That's a fact, and anybody can ask Avery if they don't believe me.

I'm thinking stopping VDub asf was an ill considered move that hurt the format considerably with the video community.
Agreed, plus it brought a lot of bad PR, unjustly. I've explained many times on this forum that Microsoft never threatened Avery, yet that's still the prevalent forum myth.

Seems a different climate now.
Indeed it is, and I'm glad you noticed. If anyone ever wants to implement Windows Media technologies and wants to know if it's legal, all they have to do is send me a PM and I can get them in touch with the right legal people. Even if the answer turns out to be NO, I don't want anybody to think that they can't ask or that bad things will happen if they do.

[Guest]

Sheesh, it's not illegal to implement ASF. :)


Even if the answer turns out to be NO... How do you reconcile these two statements?

[zambelli]

How do you reconcile these two statements?
I was speaking generally in the second statement, implying that it's decided on a case-by-case basis and that not everybody's idea of WM implementation is the same so there can't be a "one size fits all" verdict. Somebody cracking WM DRM and somebody adding ASF reading support to their app are not the same thing. :)

All I'm saying is: if you want to add DVR-MS support to DGIndex, don't just assume it's off limits and that "Microsoft would crush your balls" if you ever tried. If you want to know whether it's plausible, just ask and I can dig around and find out for you. There might be some jumping through hoops required to work around the clashing of Microsoft's end user licenses with GPL licenses, but in general - implementing ASF support is certainly doable and perfectly legal.

[Guest]

I can dig around and find out for you. Please do. Thank you.

[DDogg]

Z, I have admired your professionalism on this forum and think you have done a lot to repair some of the bad feelings. This may be an opportunity to provide another positive contribution. Namely a blanket statement for the GPL community, not just a one off for DG. I would think even you would roll your eyes with trepidation at making the attempt to dig that through the bureaucracy, but it may be worth going through the bigger hassle once, versus many little ones?

[zambelli]

Z, I have admired your professionalism on this forum and think you have done a lot to repair some of the bad feelings. This may be an opportunity to provide another positive contribution. Namely a blanket statement for the GPL community, not just a one off for DG. I would think even you would roll your eyes with trepidation at making the attempt to dig that through the bureaucracy, but it may be worth going through the bigger hassle once, versus many little ones?
Thank you, I appreciate the kind words. I think it'd be really difficult to make a blanket statement for the GPL community here because it'd have to come from MS legal people on the highest level - and I'm afraid my influence doesn't go that far, plus I'm sure any such statement would have to be examined very carefully and deliberately in order to make sure it's not opening an unwanted can of worms.
I'm most familiar with the world of Windows Media, so that's where I can be of most help.

[DDogg]

My use of "blanket statement" was overly broad. I was suggesting a situation where MS would clearly grant permission for the gpl community to "open" a non drm protected ASF file and rewrite it into another standard, non drm, format. Certainly such a thing must exist as this is exactly what VideoReDo does. [This would go a long way on putting to rest the "that's still the prevalent forum myth" thing you mentioned above, regarding the PR fiasco with VDub asf support]

[bond]

iirc avery referred to a specific patent owned by m$ on asf. unless m$ states that you can use this patent freely you will always be at risk to get legal problems when using it without approval of m$

[Wilbert]

iirc avery referred to a specific patent owned by m$ on asf. unless m$ states that you can use this patent freely you will always be at risk to get legal problems when using it without approval of m$
Yes, but not only that ...

(f) For a variety of reasons, including without limitation, because you do not have the right to sublicense the Necessary Claims, your license rights to the Specification are conditioned upon your not creating or distributing your Implementations in any manner that would cause ASF (whether embodied in your Implementation or otherwise) to become subject to any of the terms of an Excluded License. An “Excluded License” is any license that requires as a condition of use, modification and/or distribution of software subject to the Excluded License, that such software or other software combined and/or distributed with such software be (x) disclosed or distributed in source code form; (y) licensed for the purpose of making derivative works; or (z) redistributable at no charge;
http://forum.doom9.org/showthread.php?p=825692#post825692

So, it's NOT legal to add ASF support and to release DGIndex as GPL (as should be done). Since zambelli replied in that thread, so he should know this.

[Guest]

How about if I make a DLL for the DVR-MS parsing and then not GPL that?

[Wilbert]

How about if I make a DLL for the DVR-MS parsing and then not GPL that?
That should be legal, apart from the patent problem Bond is talking about. But luckely there are not many places where software patents are valid :)

[zambelli]

So, it's NOT legal to add ASF support and to release DGIndex as GPL (as should be done). Since zambelli replied in that thread, so he should know this.
I mentioned earlier that there might be some jumping through hoops required to get around that. I see that as a minor issue. Licenses are just licenses - as the author of DGIndex, Donald can choose what license he wants to apply and how he wants to implement his own software.

I'm not sure what patent limitations Bond is referring to. http://www.microsoft.com/windows/windowsmedia/licensing/final.aspx state that implementing ASF readers and writers is free of charge. Also, if Donald wants to use the WMF SDK to do the parsing, that's free too - http://www.microsoft.com/windows/windowsmedia/licensing/licensing.aspx - and I think it might even be GPL-compatible because it's just a public SDK. There are a number of options, and I'll try to get legal advice from within MS to make sure we're not just speculating.

[Guest]

Licenses are just licenses - as the author of DGIndex, Donald can choose what license he wants to apply and how he wants to implement his own software. I'm not free, because DGMPGDec is an evolution of DVD2AVI + mpeg2dec, both of which are GPL. That's why I suggested the DLL route.

In any case, I'd appreciate hearing what you find out.

[Inventive Software]

MS IIRC state that proper DVR-MS support be implemented through DirectShow. Thoughts?

[Guest]

It will be much easier for me to just add an ASF parsing layer to DGIndex/DGDecode.

[Inventive Software]

Completely forgive my ignorance, but is essentially DVR-MS the ASF container with MPEG-2 video and MPEG-1 layer 2 audio?

[bond]

y e s

[DDogg]

Isn't the video a variable framerate? [To use DS and achieve audio sync requires convertfps=true]

[DDogg]

<Bump>

Any further progress of these discussions?

[Guest]

It is so easy to demux the audio and video from DVR-MS using GraphEdit that my motivation to complete adding it to DGMPGDec took a nosedive.

[buzzqw]

not all average user are smart enough to use graphedit, but dgindex is surely a friendly application for common usage.

take your time and feel free to add this features or not

thanks!

BHH

[Mr_Odwin]

To add to this, I think it would be nice to have access to this feature in an application that is cli friendly, such as DGIndex.

[DDogg]

It is so easy to demux the audio and video from DVR-MS using GraphEdit that my motivation to complete adding it to DGMPGDec took a nosedive.I know you like to see the justification for the request, so here goes:

1> Many (near all?) GUI frontends use DGIndex. Many of these programs, AutoMKV and FAVC, for example would like to support DV-MRS files in a similar fashion as they support Mpeg2. Thus, there is a need for the "standard" dgindex method so that support for these files can be added without resorting to lesser methods.
2> GraphEdit is not easy to understand or use by a large percentage of users.
3> DirectShowSource is not a workable method for allowing editing of commercials, etc as seeking is near impossible. Try it in VDub and I promise you will not have any hair left, and your cat's will have a sore backside for weeks.
4> As with another discussion we had, if DGIndex does not support it, it ain't a real format and will never mainsteam into 'our' community.
5> Nearly every Windows XP and Vista machine sold now has a variant of WMC. The need for support will increase exponentially as people add tuners.
6> Please.

[lazyn00b]

Completely forgive my ignorance, but is essentially DVR-MS the ASF container with MPEG-2 video and MPEG-1 layer 2 audio?

Hey, don't forget those of us recording HDTV - DVR-MS files also can contain AC3 audio!

[Inventive Software]

Hey, don't forget those of us recording HDTV - DVR-MS files also can contain AC3 audio!

I read it from Microsoft's site. Blame them! ;)

It is so easy to demux the audio and video from DVR-MS using GraphEdit that my motivation to complete adding it to DGMPGDec took a nosedive.

Care to share then? :rolleyes:

[Pookie]

UPDATE - The ffmpeg command line method didn't work consistently on additional dvr-ms samples :(


I don't have a dvr-ms file to test, but doesn't ffmpeg have demuxing support for this ? Once you've freed the prisoners of microban, you should have a plain ol' mpeg2 that dgindex can read.

Another possible option is a command line util that runs .grf files
http://www.merage.uci.edu/~dvdotnet/graphviewer.txt
http://www.merage.uci.edu/~dvdotnet/downloads/graphviewer10.zip

Edit - ffmpeg seems to work on a test clip - EDIT - Nope, I'm wrong.
http://tirnanog.fate.jp/mirror/ffmpeg/ffmpeg.rev7375.7z
http://sourceware.mirrors.tds.net/pub/sourceware.org/pthreads-win32/dll-latest/lib/pthreadGC2.dll
ftp://ftp.mplayerhq.hu/MPlayer/samples/dvr_ms

ffmpeg -i test.dvr-ms -vcodec copy -an -y test.m2v
ffmpeg -i test.dvr-ms -acodec copy -vn -y test.ac3

Resulting .m2v opened in DGindex

[DDogg]

For folks that want to deal with GraphEdit, here is a decent article (http://msdn.microsoft.com/smartclient/default.aspx?pull=/library/en-us/dnxpmce/html/dvr-ms.asp#dvr-ms_topic9)

@pookie, have you tried the above with a stereo ms-dvr and remuxed it to check audio sync?

[Pookie]

Nope... I'll try it now. Dump that dvr-ms nonsense and get GBPVR ;)

Edit - Moot point to even try it, as I noticed the commands above did NOT work on every test file I've attempted it with - actually, most didn't.
Arghhh...frustrating. I'm sure you guys have tried the VLC method- once again seems to work nicely on my 3 test files - added advantage that it doesn't need to demux - it just changes the container to mpeg2-ps :)

Vlc 0.86a

File-->Wizard-->Transcode/Save--> Select-->(Bypass Transcode screen)-->Encapsulation (Choose Mpeg-PS)-->Save (as *.mpg)


* Will someone try the VLC Method above on a longer-than-20-second clip ? *

The basic flow then, is:

.Drv-ms-->VLC to MPG container -->Open MPG in Dgindex-->You know the rest

A/V Sync was perfect, btw

[Jim_IT]

Vlc 0.86a

File-->Wizard-->Transcode/Save--> Select-->(Bypass Transcode screen)-->Encapsulation (Choose Mpeg-PS)-->Save (as *.mpg)


* Will someone try the VLC Method above on a longer-than-20-second clip ? *

Pookie,

I tried on a 1 minute ATSC dvr-ms segment, I extracted with tool from the sample code in the Fun with DVR-MS msdn article Ddogg linked to in his above post.

VLC test performed terribly. Resulting mpg file was way out of sync.

I have been playing around with getting the proper formula correct for working with HD dvr-ms recordings for the past 2 years. Mostly, I am successful in most conversions, but still there is no reliable way to process OTA HD material.

My latest testing, has provided a few interesting findings.

First, ATI Avivo converter is fast and sync, but has ok to poor quality, depending on your tastes. This led me to look at using ATI's muxer in grapedit. It has been recently updated, and it has a HD mpeg profile. The output mpeg2-ps remains in sync and stutter free. Used projectX to make cuts.

Second, using ffmpeg to create valid psp avc files in one step seemed obivous. ffmpeg, chokes on an invalid codec when the dvr-ms is directly processed, but encodes successfully when given mpeg2-ps muxed file. The encoding time is about 3 times program length on 3200+ and files look great, but sometimes stop inexplicably on the PSP (if you can ff over, file will continue to play).

Last, there are some recordings you simply cannot process. My last attempt at a 30 min episode, will not remain in sync after removal from ms asf wrapper.

What sucks is you set up a bunch of encodings, fill the memory stick up with a bunch of episodes of a series and the 2nd or 3rd will not play, so you can't watch the remaining episodes.

What I'd give to find something more reliable. I have used, tried almost everything out there.

I'd be happy to upload or grant ftp access for, any size HD dvr-ms file samples.

Jim

[Pookie]

Bummer...

Thanks for testing it out, Jim_IT. Presumptuous of me to think I could find the panacea after a couple of days of clicking around.

[DDogg]

Don, as you can see from these posts, there is very much a need for the dgindex standardized method to handle standard stereo ms-dvr files as well as the HD type with AC3. I continue to hope you might consider adding this functionality when time allows, but I do not know enough about the ms-dvr format to know if adding support in dgindex is even possible. Your thoughts on that?

[Inventive Software]

I'll see about providing some sample DVR-MS British broadcast streams when I get my TV tuner.

[nnigam]

I can provide DVR-MS samples of OTA broadcast in dc metro area. Let me know what you want and how to send it to you. I can record any show you want. These will not be HD since my HD card is not supported with Windows Media Center.

Neeraj

[DDogg]

Don, just curious if we should consider this a dead issue, or if you continue to evaluate the possibility. Your new work dgavcdec for 264 is great and I would still like to hope for something similar for DVR-MS. Completely understand if this is an area you would prefer not to explore further. If so, just let us know and this old thread can die without future bumps.

[Guest]

I'm moving on to the H.264 world. There is an easy solution for DVR-MS with Graphedit. Anyway, I don't use DVR-MS so there is nothing in it for me.

[Inventive Software]

Darn. Looks like I'll have to have a go at knocking something up then.......

[sjchmura]

So with graphedit do we just output the the filewriter?

[Guest]

Use dump.ax from here. There are other dumper filters, too. Take your pick.

http://www.apecity.com/tivo/

[DDogg]

The main thrust of the request was to allow the many gui frontends a standardized way of dealing with these files. Using Graphedit can't accomplish this. DGIndex is the standard and it was my thought that adding DVR-MS capability to it would allow many of our best GUI frontends to handle DVR-MS files in a clean and efficient manner.

Directshowsource with convertfps=true will handle the SD stereo files, but leaves those with HD and 5.1 somewhat out in the cold. Of course, without an index we cannot seek to edit and this is problematic as most would want to cut commercials. I think it would have been a good thing for the community, but know your time is limited and really do appreciate all that you do for us.

Thanks for the reply.

[mdkusr]

Has the legallity of this been figured out yet? I use dvr-ms conversion tools all the time, so I am curious as to why this would be a problem to add to DGDecode.

I use DVRMS Toolbox (http://babgvant.com/default.aspx) to rip the MPEG's out of dvr-ms files, and then process them with DGDecode, so if this guy gets away with it, why can't DGDecode? This would totally eliminate a step for me in cutting commercials from DVR-MS files.

There is an auto commercial cutter in DVRMSToolbox, but I don't trust it. I currently convert my recordings to MPEG, then use DGDecode on them to strip out the audio and video, and then use Cuttermaran to chop out the commercials. Takes about 2 to 3 minutes of effort, but anything that would make this move quicker would be greatly appreciated.

-M

PS. According to my calculations, a DVR-MS file has about 20% bloat. Just simply ripping them into MPEG's files results in a 20% smaller file size in case anyone cares.

[Inventive Software]

The issue is licensing. DGIndex is GPL, whereas ASF (which is what DVR-MS is) is proprietary. BUT, ASF can be decoded legitimately with DirectShow or Windows Media Format SDK. So it's working out how best to do it, and make it available. Any solution would have to work seamlessly with DGIndex, and comply not only with the GPL, but Microsoft's licenses too.

And the bloat is needed for transport stream compliance IIRC... since a program needs to be able to join the stream anywhere. MPEG-2 TS is much the same. DVR-MS probably has less "bloat" than TS anyway, but I haven't compared them. ;)