Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 AVC / H.264

Reply
 
Thread Tools Search this Thread Display Modes
Old 23rd September 2009, 15:16   #1  |  Link
FredThompson
Registered User
 
FredThompson's Avatar
 
Join Date: Feb 2002
Location: Charlotte, NC USA
Posts: 1,972
MKV as DVD replacement possible?

I'd like to convert a few thousand DVDs into AVC MKV files. My hope is to convert them into a format with data and physical size advantage which should also be somewhat independent of storage medium and playable on a variety of soft and hard decoders such as WD TV, PC, BluRay player, etc. I am not interested in downsizing or deinterlacing at all. The original frame size and interlace/progressive structure must remain.

The DVDs have 2 primary structures:

1) Homemade captures with very simple menus (no chapters, no angles but they DO frequently have CC embedded in the VOB)

2) Full commercial discs which can have very complex structures.

Converting #1 above to AVC en masse is simple. The data will be ripped into various source directories using basic DVD ripping software to yield single-file source, even if the file is longer than the 1G DVD VOB standard. When a batch has been ripped, they'll be encoded with x264. Goal is to use VBR to yield files half the size of the original. This will allow comparison with new captures from DVR (DirecTV with TiVo) of the same file. If a new MPEG2 version of an existing source is significantly more than 2X the size of the new MKV it will probably be encoded and stored as the new baseline. DirecTV varies the bitrate, sometimes dramatically, which can affect quality, of course.

However, IF simple profile MPEG4 can truly give the 2:1 data ratio I seek without perceptible quality loss, I'd prefer to use it for such simple sources and convert the CC into subtitle files to accompany the encodes. Is simple profile MPEG4 capable of this?

Converting #2 is not so simple. In many cases, the menu structure is critical. If at all possible, I'd like to keep all the functions of the DVD control structure such as chapters, angles, audio tracks, subtitles, etc.

I have VERY limited experience with h.264 and mkv. The last time I dabbled with MPEG4 was pre-AVC/VC-1 when the propaganda stated equal quality with MPEG2 in half the size...it didn't pan out.

Now, mkv supports multiple audio and subtitle tracks. There are quite a few threads here about converting commercial SD and HD discs to mkv. What I don't see, or haven't found, is something akin to DVD RB Pro which can take a DVD structure and yield an mkv equivalent, preserving all the user interface of the source DVD. Does such a thing exist? If so, where?

If not, are there any automated ways to take a DVD IFO/VOB set and generate the various components which go into the mkv? The more automation, the better, as long as 2-pass x264 and custom AviSynth filters can be used. In this case, the highly complex DVDs would not be processed, only the simple ones.
__________________
Reclusive fart.
Collecting Military, Trains, Cooking, Woodworking, Fighting Illini, Auburn Tigers

Last edited by FredThompson; 23rd September 2009 at 15:23. Reason: I'm AD...oh, look, a bird!
FredThompson is offline   Reply With Quote
Old 23rd September 2009, 15:34   #2  |  Link
thewebchat
Advanced Blogging
 
Join Date: May 2009
Posts: 483
Why would you want to re-encode something without deinterlacing it? Especially for all those cases of telecine and not actual interlacing. As far as I know, you just need the following:

Decrypt
DGIndex
DVD2MKV chapters thingy
Automagic encoding
Automagic muxing

Looks easy enough. I think MEGUI might even cover most of these.

Edit: Oh, right, if by "all the user interface," you mean DVD menus, have fun with that. MKV menus have been vaporware from the start so the chances of that happening are less than 0.
thewebchat is offline   Reply With Quote
Old 23rd September 2009, 17:10   #3  |  Link
FredThompson
Registered User
 
FredThompson's Avatar
 
Join Date: Feb 2002
Location: Charlotte, NC USA
Posts: 1,972
You made a lot of erroneous assumptions.

Interlace /= telecine.

Interlaced video has twice the temporal information of progressive video with the same frame rate.

"Converting" Interlaced video to progressive video destroys the temporal information. No matter how sophisticated the conversion, there are always problems. Try it on a subject which is spinning perpendicular to the camera. Look at high motion subjects on regular and double-rate LCD displays and you'll see a huge difference.

"DVD2MKV chapters thingy" is a concept. I'm looking for a manifested tool for the concept. There is precedent:

http://forum.corecodec.com/viewtopic.php?f=15&t=54
http://forum.corecodec.com/viewtopic.php?f=14&t=1973
http://forum.corecodec.com/viewtopic.php?f=14&t=2966

Given MKV is a container which supports chapters and some rudimentary menus have been seen "in the wild" there is ample reason to believe robust control structures should be possible. MKV is a container, just as ISO or AVi are containers. DVD Menus are nothing more than video "chapters" with some control logic.

I should have stated MKV is not a requirement. Functionality is the goal, MKV is a perceived means to portability.
__________________
Reclusive fart.
Collecting Military, Trains, Cooking, Woodworking, Fighting Illini, Auburn Tigers

Last edited by FredThompson; 23rd September 2009 at 17:13.
FredThompson is offline   Reply With Quote
Old 23rd September 2009, 17:37   #4  |  Link
nm
Registered User
 
Join Date: Mar 2005
Location: Finland
Posts: 2,643
Quote:
Originally Posted by FredThompson View Post
"Converting" Interlaced video to progressive video destroys the temporal information.
Not if you bob it or use some advanced full-rate deinterlacing filter. Otherwise you'd need an old CRT tube to display those interlaced sources properly. Modern display equipment is progressive.

Quote:
No matter how sophisticated the conversion, there are always problems.
That's true to some extent, but the better deinterlacers can do a pretty decent job -- better than what you get by letting the cheap LCD TV deinterlace on playback.

If I were you, I'd just rip to full disc images. If some player can't handle them, write a script to encode the video titles to a supported format but keep the original images around.
nm is offline   Reply With Quote
Old 23rd September 2009, 19:43   #5  |  Link
creamyhorror
Registered User
 
Join Date: Mar 2008
Posts: 118
Quote:
Originally Posted by FredThompson View Post
Given MKV is a container which supports chapters and some rudimentary menus have been seen "in the wild" there is ample reason to believe robust control structures should be possible. MKV is a container, just as ISO or AVi are containers. DVD Menus are nothing more than video "chapters" with some control logic.
I would like to see some of these rudimentary "menus".

I would also like a tool that extracted all parts of a DVD separately and encoded them all according to settings I specified, but I doubt it exists as yet.
creamyhorror is offline   Reply With Quote
Old 23rd September 2009, 19:53   #6  |  Link
thewebchat
Advanced Blogging
 
Join Date: May 2009
Posts: 483
After a very brief googling, I found http://www.videohelp.com/tools/ChapterXtractor

I believe that if you add this to the chain I proposed earlier, that essentially leaves us with only the menus missing. Since MKV menu implementation has a 0% chance of ever being finished, as MKV development is essentially dead, I think this is as close as you'll ever get.

Last edited by thewebchat; 23rd September 2009 at 19:56.
thewebchat is offline   Reply With Quote
Old 23rd September 2009, 20:09   #7  |  Link
Guest
Guest
 
Join Date: Jan 2002
Posts: 21,923
Quote:
Originally Posted by thewebchat View Post
Since MKV menu implementation has a 0% chance of ever being finished, as MKV development is essentially dead
What is your reliable source for this claim?
Guest is offline   Reply With Quote
Old 23rd September 2009, 21:13   #8  |  Link
creamyhorror
Registered User
 
Join Date: Mar 2008
Posts: 118
Quote:
Originally Posted by thewebchat View Post
After a very brief googling, I found http://www.videohelp.com/tools/ChapterXtractor
Wouldn't DVD-Decrypter extract chapters just the same? (Although I think I've actually used ChapterXtractor before...)

In any case, the kind of chaptering FredThompson was looking for is actually a command structure as well as a playback interface it. Which, as you pointed out, doesn't exist and may not for some time to come.

A command/menu structure would be nice to have, it's true.
creamyhorror is offline   Reply With Quote
Old 23rd September 2009, 21:33   #9  |  Link
FredThompson
Registered User
 
FredThompson's Avatar
 
Join Date: Feb 2002
Location: Charlotte, NC USA
Posts: 1,972
Quote:
Originally Posted by nm View Post
Not if you bob it or use some advanced full-rate deinterlacing filter. Otherwise you'd need an old CRT tube to display those interlaced sources properly. Modern display equipment is progressive.
Yes, but that's only a requirement during playback. My goal is to reduce physical space and data size. Bear with me as I explain some of the background thought which led to this post: I was merrily going along using DVD RB Pro and HC but now, the physical size and cost of optical discs has become prohibitive compared to SATA drives. I've been trying to devise a way to keep all my "stuff" while my physical living space is going to be cut in half (I'm getting married.) 25 years of VHS tapes which still occupy a complete wall and another complete wall of discs on spindles have created a problem for this packrat. So..what to do? One option is to rip to ISO as you suggest. But...what if I those can be shrunk in half with virtually no quality loss through current encoding? Things are very different now than they were even 5 years ago. I have more than one computer so once the files are pulled from the source disc, it would be great to re-encode them to half the current size with "unused" computer time. I've also realized that something like the Amahi home server could be used to uncouple the use of the data from its storage medium. I already do something similar with DV. The raw source is encoded to 4:2:2 interlaced MPEG with TMPGEnc at a bitrate of 8,000. The result is about half (maybe less) than the full DV data size with virtually no visual difference. I never filter the DV. MPEG2->MPEG4, in my hoped-for use, would have only some light filtering in an attempt to reduce macroblocks and mosquito noise. Have you seen the dedicated mosquito removal inline filters? Seems to me the same functionality will happen soon enough in GPUs so the goal is reducing the data set size and leaving the heavy lifting for the playback method.
Quote:
That's true to some extent, but the better deinterlacers can do a pretty decent job -- better than what you get by letting the cheap LCD TV deinterlace on playback.
No, it's true period. Interlaced is 2 interwoven sets of still images representing 50 or ~60 distinct time periods per second. Additionally, each of the two interwoven sets of images is shot from a slightly different view point. My goal is to economically use time (CPU to encode) to take advantage of the current size and cost benefits of hard drive storage. Even with motion-compensated methods I still see glitches if there is high motion but it can be visible in other places as well. There is an episode of the History Channel show "Save Our History" (I think that's the title) which has a house in the background as the camera zooms out. The thin wood pieces which separate the window panes (can never remember what those are called) play havoc with deinterlacing. Anything which is rotating can cause problems. The artificial scan lines effect of alternating black horizontal lines can cause problems. Try it on something like flipping hair or acrobatics or motor sports. All things considered, I'd prefer to compress the data size, not convert known data points into interpolated guesses with half the temporal information. A good way to see the result is to compare interlaced source and a deinterlaced version of the same source on 60 and 120 Hz large displays.
Quote:
If I were you, I'd just rip to full disc images. If some player can't handle them, write a script to encode the video titles to a supported format but keep the original images around.
I'm keeping the commercial originals in storage. The homemade stuff, which includes capturing the VHS tapes, doesn't have complex menus so AP MPEG4 seems safe enough to me now. If space and cost weren't issues, I'd just keep doing what I'm doing.
Quote:
Originally Posted by creamyhorror View Post
I would like to see some of these rudimentary "menus".
I'm still looking for a good example. One of the posts I mentioned above has a dead link. My guess is such a menu is a simple branch at the beginning and was only to pick language and subtitle. There's a post at the KMPlayer board stating MPC supports them but KMPlayer does not.
Quote:
I would also like a tool that extracted all parts of a DVD separately and encoded them all according to settings I specified, but I doubt it exists as yet.
Right. That's what I see. Parsing is handled by a number of programs such as DVDShrink, DVD Stripper, PGCEdit, DVD Rebuilder, etc. BR RB Beta looks very promising but the bluray disc format is still used. There are very few players which will mount an ISO but almost every current player works with VOB. Look at things like the WD TV players. They'll work with mkv but won't mount a virtual disc.

@neuron2, howdy! How's kicks?

edit: I'm having trouble finding the forum link I read earlier today which supposedly linked to an MKV sample file with a menu. The link didn't work but the thread mentioned a simple menu which used the first 2 chapter points of the sample data. I'll keep searching.

AutoMKV Chapters: http://forums.animesuki.com/showthread.php?t=85641
101 things you never knew you could do with Matroska: http://www.mod16.org/hurfdurf/?p=8 and http://forums.animesuki.com/showthread.php?t=59561
maybe the anime subculture will figure this out first...

I guess the easiest option would be to add DVD XML and pointers to MKV ignoring the pesky little detail of licensing. Where's derrow and why hasn't he done what I desire :P
__________________
Reclusive fart.
Collecting Military, Trains, Cooking, Woodworking, Fighting Illini, Auburn Tigers

Last edited by FredThompson; 23rd September 2009 at 22:25.
FredThompson is offline   Reply With Quote
Old 24th September 2009, 01:03   #10  |  Link
BetaBoy
CoreCodec Founder
 
BetaBoy's Avatar
 
Join Date: Oct 2001
Location: San Francisco
Posts: 1,421
Quote:
Originally Posted by neuron2 View Post
What is your reliable source for this claim?
He is talking out of his a**. We have and continue to be active with Matroska development leading towards 2.0. As I have stated here at D9 a few times we have completed EBML 2.0, which is a complete rewrite and is already included in CorePlayer 1.3.x and our upcoming 2.0 release. We are also working on LibMatroska 2.0 which will also have a new Matroska Parser Library utilizing Core-C (our modular ANSI C language), these are also a complete re-writes. However, we have sloooowed things down a bit for 2.0 to allow 1.x to gain adoption ( no need to mess with what works ), stopped devel? Hell no.
__________________
Dan "BetaBoy" Marlin
Ubiquitous Multimedia Technologies and Developer Tools

http://corecodec.com

Last edited by BetaBoy; 24th September 2009 at 01:52.
BetaBoy is offline   Reply With Quote
Old 24th September 2009, 01:33   #11  |  Link
BetaBoy
CoreCodec Founder
 
BetaBoy's Avatar
 
Join Date: Oct 2001
Location: San Francisco
Posts: 1,421
On Matroska menus... As we discussed a while back about them. The proposed menu system is 100% reliant on the means to display them (software and hardware) as the specs state. So while the elements can be defined, the means for playback still needs to know the how/where/what to do with them. So technically you could just stuff a current 'Cocktail' menu in Matroska as long as the means to handle all the 'web' elements are handled in the player.

On DVD Menu Extractor (or DMX as we call it), here it is in SVN. https://svn.matroska.org/svn/matrosk...dMenuXtractor/
__________________
Dan "BetaBoy" Marlin
Ubiquitous Multimedia Technologies and Developer Tools

http://corecodec.com

Last edited by BetaBoy; 24th September 2009 at 01:52.
BetaBoy is offline   Reply With Quote
Old 24th September 2009, 02:23   #12  |  Link
FredThompson
Registered User
 
FredThompson's Avatar
 
Join Date: Feb 2002
Location: Charlotte, NC USA
Posts: 1,972
Thanks for the link. I noticed it's gone from the closed thread in the corecodec forum.

Out of curiosity, is the mkv support in commercial devices such as WD TV a result of direct consultation with CoreCodec or did it just happen? IIRC, the main restriction with DVD and BD control structures is the copyright on the set of functions and resultant licensing fees to support...restriction of competition and an administrative staff...my opinion.

We're looking at a horse and cart situation, yes?

I did notice some discussion in the WD TV hobbyist areas of modified firmware, including discussion of menus in mkv but no realization of the concept. There's also a post in your forum about QuickTime-like menus as HTML with a browser for playback.

Maybe an IFO set with mkv component files instead of VOB might be an interim method for proof of concept but IFO includes absolute pointers based on sectors.
__________________
Reclusive fart.
Collecting Military, Trains, Cooking, Woodworking, Fighting Illini, Auburn Tigers
FredThompson is offline   Reply With Quote
Old 24th September 2009, 05:10   #13  |  Link
creamyhorror
Registered User
 
Join Date: Mar 2008
Posts: 118
Quote:
Originally Posted by FredThompson View Post
There is an episode of the History Channel show "Save Our History" (I think that's the title) which has a house in the background as the camera zooms out. The thin wood pieces which separate the window panes (can never remember what those are called) play havoc with deinterlacing.
If you're intending to backup many interlaced DVDs, you're not going to find x264 is as efficient as you hope. x264 is optimized for progressive encoding.

Quote:
I'm still looking for a good example. One of the posts I mentioned above has a dead link. My guess is such a menu is a simple branch at the beginning and was only to pick language and subtitle.
Well, if you consider a list of timeline-chapters to be a menu, then I guess you'd literally be right. But it's far from an actual GUI menu, which doesn't exist in any form yet.

Quote:
AutoMKV Chapters: http://forums.animesuki.com/showthread.php?t=85641
101 things you never knew you could do with Matroska: http://www.mod16.org/hurfdurf/?p=8 and http://forums.animesuki.com/showthread.php?t=59561
maybe the anime subculture will figure this out first...
Yes, that's all dealing with chapters and timelines/linking. MPC-HC, for example, presents them in a right-click menu. No specification for graphical display as yet, to say nothing of custom animated menus.

Quote:
I guess the easiest option would be to add DVD XML and pointers to MKV ignoring the pesky little detail of licensing. Where's derrow and why hasn't he done what I desire :P
It shouldn't be hard to create an open-source equivalent, but I'm far from a dev.
creamyhorror is offline   Reply With Quote
Old 24th September 2009, 07:20   #14  |  Link
FredThompson
Registered User
 
FredThompson's Avatar
 
Join Date: Feb 2002
Location: Charlotte, NC USA
Posts: 1,972
Hmm...if you edit a DVD menu to remove stuff like Disney's on-disc games and ignore alpha-blending and logic for play-once portions and such, what is a DVD menu, really? It's a liked list of video clips. Still images are just I-frames. IIRC, DVD Stripper completely fragmented a DVD source into all the individual components. Do that and, essentially, parse only the navigation links into MKV links. Then again, it's also almost 3 AM here so...
__________________
Reclusive fart.
Collecting Military, Trains, Cooking, Woodworking, Fighting Illini, Auburn Tigers
FredThompson is offline   Reply With Quote
Old 2nd August 2011, 05:05   #15  |  Link
lovelove
Registered User
 
Join Date: Mar 2010
Posts: 106
Quote:
Originally Posted by FredThompson View Post
I'm having trouble finding the forum link I read earlier today which supposedly linked to an MKV sample file with a menu. The link didn't work but the thread mentioned a simple menu which used the first 2 chapter points of the sample data. I'll keep searching.
@FredThompson: The posting you are searching for his here:
http://forum.doom9.org/showthread.ph...801#post768801
lovelove is offline   Reply With Quote
Old 2nd August 2011, 05:14   #16  |  Link
lovelove
Registered User
 
Join Date: Mar 2010
Posts: 106
Quote:
Originally Posted by BetaBoy View Post
On Matroska menus... As we discussed a while back about them. The proposed menu system is 100% reliant on the means to display them (software and hardware) as the specs state. So while the elements can be defined, the means for playback still needs to know the how/where/what to do with them. So technically you could just stuff a current 'Cocktail' menu in Matroska as long as the means to handle all the 'web' elements are handled in the player. [...]
maybe I got your posting wrong, but it sounds like it's the same old story:
applications do not support the standards
and the standard doesn't get finished due to lacking software support.

reminds me of lyrics3 and id3v2.4 specs (which had been out for ~10 years)
lovelove is offline   Reply With Quote
Old 2nd August 2011, 06:46   #17  |  Link
Chetwood
Registered User
 
Chetwood's Avatar
 
Join Date: Nov 2001
Posts: 1,103
Quote:
Originally Posted by FredThompson View Post
Look at things like the WD TV players. They'll work with mkv but won't mount a virtual disc.
I don't know where you got your info from but the original WD TV supported ISO files as does the entire line of their players. Problem is that internal VobSubs in MKVs are problematic, check the stickies on the official forums.

While H.264 yields smaller file size at a given quality (progressive or not) I still use xvid for SD material. MakeMKV keeps chapters but does not encode but I'm pretty sure Staxrip, Handbrake or MeGUI will also transfer chapters to MKV. Thus ripping the main movie with DVDFab and encoding to MKV should be easy enough.

I just don't know why you would need menus any more. The file browser of any player is good enough to select a certain ep of a TV show, why complicate things?
__________________

MultiMakeMKV: MakeMKV batch processing (Win)
MultiShrink
: DVD Shrink batch processing
Offizieller ‹bersetzer von DVD Shrink deutsch
Chetwood is offline   Reply With Quote
Old 2nd August 2011, 06:55   #18  |  Link
lovelove
Registered User
 
Join Date: Mar 2010
Posts: 106
some DVD have highly elaborated menus which are just part of the artwork
why don't you just accept the premise?
lovelove is offline   Reply With Quote
Old 2nd August 2011, 07:21   #19  |  Link
Chetwood
Registered User
 
Chetwood's Avatar
 
Join Date: Nov 2001
Posts: 1,103
Because there's no container that properly could handle them. Why dont you just accept that premise? Besides, players like WD TV display jpgs as thumbnails or even screen sized pictures which would allow for keeping some of this artwork. Feel free to complicate things as much as you like.
__________________

MultiMakeMKV: MakeMKV batch processing (Win)
MultiShrink
: DVD Shrink batch processing
Offizieller ‹bersetzer von DVD Shrink deutsch
Chetwood is offline   Reply With Quote
Old 2nd August 2011, 07:59   #20  |  Link
lovelove
Registered User
 
Join Date: Mar 2010
Posts: 106
There are people who can't distinguish between a still image and a full interactive menu with video, sound, links, multimedia design, etc.

Quote:
Originally Posted by Chetwood View Post
Because there's no container that properly could handle them. Why dont you just accept that premise?
... And there are also people who can't distinguidh between a premise and a conclusion.
And possibly a wrong one, that is.
Quote:
Originally Posted by http://www.matroska.org/technical/menu/index.html
Matroska Menu Specifications
[...]
What we'll try to have is a system that can do almost everything done on a DVD, or more, or better, or drop the unused features if necessary."

Last edited by lovelove; 2nd August 2011 at 08:02.
lovelove is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 17:22.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.