Is there a way to extract multiple PIP streams (Blu-ray)?

Lincoln Burrows · 30th November 2010, 01:35

The first issue I had with Blu-rays was described in the MakeMKV thread regarding slideshows:

http://www.makemkv.com/forum2/viewtopic.php?f=1&t=2111

It seems a simple convertion won't give anyone MKV files that resemble the original authoring. Now I am facing another problem and I need to know if this can be done as well, because I am not seeing how. And it's even worse than slideshows (or multiple video angles, from Star Trek: TOS).

The title I have tested is "The Sound of Music" (1965) from Fox Film, in Blu-ray (first disc).

It's possible to convert (into Matroska) anything on this disc, except for this:

http://www.blu-ray.com/movies/The-So.../13695/#Review

Quote:

Your Favorite Things - An Interactive Celebration

Like most of Fox's tent pole Blu-ray productions, The Sound of Music comes with an all-new interactive bonus features interface that can be accessed while watching the film.

When you select "Your Favorite Things" from the extras tab, a brief video tutorial plays and you're taken to a menu where you can customize the experience. There are four options here, and you can select as many or as few of them as you'd like.

Making Music: A Journey in Images is a picture-in-picture mode that displays many never-before-seen storyboards and photographs of the film's production in the upper right corner of the screen, The Sing-Along Experience provides karaoke-style lyrics across the bottom of the screen, Many a Thing to Know is a trivia track about the making of the film and the real Maria, and Where Was it Filmed? is an ongoing multiple-choice quiz.

With all four options turned on, I find that the interface obscures too much of the film, but I can see many fans taking advantage of the karaoke mode.

As you can see, if we select one of those options in the Blu-ray menu, while we are watching the movie, a window appears either showing another video or (and I am not talking about movie subtitles) with movie curiosities written in english (or portuguese, it all depends of the language Total Media Theatre player from Arcsoft is using).

I am not seeing how I am going to extract this PIP content. Is it possible to do that?

Have you guys already saw any Bu-ray with this kind of content?

About this thread:
http://forum.doom9.org/showthread.php?t=157956

I have successfully extracted one PIP stream using TSMuxer (720x480, using 3.2 Profile). However, it was only one of them, I am not seeing the curiosity PIP track anywhere. Neither "Where was it filmed", which is confirmed to be another PIP track!

This is what I am talking about:

P.S. The m2ts from the movie is a unique file, and it has 42.1 GB (00360.m2ts).

mrr19121970 · 30th November 2010, 10:58

Quote:

Originally Posted by Lincoln Burrows

As you can see, if we select one of those options in the Blu-ray menu, while we are watching the movie, a window appears either showing another video or (and I am not talking about movie subtitles) with movie curiosities written in english (or portuguese, it all depends of the language Total Media Theatre player from Arcsoft is using).

I am not seeing how I am going to extract this PIP content. Is it possible to do that?

The PIP does not have to be in the same .m2ts as the main feature. Please use BDInfo to see if there is a subpath branching off to a completely different .m2ts.

You can also see this in Clown_BD BD Copier too.

mrr19121970 · 30th November 2010, 18:09

The 'Many Things To Know' and 'Where Was It Filmed' are in this case not PIP videos. They are presentation graphics streams (PGS), ie subtitles stored as PNG images with timing information. Use BDSup2Sub to confirm which ones after demux all.

Lincoln Burrows · 30th November 2010, 19:44

mrr19121970, yes, they are. I already checked all audio/PGS streams. This movie has the following, according to ClownBD:

Quote:

PID = 4113
Stream Type = Video MPEG4 AVC
Language =
Other Details = 1080p 24/1.001 (16:9)

PID = 4352
Stream Type = Audio DTS-HD Master
Language = English
Other Details = 48kbps Multi_Channel

PID = 4353
Stream Type = Audio Dolby Digital
Language = English
Other Details = 48kbps Multi_Channel

PID = 4354
Stream Type = Audio Dolby Digital
Language = Portuguese
Other Details = 48kbps Multi_Channel

PID = 4355
Stream Type = Audio DTS
Language = French
Other Details = 48kbps Multi_Channel

PID = 4356
Stream Type = Audio Dolby Digital
Language = Spanish
Other Details = 48kbps Multi_Channel

PID = 4357
Stream Type = Audio Dolby Digital
Language = English
Other Details = 48kbps Stereo

PID = 4358
Stream Type = Audio Dolby Digital
Language = English
Other Details = 48kbps Stereo

PID = 4608
Stream Type = Subtitles PGS
Language = English
Other Details =

PID = 4609
Stream Type = Subtitles PGS
Language = French
Other Details =

PID = 4610
Stream Type = Subtitles PGS
Language = Portuguese
Other Details =

PID = 4611
Stream Type = Subtitles PGS
Language = Danish
Other Details =

PID = 4612
Stream Type = Subtitles PGS
Language = Finnish
Other Details =

PID = 4613
Stream Type = Subtitles PGS
Language = Spanish
Other Details =

PID = 4614
Stream Type = Subtitles PGS
Language = Dutch
Other Details =

PID = 4615
Stream Type = Subtitles PGS
Language = Norwegian
Other Details =

PID = 4616
Stream Type = Subtitles PGS
Language = Swedish
Other Details =

PID = 4617
Stream Type = Subtitles PGS
Language = English
Other Details =

PID = 4618
Stream Type = Subtitles PGS
Language = French
Other Details =

PID = 4619
Stream Type = Subtitles PGS
Language = Danish
Other Details =

PID = 4620
Stream Type = Subtitles PGS
Language = Finnish
Other Details =

PID = 4621
Stream Type = Subtitles PGS
Language = Dutch
Other Details =

PID = 4622
Stream Type = Subtitles PGS
Language = Norwegian
Other Details =

PID = 4623
Stream Type = Subtitles PGS
Language = Swedish
Other Details =

PID = 4624
Stream Type = Subtitles PGS
Language = English
Other Details =

PID = 4625
Stream Type = Subtitles PGS
Language = French
Other Details =

PID = 4626
Stream Type = Subtitles PGS
Language = Danish
Other Details =

PID = 4627
Stream Type = Subtitles PGS
Language = Finnish
Other Details =

PID = 4628
Stream Type = Subtitles PGS
Language = Dutch
Other Details =

PID = 4629
Stream Type = Subtitles PGS
Language = Norwegian
Other Details =

PID = 4630
Stream Type = Subtitles PGS
Language = Swedish
Other Details =

PID = 4631
Stream Type = Subtitles PGS
Language = English
Other Details =

PID = 4632
Stream Type = Subtitles PGS
Language = English
Other Details =

PID = 4633
Stream Type = Subtitles PGS
Language = French
Other Details =

PID = 4634
Stream Type = Subtitles PGS
Language = Spanish
Other Details =

PID = 4635
Stream Type = Subtitles PGS
Language = Dutch
Other Details =

PID = 4636
Stream Type = Subtitles PGS
Language = English
Other Details =

PID = 4637
Stream Type = Subtitles PGS
Language = Dutch
Other Details =

tsMuxeR cannot handle secondary video (actually it can, but without audio)
You cannot select this

PID = 6912
Stream Type = Video MPEG4 AVC
Language =
Other Details = 480p 24/1.001 (16:9)

The ones that I was looking for:

English CC (movie) - confirmed
Portuguese (movie) - confirmed
English (1st audio commentary)
English (2nd audio commentary)
English (Karaoke stream, Sing-Along experience)
English CC (movie, but without symbols in the lyrics, and with a different font)
English (Karaoke stream, Sing-Along experience)

Let me remind you the PIP stream called "Many a thing to know" is presented in different languages. I recognized at least two while verifying in TMT from Arcsoft: english and portuguese.

See the picture below with the english option selected on Total Media Theatre player (menus and etc):

If I had selected "Portuguese" here:

The picture above would have showed me the "About 500 productions..." written in portuguese. So it's not a PGS/subtitle stream.

As I explained to you, there are no other m2ts files, they were all checked and confirmed to be something else (FOX logo, piracy warning, that sort of stuff).

See? No luck. I tried with BDInfo as well.

Those programs are simply ignoring the remaining PIP streams:

"Many a thing to know"
"Where was it filmed?"

They are not hidden somewhere that I can't find it. They are not being recognized by any software. And my guess is they are all inside the movie's m2ts file with 42,1 GB, but I can only see them with the full disc backup.

mrr19121970 · 30th November 2010, 20:07

OK, I had a closer look. You'll find them in the BDMV/JAR folder. Checkout 01001/SOM_WW_D1_ENG_Composite*.png

Also checkout 77*.JAR

I don't think you're going to get these out. You can proove this by opening the disk in Clone_BD BD Copier, deselect all, select menus and create an ISO. All your favourite things...

rik1138 · 30th November 2010, 21:59

Yeah, it's not a presentation graphic stream, it's a Java application that just displays a PNG graphic and certain time intervals throughout the film. If it has to be regionalized (translated) that would amount to WAY too much video and PIP streams... It's most likely that the blue background is one PNG, and the text itself is either another PNG, or just dynamic text created using font strips. You aren't going to be able to 'extract' those streams and keep them...

Lincoln Burrows · 30th November 2010, 21:59

The ISO is created with all menus but you will still need the m2ts from the movie to actually see this content. The whole idea was to extract both of these contents into single Matroska files...

So there's no way to extract those additional PIP streams, then. The first one was extracted but only because TSMuxer recognized as a second video angle and there is a warning (? - see post above) saying it does not extract with audio.

Quote:

Originally Posted by rik1138

It's most likely that the blue background is one PNG, and the text itself is either another PNG, or just dynamic text created using font strips. You aren't going to be able to 'extract' those streams and keep them...

I think that might be the case. This is the list of files, except for the BDMV/STREAM folder, if you wanna check:

http://www.easy-share.com/1913149484/files.7z

rik1138 · 30th November 2010, 22:02

I don't think TSMuxer can recognize DTS secondary audio, that's the problem you having there... I'm not sure what alternative there is for that?

Lincoln Burrows · 1st December 2010, 01:28

In this case there is no need for audio, the first PIP stream only features a video with static images and the audio is from the movie itself, not unique from that PIP stream. But I wonder if there was any audio if it could be extracted by TSMuxer.

I don't know about the remaining PIP streams but it's sad that we still have relevant contents from Blu-rays that can't be converted into MKV properly. I hope the developers can add that functionality one day (specially MakeMKV, that can't even extract the first PIP stream).

rik1138 · 1st December 2010, 03:48

Well, the other content is not a PIP stream... It's a program that displays graphics in real-time as you are watching the film... It's like trying to make a video if you surfing the internet or something... There's probably an infinite number of ways to write the java code, and the 'converter' would have to know to search for all of them to recognize that there's graphic elements being displayed at certain times...

You could take all the individual PNGs for the mode in question, re-apply them to the feature video as a subtitle stream and use that, but that would be a lot of work...

The easiest way to do it would be to play the movie with that mode on and record it...

Lincoln Burrows · 1st December 2010, 19:07

What PNG's are you talking about? I am only seeing PNGs from menus in those folders!

mrr19121970 · 1st December 2010, 20:08

There are no .PNGs. The information is stored in a .DAT file in the 77*.JARs.

rik1138 · 1st December 2010, 21:53

No, the PNGs are there. Look in BDMV\JAR\01002.
Look at:
YFT_APP_ENG_Composite1.png
(Assuming the US disc, that is)
You will notice that just beneath the image with the picture is the blue square with the little green 'book' icon in it that Lincoln circled in his first post. That's the PNG that's being displayed (it's a mosaic, so the java code is only displaying a tiny chunk of the whole PNG file). The text itself is dynamically created on top of that graphic. The font for the text is the .PNG file called:
som_YFT_GoudyOS_30pt_Bold_grey.png
They don't have an individual PNG for every text box that appears, just the one (or maybe 2 or 3) background images, and text is generated on top of that.

30th November 2010, 18:09	#3 \| Link
mrr19121970 Registered User Join Date: Mar 2008 Posts: 305	The 'Many Things To Know' and 'Where Was It Filmed' are in this case not PIP videos. They are presentation graphics streams (PGS), ie subtitles stored as PNG images with timing information. Use BDSup2Sub to confirm which ones after demux all. __________________ Blu-Ray Movie Only Copies : Clown_BD Blu-Ray Movie Only DVD Conversion : Clown_BD Blu-Ray Copies With Menus : Clown_BD BD Copier

30th November 2010, 20:07	#5 \| Link
mrr19121970 Registered User Join Date: Mar 2008 Posts: 305	OK, I had a closer look. You'll find them in the BDMV/JAR folder. Checkout 01001/SOM_WW_D1_ENG_Composite.png Also checkout 77.JAR I don't think you're going to get these out. You can proove this by opening the disk in Clone_BD BD Copier, deselect all, select menus and create an ISO. All your favourite things... __________________ Blu-Ray Movie Only Copies : Clown_BD Blu-Ray Movie Only DVD Conversion : Clown_BD Blu-Ray Copies With Menus : Clown_BD BD Copier Last edited by mrr19121970; 30th November 2010 at 20:18.

1st December 2010, 01:28	#9 \| Link
Lincoln Burrows Registered User Join Date: Mar 2008 Posts: 280	In this case there is no need for audio, the first PIP stream only features a video with static images and the audio is from the movie itself, not unique from that PIP stream. But I wonder if there was any audio if it could be extracted by TSMuxer. I don't know about the remaining PIP streams but it's sad that we still have relevant contents from Blu-rays that can't be converted into MKV properly. I hope the developers can add that functionality one day (specially MakeMKV, that can't even extract the first PIP stream). __________________ "Whoever has not known himself has known nothing. But whoever has known himself has simultaneously achieved knowledge about the depth of all things."

1st December 2010, 19:07	#11 \| Link
Lincoln Burrows Registered User Join Date: Mar 2008 Posts: 280	What PNG's are you talking about? I am only seeing PNGs from menus in those folders! __________________ "Whoever has not known himself has known nothing. But whoever has known himself has simultaneously achieved knowledge about the depth of all things."

1st December 2010, 20:08	#12 \| Link
mrr19121970 Registered User Join Date: Mar 2008 Posts: 305	There are no .PNGs. The information is stored in a .DAT file in the 77*.JARs. __________________ Blu-Ray Movie Only Copies : Clown_BD Blu-Ray Movie Only DVD Conversion : Clown_BD Blu-Ray Copies With Menus : Clown_BD BD Copier

30th November 2010, 21:59	#6 \| Link
rik1138 Registered User Join Date: Mar 2008 Location: LA Posts: 620	Yeah, it's not a presentation graphic stream, it's a Java application that just displays a PNG graphic and certain time intervals throughout the film. If it has to be regionalized (translated) that would amount to WAY too much video and PIP streams... It's most likely that the blue background is one PNG, and the text itself is either another PNG, or just dynamic text created using font strips. You aren't going to be able to 'extract' those streams and keep them...

30th November 2010, 22:02	#8 \| Link
rik1138 Registered User Join Date: Mar 2008 Location: LA Posts: 620	I don't think TSMuxer can recognize DTS secondary audio, that's the problem you having there... I'm not sure what alternative there is for that?

1st December 2010, 03:48	#10 \| Link
rik1138 Registered User Join Date: Mar 2008 Location: LA Posts: 620	Well, the other content is not a PIP stream... It's a program that displays graphics in real-time as you are watching the film... It's like trying to make a video if you surfing the internet or something... There's probably an infinite number of ways to write the java code, and the 'converter' would have to know to search for all of them to recognize that there's graphic elements being displayed at certain times... You could take all the individual PNGs for the mode in question, re-apply them to the feature video as a subtitle stream and use that, but that would be a lot of work... The easiest way to do it would be to play the movie with that mode on and record it...

1st December 2010, 21:53	#13 \| Link
rik1138 Registered User Join Date: Mar 2008 Location: LA Posts: 620	No, the PNGs are there. Look in BDMV\JAR\01002. Look at: YFT_APP_ENG_Composite1.png (Assuming the US disc, that is) You will notice that just beneath the image with the picture is the blue square with the little green 'book' icon in it that Lincoln circled in his first post. That's the PNG that's being displayed (it's a mosaic, so the java code is only displaying a tiny chunk of the whole PNG file). The text itself is dynamically created on top of that graphic. The font for the text is the .PNG file called: som_YFT_GoudyOS_30pt_Bold_grey.png They don't have an individual PNG for every text box that appears, just the one (or maybe 2 or 3) background images, and text is generated on top of that.