Secondary Audio And Video (PIP)

[DieHard]

I am looking to test out some async PIP (hopefully) but am having a bit of trouble in getting everything into Scenarist HDMV and then setting it up properly.

After many different tests I was finally able to load in a video with 29.97fps and 480i into the secondary video track. My problem now rests with the audio. I tried both an ac3 file and also an LPCM wav file. Both automatically loaded into the Primary Audio track and won't let me change them to the secondary from what I can see. Do I need an empty track in the primary first or do I need a specially formatted audio file to use in the secondary position?

[jokin]

Not sure I try to get PiP and PDVD opens it but rimary video does not play at all just black screen. Choose Secondary Vid stream and it freezes. If I remove 2nd vid it muxes and plays perfectly.

[2themax]

There are some very tight constraints on PIP A/V. The most notable are that the secondary video must have the same frame rate or multiple of the primary video's frame rate, i.e. 24p - 24p or 60p - 30i. The secondary audio has to be DTS Express or Dolby Digital Plus. That pretty much rules out non-professional authoring at this point.

[jokin]

My video streams had same framerate. It just froze powerdvd after it was muxed. No errors or warnings during muxing. did not have any audio for the test. both were 1920x1080p with framerate of 23.976.

[2themax]

The PIP currently can not be HD. The authoring tool supports it, but players don't as it is optional per spec. Try converting it to SD resolution and then remux.

[DieHard]

What I am going to attempt to do (although not 100% sure yet if it can be accomplished, although with everything I've read and heard it should be with Profile 1.1) is this:

1. Have a main video and audio stream playing.
2. Call a pop-up menu.
3. Select a PIP option.
4. Have the main video pause as a secondary video and audio stream start within a PIP window. There will be a 'Fullscreen' button under the PIP window.
5. If the button is pressed the video will go fullscreen with new buttons appearing 'PIP Window' and 'Resume'.
6. If the 'Resume' button is clicked or the end of the secondary video stream is reached, the main video reappears (or PIP window vanishes) and the main video continues playing from where it paused.

Now as far as I'm aware using the current tools (Scenarist HDMV or Blu-Print) and playing it on a Profile 1.1-capable player (PS3) this should all be possible. The only problem for me really is finding the time to attempt it all and most importantly fiddle with all the little things that are sure to frustrate me along the way of getting to the end result. Has anyone else attempted anything like this yet?

Secondary audio has to be DTS Express or Dolby Digital Plus? I read that it had be to LPCM wav only? I'm not sure I've run across any programs to convert aac/ac3 to those formats. I thought DD+ was just an enhanced ac3 format, although a normal ac3 audio file won't load as a secondary audio file. (Unless you need a primary audio track or something first.)

[2themax]

What I am going to attempt to do (although not 100% sure yet if it can be accomplished, although with everything I've read and heard it should be with Profile 1.1) is this:

1. Have a main video and audio stream playing.
2. Call a pop-up menu.
3. Select a PIP option.
4. Have the main video pause as a secondary video and audio stream start within a PIP window. There will be a 'Fullscreen' button under the PIP window.
5. If the button is pressed the video will go fullscreen with new buttons appearing 'PIP Window' and 'Resume'.
6. If the 'Resume' button is clicked or the end of the secondary video stream is reached, the main video reappears (or PIP window vanishes) and the main video continues playing from where it paused.

Now as far as I'm aware using the current tools (Scenarist HDMV or Blu-Print) and playing it on a Profile 1.1-capable player (PS3) this should all be possible. The only problem for me really is finding the time to attempt it all and most importantly fiddle with all the little things that are sure to frustrate me along the way of getting to the end result. Has anyone else attempted anything like this yet?

Secondary audio has to be DTS Express or Dolby Digital Plus? I read that it had be to LPCM wav only? I'm not sure I've run across any programs to convert aac/ac3 to those formats. I thought DD+ was just an enhanced ac3 format, although a normal ac3 audio file won't load as a secondary audio file. (Unless you need a primary audio track or something first.)
I've done synchronous PIP, your attempting asynchronous, but should be possible. I think the concept is to have asynchronous PIP playing from the local storage or the network and not from the disc.

DTS Express and Dolby Digital Plus are the only 2 supported secondary audio codecs. I checked this morning in the spec to make sure it did not change and it didn't. LPCM is only available as primary or button audio.

[DieHard]

Has anyone gotten async PIP to work at all?

At the moment I have a button in a pop-up menu that has this set:

SetSecondaryStream SecondaryVideo:No=1 [DispON] SecondaryAudio:[DispOFF] PiP_PG_textST:[DispOFF] [PiP PG TextST Valid OFF]

The video it is supposed to reference was added to a clip set up as application type 'One or more elementary streams path' and that clip was added as a SubPath to the main video PlayList as '[Out of Mux] Asynchronous Type of PiP'. However, when I click on the button nothing happens.

At the moment the secondary video is 720p and from what I understand it must be SD resolution. So my question is, if I re-encode it down to 480p with everything set up the same way, should it work or is there anything else I'm still missing?

[jokin]

I still can not get a primary video stream to play if a secondary video stream is in the same clip. Just freezes powerdvd. My secondary video stream is SD resolution.

[DieHard]

I believe they have to be in seperate clips just like IGs, etc. Try creating a clip that is 'One or more elementary streams path' (as I think that was the only type I could load into a SubPath as a seconary video) and adding that as an Out of Mux SubPath and see what happens. I can only assume that is what needs to be done for PiP. Maybe I should try it in the same clip and see what happens.

[2themax]

I still can not get a primary video stream to play if a secondary video stream is in the same clip. Just freezes powerdvd. My secondary video stream is SD resolution.
Make sure your using the latest PowerDVD. And if not, you'll need to add a pre-command that checks player profile and then plays the appropriate playlist that either has or doesn't have the PIP stream. Older versions of PowerDVD will freeze if they try to play a playlist with PIP.

[2themax]

I believe they have to be in seperate clips just like IGs, etc. Try creating a clip that is 'One or more elementary streams path' (as I think that was the only type I could load into a SubPath as a seconary video) and adding that as an Out of Mux SubPath and see what happens. I can only assume that is what needs to be done for PiP. Maybe I should try it in the same clip and see what happens.
If the PIP is synchronous, it needs to be placed in the same clip as the primary A/V.

[DieHard]

What is the '[In Mux] Synchronous Type of PiP' SubPath for then since if you have the secondary video in the same clip as the main video? For async PiP (in a seperate clip added as a subpath) do you know if you still use that same SetSecondaryStream command or do you have to do something else?

[2themax]

What is the '[In Mux] Synchronous Type of PiP' SubPath for then since if you have the secondary video in the same clip as the main video? For async PiP (in a seperate clip added as a subpath) do you know if you still use that same SetSecondaryStream command or do you have to do something else?
Synchronous PIP is what you're used to seeing on titles so far. Here's how it works in Scenarist:

- Create a new Clip
- Add the primary and secondary A/V to the that Clip
- Create a new Playlist
- Create a PlayItem and an In Mux SubPath within that Playlist
- Add the Clip to the Playitem using only the primary A/V
- Add the same Clip to the In Mux SubPath using only the secondary A/V
- Change the Audio Mix Flag for the Playlist
- Set the SetSecondaryStream command as needed
- Right click on the secondary video and select Link PIP
- Choose the corresponding secondary audio
- Right click on the secondary video again and select PIP Editor
- Configure your PIP placement, size, etc.

[DieHard]

Hmmm. I tried that using a button to set the secondary video and nothing happened, although the secondary video is 720p. Shouldn't it still do something, just not show/play properly?

Also, is it possible to have an avc secondary video stream if it has to be 480i/480p to work? I tried converting the 720p avc stream down to 480p but Scenarist won't let me add it to the project. I get one of those 'Error Content: Value of VideoFormat is not supported in the BD standard.' The problem is that if I convert it to mpeg2 instead it won't allow me to use it as a secondary stream for an avc primary stream due to the different video formats.

So the real question is, is it possible to create a PiP setup without both primary and secondary streams being anything other than same framerate mpeg2 streams?

[2themax]

Hmmm. I tried that using a button to set the secondary video and nothing happened, although the secondary video is 720p. Shouldn't it still do something, just not show/play properly?

With the 720p video, you won't see anything happen.

Also, is it possible to have an avc secondary video stream if it has to be 480i/480p to work? I tried converting the 720p avc stream down to 480p but Scenarist won't let me add it to the project. I get one of those 'Error Content: Value of VideoFormat is not supported in the BD standard.' The problem is that if I convert it to mpeg2 instead it won't allow me to use it as a secondary stream for an avc primary stream due to the different video formats.

The PIP video has certain encoding constraints depending on codec. Something in the encoder isn't set to BD PIP spec.

So the real question is, is it possible to create a PiP setup without both primary and secondary streams being anything other than same framerate mpeg2 streams?
Yes. I've done so with both VC-1 and AVC. There's also certain restrictions on the combination of codecs. For instance, a VC-1 primary video stream has to have a VC-1 secondary video stream. Same for AVC. But, a MPEG2 primary video stream can have any of the three as a secondary video stream.

[DieHard]

Any info on an encoder that should work for AVC PIP or what needs to be set properly? I've tried using the MainConcept Reference Blu-Ray (480i/p) template and it gives me one error: 'Error Content: Value of FrameRateCode is not supported in the BD standard.' when I used 23.976 fps. I tried using the AVC DVD template and got the error: 'Error Content: Value of Gaps_in_frame_num_value_allowed_flag is not supported in the BD standard.' And when I tried using x264 or meGUI (can't remember which now) I believe I got the error: 'Error Content: Value of VideoFormat is not supported in the BD standard.'

It seems every different thing I try I end up with a different error result but an error nonetheless. Any info on a command-line I could use for x264 or meGUI or really any template/program would help since I'm not really sure of all the flags and whatnot.

All I want is to try to get a simple AVC PIP working with a framerate of 23.976 since all my HD footage is 23.976 AVC, which is what I'll be using for my primary streams.

[2themax]

Unfortunately, there isn't anything in the free market to do so yet. I use CineVision and CineVision PSE at work, both of which are obviously made for this stuff. The MainConcept encoder has a bug in it somewhere. I've gotten the same error when trying it with 23.976p content. However, it will work with 29.97i content.

A free encoder that you might want to give a try is the VC-1 SDK from Microsoft. I know, it's not AVC. There should be enough control via command line to get the right output. I may try this myself and I'll let you know if it works.

[DieHard]

CineVision I have found absolutely horrible. It doesn't ever want to accept anything as input (for some reason can't scan and read all input properties of ANY input file properly and most things it says 'It is not possible to encode to AVC for Blu-Ray Disc with this file selection. Choose a different file and try again') and anything it does accept which is very little it seems to chop up the output. I am running it on Vista and beginning to wonder if that is the reason why. I'm also running version 1.2 since the newer 2.0 just straight out crashes.

The very few files it does accept all seem to be mkv (which isn't even listed as an accepted filetype in its list) and it then forces specific output. If the input file is 1280x720p it forces a change to that resolution.

I would ask if it was just a Vista problem but the fact that it actually does accept and work with something makes me wonder if it is something else.

Edit: I tried CineVision 2.0 on WinXP Home and it seemed to do the same thing.

[DieHard]

I did manage to get one PIP working with a 29.97fps 1080i MPEG2 primary stream and a 29.97fps 480i MPEG2 secondary stream.

This means that hypothetically if I were to convert all my content from 23.976fps AVC to 29.97fps MPEG2 I should be able to use PIP with it. Herein lies another problem however. I know that to convert 23.976fps to 29.97fps you need to use pulldown but when I tried encoding it with MainConcept Reference and loading it into Scenarist HDMV it gave me the error: 'Error Content: Value of ProgressiveSequence is not supported in the BD standard.' which I assume has something to do with the pulldown flag? The question is, how can I get it compliant now?

Edit: I tried re-encoding a pulldowned MPEG2 stream to get rid of the pulldown flag and I'm still getting that error so it appears not to be that but I still can't figure out just what it is refering to then. I tried encoding both progressive and interlaced and it made no difference.

[DieHard]

Update: I have actually managed to make my first Async PIP now although I have run into more issues and odd behaviour.

First off, unlike what everyone has been saying, async PIP actually doesn't pause the primary video from playing. The play simultaneously just like with sync PIP, the only difference being that with async you can watch the PIP at any point during watching the primary stream. My question in regards to this is: does anyone know how to pause the primary video stream? It would be good to have both options available.

Next, I added a button to make it fullscreen and have it set as: SetSecondaryStream SecondaryVideo:No=1 [DispON] [FullScreen] and it increases the size but about half the screen is still black. I'm assuming it has something to do with the 480i video trying to fit into a 1080i space, but not totally sure.

Also, I still am unable to get working Secondary Audio. I know it has to be EAC3 (DD+) or DTS-HD but as of yet I have been unable to find any 'working' program to convert normal ac3 or any other format to DD+. Any suggestions would be nice, (other than Dolby Media Encoder since that is pretty much out of the question due to price/etc right now.)

[johnsonlee]

Is there anyone who can provide the sample image for verification of the PIP function? It will be appreciated very much.

[speedyrazor]

Hi, Im reading this thread with great interest, trying the same thing myself, all working accept the secondary audio. Just to let you all know that theres a new forum dedicated to Blu-Ray Authoring over here www.bdjforum.com, it has a great HDMV section, more and more tips growing daily.

Regards.