Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > (HD) DVD, Blu-ray & (S)VCD > (HD) DVD & Blu-ray authoring

Reply
 
Thread Tools Search this Thread Display Modes
Old 18th February 2009, 04:37   #1  |  Link
quantum
Registered User
 
Join Date: Nov 2002
Location: USA
Posts: 528
tsMuxeR Blu-ray output and Fixclpi

[ EDIT - This issue has been resolved in recent builds of tsMuxeR. No further patching is required ]

[ EDIT - jdobbs has posted an updated fixclpi that seems to solve all issues, including playback on Panasonic standalones. Get the new version here:
http://forum.doom9.org/showpost.php?...&postcount=106 ]

For those interested in using tsMuxeR to mux Blu-ray, jdobbs identified an issue with the .clpi files being generated by tsMuxer. He indicated the issue manifested itself with problems in fast forward and rewind, and chapter skipping.

This is his first post on the matter:
http://forum.doom9.org/showpost.php?...postcount=2159

He then posted a program designed to fix the issue:
http://forum.doom9.org/showpost.php?...postcount=2188

First let me say I'm grateful to jdobbs for pointing out the issue, and for creating and posting his utility.

The point of this post is to try to learn more about the issue, and see how people are being affected by using fixclpi.

My own experience: I have a Pioneer BDP-51FD standalone. I demux from original to elementary streams with eac3to and then mux to Blu-ray with tsMuxer and burn to BD-R 25GB media. Before using fixclpi, I had no problems with ff/rew, however I did notice that chapter jumps were sometimes slow, sometimes taking a few seconds. Chapter jumps were accurate, even if slow.

After using fixclpi, the chapter jumps were indeed faster. However, I then noticed artifacts when using ff/rew which previously were not an issue. The screen was not being correctly refreshed while using ff/rew. It looked like various portions of previous screens were being left behind. Or, to put it another way, only a portion of each new screen, near the top, was being drawn before the next screen was starting to be drawn. This could be described as a 'tearing' effect.

I tested the original store bought BD-Roms and they have no issues with ff/rew. Other burned BD-R's not treated with fixclpi have no issues with ff/rew. In these cases, which I consider normal, using ff/rew shows frames in rapid succession, in a slide show type effect. I don't have any theories about what is happening, or how fixclpi is involved or not involved. I'm just posting my observations.

[EDIT: I tested PowerDVD 7.3 and FF/REW ( f and b keys ) is also affected by fixclpi. It's easily noticeable at 2x and above, where frames are no longer drawn. It's easy to test this by swapping the clpi files back and forth. Note that only FF seems to display frames in PowerDVD, regardless of fixclpi. REW doesn't display frames for me in either case.]

I guess I could live with the fixclpi effect, but I'm concerned that the changes made by fixclpi may not be having the intended effect, which is, I hope, to make the clpi files totally compliant.

I'm equally concerned with leaving the clpi files untreated. I am seeing slow chapter seeking, so I'm concerned that the tsMuxer clpi files may in fact be non compliant as jdobbs reported. While they work fine now in my BDP-51FD (and PowerDVD), I'm wondering about my future players and how such disks will be tolerated.

What I'm asking for is any and all comments about experiences using tsMuxer and fixclpi, and the effect on standalone players, especially with ff/rew and chapter seeking. Maybe if enough detail is provided, jdobbs can spot something that might be helpful.

I'd also appreciate any comments or links to specifications about the clpi files. As a programmer myself, I'd like to examine this in detail (unless jdobbs comes to the rescue, in which case I'll gladly step aside).

I very much want to use tsMuxer as it does a great job in muxing basic Blu-rays. I also want to be confident that the output is as compliant as it can be.

Last edited by quantum; 3rd June 2009 at 22:46.
quantum is offline   Reply With Quote
Old 18th February 2009, 16:24   #2  |  Link
shon3i
BluRay Maniac
 
shon3i's Avatar
 
Join Date: Dec 2005
Posts: 2,419
Did you recompress video? and how?
shon3i is offline   Reply With Quote
Old 18th February 2009, 16:49   #3  |  Link
quantum
Registered User
 
Join Date: Nov 2002
Location: USA
Posts: 528
Quote:
Originally Posted by shon3i View Post
Did you recompress video? and how?
I'm demuxing to elementary streams with eac3to and muxing with tsMuxer. I have not reencoded the video.

My recent tests were with VC1.

Last edited by quantum; 18th February 2009 at 16:52.
quantum is offline   Reply With Quote
Old 18th February 2009, 16:56   #4  |  Link
laserfan
Aging Video Hobbyist
 
Join Date: Dec 2004
Location: Off the Map
Posts: 2,461
I can only say quantum that in my experience, failure to Fixclpi on tsMuxeR output results in PowerDVD missing "some" chapter marks, and "some" rather badly at that (i.e. not by just a frame or two, but way off). Not all, but some. So I don't know what Fixclpi does exactly, but it clearly fixes some serious glitch in tsMuxeR output and so I use it religiously.

But I have also observed that PowerDVD does not like the contSPS option in tsMuxeR in that if I leave that option OFF, then PDVD hits the chapter marks every time, whereas with contSPS enabled (checked in the GUI or added to the .meta file as most GUIs do) then again PDVD misses some chapter marks i.e. hits them early by a frame. I suspect contSPS to mess with timecodes somehow.

I don't use FF/RW when watching films (at all) so can't comment on that behavior. So this is all FWIW.
laserfan is offline   Reply With Quote
Old 18th February 2009, 17:35   #5  |  Link
quantum
Registered User
 
Join Date: Nov 2002
Location: USA
Posts: 528
Shon3i's post brings to mind that I'll post a list of mux details, with video (vc1 or h264 and wether reencoded) and if h264, the status of "add picture timing info" and "continually insert SPS/PPS" check boxes.
quantum is offline   Reply With Quote
Old 18th February 2009, 18:06   #6  |  Link
quantum
Registered User
 
Join Date: Nov 2002
Location: USA
Posts: 528
Quote:
Originally Posted by laserfan View Post
..But I have also observed that PowerDVD does not like the contSPS option in tsMuxeR in that if I leave that option OFF, then PDVD hits the chapter marks every time, whereas with contSPS enabled (checked in the GUI or added to the .meta file as most GUIs do) then again PDVD misses some chapter marks i.e. hits them early by a frame.
So can't you just leave "add picture timing info" and "continually insert SPS/PPS" unchecked and not run fixclpi?
quantum is offline   Reply With Quote
Old 18th February 2009, 18:32   #7  |  Link
laserfan
Aging Video Hobbyist
 
Join Date: Dec 2004
Location: Off the Map
Posts: 2,461
Quote:
Originally Posted by quantum View Post
So can't you just leave "add picture timing info" and "continually insert SPS/PPS" unchecked and not run fixclpi?
No, and maybe my contSPS comment is a distraction (i.e. a completely separate issue).
laserfan is offline   Reply With Quote
Old 19th February 2009, 01:27   #8  |  Link
quantum
Registered User
 
Join Date: Nov 2002
Location: USA
Posts: 528
I just finished 4 test muxes using a BD-RE. The movie was AVP Requiem. Video is H264 as demuxed with eac3to and no reencode, audio is DTS, no subtitles, with chapter stops.

+ indicates option active, - indicates option not used

1) -Fixclpi, -add picture timing, -continually insert SPS/PPS = good FF/REW, slower chapter seek
2) -Fixclpi, +add picture timing, +continually insert SPS/PPS = good FF/REW, slower chapter seek
3) +Fixclpi, +add picture timing, +continually insert SPS/PPS = bad FF/REW, good chapter seek
4) +Fixclpi, -add picture timing, -continually insert SPS/PPS = bad FF/REW, good chapter seek
5) +Fixclpi, +add picture timing, -continually insert SPS/PPS = bad FF/REW, good chapter seek

I didn't get this thorough with my last VC1 test, but I think the results are consistent with H264 and VC1. I don't recall having an instance were fixclpi was applied and ff/rew worked without obvious video artifacts.

Last edited by quantum; 19th February 2009 at 03:53.
quantum is offline   Reply With Quote
Old 19th February 2009, 03:12   #9  |  Link
laserfan
Aging Video Hobbyist
 
Join Date: Dec 2004
Location: Off the Map
Posts: 2,461
I don't see +Fixclpi, +add picture timing, -continually insert SPS/PPS
laserfan is offline   Reply With Quote
Old 19th February 2009, 03:53   #10  |  Link
quantum
Registered User
 
Join Date: Nov 2002
Location: USA
Posts: 528
Quote:
Originally Posted by laserfan View Post
I don't see +Fixclpi, +add picture timing, -continually insert SPS/PPS
Okay, in the interest of completeness, I just burned that and updated the summary post. No difference.

However, it occurs to me, that no matter how I set those two settings, I'm not seeing any message in tsMuxer GUI that anything is changing. I see something like this for an x264 encode:

H264 bitstream changed: insert pict timing and buffering period SEI units

But nothing for the H264 pulled straight off the disk. I assume this means the H264 as encoded from the studio has all the necessary timing and SPS/PPS, and tsMuxer does not attempt to re-add or change them.

So I don't think changing "add picture timing" or "continually insert SPS/PPS" settings makes any difference for this case, and these settings are not available at all for VC1.
quantum is offline   Reply With Quote
Old 19th February 2009, 04:30   #11  |  Link
laserfan
Aging Video Hobbyist
 
Join Date: Dec 2004
Location: Off the Map
Posts: 2,461
Quote:
Originally Posted by quantum View Post
Okay, in the interest of completeness, I just burned that...
Ok, but the only one you missed was the one I commented on!

Quote:
I'm not seeing any message in tsMuxer GUI that anything is changing.
And I'm not sure you would.
laserfan is offline   Reply With Quote
Old 19th February 2009, 13:17   #12  |  Link
colinhunt
Registered User
 
Join Date: Dec 2002
Posts: 1,022
I was fooling around with a project last night and bumped into something that relates to this issue.

Starting point:
A Blu-ray with slightly more than 26 gigabytes of data
- main movie as 00000.m2ts

Goal:
- make 00000.m2ts small enough to fit all data on a single-layer BD-RE while keeping original menus

Idea:
step 1) Strip one of the two audiotracks and cut ending credits out, then use tsMuxeR 1.8.18(b) to create a new 00000.m2ts
step 2) Replace original 00000.m2ts with the tsMuxeR-created one in original BDMV/STREAM directory
step 3) Burn data on a BD-RE and see what happens

Process:
- Dropped the original 00000.m2ts into tsMuxeR
- unticked LPCM track
- left Add picture timing info and Continually insert PS/PPS at defaults, i.e. ticked
- Selected No chapters in the Blu-ray tab
- Enabled cutting after 78 mins, thus removing ending credits
- Output muxed into 00000.m2ts; resulting file 22,3 GB (original 23,9 GB)
- Moved original 00000.m2ts out of the BDMV/STREAM directory; moved new 00000.m2ts into BDMV/STREAM
- Burned data on a BD-RE and played the disc on a PS3

Result: partial success. BD-RE behaves almost exactly like the original BD-Rom - except FF and RW tend to get stuck, and chapter skipping results in a black screen. Top menu and pop-up menu work perfectly.

Note that I did not output a Blu-ray structure with tsRemuX. Instead I simply replaced the original 00000.m2ts with a new, smaller one, thus using the original disc's structure. In other words, I'm using the original .clpi files and not ones created by tsMuxeR.

Arcsoft TMT plays the BD-RE a lot better. Chapter skips take about a second, but they work. FF works fine up to 4x, but at faster speeds there's visible image breakup and artifacting. RW doesn't work properly but I've noticed it works just as badly with original BD-Roms.

On a Panasonic BD55 the BD-RE functions exactly like the original BD-Rom, as long as I don't touch FF, RW and chapter skip buttons. Chapter skipping takes approx. 7 seconds after which the first frame of the next chapter is displayed for a few seconds, then image disappears, player handshakes over HDMI and then displays Top Menu. Selecting Resume Movie starts playback from the beginning, not from the chapter I skipped to earlier.

Last edited by colinhunt; 19th February 2009 at 13:42.
colinhunt is offline   Reply With Quote
Old 19th February 2009, 14:15   #13  |  Link
quantum
Registered User
 
Join Date: Nov 2002
Location: USA
Posts: 528
Quote:
Originally Posted by colinhunt View Post
Note that I did not output a Blu-ray structure with tsRemuX. Instead I simply replaced the original 00000.m2ts with a new, smaller one, thus using the original disc's structure. In other words, I'm using the original .clpi files and not ones created by tsMuxeR.
Then I don't think this is related. You're trying to merge two different Blu-ray structures.

My post is about tsMuxer generated Blu-ray output and the effect of fixclpi.
quantum is offline   Reply With Quote
Old 19th February 2009, 14:31   #14  |  Link
colinhunt
Registered User
 
Join Date: Dec 2002
Posts: 1,022
Quote:
Originally Posted by quantum View Post
You're trying to merge two different Blu-ray structures.
I'm using one and only one BD structure from the original BD-Rom. The 00000.m2ts file is not a 'structure' on its own; it's simply a container file with multiplexed streams inside.

Quote:
My post is about tsMuxer generated Blu-ray output and the effect of fixclpi.
I'm aware of this. My point is that you are trying to solve the same problem I'm having by applying some jdobbs magic to .clpi files created by tsMuxer. My experience of last night hints that the issue is not within tsMuxer-generated .clpi files but in the tsMuxer-generated .m2ts files.
colinhunt is offline   Reply With Quote
Old 20th February 2009, 05:55   #15  |  Link
quantum
Registered User
 
Join Date: Nov 2002
Location: USA
Posts: 528
For what it's worth, and I don't intend to keep harping on this if no one is interested, I just tested PowerDVD 7.3 and played some test Blu-ray muxes (as Blu-ray disks using the "open movie file on hard disk" feature). Fast forward and rewind in PowerDVD ( f and b keys) was affected by fixclpi. With the mux direct from tsMuxer, ff/rew worked as expected. With the mux treated by fixclpi, ff/rew at 2x and higher would usually not display any frames, although the counter would advance. The difference was obvious when switching the clpi files back and forth.
quantum is offline   Reply With Quote
Old 20th February 2009, 11:37   #16  |  Link
colinhunt
Registered User
 
Join Date: Dec 2002
Posts: 1,022
I think this issue needs to be harped on Perhaps someone could alert jdobbs & co. that the issue still exists.
colinhunt is offline   Reply With Quote
Old 20th February 2009, 16:04   #17  |  Link
laserfan
Aging Video Hobbyist
 
Join Date: Dec 2004
Location: Off the Map
Posts: 2,461
But no one has defined "the issue"--there have only been end-user observation of oddities on playback (both quantum's and my own and others) without any clear/definitive specification of where the problem(s) might lie.

I for example think contSPS creates problems for seeking, at least to chapter marks, but I have nothing to offer re: how it changes tsMuxeR's output i.e. .m2ts structure. I know that it DOES, because I've noticed that .m2ts sizes are different if "contSPS" vs. "no contSPS" but beyond that I have no clue what it's doing, nor apparently does anyone else as I've searched the Internet and come up with zip about it.

I'm just guessing, but it seems these problems aren't getting much attention because people are happy to just get their bloody programs to PLAY on their bloody players first! But then there are guys like me that anal-yze and stress over hitting chapter advances perfectly, or others like quantum who FF & REW during a film (I never do that, for example). Something is rotten in the state of tsMuxeR output, and I hope eventually someone with the knowledge/skills/toolset to look at this will ID where the muxing is "not right".
laserfan is offline   Reply With Quote
Old 20th February 2009, 16:36   #18  |  Link
deank
Programmer (or just 教务长)
 
deank's Avatar
 
Join Date: Oct 2008
Location: Valencia, Spain
Posts: 4,251
It is strange that you experience chapter seeking problems... One thing for sure is that chapter information is not stored in .clpi files but in .mpls files, so I don't see how fixclpi could affect chapters. (edit: probably it is because mpls refer to a table in .clpi which in its turn points to offsets used for jump-points).

I'm using PS3 and never used fixclpi. If I use FF/REW I first click the button once... then the player start seeking... Then I wait a bit more and then click the button again and it starts seeking faster and faster and I never had problems.

The problem with FF/REW and chapter seeking is with the transport stream itself and it depends on the type of player used.

These three: the sequence parameter set (SPS), the picture parameter set (PPS) and the supplemental enhancement information (SEI) define the way your player will try to decode video at the jump point. Often the jump point can refer to what you may call a B-frame and then your player needs to go forward to find the next I-frame and then obtain SPS/PPS information on where to find the previous IDR (I-frame or keyframe - you name it) needed to decode the exact jump-point video frame (I-frames are called IDR). Big authoring studios/companies would never allow a jump point (chapter) to point at non-IDR frame and also would (should?!) always perfect the way all streams are muxed in their final productions, as it is of great importance when using random access in a transport stream (m2ts) and keeping the overhead at minimum.

So unless chapter marks are set exactly where a transport unit (containing IDR frame) starts, then it all depends on your player's performance and the quality of the optical media in use. A table in clpi files defines the jump point and the closer it happens to be to the IDR the faster video playback is resumed. jdobbs' fix for these clpi files does this. It fixes the pointers so skipping and waiting are set to minimum, but it can't do miracles. To get best results (with tsmuxer's output) a combination of jdobbs' program and another utility is needed (or a fix for tsmuxer ). This utility should scan the entire m2ts stream and find the correct values to be put in clpi tables.

Some players/decoders use larger buffers to store SPS/PPS/SEI information thus making it easier and faster to perform seeking - others don't. Playstation3 is a really fast machine and probably this is the reason these problems do not affect it (when SPS/SEI information is present in all needed transport units).

As I already posted in another thread, m2ts inherited ts (which was never planned to be used for random access) and with this burden it needed additional set of controls (included in the NAL) to define means for random access.

You all already know that REWIND (will not mention reverse playback at all!) requires much more processing power than frame advance or fast forward... if your player is not powerful enough - no matter what fixes you apply - it will never get better.

And another thing - ff/rew problems with tsmuxer should manifest themselves only in certain parts of the movie (if jdobbs' fix is not applied).

Also - don't rely on SOFTWARE players for testing and observing. Always use your hardware player - it is supposed that the manufacturer of the hardware-player (with its on-chip software) did a much better job and is following standards strictly.

... and of course I may be wrong who knows - I hope my post sheds some light on the subject.
__________________
multiAVCHD - donate | popBD | uncropMKV | mkv2avi | easySUP

Last edited by deank; 20th February 2009 at 18:18.
deank is offline   Reply With Quote
Old 20th February 2009, 23:29   #19  |  Link
laserfan
Aging Video Hobbyist
 
Join Date: Dec 2004
Location: Off the Map
Posts: 2,461
Thank you for that deank--I can't say I understand every nuance but I appreciate your thorough attempt to educate! Regarding the I-frame question, I have worked very hard to assure that there exist I-frames at the desired landing points (jump points) for chapters. I double-check the exact timecodes to be included in tsmuxer's --custom-chapter option, by using neuron2's tools (to assure precise "seek") both with the original h264, MPEG2, or VC-1, and then again after encoding by x264 using DGAVCDecNV again and double-checking these timecodes. But then, after muxing, chapter advance doesn't always work precisely until fixclpi is applied, so there must be something about the clpi files tsMuxeR is generated that is faulty for sure.

That PDVD still doesn't always land correctly even after fixclpi might indeed suggest a PDVD problem (my TMT and hardware players seem to work perfectly) except that when I leave-out contSPS from tsMuxeR even IT works perfectly. I need to ask what is probably a stoopid newbie question: MUST blu-ray .m2ts files ALWAYS include ALL OF SPS, PPS, and SEI information? And which of these is ALREADY INCLUDED in x264 output? I saw elsewhere here some comment from neuron2 that suggested that the act of using his tools as input to x264 included already SPS/PPS info anyway. Another: any easy way for a dummy like me to look at an .264 file from x264, and then at the .m2ts file produced by tsMuxeR, and see if tsMuxer has tromped-on any/all of SPS/PPS/SEI? Or maybe better yet, to make files with-and-without tsMuxeR's insertSEI and contSPS options and see how the info might have changed?

My spirit is willing, though (if this ain't completely obvious) my understanding is weak!
laserfan is offline   Reply With Quote
Old 21st February 2009, 00:02   #20  |  Link
deank
Programmer (or just 教务长)
 
deank's Avatar
 
Join Date: Oct 2008
Location: Valencia, Spain
Posts: 4,251
Going backwards as for your questions order:

You can always use the good old FC.exe (present in all windows distributions). If you need to compare 2 files and find how they differ, just open a command prompt (start->run..->cmd) and type:

fc /b file1 file2

and you will get all offsets where these files differ. I doubt that it will help you but still it is good to know that MS kept this command line utility

***

About your question about comparing 264 output from x264 and tsMuxer... how to put it... x264 produces an elementary stream | tsmuxer produces a file with the container format.

tsmuxer uses the elementary stream from x264, encapsulates it in 192 byte packets and produces m2ts. There is no way to compare such pair of files as they will differ from the first byte. MPEG-2 Transport Stream (M2TS) is a container that can contain a h.264 stream (AVC or x264 output) and many other streams (such as audio/subtitles/etc - but you already know it).

I don't know about neuron's tool (please give a link for more info). There is not such thing as "bluray m2ts". It is just a container format.

x264 produces elementary stream and it does not contain anything else but the video itself - no SPS/PPS/SEI. These three are for m2ts and have nothing to do with the streams inside m2ts. It is a MUXER decision where to insert this information.

It really is not easy to explain all in few words (me - myself - i'm still reading and learning about all these things).

And again about clpi files - yes - jdobbs' program fixes tsmuxer's clpi output to an extent.

And about chapters - if you use SECONDS as a general value you can never get 100% exact entry point / jump point for your chapters.

That's it for now.
__________________
multiAVCHD - donate | popBD | uncropMKV | mkv2avi | easySUP
deank is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 12:26.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.