tsMuxeR Blu-ray output and Fixclpi

[quantum]

[ EDIT - This issue has been resolved in recent builds of tsMuxeR. No further patching is required ]

[ EDIT - jdobbs has posted an updated fixclpi that seems to solve all issues, including playback on Panasonic standalones. Get the new version here:
http://forum.doom9.org/showpost.php?p=1257320&postcount=106 ]

For those interested in using tsMuxeR to mux Blu-ray, jdobbs identified an issue with the .clpi files being generated by tsMuxer. He indicated the issue manifested itself with problems in fast forward and rewind, and chapter skipping.

This is his first post on the matter:
http://forum.doom9.org/showpost.php?p=1206841&postcount=2159

He then posted a program designed to fix the issue:
http://forum.doom9.org/showpost.php?p=1207492&postcount=2188

First let me say I'm grateful to jdobbs for pointing out the issue, and for creating and posting his utility.

The point of this post is to try to learn more about the issue, and see how people are being affected by using fixclpi.

My own experience: I have a Pioneer BDP-51FD standalone. I demux from original to elementary streams with eac3to and then mux to Blu-ray with tsMuxer and burn to BD-R 25GB media. Before using fixclpi, I had no problems with ff/rew, however I did notice that chapter jumps were sometimes slow, sometimes taking a few seconds. Chapter jumps were accurate, even if slow.

After using fixclpi, the chapter jumps were indeed faster. However, I then noticed artifacts when using ff/rew which previously were not an issue. The screen was not being correctly refreshed while using ff/rew. It looked like various portions of previous screens were being left behind. Or, to put it another way, only a portion of each new screen, near the top, was being drawn before the next screen was starting to be drawn. This could be described as a 'tearing' effect.

I tested the original store bought BD-Roms and they have no issues with ff/rew. Other burned BD-R's not treated with fixclpi have no issues with ff/rew. In these cases, which I consider normal, using ff/rew shows frames in rapid succession, in a slide show type effect. I don't have any theories about what is happening, or how fixclpi is involved or not involved. I'm just posting my observations.

[EDIT: I tested PowerDVD 7.3 and FF/REW ( f and b keys ) is also affected by fixclpi. It's easily noticeable at 2x and above, where frames are no longer drawn. It's easy to test this by swapping the clpi files back and forth. Note that only FF seems to display frames in PowerDVD, regardless of fixclpi. REW doesn't display frames for me in either case.]

I guess I could live with the fixclpi effect, but I'm concerned that the changes made by fixclpi may not be having the intended effect, which is, I hope, to make the clpi files totally compliant.

I'm equally concerned with leaving the clpi files untreated. I am seeing slow chapter seeking, so I'm concerned that the tsMuxer clpi files may in fact be non compliant as jdobbs reported. While they work fine now in my BDP-51FD (and PowerDVD), I'm wondering about my future players and how such disks will be tolerated.

What I'm asking for is any and all comments about experiences using tsMuxer and fixclpi, and the effect on standalone players, especially with ff/rew and chapter seeking. Maybe if enough detail is provided, jdobbs can spot something that might be helpful.

I'd also appreciate any comments or links to specifications about the clpi files. As a programmer myself, I'd like to examine this in detail (unless jdobbs comes to the rescue, in which case I'll gladly step aside).

I very much want to use tsMuxer as it does a great job in muxing basic Blu-rays. I also want to be confident that the output is as compliant as it can be.

[shon3i]

Did you recompress video? and how?

[quantum]

Did you recompress video? and how?
I'm demuxing to elementary streams with eac3to and muxing with tsMuxer. I have not reencoded the video.

My recent tests were with VC1.

[laserfan]

I can only say quantum that in my experience, failure to Fixclpi on tsMuxeR output results in PowerDVD missing "some" chapter marks, and "some" rather badly at that (i.e. not by just a frame or two, but way off). Not all, but some. So I don't know what Fixclpi does exactly, but it clearly fixes some serious glitch in tsMuxeR output and so I use it religiously.

But I have also observed that PowerDVD does not like the contSPS option in tsMuxeR in that if I leave that option OFF, then PDVD hits the chapter marks every time, whereas with contSPS enabled (checked in the GUI or added to the .meta file as most GUIs do) then again PDVD misses some chapter marks i.e. hits them early by a frame. I suspect contSPS to mess with timecodes somehow.

I don't use FF/RW when watching films (at all) so can't comment on that behavior. So this is all FWIW.

[quantum]

Shon3i's post brings to mind that I'll post a list of mux details, with video (vc1 or h264 and wether reencoded) and if h264, the status of "add picture timing info" and "continually insert SPS/PPS" check boxes.

[quantum]

..But I have also observed that PowerDVD does not like the contSPS option in tsMuxeR in that if I leave that option OFF, then PDVD hits the chapter marks every time, whereas with contSPS enabled (checked in the GUI or added to the .meta file as most GUIs do) then again PDVD misses some chapter marks i.e. hits them early by a frame.
So can't you just leave "add picture timing info" and "continually insert SPS/PPS" unchecked and not run fixclpi?

[laserfan]

So can't you just leave "add picture timing info" and "continually insert SPS/PPS" unchecked and not run fixclpi?No, and maybe my contSPS comment is a distraction (i.e. a completely separate issue).

[quantum]

I just finished 4 test muxes using a BD-RE. The movie was AVP Requiem. Video is H264 as demuxed with eac3to and no reencode, audio is DTS, no subtitles, with chapter stops.

+ indicates option active, - indicates option not used

1) -Fixclpi, -add picture timing, -continually insert SPS/PPS = good FF/REW, slower chapter seek
2) -Fixclpi, +add picture timing, +continually insert SPS/PPS = good FF/REW, slower chapter seek
3) +Fixclpi, +add picture timing, +continually insert SPS/PPS = bad FF/REW, good chapter seek
4) +Fixclpi, -add picture timing, -continually insert SPS/PPS = bad FF/REW, good chapter seek
5) +Fixclpi, +add picture timing, -continually insert SPS/PPS = bad FF/REW, good chapter seek

I didn't get this thorough with my last VC1 test, but I think the results are consistent with H264 and VC1. I don't recall having an instance were fixclpi was applied and ff/rew worked without obvious video artifacts.

[laserfan]

I don't see +Fixclpi, +add picture timing, -continually insert SPS/PPS

[quantum]

I don't see +Fixclpi, +add picture timing, -continually insert SPS/PPS

Okay, in the interest of completeness, I just burned that and updated the summary post. No difference.

However, it occurs to me, that no matter how I set those two settings, I'm not seeing any message in tsMuxer GUI that anything is changing. I see something like this for an x264 encode:

H264 bitstream changed: insert pict timing and buffering period SEI units

But nothing for the H264 pulled straight off the disk. I assume this means the H264 as encoded from the studio has all the necessary timing and SPS/PPS, and tsMuxer does not attempt to re-add or change them.

So I don't think changing "add picture timing" or "continually insert SPS/PPS" settings makes any difference for this case, and these settings are not available at all for VC1.

[laserfan]

Okay, in the interest of completeness, I just burned that...Ok, but the only one you missed was the one I commented on!

I'm not seeing any message in tsMuxer GUI that anything is changing.And I'm not sure you would.

[colinhunt]

I was fooling around with a project last night and bumped into something that relates to this issue.

Starting point:
A Blu-ray with slightly more than 26 gigabytes of data
- main movie as 00000.m2ts

Goal:
- make 00000.m2ts small enough to fit all data on a single-layer BD-RE while keeping original menus

Idea:
step 1) Strip one of the two audiotracks and cut ending credits out, then use tsMuxeR 1.8.18(b) to create a new 00000.m2ts
step 2) Replace original 00000.m2ts with the tsMuxeR-created one in original BDMV/STREAM directory
step 3) Burn data on a BD-RE and see what happens

Process:
- Dropped the original 00000.m2ts into tsMuxeR
- unticked LPCM track
- left Add picture timing info and Continually insert PS/PPS at defaults, i.e. ticked
- Selected No chapters in the Blu-ray tab
- Enabled cutting after 78 mins, thus removing ending credits
- Output muxed into 00000.m2ts; resulting file 22,3 GB (original 23,9 GB)
- Moved original 00000.m2ts out of the BDMV/STREAM directory; moved new 00000.m2ts into BDMV/STREAM
- Burned data on a BD-RE and played the disc on a PS3

Result: partial success. BD-RE behaves almost exactly like the original BD-Rom - except FF and RW tend to get stuck, and chapter skipping results in a black screen. Top menu and pop-up menu work perfectly.

Note that I did not output a Blu-ray structure with tsRemuX. Instead I simply replaced the original 00000.m2ts with a new, smaller one, thus using the original disc's structure. In other words, I'm using the original .clpi files and not ones created by tsMuxeR.

Arcsoft TMT plays the BD-RE a lot better. Chapter skips take about a second, but they work. FF works fine up to 4x, but at faster speeds there's visible image breakup and artifacting. RW doesn't work properly but I've noticed it works just as badly with original BD-Roms.

On a Panasonic BD55 the BD-RE functions exactly like the original BD-Rom, as long as I don't touch FF, RW and chapter skip buttons. Chapter skipping takes approx. 7 seconds after which the first frame of the next chapter is displayed for a few seconds, then image disappears, player handshakes over HDMI and then displays Top Menu. Selecting Resume Movie starts playback from the beginning, not from the chapter I skipped to earlier.

[quantum]

Note that I did not output a Blu-ray structure with tsRemuX. Instead I simply replaced the original 00000.m2ts with a new, smaller one, thus using the original disc's structure. In other words, I'm using the original .clpi files and not ones created by tsMuxeR.
Then I don't think this is related. You're trying to merge two different Blu-ray structures.

My post is about tsMuxer generated Blu-ray output and the effect of fixclpi.

[colinhunt]

You're trying to merge two different Blu-ray structures.
I'm using one and only one BD structure from the original BD-Rom. The 00000.m2ts file is not a 'structure' on its own; it's simply a container file with multiplexed streams inside.

My post is about tsMuxer generated Blu-ray output and the effect of fixclpi.
I'm aware of this. My point is that you are trying to solve the same problem I'm having by applying some jdobbs magic to .clpi files created by tsMuxer. My experience of last night hints that the issue is not within tsMuxer-generated .clpi files but in the tsMuxer-generated .m2ts files.

[quantum]

For what it's worth, and I don't intend to keep harping on this if no one is interested, I just tested PowerDVD 7.3 and played some test Blu-ray muxes (as Blu-ray disks using the "open movie file on hard disk" feature). Fast forward and rewind in PowerDVD ( f and b keys) was affected by fixclpi. With the mux direct from tsMuxer, ff/rew worked as expected. With the mux treated by fixclpi, ff/rew at 2x and higher would usually not display any frames, although the counter would advance. The difference was obvious when switching the clpi files back and forth.

[colinhunt]

I think this issue needs to be harped on :) Perhaps someone could alert jdobbs & co. that the issue still exists.

[laserfan]

But no one has defined "the issue"--there have only been end-user observation of oddities on playback (both quantum's and my own and others) without any clear/definitive specification of where the problem(s) might lie.

I for example think contSPS creates problems for seeking, at least to chapter marks, but I have nothing to offer re: how it changes tsMuxeR's output i.e. .m2ts structure. I know that it DOES, because I've noticed that .m2ts sizes are different if "contSPS" vs. "no contSPS" but beyond that I have no clue what it's doing, nor apparently does anyone else as I've searched the Internet and come up with zip about it.

I'm just guessing, but it seems these problems aren't getting much attention because people are happy to just get their bloody programs to PLAY on their bloody players first! But then there are guys like me that anal-yze and stress over hitting chapter advances perfectly, or others like quantum who FF & REW during a film (I never do that, for example). Something is rotten in the state of tsMuxeR output, and I hope eventually someone with the knowledge/skills/toolset to look at this will ID where the muxing is "not right".

[deank]

It is strange that you experience chapter seeking problems... One thing for sure is that chapter information is not stored in .clpi files but in .mpls files, so I don't see how fixclpi could affect chapters. (edit: probably it is because mpls refer to a table in .clpi which in its turn points to offsets used for jump-points).

I'm using PS3 and never used fixclpi. If I use FF/REW I first click the button once... then the player start seeking... Then I wait a bit more and then click the button again and it starts seeking faster and faster and I never had problems.

The problem with FF/REW and chapter seeking is with the transport stream itself and it depends on the type of player used.

These three: the sequence parameter set (SPS), the picture parameter set (PPS) and the supplemental enhancement information (SEI) define the way your player will try to decode video at the jump point. Often the jump point can refer to what you may call a B-frame and then your player needs to go forward to find the next I-frame and then obtain SPS/PPS information on where to find the previous IDR (I-frame or keyframe - you name it) needed to decode the exact jump-point video frame (I-frames are called IDR). Big authoring studios/companies would never allow a jump point (chapter) to point at non-IDR frame and also would (should?!) always perfect the way all streams are muxed in their final productions, as it is of great importance when using random access in a transport stream (m2ts) and keeping the overhead at minimum.

So unless chapter marks are set exactly where a transport unit (containing IDR frame) starts, then it all depends on your player's performance and the quality of the optical media in use. A table in clpi files defines the jump point and the closer it happens to be to the IDR the faster video playback is resumed. jdobbs' fix for these clpi files does this. It fixes the pointers so skipping and waiting are set to minimum, but it can't do miracles. To get best results (with tsmuxer's output) a combination of jdobbs' program and another utility is needed (or a fix for tsmuxer :) ). This utility should scan the entire m2ts stream and find the correct values to be put in clpi tables.

Some players/decoders use larger buffers to store SPS/PPS/SEI information thus making it easier and faster to perform seeking - others don't. Playstation3 is a really fast machine and probably this is the reason these problems do not affect it (when SPS/SEI information is present in all needed transport units).

As I already posted in another thread, m2ts inherited ts (which was never planned to be used for random access) and with this burden it needed additional set of controls (included in the NAL) to define means for random access.

You all already know that REWIND (will not mention reverse playback at all!) requires much more processing power than frame advance or fast forward... if your player is not powerful enough - no matter what fixes you apply - it will never get better.

And another thing - ff/rew problems with tsmuxer should manifest themselves only in certain parts of the movie (if jdobbs' fix is not applied).

Also - don't rely on SOFTWARE players for testing and observing. Always use your hardware player - it is supposed that the manufacturer of the hardware-player (with its on-chip software) did a much better job and is following standards strictly.

... and of course I may be wrong :) who knows - I hope my post sheds some light on the subject.

[laserfan]

Thank you for that deank--I can't say I understand every nuance but I appreciate your thorough attempt to educate! Regarding the I-frame question, I have worked very hard to assure that there exist I-frames at the desired landing points (jump points) for chapters. I double-check the exact timecodes to be included in tsmuxer's --custom-chapter option, by using neuron2's tools (to assure precise "seek") both with the original h264, MPEG2, or VC-1, and then again after encoding by x264 using DGAVCDecNV again and double-checking these timecodes. But then, after muxing, chapter advance doesn't always work precisely until fixclpi is applied, so there must be something about the clpi files tsMuxeR is generated that is faulty for sure.

That PDVD still doesn't always land correctly even after fixclpi might indeed suggest a PDVD problem (my TMT and hardware players seem to work perfectly) except that when I leave-out contSPS from tsMuxeR even IT works perfectly. I need to ask what is probably a stoopid newbie question: MUST blu-ray .m2ts files ALWAYS include ALL OF SPS, PPS, and SEI information? And which of these is ALREADY INCLUDED in x264 output? I saw elsewhere here some comment from neuron2 that suggested that the act of using his tools as input to x264 included already SPS/PPS info anyway. Another: any easy way for a dummy like me to look at an .264 file from x264, and then at the .m2ts file produced by tsMuxeR, and see if tsMuxer has tromped-on any/all of SPS/PPS/SEI? Or maybe better yet, to make files with-and-without tsMuxeR's insertSEI and contSPS options and see how the info might have changed? :confused:

My spirit is willing, though (if this ain't completely obvious) my understanding is weak! ;)

[deank]

Going backwards as for your questions order:

You can always use the good old FC.exe (present in all windows distributions). If you need to compare 2 files and find how they differ, just open a command prompt (start->run..->cmd) and type:

fc /b file1 file2

and you will get all offsets where these files differ. I doubt that it will help you but still it is good to know that MS kept this command line utility :)

***

About your question about comparing 264 output from x264 and tsMuxer... how to put it... x264 produces an elementary stream | tsmuxer produces a file with the container format.

tsmuxer uses the elementary stream from x264, encapsulates it in 192 byte packets and produces m2ts. There is no way to compare such pair of files as they will differ from the first byte. MPEG-2 Transport Stream (M2TS) is a container that can contain a h.264 stream (AVC or x264 output) and many other streams (such as audio/subtitles/etc - but you already know it).

I don't know about neuron's tool (please give a link for more info). There is not such thing as "bluray m2ts". It is just a container format.

x264 produces elementary stream and it does not contain anything else but the video itself - no SPS/PPS/SEI. These three are for m2ts and have nothing to do with the streams inside m2ts. It is a MUXER decision where to insert this information.

It really is not easy to explain all in few words (me - myself - i'm still reading and learning about all these things).

And again about clpi files - yes - jdobbs' program fixes tsmuxer's clpi output to an extent.

And about chapters - if you use SECONDS as a general value you can never get 100% exact entry point / jump point for your chapters.

That's it for now.

[laserfan]

Thanks again deank, I was thinking not about FC-type compare (or WinMerge) but rather comparing SEI/SPS/PPS flags, but you answer part of this too ("x264 produces elementary stream and it does not contain anything else but the video itself - no SPS/PPS/SEI.") So I guess my remaining ? might be how to see in the file structure the effects of tsMuxeR's insertSEI, contSPS which is almost back where we started w/the OP asking what fixclpi does exactly.

Hey deank can you write a better blu-ray muxer maybe than smlabs' Roman!? ;)

Re: chapters I was referring simply to xx:xx:xx.xxx timecode specifications to the --custom-chapters option. The tools from Donald Graft like DGAVCDecNV here (http://www.neuron2.net/dgavcdecnv/dgavcdecnv.html) always seem to ID timecodes by frame exactly right!

[setarip_old]

@quantum

Hi!I just tested PowerDVD 7.3 and played some test Blu-ray muxes (as Blu-ray disks using the "open movie file on hard disk" feature).Forgive my all-too-obvious ignorance regarding this procedure, but since I've had no success doing this myself, so I'll ask - when using "open movie file on hard disk" for BluRay hard drive "packages", what file or folder of the "package" do you click on in order for the file selector to make its "OK" radiobutton available?

[quantum]

...when using "open movie file on hard disk" for BluRay hard drive "packages", what file or folder of the "package" do you click on in order for the file selector to make its "OK" radiobutton available?
Either the folder above BDMV, or the BDMV folder. However, I've heard that this functionality may have been removed in later versions. I'm using 7.3. Some complex Blu-ray disks may not have started for me, but the simple movie-only muxes from tsMuxer always work fine.

[setarip_old]

@quantum

Thanks for the information ;>}

**EDIT** Strange, it only works for standard DVD "packages" on the hard drive, not BluRay "packages" (I also have v.7.3)

[laserfan]

[B]**EDIT** Strange, it only works for standard DVD "packages" on the hard drive, not BluRay "packages" (I also have v.7.3) [/COLOR]Should work, I have same version as you & quantum and it works well for everything.

[setarip_old]

@laserfan
Should workWould that it were so. Just another anomaly in the world after punchcards ;>}

[anode]

I have been analysing the output *clpi from tsMuxeR 1.8.8b for my own. This files contain a table with PTS (timecode) to SPN (source packet number in m2ts-file) relationship for all I-IDR(?)-frames.
So if you skip (chapters) or ff/rw these tables are used to determinate where is the location of the next i-frame to start decoding.
tsMuxeR generates these tables wrong. The tables have first a COARSE section, and second a FINE section. The full PTS and SPN Numbers are divided in this coarse and fine tables. If the fine entry of PTS and/or SPN rolls over, you have to make a new coarse entry. tsMuxeR does this for PTS, but not for SPN.

djobbs fixclpi fixes this, but as far i can see, introduces another error: As PTS fine and coarse entries share one bit together, you have to generate a new coarse entry for PTS when the 2nd most significant bit of the fine PTS rolls over, since the most significant bit of the finePTS is the least significant bit of the coarsePTS (shared). So here should a new coarse entry be created and fixclpi does not do this.
(Sorry for this technical stuff, maybe it is of use for somebody)

I am trying to fix this for myself to see if seeking/ff/rw behaviour is changing. Weren't there some Panasonic Players who reject discs with fixclpi'ed-files?
Has somebody researched this topic,too?

[laserfan]

Well, of course jdobbs has looked hard at clpi files and if you are correct about fixclpi you should somehow direct him to this thread. Here are a couple of his posts on this:

http://forum.doom9.org/showthread.php?p=1223405#post1223405

http://forum.doom9.org/showthread.php?p=1207492#post1207492

I'm guessing the latter thread would surely get his attention, but maybe instead you could PM him and direct him here in case he misses your post.

[rack04]

I am trying to fix this for myself to see if seeking/ff/rw behaviour is changing. Weren't there some Panasonic Players who reject discs with fixclpi'ed-files?

Yes, the Panasonic BD-35 for example.

[nwg]

What is meant by bad ff/rw? I have done some Blu Rays and the Sony S350 is not ff/rw very well. It does work for a few minutes then freezes the image until I press play again. This is with fixclpi used and add picturing timing and insert sps is greyed out so I don't know how to select them

[quantum]

...significant bit of the finePTS is the least significant bit of the coarsePTS (shared). So here should a new coarse entry be created and fixclpi does not do this.
(Sorry for this technical stuff, maybe it is of use for somebody)
I'm grateful that you are taking an interest in this, and even better, that you have the technical knowledge of the Blu-ray format. If I can offer any help at all (especially as a programmer), I'd be glad to be of service.

[jdobbs]

I have been analysing the output *clpi from tsMuxeR 1.8.8b for my own. This files contain a table with PTS (timecode) to SPN (source packet number in m2ts-file) relationship for all I-IDR(?)-frames.
So if you skip (chapters) or ff/rw these tables are used to determinate where is the location of the next i-frame to start decoding.
tsMuxeR generates these tables wrong. The tables have first a COARSE section, and second a FINE section. The full PTS and SPN Numbers are divided in this coarse and fine tables. If the fine entry of PTS and/or SPN rolls over, you have to make a new coarse entry. tsMuxeR does this for PTS, but not for SPN.

djobbs fixclpi fixes this, but as far i can see, introduces another error: As PTS fine and coarse entries share one bit together, you have to generate a new coarse entry for PTS when the 2nd most significant bit of the fine PTS rolls over, since the most significant bit of the finePTS is the least significant bit of the coarsePTS (shared). So here should a new coarse entry be created and fixclpi does not do this.
(Sorry for this technical stuff, maybe it is of use for somebody)

I am trying to fix this for myself to see if seeking/ff/rw behaviour is changing. Weren't there some Panasonic Players who reject discs with fixclpi'ed-files?
Has somebody researched this topic,too? Hmmm... I thought that was how I did it, and it was done to fix exactly the situation you are describing. Have you dumped it out and seen this on "fixed" clips?

I'll go back and look. Thanks for the hint... I was doing a logical OR of the COARSE/FINE portions together when I was testing the output, so I may have missed it. It definitely fixed the FF/RW issues I had -- but maybe my player's firmware doesn't notice this because it works the same way I did. I even went so far as to put a "don't fix the clip" option in BD Rebuilder so the Panasonics could still use it.

[laserfan]

I'll go back and look.Thanks for checking. I hope anode comes back here and is not just a "one-post wonder"! ;)

Your fixclpi repairs chapter-seek problems with 2 out of 3 playback options I have (a stb, TMT, and PDVD). But with PDVD I still sometimes have odd seek problems and I dunno if it is PDVD or maybe this thing anode suspects in the clpis (or a different tsMuxeR bug for that matter).

[jdobbs]

He's right. I was checking for rollover on the MSB of the fine portion of the PTS, when I should have been looking for change in the coarse part or rollover on the 2nd MSB... I've fixed it and am testing it. I also added a check that would correct any previously "fixed" CLPI files.

[quantum]

Let me be the first to say THANK YOU jdobbs and anode :thanks: :) :) :) ;)

Also I'll be ready to test the new update as soon as it comes out on my Standalone and on PowerDVD.

[jdobbs]

Well, not yet... I adjusted it so the it creates a COARSE entry when the 2nd MSB rolls over. But that caused the output FF/RW and chapter jumping to fail completely (with picture freeze). What anode says make sense, but my Sony SAP certainly doesn't like it. Still experimenting/testing. I may be missing something.

[jdobbs]

It turned out to be the source I was working with. You can now download a new version of FIXCLPI (v2.1) from this link. (http://www.jdobbs.net/freeware/fixclpi_v21.zip)

I'm hoping this one fixes the previously reported issues with Panasonic players. I'd appreciate it if someone would report back on that.

I added a new mode in which if you pass the program a directory name (rather than a file name) it will correct any CLPI files it finds within that directory tree. That should save a little time.

I tested it on a couple of sources and it appears to work correctly -- but like all freeware, there are no assurances made.

My thanks to anode for pointing out the issue.

[MadMonkey57]

Sounds promising !!! Thanks anode and jdobbs.
I'll report to let you know what my Pana BD35 thinks about it...

[rack04]

Burning a disc now to test on my BD35. Will report back soon. Thanks jdobbs.

[jdobbs]

Just updated it again, download from this link (http://www.jdobbs.net/freeware/fixclpi_v21.zip). The folder update option was only updating the first CLPI in each subfolder. It will now do all of them. I also updated the link above, so it will also get this version.

[quantum]

Tested a shortened Blu-ray using tsMuxer 1.8.18b and the new fixclpi:
- No changes in PowerDVD - still either no frames drawn with FF or sporadic frames drawn.
- No changes in Pioneer standalone - FF/REW appears visually corrupted.

In both cases, chapters work fine.

I did see that .clpi file size has changed between the old and new fixclpi versions. New version creates slightly larger files.

[jdobbs]

Tested a shortened Blu-ray using tsMuxer 1.8.18b and the new fixclpi:
- No changes in PowerDVD - still either no frames drawn with FF or sporadic frames drawn.
- No changes in Pioneer standalone - FF/REW appears visually corrupted.

In both cases, chapters work fine.

I did see that .clpi file size has changed between the old and new fixclpi versions. New version creates slightly larger files.
Don't know what "no frames drawn" means.

Do these symptoms show with the original "not fixed" source from TSMUXER (run directly, not from BD-RB -- because it fixes automatically)? I'm guessing it does, because I can't see how the issues in these tables would cause either of those symptoms. Normally when the tables are bad they cause problems in FF/RW and problems seeking chapters. The FF/RW symptoms are more issues like going backward and suddenly you are going forward again (while still holding down the RW button) or freezing.

So TSMUXER v1.8.18 still creates bad tables? That's something I thought they would have fixed right away given how easy it is.

Try using v1.8.4. I know it works, and that way we can eliminate any new issues introduced. There is usually a reason why the website still has an older version linked even though there are newer test versions.

[rack04]

Sad to report that fixclpi still creates a unplayable disc on my BD35.

[jdobbs]

Frankly, then, I have no idea why the Panasonics are having a problem. I've been comparing the output tables of this version (forced to "fix") with the original commercial CLPI files -- and so far they are identical. I'll keep looking.

[edit] Well... in dumping, maybe not. Still looking.

[quantum]

Don't know what "no frames drawn" means.
When using the FF feature in PowerDVD ( the F key ) while playing as a Blu-ray, I see a rapid slide show of frames, similar to what my Pioneer standalone displays. This works as expected with the untreated Blu-ray output direct from tsMuxer.

When treated with fixclpi, PowerDVD FF tends to not display anything, or will display a single frozen frame, but the counter is advancing. I have been using this as a quick test, as it seems to be consistent with my standalone having FF/REW visual artifacts. If it works in PowerDVD, it seems to work on my standalone.

Note that I haven't used BD-RB, just demuxed with eac3to and remuxed with tsMuxer ( and optionally treated with fixclpi ).


..because I can't see how the issues in these tables would cause either of those symptoms.
I wish I had the answer. Unfortunately I don't have a clue about the inner workings of the Blu-ray structure. Or maybe the issue is on my setup.

Try using v1.8.4.
Just muxed a small sample with 1.8.4. Still no change. I'll try some more tests.

[laserfan]

Just updated it again, http://www.jdobbs.net/freeware/fixclpi_v21.zip.Just want to confirm you've packed-up the right file: this shows 2.00 on launch

C:\>fixclpi
FixCLPI v2.00, jdobbs softworks

Usage:
file mode: fixclpi "c:\output\path\00001.clpi"
folder mode: fixclpi "c:\output\path"
I'm testing it right now myself. :)

[jdobbs]

Forgot to update that text, it should be the right one... I'm still working on this, so don't use it for anything but testing, ok?

[jdobbs]

Ok... try this one. (http://www.jdobbs.net/freeware/fixclpi_v22.zip) Make sure you use it against a source that hasn't been "fixed" before (preferably something straight from TSMUXER). Let me know if it works with the Panasonic.

Thanks.

[quantum]

My first PowerDVD test with fixclpi 2.2 looks promising :) FF seems to be normal.

I'm going to burn a BD-RE and test on the Pioneer :cool:

[jdobbs]

If you downloaded before this message... download it again. I'd accidentally left some test code in...

[jdobbs]

My first PowerDVD test with fixclpi 2.2 looks promising :) FF seems to be normal.

I'm going to burn a BD-RE and test on the Pioneer :cool:Great. Fingers are crossed.

[quantum]

Great. Fingers are crossed.

I just re-applied the new 2.2 (on a clean untreated source) and retested in PowerDVD, just to be sure ;-)

I noticed something: before applying fixclpi (using straight tsMuxer output), FF worked in PowerDVD, but REW never displayed any frames, and tended to max the CPU. After the new fixclpi, both FF and REW work great, and both show frames as you would expect, even up to 4x (didn't test any higher). I thought a little positive news would be appreciated :)

Now burning a BD-RE to test the Pioneer.

[jdobbs]

Definitely appreciated. If this works correctly I'm going to integrate the updates it into BD-RB.

[laserfan]

I have always used fixclpi with all my muxes, and am interested in 2.2 because I sometimes have chapter marks that fail to land properly (usually a bad one will land a few frames early, which can be very disconcerting). Using 2.1, I made a whole bunch of different tries, including with/without insertSEI, contSPS, and fps= options w/tsmuxer, and with versions 1.8.8 and then 1.8.4. With these tests it was clear that fixclpi made things much better ie. sometimes a chapter advance would not only land MANY SECONDS early, it would also show an incorrect timecode for the landing, i.e. the timecode would be for the landing point of the mark, but the wrong frame!

With all combos of tsMuxeR and its opts and fixclpi 2.1, I was still getting one mark that wasn't right (a frame or two early). Finally I tried 1.8.18 and fixclpi and it all now is perfect! :eek:

So IMO there's at least one problem with tsMuxeR 1.8.4/5 and 1.8.8 that 1.8.18 latest fixes, altho fixclpi is still needed of course.

I'll fiddle with 2.2 next but I therefore suggest jdobbs that you try 1.8.18 yourself.

P.S. with my players and x264 encodings as input to the muxes, it appears I don't need ANY OF tsMuxeR's fps=, insertSEI, or contSPS options!

Sorry that I have nothing to contribute wrt FF/REW or compatibility--these muxes always work on my BH200 player.

[quantum]

Tested a shortened tsMuxer 1.8.4 Blu-ray mux after applying fixclpi 2.2 on my Pioneer standalone. Everything looks perfect! I'm going to test a full length disk from another source where I know exactly where the chapters are, and I remember distinctly how slow chapter advances were before fixclpi. Of course I'll also double check FF/REW, but I'm thinking we have a winner here.

Very much appreciated jdobbs!

[nwg]

Will the current Fixclpi gui work ok with the new version?

[anode]

Hello jdobbs and the others,

I'm glad that you found some issues according to my first post.
Thank you for reacting so fast.
I tried to fix it for myself but i have not much programming skills and so my own tests take always some time.
For me (i have a Samsung BD-P2500 standalone) tests with the fixed clpi-files did improve ff/rw: now the picture displayed while searching is not longer divided in some parts, but a complete frame is shown. This is nothing very important, since chapter skipping did work with the older, fixed clpi.
Sometimes after chapter skip there are audio/video resync issues. This may have to do with the chapter point not being placed exactly on a I-Frame or (as i use hd-broadcasts) the streams have longer GOPs than within blu-ray specs. Will have to do some tests....

For anybody interested in clpi-file format i have found 2 patents describing the binary structure:
EP 1873780 and EP 1715686 (sorry, you have to google for them. Get the pdf with the pictures...)

Another issue with the clpi's from tsMuxeR is that the "NumberOfSourcePackets" in the ClipInfo-Section is wrong. This should be the number of 192-byte Packets in the corresponding m2ts-stream.
I have found two tools which fixes this, one "AVCHDme 1.4" and another "ClipInf Editor v0.01b". The second one calculates this number sometimes wrong by one packet?
Perhaps we can correct all this little tsMuxeR-bugs, so that even the Panasonics should accept the clpi's. (Hard to understand that they play with the obvious wrong tables!)

[laserfan]

Hey anode great to have you back, awesome first & second posts!!! ;)

I can vouch for audio sync issues following a chapter/I-frame mismatch, but even if this is the case the audio syncs-up again probably at the next GOP.

Welcome to doom9! :)

[quantum]

I did another test, this time using a different source, and with tsMuxer 1.8.18b. Everything is perfect in PowerDVD and my Pioneer standalone. FF/REW works fine, and chapters are fast and accurate.

Now we just need confirmation from a Panasonic owner.


Another issue with the clpi's from tsMuxeR is that the "NumberOfSourcePackets" in the ClipInfo-Section is wrong. This should be the number of 192-byte Packets in the corresponding m2ts-stream..
...Perhaps we can correct all this little tsMuxeR-bugs, so that even the Panasonics should accept the clpi's. (Hard to understand that they play with the obvious wrong tables!)
Thank you for your excellent observations anode. Hopefully jdobbs will turn fixclpi into a complete tsMuxer fixer!

I do notice that Clipinf Editor 0.02b reports the clipinf packets are not sized correctly.

[jdobbs]

Will the current Fixclpi gui work ok with the new version? What kind of GUI do you want? I can make one very easily as a part of it.

[jdobbs]

Hello jdobbs and the others,

I'm glad that you found some issues according to my first post.
Thank you for reacting so fast.
I tried to fix it for myself but i have not much programming skills and so my own tests take always some time.
For me (i have a Samsung BD-P2500 standalone) tests with the fixed clpi-files did improve ff/rw: now the picture displayed while searching is not longer divided in some parts, but a complete frame is shown. This is nothing very important, since chapter skipping did work with the older, fixed clpi.
Sometimes after chapter skip there are audio/video resync issues. This may have to do with the chapter point not being placed exactly on a I-Frame or (as i use hd-broadcasts) the streams have longer GOPs than within blu-ray specs. Will have to do some tests....

For anybody interested in clpi-file format i have found 2 patents describing the binary structure:
EP 1873780 and EP 1715686 (sorry, you have to google for them. Get the pdf with the pictures...)

Another issue with the clpi's from tsMuxeR is that the "NumberOfSourcePackets" in the ClipInfo-Section is wrong. This should be the number of 192-byte Packets in the corresponding m2ts-stream.
I have found two tools which fixes this, one "AVCHDme 1.4" and another "ClipInf Editor v0.01b". The second one calculates this number sometimes wrong by one packet?
Perhaps we can correct all this little tsMuxeR-bugs, so that even the Panasonics should accept the clpi's. (Hard to understand that they play with the obvious wrong tables!) I made a lot of streams a while back that had long GOPs -- I can confirm that when you use them you will get video delays at chapter points. For Blu-ray compatibility you need to make sure the GOPs are no more than one second long. In BD-RB I set them to 24 as the default. If you stick to that rule the chapters should sync up immediately.

I forgot about the number of packets bug in TSMUXER. I'd fixed it in BD-RB but not in FIXCLPI. I'll put another version up shortly that fixes that too (as long as FIXCLPI runs in folder mode and the BDMV\STREAM folder can be found).

[nwg]

What kind of GUI do you want? I can make one very easily as a part of it.

There has been one available for a while now from here somewhere done by someone else. It just allows the user to select the files/folders and process them. I tried it with 2.2 and it works fine.

I am trying it in a minute on a ripbot avchd (BDMV folder on a DVD). When I play it them in a Sony S350. The picture freezes when trying to ff/rwd. Skip chapters work but there is no sound for a few seconds and then it plays normally. I have been blaming the player as the same disc works perfectly in a Samsung 1500 player.

[jdobbs]

Hello jdobbs and the others,

I'm glad that you found some issues according to my first post.
Thank you for reacting so fast.
I tried to fix it for myself but i have not much programming skills and so my own tests take always some time.
For me (i have a Samsung BD-P2500 standalone) tests with the fixed clpi-files did improve ff/rw: now the picture displayed while searching is not longer divided in some parts, but a complete frame is shown. This is nothing very important, since chapter skipping did work with the older, fixed clpi.
Sometimes after chapter skip there are audio/video resync issues. This may have to do with the chapter point not being placed exactly on a I-Frame or (as i use hd-broadcasts) the streams have longer GOPs than within blu-ray specs. Will have to do some tests....

For anybody interested in clpi-file format i have found 2 patents describing the binary structure:
EP 1873780 and EP 1715686 (sorry, you have to google for them. Get the pdf with the pictures...)

Another issue with the clpi's from tsMuxeR is that the "NumberOfSourcePackets" in the ClipInfo-Section is wrong. This should be the number of 192-byte Packets in the corresponding m2ts-stream.
I have found two tools which fixes this, one "AVCHDme 1.4" and another "ClipInf Editor v0.01b". The second one calculates this number sometimes wrong by one packet?
Perhaps we can correct all this little tsMuxeR-bugs, so that even the Panasonics should accept the clpi's. (Hard to understand that they play with the obvious wrong tables!) I did a lot of file dumps and analysis to find those table formats... it sure would have been nice to find docs a month or two ago :)

[nwg]

I think the new version has improved the ripbot files. Both the Sony S350/S550 doesn't have great navigation with discs. Even at full speed it is pretty slow going through a film but it jumps from one bit to another as it freezes periodically on frame while the counter does the same thing. It can be quite hard to find certain bits sometimes. The new Fixclpi has managed to make the navigation a little smoother.

I am going to compare a TsMuxed BD that has the first fixclpi done to it and then redo it with the 2.2 version.

[rack04]

I made a lot of streams a while back that had long GOPs -- I can confirm that when you use them you will get video delays at chapter points. For Blu-ray compatibility you need to make sure the GOPs are no more than one second long. In BD-RB I set them to 24 as the default. If you stick to that rule the chapters should sync up immediately.

Are you referring to --keyint 24?

[jdobbs]

Yes. And, of course, you have to also use "--min-keyint 1" along with it.

[jdobbs]

And here is one more release (http://www.jdobbs.net/freeware/fixclpi_v23.zip) (v2.30) of FixCLPI. In this one if you pass it a folder as the parameter, it will also automatically fix the packet_count variable (the one written by TSMUXER is often if not always wrong). Of course it will only correct it if the CLPI is in a standard BD format (CLPI is in either the BACKUP/CLIPINF or the CLIPINF folder, and the M2TS is in the STREAM folder). The program is verbose, so if it gets changed, you'll know it.

[quantum]

And here is one more release (http://www.jdobbs.net/freeware/fixclpi_v23.zip) (v2.30) of FixCLPI. In this one if you pass it a folder as the parameter, it will also automatically fix the packet_count variable (the one written by TSMUXER is often if not always wrong). Of course it will only correct it if the CLPI is in a standard BD format (CLPI is in either the BACKUP/CLIPINF or the CLIPINF folder, and the M2TS is in the STREAM folder). The program is verbose, so if it gets changed, you'll know it.
Awesome work! I double checked the math, based on anode's comments, and it looks like you have it right. Clipinf Editor reports it wrong in my test, off by one packet as anode said.

[turbojet]

fixclpi 2.30 worked great on Panasonic BD30, ff, rw, chapter skips work like retail BD's which is a lot better then no fixclpi.
previous fixclpi broke playback in this player.

[MadMonkey57]

Pana BD35 latest fw + fixclpi 2.3: chapter seeking as good as without fixclpi, FF/RW seems a little "smoother" than without fixclpi.
Same as the BD30 regarding previous version of fixclpi.

[jdobbs]

Looks like we're good to go then.

I'm removing all previous versions of FixCLPI from my site. So v2.30 should be the one to use.

Thanks to all for testing and to anode for his very observant and helpful report. :)

[turbojet]

Your welcome and thanks for the fix jdobbs and for reporting the issue anode.

[MadMonkey57]

Well thank YOU jdobbs and anode.

[Elesias]

** Edited **

[laserfan]

Looks like we're good to go then.

I'm removing all previous versions of FixCLPI from my site. So v2.30 should be the one to use.

Thanks to all for testing and to anode for his very observant and helpful report. :)It appears "fixclpi" is built-in to BD-RB and not an external Tool, so can we expect you will update that shortly? It seems the new version does not work "over" BD-RB output as it stands right now. Thanks jdobbs for all the energy you put-in to this the last couple days!

[jdobbs]

It appears "fixclpi" is built-in to BD-RB and not an external Tool, so can we expect you will update that shortly? It seems the new version does not work "over" BD-RB output as it stands right now. Thanks jdobbs for all the energy you put-in to this the last couple days! Yes. You should see a new BD-RB later today that includes (internally) the updates included in FixCLPI.

[MB2]

Hi all... i have done quite a few encodes with BD-RB, and yes they dont work on the Panasonic BD35 (or the 55)

Is there anyway I can copy them back to my pc and correct them in anyway without reencoding them again? And how do i do it??

Many Thanks!

[jdobbs]

Hmmm... I'll update FixCLPI.EXE to recognize that condition and rewrite the CLPI files. I have quite a few that, even though they work for me, need to be corrected. I like things to be straight on my backups. Luckily I keep an ISO for almost all my backups on a harddrive.

[EDIT] Ok. I stand corrected. As long as they weren't done by one of the versions from the past couple of days, you can run FIXCLPI v2.30 against them and it should fix them. :)

I just did it on a couple of mine. So, no need for a new version.

Example (for APOCALYPTO):

1. Create a folder called c:\APOCALYPTO
2. Copy the root folders (BDMV and CERTIFICATE) to the new APOCALYPTO folder.
3. run "c:\path\to\fixclpi.exe" c:\APOCALYPTO
4. you'll see messages for CLPI files that are corrected
5. Reburn or rewrite to UDF 2.5 ISO or disc.

[EDIT AGAIN] Not positive if this will work for the Panasonic on previously "fixed" files, waiting for feedback.

[MB2]

EDIT!!! YOU STAR!!! Thank you will try in the morning!





Thanks for that - i have about 30 i might need to redo... eek! Just donated aswell... i encourage others to also :)
LMK when a simpleton like me can redo em all LOL

[MB2]

I tell you what ive done

Ive ripped everything using TSremux 1.8.8 movie only... stored on hd.
Then gone to mainly the v.1.9.01 of rebuilder and done quite a few...

Excuse my igorance, what do i then do? Copy the BDMV folders etc to pc and run FixCLPI.EXE

I double click on it and im not sure whats its doing...

Ive also started your new version of BDRB on 2 films - these should play ok now on the BD35/55 if im correct??

[jdobbs]

EDIT!!! YOU STAR!!! Thank you will try in the morning!





Thanks for that - i have about 30 i might need to redo... eek! Just donated aswell... i encourage others to also :)
LMK when a simpleton like me can redo em all LOLTry it once and let me know if it works on the Panasonic... I just remembered there was one change that may not make it through if the source had been "fixed" before -- but I'm not sure if it matters...

Please let me know when you find out for sure, ok?

[jdobbs]

I tell you what ive done

Ive ripped everything using TSremux 1.8.8 movie only... stored on hd.
Then gone to mainly the v.1.9.01 of rebuilder and done quite a few...

Excuse my igorance, what do i then do? Copy the BDMV folders etc to pc and run FixCLPI.EXE

I double click on it and im not sure whats its doing...

Ive also started your new version of BDRB on 2 films - these should play ok now on the BD35/55 if im correct??Run CMD.EXE and then type in the lines from the command window. Just double clicking isn't going to do anything, it is a console command.

[~bT~]

I tell you what ive done

Ive ripped everything using TSremux 1.8.8 movie only... stored on hd.
Then gone to mainly the v.1.9.01 of rebuilder and done quite a few...

Excuse my igorance, what do i then do? Copy the BDMV folders etc to pc and run FixCLPI.EXE

I double click on it and im not sure whats its doing...

Ive also started your new version of BDRB on 2 films - these should play ok now on the BD35/55 if im correct??
try the gui if u dont know cmd.

http://forum.doom9.org/showpost.php?p=1207707&postcount=2196

[jdobbs]

Can you select a directory with the GUI?

[MB2]

Ive seemed to have got it to work, i will send em to a mate today, and should know tomorrow if they worked, thanks ;-)

[~bT~]

Can you select a directory with the GUI?

yes u can.

edit: i guess u can't. i was thinking of the CLIPINF folder.

[BZeeme]

yes u can.

If I select the BDMV folder, the GUI says something like it can't find any files in that folder. I guess you have to go to each CLIPINF folder (or use CLI)

[turbojet]

With the gui even after choosing the clipinf directories I notice the packet header doesn't get checked/fixed.

It would be really nice to have a simple gui with a browse for BDMV directory or something in a future version of fixclpi all in one little exe, no need to hunt for other downloads.

[jdobbs]

With the gui even after choosing the clipinf directories I notice the packet header doesn't get checked/fixed.

It would be really nice to have a simple gui with a browse for BDMV directory or something in a future version of fixclpi all in one little exe, no need to hunt for other downloads.That's because you have to select a directory (not a file) in order to get the sizing fixed.

I'll compile a GUI version. I just wanted to make a command line version in order to keep it simple. It was never meant to become a full-up project... only a quick fix until TSMUXER is fixed. But I guess it isn't a priority with the developers.

[turbojet]

Possible to just add a gui while keeping cli?

Also I noticed file version is 2.1.0.0 in current exe.

[jdobbs]

Possible to just add a gui while keeping cli?

Also I noticed file version is 2.1.0.0 in current exe. I probably forgot to update it. It should say "v2.30" when you run it, though.

[G_M_C]

That's because you have to select a directory (not a file) in order to get the sizing fixed.

I'll compile a GUI version. I just wanted to make a command line version in order to keep it simple. It was never meant to become a full-up project... only a quick fix until TSMUXER is fixed. But I guess it isn't a priority with the developers.

Is fixCLPI still needed, even in version 1.8.19 of tsMuxeR ?

(http://forum.doom9.org/showthread.php?p=1256299#post1256299)

[jdobbs]

I don't know. I'll run a mux with .19 and see...

[G_M_C]

I don't know. I'll run a mux with .19 and see...

YEah, hopefully it is fixed now :) But at least the TrueHD problems seems to have been fixed !

Speaking of fixed problems: Thanx Jdobbs for fixing the problems we saw with fixCLPI and the Panny's BD30/BD35 :)

[jdobbs]

Yep, still needed. The packet_size parameter was set correctly on the one I just tested, so apparently that was fixed, but the ep tables (COARSE/FINE) are still wrong.

[mouw]

new FixClip v2.3 seems smoother w/ PowerDVD v7.3

however on my Sony BDP-S350 maybe a little FASTER on Chapter jumps
but Sound still a couple seconds behind video at jump
and FF/RW really doesn't work

FixClipGUI works just find in Vista
just copy FixClip v2.3 (replace old ver) into directory w/FixClipGUI
and use a ShortCut to run program...

i only FIX the Clipfile in BDMV\CLIPINFO
never bother with BDMV\BACKUP\CLIPINFO
any1 think this is a problem??

[Duppie]

jdobbs

Does your fixclpi work on AVCHD format for PS3. I have noticed you have mentioned only Blu-Ray structure?

Thanks

[deank]

Most of the time I'm getting 2 error messages


D:\multiAVCHD>"c:Users\Dean Kasabow\Desktop\multiAVCHD\tools\fixclpi.exe" AVCHD_testpip

FixCLPI v2.30, jdobbs softworks

Fixing CLPI files in directory tree.
- AVCHD_testpip\BDMV\BACKUP\CLIPINF\00000.clpi ep tables corrected.
- AVCHD_testpip\BDMV\BACKUP\CLIPINF\00001.clpi ok.
Target does not appear to be a CLPI file.
- AVCHD_testpip\BDMV\BACKUP\CLIPINF\01300.clpi ok.

After 01300.clpi I get runtime error 5 "File already open" and I get the same error with all files with names above 01000.clpi. (my menu files) and the files are not open in any other application.

[mrr19121970]

I got a Run-time error '6': Overflow

[mrr19121970]

With the gui even after choosing the clipinf directories I notice the packet header doesn't get checked/fixed.

It would be really nice to have a simple gui with a browse for BDMV directory or something in a future version of fixclpi all in one little exe, no need to hunt for other downloads.


Try this...

fixCLPI GUI v0.02b.exe (http://clownbd.techxt.com/Downloads/fixCLPI GUI v0.02b.exe)

Includes v2.31

[jdobbs]

Most of the time I'm getting 2 error messages


D:\multiAVCHD>"c:Users\Dean Kasabow\Desktop\multiAVCHD\tools\fixclpi.exe" AVCHD_testpip

FixCLPI v2.30, jdobbs softworks

Fixing CLPI files in directory tree.
- AVCHD_testpip\BDMV\BACKUP\CLIPINF\00000.clpi ep tables corrected.
- AVCHD_testpip\BDMV\BACKUP\CLIPINF\00001.clpi ok.
Target does not appear to be a CLPI file.
- AVCHD_testpip\BDMV\BACKUP\CLIPINF\01300.clpi ok.

After 01300.clpi I get runtime error 5 "File already open" and I get the same error with all files with names above 01000.clpi. (my menu files) and the files are not open in any other application.Interesting. I've done a lot of testing and haven't had issues. I also ran it against CLPI files that have high numbers (e.g. "20001.CLPI"). As for the "Target does not appear to be a CLPI file." -- it is designed specifically for TSMUXER output, that means the header on the file has to be "HDMV0200" -- it's possible that some AVCHD files may have "HDMV0100" (if something has patched it after TSMUXER). I guess I could go back and change it so it accepts both.

I'll check and make sure the files are closed... it's possible I may have left something open when I get an error.

[Edit] Yep, just checked. I don't close the file before exiting the module when I give the "Target does not appear to be a CLPI file." error. Bad programming... I'll fix it and make a new release. I'll also allow "HDMV0100" as the header.

[deank]

Thanks. And - yes - those with 'not valid' error are with 0100 header. :)

[MB2]

OK, my mate has had a chance to test some discs i sent him...

2 of which were done on earlier versions of BDRB but run with FIXCLPI - They dont work, have sound tho.

I did also Predator 2 with the new version of BDRB 0.20.1 - and that doesnt work :-(

[jdobbs]

OK, my mate has had a chance to test some discs i sent him...

2 of which were done on earlier versions of BDRB but run with FIXCLPI - They dont work, have sound tho.

I did also Predator 2 with the new version of BDRB 0.20.1 - and that doesnt work :-( Use the new version that will be posted in a few minutes. Can't make any recommendations on the BDRB 0.20.1 issue. I have reports from those with Panasonic that say it works ok now. Try the new version that will be posted in a few minutes.

[rack04]

Use the new version that will be posted in a few minutes. Can't make any recommendations on the BDRB 0.20.1 issue. I have reports from those with Panasonic that say it works ok now. Try the new version that will be posted in a few minutes.

I can confirm. My BD35 plays flawlessly now.

[jdobbs]

I've compiled a new version of FixCLPI.EXE (v2.31). You can download it from this link (http://www.jdobbs.net/freeware/fixclpi_v231.zip). Updates to this one:

- Added code to recognize issues in previously "fixed"
CLPI files, and correct.
- Corrected an error in that could result in "file
already open" error in directory mode.
- Added "HDMV0100" as an acceptable header for the
identification of valid CLPI files.

[MB2]

His player is the BD55. Sorry, should have said.

Will try the new fixclpi

[MB2]

Just as a side note, what version of TSremux should i be using? Have been using 1.8.8b

[jdobbs]

I have been using 1.8.4 because it is that last "public" release. But I think 1.8.8 is ok. I've had issues reported with BDRB with the newer ones (1.8.18 & 1.8.19), causing me to move back to 1.8.4 in my release -- but can't confirm them myself.

[MB2]

So in theory,

Having stripped Predator 2 using TSremux 1.8.8b, then doing rebuilder in v.0.20.1 should this disc have worked (in the BD55) without the need for fixclpi?

[jdobbs]

If you preprocessed the original with TSMUXER before running BDRB than I can't guarantee anything. BD-RB is meant to be run against original sources (excluding AnyDVD processing, since that has been thoroughly tested).

I can say that I've seen problems with TSMUXER processing sources that were previously run through TSMUXER. So at best I can say its unpredictable.

[MB2]

Is there anything youd suggest i do to the source created by TSremux before i start with BDRB... I must have well over 120 on HD's (all movie only using TSremux). Would doing the FIXCLPI before help then running BDRB?

[jdobbs]

I wish I could help, but I'm not even sure what the issue is. Here's an example: If I take a multipart video that has been encoded, mux it into several M2TS files, and then try to use those as a source for a multipart mux with a single audio stream, it will be way out of sync. If I do the exact same mux, but use the exact same .264 files that were used to create the M2TSs, it will work correctly with perfect sync. The only difference is the fact that in the first method I am remuxing from previously TSMUXER created M2TS files. I've had other issues as well. In fact, I've tried, in the past, to demux to elementary streams so I can start over again from scratch and even that didn't even seem to work.

Sorry. But all I can say is that TSMUXER seems to be fine as long as it isn't working with a source it created itself. Either that, or I've just run into some uncommon strangeness. The example I used above, though, forced me to rewrite part of BD-RB because it was happening to others too.

If those HDs are in blu-ray formant you use FixCLPI on them, they should be corrected and be fine. I just wouldn't run them through BD-RB... since it is going to use TSMUXER again.

Just so you know, it is my intention to eventually build my own muxer for BD-RB, and at that point maybe this will become a non-issue.

[setarip_old]

@jdobbsIf you preprocessed the original with TSMUXER before running BDRB than I can't guarantee anything. BD-RB is meant to be run against original sources (excluding AnyDVD processing, since that has been thoroughly tested).

I have been using "MakeMKV" (to decrypt and convert to MKV) instead of "AnyDVD HD" and then "TSMuxer" (To create a proper BluRay movie-only "package") for all of my BluRay ripping. I have then successfully, without incident) used BD-RB to compress several of them to fit on DVD-9s (Notwithstanding the difficulty I recently encountered regarding the small clip from "The Santa Clause 3")...

[mrr19121970]

I got a Run-time error '6': Overflow


I still get this error with V0.231.

Here's the offending file:

http://www.megaupload.com/?d=N8YJAOBT

[jdobbs]

I still get this error with V0.231.

Here's the offending file:

http://www.megaupload.com/?d=N8YJAOBTThanks for providing the file. I ran against it and got the same error. The PTS value was wildly high and caused an overflow when I used the upper bits in a calculation. I'll have to figure out a way to fix it -- because even though the value is exceptionally unusual, it is completly legal.

[~bT~]

Try this...

fixCLPI GUI v0.02b.exe (http://clownbd.techxt.com/Downloads/fixCLPI GUI v0.02b.exe)

Includes v2.31
:thanks: mr

[shon3i]

jdobbs tsMuxeR_1.8.20(b) is out, can you check please is there same problem?

[jdobbs]

I just tried v1.8.20(b) The packet count variable did not need correction in the blu-ray output I tested for this version, but the COARSE/FINE tables still needed to be corrected.

[laserfan]

I just muxed something with 1.8.20, my "thing" being accurate chapter marks (seeks, landings, whatever you want to call skipping-ahead accurately by chapter) and the resulting mux had a few issues. Applying fixclpi 2.31 made those issues go away, so I'm glad to find out that fixclpi still works despite that it wasn't developed with 1.8.20 output.

I will say tho also that the problems were with only a few chapter marks out of 38. And also, I muxed another program the other day where fixclpi was not needed at all i.e. the chapter landings worked fine (I applied fixclpi anyway).

So I don't know what's up with tsmuxer in that sometimes it's good and sometimes it's not--I might have guessed that if the 00001.clpi file had problems, you'd always be able to see them on playback but apparently not.

[jdobbs]

Some very small encodes may not need it... but pretty much every one I've tested does...

It's a really easy fix... I hope they get to it soon.

[psme]

Thanks for the great effort. But the fixCLPI GUI does not work with 8.3 filename. Took me a while to notice and I need to change back all those old encoding in PS3 format, back to blu-ray file naming to apply the new fix...

regards,

Li On

[jdobbs]

I don't think I understand what you mean?

[mrr19121970]

I guess he means you're looking for *.CLPI, whereas he has *.CPI after running AVCHMe.

[jdobbs]

Hmm... but is that BD compatible? It sure doesn't sound like it.

[deank]

Right... BD is .clpi/.mpls/.m2ts/.bdmv (indx/mobj/hdmv 0200), AVCHD is .cpi/.mpl/.mts/.bdm (indx/mobj/hdmv 0100).

[mrr19121970]

@ JDobbs.

Can you please look at these 2 files:

Original (http://clownbd.techxt.com/Downloads/While She Was Out/Original/00000.clpi)

tsMuxeR-fixCLPI (http://clownbd.techxt.com/Downloads/While She Was Out/tsMuxeR-fixCLPI/00001.clpi)

In the original you can jump chapters & fast forward etc. However in tsMuxeR (every version) jumping chapter or ffwding just takes you back right to the start.

Any ides ?

Thanks.

[jdobbs]

Where did these come from? The first one is a legitimate CLPI. The second one looks trashed and only has 7 COARSE and 7 FINE entries. It wasn't created by FixCLPI -- because I just ran FixCLPI against the first one and it found nothing to change (and left it intact). Also, the second one looks like it has nothing in common with the first one (completely different SPN and PTS values)

[nwg]

Is the m2ts packet size to match the clipinf size?

[jdobbs]

Is the m2ts packet size to match the clipinf size? Was this directed at me? If so, I don't think I understand the question.

[nwg]

Was this directed at me? If so, I don't think I understand the question.

It was a general question but I hoped you would know.

After fixclpi the clipinf and m2ts packet size are different. The prog by mrr19121970 makes them the same using his clipinf editor.

so are they supposed to be the same or different as your fixclpi prog and his does them differently.

http://www.apah20.dsl.pipex.com/pics/clipinf.jpg

[deank]

jdobbs' fixclpi fixes different things, not m2ts packet information stored in clpi files.

[jdobbs]

Actually it does fix the packet count... if you use a folder as the input parameter and it finds a correspoinding "..\STREAM\" entry in the path. But in the CLPI files posted there is no way of knowing if they match, because I don't have the size of the M2TS to compare.

[nwg]

jdobbs' fixclpi fixes different things, not m2ts packet information stored in clpi files.

But they interfere with each other and change each other. If I use that clipeditor to fix the length so they both are the same as what m2ts is then re-run fixclpi, the clipinf packet size is back to what is was before. Just want to know if both programs are needed.

I using the fixclpi gui 0.02 and using the Blu Ray folder as input.

[mrr19121970]

Where did these come from? The first one is a legitimate CLPI. The second one looks trashed and only has 7 COARSE and 7 FINE entries. It wasn't created by FixCLPI -- because I just ran FixCLPI against the first one and it found nothing to change (and left it intact). Also, the second one looks like it has nothing in common with the first one (completely different SPN and PTS values)

The 1st one is from the original disc, and the 2nd one from tsMuxer (1.8.22) and fixCLPI (2.31)

FixCLPI v2.31, jdobbs softworks

Fixing CLPI files in directory tree.
- D:\DEMUX\While She Was Out\While She Was Out\BDMV\BACKUP\CLIPINF\00001.clpi packet count fixed
- D:\DEMUX\While She Was Out\While She Was Out\BDMV\BACKUP\CLIPINF\00001.clpi ok.
- D:\DEMUX\While She Was Out\While She Was Out\BDMV\CLIPINF\00001.clpi packet count fixed
- D:\DEMUX\While She Was Out\While She Was Out\BDMV\CLIPINF\00001.clpi ok.


I'll run it through tsMuxeR again (without fixCLPI)

[jdobbs]

If you are using one of the newer versions of TSMUXER, the value is correct and shouldn't be changed (at least on single M2TS muxes).

[jdobbs]

The 1st one is from the original disc, and the 2nd one from tsMuxer (1.8.22) and fixCLPI (2.31)

FixCLPI v2.31, jdobbs softworks

Fixing CLPI files in directory tree.
- D:\DEMUX\While She Was Out\While She Was Out\BDMV\BACKUP\CLIPINF\00001.clpi packet count fixed
- D:\DEMUX\While She Was Out\While She Was Out\BDMV\BACKUP\CLIPINF\00001.clpi ok.
- D:\DEMUX\While She Was Out\While She Was Out\BDMV\CLIPINF\00001.clpi packet count fixed
- D:\DEMUX\While She Was Out\While She Was Out\BDMV\CLIPINF\00001.clpi ok.


I'll run it through tsMuxeR again (without fixCLPI) Well the tables won't necessarily match because the SPN values will almost undoubtedly change, and since TSMUXER resets the PTS baseline, the PTS values won't match either. Let's see what the CLPI looks like just from TSMUXER (no FixCLPI) -- as that is more likely to be the source of the problem. FixCLPI just uses the values in the table and corrects the missing COARSE entries related to SPN roll-over.

If the problem is thought to be FixCLPI, I would need to see the same CLPI before and after FixCLPI (not the original).

[nwg]

If you are using one of the newer versions of TSMUXER, the value is correct and shouldn't be changed (at least on single M2TS muxes).

If that is to me then I used 0.21 of TsMuxer. Thank you.

[jdobbs]

I just went back and did some testing. It appears that if you do a simple blu-ray mux, the packet count is correct (at least on the three I tried), but if you use splitting, all the CLPI packet count values are still wrong even with the latest versions.

[mrr19121970]

I guess that tsMuxeR has FBR the CLPI file. Here is the original created by v1.8.22:

tsMuxeR only (http://clownbd.techxt.com/Downloads/While She Was Out/tsMuxeR/00001.clpi)

[jdobbs]

Yep. This one looks goofy also.

This is why I keep using v1.8.4 for my BD-RB muxing -- because, even though I know it isn't perfect, I know what doesn't work right consistently (and I can fix it), and I don't have to worry about new issues that are introduced.

[MB2]

@jdobbs

Which version of TSremux is inside BDRB?

[jdobbs]

v1.8.4. The last one that was publicly released by smlabs.

[mrr19121970]

tsMuxeR v1.8.4 didn't do much better:

V1.8.4 (http://clownbd.techxt.com/Downloads/While She Was Out/tsMuxeR_V184/00001.clpi)

[jdobbs]

Is there something odd about the source, or do you have the encode set to some incredibly long GOP length (keyint) or structure? That could affect the spacing of the PTS values used for the EP tables. Also, if you set the chapter values in TSMUXER to something odd, you might get something like this.

[mrr19121970]

No the source is an original BD demuxed with eac3to (While She Was Out), the video stream is VC1 1080i/50 which tsMuxeR GUI simply refuses to accept. The CLI accepts it, and detects it correctly:


D:\DEMUX\While She Was Out>"E:\TVIX\tsMuxeR\tsMuxeR_1.8.4(b)\tsMuxeR.exe" "D:\DE
MUX\While She Was Out\While She Was Out.meta" "D:\DEMUX\While She Was Out\While
She Was Out_184"
SmartLabs tsMuxeR. Version 1.8.4(b) http://www.smlabs.net
VC-1 muxing fps not set. Get fps from stream.
Decoding VC-1 stream (track 1): Profile: Advanced@3 Resolution: 1920:1080i Frame rate: 25
Decoding DTS stream (track 2): Bitrate: 1536Kbps Sample Rate: 48KHz Channels:

[laserfan]

Gosh another version 1.8.23 today...obviously with all the recent new releases there is alot of work being done on tsMuxeR at present and we can only hope I guess that Roman or whoever is working on it is paying some attention to these Doom9 threads!!??!! One might also assume they are seeing some very large number of hits on their ftp site and are aware of the outside scrutiny!

[laserfan]

I tried tsMuxeR 1.8.23 (latest as of the time of this post!) and it had one or two messed-up chapter marks/time codes and applied fixclpi 2.31 and it fixed the problems. So tsMuxeR is still broke but thankfully fixclpi still works:

C:\>fixclpi e:\video.bluray
FixCLPI v2.31, jdobbs softworks

Fixing CLPI files in directory tree.
- e:\video.bluray\BDMV\BACKUP\CLIPINF\00001.clpi packet count fixed
- e:\video.bluray\BDMV\BACKUP\CLIPINF\00001.clpi corrected.
- e:\video.bluray\BDMV\CLIPINF\00001.clpi packet count fixed
- e:\video.bluray\BDMV\CLIPINF\00001.clpi corrected.

[Visor]

I made a lot of streams a while back that had long GOPs -- I can confirm that when you use them you will get video delays at chapter points. For Blu-ray compatibility you need to make sure the GOPs are no more than one second long. In BD-RB I set them to 24 as the default. If you stick to that rule the chapters should sync up immediately.
Are you referring to --keyint 24?
Yes. And, of course, you have to also use "--min-keyint 1" along with it.

So that's it! I've been pulling my hair out all week trying to figure out why my Sony BDP-S350 was having a hard time fast-forwarding, rewinding, and chapter-jumping certain downloaded AVCHDs/BD9s. I would've assumed it was just a limitation of the player, but some other discs scanning just fine. I tried using FixClpi, remuxing with TSRemux, using h264info to write PPS... nothing worked to improve scanning or chapter response. Finally, I stumbled onto a couple of very interesting links in addition to this one:

http://forum.handbrake.fr/viewtopic.php?f=14&t=6346
http://forum.doom9.org/showthread.php?t=136327&highlight=keyint

I downloaded MediaInfo and compared slow AVCHDs vs. fast ones, and guess what? The keyints were vastly different. A fast disc would have a typical keyint of 24 and a minimum of 2, whereas a slow disc would have numbers at least 10 times that size (eg. 240/24). According to the above second link, MeGUI's "SA-Blu-ray" profile defaults to low keyint numbers, which in turn leads towards a smoother scanning & chapter jumping experience. I wish people would use that profile more often. I know it could potentially result in a small sacrifice in video quality, but I would prefer to have the same smooth scanning capabilities as an actual Blu-ray Disc.

Thanks guys! I can sleep again. :p

Visor

[G_M_C]

It's actually very simple;
GOP length in Bluray specifications is named as 1 second if you use max bitrate > 15 Mbps. GOP length is 2 sec for max bitrate < 15 Mbps. Out of this it follows that key-int is always equal to the framerate (rounded of to the nearest whole number) when bitrate is > 15 Mbps, and 2 x framerate for streams with < 15 Mbps.

In the same spec is determined that minimum key-int is always set to 1.

See also this usefull post, where i tried to centralize al these sort of questions; But sadly the thread did not "take off" http://forum.doom9.org/showthread.php?t=141376

[nwg]

So that's it! I've been pulling my hair out all week trying to figure out why my Sony BDP-S350 was having a hard time fast-forwarding, rewinding, and chapter-jumping certain downloaded AVCHDs/BD9s. I would've assumed it was just a limitation of the player, but some other discs scanning just fine. I tried using FixClpi, remuxing with TSRemux, using h264info to write PPS... nothing worked to improve scanning or chapter response. Finally, I stumbled onto a couple of very interesting links in addition to this one:

http://forum.handbrake.fr/viewtopic.php?f=14&t=6346
http://forum.doom9.org/showthread.php?t=136327&highlight=keyint

I downloaded MediaInfo and compared slow AVCHDs vs. fast ones, and guess what? The keyints were vastly different. A fast disc would have a typical keyint of 24 and a minimum of 2, whereas a slow disc would have numbers at least 10 times that size (eg. 240/24). According to the above second link, MeGUI's "SA-Blu-ray" profile defaults to low keyint numbers, which in turn leads towards a smoother scanning & chapter jumping experience. I wish people would use that profile more often. I know it could potentially result in a small sacrifice in video quality, but I would prefer to have the same smooth scanning capabilities as an actual Blu-ray Disc.

Thanks guys! I can sleep again. :p

Visor



I have been getting the same thing happen with ripbot BD structures on my S350. I too thought it was the S350 causing it. I downloaded mediainfo and loaded a ripbot example in it. It is Keyint= 250 keyint_min =25. RB Builder files are ok and ff/rw as they should.

Anyway to change this without have to reencode.

[deank]

No, unfortunately there is no way. It is good to follow the standards, but it's not the case when software developers build applications for themselves :) and users suffer from the fact.

I'm, too, glad that jdobbs rejected all "suggestions" for increasing keyint in BD-RB, just because it works for some people.

Dean

[G_M_C]

I have been getting the same thing happen with ripbot BD structures on my S350. I too thought it was the S350 causing it. I downloaded mediainfo and loaded a ripbot example in it. It is Keyint= 250 keyint_min =25. RB Builder files are ok and ff/rw as they should.

Anyway to change this without have to reencode.

Well, no way to chage that without reencoding. But one lesson learned: Don't use Ripbot for BD9/AVCHD stuff (there's no need to use it at all, "DIY" is always better as you have more control over results; But yes, you'll have to get used to scripting Avisynth and using CLI's).

[DreckSoft]

I know this in a very old thread. But I have some issues with MultiAVCHD BDs. It uses tsMuxeR 1.10.6 and SOME of the BDs won't work properly. Skipping to the next chapter either does not work at all or takes ages. If I mux the same files with a recent version of tsMuxeR it works. However, then my player does not recognize (non-standard) EAC3 audio so I'm kinda stuck with the old version.

fixclpi won't change anything on the file it says OK for all. Can someone tell me where exactly the issue was so I can check if I find something in the newer files?