TSPE 0.700 Public Beta out now!

[Ventolin]

Hi folks,

After a couple years of quiet development in the darkest corners of the Internet, another public beta of TSPE is now available!

http://www.bitstreamtools.com

For those that don't know, TSPE (Transport Stream Packet Editor) is a non-linear Editor / Analyser.

So, what's new...

Supported Formats:
MPEG2 / H.264 / VC-1 / AC-3 / DDP / MLP / DTS / DTS-HD / DTS-HD MA / AAC / MPEG1 Audio / MPEG2 Audio / PCM (everything? :)

.TS and .M2TS support including stripping the m2ts header to make normal 188 byte .TS files.

No re-encoding of the stream to preserve maximum quality, I frame to P frame accurate, clean video/audio editing, timestamp and table correction.

You can cut commercials from captured sources, detect commercials using format changes, scan for errors, import TSPE and M2R logs, fix broken streams including timestamp wraparounds, continuity errors, timestamp gaps, realign to start codes and resync audio.

Stream stripping with table correction, PCR remapping, Demuxing, Core Audio from HD audio extraction.

Video preview is much more accurate, uses your own directshow filters, test and full edits are previewable with TSPE's own internal viewer, multi-part ts file joining with autosorting EDL (Edit decision List). Marker List to enable bookmarking of positions.

Packet Header Decode for analysis. Resizable interface.

Plenty more too, visit the website for details.

TSPE is now shareware, free trial period is 45 days. I wanted to release TSPE once I was sure it performed well enough to justify payment. It is still in beta testing, but most of the major bugs have been ironed out (I hope!). Still plenty more work to be done, eg. m2ts and vc1 support is still sketchy, but it does do the job just about.

Please let me know how you get along; of course I would like to know your feedback good or bad and especially if you have some streams which don't work properly.

Have fun,

Regards,

Vent

[Atak_Snajpera]

Those two ts samples do not work (no preview)
http://www.mediafire.com/?sharekey=fc577931a2d88b41d2db6fb9a8902bda

One more thing. Instead of small 'loading' window I would use window with graphic (splash screen)

[Ventolin]

Those two ts samples do not work (no preview)
http://www.mediafire.com/?sharekey=fc577931a2d88b41d2db6fb9a8902bda

One more thing. Instead of small 'loading' window I would use window with graphic (splash screen)
Hi,

This is a snippit of the status log when you try to load either of the two .ts files:

Opening: X:\tmp\Batman-5.TSSplit.1-57.ts
Opening File...
Auto Finding PIDs
Unable To Find PAT

Both streams do not have a PAT. The PAT is essential for a Transport stream and there should be one every half second or so. A transport stream should have at least 1 PAT with PID 0, 1 PMT and 1 stream (which can be video or audio).

Try loading the full source into TSPE instead.

Thanks for the note about the splash screen, I haven't gotten around to doing one yet, same with an icon. These will be added in due course, once I've learnt a bit about graphic design :)

Cheers,

Vent

[mediator]

many real life streams don't come with PAT (e.g. captures from DVB, seems often the PAT is just stripped during the recording process). Maybe you should consider supporting them anyway, even if they don't strictly comply with ISO.

[Ventolin]

many real life streams don't come with PAT (e.g. captures from DVB, seems often the PAT is just stripped during the recording process). Maybe you should consider supporting them anyway, even if they don't strictly comply with ISO.

PAT and PMT really are essential to a Transport stream. If a capture program strips them out of a stream, then the capture program is at fault. TSPE would then need to repair the stream to fix the poor design of the capture program. What real life streams have you come across without PAT and PMTs? If a specific capture program is stripping out the PAT and PMT perhaps the authors could fix their software?

The PAT and PMT tell the decoder what is in the transport stream. The PAT gives a list of the Programmes available in the form of PIDs for PMTs. Once the PID of the PMT is known, the PMT can be read to find out the what PIDs, types and timing information (PCR) are of the streams that make up that programme. When capturing a complete mux you may have *hundreds* of streams in the transport stream. How is anything supposed to know what streams belong to what programme if you dont have the PAT and PMTs?

Nevertheless, in the most basic mode TSPE *still* works even without the PAT and PMT. If you are able to find a working directshow filter setup (i.e. one that works in graphedit) you can then have video preview. TSPE will still allow you to set start and end points and make an edit if you turn the fix options off with or without the video preview.

Since I couldn't get any filter combination to work in graphedit, I couldn't test what happens with TSPE completely, but I did edit a section out successfully nonetheless.

Full table editing (for PAT and PMT) is already on the TODO list however. Once that is implemented it will allow TSPE to repair streams without PAT or PMTs. Perhaps I could get TSPE to launch an external viewer like MPC instead of it's internal viewer for test edits in the mean time. I could code that in if there was enough demand...

To edit these streams "blind" switch to the decode window (using the View) button. Then use the decoded PTS/DTS timestamps as a guide to the real timecode of the stream. Then you can navigate to test out positions. Make sure you turn off Fix Tables and Fix Timecodes and turn off the internal viewer from the menu options. You can then make test and full edits and preview the filename.test.ts or filename.edit.ts files generated with something like MPC. Scanning the file revealed a timestamp gap of 49 seconds or so at the start in the Batman sample; the scan dropped a marker at the end of the gap so you can navigate there easily.

Hope that helps,

Regards,

Vent

[canTsTop]

hello,
i have this sample http://www.megaupload.com/?d=5A68THW9 is it possible to fix it so i have audio / video synch after encode, and how?

[Atak_Snajpera]

once I've learnt a bit about graphic design
I can create something for you for free.

Another thoughts:
1) Main form should appear in center of the screen
2) Movie slider should start at first decodable key frame instead of P or B frame

I forgot to say. Excellent job!

i have this sample http://www.megaupload.com/?d=5A68THW9
Are you joking ? :) This sample is like wooden wall destroyed by Minigun. Too many holes :) ...

[Ventolin]

hello,
i have this sample http://www.megaupload.com/?d=5A68THW9 is it possible to fix it so i have audio / video synch after encode, and how?

Woah, this is a glitchy sample!

After loading into TSPE, do a scan. This will reveal:

Continuity Errors: 764, Transport Error Indicator Errors: 0
Timestamp gaps: 16, PCR gaps: 7
Total number of separate discontinuous regions: 655

The Marker List will now contain alot of entries. Looking through there are 4 points which we can use:

A. First entry is at offset 1186092 (5 seconds in marker list)
B. Last entry is at offset 5670456 (24 seconds in marker list)
Gap in the middle between:
C. offset 2977168 (13 secs)
D. offset 5000236 (22 secs)

Every other marker entry is too close to each other to be of use, the biggest gap being about 3 seconds, with not much useable inbetween. Note that the marker list time values aren't quite accurate, once the marker is double clicked a time based on the nearest timestamp in the stream will be used (assuming timestamp mode is running which it will be by default). The byte position is the only thing we should be concerned with.

So, we need to make 3 cuts:
from start to point A.
from point C to point D.
from point B to the end

So:
1. double click the first entry, then click "Set" under End Point
2. Click on EDL tab to switch to EDL view
3. Click Add under EDL view (first entry will appear)
5. Click on Markers tab
6. Scroll to point C in Markers (look for offset 2977168 (13 secs)
7. Double click entry - this moves current position to that offset
8. Click Set under Start Point
9. Scroll to point D (offset 5000236 (22 secs))
10. Click Set under End Point
11. Click on EDL Tab to switch to EDL view
12. Click Add under EDL view (second entry will appear)
13. Click on Markers tab
14. Scroll down to the bottom of the list to the last entry (offset 5670456 (24 seconds in marker list) )
15. Double click last entry
16. Click on Set under the Start Point
17. Scroll to the very end of the video with the trackbar and using >> to make sure you're at the very end
18. Click on Set under End Point
19. Click on EDL tab
20. Click on Add under EDL view - 3rd entry should appear.

Lots of steps to create a 3 entry EDL, but it's simpler once you do it.

The EDL should now have something like this:
00:00:00.00 0 00:00:03.80 1186280
00:00:15.08 2977168 00:00:21.96 5000424
00:00:29.79 5670456 00:00:46.01 10440580

If you click Edit now, you will generate a playable, non-crashing clip. The audio cuts out a bit too much though but stays in sync.

To try and improve the audio, Uncheck Menu Edit->Clean Edit Point->Clean Audio Frames. This cuts audio at byte positions instead of looking at timestamp positions from the video stream at the expense of possible broken audio packets.

Now clicking edit produces a playable stream with better audio and we're done!

I hope the rest of the stream is clean, if you have that many errors all over the stream, it's really too damaged to be of much use.

If you're going to be re-encoding from this source, you'll have to do some more experimentation, but from a playback point of view, the edited clip is in sync.

Good Luck!

Regards,

Vent

[Flaarn]

Thanks for this ventolin, checking it out on a couple of h264 transport streams now, bbc hd and channel 4 hd.
The big issue for me has always been frame accurate cutting so that no extra 2.0 audio frames are present if the main broadcast is 5.1 but adverts are 2.0, many conversion tools tend to throw a wobbly when there are audio changes, more so when the transport stream contains some 2.0 ac3 frames at the start, sure it can be patched when parsing, but it adds another step in the process which in my opinion is a nuisance, with this new release I'm hoping that I can bypass this step and have a clean ts to play with, thanks again for your ongoing work.

[Ventolin]

I can create something for you for free.
Seriously? Thanks! I can create one for free too :P but if you really want to have a go, we'd be happy to look at it :)

Another thoughts:
1) Main form should appear in center of the screen
2) Movie slider should start at first decodable key frame instead of P or B frame
Thanks for your suggestions, I'll think about 1) but might be confusing with multiple instances. 2) Slider should be at the very start of the stream as this is a *packet* level editor and analyser - the PAT usually is at the first packet!

I forgot to say. Excellent job!
Thank you kindly!

Are you joking ? :) This sample is like wooden wall destroyed by Minigun. Too many holes :) ...
LOL, but still possible to get something out, see above :)

Cheers,

Vent

[Ventolin]

Thanks for this ventolin, checking it out on a couple of h264 transport streams now, bbc hd and channel 4 hd.
The big issue for me has always been frame accurate cutting so that no extra 2.0 audio frames are present if the main broadcast is 5.1 but adverts are 2.0, many conversion tools tend to throw a wobbly when there are audio changes, more so when the transport stream contains some 2.0 ac3 frames at the start, sure it can be patched when parsing, but it adds another step in the process which in my opinion is a nuisance, with this new release I'm hoping that I can bypass this step and have a clean ts to play with, thanks again for your ongoing work.
Hi,
I've found the audio switchover point to be less than accurate on the broadcaster's end which means that you might chop off the start of the audio at the start of a segment. I hate this alot as my offboard processor will mute for a second at the switchover point. I did complain to the BBC some time ago, they tried to fix it but screwed things up alot and promptly gave up.

Hope you find TSPE useful though, look forward to your feedback.

Regards,

Vent

[Atak_Snajpera]

2) Slider should be at the very start of the stream as this is a *packet* level editor and analyser - the PAT usually is at the first packet!
I'm not saying about discarding everything before first keyframe.
Slider should be rewinded to first key-frame by default. See screenshot
http://img210.imageshack.us/img210/4282/new1ep5.th.png (http://img210.imageshack.us/my.php?image=new1ep5.png)

I see no reason to cut stream at P or B frame. Only cutting at key frame maintains audio synchronization if we want to transcode file further. This is main reason why I use this tool.

[canTsTop]

Woah, this is a glitchy sample!

After loading into TSPE, do a scan. This will reveal:

Continuity Errors: 764, Transport Error Indicator Errors: 0
Timestamp gaps: 16, PCR gaps: 7
Total number of separate discontinuous regions: 655

The Marker List will now contain alot of entries. Looking through there are 4 points which we can use:

A. First entry is at offset 1186092 (5 seconds in marker list)
B. Last entry is at offset 5670456 (24 seconds in marker list)
Gap in the middle between:
C. offset 2977168 (13 secs)
D. offset 5000236 (22 secs)

Every other marker entry is too close to each other to be of use, the biggest gap being about 3 seconds, with not much useable inbetween. Note that the marker list time values aren't quite accurate, once the marker is double clicked a time based on the nearest timestamp in the stream will be used (assuming timestamp mode is running which it will be by default). The byte position is the only thing we should be concerned with.

So, we need to make 3 cuts:
from start to point A.
from point C to point D.
from point B to the end

So:
1. double click the first entry, then click "Set" under End Point
2. Click on EDL tab to switch to EDL view
3. Click Add under EDL view (first entry will appear)
5. Click on Markers tab
6. Scroll to point C in Markers (look for offset 2977168 (13 secs)
7. Double click entry - this moves current position to that offset
8. Click Set under Start Point
9. Scroll to point D (offset 5000236 (22 secs))
10. Click Set under End Point
11. Click on EDL Tab to switch to EDL view
12. Click Add under EDL view (second entry will appear)
13. Click on Markers tab
14. Scroll down to the bottom of the list to the last entry (offset 5670456 (24 seconds in marker list) )
15. Double click last entry
16. Click on Set under the Start Point
17. Scroll to the very end of the video with the trackbar and using >> to make sure you're at the very end
18. Click on Set under End Point
19. Click on EDL tab
20. Click on Add under EDL view - 3rd entry should appear.

Lots of steps to create a 3 entry EDL, but it's simpler once you do it.

The EDL should now have something like this:
00:00:00.00 0 00:00:03.80 1186280
00:00:15.08 2977168 00:00:21.96 5000424
00:00:29.79 5670456 00:00:46.01 10440580

If you click Edit now, you will generate a playable, non-crashing clip. The audio cuts out a bit too much though but stays in sync.

To try and improve the audio, Uncheck Menu Edit->Clean Edit Point->Clean Audio Frames. This cuts audio at byte positions instead of looking at timestamp positions from the video stream at the expense of possible broken audio packets.

Now clicking edit produces a playable stream with better audio and we're done!

I hope the rest of the stream is clean, if you have that many errors all over the stream, it's really too damaged to be of much use.

If you're going to be re-encoding from this source, you'll have to do some more experimentation, but from a playback point of view, the edited clip is in sync.

Good Luck!

Regards,

Vent

Thank You for answer.

This sample intentionally so damaged :)

If i correctly understand this method cuts out damaged parts off file. I did exactly as you described (with Clean Audio Frames unchecked) and it kind off worked (DGAVCIndex didn't crash). But if i again open already edited (fixed) TS file in TSPE, scan, and there is Continuity Errors: 17. It would be good to have some kind of automation of this method :) if for example there is 3 hours recording where we need to do ~20 cuts :)


Is this is only way to fix damaged TS files? Maybe something like ProjectX, it replaces damaged frames with empty frames?

[Ventolin]

I'm not saying about discarding everything before first keyframe.
Slider should be rewinded to first key-frame by default. See screenshot
http://img210.imageshack.us/img210/4282/new1ep5.th.png (http://img210.imageshack.us/my.php?image=new1ep5.png)

I see no reason to cut stream at P or B frame. Only cutting at key frame maintains audio synchronization if we want to transcode file further. This is main reason why I use this tool.
Lets take the following two scenarios:

TV capture:
When I capture a TV programme I like to start capturing a few minutes before the start of the actual programme itself - that way I won't miss the very start if it's broadcast early. Then when I load the captured .ts file into TSPE I look for the start of the programme. This almost never is at the first I frame, but usually a minute or two into the file. So I dont really need to look at the first I frame. However, I might be interested at looking at the PAT and PMTs which could be at the very start of the stream.

BD analysis:
Suppose I want to look at a stream from a BD. Again the first few packets are the PAT/PMT, followed by an I-Frame for the start of the video. In this situation, I am already at the point I want to edit with.

So, I think in most situations starting from the very beginning of the stream is correct?

I guess the only time I'd want the very first I frame is if I have missed the start of the captured programme, in which case it's a simple case of navigating to the first I frame.

In any event, it's certainly possible to make sure the edit always starts with an I frame so you don't have to look for it manually. I'll look into that possiblity further, thanks :)

Regards,

Vent

[Ventolin]

Thank You for answer.

This sample intentionally so damaged :)

If i correctly understand this method cuts out damaged parts off file. I did exactly as you described (with Clean Audio Frames unchecked) and it kind off worked (DGAVCIndex didn't crash). But if i again open already edited (fixed) TS file in TSPE, scan, and there is Continuity Errors: 17. It would be good to have some kind of automation of this method :) if for example there is 3 hours recording where we need to do ~20 cuts :)


Is this is only way to fix damaged TS files? Maybe something like ProjectX, it replaces damaged frames with empty frames?
I think you are looking for similar functionality to Mpeg2Repair where bad mpeg2 frames are repaired. Whilst this is possible, it's certainly going to take some time to implement, especially as M2R is such an awesome tool and I'm still developing my c0dering skill0rz :) It would require a detailed knowledge of mpeg2/h264/vc1 encoding.

For playback purposes, having minor glitches in place is preferable in my opinion to missing a chunk of the programme.

TSPE actually does do some repair automatically eg. timestamp gaps, so for some glitches it is automated. It all depends on the type of glitch and severity.

Cheers,

Vent

[JonRead]

Thanks, I will test out with UK satellite broadcast HD h.264 streams and let you know if I find any bugs.

[robinsj]

In your editor is there a way to manually enter a time you want to go to in a video?

[Ventolin]

Thanks, I will test out with UK satellite broadcast HD h.264 streams and let you know if I find any bugs.

Hi, Please do let me know how you are getting on!

Vent

[Ventolin]

In your editor is there a way to manually enter a time you want to go to in a video?

I'm just going to add that in right now, this very minute! Well OK, maybe the next minute or two as I'm still typing :)

Cheers,

Vent

[robinsj]

Okay another question.... I am trying to edit a few movies that I have that my wife really likes, but she really hates hearing the F-word in movies.... So I am trying to use your software to edit them out. (not really interested in getting into an arguement about editing a movie to take out language, so please leave that alone to whomever is interested in arguing)

So what is the best way to do it..... Here is what I have tried to do... I try playing the movie in the preview window, and then I just hit pause right before he says it and mark that as the end, then add that to the edits. Then I play it again until it has passed that he said it, usually only is like half a second or so... and then mark that as the beginning, and then go all the way to the end of the movie and mark that as the end and then add that to the edits. So basically I have a very small portion that is maybe 15-25 frames long that is being cut out. Well, then when I edit it and go to play it, it seems it totally misses the swear word and cuts something else out within frames from where i was trying to cut. Does this have to do with the PID's? Or the I-Frames? I tried unselecting Fix tables and timecodes, and that seemed to work, but then all I got was digital snow instead of just a clean cut...??

Any Help?

[Ventolin]

Okay another question.... I am trying to edit a few movies that I have that my wife really likes, but she really hates hearing the F-word in movies.... So I am trying to use your software to edit them out. (not really interested in getting into an arguement about editing a movie to take out language, so please leave that alone to whomever is interested in arguing)

So what is the best way to do it..... Here is what I have tried to do... I try playing the movie in the preview window, and then I just hit pause right before he says it and mark that as the end, then add that to the edits. Then I play it again until it has passed that he said it, usually only is like half a second or so... and then mark that as the beginning, and then go all the way to the end of the movie and mark that as the end and then add that to the edits. So basically I have a very small portion that is maybe 15-25 frames long that is being cut out. Well, then when I edit it and go to play it, it seems it totally misses the swear word and cuts something else out within frames from where i was trying to cut. Does this have to do with the PID's? Or the I-Frames? I tried unselecting Fix tables and timecodes, and that seemed to work, but then all I got was digital snow instead of just a clean cut...??

Any Help?

Small edits are hard to deal with as what you really want is frame accurate video. TSPE can only do I frame for the start point and P frame for the end point, so you may find your edits do not line up exactly where you want them.

However, as TSPE can process at the packet level, there is a way to edit just the audio out but leave the video intact.

This isn't so easy to do however, and will take some experimentation to get to know how to do it easily. Level of difficulty for this is 5/10.

Lets take 1 swear word as an example. You will have 3 edits:

1. The start of the movie to the beginning of the swear point (section A)
2. The swear point (section B)
3. The end of the swear point to the end of the movie (section C)

To do audio editing, set the "Frame Step" mode to be audio instead of Video (bottom left hand corner). This makes the F+ and F- buttons jump to audio frames instead of video frames.

Now do the following:
Setup section A:
1. Set the start point to the start of the movie.
2.Set the end point to the start of the swear word.
3. Click Add to add this section to the EDL.

Setup section B (the swear word):
4. Set the start point to be the start of the swear word.
5. Set the end point to the be end of the swear word.
6. In the PID list, uncheck the audio stream. (This strips out the audio stream for section B)
7. Click Add to add this section to the EDL.

Setup section C (from the end of the swear word to the end of the file)
8. Set the start point to the end of the swear word.
9. Set the end point to the end of the file.
10. In the PID list, re-check the audio stream.
11. Click Add to add this section to the EDL.

To complete the edit:
12. Turn off "Clean Video" and "Clean Audio" under "Clean Edits" from the menus.
13. Click the Edit button.

This should now copy everything up to the swear word intact, then strip the audio only for the swear word itself, then copy everything after the swear word intact.

As long as you play

[rebkell]

I'm trying to understand the PTS stamps, I have a common problem I'm encountering with the Hauppauge HDPVR unit, it has audio dropouts and then it resets and basically just starts a new recording at that spot in the stream, unfortunately the video usually continues for a half a dozen frames or so, before it restarts. I can pretty much cut out the bad place and maintain sync, I know exactly what I want to cut out, but I can't seem to match up the timestamps(PTS) in xport(which are exact matches for what are exactly in the ts file) with the ones you show.

For example the first PTS of the ts file is always
43843, but according to TSPE the first PTS is:
00:00:00:14 (29) 487 .. how to I match these two PTS stamps up? I'm not seeing a correlation between the two, I tried changing from PCR to PTS based in the Options->Timestamp Mode, but I didn't see any difference.

Upon looking I do see how they correlate, 487 is the ms value of 43843, is there anyway I can get the raw 43843 to show in TSPE

[Ventolin]

I'm trying to understand the PTS stamps, I have a common problem I'm encountering with the Hauppauge HDPVR unit, it has audio dropouts and then it resets and basically just starts a new recording at that spot in the stream, unfortunately the video usually continues for a half a dozen frames or so, before it restarts. I can pretty much cut out the bad place and maintain sync, I know exactly what I want to cut out, but I can't seem to match up the timestamps(PTS) in xport(which are exact matches for what are exactly in the ts file) with the ones you show.

For example the first PTS of the ts file is always
43843, but according to TSPE the first PTS is:
00:00:00:14 (29) 487 .. how to I match these two PTS stamps up? I'm not seeing a correlation between the two, I tried changing from PCR to PTS based in the Options->Timestamp Mode, but I didn't see any difference.

Upon looking I do see how they correlate, 487 is the ms value of 43843, is there anyway I can get the raw 43843 to show in TSPE

Hi,

TSPE shows the PTS in a number of formats:

In the decode window you have:

PID: 0x0203
Continuity Counter: 4
PTS: 13:02:29:07 (15) 46949.304s
DTS: 13:02:29:04 (09) 46949.184s
MPEG2 Timestamp: 08:54:15:09
...
PES Header:
Stream ID: 224: Video Stream: 0
...
PTS Ticks: 4225437376
DTS Ticks: 4225426576

Here you can see the actual PTS and DTS "ticks" of the frame. To get the actual seconds this represents, divide by 90000 (90KHz)

eg: 4225437376 / 90000 = 46949.304 seconds
Converting from seconds to actual timestamp gives you the timestamp shown at the start of the packet.

I'm not sure how xport calculates it, but I think TSPE does all xport does anyway as far as I know (but I haven't used xport in a while).

Hope that helps,

Vent

[rebkell]

Hi,
...
PES Header:
Stream ID: 224: Video Stream: 0
...
PTS Ticks: 4225437376
DTS Ticks: 4225426576[/CODE]

Here you can see the actual PTS and DTS "ticks" of the frame. To get the actual seconds this represents, divide by 90000 (90KHz)

eg: 4225437376 / 90000 = 46949.304 seconds
Converting from seconds to actual timestamp gives you the timestamp shown at the start of the packet.

I'm not sure how xport calculates it, but I think TSPE does all xport does anyway as far as I know (but I haven't used xport in a while).

Hope that helps,

Vent

Got it, it's the ticks, is there anyway to make the font bigger in the PES Header view(Packet Decode Window)? It's awfully small and difficult to read.

[Ventolin]

Got it, it's the ticks, is there anyway to make the font bigger in the PES Header view(Packet Decode Window)? It's awfully small and difficult to read.

Funny, I was thinking exactly the same thing when I wrote that example just now!

Added to the list!

Cheers,

Vent

[rebkell]

Funny, I was thinking exactly the same thing when I wrote that example just now!

Added to the list!

Cheers,

Vent

Another request, you have an NPTS button, which will jump to the next PTS no matter the PID, would it be possible to add a PPTS button, that will jump to the previous PTS no matter the PID.

I was writing out an edited file, that was 29.97FPS and it kept reporting : Detected framerate 12.500 fps. The edited file seems to be fine, but just wondering where it got the 12.5 FPS number from, or am I totally misunderstanding what it means, like I said it's no biggie because the edited file was just fine.

Another question about cutting/editing. When I set an end point, if for instance I set it on the first packet of an audio frame, will that frame be included or excluded from the edit. You also said it has I-Frame and P-Frame accuracy, so if I'm trying to fine tune my cuts, would I position the end cut on a P-Frame or just prior to the P-Frame? Sorry for all the questions, but I think I can work very well with this program, but I just want to understand how the cut points function, exactly.

Edit: Ignore the end point question, I found the option to include it or not include it in the EDL file.

[Ventolin]

Another request, you have an NPTS button, which will jump to the next PTS no matter the PID, would it be possible to add a PPTS button, that will jump to the previous PTS no matter the PID.
There used to a button that did that, but the usefulness of the NPTS / PPTS buttons are a bit questionable. It's trivial to add this functionality back in, but I think that will have to wait for user customisable toolbars which is for a later release.

I was writing out an edited file, that was 29.97FPS and it kept reporting : Detected framerate 12.500 fps. The edited file seems to be fine, but just wondering where it got the 12.5 FPS number from, or am I totally misunderstanding what it means, like I said it's no biggie because the edited file was just fine.
This looks to be a bug, can you send me a small sample file please (10MB should be enough).

Another question about cutting/editing. When I set an end point, if for instance I set it on the first packet of an audio frame, will that frame be included or excluded from the edit. You also said it has I-Frame and P-Frame accuracy, so if I'm trying to fine tune my cuts, would I position the end cut on a P-Frame or just prior to the P-Frame? Sorry for all the questions, but I think I can work very well with this program, but I just want to understand how the cut points function, exactly.
With "Clean Edits" on, the exact edit location is automatically determined so you don't have to worry. You can do a "Test Edit Point" by right click 2 selected EDL entries. This will make a short test file with the last few seconds of the first entry together with the first few seconds of the second entry so you can see how the final edit will look. From that you can fine tune the start/end points of the edit. Note that most of the video codecs themselves have trouble moving to a B frame directly so finding the exact edit point is really a bit of trail and error.

Edit: Ignore the end point question, I found the option to include it or not include it in the EDL file.
The "Include packet at end point" option is useful for packet accurate editing with "clean edits" off. This lets you decide if you want the current packet to be included at the end point or not. Usually if you are at an I Frame you should have this off but have it on to include the very last packet of the file. This should be on if you are merging multi-part ts files in the edl for byte-perfect merging. With "clean edits" on however, this option has no effect, which is the default behaviour.

Hope that helps,

Regards,

Vent

[rebkell]

Ahh, clean edits, ok.

Also what is the significance of Auto Find Sensitivity? I lowered it and it found some discontinuities that were giving me trouble, I lowered it to 10 and it seems to find all the bad places in this file I'm working with. What do the numbers represent? The default is 30 I believe.

I'll try to upload a piece of this file for you to look at the 12.5 FPS thing, it's just one of the standard 1080i, Hauppauge recordings.

[Ventolin]

Ahh, clean edits, ok.

Also what is the significance of Auto Find Sensitivity? I lowered it and it found some discontinuities that were giving me trouble, I lowered it to 10 and it seems to find all the bad places in this file I'm working with. What do the numbers represent? The default is 30 I believe.

I'll try to upload a piece of this file for you to look at the 12.5 FPS thing, it's just one of the standard 1080i, Hauppauge recordings.
If you could upload the glitch part with a bit on either side, I can have a look at the glitches you are getting at the same time. +/- 10 seconds from a glitch would be great, thanks.

The Auto Find Sensitivity adjusts the depth of the search it does when you first load a file. Higher is better but slower, though in your particular file it just so happens to pick up the glitches you have.

Cheers,

Vent

[rebkell]

Ok, I gave you bad info, I actually don't find anything with Auto Find, the one it does when it loads a new file. It was actually Scan that finds all the problems, I think I was overlooking some bad places it was putting in the Markers tab section. I just cut out the file and demuxed it and then remuxed it and the a/v sync was pretty much dead on.

I'm sorry for the bad info I was giving, but I did learn a lot and I know my problem areas pretty well and this looks like I can fix them, I'm going to have to cut out anywhere from 6 to 8 frames to maintain sync, but it's a bad place in the file anyway, so it's really no big loss considering it was a bad place anyway. The good thing is that everytime I have an audio dropout, when the unit recovers it always starts afresh with an IDR frame and the audio is exactly 100 ms ahead of the audio.

We/you really need some of this stuff documented, I know it's a pain, but it's hard to determine some of this stuff even with trial and error, I probably would have never figured out the clean edits thing, since it seems to basically override other things.

[rebkell]

One more feature request, when you display the Frame type in the bottom right status bar, it would be convenient if you signified if the I Slice was an IDR frame.

Another thing I've noticed, it may be by design, is that when you are navigating on the track bar and you click to the right or left of the slider, it sometimes jumps two minutes and if held down it will jump numerous times, which is all well and good, but it doesn't display any of the frames when it jumps, only the last one when it stops.

All in all, this program is really nice, it has a lot of controls and it's just a matter of learning how to use them for my needs.

[Ventolin]

One more feature request, when you display the Frame type in the bottom right status bar, it would be convenient if you signified if the I Slice was an IDR frame.

Another thing I've noticed, it may be by design, is that when you are navigating on the track bar and you click to the right or left of the slider, it sometimes jumps two minutes and if held down it will jump numerous times, which is all well and good, but it doesn't display any of the frames when it jumps, only the last one when it stops.

All in all, this program is really nice, it has a lot of controls and it's just a matter of learning how to use them for my needs.
Hi,
Glad to hear you are able to get your glitches sorted! I got your sample file thanks, will look into the framerate bug.

IDR frame now identified in the bottom right status corner as requested, also IF+/- jumps to IDR frames if they are present in the stream. Need to do some more testing with that though.

Documentation is important, I know, trouble is the program is in quite a state of flux at the moment, so some of the options are changing and a whole lot of others are being added. Alot of the options are there so you can turn them off if something goes wrong though, normally the option defaults are the ones you should always use. I do feel though that you shouldn't really have to RTFM if you want to know how to do something, the program should tell you or it should be obvious, so that's where I would like to head towards.

Trackbar behaviour is by design, requests are actually made for frames, but the video decoders just can't handle that level of "scrubbing" - it takes alot to decode the video let alone deal with hundreds of requests in a split second.

To aid navigation, quite a few keyboard shortcuts and mouse controls have been added. Enough to please both keyboard and mouse jockeys alike!

That with Directshow filter control will be in the next release due shortly.

Regards,

Vent

[rebkell]

Are there any keyboard shortcuts for navigating the preview window? The trackbar control may be by design, but it jumps 2 minutes and sometimes 3 minutes, no matter how fast I click and release the mouse button. It can actually jump over an entire commercial with this behavior.

Keep up the good work, is there a new version available, you said the IDR shows up in the window, is a new beta version available for download?

[DreckSoft]

PAT and PMT really are essential to a Transport stream. If a capture program strips them out of a stream, then the capture program is at fault. TSPE would then need to repair the stream to fix the poor design of the capture program. What real life streams have you come across without PAT and PMTs? If a specific capture program is stripping out the PAT and PMT perhaps the authors could fix their software?


Offline Descrambler produces strange output. Till now I haven't found a program which can handle the output. The output of OD seems to be ok for the few pre-defined CAIDs but decrypting for example ORF1HD with Cryptoworks (0x0D05) creates a completely unusable file.

So a "fix my stream" would be a nice function.

[Ventolin]

Are there any keyboard shortcuts for navigating the preview window? The trackbar control may be by design, but it jumps 2 minutes and sometimes 3 minutes, no matter how fast I click and release the mouse button. It can actually jump over an entire commercial with this behavior.

Keep up the good work, is there a new version available, you said the IDR shows up in the window, is a new beta version available for download?
Hi, thanks, I'm looking into the trackbar sensitivity. Almost all the keyboard and mouse shortcuts are for navigation.

New public version isn't available yet, though fully paid users may have a chance to test before the next public beta is released.

Regards,

Vent

[Ventolin]

Offline Descrambler produces strange output. Till now I haven't found a program which can handle the output. The output of OD seems to be ok for the few pre-defined CAIDs but decrypting for example ORF1HD with Cryptoworks (0x0D05) creates a completely unusable file.

So a "fix my stream" would be a nice function.

Hi,

If the output is still scrambled, there isn't anything TSPE will do to descramble the output. However, if it's a case of missing tables, there is a possiblity. Table editing support is high on the priority list, though I'm not sure it will make it to the next public beta.

If you can send me a sample, I'll see if there's anything that can be done.

Regards,

Vent

[robinsj]

As long as you play

Did you mean to say something after this? Seems like an incomplete statement. Thanks for those directions for editing out audio, that should work perfectly. Next thing I know, my wife will be complaining that she can't hear it, but see can see them mouth it... haha... So just thought I would ask, is there anyway out there, to blur the mouth or anything like that. I mean, I think for the most part just cutting the audio would work great, but some videos are really close up on a face or something. I know this is out of taste for some people, and like I said before, not looking to get into arguements about what I want to edit. I seriously doubt its possible, but let me know if you have any ideas....

[Ventolin]

Did you mean to say something after this? Seems like an incomplete statement. Thanks for those directions for editing out audio, that should work perfectly. Next thing I know, my wife will be complaining that she can't hear it, but see can see them mouth it... haha... So just thought I would ask, is there anyway out there, to blur the mouth or anything like that. I mean, I think for the most part just cutting the audio would work great, but some videos are really close up on a face or something. I know this is out of taste for some people, and like I said before, not looking to get into arguements about what I want to edit. I seriously doubt its possible, but let me know if you have any ideas....

I did have some text after that line, I tried to edit the post to add it in, but looks like it didn't stick. Here's the rest of the text...

"As long as you play the file as a transport stream you will maintain sync, if you demux it you will loose sync at the edit point. To fix this, just demux in separate parts for processing then reassemble afterwards."

As for blurring of mouth, a quick hand infront of the eyes works well, as does changing the video input on the TV with a quick flick of the remote. You should know exactly where the swear words are if you cut them out!

If you want to do it in the video, you'll have to re-encode with a blur filter (using a digital compositor or AVS script for the blur), or just cut the whole GOP out (from I/P frame to I frame).

Cheers,

Vent

[robinsj]

Yeah, i figured that was gonna be the answer but thought I would ask. I honestly don't care, cutting the audio is good enough, and I absolutely don't want to re-encode. What I have been doing, is just ripping the sub titles and doing a search through those for the f words, and it gives me the timestamp right there which is within pretty much a second of when it plays... so then I just use your fine piece of software to pinpoint it and snip it out. Great detailed instructions you gave by the way. This is the reason why I was wondering if you can just enter in a time that you wanted to goto in the video? Did you add this feature, I noticed you said you were going to. Was wondering if it has been added to the downloadable file yet?

[Renzz]

I downloaded TSPE yesterday along with the trial license and managed to do a few edits ok. However, today it will no longer edit. I can open files, move through them, select edit points etc, but when I click on "Edit" to do the final edit, it always immediately fails with

Error:
Index was outside the bounds of the array. at FmX1GV7Hfyex.3rSrxSTS.3I9TgLAHWw6yjYty6cd()
PID: 0, iPii: 0, ProgIndex: 0
Warning: No identified video PID found in this section.
Total time taken 0.06 seconds, write speed 0.0 MB/s

Edit complete: successfully written file with 376 bytes.

It is doing this on the file I successfully edited yesterday as well as others. Not sure what I am doing wrong.

(Oh and BTW, the email address under Help - betatest@bitstreamtools.com - is bouncing as unknown).

[blubberbirne]

Same Problem here......

Maybe the Beta Key Expired?

[Ventolin]

Thanks for that notification, looking into it.

Vent

EDIT: I have updated the licence on the website, please redownload the licence! This will only affect trial licences, people with fully paid licences will not have an issue.

Please let me know how you get on with the new licence.

Newer version should be out very soon btw.

[Renzz]

Nope, still failing for me:

Error:
Index was outside the bounds of the array. at FmX1GV7Hfyex.3rSrxSTS.3I9TgLAHWw6yjYty6cd()
PID: 0, iPii: 0, ProgIndex: 0
Warning: No identified video PID found in this section.
Total time taken 0.09 seconds, write speed 0.0 MB/s

Edit complete: successfully written file with 188 bytes.

[Ventolin]

Nope, still failing for me:

Error:
Index was outside the bounds of the array. at FmX1GV7Hfyex.3rSrxSTS.3I9TgLAHWw6yjYty6cd()
PID: 0, iPii: 0, ProgIndex: 0
Warning: No identified video PID found in this section.
Total time taken 0.09 seconds, write speed 0.0 MB/s

Edit complete: successfully written file with 188 bytes.

Thanks, please stay tuned...

Vent

[Ventolin]

New Version 0.740!

Hi folks, as the 0.700 licence has now expired, please try the NEW version 0.740!

Please note, this version is unstable and needs testing, please do let me know how you get on. Check the website for the latest updates.

You will need both the new 0.740 (01/01/09) .rar file and the beta test licence (01/01/09) file.

Whats New:

Directshow Filter Control. - Choose your directshow video and audio decoder filters (under options menu).
Keyboard and Mouse shortcuts - see under Help - Keyboard Shortcuts.
GUI now remembers positions (including preview window)
Video Preview should now be more accurate - two modes added, Preview Mode and Auto-Preview - see below.
H.264 IDR frames now clearly identified in bottom corner status, IF+/- snaps to IDR frames if present (needs some testing)
Internal Viewer now stays active, can be full screen via keyboard shortcut.
Decode Packet window font size increased slightly.
Jump To Timecode - just type in a new timecode in the Current Time box above the preview window.


Preview Mode:
Pressing CTRL-K will do a very fast test and show the first frame in the main viewer window. This lets you quickly see if your start point is accurate (jump to an IFrame first!).

Autopreview mode (under options - timestamp mode)
This disables realtime preview and instead creates a fast test edit and displays the first frame from that. This is much slower than realtime preview and is only recommended for use if you really have inaccurate preview. (If you do, please send me a sample file!)

In almost all cases, preview is accurate with CoreAVC, Cyberlink and FFDShow filters. However, you can adjust the preview window by tweaking the "Preview Adjust" option (under options - timestamp mode)

Directshow Control:
This needs to be enabled if you want to use your own filter choices instead of the automatic filter choice. Note, incorrectly setting the filters may leave TSPE unstable!

Right clicking the video display will show the filters in use. If verbose logging is enabled (under options), you can get a dump of the directshow filter chain in use in the status window.

Hope this release works OK for you, feedback is always appreciated!


Regards,

Vent

[Renzz]

Thanks, new version working ok again for me.

A couple of things I have noticed as I use it today:

1. If I open a stream with several audio tracks (BBC HD stream for example which has both AC3 and MPEG soundtracks), it detects both when it scans the file, but only lists the AC3 one under the PID tree, and only retains this track in the edited file. How do you get it to preserve all audio tracks?

2. Opening a stream from ITV HD, it detects it as 50fps, when it is 25 - possibly due to the MBAFF encoding they use?

[Renzz]

A quick note about editing commercials out and keeping sync.

I have captured a stream from ITV-HD that I wanted to edit the commercials from, demux and encode to make it smaller, then remux the original AC3 soundtrack back in. However, on the default settings in TSPE, I could successfully do the edits and the resulting TS file played fine and in sync. I then used DGAVCDECNV to index the stream and demux the AC3 out. I then encoded the video using MEGUI to something smaller and then remuxed the AC3 back. The start of the resulting MKV was fine and in sync, but at each cut point it would go slightly out of sync, so by the end it was out by quite a lot.

I have since discovered that if I go to Edit/Clean Edit Point and untick Clean Audio Frames, it fixes this and keeps everything in sync. I do get the occasional squeak at the edit point where I guess the AC3 is slightly corrupt, but at least sync is now good.

Is this the only way I can achieve good sync when cutting h264 streams at an IDR frame?

[rebkell]

A quick note about editing commercials out and keeping sync.

I have captured a stream from ITV-HD that I wanted to edit the commercials from, demux and encode to make it smaller, then remux the original AC3 soundtrack back in. However, on the default settings in TSPE, I could successfully do the edits and the resulting TS file played fine and in sync. I then used DGAVCDECNV to index the stream and demux the AC3 out. I then encoded the video using MEGUI to something smaller and then remuxed the AC3 back. The start of the resulting MKV was fine and in sync, but at each cut point it would go slightly out of sync, so by the end it was out by quite a lot.

I have since discovered that if I go to Edit/Clean Edit Point and untick Clean Audio Frames, it fixes this and keeps everything in sync. I do get the occasional squeak at the edit point where I guess the AC3 is slightly corrupt, but at least sync is now good.

Is this the only way I can achieve good sync when cutting h264 streams at an IDR frame?

Commercials are a pain for sure, I've pretty much just started encoding the whole thing before I edit it, I still have problems with sync sometimes, but I've decided I spend less time getting a good finished product, than if I try to cut commercials before encoding, The extra time it takes to encode I figure I would spend trying to edit it out and clean it up anyway. Of course in my case I'm re-encoding back to h264 and then muxing back to a ts file, then I use either TSPE or h264ts_cutter to cut out commercials.

[Ventolin]

A quick note about editing commercials out and keeping sync.

I have captured a stream from ITV-HD that I wanted to edit the commercials from, demux and encode to make it smaller, then remux the original AC3 soundtrack back in. However, on the default settings in TSPE, I could successfully do the edits and the resulting TS file played fine and in sync. I then used DGAVCDECNV to index the stream and demux the AC3 out. I then encoded the video using MEGUI to something smaller and then remuxed the AC3 back. The start of the resulting MKV was fine and in sync, but at each cut point it would go slightly out of sync, so by the end it was out by quite a lot.

I have since discovered that if I go to Edit/Clean Edit Point and untick Clean Audio Frames, it fixes this and keeps everything in sync. I do get the occasional squeak at the edit point where I guess the AC3 is slightly corrupt, but at least sync is now good.

Is this the only way I can achieve good sync when cutting h264 streams at an IDR frame?

Hi,

Did you try to demux from within TSPE? You can edit and demux in the same operation.

Regarding the ITV-HD mpeg audio stream, it might be that the stream is there but perhaps not identified clearly. Do you have any other streams in the PID tree?

Vent

[Renzz]

Hi,

Did you try to demux from within TSPE? You can edit and demux in the same operation.

Regarding the ITV-HD mpeg audio stream, it might be that the stream is there but perhaps not identified clearly. Do you have any other streams in the PID tree?

Vent

No, I haven't tried editing and demuxing at the same time - I'll give that a go.

Regarding the MPEG audio stream (actually this is from BBC HD not ITV), the only streams in the PID tree are are the AVC video and AC3 audio. However, when you open the file initially, it does detect it:

Opening: G:\Recordings\Doctor Who at the Proms (BBC HD) 2009-01-01 13-47-02.ts
Opening File...
Video Loaded.
Auto Finding PIDs
Debug: PMT Version Change Detected near 0x6A414
Debug: PMT Version Change Detected near 0x1DF8C
Debug: PMT Version Change Detected near 0x6A414
PID Attribute List:
PID: 0x0918: H.264 1440x1088 (1920x1080) @ 25.000 fps
PID: 0x0919: AC3: 3/0.1: L, C, R, LFE 44.1KHz 256Kbit/s
PID: 0x091A: MPEG1 Audio Layer 2, 64Kbit/s @ 44100Hz

Scanning For Format Changes
File size: 8707612356 Bytes
Video Resolution from Codec: 1440 x 1080 @ 25.00 fps
Duration from Codec: 4077.44
Duration from Timestamps: 4077.56
Average System Bitrate: 17083966.60
Done.