Project X, Cuttermaran, and HD ATSC

QuadraQ · 27th February 2009, 08:11

I'm having the same issue described in this old post:

http://forum.dvbtechnics.info/archiv...hp/t-2232.html

I have a MPEG2 capture of an ATSC 1080i AC-3 broadcast, and want to use Cuttermaran to edit it. I'm trying to use ProjectX to demultiplex it first. Other tools such as DGIndex have serious sync problems when used with cuttermaran - or at least this particular recording does for sure.

I can get ProjectX to demultiplex the source with sync, but it drops thousands of necessary B frames that appear to be missing PTS information. This makes the result stutter and as a result unusable.

Can ProjectX handle ATSC broadcasts? Is there some option I need to try? If it doesn't support ATSC is there any real chance that could be added?

I could make part of the recording available to any developers who may need it for testing. Thanks.

drmpeg · 27th February 2009, 12:10

If you're willing to use a Command Prompt tool (and I realize many do not like to), you can try my demuxer.

http://www.w6rz.net/xport.zip

It will automatically cut the audio to be in sync with the video. If you have any problems, just post here and I'll help you out.

BTW, the problem with ProjectX is that it doesn't support video streams with inverse telecine or video streams that don't have GOP headers.

Ron

stenews · 27th February 2009, 14:08

Hi drmpeg nice to talk with you

Could you explain me better how "MPEG-2 Transport Stream Test Patterns and Tools" works?
I mean, Can I check with it the integrity of my TS files or what?
I see that it's an ISO file...why?

Thanks for any information you'll give to me.
Stefano.

QuadraQ · 28th February 2009, 03:06

Quote:

Originally Posted by drmpeg

If you're willing to use a Command Prompt tool (and I realize many do not like to), you can try my demuxer.

http://www.w6rz.net/xport.zip

It will automatically cut the audio to be in sync with the video. If you have any problems, just post here and I'll help you out.

BTW, the problem with ProjectX is that it doesn't support video streams with inverse telecine or video streams that don't have GOP headers.

Ron

Well I'm capable of using command line just fine when needed.

OK according to your readme file the following is the usage:

Quote:

xport <-pavtdszgh> <infile> <program number> <video stream number> <audio stream number>

typically:

xport something.ts 1 1 1

The output files are always named bits0001.mpv for video and bits0001.mpa for audio
(note that bits0001.mpa can be an AC-3 file). For AC-3, MPEG-1 Layer 2 and Blu-ray
LPCM audio, the demuxer will time align the beginning of the two elementary streams
so that a remux to PS or back to TS will have correct A/V sync. The -n option has
been deleted.

Options:

p = parse only, do not demux to video and audio files.
a = dump audio PTS.
v = dump video PTS.
t = GOP timecode mode, count repeated fields/frames.
d = dump all PID's (useful for debugging muxers, but tons of output).
s = suppress TS rate dumping (useful when piping output to a file).
z = demux to PES streams (instead of elementary streams).
g = dump ATSC PSIP information.
h = input file is in HDMV (AVCHD and Blu-ray) format (192 byte packets)
2 = only extract 2 channels of HDMV LPCM audio from multi-channel tracks.

So I'm no format expert. What is a simple way to discover the "program number," the "video stream number," and the "audio stream number?"

My source doesn't record in a TS stream but is DVR-MS format originally and then converted to an mpg file using ffmpeg. I'm able to create a TS file by running the mpg through ProjectX in the TS mode. Is that going to be a problem?

Also my source has AC-3 audio so when the demux is finished can I simply change the extension from .mpa to .ac3? Thanks again for your help.

QuadraQ · 28th February 2009, 04:15

OK I went ahead and tried it with the format your readme suggests "xport Test.ts 1 1 1" and it seemed to work fine. Here are the results:

C:\Users\Isaac\Videos\Temp Files>xport Test.ts 1 1 1
xport Transport Stream Demuxer 1.01
program = 1, video channel = 1, audio channel = 1
Program Number = 1 (0x0001), Program Map PID = 16 (0x0010)
program descriptor = 0x05, 0x04, 0x47, 0x41, 0x39, 0x34
Video PID = 17 <0x0011>, type = 0x02
Audio PID = 20 <0x0014>, type = 0x81
Sequence Header found
0 frames before first Sequence Header
Horizontal size = 1920
Vertical size = 1080
Aspect ratio = 16:9
Frame rate = 29.97
Sequence header bitrate = 65000000 bps
Progressive Sequence = 0
First Video PTS = 0x0000c6fc
Audio Bitrate = 384000, Audio Sampling Rate = 48000
Audio Mode = 3/2, bsid = 8, bsmod = 0
First Audio PTS = 0x0000c648, -180
ts rate = 18329320, video rate = 15658794
packets for pid 0 <0x0000> = 283163, first = 1, last = 45755046
packets for pid 16 <0x0010> = 283163, first = 2, last = 45755047
packets for pid 17 <0x0011> = 44109240, first = 3, last = 45755169
packets for pid 20 <0x0014> = 1079603, first = 281, last = 45752466
coded pictures = 95796, video fields = 226379
Ending audio to video PTS difference = -25470 ticks, -283.000000 milliseconds

Next I renamed the mpa file to ac3. I placed the mpv file in VLC - Good job! Looks just as good as the original, without the fits and starts that ProjectX created. Did the same with the ac3 file and everything seems in order.

Opened Cuttermaran and loaded the mpv file. It processes it and doesn't give me any error messages. (DGIndex demuxing always gave me error messages about 'Temporal errors' or something like that.) I load the ac3 file and it processes it OK - no errors. I then change the delay to -283 ms as reported by xport.

Here's the big challenge - Will the audio and video stay in sync?

....
No.
....

I'm experiencing the same sync issues as before. What's interesting is that it's not simply a matter of delay. The audio gets more and more out of sync the further I jump ahead in the video (It is in sync at the beginning, but gets out of sync pretty fast, probably around 35 seconds into it). I remember reading somewhere on these forums that sometimes the audio needs to be 'stretched' to fit the video.

Anyway, DrMpeg do you have any idea what to do next? I'd be happy to provide you with any logs, or video material if you need it. Does Cuttermaran have some known issue with audio of the above format? Anyway, thanks for the help and insight.

QuadraQ · 28th February 2009, 04:40

Another quick test - I loaded the audio/video files into Mpeg2Schnitt and it loaded them successfully. I set the audio delay to -283ms again and pressed play to preview it.

Same issue. Not sure if it's useful or not but here's the info that Mpeg2Schnitt reports:

Video file size: 7094.5 MB
Video bitrate: 18618709 bps
------------------------------------
Sequence header:
Frame width: 1920
Frame height: 1080
Aspect ratio: 9/16
Frame rate: 29,97 fps
Bitrate: 65000000 bps
VBV buffer: 976 kB
Profile/Level: Main/High
Colour format: 4:2:0
Low Delay: No
------------------------------------
Frame header:
Frame type: I-Frame
DC Precision: 10
Frame structure: Frame
Top field first: Yes
------------------------------------
Audio information:
AC3 sound
Bitrate: 384 Kbps
Samplerate: 48000 Hz
Complete Main (CM)
3/2 - L, C, R, SL, SR
Low Frequency Effects
Frame size: 1536 Byte
Frame time: 32 ms

I also noticed something odd. There is a significant disparity between the video length and audio length that is reported:

Video: 00:53:16:12
Audio: 01:02:56:736

In this case the audio length appears to be essentially correct, while the video length is too short.

I suppose that it's no coincidence that both of these programs have difficulty (Cuttermaran and Mpeg2Schnitt) and both are developed in Germany. Perhaps they have a PAL bias.

Are there comparable programs with a more NTSC/ATSC focus? Again any help is much appreciated.

drmpeg · 28th February 2009, 05:05

The -283 millisecond figure at the end of the run is not the audio delay. It's just some information I added when we were looking at audio overlap in multi-segment Blu-Ray rips. The actual audio delay is the figure that gets printed after the line:

First Audio PTS = 0x0000c648, -180

It's in PTS tick units, so the residual audio delay is only -180/90000 or -2 milliseconds.

The drift in A/V sync is a whole other problem. Sometimes it depends on how the bitstream was originally muxed. One thing to check is the PTS increment of the audio and video streams. You can use xport to do this. To check the audio PTS increment, use:

xport -pas Test.ts 1 1 1

This will dump all of the audio PTS's and the increment between them. If any increment is not 2880, then you will have drift issues. For the video PTS, use:

xport -pvs Test.ts 1 1 1

Likewise, if the video PTS increment is not 3003 (or 4504/4505 if there's a repeat field), then you'll have drift problems.

BTW, your bitstream does have telecine flags, which is why ProjectX didn't work for you. The clue is this line:

coded pictures = 95796, video fields = 226379

It's reporting the number of MPEG-2 pictures and the number of fields they represent. If the number of field is greater than coded pictures times 2, then there were frames with repeat_first_field = 1.

Ron

QuadraQ · 28th February 2009, 09:26

OK here's the result of the audio test (greatly abbreviated):

xport Transport Stream Demuxer 1.01
program = 1, video channel = 1, audio channel = 1
Program Number = 1 (0x0001), Program Map PID = 16 (0x0010)
program descriptor = 0x05, 0x04, 0x47, 0x41, 0x39, 0x34
Video PID = 17 <0x0011>, type = 0x02

Audio PID = 20 <0x0014>, type = 0x81

Sequence Header found
0 frames before first Sequence Header
Horizontal size = 1920
Vertical size = 1080
Aspect ratio = 16:9
Frame rate = 29.97
Sequence header bitrate = 65000000 bps
Progressive Sequence = 0
Audio PTS = 50760, 50760

First Video PTS = 0x0000c6fc
Audio PTS = 53640, 2880

Audio Bitrate = 384000, Audio Sampling Rate = 48000
Audio Mode = 3/2, bsid = 8, bsmod = 0
First Audio PTS = 0x0000c648, -180
Audio PTS = 56520, 2880
Audio PTS = 62280, 5760
Audio PTS = 65160, 2880
Audio PTS = 68040, 2880
Audio PTS = 73800, 5760
Audio PTS = 76680, 2880
Audio PTS = 79560, 2880
Audio PTS = 85320, 5760
Audio PTS = 88200, 2880
Audio PTS = 91080, 2880
Audio PTS = 96840, 5760
Audio PTS = 99720, 2880
...
Audio PTS = 339944310, 2880
Audio PTS = 339947190, 2880
Audio PTS = 339952950, 5760

packets for pid 0 <0x0000> = 283163, first = 1, last = 45755046
packets for pid 16 <0x0010> = 283163, first = 2, last = 45755047
packets for pid 17 <0x0011> = 44109240, first = 3, last = 45755169
packets for pid 20 <0x0014> = 1079603, first = 281, last = 45752466
coded pictures = 95796, video fields = 226379
Ending audio to video PTS difference = -25470 ticks, -283.000000 milliseconds

Obviously some of the increments are not 2880, but the only other value I see is 5760.

The video test is more complex and less consistent, so I've attached it for full review. Here's a snippet:

xport Transport Stream Demuxer 1.01
program = 1, video channel = 1, audio channel = 1
Program Number = 1 (0x0001), Program Map PID = 16 (0x0010)
program descriptor = 0x05, 0x04, 0x47, 0x41, 0x39, 0x34
Video PID = 17 <0x0011>, type = 0x02

Audio PID = 20 <0x0014>, type = 0x81

Video PTS(P) = 83340, DTS(P) = 47970
Sequence Header found
0 frames before first Sequence Header
Horizontal size = 1920
Vertical size = 1080
Aspect ratio = 16:9
Frame rate = 29.97
Sequence header bitrate = 65000000 bps
Progressive Sequence = 0
Video PTS(B) = 50940, 2970
First Video PTS = 0x0000c6fc
Video PTS(B) = 53910, 2970
Audio Bitrate = 384000, Audio Sampling Rate = 48000
Audio Mode = 3/2, bsid = 8, bsmod = 0
First Audio PTS = 0x0000c648, -180
Video PTS(B) = 86310, 32400
Video PTS(B) = 89280, 2970
Video PTS(B) = 95220, 5940
Video PTS(B) = 98190, 2970
Video PTS(B) = 104130, 5940
Video PTS(B) = 107100, 2970
Video PTS(B) = 113040, 5940
Video PTS(B) = 116010, 2970
Video PTS(B) = 121950, 5940
Video PTS(B) = 124920, 2970
Video PTS(B) = 130860, 5940
Video PTS(B) = 133830, 2970
Video PTS(B) = 139770, 5940
Video PTS(B) = 142740, 2970
Video PTS(B) = 148680, 5940
Video PTS(B) = 151650, 2970
Video PTS(B) = 157590, 5940
Video PTS(B) = 160560, 2970
Video PTS(P) = 173430, DTS(P) = 163530, 2970
...
Video PTS(B) = 339965730, 4500
Video PTS(P) = 339981210, DTS(P) = 339968700, 2970
Video PTS(B) = 339973200, 4500
Video PTS(B) = 339976170, 2970

packets for pid 0 <0x0000> = 283163, first = 1, last = 45755046
packets for pid 16 <0x0010> = 283163, first = 2, last = 45755047
packets for pid 17 <0x0011> = 44109240, first = 3, last = 45755169
packets for pid 20 <0x0014> = 1079603, first = 281, last = 45752466
coded pictures = 95796, video fields = 226379
Ending audio to video PTS difference = -25470 ticks, -283.000000 milliseconds

Is there any hope?

QuadraQ · 1st March 2009, 03:29

BTW I realized I hadn't said thank you, DrMpeg for your help. I'm very interested in video technology and how it works "behind the curtain." Thank you for taking the time to help educate me!

27th February 2009, 08:11	#1 \| Link
QuadraQ Registered User Join Date: Jun 2003 Location: Loomis, CA Posts: 109	Project X, Cuttermaran, and HD ATSC I'm having the same issue described in this old post: http://forum.dvbtechnics.info/archiv...hp/t-2232.html I have a MPEG2 capture of an ATSC 1080i AC-3 broadcast, and want to use Cuttermaran to edit it. I'm trying to use ProjectX to demultiplex it first. Other tools such as DGIndex have serious sync problems when used with cuttermaran - or at least this particular recording does for sure. I can get ProjectX to demultiplex the source with sync, but it drops thousands of necessary B frames that appear to be missing PTS information. This makes the result stutter and as a result unusable. Can ProjectX handle ATSC broadcasts? Is there some option I need to try? If it doesn't support ATSC is there any real chance that could be added? I could make part of the recording available to any developers who may need it for testing. Thanks.

27th February 2009, 12:10	#2 \| Link
drmpeg Registered User Join Date: Jan 2003 Location: Silicon Valley Posts: 455	If you're willing to use a Command Prompt tool (and I realize many do not like to), you can try my demuxer. http://www.w6rz.net/xport.zip It will automatically cut the audio to be in sync with the video. If you have any problems, just post here and I'll help you out. BTW, the problem with ProjectX is that it doesn't support video streams with inverse telecine or video streams that don't have GOP headers. Ron __________________ HD MPEG-2 Test Patterns http://www.w6rz.net

28th February 2009, 05:05	#7 \| Link
drmpeg Registered User Join Date: Jan 2003 Location: Silicon Valley Posts: 455	The -283 millisecond figure at the end of the run is not the audio delay. It's just some information I added when we were looking at audio overlap in multi-segment Blu-Ray rips. The actual audio delay is the figure that gets printed after the line: First Audio PTS = 0x0000c648, -180 It's in PTS tick units, so the residual audio delay is only -180/90000 or -2 milliseconds. The drift in A/V sync is a whole other problem. Sometimes it depends on how the bitstream was originally muxed. One thing to check is the PTS increment of the audio and video streams. You can use xport to do this. To check the audio PTS increment, use: xport -pas Test.ts 1 1 1 This will dump all of the audio PTS's and the increment between them. If any increment is not 2880, then you will have drift issues. For the video PTS, use: xport -pvs Test.ts 1 1 1 Likewise, if the video PTS increment is not 3003 (or 4504/4505 if there's a repeat field), then you'll have drift problems. BTW, your bitstream does have telecine flags, which is why ProjectX didn't work for you. The clue is this line: coded pictures = 95796, video fields = 226379 It's reporting the number of MPEG-2 pictures and the number of fields they represent. If the number of field is greater than coded pictures times 2, then there were frames with repeat_first_field = 1. Ron __________________ HD MPEG-2 Test Patterns http://www.w6rz.net

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

27th February 2009, 14:08	#3 \| Link
stenews Registered User Join Date: Oct 2008 Location: somewhere in Italy Posts: 217	Hi drmpeg nice to talk with you Could you explain me better how "MPEG-2 Transport Stream Test Patterns and Tools" works? I mean, Can I check with it the integrity of my TS files or what? I see that it's an ISO file...why? Thanks for any information you'll give to me. Stefano.

28th February 2009, 04:15	#5 \| Link
QuadraQ Registered User Join Date: Jun 2003 Location: Loomis, CA Posts: 109	OK I went ahead and tried it with the format your readme suggests "xport Test.ts 1 1 1" and it seemed to work fine. Here are the results: C:\Users\Isaac\Videos\Temp Files>xport Test.ts 1 1 1 xport Transport Stream Demuxer 1.01 program = 1, video channel = 1, audio channel = 1 Program Number = 1 (0x0001), Program Map PID = 16 (0x0010) program descriptor = 0x05, 0x04, 0x47, 0x41, 0x39, 0x34 Video PID = 17 <0x0011>, type = 0x02 Audio PID = 20 <0x0014>, type = 0x81 Sequence Header found 0 frames before first Sequence Header Horizontal size = 1920 Vertical size = 1080 Aspect ratio = 16:9 Frame rate = 29.97 Sequence header bitrate = 65000000 bps Progressive Sequence = 0 First Video PTS = 0x0000c6fc Audio Bitrate = 384000, Audio Sampling Rate = 48000 Audio Mode = 3/2, bsid = 8, bsmod = 0 First Audio PTS = 0x0000c648, -180 ts rate = 18329320, video rate = 15658794 packets for pid 0 <0x0000> = 283163, first = 1, last = 45755046 packets for pid 16 <0x0010> = 283163, first = 2, last = 45755047 packets for pid 17 <0x0011> = 44109240, first = 3, last = 45755169 packets for pid 20 <0x0014> = 1079603, first = 281, last = 45752466 coded pictures = 95796, video fields = 226379 Ending audio to video PTS difference = -25470 ticks, -283.000000 milliseconds Next I renamed the mpa file to ac3. I placed the mpv file in VLC - Good job! Looks just as good as the original, without the fits and starts that ProjectX created. Did the same with the ac3 file and everything seems in order. Opened Cuttermaran and loaded the mpv file. It processes it and doesn't give me any error messages. (DGIndex demuxing always gave me error messages about 'Temporal errors' or something like that.) I load the ac3 file and it processes it OK - no errors. I then change the delay to -283 ms as reported by xport. Here's the big challenge - Will the audio and video stay in sync? .... No. .... I'm experiencing the same sync issues as before. What's interesting is that it's not simply a matter of delay. The audio gets more and more out of sync the further I jump ahead in the video (It is in sync at the beginning, but gets out of sync pretty fast, probably around 35 seconds into it). I remember reading somewhere on these forums that sometimes the audio needs to be 'stretched' to fit the video. Anyway, DrMpeg do you have any idea what to do next? I'd be happy to provide you with any logs, or video material if you need it. Does Cuttermaran have some known issue with audio of the above format? Anyway, thanks for the help and insight.

28th February 2009, 04:40	#6 \| Link
QuadraQ Registered User Join Date: Jun 2003 Location: Loomis, CA Posts: 109	Another quick test - I loaded the audio/video files into Mpeg2Schnitt and it loaded them successfully. I set the audio delay to -283ms again and pressed play to preview it. Same issue. Not sure if it's useful or not but here's the info that Mpeg2Schnitt reports: Video file size: 7094.5 MB Video bitrate: 18618709 bps ------------------------------------ Sequence header: Frame width: 1920 Frame height: 1080 Aspect ratio: 9/16 Frame rate: 29,97 fps Bitrate: 65000000 bps VBV buffer: 976 kB Profile/Level: Main/High Colour format: 4:2:0 Low Delay: No ------------------------------------ Frame header: Frame type: I-Frame DC Precision: 10 Frame structure: Frame Top field first: Yes ------------------------------------ Audio information: AC3 sound Bitrate: 384 Kbps Samplerate: 48000 Hz Complete Main (CM) 3/2 - L, C, R, SL, SR Low Frequency Effects Frame size: 1536 Byte Frame time: 32 ms I also noticed something odd. There is a significant disparity between the video length and audio length that is reported: Video: 00:53:16:12 Audio: 01:02:56:736 In this case the audio length appears to be essentially correct, while the video length is too short. I suppose that it's no coincidence that both of these programs have difficulty (Cuttermaran and Mpeg2Schnitt) and both are developed in Germany. Perhaps they have a PAL bias. Are there comparable programs with a more NTSC/ATSC focus? Again any help is much appreciated.

1st March 2009, 03:29	#9 \| Link
QuadraQ Registered User Join Date: Jun 2003 Location: Loomis, CA Posts: 109	BTW I realized I hadn't said thank you, DrMpeg for your help. I'm very interested in video technology and how it works "behind the curtain." Thank you for taking the time to help educate me!