Need specification for this strange AC3 stream

[gumballguy]

Hi

My fusionhdtv card allows to save to mpeg instead of ts. When I first got the card, I thought this was a safer option, so I always recorded to mpeg. However, I cannot demux and cut streams when editing/encoding. Can anyone demux this sample file?

So far I have tried:
Projectx
PVAStrumento
TMPEG
simply cutting without demuxing using mpeg2schnitt then attempting to work with the output. Obviously, bad.

There is a sample clip at http://www.arach.net.au/~watto/2005-06-06.zip

Additionally, I fear that without using a program such as projectx, I'm going to end up with sync issues. Still, first step is to actually demux this thing!

[Guest]

I can demux the video with no problem using DGIndex 1.4.6b1. But the stream does not contain any audio, so there is no demuxed audio stream. At least there's no need to worry about sync. :)

[gumballguy]

Ah, there IS audio though! Play it in mpc (or wmp 6.4)! :)

I downloaded my own file and investigated to make sure I hadnt uploaded the wrong thing... and thats the correct file. Notably, it WONT play audio in VLC, which I only just discovered.

Im not sure if the PID's stay constant across filetypes (I suspect they dont), but a recording from the same tv channel to .ts format gives this:
Video: PID: 0x20A
Audio: PID: 0x2EE(AC-3)(eng)_PD
The ac3 stream will be 48000/384.

[Guest]

Hmm, interesting. I'll trace through it with the debugger and find out why DGIndex isn't seeing the audio. I'm home sick today. :)

[Guest]

It's not standard AC3, so I can't help with this without further information. It appears on private stream 1 as it should, but the audio substream ID is 0x0b, when it should be 0x80-0x87. And then the rate has a reserved value. I stopped looking after that. Patching the ID to 0x80 did not work.

I would consider supporting it if you can point to a spec that describes it.

Moving to the Audio forum and changing title.

[guada 2]

Hello gumballguy,

Sorry, PVASTRUMENTO perfectly functions;
Look at the result:

PVAStrumento 2.1.0.12
*** running at 10-18-2005 19:10
***
Pre-Scanning input. Takes some time.
Stream info for
C:\Documents and Settings\desbonne\Mes documents\log récent\Nouveau dossier\2005-06-06.mpg

Found 1 video stream.
Found 1 AC3 audio stream.

VIDEO #1
Resolution 1440 x 1088
Aspect ratio is 16:9
Frame rate 25.00 fps
Nominal bitrate 12800000 bps
First PTS: 00:00:00.724

AC3 AUDIO #1
Format: 2/0
Sampled at 48.0 kHz.
First PTS: 00:00:00.112


<<DEMUXING>>
Input: C:\Documents and Settings\desbonne\Mes documents\log récent\Nouveau dossier\2005-06-06.mpg
Options
.. fixing stream synch.
.. keeping track of audio offset accumulation.
.. strict checking of audio framing.
.. dropping GOPs longer than 980 KB
.. splitting on change of video format.
.. splitting on change of audio format.
.. splitting on number of audio tracks.
.. clearing CDF
.. creating new GOP timecodes.

Closing starting GOP.
Aiming for sync at 00:00:00.804
Cutting audio.
AC3 stream 1 now starts at 00:00:00.816
Video stream 1 now starts at 00:00:00.804
New MPEG video : \2005-06-06.mpv
New AC3 audio : \2005-06-06.ac3
****FileSpan for PES packet
Adjusting audio to frames in last GOP at EOF
Last fileset:
8494381 total bytes written
Video 1: 130 frames, 00:00:05.200
AC3 1: 150 frames, 00:00:04.800

Video statistics
================
GOP length (min/avg/max):
10 / 11 / 12 [frames]
690317 / 758101 / 767837 [Bytes]
GOP bitrate (min/avg/max):
12304 / 12844 / 13806 [kbps]


(Autosaving log to \2005-06-06.txt)

Bye.

[gumballguy]

Neuron2, im not so sure that its the ac3 stream as to how its placed into the file. I doubt that my hd card alters the stream according to whether it saves it into an mpeg or ts.

guada2, I shall have another attempt at it soon. From memory, the problem with pvastrumento is that it stops part of the way through a large file for an unknown reason. Let me verify that statement first though, and make sure I was running the newest PVAs at the time.

Replies may be delayed, I am under extreme pressure to finish writing my thesis! :) Hopefully I will find a few minutes spare tomorrow... its 2am atm, so g'night.

[guada 2]

gumballguy,

* What is the exact reference of your card?

* I wondered if Dvico' S was compatible with the mode of recognition of DGindex. Because one not to forget that it has his own AC3, AAC, MP2 and MP1 audio decoder.......

[Guest]

Apparently it's related to the difference between Australian and ATSC standards. I could try to reverse engineer PVAStrumento, but it would be easier if someone gave me the spec.

Do you have the Australian format card or the ATSC format card?

[setarip_old]

@gumballguy

Hi!

Can anyone demux this sample file?

So far I have tried:
Projectx
PVAStrumento
TMPEG

I was able to demultiplex your 5 second clip using good ol' reliable TMPGEnc v.1.2 (I'm fairly certain all more current versions would work as well), as follows:

TMPGEnc>>File>>MPEGTools>>Demultiplex (NOT "Simple Demultiplex")

I doubleclicked on the "private_stream_1 (0x0B)" and saved it with an .AC3 extender
I then doubleclicked on the "video_stream 0x00" and saved it with an .M2V extender

Each individual stream played normally under Windows Media Player...

[gumballguy]

Aha.. private stream. Id not thought of saving that as an ac3 file, I had imagined it to be a teletext file or something.

Neuron2, I will find you that info if possible in the near future. Will check product manual and then email the makers of the card. I shall pm you, so no need to check the thread.

Please note that this will be my last reply for a while (could be a couple weeks), I woke up to find my hdd failing miserably. My first priority is to recover that data before any more forum use ;)

[guada 2]

gumballguy,

I have an idea on the choice of your card. I understand your discretion.... :)

However, if you want to find your data, uses this software: EASYRECOVERY PROFESSIONAL.

Bye.

[guada 2]

With ADCTools I found this:

bitRate = 1536000
samplerate = 48000
channels = 2
bitPerSample = 16


MPC

Video: MPEG2 Video 1440x1088 (16:9) 25.00fps 12800Kbps
Audio: Dolby AC3 48000Hz stereo 256Kbps
Video: äS


Where is the truth?

[eb]

not for every case but for most cases bbtool1.9 is very useful

bbINFO - version 1.9, by Brent Beyeler (beyeler@home.com)
speed increases by, Apachez and Christian Vogelgsang

File 1 is an MPEG-2 Program Stream


PACK #0, pack_start_code = 000001BA, SCR = 112.280778 ms
stream_id = BD Private Stream 1, packet #0, PTS = 112.277778 ms, underflow

PACK #1, pack_start_code = 000001BA, SCR = 112.280778 ms
stream_id = BD Private Stream 1, packet #1,

PACK #2, pack_start_code = 000001BA, SCR = 112.280778 ms
stream_id = BD Private Stream 1, packet #2,

PACK #3, pack_start_code = 000001BA, SCR = 112.280778 ms
stream_id = E0 Video Stream 0, packet #0, PTS = 804.644444 ms, DTS = 684.644444 ms
0 - sequence_header_code
140 - extension_start_code
150 - user_data_start_code
161 - group_of_pictures header
169 - picture_start_code
178 - extension_start_code

PACK #4, pack_start_code = 000001BA, SCR = 112.280778 ms
stream_id = E0 Video Stream 0, packet #1,

Summary:

MPEG Packs = 1263
Private Stream 1 packets = 176, total bytes = 180224
Video stream 0 packets = 1087, total bytes = 8882053

[guada 2]

Hello eb,

For the video it is not a problem.
And for the audio?

What utilitarian other that you know?
Perhaps Mr question man ?????

[eb]

use bbdmux 1 0xBD 1.ac3
where 1 is above file renamed

EDIT:
and to demux video use
bbdmux 1 0xE0 1.m2v

[calinb]

Hi

My fusionhdtv card allows to save to mpeg instead of tp. When I first got the card, I thought this was a safer option, so I always recorded to mpeg. However, I cannot demux and cut streams when editing/encoding. Can anyone demux this sample file?

gumballguy,

I've had the same problem with mpeg files created by my ATSC Fusion card. I suggest using .tp (Fusion transport stream), if you can. I've been demuxing / recoding the .tp files for a couple of years, with, generally, no sync problems. On the rare occasion that I have a sync problem, I use MPC to determine the delay required and remux with avimuxGUI or mkvmerge/mmg.

Unfortunately for me, I must use mpeg on one QAM channel only. If I use .tp output, the stream is corrupted. I'm working this issue with Dvico customer support.

I've had good luck demuxing the mpeg files with the Windows mpeg demultiplexer DSF (not the Windows mpeg "splitter"). See my previous post. Although it's possible, with the right set of DSFs, to save the audio to .ac3, I just save to a .wav file and use besplit to "fix" the ac3 wav file to a .ac3 file afterwards. If you have trouble, I'll try to find time to post my graphedit filter graph for you.

[Guest]

I finally figured this out!

Private stream 1 carries the AC3. But the way it is packed in there is different for DVD and non-DVD streams. For DVD streams, there is an audio substream number and an associated DVD-specific header. For plain vanilla MPEG, there is no audio substream number and header. I revised DGIndex to detect the difference and handle it. I tested it on one Fusion card capture and it worked. Please test this on your Fusion MPG streams and notify me if anything doesn't work. Thank you.

BTW, the 0x0b, which I was trying to interpret as the substream id, was actually the first byte of the AC3 sync word!

Of course, because DVD2AVI was designed for DVD streams, it didn't know or care about this. And because DGIndex was derived from DVD2AVI...

http://neuron2.net/dgmpgdec/dgmpgdec147b12.zip

[calinb]

Thanks, Donald! Initial testing with a Fusion .mpg file indicates the new feature is working. However, I ran into a problem with Fusion .tp files that must've crept in somewhere; preview is not currently working on .tp files, though I can move the cursor along the timeline okay. Upon hitting F5, the information window opens but it contains no information and DGIndex freezes. It demuxes and makes project files though.

Let me know if you need any sample clips and :thanks:, as always!

[Guest]

Maybe you left the playback speed set to single-step. But if not...

Which version?

If it does that with 1.4.7b12, please upload a stream to my FTP. I'll PM you the password. Good timing, I was just about to release 1.4.7.

Thank you for pointing it out.

[squid_80]

Try waiting a while. It doesn't freeze, just pauses for a long time.

[Guest]

Not that again! I thought I fixed that. Well, if you want to give me the stream (up to 500MBytes), I'll try again. :(

[squid_80]

Hmmm, problem is the pause isn't noticeable unless the file is fairly large (>1gb). Also it doesn't happen with all files, just some; The bad files come up with "Could not find PAT/PMT tables!" when the Detect PIDs: PAT/PMT option is selected, while the good files (all are recorded from the same broadcast channel) show a selection of PIDs. However even with a good file it's possible to make it pause by setting the audio PID to something non-existent or to a PID which contains no audio data.

[Guest]

The problem is that DGIndex is trying to determine the audio type. He does it by first parsing the file for PAT/PMT that describes the streams. DGIndex reads a lot of the file looking for the PAT/PMT. If that fails, DGIndex reverts to raw PID parsing and some good guessing. It's more relable to use PAT/PMT, but they may not be present.

What is the longest pause you see with this type of stream?

I wonder if these streams really do have PAT/PMT and my parser is confused. Can you give me 100MBytes of such a stream?

[calinb]

I've only encountered one program (Lateshow with David Letterman) that has this problem, but it's a problem with all three .tp recordings that I've captured of the show (two are 1080i and one is 480i--all with 2ch AC3). I'm using 1.4.7b12 in normal playback speed. I'll do more testing and capture some short clips next time the show airs.

Strange!

[Guest]

Are you able to answer this, please?

"What is the longest pause you see with this type of stream?"

[eb]

In this new file from post 1 there are no audio, there are only paddings as PID BE.
File converted to Xvid 3MB.
ftp://www.eb.enterpol.pl:eb@www.eb.enterpol.pl/emotion.avi right click and save

eb
EDIT Longer sample needed

[Guest]

Sorry, eb, I missed your point entirely. Can you explain better please?

[squid_80]

I tested the .tp files I have available and the longest pause is 55 seconds, from a 1.8gb file.

120mb sample (recorded with a Fusion DVB-T lite) available <removed>, if the server acts up and you can't get it just pm me your ftp password and I'll upload it.

[eb]

bbINFO - version 1.9, by Brent Beyeler (beyeler@home.com)
speed increases by, Apachez and Christian Vogelgsang

File 1 is an MPEG-2 Program Stream


PACK #0, pack_start_code = 000001BA, SCR = 0.000000 ms

system_header_start_code = 000001BB
stream_id = BE Padding Stream, packet #0,

PACK #1, pack_start_code = 000001BA, SCR = 49.473926 ms
stream_id = E0 Video Stream 0, packet #0, PTS = 95443665.611111 ms, DTS = 95443632.244444 ms
0 - sequence_header_code
12 - extension_start_code
22 - extension_start_code
34 - group_of_pictures header
42 - picture_start_code
50 - extension_start_code

PACK #2, pack_start_code = 000001BA, SCR = 49.473926 ms
stream_id = E0 Video Stream 0, packet #1,
......
Summary:

MPEG Packs = 18432
System headers = 1
Padding Stream packets = 484, total bytes = 471145
Video stream 0 packets = 18431, total bytes = 36847393

Info for file from post 1

Link from last post not working

EDIT: site data deleted

[Guest]

Why are you giving me an Xvid Stream and what do you want me to do with it?

Now you are asking me to upload something.

I'm totally baffled. Does anyone else understand this?

[eb]

Sorry neuron2,
it was not directed to you exactly, as usually we have troubles to understand each other.

Ask for upload directed to squid_80

Xvid presented as example that pulldown3.2 is needed to go to 23.976 f/s and to fluent display.

Greetings to you neuron2.

[Guest]

Ah, sorry. Yes, that link is hosed. I sent squid the password to my FTP also.

Sorry for being so dense (two Kirin Ichibans will do that), and best wishes to you!

[Guest]

I think I have this long delay issue fixed once and for all now:

http://neuron2.net/dgmpgdec/dgmpgdec147rc1.zip

[calinb]

Yup--147rc1 took care of it.:thanks: I don't have to use raw PID detection on these files anymore either; PAT/PMT works!

Excellent work again, Donald. I don't know how we'd get by without your tools contributions.

[Guest]

Great!

Thank you for the feedback and calling attention to the problem.

And thanks to squid_80 for the test file.

[calinb]

I'm sending some Fusion mpg test files now. 147rc1 is having trouble finding the AC3 audio in Fusion mpg files again.

Donald, have you ever played "whack a mole?" :eek:

[Guest]

Um, it appears that you did not set the option for "Force Fusion-Style Audio". If you do that, the "no audio" stream is fine.

The "misreported" stream then shows no audio, nor does any show up in Mplayer Classic, or any other player I have. I'll look at it in the debugger.

EDIT: I can't see any recognizable audio in the "misreported" stream.

Conclusion: no problems found with DGIndex.

[calinb]

Um, it appears that you did not set the option for "Force Fusion-Style Audio". If you do that, the "no audio" stream is fine.
Doh! I forgot to do that. I guess I blew away my .ini file with RC1 and forgot to re-enable Fusion mode.

-Cal
:stupid: