Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 AVC / H.264

Closed Thread
 
Thread Tools Search this Thread Display Modes
Old 23rd May 2008, 16:52   #1061  |  Link
MBoufleur
Registered User
 
Join Date: Sep 2006
Location: Brazil
Posts: 23
AVC Stream oddities

Hi neuron2,

I've recently received three TS files with unusual SI tables. Those are all files captured off the air in SBTVD format (similar to ISDB-T), so it's not DVB, nor ATSC.
SBTVD is the new Terrestrial Digital Television Standard in Brazil, recently developed and taking it's basis from Japanese ISDB-T, but using H.264 HD for video and AAC for audio.
It is usually has 2 programs inside the transport stream: 1 for HD in 1080i and another one for mobiles using Baseline profile.

It seems no computer software is able to play these files correctly to date. Most can display video, but no audio.

I was able to change between programs in the TS File using Graphedit and ArcSoft MPEG Demuxer, but I could'nt find any AAC decoder filter that would connect to the ArcSoft Demuxer, and the ArcSoft audio filter doesn't seem to play AAC. MainConcept Demuxer also works, but only extracts the Baseline part, so I'm pretty much stuck there.

Strangelly, some playout applications incorrectly identify the AAC streams as MPEG-1 audio, such as MPC and DGAVCIndex as well (see below).

MediaInfo seems to be an app that can identify all streams correctly, but it does not display tables information, nor does it separate streams by programs.

I'm making these three TS files available for download, for testing purposes. There is a full 3 min. version for each of these three files (around 250MB each), and smaller 2MB versions for quick tests.
You can download these files here.
Username: Manhattan
Password: M4nh4tt4n

Some things I noticed using DGAVCindex:
- Sample 1 shows Mpeg Audio available in Audio Demux, but the same PID is identified as AAC LATM/LOAS in "Detect PIDs" menu;
- Sample 2 makes DGAVCIndex to crash while saving project with the complete file;
- Sample 2 is the only one of the three samples with 2 programs. If you click on "Set PID" Menu in DGAVCIndex, it automatically shows the Program 1 Vídeo PID with the PCR PID from Program 2.
- Sample 2 shows 25fps for the Baseline information window. In MediaInfo, it states that the stream is VFR.
- Sample 3 shows no audio for demuxing in "Audio Demux" window.

I know H.264 is itself is a relative new standard, and DGAVCIndex is in it's beginning. If you think these files might be worth of debugging, I would be very grateful.

Last edited by MBoufleur; 23rd May 2008 at 18:12.
MBoufleur is offline  
Old 23rd May 2008, 17:02   #1062  |  Link
Guest
Guest
 
Join Date: Jan 2002
Posts: 21,922
@ChiDragon and Gannjunior

I have your streams. I want to get DGMPGDec 1.5.0 out the door today and then I will look at them.

@MBoufleur

Your link to the streams is not working for me.
Guest is offline  
Old 23rd May 2008, 17:08   #1063  |  Link
MBoufleur
Registered User
 
Join Date: Sep 2006
Location: Brazil
Posts: 23
After you click the link, you'll be redirect to a web page with a dialog box. You just have to press OK to view the page content.

Edit:
I just found out that the link I previously written doesn't work with IExplorer due to security concerns (it was working fine with Firefox, though...).
I removed the username and password from the link itself and unfortunately, one will have to type them when accessing the webpage.

Last edited by MBoufleur; 23rd May 2008 at 18:15.
MBoufleur is offline  
Old 23rd May 2008, 18:12   #1064  |  Link
Guest
Guest
 
Join Date: Jan 2002
Posts: 21,922
That audio is AAC LATM/LOAS. I don't support that yet, and I can't predict when I may get to it.
Guest is offline  
Old 23rd May 2008, 18:48   #1065  |  Link
MBoufleur
Registered User
 
Join Date: Sep 2006
Location: Brazil
Posts: 23
When you mean support, it means that the file cannot be demuxed, or decoded (like the function "convert to WAV" from DGIndex)?
MBoufleur is offline  
Old 23rd May 2008, 19:00   #1066  |  Link
Inventive Software
Turkey Machine
 
Join Date: Jan 2005
Location: Lowestoft, UK (but visit lots of places with bribes [beer])
Posts: 1,953
Decoded. DGAVCDec primarily uses libavcodec for audio and video decoding, so if that doesn't support it, DGAVCDec won't.
__________________
On Discworld it is clearly recognized that million-to-one chances happen 9 times out of 10. If the hero did not overcome huge odds, what would be the point? Terry Pratchett - The Science Of Discworld
Inventive Software is offline  
Old 23rd May 2008, 23:35   #1067  |  Link
Schmendrick
Registered User
 
Join Date: Oct 2003
Location: Germany
Posts: 91
@Mboufleur: You can demux all the streams with TSRemux. The "MPEGaudio"-files you have to rename *.aac. Winamp is able to play these files and also to convert them to Wav-files. Thus you could e.g. reencode them either to real MPEG-audio or to AC3 and then remux these audiofiles back with TSMuxer.

In the meantime I have downloaded one full sample video. Using TSRemux I demuxed the video- and the audio-stream as avc- and aac-file resepctively. The aac-file I converted into a stereo-wav-file using Winamp with its file-output-plugin. Then I encoded this wav-file with Belight using the AC3enc-encoder into a stereo AC3-256kbit-file. Opening the original "mpg"-ts-file with a hex editor I searched for the first PTS-value of the video stream which can be found if you search for the first occurence of the video flag string "00 00 01 E0" and then the five bytes of the PTS value which is located with a 5 bytes offset after the video flag. In the first sample it is "31 00 CD 33 9D". Then you have to search for the first PTS-value of the audio stream which can be found if you search for the first occurence of the MPEG-audio flag string "00 00 01 C0" and then the five byted of the PTS value which is located with a 5 bytes offset after the MPEG-audio flag. Here it is "21 00 C7 F1 91". As these values are different you have to calculate the time offset so how many seconds the audio is playing too early in this case as the audio-PTS is smaler than the video-PTS, as the least significant bit of the first byte of the PTS values can be ignored so that the "31"-value is equal to the "21"-value of the different PTS-values. As the calculation of the PTS values which is documented in the MPEG ISO standards is quite complicated I have written a simple VideoPTS-Calculator programme:
http://www.alice-dsl.net/schmendrick/VideoPTSCalc.exe
which uses .NET-2.0. You can enter the video PTS-value as the ASCII-string of the hex values of the first five bytes "21 00 CD 33 9D" in the left input box, then guess the time offset value (shift(sec)) in seconds e.g. "-0.82211" which is -822.11 milliseconds, then click on "compute" and obtain the audio PTS value in the right box "new VideoPTS". If you experiment a little with the "shift(sec)"-value you are able to find the exact offset value.

Then using TSMuxer you can remux the avc-video file with your new AC3-audio file using the time offset value in this case -822 milliseconds to obtain a perfectly synchronous ts-file which can be properly displayed e.g. using PowerDVDUltra.

I hope this method works for you. You can contact me if you have problems.

Schmendrick

Last edited by Schmendrick; 25th May 2008 at 16:08. Reason: Adding solution to problem.
Schmendrick is offline  
Old 24th May 2008, 00:08   #1068  |  Link
Gannjunior
Registered User
 
Gannjunior's Avatar
 
Join Date: Jul 2002
Location: Italy, Genova
Posts: 206
Quote:
Originally Posted by neuron2 View Post
@ChiDragon and Gannjunior

I have your streams. I want to get DGMPGDec 1.5.0 out the door today and then I will look at them.

...
don't worry, look it when you can

thanks,
ciao!
__________________
P6T| 12Gb DDR3| i7-920 D0@3.8Ghz@1.168v+NOCTUA NH-U12P| Liberty 620| Stacker|2xRAID0 (4x500gb) Seag.72k.11| Seag. 500gb 72k.12| Seag. 1.5Tb 72k.11| GTX470@668/1843| U2410 24"| Se7en x64 on VRapt 300gb
Gannjunior is offline  
Old 26th May 2008, 21:31   #1069  |  Link
MBoufleur
Registered User
 
Join Date: Sep 2006
Location: Brazil
Posts: 23
@Schmendrik: indeed, I was able to demux some streams with TSremux. It's a remarkable tool and I guess it helps lots of people. I didn't know about this one though, but I'm glad you pointed it to me!

TSremux has some small issues however:
- In stream 2, one cannot see the streams divided by structure, and it gets confusing as to which audio belongs to which video, by which PID...
- In steam 2 again: after saving the streams to elementary stream files, Upper Audio 18 is equal to the lower Audio 18, and Upper Video 17 is equal to Lower Video 17 - when it shouldn't. Those are two different audio and video streams and their output elementary stream files should not be the same. I used Manzanita do demux the same streams for comparison purposes, and all streams matched the exact size of the files from TSRemuxer (Video 17 is HD video, and Audio 18 is mobile audio). Perhaps some misconfiguration when identifying the streams without separating them by program, and then displaying them makes one equal to another?

@neuron2: Stream 2 (the bigger one) makes DGAVCIndex crash halfway when saving to DGA file. Only happens with this one, and always at the same point (frame 3414).

@Inventive Software: It seems some of the streams I'm running through DGAVCIndex and serving to VirtualDub end up showing some blocks on the middle of the image (at some points). At first I thought this was only for display, but when moving forward and backward frame by frame in VirtualDub, one can see that they will end up appearing in the final video. Is this caused by some decoding configuration in libavcodec that I could disable (In Loop Deblocking filter, maybe)?
MBoufleur is offline  
Old 27th May 2008, 00:11   #1070  |  Link
Inventive Software
Turkey Machine
 
Join Date: Jan 2005
Location: Lowestoft, UK (but visit lots of places with bribes [beer])
Posts: 1,953
Quote:
Originally Posted by MBoufleur View Post
@Inventive Software: It seems some of the streams I'm running through DGAVCIndex and serving to VirtualDub end up showing some blocks on the middle of the image (at some points). At first I thought this was only for display, but when moving forward and backward frame by frame in VirtualDub, one can see that they will end up appearing in the final video. Is this caused by some decoding configuration in libavcodec that I could disable (In Loop Deblocking filter, maybe)?
That sounds like either a libavcodec decoding error, or a DGAVCDec error. Does the same thing occur with DGAVCIndex? If so, upload samples of those streams somewhere so that neuron2 can fix the bug and improve the program further.

Does it happen if you use DirectShowSource with ffdshow's H.264 decoder enabled in its config so it's used as the decoder?
__________________
On Discworld it is clearly recognized that million-to-one chances happen 9 times out of 10. If the hero did not overcome huge odds, what would be the point? Terry Pratchett - The Science Of Discworld

Last edited by Inventive Software; 27th May 2008 at 01:40.
Inventive Software is offline  
Old 27th May 2008, 00:37   #1071  |  Link
MBoufleur
Registered User
 
Join Date: Sep 2006
Location: Brazil
Posts: 23
Quote:
That sounds like either a libavcodec decoding error, or a DGAVCDec error. Does the same thing occur with DGAVCIndex? If so, upload samples of those streams somewhere so that neuron2 can fix the bug and improve the program further.

Does it happen if you use DirectShowSource with ffdshow's H.264 decoder enabled in its config so it's used as the decoder?
As for the crash, MPC using FFDshow doesn't crash at that specific point (although it doesn't play at full speed either)...

As for the blocks while decoding, It's the same error, so it should be liabvcodec's error. I used FFdshow and MPC to play some content, paused in a frame, and compared to the same frame in DGAVCIndex. Both look identical...

I would say it looks like bad interlace implementation for H.264 in libavcodec since it's been the Achille's heel of many H.264 decoders I've seen (specially for HD). VLC for example, also used libavcodec, but it couldn't display MBAFF in a proper way.

What does it really mean when the stream is neither MBAFF nor PAFF, and the content is interlaced? I always thought that either one of these two would appear if the content would come interlaced.

Anyway, All samples remain in the same place, so if you want to use the same files for more testing and debuging, the link from my first post should be ok.

Last edited by MBoufleur; 27th May 2008 at 00:45.
MBoufleur is offline  
Old 27th May 2008, 01:45   #1072  |  Link
Inventive Software
Turkey Machine
 
Join Date: Jan 2005
Location: Lowestoft, UK (but visit lots of places with bribes [beer])
Posts: 1,953
Interlacing = PAFF or MBAFF, there is no "alternative" interlacing method for H.264/AVC. I'd pin the blame on PAFF decoding, MBAFF decoding I think has been solid for a while. I shall take a look myself tomorrow, but if it's PAFF I'd consider letting ffmpeg's developers know, since they're the primary libavcodec developers.
__________________
On Discworld it is clearly recognized that million-to-one chances happen 9 times out of 10. If the hero did not overcome huge odds, what would be the point? Terry Pratchett - The Science Of Discworld
Inventive Software is offline  
Old 27th May 2008, 16:09   #1073  |  Link
SeeMoreDigital
Life looks better in UHD
 
SeeMoreDigital's Avatar
 
Join Date: Jun 2003
Location: Notts, UK
Posts: 11,724
Quote:
Originally Posted by Inventive Software View Post
Interlacing = PAFF or MBAFF, there is no "alternative" interlacing method for H.264/AVC. I'd pin the blame on PAFF decoding, MBAFF decoding I think has been solid for a while. I shall take a look myself tomorrow, but if it's PAFF I'd consider letting ffmpeg's developers know, since they're the primary libavcodec developers.
Sony and Panasonic's AVC camcorders produce "Fields TFF" sources...
__________________
| I've been testing hardware media playback devices and software A/V encoders and decoders since 2001 | My Network Layout & A/V Gear |
SeeMoreDigital is offline  
Old 28th May 2008, 03:13   #1074  |  Link
Inventive Software
Turkey Machine
 
Join Date: Jan 2005
Location: Lowestoft, UK (but visit lots of places with bribes [beer])
Posts: 1,953
Quote:
Originally Posted by Wikipedia's H.264 page
Flexible interlaced-scan video coding features, including:
Macroblock-adaptive frame-field (MBAFF) coding, using a macroblock pair structure for pictures coded as frames, allowing 16×16 macroblocks in field mode (compared with 16×8 half-macroblocks in MPEG-2).
Picture-adaptive frame-field coding (PAFF or PicAFF) allowing a freely-selected mixture of pictures coded as MBAFF frames with pictures coded as individual single fields (half frames) of interlaced video.
Either the software you used to tell you that's wrong, or Sony & Panasonic really messed up...
__________________
On Discworld it is clearly recognized that million-to-one chances happen 9 times out of 10. If the hero did not overcome huge odds, what would be the point? Terry Pratchett - The Science Of Discworld

Last edited by Guest; 28th May 2008 at 03:40. Reason: rule 4
Inventive Software is offline  
Old 28th May 2008, 18:04   #1075  |  Link
SeeMoreDigital
Life looks better in UHD
 
SeeMoreDigital's Avatar
 
Join Date: Jun 2003
Location: Notts, UK
Posts: 11,724
Quote:
Originally Posted by Inventive Software View Post
Either the software you used to tell you that's wrong, or Sony & Panasonic really messed up...
Just so as you know: -

DGAVCIndex reports the "Frame Structure" as being "Fields TFF".

MediaInfo reports the "Scan Type" as being "Interlaced" and the "Scan Order" as being "Top Field First".


Cheers
__________________
| I've been testing hardware media playback devices and software A/V encoders and decoders since 2001 | My Network Layout & A/V Gear |
SeeMoreDigital is offline  
Old 28th May 2008, 22:09   #1076  |  Link
Inventive Software
Turkey Machine
 
Join Date: Jan 2005
Location: Lowestoft, UK (but visit lots of places with bribes [beer])
Posts: 1,953
That's fine, so nothing tells you whether it's PAFF or MBAFF?
__________________
On Discworld it is clearly recognized that million-to-one chances happen 9 times out of 10. If the hero did not overcome huge odds, what would be the point? Terry Pratchett - The Science Of Discworld
Inventive Software is offline  
Old 29th May 2008, 00:22   #1077  |  Link
Guest
Guest
 
Join Date: Jan 2002
Posts: 21,922
DGAVCIndex says "MBAFF" for MBAFF and "Fields TFF" or "Fields BFF" for PAFF.
Guest is offline  
Old 29th May 2008, 04:46   #1078  |  Link
Inventive Software
Turkey Machine
 
Join Date: Jan 2005
Location: Lowestoft, UK (but visit lots of places with bribes [beer])
Posts: 1,953
Quote:
Originally Posted by neuron2 View Post
DGAVCIndex says "MBAFF" for MBAFF and "Fields TFF" or "Fields BFF" for PAFF.
Perhaps relating the "Fields TFF" and "Fields BFF" to PAFF in the DGAVCIndex documentation would be useful. I didn't see any mention of PAFF.
__________________
On Discworld it is clearly recognized that million-to-one chances happen 9 times out of 10. If the hero did not overcome huge odds, what would be the point? Terry Pratchett - The Science Of Discworld
Inventive Software is offline  
Old 29th May 2008, 04:49   #1079  |  Link
Guest
Guest
 
Join Date: Jan 2002
Posts: 21,922
You won't see any mention of "PAFF" in the AVC specification either.
Guest is offline  
Old 29th May 2008, 05:39   #1080  |  Link
Inventive Software
Turkey Machine
 
Join Date: Jan 2005
Location: Lowestoft, UK (but visit lots of places with bribes [beer])
Posts: 1,953
Then why's it used?
__________________
On Discworld it is clearly recognized that million-to-one chances happen 9 times out of 10. If the hero did not overcome huge odds, what would be the point? Terry Pratchett - The Science Of Discworld
Inventive Software is offline  
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 21:22.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.