DGAVCDec 1.0.9

[rebkell]

I have a question about discontinuities that happen sporadically in .ts streams, how does DGAVCDec handle the incomplete video and audio units? Does it just drop them when demuxing the audio and not just serve them when frame serving?

[Guest]

It blindly demuxes what it sees in transport packets. If there is an incomplete packet, it will demux junk and then resync to the next packet sync for the next packet.

So it is completely indeterminsistic what you will get if your stream is errored.

Yes, I know it is suboptimal. But I think stream cleanup should be done in a separate application run first, similar to ProjectX. You're welcome to write one if you need it. :)

I haven't needed it yet, so I never wrote one. Sorry!

[rebkell]

It blindly demuxes what it sees in transport packets. If there is an incomplete packet, it will demux junk and then resync to the next packet sync for the next packet.

So it is completely indeterminsistic what you will get if your stream is errored.

Yes, I know it is suboptimal. But I think stream cleanup should be done in a separate application run first, similar to ProjectX. You're welcome to write one if you need it. :)

I haven't needed it yet, so I never wrote one. Sorry!

Actually I was just wondering, I haven't really noticed any problem with the junk. I was pretty sure it did what you said, but just wanted to make sure. Does the above apply to the video also, you just send what ever fragment you've got to the encoding app?

[Guest]

Video is a little different, because it is trying to parse NALUs. But the effect is the same.

[rebkell]

Video is a little different, because it is trying to parse NALUs. But the effect is the same.

Ok, thanks. I'm still trying to grasp the NAL units, but I take it if it's far enough along in the elementary stream before the corruption, that you'll serve up a frame?

[Guest]

Not necessarily; it depends on how much transport is lost.

[lchiu7]

It blindly demuxes what it sees in transport packets. If there is an incomplete packet, it will demux junk and then resync to the next packet sync for the next packet.

So it is completely indeterminsistic what you will get if your stream is errored.

Yes, I know it is suboptimal. But I think stream cleanup should be done in a separate application run first, similar to ProjectX. You're welcome to write one if you need it. :)

I haven't needed it yet, so I never wrote one. Sorry!

I personally find the tool invaluable for getting the AAC-HE LOAS streams out of our DVB-T broadcasts. But as neuron2 says, the output can be indeterministic depending on the source material.

I find occasionally if I have a poor stream (dropouts in the broadcast) the program can actually top demuxing part way through the file. I did upload a sample for DG to test and a later release fixed that particular file but the other day with A30 I found it had stopped 50% through the file (with no erros). Since the file is 3.3G I need to find the offending section and extract it for DG to test but that is proving difficult.

[rebkell]

Not necessarily; it depends on how much transport is lost.

One more question, and I'm through :) Can it be determined by parsing the .dga file or is it more of a runtime type decision on whether a particular frame is served?

[Guest]

If a frame is detected it will be served. If it is missing due to a transport drop out, then it won't.

[laserfan]

I wrote a little c# app, that analyzes the dga files for me, it may not work correctly for you depending on the designation of I, P and B frames, but it should count frames and give you the frame and approx time where the IDR frames occur. You can try it if you want.

http://rebkell.net/readDGA.exeTried your app rebkell and it worked fine for me, but I was looking to ID I-Frames along with the time code and a visual image (to check scene changes & be able to pick chapter marks). Guess I'll continue to use Avidemux, but thanks for posting!

[lchiu7]

Tried your app rebkell and it worked fine for me, but I was looking to ID I-Frames along with the time code and a visual image (to check scene changes & be able to pick chapter marks). Guess I'll continue to use Avidemux, but thanks for posting!


Have you tried TS Packedt Editor? It identifies frame types (IIRC), has visual display (you need to install the Haali media splitter for some media types), and also identifies discontinuities and other strange things in the file

[laserfan]

Have you tried TS Packedt Editor? It identifies frame types (IIRC), has visual display (you need to install the Haali media splitter for some media types), and also identifies discontinuities and other strange things in the fileAnother tool I didn't know about! Nice suggestion but Avidemux works great for my purpose--we now return on-topic thanks! :)

[hellfred]

Slow progress on the H246 decoder of libavcodec, this time from the ffdshow developers.
ffmpeg revision 15773 (http://svn.mplayerhq.hu/ffmpeg?view=rev&revision=15773)
And a call for a helping hand [1]:
Also besides these, it would be interresting to also parse PPS/SPS/SEI in
the AVParser (h264_parser.c). While that is unrelated to your patch, it
would be a step toward fixing some long standing timestamp issues with
h264 in mpeg-ps/ts and should not be too hard as the existing parsing code
could be shared.
[1] http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2008-November/055553.html

[lchiu7]

Perhaps due to atmospheric conditions or whatever, lately I have been encountering captured files from our DVB-T broadcasts which when dgavcindex tries to demux, it stop and notes ended successfully well before the end of file. This results in an AAC file which only covers part of the show.

The errors I have seen look like this
http://www.datamix.info/Capture.JPG

I normally click cancel since I press Yes the errors keep on coming.

Finally the file hasn't been processed completely.

Though they might be different issues if I open the file in TS Packet Editor and run a scan, it finds no errors in the file.

A sample (about 70Mb) can be found here

www.datamix.info/flashpoint.ts

Larry

[poisondeathray]

Ichiu - SMplayer seems to be able to play these LATM/LOAS AAC audio both separately, and decoding properly directly out of the .ts with the newest builds

I don't know enough figure out how this could be used in a dump, construct a graph, or use to the command line, but perhaps it might work?

Anyways perhaps we could carry the conversation over to the audio section?
http://forum.doom9.org/showthread.php?t=140570

[Guest]

@Larry

My transport parser is barfing. The stream seems OK. I thought I had this fixed, but apparently not. I'll get right on this.

[lchiu7]

@Larry

My transport parser is barfing. The stream seems OK. I thought I had this fixed, but apparently not. I'll get right on this.

Thanks for the quick response. I hadn't noticed this since I had developed a process for demuxing the files in batch, playing Winamp to get wav, besweet to get AC3 and then mux it back. But while watching a show so processed, the audio stopped half way through! So investigation led me to the issue noted.

[Guest]

Those streams are stupid but legal. In a transport packet with payload_unit_start_indicator=1 (so we expect a PES packet to start in this transport packet) they include enough stuffing bytes in the adaptation field such that the PES header itself gets split between transport packets. It's perverse, really. It's like, "here, let's see if this transport parser can tolerate the stupidest legal thing we can think of".

I was able to hack it to work, basically to prove my analysis correct, but it is not an acceptable solution. I'm going to have to redesign the transport parser in all the DG tools to support this. It will take some time.

Your workaround for now is to demux the stream using a demuxer that supports this silliness, such as Elecard XMuxer Pro. Then index the elementary video stream.

[lchiu7]

Those streams are stupid but legal. In a transport packet with payload_unit_start_indicator=1 (so we expect a PES packet to start in this transport packet) they include enough stuffing bytes in the adaptation field such that the PES header itself gets split between transport packets. It's perverse, really. It's like, "here, let's see if this transport parser can tolerate the stupidest legal thing we can think of".

I was able to hack it to work, basically to prove my analysis correct, but it is not an acceptable solution. I'm going to have to redesign the transport parser in all the DG tools to support this. It will take some time.

Your workaround for now is to demux the stream using a demuxer that supports this silliness, such as Elecard XMuxer Pro. Then index the elementary video stream.

Thanks for looking at it. I guess the strange thing is, this silliness is random. Sometimes I can demux a file completely with no problems, sometimes I encounter this problem. Oh well - will wait till you fix it and/or look at the Elecard XMuxer Pro solution

[Guest]

I guess the strange thing is, this silliness is random. Sometimes I can demux a file completely with no problems, sometimes I encounter this problem. It just depends on how much stuffing they add.

[Guest]

Lucky you. I figured out a quick fix that seems to work OK for these streams. I'll release it ASAP.

[Guest]

* Added support for all POC order count types.

* Made a hack for transport streams which have PES headers split across transport packets.

http://neuron2.net/dgavcdec/dgavcdec.html

[Snowknight26]

Topic title still states 1.0.4.

[lchiu7]

It just depends on how much stuffing they add.

Since the files are not pristine rips from HD-DVD or BR but in fact from UHF transmissions, could this be from dropouts in the transmission stream? Watching these streams I can occasionally see video glitches which must be present in the video stream since I always watch captured streams, not the live ones.

[Guest]

No, there are no errors at the time the silliness occurs.

[lchiu7]

No, there are no errors at the time the silliness occurs.
Out of interest when this silliness occurs, does it impact the demuxing of the audio versus handling the video frames. I ask because how hard would it be to have an option that tells the program when such errors occur to continue demuxing the audio while not guaranteeing the quality of the data in the dga file.

Slightly selfish but my main purpose of the program is to demux the audio - I don't normally load the dga file into Virtualdub or avisynth since something like MeGui is okay with handling the file with DirectShow

Thanks

[Guest]

A split PES header is not an error. It is a legal construct. I wan't handling it properly. It is fixed. Your audio should be fine.

Are you saying that you still have a problem?

[lchiu7]

A split PES header is not an error. It is a legal construct. I wan't handling it properly. It is fixed. Your audio should be fine.

Are you saying that you still have a problem?

No - timing error in the responses. I hadn't seen you had released 1.06. Downloaded it and it went through the entire video file (the one with the extract I uploaded) with no errors. Winamp was able to play the aac file fine and output the wav file I need.

Thanks a bunch for that - really appreciated.

[Guest]

Ah, good. Thanks for pointing out the problem and for your other useful contributions.

[kebulek]

neuron2: I just had a 1-2s glitch in my HD stream and DGAVCIndex 1.0.6 did it unwatchable for 12 seconds (still image with many blocks). :-( Is this DGAVCIndex's fault or just bad stream?

Sample with glitch (10MB):
http://rapidshare.com/files/168005722/hd-sample.ts.html

It's .PVA demuxed by Mencoder, and then muxed to .TS by TSMuxeR.

Note to everyone - please do not download this file before Neuron2, because rapidshare said "Your file has been saved and can now be downloaded only 10 times." :-( Thank you.

:thanks:

[new_age]

Any idea what can I do to properly decode h264 stream when I get this error message

Sure. Get a recent Nvidia card and use DGAVCDecNV, which plays it flawlessly. Looks like a libavcodec issue.

Oops, I edited your post instead of replying. Sorry.

[Comatose]

new_age: when posting about a problem like this, you should include a sample...

[new_age]

You're right. Then I need some help. How can I cut from the h264 file properly?

[Comatose]

I guess if DGAVCIndex doesn't work, then you can mux to ts and use h264ts_cutter or mux to mp4 and use Avidemux.

[new_age]

I've put the sample here:

http://rapidshare.com/files/168170121/00002_err.zip.html

Please do not download except Neuron2 because it can be download only 10 times.

[bob0r]

I've put the sample here:

http://rapidshare.com/files/168170121/00002_err.zip.html

Please do not download except Neuron2 because it can be download only 10 times.

unzipped:
http://files.x264.nl/00002_err.264

[Atak_Snajpera]

Please do not download except Neuron2 because it can be download only 10 times.
So why do you use rapidshare??????? I can't understand why people are so addicted to rapidshare?????

[new_age]

off/flame:

Because I don't know any other filesharing site!!!!!!! Do you understand that?????

[kebulek]

neuron2: I just had a 1-2s glitch in my HD stream and DGAVCIndex 1.0.6 did it unwatchable for 12 seconds (still image with many blocks). :-( Is this DGAVCIndex's fault or just bad stream?

Sample with glitch (10MB):
http://rapidshare.com/files/168005722/hd-sample.ts.html

It's .PVA demuxed by Mencoder, and then muxed to .TS by TSMuxeR.

Note to everyone - please do not download this file before Neuron2, because rapidshare said "Your file has been saved and can now be downloaded only 10 times." :-( Thank you.

:thanks:

OK here is the same sample:
http://uloz.to/996704/hd-sample.ts

It's Czech filesharing server, so copy those four red digits a click on the green button "--> Stáhnout soubor <--". :)

I hope it will work... And if there is a better filesharing, let me know - thanks.

[Atak_Snajpera]

Because I don't know any other filesharing site!!!!!!! Do you understand that?????
have you ever heard about Google? (rhetoric question)

END OFFTOPPIC !!!

[new_age]

END OFFTOPPIC !!!

"""" :thanks: """"

[Guest]

neuron2: I just had a 1-2s glitch in my HD stream and DGAVCIndex 1.0.6 did it unwatchable for 12 seconds (still image with many blocks). :-( Is this DGAVCIndex's fault or just bad stream? If your GOPs are long then a large section of video can be destroyed by a short glitch.

[Guest]

@new_age

See here:

http://forum.doom9.org/showthread.php?p=1217903#post1217903

[new_age]

@new_age

See here:

http://forum.doom9.org/showthread.php?p=1217903#post1217903

Thanks for the info. Will be hard: using laptop. :-)

Anyway I've temporaly solved by using CoreAVC and DirectShowSource but it is slower than AVCSource.

[canTsTop]

i have "bad" sample http://www.mediafire.com/?y4tej0guuns

it indexes ok, but then i load avs in virtualdub, virtualdub stops responding and after 20-25 seconds virtualdub shows green frame, then i go forward frame by frame, i see correct picture at frame 7. and from frame 7 i step backwards and some frames that were previously green, now they correct.

sorry for bad english

EDIT:
ffdshow don't understand this sample too... :(

[Guest]

i have "bad" sample Yes, you do. The transport stream is corrupted. I can't help you with that.

[komisar]

neuron2
Can you compile DGAVCDec with this patch (http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2008-November/056659.html) to support H.264 predictive lossless?

[canTsTop]

Yes, you do. The transport stream is corrupted. I can't help you with that.

sorry about that, i don't know what happened to that sample...

here is same 10mb from same recording, but this time its not corrupted (checked with TS Packet Editor) http://www.mediafire.com/?5tyzt3ljthn

[Guest]

Can you compile DGAVCDec with this patch (http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2008-November/056659.html) to support H.264 predictive lossless? If it won't apply to the older version of libavcodec I use, then I can't absorb it. I'm not able to update libavcodec, due to several critical regressions for frame accurate seeking that the developers consider low priority (which in practice means do it yourself or forget it). Unfortunately I have too much of my own work to get involved in libavcodec development.

[Guest]

here is same 10mb from same recording, but this time its not corrupted (checked with TS Packet Editor) http://www.mediafire.com/?5tyzt3ljthn The stream contains no IDRs. You'll have to enable Treat I as IDR and accept that the first few frames will be trash.

Is that from the Hauppauge HD-PVR? If so, you can use the Graphedit hack to enable IDRs.