DGAVCDec 1.0.9

[bob0r]

Alpha 23 detects an AC3 and an MP2 and I can demux both of them.

Hmm indeed i see it also now, weird, thanks for checking them!

[bob0r]

I tried mkvtoolnix 2.2.0 and set the FPS in the Format specific options tab. Everything worked fine and the output was an MKV that played at 25fps and was reported as such by MPC. I don't know if it is relevant but I named the raw AVC file with a '.264' extension.

mkvtoolnix: 2.1.0: http://files.x264.nl/itv.hd.remux.choppy.mkv
mkvtoolnix: 2.2.0: same result, can you share a sample of your .mkv?

[Guest]

mkvtoolnix: 2.1.0: http://files.x264.nl/itv.hd.remux.choppy.mkv
mkvtoolnix: 2.2.0: same result, can you share a sample of your .mkv? No point, really, because yours plays just fine and smooth for me.

What AVC codec are you using?

[Flaarn]

Ok , I'll jump in here, there will be some oddities if you capture further test streams with itv hd, as they are playing with the parameters right now, in the last few hours they have been playing with the audio channels, ie left/right channel testing.
It reminds me of the very early bbc hd testing, but at least there are some dvb apps out there that can handle it now, basically expect some further strangeness for a few days until it goes "live".

[bob0r]

CoreAVC Decoder :O)

Guess we just found another .mkv/coreavc bug (as .ts plays just fine)


I have another question for you Neuron:
Pengvado just told me a little about colorspaces and the matrixes used (tv/pc level) My question: Does dgavcdec (libavcodec) always set the same level as the source is?

"Is dgavcdec "decoding" the video and serving the results to the encoder, or does it work another way"?

[Guest]

Pengvado just told me a little about colorspaces and the matrixes used (tv/pc level) My question: Does dgavcdec (libavcodec) always set the same level as the source is? DGAVCDec passes the YV12 through as is.

"Is dgavcdec "decoding" the video and serving the results to the encoder, or does it work another way"? DGAVCDec decodes the video using libavcodec and passes uncompressed video out via Avisynth using DGAVCDecode.dll.

[ChiDragon]

I'm having issues with another movie now... for one thing the # of frames is too short and thus the length doesn't match up with DirectShowSource or the PS3.

Here's a chunk (http://chidragon.thedessie.com/00000.TSSplit.1-670.m2ts), I'm not sure how much this will help since it doesn't seem to have the same shorter length issue. Here the # of frames appears the same in DGAVCIndex and DirectShowSource but less with AVCSource. The full movie has seeking issues with AVCSource, and so does this clip. "Go to" frame 67 and it's different than if you seek to it from 0. "Go to" frame 100 = Visual C++ runtime error.

[Gannjunior]

Hi neuron,

i'm doing a test and i'm simulating an "avc-intra" compression.
So, to get it, i managed megui to have GOP=1 and minimum i-frame interval to 1, to be sure to have only I-frame encode.

Since i'm calculating some sessions of SSIM and PSNR, i used DGAVC to prepare the comparison script.
but i've noticed that dgavc sees only one frame of my clip. (i tried to play it in mpc and it is ok).
i hope to have explained myself...

pls let me know if there is a solution.

thanks!
ciao

[Guest]

@Gannjunior

I need a stream that allows me to duplicate your issue.

[Gannjunior]

here it is neuron. thanks!

http://www.savefiles.net/d/85otcvj2jh5yh.html

[MBoufleur]

Hi neuron2,

I've recently received three TS files with unusual SI tables. Those are all files captured off the air in SBTVD format (similar to ISDB-T), so it's not DVB, nor ATSC.
SBTVD is the new Terrestrial Digital Television Standard in Brazil, recently developed and taking it's basis from Japanese ISDB-T, but using H.264 HD for video and AAC for audio.
It is usually has 2 programs inside the transport stream: 1 for HD in 1080i and another one for mobiles using Baseline profile.

It seems no computer software is able to play these files correctly to date. Most can display video, but no audio.

I was able to change between programs in the TS File using Graphedit and ArcSoft MPEG Demuxer, but I could'nt find any AAC decoder filter that would connect to the ArcSoft Demuxer, and the ArcSoft audio filter doesn't seem to play AAC. MainConcept Demuxer also works, but only extracts the Baseline part, so I'm pretty much stuck there.

Strangelly, some playout applications incorrectly identify the AAC streams as MPEG-1 audio, such as MPC and DGAVCIndex as well (see below).

MediaInfo (http://mediainfo.sourceforge.net/en) seems to be an app that can identify all streams correctly, but it does not display tables information, nor does it separate streams by programs.

I'm making these three TS files available for download, for testing purposes. There is a full 3 min. version for each of these three files (around 250MB each), and smaller 2MB versions for quick tests.
You can download these files here (http://www.cbclip.com.br/Manhattan).
Username: Manhattan
Password: M4nh4tt4n

Some things I noticed using DGAVCindex:
- Sample 1 shows Mpeg Audio available in Audio Demux, but the same PID is identified as AAC LATM/LOAS in "Detect PIDs" menu;
- Sample 2 makes DGAVCIndex to crash while saving project with the complete file;
- Sample 2 is the only one of the three samples with 2 programs. If you click on "Set PID" Menu in DGAVCIndex, it automatically shows the Program 1 Vídeo PID with the PCR PID from Program 2.
- Sample 2 shows 25fps for the Baseline information window. In MediaInfo, it states that the stream is VFR.
- Sample 3 shows no audio for demuxing in "Audio Demux" window.

I know H.264 is itself is a relative new standard, and DGAVCIndex is in it's beginning. If you think these files might be worth of debugging, I would be very grateful.

[Guest]

@ChiDragon and Gannjunior

I have your streams. I want to get DGMPGDec 1.5.0 out the door today and then I will look at them.

@MBoufleur

Your link to the streams is not working for me.

[MBoufleur]

After you click the link, you'll be redirect to a web page with a dialog box. You just have to press OK to view the page content.

Edit:
I just found out that the link I previously written doesn't work with IExplorer due to security concerns (it was working fine with Firefox, though...).
I removed the username and password from the link itself and unfortunately, one will have to type them when accessing the webpage.

[Guest]

That audio is AAC LATM/LOAS. I don't support that yet, and I can't predict when I may get to it.

[MBoufleur]

When you mean support, it means that the file cannot be demuxed, or decoded (like the function "convert to WAV" from DGIndex)?

[Inventive Software]

Decoded. DGAVCDec primarily uses libavcodec for audio and video decoding, so if that doesn't support it, DGAVCDec won't.

[Schmendrick]

@Mboufleur: You can demux all the streams with TSRemux. The "MPEGaudio"-files you have to rename *.aac. Winamp is able to play these files and also to convert them to Wav-files. Thus you could e.g. reencode them either to real MPEG-audio or to AC3 and then remux these audiofiles back with TSMuxer.

In the meantime I have downloaded one full sample video. Using TSRemux I demuxed the video- and the audio-stream as avc- and aac-file resepctively. The aac-file I converted into a stereo-wav-file using Winamp with its file-output-plugin. Then I encoded this wav-file with Belight using the AC3enc-encoder into a stereo AC3-256kbit-file. Opening the original "mpg"-ts-file with a hex editor I searched for the first PTS-value of the video stream which can be found if you search for the first occurence of the video flag string "00 00 01 E0" and then the five bytes of the PTS value which is located with a 5 bytes offset after the video flag. In the first sample it is "31 00 CD 33 9D". Then you have to search for the first PTS-value of the audio stream which can be found if you search for the first occurence of the MPEG-audio flag string "00 00 01 C0" and then the five byted of the PTS value which is located with a 5 bytes offset after the MPEG-audio flag. Here it is "21 00 C7 F1 91". As these values are different you have to calculate the time offset so how many seconds the audio is playing too early in this case as the audio-PTS is smaler than the video-PTS, as the least significant bit of the first byte of the PTS values can be ignored so that the "31"-value is equal to the "21"-value of the different PTS-values. As the calculation of the PTS values which is documented in the MPEG ISO standards is quite complicated I have written a simple VideoPTS-Calculator programme:
http://www.alice-dsl.net/schmendrick/VideoPTSCalc.exe
which uses .NET-2.0. You can enter the video PTS-value as the ASCII-string of the hex values of the first five bytes "21 00 CD 33 9D" in the left input box, then guess the time offset value (shift(sec)) in seconds e.g. "-0.82211" which is -822.11 milliseconds, then click on "compute" and obtain the audio PTS value in the right box "new VideoPTS". If you experiment a little with the "shift(sec)"-value you are able to find the exact offset value.

Then using TSMuxer you can remux the avc-video file with your new AC3-audio file using the time offset value in this case -822 milliseconds to obtain a perfectly synchronous ts-file which can be properly displayed e.g. using PowerDVDUltra.

I hope this method works for you. You can contact me if you have problems.

Schmendrick

[Gannjunior]

@ChiDragon and Gannjunior

I have your streams. I want to get DGMPGDec 1.5.0 out the door today and then I will look at them.

...

don't worry, look it when you can ;)

thanks,
ciao!

[MBoufleur]

@Schmendrik: indeed, I was able to demux some streams with TSremux. It's a remarkable tool and I guess it helps lots of people. I didn't know about this one though, but I'm glad you pointed it to me! :thanks:

TSremux has some small issues however:
- In stream 2, one cannot see the streams divided by structure, and it gets confusing as to which audio belongs to which video, by which PID...
- In steam 2 again: after saving the streams to elementary stream files, Upper Audio 18 is equal to the lower Audio 18, and Upper Video 17 is equal to Lower Video 17 - when it shouldn't. Those are two different audio and video streams and their output elementary stream files should not be the same. I used Manzanita do demux the same streams for comparison purposes, and all streams matched the exact size of the files from TSRemuxer (Video 17 is HD video, and Audio 18 is mobile audio). Perhaps some misconfiguration when identifying the streams without separating them by program, and then displaying them makes one equal to another?

@neuron2: Stream 2 (the bigger one) makes DGAVCIndex crash halfway when saving to DGA file. Only happens with this one, and always at the same point (frame 3414).

@Inventive Software: It seems some of the streams I'm running through DGAVCIndex and serving to VirtualDub end up showing some blocks on the middle of the image (at some points). At first I thought this was only for display, but when moving forward and backward frame by frame in VirtualDub, one can see that they will end up appearing in the final video. Is this caused by some decoding configuration in libavcodec that I could disable (In Loop Deblocking filter, maybe)?

[Inventive Software]

@Inventive Software: It seems some of the streams I'm running through DGAVCIndex and serving to VirtualDub end up showing some blocks on the middle of the image (at some points). At first I thought this was only for display, but when moving forward and backward frame by frame in VirtualDub, one can see that they will end up appearing in the final video. Is this caused by some decoding configuration in libavcodec that I could disable (In Loop Deblocking filter, maybe)?

That sounds like either a libavcodec decoding error, or a DGAVCDec error. Does the same thing occur with DGAVCIndex? If so, upload samples of those streams somewhere so that neuron2 can fix the bug and improve the program further. :)

Does it happen if you use DirectShowSource with ffdshow's H.264 decoder enabled in its config so it's used as the decoder?

[MBoufleur]

That sounds like either a libavcodec decoding error, or a DGAVCDec error. Does the same thing occur with DGAVCIndex? If so, upload samples of those streams somewhere so that neuron2 can fix the bug and improve the program further. :)

Does it happen if you use DirectShowSource with ffdshow's H.264 decoder enabled in its config so it's used as the decoder?

As for the crash, MPC using FFDshow doesn't crash at that specific point (although it doesn't play at full speed either)...

As for the blocks while decoding, It's the same error, so it should be liabvcodec's error. I used FFdshow and MPC to play some content, paused in a frame, and compared to the same frame in DGAVCIndex. Both look identical...

I would say it looks like bad interlace implementation for H.264 in libavcodec since it's been the Achille's heel of many H.264 decoders I've seen (specially for HD). VLC for example, also used libavcodec, but it couldn't display MBAFF in a proper way.

What does it really mean when the stream is neither MBAFF nor PAFF, and the content is interlaced? I always thought that either one of these two would appear if the content would come interlaced.

Anyway, All samples remain in the same place, so if you want to use the same files for more testing and debuging, the link from my first post should be ok.

[Inventive Software]

Interlacing = PAFF or MBAFF, there is no "alternative" interlacing method for H.264/AVC. I'd pin the blame on PAFF decoding, MBAFF decoding I think has been solid for a while. I shall take a look myself tomorrow, but if it's PAFF I'd consider letting ffmpeg's developers know, since they're the primary libavcodec developers.

[SeeMoreDigital]

Interlacing = PAFF or MBAFF, there is no "alternative" interlacing method for H.264/AVC. I'd pin the blame on PAFF decoding, MBAFF decoding I think has been solid for a while. I shall take a look myself tomorrow, but if it's PAFF I'd consider letting ffmpeg's developers know, since they're the primary libavcodec developers.Sony and Panasonic's AVC camcorders produce "Fields TFF" sources...

[Inventive Software]

Flexible interlaced-scan video coding features, including:
Macroblock-adaptive frame-field (MBAFF) coding, using a macroblock pair structure for pictures coded as frames, allowing 16×16 macroblocks in field mode (compared with 16×8 half-macroblocks in MPEG-2).
Picture-adaptive frame-field coding (PAFF or PicAFF) allowing a freely-selected mixture of pictures coded as MBAFF frames with pictures coded as individual single fields (half frames) of interlaced video.

Either the software you used to tell you that's wrong, or Sony & Panasonic really messed up...

[SeeMoreDigital]

Either the software you used to tell you that's wrong, or Sony & Panasonic really messed up...Just so as you know: -

DGAVCIndex reports the "Frame Structure" as being "Fields TFF".

MediaInfo reports the "Scan Type" as being "Interlaced" and the "Scan Order" as being "Top Field First".


Cheers

[Inventive Software]

That's fine, so nothing tells you whether it's PAFF or MBAFF?

[Guest]

DGAVCIndex says "MBAFF" for MBAFF and "Fields TFF" or "Fields BFF" for PAFF.

[Inventive Software]

DGAVCIndex says "MBAFF" for MBAFF and "Fields TFF" or "Fields BFF" for PAFF.

Perhaps relating the "Fields TFF" and "Fields BFF" to PAFF in the DGAVCIndex documentation would be useful. I didn't see any mention of PAFF.

[Guest]

You won't see any mention of "PAFF" in the AVC specification either. :devil:

[Inventive Software]

Then why's it used? :p

[Renzz]

Hi Neuron - I wonder if you might take a look at another sample TS from the new ITV HD channel - sample can be downloaded here (http://www.savefile.com/files/1588061).

I'm getting some odd effects when I try to deinterlace it (strange blockiness around moving edges). I have a feeling dgavcindex isn't handling it too well, resulting in a messed up DGA file.

If it's not dgavcindex that's at fault, can anyone suggest a good way to deinterlace this?

(I should point out that the new ITV HD channel, although broadcasting FTA, is using some sort of non-standard encoding so that only a certain type of satellite receiver can pick it up).

[SeeMoreDigital]

(I should point out that the new ITV HD channel, although broadcasting FTA, is using some sort of non-standard encoding so that only a certain type of satellite receiver can pick it up).May I ask where have you read this?


Cheers

[Guest]

It's a known problem with libavcodec. Demux the AVC video (you can use DGAVCIndex) and then open it with DirectShowSource().

[Renzz]

May I ask where have you read this?


Cheers

http://www.digitalspy.co.uk/forums/showthread.php?p=23341243&highlight=h222#post23341243

although it's a bit speculative, it seems the only receivers that can pick this up easily are Freesat ones. It is possible on various other FTA receivers but requires more work, since a simple transponder scan will not pick the channel up (it gets identified as a data service with no video PID). As it's being publicised as "Freesat-exclusive" the thinking is that this type of encoding is a way of achieving this.

[SeeMoreDigital]

http://www.digitalspy.co.uk/forums/showthread.php?p=23341243&highlight=h222#post23341243

although it's a bit speculative, it seems the only receivers that can pick this up easily are Freesat ones. It is possible on various other FTA receivers but requires more work, since a simple transponder scan will not pick the channel up (it gets identified as a data service with no video PID). As it's being publicised as "Freesat-exclusive" the thinking is that this type of encoding is a way of achieving this.Some of those guys over on DigitalSpy must be having a laugh...

MediaPlayer Classic with FFdshow can play the ITV-HD sample (although my 1-core 2.8Ghz P4 is not fast enough to play it at full frame rate) and displays the samples attributes correctly, i.e: Video: MPEG4 Video (H264) 1440x1088 [Video]
Audio: Dolby AC3 48000Hz stereo 256Kbps [Audio].

DGAVCIndex reports a whole load of stuff on various PIDs that may or may not be active in the transport stream.

Also, after running the ITV-HD sample thru' TSremux, my Ziova CS615 (pre-release sample unit) had no trouble playing it in hardware.

Anyway, until ITV-HD officially launches they can use what-ever PID's they like during their test transmission phase!


Cheers

[j7n]

I apologize if this trouble has been discussed before. This thread is 27 pages long.

I have a problem with an H.264 elementary stream. It plays longer than is indicated by DGAVCIndex and the very end (6 seconds out of two hours) is cut off. This was discovered only after encoding with x264. When seeking to the end, the last frames display properly. Problem only occurs when playing through a significant portion of the file, or maybe the entire file.

I had to make this AVS script to encode the entire file:
raw = AVCSource("The_Fifth_Element_-_Remastered.dga", deblock=true)
raw = crop(raw,4,148,-4,-144)
raw = LanczosResize(raw,1280,528)
endraw = trim(raw,180800,0)
end = trim(endraw,14,0)
raw + end

I can't think of a way to show this in action. Obviously it's impossible to upload the whole two hours of video. (Possible but probably not worth it.)

[Guest]

Please post a link to the DGA file or attach it.

[j7n]

http://j7n.sytes.net/temp/The_Fifth_Element_-_Remastered.dga.rar

[Guest]

Counting FRM in the DGA gives 180967 frames.

I don't understand what you are doing with the trims in your script.

First, you set endraw to include the last 167 frames. Then you set end to include the last 14 frames of endraw. You could have done that in one operation. Then you concatenate the original file with the last 14 frames. Why are you doing all this?

Please make a DGA file for the encoded x264 file and post that. If the frame count is correct, then it is a player issue.

[j7n]

If multiplexed together with unmodified audio, both streams are possibly off by no more than 100ms at the start of credits – not by 5-6 seconds as might be expected if there were doubled frames.

I demuxed the file transcoded (http://j7n.sytes.net/temp/demuxed.DGA.rar) from this AVS with mkvextract. DirectShowSource("x264output.mkv") with Haali splitter and ffdshow tells that there are 181122 frames [2:05:54]. However AVCSource("x264output_demuxed.dga") says there are only 180752 frames [2:05:38].

I don't understand what you are doing with the trims in your script.
Seeking with DGAVCIndex in the original *.264 produces artifacts (http://j7n.sytes.net/temp/demuxed-artifacts.jpg). Now that I think about it several trim()s would probably not made a difference. But the result was perfect, so I left it that way.

[Guest]

You didn't tell me what the trims and concatenation are for.

I'm not clear on what problem you are reporting. DGAVCDec gets less frames than DirectShow? You get artifacts when decoding a stream? Both?

Can you demonstrate the problem with a smaller stream that you can give me? If not, it's going to be difficult to do anything for you, unless you want to mail it to me.

[j7n]

You didn't tell me what the trims and concatenation are for.
When playing the whole stream with AVCSource() in one shot, I get slightly less frames than there actually are. It is possible to recover and use the lost frames using trim().

Double trimming was indeed pointless. Articacts only appear when seeking backwards.

[Guest]

I can't help you without any streams.

[G_M_C]

Hi Neuron2,

As you might know, i've been a long-time user of DGAVCDec (since the first Alpha as a matter of fact). I just drop in now and then to thank you for your effords, and for your work; Just as i do now.

But we've been on alpha's a long time now, and alpha 23 seems to "be around" longer than other, previous, versions. And i wonder if this time between updates is because you are planning a new update, and if the "beta-stage" is allready in sight ....

:)

[Guest]

I was working on DGMPGDec 1.5.0. Now I'm taking a little rest before doing another spurt on DGAVCDec. I have less time nowadays due to lifestyle developments. :)

[audyovydeo]

have less time nowadays due to lifestyle developments. :)

<trivia>

"One small lifestyle development for one man, one giant loss for the AVC community"

</trivia>

[Guest]

It's not so small. :)

[G_M_C]

It's not so small. :)

Might not be small but it is :) aparently ... Can we/shall we congratulate you on anything maybe ?

[Guest]

Not small but not quite that big! But thank you.

[Guest]

i'm doing a test and i'm simulating an "avc-intra" compression.
So, to get it, i managed megui to have GOP=1 and minimum i-frame interval to 1, to be sure to have only I-frame encode.

Since i'm calculating some sessions of SSIM and PSNR, i used DGAVC to prepare the comparison script.
but i've noticed that dgavc sees only one frame of my clip. (i tried to play it in mpc and it is ok).
i hope to have explained myself...

pls let me know if there is a solution.
I have a fix for this and will post it this evening. Thank you for pointing this out. My code for detecting a new picture was missing a clause.