GSpot v2.60 b00

[GodofaGap]

No. AFAIK both AVI and mp4 use decoding order. Of course MP4 does not need the N-VOP for padding, frames are re-aranged automatically. Although, I am not yet convinced the hack was really necessary for AVI, it is however for VFW. A Directshow (or any private API) decoder should be able to buffer as much as it wants, I think.

[bond]

my description of packed crapstream:
http://forum.doom9.org/showthread.php?s=&threadid=80430

[Wilbert]

@bond,

Could you post this stuff (or a reference to it) in one of the faqs?

[SeeMoreDigital]

Originally Posted by Wilbert

@bond,

Could you post this stuff (or a reference to it) in one of the faqs?

Indeed... good idea!

Is the information still fully current. Does any of it require updating/tweaking?

[stegre]

Originally Posted by foxyshadis

Based on stegre's explanation (which belongs in any video glossary!), I'd guess that they're P-VOPs with a not-coded bit, aka N-VOPs, only unlike packed bitstream these ones have an actual duration. I need to decompose some elementary streams and look at this myself!

Thanks :) I actually got "distracted" (in a good way, I guess) and ended up spending so much time on that "composite screenshot" and the associated post last night that I now hardly have time to read, much less respond to, many of the other (both older & newer) posts in this thread. But I'll be back here as soon as I have some time.

But I do want to mention that, yes, foxyshadis, you are absolutely 100% correct. Which is why I posted earlier that SMD's file he mentions above was so "interesting" to me: I so rarely, if ever, run into the "regular" NVOPS - ones with non-zero time - that I had, in some spots in the code, assumed that any "not coded" frame was just a "packed bitstream marker", without bothering to check for the "zero-time" requirement (even though I was actually aware of this). The currently released beta version does have this bug in a number of spots, causing some functions (like the GSpot's "VGS" display of SMD's file) to display quite erroneously.

But sure, definitely look into it yourself if you're so inclined! - it'll help me make sure I've got this all straight myself.

[Ezhihua]

Originally Posted by stegre

I've just released a new GSpot, v2.60 b00 - the first update in almost two years. So far it's kind of a pre-pre-beta, but it may be of particular interest to readers of this forum as many of the new features are of a highly technical nature. See the screenshots at the site to get an idea of what I'm talking about, and I'll be adding new info to the site as time permits.

I'll be checking back here on this thread (or others) for any comments - thanks..

http://www.headbands.com/gspot/

- Steve G

could you release a unicode version?

[bond]

Originally Posted by Wilbert

@bond,

Could you post this stuff (or a reference to it) in one of the faqs?

i posted a link to it in the mp4 faq already for a long time

people just really need to read it

[stegre]

Ahh, indeed, Bond's description is an excellent post, thanks. I now understand the reasons why they did it.

Originally Posted by squid_80

It was more the other way around. Packing 2 frames together was done for frame accurate editing of .avi files (with unpacked files there is a 1 frame lag) and the n-vop was added to the end so the final frame (the one packed with the b-vop) could be returned.

Correct, I now see that, having read Bond's post.

Originally Posted by SeeMoreDigital

...DivX elected to use "packed bit-stream" when generating ASP streams with 1B-VOP was because the frame order in AVI is a compromise ie: generating ASP streams in MP4 offers a slightly different frame order!

No, as GodofaGap's answer to your post also was saying in a slightly different way, the MPEG people had already "optimized" the coding order by making it different from the display order. Any info required to display "current" B frame, which requires info from a "future" I or P frame, was solved by actually including that I or P earlier in the file than the time corresponding to when it is to be displayed. And the process works for any number of consecutive B's, not just one. So the AVI file is not in any way "re-ordered" from the MPEG; it's just framed differently.

Originally Posted by squid_80

A not coded P-VOP (N-VOP) means show the last reference frame the codec received. In the case of an N-VOP inserted for packed bitstream reasons, the last reference frame is the P from the P+B packed combo. In the case of your samples, the reference frames are the I-VOPs.

VOPs have timestamps, not durations. N-VOPs inserted for packed bitstream have the same timestamp as the reference frame which replaces them.

Yes, correct too, I referred to them as "zero duration" for simplicity, indeed there is no "zero" to be found, you only "see" zero when you subtract its "timestamp" from an identical "timestamp" on a previous reference frame, as you say.

Though, (just to be argumentative ;) the values in the elementary stream are not exactly "timestamps" either - not in an absolute sense, like an MPEG System Stream has - one where you could look at a frame and say "oh, this is 12 min 57 sec into the file". But you're right, neither are they a relative "duration" for how long to display the frame for, nor are they even a relative duration for "how long to wait from the previous frame before playing this one". It's actually kind of a hybrid - it's normally a "timestamp" for when to play this frame relative to some other "reference" frame, but within a given one second interval only. Associated with it is a "carry value" (almost always "1" when it's not zero). When set to "1", it means you "lap" to a the next second. Then the "timestamp" values are become relative again. So in at least in a sense it's a "relative" ("duration-like") vs. "absolute" ("timestamp-like") system, though I agree your terminology is probably more accurate in this case.

Just to digress a bit, though: what's interesting, is what actually constitutes the "last reference" frame". It's usually the previous "non-B" frame, so unless you have a very slow frame rate or a lot of consecutive B's, this "carry value" mentioned above is usually zero and never exceeds one. But another interesting thing about SMD's "NVOP file" he created is that the timestamp does continue to increase and the carry values do keep going up, to 2, 3, 4 and even higher (even there are no B-frames at all). I assume that the encoder did its job correctly, though I'd like to check into that further.

Since the people who created the MPEG spec expected low values for that filed, almost always zero or one, they encoded it in an unusual fashion which is efficient only for small numbers. The variable length field's value is simply equal to the number of consecutive "1" bits in it until you hit a "0". So the otherwise semi-identical NVOP frames in SMD's file actually get keep getting longer and longer. If it got to say, 31 seconds of continuous N-VOPS without an I frame, that frame would be 4 bytes longer than the first frame. Not that that's a lot, and it's still a great savings overall compared to the same file encoded in a more "normal" way, i.e not using those N-VOPS.

But it's weird using what is at that point a 32 bit field to represent the number between zero and 31, when a normal 32 bit binary encoded field could represent a any number between zero and over 4 billion. But by making the field variable length like that, the field only takes 1 or 2 bits the great majority of the time, so they've obviously thought this all out....

[bond]

Originally Posted by SeeMoreDigital

Is the information still fully current. Does any of it require updating/tweaking?

yes, the info is still correct and also valid for avc for example

what is missing is what is done when using b-frames as references (as possible in x264) as this needs an even longer time lag as when using "normal" b-frames

[SeeMoreDigital]

Hi Steve,

I've generated another N-VOP file for your collection. This sample has a bit of motion... and helps to show why N-VOP's are so useful: -

Speaker Mapping File (MPEG-4+6Ch AC3).7z


Cheers

[unskinnyboy]

v2.60 b00 still indicate the number of AC3 channels wrong. It was a problem with the previous builds too. For e.g: a file with 6Ch AC3, GSpot says it is 2Ch. VirtualDubMoD reports the channel info correctly.

One very common and annoying problem with the previous builds was that, on some files, it would report the wrong audio information and then use *that* to calculate the video bitrate, and in the end report wrong video bitrate too. In v2.60 b00, since the video is being demuxed out first, the video bitrate is accurate, so at least that part is OK. Just the audio info is wrong.

[SeeMoreDigital]

Originally Posted by unskinnyboy

v2.60 b00 still indicate the number of AC3 channels wrong. It was a problem with the previous builds too. For e.g: a file with 6Ch AC3, GSpot says it is 2Ch. VirtualDubMoD reports the channel info correctly.

In my experience, previous GSpot builds would report 6Ch AC3 streams as having 5 channels!

I've just used VDM to mux an MPEG-4 video stream together with an 6Ch AC3 stream and got this: -




Cheers

[clsid]

Perhaps it only reports the numbe of channels incorrect in specific situations. If unskinnyboy provides a sample file, then Stegre can try to reproduce and fix the problem.

[unskinnyboy]

Originally Posted by SeeMoreDigital

In my experience, previous GSpot builds would report 6Ch AC3 streams as having 5 channels!

Note that your screenshot says 5.1 and not 5, and that is correct. 6 Channels is also referred to as 5.1, where the 5 stands for the normal-range speaker outputs (RF, C, LF, RR, LR) and the 0.1 for the LFE. So what you see is correct.

Originally Posted by clsid

Perhaps it only reports the numbe of channels incorrect in specific situations. If unskinnyboy provides a sample file, then Stegre can try to reproduce and fix the problem.

Yes, it happens only with certain files. I will upload a sample later.

[SeeMoreDigital]

Originally Posted by unskinnyboy

Note that your screenshot says 5.1 and not 5, and that is correct. 6 Channels is also referred to as 5.1, where the 5 stands for the normal-range speaker outputs (RF, C, LF, RR, LR) and the 0.1 for the LFE. So what you see is correct.

Yes... I know it's correct!

I am however, unable to replicate what you are seeing. So as clsid suggests, can you provide a sample?


Cheers

[stegre]

Yeah, as unskinny boy says "5.1" is correct and technically more informative than "6" (well, in this particular case, for AC3, "6" would have to mean "5.1", as AC3 only supports 5 "full bandwidth" channels max so a sixth would have to be an LFE (i.e. "Low Frequency Effects" i.e. a "subwoofer" channel). The "5.1" style terminology is not something I made up, btw, it's commonly used by audiophiles and in audio equipment specs.

As you can see in the spec below, "nfchans" ("number of full bandwidth channels") maxes out at 5. For clarity I could change the terminology to say ("6 chnls [5.1.]") or something.

The existence of an extra "LFE" channel can be added to any of the modes below, so there are really 16 different combinations ( at least in theory): any of the eight below with an extra LFE and any of them without one. They treat the LEF channel as a separate entity presumably because it's coded to use less bandwidth than the "full" channels (since there's no need - in fact it's probably undesirable - to encode all the hi-frequency components to send to a subwoofer).

So actually, when I use the "dot" terminology, it's more informative. Note the chart below (where, btw, "S" means "surround" aka "rear": e.g. "SL" is "rear left"). Ignoring the first oddball mode "0", if I said "2" channels it would have to be "L + R". I'd say "2.1" if it were "L, R" plus subwoofer. In the latter case VirtualDubMod, I assume, would say "3", which could mean either L + R (2 full range channels) with one LFE subwoofer channel, or it could also mean either L, R, C or L, R, S (each of those being 3 full range channels).

In fact, even my current terminology doesn't fully specify speaker placement, (my "3" could mean L, R, C or L, R, S) but at least it pares down the combinations in that regard, and in any event always identifies the LFE channel as separate entity, as it is indeed "different" than the other channels.

[stegre]

Originally Posted by unskinnyboy

...For e.g: a file with 6Ch AC3, GSpot says it is 2Ch. VirtualDubMoD reports the channel info correctly.

But back to this original question, yes, if it's reporting 2 channels for a "5.1" / 6 chnl AC3 stream that's clearly in error. I was about to post a GSpot update, beta 01, as early as today -- but let me take a quick look at the AC3 handling first. I'll test it on a bunch of files I have here, but it would be great if you could post your testfile as well in case I can't reproduce it. And I admit, I'm already seeing some weird things in a few very quick tests I did in the last few minutes, so there are indeed probably some bugs in that area.

Originally Posted by unskinnyboy

...it would report the wrong audio information and then use *that* to calculate the video bitrate, and in the end report wrong video bitrate too. In v2.60 b00, since the video is being demuxed out first, the video bitrate is accurate, so at least that part is OK. Just the audio info is wrong.

Yes, what you're describing is exactly what I did in earlier versions: In the absence of better info, I'd often use (when I only had the movie's duration, audio bitrate and overall total filesize):

TotalAudioSize = AudioBitrate * Duration
TotalVideoSize = TotalFileSize - TotalAudioSize;
VideoBitRate = TotalVideoSize / Duration

But I'd like to use that only as a last resort in these newer versions. As you say, one disadvantage of the above is that if there IS an error in the audio calc, it gets propagated to the video. It also introduces errors related to container overhead, and possibly significant errors if the container contains additional unidentified streams (though I do account for the simple case multiple audio streams when I do the above calculation).

As I add more capabilities to GSpot internally, my options increase - in fact to list a few, all of which are preferred over the above (though not necessarily in this order):

1) Display VideoBitRate as specified in a container, if available.
2) Display Video Bitrate as specified in a "header" chunk of a video stream demuxed from the container, if available, and if GSpot "understands" the format it's demuxing.
3) Display VideoBitRate calculated after demuxing the entire video stream, counting the bytes, and dividing by the duration (which doesn't necessarily require "understanding" the format of the demuxed video stream).

I have been improving, as you mentioned, and will continue to attempt to improve on using the best method available for each given situation.

[unskinnyboy]

Houston, we have a problem. On the whole file (which is a full length encode), GSpot reports 48000Hz 384 kb/s total (2 chnls), but on a sample cut out of it, GSpot says 48000Hz 384 kb/s total (5.1 chnls), the latter being the correct info.

Steve, I don't mind uploading the full file to you if it will help troubleshoot this issue. I will follow up with a PM.

[unskinnyboy]

Steve, what exactly in the avi are you looking at when you report 'garbage at end'? And why only for OpenDML avis > 2 GB? Would like to hear you out on this. Thanks.

FYI - http://forum.doom9.org/showthread.php?t=116852

[stegre]

I'll get back to that question and some other stuff about various updates I've made, especially ones related posts in this thread, but I just wanted to quickly mention that I've just posted an update: so here's quick links to the GSpot v2.60 b01 download page and the associated list of changes page. BTW, regarding representation of AC3 channels (earlier in this thread) I did my "doom9 homework" and found this sticky thread under Doom9's Forum > General > Audio encoding, where the fifth section down, "Basic Parameters", specifically discusses the proper "official" way to represent the number of full range channels, LFE channel, and speaker layout as well - a representation basically backed up the referenced specification document there. So I've adapted the terminology described there in this new version, as mentioned in my update notes (along with a small screenshot).

Anyway, I'll be back - I need a brief rest now ;)