How to demux VC1 elementary stream from WMV file?

[vsv]

StreamInfo VC1 ES (HD-DVD compliant)
taken by Elecard Stream Analyzer

===<<< Stream >>>========

0x00000018 Entry Point Layer
broken_link = 0
closed_entry = 1
panscan_flag = 0
refdist_flag = 0
loopfilter = 1
fastuvmc = 1
extended_mv = 0
dquant = 0
vstransform = 0
overlap = 0
quantizer = 2
hrd_fullness():
hrd_fullness[0] = 0
coded_size_flag = 0
range_mapy_flag = 0
range_mapuv_flag = 0

0x00000020 Picture Layer - I Frame
ptype = 6
rndctrl = 0
pqindex = 10
transacfrm = 0
transacfrm2 = 0
transdctab = 1

0x0000AF03 Picture Layer - P Frame
ptype = 0
rndctrl = 0
pqindex = 11
mvmode = 0001b
mvtab = 1
cbptab = 2
transacfrm = 2
transdctab = 0

0x00013E98 Picture Layer - B Frame
ptype = 2
rndctrl = 0
bfraction = 1
pqindex = 12
mvmode = 0
mvtab = 2
cbptab = 1
transacfrm = 0
transdctab = 0

0x00014FB9 Picture Layer - B Frame
ptype = 2
rndctrl = 0
bfraction = 2
pqindex = 12
mvmode = 0
mvtab = 1
cbptab = 1
transacfrm = 0
transdctab = 0

Small sample (1,91 MB) VC1 ES 1440x1080i29,97 encoded from HDV source:
http://media04.videomaker.com/content/video/article/13066/hdfx7_capture_05.m2t
who interested can download there:
http://files-upload.com/files/43174/VC1ES.rar.html
passw: doom9

[bond]

Small sample (1,91 MB) VC1 ES 1440x1080i29,97 encoded from HDV source:
http://media04.videomaker.com/content/video/article/13066/hdfx7_capture_05.m2t
who interested can download there:
http://files-upload.com/files/43174/VC1ES.rar.html
passw: doom9which tool produced that sample?

[Guest]

@vsv

What are you trying to tell us?

[crypto]

@vsv
Very cool, your file passes all tests. How did you encode it? Do you have the corresponding wmv file?

[vsv]

@crypto, this encoded by cinevision to my request from HDV mpeg2 TS.
You can encode to wmv AP, than use mplayer demux
to VC1 and compare to uploaded sample.

[stegre]

I've just spent a few minutes "reverse engineering" the VC-1 elementary stream. It uses the typical MPEG {0x00 0x00 0x01 0xnn } 4 byte sequence as a start codes to identify "payload chunks" or "packets" or "packs" whatever you want to call them. I'll just use the word "packets" for now.

The sample contains 3 packet types, "0x0d", "0x0e", and "0x0f" (the actual values corresponding to where I've written "0xnn" above).

The sample contains 12 groups of 20 packets. Each group of 20 contains one 0x0f "header" packet, immediately followed by a different type 0x0e "header" packet - that pair is then always followed by eighteen 0x0d "frame" packets. Then the pattern repeats (keyframe interval perhaps?) The file is actually halfway thru a 13th such group when it comes to an end.

I'm strictly guessing when I use the words "header" and "frame" above, but that's what they resemble if I were to guess. The first few bytes of the 0x0e packets have a linearly increasing "big endian" number which looks very much like a timecode of some sort, though I didn't look any further into it than that.

More to the point, the idea that the "0x0d" packets are "frames" works out because the original sample had 225 frames of MPEG-2 after it's demuxed from the TS. The VC-1 file has 12 x 18 = 216 plus another six in the partial 13th group makes 222, which is close enough to 225 for me, taking into account that it was totally re-encoded & my counts may be off by 1 or 2 anyway.

So in summary, this is at least consistent with what I thought & previously posted: an elementary stream has the basic headers needed for "info" and parsing frame boundaries, even if you started playing the stream at an arbitrary point.

The start codes definitely not present in raw VC1 demuxed from either WMV or AVI, I've scanned 18MB samples and found there was not a single instance of the three byte sequence 0x00, 0x00, 0x01 - not even an occasional "accidental" one - which makes sense because MPEG specs tend to forbid the sync sequence from coming up "accidentally", thus simplifying syncing to the stream at any point.

So I'd say this pretty clearly establishes that an "elementary" VC-1 has more structure than "raw" VC-1 as obtained by demuxing a containerized miovie, so you're not going to get one that way. And, as I think I mentioned before, it may be close to, if not totally, impossible to in any way play or "reconstruct" a raw demuxed stream into anything useful ever again. The ticket, as neuron2 suggested, would be a single small app that added in the headers as it demuxed, performing WMV / AVI to elementary VC1 in a single step.

[zambelli]

So I'd say this pretty clearly establishes that an "elementary" VC-1 has more structure than "raw" VC-1 as obtained by demuxing a containerized miovie, so you're not going to get one that way. And, as I think I mentioned before, it may be close to, if not totally, impossible to in any way play or "reconstruct" a raw demuxed stream into anything useful ever again. The ticket, as neuron2 suggested, would be a single small app that added in the headers as it demuxed, performing WMV / AVI to elementary VC1 in a single step.
Sounds like a great opportunity for a collaboration with Crypto on this one: http://forum.doom9.org/showthread.php?p=926161#post926161.

[Trahald]

im working on something in my spare time to add the required headers to a h264 stream that lacks them to make them hddvd ready. btw mpeg2 / h264 use 00 00 00 01 XX . its a bit easier with h264 than vc-1 since as you elude, it already has the structure, just that many streams are missing some required headers and/or headers out of sequence (there is some info in one header that is needed by another header so later header should appear after the other). also doesnt hurt that there is open source avc stream analyzer software available.vc-1 info may be harder to get.

[vsv]

Trahald try demo of Semaphore:
Semaphore gives you the flexibility to analyze digital media content
in any advanced encoding format – Windows Media (WMV), VC-1 or AVC (H.264/MPEG-4).
http://www.inlethd.com/products/semaphore.html

[bond]

im working on something in my spare time to add the required headers to a h264 stream that lacks them to make them hddvd ready. - SEI messages
- HRD info or
- EOS ?

[Trahald]

for now just concentrating on sei(buffering and timing) and hrd. if that is insufficient i will explore eos. im not really close to done since, even with the help of mpeg4ips code, although i have access to alot of the header info. it isnt complete so i will have to google up the rest.

[bond]

sounds cool

i mean, if you are starting to code this, maybe it would be possible to add it right away to x264 instead of patching already encoded streams?

[Trahald]

well... coding it seperatly is serving as a learning experience for me (learning the details of h264 stream settings etc). if i went straight into x264 i'd break stuff in what is solid code. doing it this way i can test the streams out. learns the ins and outs of the settings. then perhaps i would consider that as i can go right in and add whats needed without uncertainty.

[bond]

well... coding it seperatly is serving as a learning experience for me (learning the details of h264 stream settings etc). if i went straight into x264 i'd break stuff in what is solid code. doing it this way i can test the streams out. learns the ins and outs of the settings. then perhaps i would consider that as i can go right in and add whats needed without uncertainty.checkout this:
http://forum.doom9.org/showthread.php?p=928347#post928347

[Trahald]

very nice... he had some of the concerns i had (ie him guessing at the calculation for initial_cpb_removal_delay ).. definately good that someone did that. i'll still work on mine.. if just for the learning experience.

[stegre]

OK, back to VC1, I think I've sorted out a lot of the whole VC-1 ES business, and can now (in the "lab") losslessly remux MS created, VC1 AP (advanced profile) from WMV to AVI and I can now demux it into a 100% conformant AP "VC-1 elementary bitstream". Which was the original question, here. I may start a separate new thread soon because this subject has become fragmented into several threads at this point - and there actually is a lot to know & discuss, and a lot of terminology that needs clarification.

I'm going to try to whip together a command line tool (and I'll make it open source, too) that will create these ES streams from WMV or AVI; it'll probably just be something rather crude and simple and may require the user to execute several steps to complete.

Anyway, the point for now is this, for anyone who's interested these are pretty nice samples for the bitrate.

1) An AP VC-1 WMV test file (http://ftyps.com/unrelated/vc1_in_wmv.wmv) (which doesn't happen to have the sound, btw).

2) A visually identical (i.e. remuxed from the WMV above) AVI equivalent (http://ftyps.com/unrelated/vc1_in_avi.avi). This sample happens to have the original muxed back in (as VBR MP3). I actually posted this sample last week. It should play on any DirectShow player if you have AP VC-1 codec(s) installed.

3) This file is new, and is the point of this post: I believe it's a fully compliant VC-1 "elementary bit stream" (http://ftyps.com/unrelated/vc1_es.es). It's also muxed from the same encoding as the above two. No sound of course. It's not of much use to regular Windows (or Linux?) users - I personally don't know of any media player that will play this file directly. But for the person who started this thread, and for the others who were asking: please let me know if it seems OK. This is the first simple test file I've made - it doesn't have the periodic "resync" info during the two minutes - something that's normally desirable insofar as it would allow someone to quickly "tap into" it, even if it were streaming & the person arrived "mid-stream". But that shortcoming I could easily corrected (I think), and in any event there's no requirement in the spec concerning how frequently such packets must be inserted.

[Guest]

3) This file is new, and is the point of this post: I believe it's a fully compliant VC-1 "elementary bit stream" (http://ftyps.com/unrelated/vc1_es.es). How did you make it?

[crypto]

Very cool. But there is still some info missing. The parser stops at the frame rate info:
http://img344.imageshack.us/img344/9361/vc1es18pe.th.jpg (http://img344.imageshack.us/my.php?image=vc1es18pe.jpg)

There are also no repetitions of the sequence layer and the entry-point header, although this might not by a problem when importing into authoring apps.

[SeeMoreDigital]

DVDPortal "VC-1 Stream Inspector".... Where can one obtain this little beauty?


Cheers

[crypto]

Hi SeeMoreDigital,

its a tool I use internally for conformance tests. There are many coverage tests still missing and I consider it work in progress. I reckon its too early to share it, but I will make it available at least for D9 members working on VC-1.

[SeeMoreDigital]

Hi SeeMoreDigital,

its a tool I use internally for conformance tests. There are many coverage tests still missing and I consider it work in progress. I reckon its too early to share it, but I will make it available at least for D9 members working on VC-1.Nice one ;)

[vsv]

3) This file is new, and is the point of this post: I believe it's a fully compliant VC-1 "elementary bit stream".
Error report from Scenarist SCA:
Error D:\vc1_es.es
Error : Frame rate reserved is wrong.
Error : TV System should be NTSC, PAL, HD60 or HD50
Error : Stream contains a GOP with a number of fields = 4546; it should not exceed 0.
Warning : Number of SequenceEndCode is 0.

Info 0 file(s) accepted, 1 file(s) rejected

[vsv]

Elecard StreamEye Analyzer:
FileSize: 24 803 972
StreamType: VC1
OverHead:
CountPackets: 2 275
---------------------------
0x00000000 Sequence Layer
profile = 3 (Advanced)
level = 2
chroma_format = 1 (4:2:0)
frmrtq_postproc = 6
bitrtq_postproc = 31
postprocflag = 0
max_coded_width = 325 (652)
max_coded_height = 239 (480)
pulldown = 1
interlace = 0
tfcntrflag = 0
finterpflag = 0
display_ext = 0
hrd_param_flag = 0

0x0000000E Entry Point Layer
broken_link = 0
closed_entry = 0
panscan_flag = 0
refdist_flag = 1
loopfilter = 0
fastuvmc = 0
extended_mv = 0
dquant = 0
vstransform = 1
overlap = 0
quantizer = 0
coded_size_flag = 1
coded_width = 325
coded_height = 239
range_mapy_flag = 0
range_mapuv_flag = 0

0x0000001C Picture Layer - I Frame
ptype = 6
rptfrm = 0
rndctrl = 0
pqindex = 4
halfqp = 0
transacfrm = 0
transacfrm2 = 0
transdctab = 0

0x0000B3A0 Picture Layer - P Frame
ptype = 0
rptfrm = 0
rndctrl = 1
pqindex = 4
halfqp = 0
mvmode = 01b
mvtab = 0
cbptab = 3
transacfrm = 0
transdctab = 1

[Isochroma]

First off, a big thank-you to all involved, and especially stegre for contributing the three sample files. I'll have more to say about my tests with them below.

Next on the agenda is a bit of bookkeeping. It has been observed that there are a number of threads dealing with what is basically the same topic cluster - muxing/demuxing VC-1:

How to demux VC1 elementary stream from WMV file? (http://forum.doom9.org/showthread.php?t=119102) (This thread)

Muxing and demuxing vc-1 (http://forum.doom9.org/showthread.php?t=119246)

Mux VC1 on WMV (http://forum.doom9.org/showthread.php?t=119785)

Muxing VC-1 in ASF (http://forum.doom9.org/showthread.php?t=120407)

To avoid confusion and isolated development islands, it is recommended to merge these threads. Such request is now made of the local moderator.

Now back to the files stegre posted.

1. The WMV file can be played back in MPC using the MS VC-1 decoder, but not the Sonic CineMaster decoder.

2. The AVI file also plays fine in MPC using the MS VC-1 decoder, but if using Sonic's, the audio plays with no video.

3. Using GraphEdit, the VC-1 ES file can be remuxed into MKV using Haali's Matroska Muxer:

http://www.isochroma.com/Testfiles/Misc/doom9/vc1_es_hmux.png

Note the "VC1" label on the Elecard MPEG Demultiplexer's output pin. It can parse MPEG-2, AVC and VC-1 ES this way, but it must be the latest build.

4. The MKV file produced, while not having any timestamps or framerate at all, can be remuxed into another MKV using MKVToolnix with a timecode file specifying 24.000 fps:

# timecode format v1
assume 24.000

5. The resulting MKV file is mostly valid and shows the specified framerate; however, there's an odd character in the field "Extra Codec Info". It plays fine in MPC (seekable) using Sonic's decoder, but using the MS decoder, only a blank frame is shown.

[stegre]

OK, I've been working on it all day; it'll be a DirectShow "filewriter" type filter so converting a VC-1 WMV will amount to as setting up this in GraphEdit and hitting "play":

http://www.ftyps.com/unrelated/vc1_filter.png

The filter on the left is just the standard "Windows Media Source Filter" and the one on the right is what I'm working on. It's just about working already, so I should have it in a day or so.

... There are also no repetitions of the sequence layer and the entry-point header, although this might not by a problem when importing into authoring apps.Yes, I know, that's what I was informally referring to when I said "...it doesn't have the periodic "resync" info during the two minutes - something that's normally desirable..." - but in any event that's already fixed. And I did also find a bug which I hope might be the cause of the "parser stopping at the frame rate info" problem.

Anyway, more later, lemme get back to work...

[crypto]

Wow. That's what we are all waiting for (and are working on ;)). Keep it up. BTW. Can you give us details on how the AVI was done?

[stegre]

OK, it's kind of a "quick hack", but it does seem to be working now. It's a bit late though - tom'w evening I'll put together a little readme.txt and zip file package with the filter and also post an updated sample file made from it. I'll also post some more technical info on what I'm doing and discuss some of the issues above. Might even be a good time to start a new thread - maybe I'll do that - to address the issue that someone mentioned about how this subject is now splattered around four different threads. Of course, then there will be five threads, haha, but at least all further discussion on this specific subject can be consolidated...

[stegre]

WMF VC1 Advanced => VC-1 ES Filewriter filter for use with GraphEdit (http://www.ftyps.com/unrelated/vc1_es.v0.01.zip) (usage & other info included). Here's an updated sample file made (24MB) (http://www.ftyps.com/unrelated/vc1_es.v0.01.es) I with it. It now puts Entry Point and Sequence headers before each keyframe.

[zambelli]

http://www.ftyps.com/unrelated/vc1_es.v0.01.es
Minor note: the file extension used for VC-1 ES thus far has been .vc1.

[crypto]

Very cool. Thanks for the VC-1 ES Filewriter filter. I just dumped an ES which is syntactically correct. This is a big step forward to home made VC-1 HD-DVDs.

[vsv]

Anyone can provide *.prx HD-DVD compliant profile (GOP at 0.6sec and another tricks) for encoding, which possible demux with Stegre's filter to VC1 es stream? Thanks in advance.

[crypto]

Anyone can provide *.prx HD-DVD compliant profile (GOP at 0.6sec and another tricks) for encoding, which possible demux with Stegre's filter to VC1 es stream? Thanks in advance.

Voilá, I used this one:


<profile version="589824"
storageformat="1"
name="[WVC1] 1080p30cbr HDDVD"
description="[WVC1] Advanced HD-DVD, 1920 x 1080 Pixel, 29.97 FPS">
<streamconfig majortype="{73646976-0000-0010-8000-00AA00389B71}"
streamnumber="1"
streamname="Video Stream"
inputname="Video409"
bitrate="14000000"
bufferwindow="1000"
reliabletransport="0"
decodercomplexity="AU"
rfc1766langid="en-us"
>
<videomediaprops maxkeyframespacing="5000000"
quality="100"/>
<wmmediatype subtype="{31435657-0000-0010-8000-00AA00389B71}"
bfixedsizesamples="0"
btemporalcompression="1"
lsamplesize="0">
<videoinfoheader dwbitrate="14000000"
dwbiterrorrate="0"
avgtimeperframe="333667">
<rcsource left="0"
top="0"
right="1920"
bottom="1080"/>
<rctarget left="0"
top="0"
right="1920"
bottom="1080"/>
<bitmapinfoheader biwidth="1920"
biheight="1080"
biplanes="1"
bibitcount="24"
bicompression="WVC1"
bisizeimage="0"
bixpelspermeter="0"
biypelspermeter="0"
biclrused="0"
biclrimportant="0"/>
</videoinfoheader>
</wmmediatype>
</streamconfig>
</profile>

[vsv]

crypto, thank you for help!
But i can not load this profile into canopus procoder...

[crypto]

You need to save it as unicode little endian UCS-2.

1. Paste it into notepad
2. Save As..
3. Select Coding Unicode
4. Enter a filename xxx.prx
5. Click ok

If you still have trouble, I will upload it somewhere. Too bad D9 can't handle attachments.

P.S. I chose a cbr of 14 MB/s. You can push this up to 23 MB/s. GOPs are at 0.5 secs. With the allowed maximum of 0.6 secs, I got warnings in authoring.

P.P.S The profile editor does not allow to enter GOPs under 1 sec. When you change anything, make sure to patch back the 0.5 secs in the XML file. (maxkeyframespacing="5000000")

[McCrash]

OK, I've been working on it all day; it'll be a DirectShow "filewriter" type filter so converting a VC-1 WMV will amount to as setting up this in GraphEdit and hitting "play":

http://www.ftyps.com/unrelated/vc1_filter.png


I have a problem connecting the two, does it mean the wmv i am using is not advanced profile ?

[Jay Bee]

WMF VC1 Advanced => VC-1 ES Filewriter filter for use with GraphEdit (http://www.ftyps.com/unrelated/vc1_es.v0.01.zip) (usage & other info included). Here's an updated sample file made (24MB) (http://www.ftyps.com/unrelated/vc1_es.v0.01.es) I with it. It now puts Entry Point and Sequence headers before each keyframe.

So can these demuxed VC-1 files be used to author a HD-DVD with Sonic Scenarist?

[stegre]

Tiny! (13KB, 6KB zip) cmnd line util (http://ftyps.com/unrelated/VC1_Info.001.zip) I wrote will parse & display Sequence and Entry Point layer info from either an elementary .VC1 or directly from a WVC1 WMV file. Very untested so far. I'll post source too, soon as I fix it up a bit.

@McCRash - run it on your .wmv, see what it says.

[crypto]

@Jay Bee
Yes, the ES streams can be used for authoring. Scenarist however does not like the short sequence headers. I rewrote those with the frame rate info and then the ES was accepted.

@stegre,
thanks for the tool. Can you add the frame rate info in the sequence header? I believe vc1_info is not parsing it correctly. I have discrepancies between my tool and yours:

yours:

--------- Sequence Level Header -----------

PROFILE: 3
LEVEL: 3
COLORDIFF_FORMAT: 1 (=4:2:0)
FRMRTQ_POSTPROC: 7
BITRTQ_POSTPROC: 31
POSTPROCFLAG: 0
MAX_CODED_WIDTH: 959 (=1920)
MAX_CODED_HEIGHT: 539 (=1080)
PULLDOWN: 1
INTERLACE: 1
TFCNTRFLAG: 0
FINTERPFLAG: 0
RESERVED: 1
PSF: 0
DISPLAY_EXT: 1
DISP_HORIZ_SIZE: 1919 (=1920)
DISP_VERT_SIZE: 1079 (=1080)
ASPECT_RATIO_FLAG: 1
ASPECT_RATIO: 1 (PAR=1:1(square))
ASPECT_HORIZ_SIZE: 128 (PAR= 129/203 = 0.635)
ASPECT_VERT_SIZE: 202 (PAR= 129/203 = 0.635)
FRAMERATE_FLAG: 0
FRAMERATEIND: --
FRAMERATENR: --
FRAMERATEDR: --
FRAMERATEEXP: --
COLOR_FORMAT_FLAG: 0
COLOR_PRIM: --
TRANSFER_CHAR: --
MATRIX_COEF: --
HRD_PARAM_FLAG: 0
HRD_NUM_LEAKY_BUCKETS: --
BIT_RATE_EXPONENT: --
BUFFER_SIZE_EXPONENT: --

--------- Entry Point Level Header -----------

BROKEN_LINK: 0
CLOSED_ENTRY: 1
PANSCAN_FLAG: 0
REFDIST_FLAG: 1
LOOPFILTER: 1
FASTUVMC: 0
EXTENDED_MV: 1
DQUANT: 1
VSTRANSFORM: 1
OVERLAP: 0
QUANTIZER: 3
CODED_SIZE_FLAG: 1
CODED_WIDTH: 4083 (=8168)
CODED_HEIGHT: 3058 (=6118)
EXTENDED_DMV: --
RANGE_MAPY_FLAG: 0
RANGE_MAPY: --
RANGE_MAPUV_FLAG: 0
RANGE_MAPUV: --


mine:

[00000000] Sequence Layer

profile: 3 (advanced)
level: 3 (AP@L3)
chroma_format: 1 (4:2:0)
frmrtq_postproc: 7
bitrtq_postproc: 31
postprocflag: 0
max_coded_width: 959
max_coded_height: 539
pulldown: 1
interlace: 1
tfcntrflag: 0
finterpflag: 0
display_ext: 1
disp_horiz_size: 1919
disp_vert_size: 1079
aspect_ratio_flag: 1
aspect_ratio: 1
framerate_flag: 1
framerateind: 0
frameratenr: 3
frameratedr: 2
frame rate: 29,97
m_color_format_flag: 1
color_prim: 1
transfer_char: 1
matrix_coef: 1
hrd_param_flag: 1
hrd_num_leaky_buckets: 1
bit_rate_exponent: 3
buffer_size_exponent: 4
hrd_rate[0] = 44920 (22999552)
hrd_buffer[0] = 57599 (14745600)

[00000019] Entry-Point Header

broken_link: 0
closed_entry: 1
panscan_flag: 0
refdist_flag: 1
loopfilter: 1
fastuvmc: 0
extended_mv: 1
dquant: 1
vstransform: 1
overlap: 0
quantizer: 3
hrd_fullness[0]: 255
coded_size_flag: 1
coded_width: 959
coded_height: 539
extended_dmv: 0
range_mapy_flag: 0
range_mapuv_flag: 0


Note: This is the dump of a VC-1 ES with the full headers. The frame rate info can not be read from the WMV file as it seems to use short headers. (framerate_flag = 0)

[stegre]

crypto (or anyone) see if this version (http://ftyps.com/unrelated/VC1_Info.002.zip) fixes the more obvious problems (crazy PAR values and CODED_WIDTH, etc.), and I'll discuss the other issues when I get home later tonight.

[crypto]

Yeah, works perfect on all samples I have.

[stegre]

@stegre,
thanks for the tool. Can you add the frame rate info in the sequence header? I believe vc1_info is not parsing it correctly. I have discrepancies between my tool and yours...
Yes, I found a few bugs, if you get a chance, please check v2 as mentioned in the above post....


Note: This is the dump of a VC-1 ES with the full headers. The frame rate info can not be read from the WMV file as it seems to use short headers. (framerate_flag = 0)
That's exactly correct. Which is to say there are two different couple of issues here:

One is miscellaneous bugs in v1 of my "info tool"; As I say, please try v2 above & see if it matches your tool, for any given file you run it against. If there are still a few more discrepancies, I may need to obtain a few .VC1 sample files to properly debug it, but I think I should be able to get it working correctly pretty easily. But...

Once the info tool is working - or if you use your own - you'll will note that any *.VC1 files created by my VC-1 writer filter WILL have "short headers", as you put it. Actually many fields are optional, but I know what you mean: leaving out the entire "display extension" field is a biggie, in particular because framerate is in there. Like other less important fields, leaving it out is "legal" per VC-1 specifications, but from the discussion above I see that Scenarist gets unhappy about when that particular one is left out. And who could blame it? It can't determine the framerate - a rather basic piece of info!

So the "info tool" aside, the real problem here is that my VC1 "writer filter" puts out "short headers". The thing is, that wasn't my choice, as it doesn't "generate" the VC-1 ES headers bit-by-bit - which would have been much more difficult - but it rather finds the "complete" and ready-to-go pair of headers (which, I found out, turn out to be appended as auxiliary data in the "ASF_Video_Media" object of a WVC1 WMV file). So that made my job a lot easier - my VC-1 ES writer filter doesn't currently do much work beyond finding those headers and then inserting them verbatim at appropriate intervals in the .VC1 output file.

But the headers that Microsoft's WVC1 encoder puts there are "short" ones, as you correctly noted. They do indeed lack the entire "display extension" - I assume because Microsoft figures all that that info is available elsewhere in the various other WMV header objects.

So the trick now would be for me to make a change to my VC1 ES file writer - and I'm looking into this. I should grab the short Microsoft VC-1 Sequence Layer and Entry Point headers verbatim, as I do now. Then I should extract, at minimum, the framerate info from elsewhere in the WMV file's headers, and modify the "as found" Sequence Layer Header it by "stuffing in" the extra bits so as to include that info. And use the modified, "more informative", "long version" Sequence Layer Header as the one I write to the resulting VC-1 ES files.

This may be a bit of work, but I'm looking at it now. But if I do that, then, from what I'm reading here, I guess my "writer filter" will be a "complete solution" in the sense that one could, in a single step, read in a WVC1 WMV file, run it thru the VC-1 ES writer filter & feed that resulting file straight to Scenarist.

I don't think I'll exactly have that all ready by tomorrow or anything, but it seems pretty feasible, and not a huge amount of work. I happen to also updating GSpot to handle WMV in general right now, so coincidentally I was already in the middle of coding WMV header parsing code anyway. So the timing is right - I should be able that part of it for both apps.

I'll keep you up to date on the progress of all this.

-----
Edit: @crypto -Just saw your latest "works perfect" post above just after I finished typing this whole post here, so ignore the few related sentences above. And that's good news - it's a good start to all this...

[crypto]

@stegre
That sounds like a plan. :)

P.S.: Have you decided yet, if your filter may be used in own programs? I would like to add ES output to my wmvmuxer (http://forum.doom9.org/showthread.php?t=120320&highlight=wmvmuxer).

[LOGiC]

@Jay Bee
Yes, the ES streams can be used for authoring. Scenarist however does not like the short sequence headers. I rewrote those with the frame rate info and then the ES was accepted.



Crypto, I had this problem a couple of times that scenarist is crashing. How do you rewrite the headers ?

[crypto]

Crypto, I had this problem a couple of times that scenarist is crashing. How do you rewrite the headers ?

I have done that with a little program just for testing purposes. Stegre is adding the necessary code to the ES file writer filter. Stay tuned for the next version.

[stegre]

I started working on it already & it's going well. And yes, I'll release it, with source, with a license like this (http://www.opensource.org/licenses/mit-license.php) - so you can do pretty much whatever you want with it - either the binary or source - even if it's something commercial.

[crypto]

Great News.

P.S.: I have PM'ed you some details from an app developers point of view.

[bond]

Tiny! (13KB, 6KB zip) cmnd line util (http://ftyps.com/unrelated/VC1_Info.001.zip) I wrote will parse & display Sequence and Entry Point layer info from either an elementary .VC1 or directly from a WVC1 WMV file. Very untested so far. I'll post source too, soon as I fix it up a bit.

@McCRash - run it on your .wmv, see what it says.stegre, this tool works fine on ap .vc1 streams, but not on main and simple profile .rcv files from the reference collection

are those different formats?

[stegre]

Yes, in fact quite different - surprisingly much so. So, yes, I just ignored them totally for now - maybe I'll go back & take a closer look at what would be involved in supporting them too, soon as I finish the demuxer thing.

[stegre]

WMV WVC1 to VC-1 "ES" writer filter, v0.2 (http://www.ftyps.com/unrelated/vc1_es.v0.2.zip)

Adds "Display Extension" to the VC-1 headers which include framerate as derived from the WMV.

Here's the top part of the readme describing the updates, the entire readme is in the zip. The "other" related project, the small command line info, is in the zip as well.

WMV VC-1, Advanced Profile" to VC-1 "ES" ("Encapsulated Stream") converter.

v0.2 04 Feb 2007

This version generates a "Display Extension" as part of the Sequence Header, and includes the frame rate info within that extension. From info previously posted, the theory is that this added piece of info "should" make the resulting VC-1 files usable with apps like "Scenarist" - thus permitting the creation of HD DVD's from VC-1 content created by Windows Media Encoder by using this filter to convert the file the WVC1 WMV to a *.VC1 elementary stream.

But that hasn't been tested yet, in fact, as I type this, the filter has only been tested on a single file altogether. I'm releasing it anyway in the interest of getting feedback.

Working on this did bring up another issue, which will be taken care of in v0.3. Now that I'm including the "Display Extension", I must also specify the images "displayed size", which will vary from its the "coded size" for files with non-square pixels. Plus there's another field in "Display Extension" where the pixel aspect ratio itself can be explicitly designated as well.

For this 0.2 version, I skipped that latter explicit PAR field, as it's optional even within "Display Extension", but I as far as "display size" I simply copied the values from "coded size". So this version will create VC-1 file which will not display the correct aspect ratio if non-square pixels are specified. And WMV encoder *does* handle non-square pixels, perhaps even as a default. As a test, I created a WVC1 WMV from a standard DVD and told it to store the compressed data internally using the same 720 x 480 pixel storage of the original MPEG-2, and then told it to use NTSC 10:11 PAR.

On WMP or any other player with proper support, this will be displayed at 640 x 480 (or technically 656 x 480 with 8 pixels cropped from either side), but if you convert such a file with this v0.2 filter, the PAR and/or "display size" information will not "make its way" to the resulting VC1 file, so I imagine the result will incorrectly be displayed as 720 x 480.

However, fixing this will be relatively simple, using the code from this v0.2 filter as a baseline. Which is to say that now that I'm already generating custom VC-1 headers based on info from misc. other WMV header objects, I'll simply also retrieve "display size" and/or PAR (I plan on doing both) in the same way I'm already doing with framerate. I'll then "rewrite" that part of the "short" VC-1 sequence header as well.

So just stay tuned for that.

- Steve G

Note: I've also added my "other" related VC-1 project to this zip file, the command line util VC1_Info. You can run this on *either* a WVC1 WMV or a VC-1 ES (advanced profile only). This complements this filter nicely, as you can see exactly which fields have been added to the VC1 that weren't in the original WMV "short headers" by simply running the util on the resulting VC-1 and comparing it to the original WMV.

[zambelli]

Working on this did bring up another issue, which will be taken care of in v0.3. Now that I'm including the "Display Extension", I must also specify the images "displayed size", which will vary from its the "coded size" for files with non-square pixels. Plus there's another field in "Display Extension" where the pixel aspect ratio itself can be explicitly designated as well.
I must interrupt you here before you continue down this path... Be careful not to confuse "display size/coded size" with Pixel Aspect Ratios - they're not the same concepts. Display size/codec size allows you to encode a video at a resolution different than the one it will be decoded at, such as storing a 720x480 SD video in a sequence designed to be decoded at 1280x720 HD resolution. WMV9 encoder supports this - take a look at the Encoding Height/Width (http://www.microsoft.com/windows/windowsmedia/howto/articles/codecadvancedsettings.aspx#Encoding_Height_and_Encoding_Width__BIOJ) options. When encoding with WME9/WMF, Display Size will always get set to whatever the output resolution is. Pixel Aspect Ratio, on the other hand, is set separately - so they're mutually exclusive concepts, even though you could theoretically use one to emulate the other.

WMV decoder will respect display/coded size data when it's present, but due to some architectural obstacles, it will never use DXVA for decoding when coded size != display size. No such restriction exists for pixel aspect ratio.. see the paragraph below.

As a test, I created a WVC1 WMV from a standard DVD and told it to store the compressed data internally using the same 720 x 480 pixel storage of the original MPEG-2, and then told it to use NTSC 10:11 PAR.

On WMP or any other player with proper support, this will be displayed at 640 x 480 (or technically 656 x 480 with 8 pixels cropped from either side), but if you convert such a file with this v0.2 filter, the PAR and/or "display size" information will not "make its way" to the resulting VC1 file, so I imagine the result will incorrectly be displayed as 720 x 480.

That's another gotcha. In WME9/WMFSDK the PAR is actually written into the ASF header and/or the ASF Data Unit Extensions. That's where the WMV decoder gets its PAR info from. I'm not so sure that the WMV decoder even knows to look into the VC-1 bitstream for PAR data, so unless that data isn't specified in the ASF container - the WMV decoder will just use square pixels.

The situation could get particularly tricky if you had both display/coded size set in the VC-1 bitstream AS WELL as PAR data in the ASF container - you might actually end up with a double applied aspect ratio adjustment. :)