VC1Source Plugin

[Warren]

Hi guys, this is my first foray into AviSynth plugin writing and I've had some success after a little bit of a learning curve.

There is still no easy way to decode a VC-1 Elementary Stream into raw pictures without going through a long convoluted directshow chain of filters in AviSynth so I decided to make a VC1Source plugin, sort of like the MPEG2Source plugin from dgIndex.

Using some reference VC-1 decoder code that also parses the VC-1 Elementary Stream I now have a plugin that will decode pictures, albeit slowly (~4.5fps on a Core2Duo E6600). The problem i am having currently is to do with framerate, the VC-1 ES reports the FPS as 29.97 even though I know for a fact that the video is 24 FPS since it comes from a HD DVD. Now it does set a pulldown flag in one of the data structures, but I'm not sure exactly how that is supposed to work.

Does anyone have any ideas or comments?

[IanB]

Perhaps putting up some source code for comment and some reference URL's might inspire some discussion.

As a guess pulldown possible means do telecine (3:2 pulldown).

[Warren]

http://pastebin.ca/331855 is a log of some of the flags for each frame that is decoded using my plugin. Notice that every 13 frames two consecutive frames have repeat field flag set to 1 - this is where judder occurs if you view the resulting YUV I420 file. Note that the clip use is progressive, not interlaced.

[Mug Funky]

wow! i thought it'd be much longer before something like this comes up.

i'll be following this thread closely

btw, an AVCsource would be cool too... dshow isn't all that reliable. all in good time though - i'm just teasing.

[IanB]

So you are proposing the field pulldown sequence would be this

00 11 12 23 33 44 55 56 67 77 88 99 9A AB BB CC Cd de ee ff ...

i.e. 26 frames results in 66 fields. Which sort of implies the input progressive rate was 23.61275 fps.

How does Directshow handle the stream, particularly the time indexes it assigns to each frame.

[crypto]

...the VC-1 ES reports the FPS as 29.97 even though I know for a fact that the video is 24 FPS since it comes from a HD DVD

I also noticed this. The HD-DVD streams are also flagged as pulldown and interlaced. Very strange. This might also explain why Toshiba players are struggling with 24p playback as it would require a reverse pulldown.

@IanB
No, 26 frames exeed the HD-DVD spec.

[zambelli]

Originally Posted by crypto

I also noticed this. The HD-DVD streams are also flagged as pulldown and interlaced. Very strange. This might also explain why Toshiba players are struggling with 24p playback as it would require a reverse pulldown.

AFAIK, all VC-1 encoded movie titles on HD-DVD are 23.976p with 3:2 pulldown, creating 29.97i compliant decoded streams.

[IanB]

Originally Posted by crypto

@IanB
No, 26 frames exceed the HD-DVD spec.

I doubt it, I see no limit for wierd pulldown cycle length in any HD-DVD specs.

If I had said a "repeating cycle of 13 film frames results in 33 interlaced fields", would you have been happier.

This is of course at odds with what I (and Zambelli and you) were expecting to see, i.e. a normal 3:2 pulldown with 4 film frames resulting in 10 interlaced fields.

Unfortunatly Warren is not sharing his references or code so we do the best clairvoyant interpolation of the data possible.

Maybe the stream has wierd flags, maybe Warren's code is bugged, maybe I am reading the log incorrectly. Whatever it is the logfile provides has a 13 frame cycle anomoly that I would like explained.

[crypto]

Originally Posted by zambelli

AFAIK, all VC-1 encoded movie titles on HD-DVD are 23.976p with 3:2 pulldown, creating 29.97i compliant decoded streams.

Thanks for confirming. Now to the difficult part. What parameters would you set for WME9? 29.97i ? Where does the pulldownflag come from? I did not find any setting or does interlaced imply pulldown?

I setup a HD-DVD profile, that I posted here I believe now, that I have to add interlaced.

[Warren]

Originally Posted by IanB

I doubt it, I see no limit for wierd pulldown cycle length in any HD-DVD specs.

If I had said a "repeating cycle of 13 film frames results in 33 interlaced fields", would you have been happier.

This is of course at odds with what I (and Zambelli and you) were expecting to see, i.e. a normal 3:2 pulldown with 4 film frames resulting in 10 interlaced fields.

Unfortunately Warren is not sharing his references or code so we do the best clairvoyant interpolation of the data possible.

Maybe the stream has weird flags, maybe Warren's code is bugged, maybe I am reading the log incorrectly. Whatever it is the logfile provides has a 13 frame cycle anomaly that I would like explained.

I'm using SMPTE's reference decoder so my decoder is not bugged (unless theirs is, which I somewhat doubt seeing as they made the spec). I'm currently using r6 of their decoder but am going to update my code to r7 tonight. You can grab that from http://www.smpte-vc1.org/AHG/Test/Decoder/FDS1/

[Warren]

Originally Posted by crypto

Thanks for confirming. Now to the difficult part. What parameters would you set for WME9? 29.97i ? Where does the pulldownflag come from? I did not find any setting or does interlaced imply pulldown?

I setup a HD-DVD profile, that I posted here I believe now, that I have to add interlaced.

Please stay on topic.

[Warren]

Originally Posted by Warren

I'm using SMPTE's reference decoder so my decoder is not bugged (unless theirs is, which I somewhat doubt seeing as they made the spec). I'm currently using r6 of their decoder but am going to update my code to r7 tonight. You can grab that from http://www.smpte-vc1.org/AHG/Test/Decoder/FDS1/

It seems there is very little change from r6 to r7 and it is nothing that would affect the way the frames are coming out. I'm going to do a dump with their reference decoder.exe and compare the YUV output to that of my plugin to eliminate my plugin glue code from the list of suspects.

Update:

I just checked the YUV output of their reference decoder.exe and it produces the same judder pattern. I quite frankly don't know where to go from here....

Advice?

[Warren]

Well I found the problem and fixed it. It seems that the SMPTE Reference VC-1 decoder (up to r7 at least) has a bug in it where it will ignore the first frame in each Sequence (~13 frames) and not only the first frame of the video like it's supposed to.

My beta tester is giving the plugin a run through and if it passes I'll upload a binary here for people to enjoy.

[IanB]

Well at least thats explained

[Warren]

Sorry for the silence, I've been trying to figure out how to figure out how many frames are in the stream. I may have to end up scanning the entire stream - not sure.

[IanB]

I am sure Don will have much sympathy for you over this issue, hence the DGIndex component.

Iff! there are reliable timestamp packets in the file you might only need to count the frames before the first timestamp and after the last timestamp in the file.

Reverse scanning the packets from the end of file to find that last timestamp will test your metal.

Likewise to do frame accurate seeking without an index on a VBR stream. Guess a file position based on average bitrate, then binary search for a slightly early timecode then count forwards for the right frame.

For bonus points you could remember the index points as the stream plays so any backwards seeking would be fast and accurate, any forwards seeking could use the above algorithm or do a hard scan forwards if there are no timecodes.

[Warren]

I wonder if this information is contained in the .map files or if the HD DVD players implement an algorithm like you suggest.

[Neo Fagin]

For what it's worth I'm one of the people 'beta testing' this plugin as it were.

I've done a few tests, and finally tried reencoding an entire feature with the plugin. The source was the movie "The Thing" demuxed from EVO with evodemux 0.541 to an elementary vc-1 stream. This was the result:

F:\Documents and Settings\encode>f:\x264\x264.exe --pass 2 --bitrate 5305 --stats "g:\thing.stats" --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid \
--b-rdo --bime --weightb --direct auto --deblock -3:-3 --subme 7 --analyse all --8x8dct --me umh --threads auto --thread-input --cqmfile "F:\x264\eqm_avc_hr.cfg" \
--progress --no-psnr --no-ssim --output "g:\thing.mkv" "g:\thing.avs" --trellis 1

avis [info]: 1280x544 @ 23.98 fps (156380 frames)
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2
WARNING: Hypothetical Reference Decoder Underflow i=0 F=789639
WARNING: HRD: Buffer underflow, bit count = 1398928
WARNING: Hypothetical Reference Decoder Underflow i=0 F=700690
WARNING: HRD: Buffer underflow, bit count = 884952
WARNING: Hypothetical Reference Decoder Underflow i=0 F=700689
WARNING: HRD: Buffer underflow, bit count = 898496
WARNING: Hypothetical Reference Decoder Underflow i=0 F=700689
WARNING: HRD: Buffer underflow, bit count = 1372192
WARNING: Hypothetical Reference Decoder Underflow i=0 F=799012
WARNING: HRD: Buffer underflow, bit count = 1046576
WARNING: Hypothetical Reference Decoder Underflow i=0 F=773289
WARNING: HRD: Buffer underflow, bit count = 774760
WARNING: Hypothetical Reference Decoder Underflow i=0 F=747026
WARNING: HRD: Buffer underflow, bit count = 759472
WARNING: Hypothetical Reference Decoder Underflow i=0 F=700689

[...around 200 more of the same error...]

WARNING: HRD: Buffer underflow, bit count = 1192360
WARNING: Hypothetical Reference Decoder Underflow i=0 F=1003460
WARNING: HRD: Buffer underflow, bit count = 1190344
WARNING: Hypothetical Reference Decoder Underflow i=0 F=14745600
WARNING: HRD: Buffer underflow, bit count = 89821568
x264 [info]: slice I:1036  Avg QP:11.73  size:105568a 0:00:00
x264 [info]: slice P:48614 Avg QP:11.98  size: 50272
x264 [info]: slice B:106730 Avg QP:12.78  size: 16601
x264 [info]: mb I  I16..4: 48.7% 45.8%  5.5%
x264 [info]: mb P  I16..4:  5.5% 11.2%  0.4%  P16..4: 27.6% 19.1% 20.3%  0.2%  0.2%    skip:15.5%
x264 [info]: mb B  I16..4:  6.7%  1.5%  0.0%  B16..8: 14.5%  2.0%  9.9%  direct: 1.4%  skip:63.9%
x264 [info]: 8x8 transform  intra:41.8%  inter:59.9%
x264 [info]: direct mvs  spatial:95.5%  temporal:4.5%
x264 [info]: ref P  41.6% 23.9% 11.3% 15.7%  7.5%
x264 [info]: ref B  29.0% 24.8% 14.7% 19.0% 12.5%
x264 [info]: kb/s:5304.9

encoded 156380 frames, 4.81 fps, 5305.03 kb/s

The movie encoded fine up to 17:36, after which the frame at 17:36 was duplicated ~115,000 times for the remainder.

The first pass ratefactor was 18.28 so I do not believe this problem occured during it.

Don't criticize/suggest stuff about my x264 commandline please unless you think it's the cause of the problem. I've used the same commandline for about two dozen reencodes from MPEG-2 and h264 sources and yes I know some options are hardly worth enabling etc.

[Isochroma]

@Neo Fagin: remember that x264 and most other encoders seek forward/backward in the stream, probably unpredictably. Since the decoder probably isn't ready for such seeking, you'll have to decompress the entire stream to an intra- only codec (Huffyuv/Zlib etc.)

Using that method, we can test the decoder's sequential decode function. When that function is stabilized, then we can work on the seeking, which once completed will allow non intra encoders to work.

[Warren]

Hrm, I guess I should try and implement seeking in the code then - this should be fun.