Quicktime Import Filter - Test Version, Still Buggy - 2006-01-09

[Mug Funky]

with QT7 (the latest? i dl'd it a second ago) i'm getting "error -2048 - not a movie file"

but it's nice and fast

[edit]

the made file does load in QTinput without any problem.

btw, thanks heaps for this! it's a real nice new year's present, i tell you

[edit 2]

scratch that, the created file doesn't load with QTinput. i was accidentally loading the old one...

[edit 3]

scratch even that... what was happening was the input plugin was crashing on my sample at around frame 205, and that meant the movie was never finished writing... changing the input made QToutput make a usable file.

which is good news! this makes me so happy i'll crack a beer open for you.

[edit 4]

output is bit identical to input! party time!

[mode editing, omg]

writing the files seems to be tricky - i figured the safest way to make a file would be to load it in virtualdub and go "preview VBR" so it played the clip beginning to end. but when it finished, virtualdub went back to the first frame, which was then written to the end of my file... perhaps space needs to be allocated first and frames written to specific byte offsets in the output file as they're accessed? that could be tricky for large files (like an 80 minute movie...), but a simpler measure that would work for the "preview VBR" method is to not allow it to write more frames than in the source clip. it would just mean random access isn't possible (which i can live with of course).

[tateu]

Originally Posted by Mug Funky

writing the files seems to be tricky

Yeah, I know. I've been using it with VirtualDub pretty much the same way...My first version, several months ago, had all of the quicktime SDK functions called directly from within the avisynth methods. I quickly got rid of that code and created separate quicktime movie classes (CQTMovieDec and CQTMovieEnc) so that I could hopefully (and easily?) create a standalone quicktime program. The plan has always been to create a cli program that reads quicktime, avi, avs and can output to quicktime, avi, raw. I haven't started working on it yet, though.

As a quick fix, I like your idea of not allowing it to output more frames than the input clip has. I'll add that in and get it out sometime tomorrow.

The problem with your other method of preallocating space is that I am using quicktime functions to handle everything except encoding the data. Instead of encoded data, I feed the raw frames to AddMediaSample and I don't see a way of specifying offset locations with this method. The reason I do it this way is so that I don't have to figure out how to write the correct header and footer data.

[Mug Funky]

ah. that makes sense.

i really appreciate this by the way without the means to write my own plugins (or the time to learn a language), i have to harrass good people like you to make stuff it's really amazing how a free tool like avisynth can do stuff to such a high quality and speed that only the most expensive of pro equipment can rival it. in fact, your plugin is unique in that it's the only way of re-saving an uncompressed mov without any loss.

perhaps you could hook up with Avery Lee and maybe QT output could be hacked into Virtualdub? that would solve the frame access problem pretty definitively, and give people a much nicer alternative to QT pro. of course, if you're making your own program for similar things, then i'll let you do that too

[Revgen]

Originally Posted by Mug Funky

perhaps you could hook up with Avery Lee and maybe QT output could be hacked into Virtualdub? that would solve the frame access problem pretty definitively, and give people a much nicer alternative to QT pro. of course, if you're making your own program for similar things, then i'll let you do that too

Remember when Microsoft went after Avery because VirtualDub had support for .asf files?

I doubt he wants to repeat that experience with Apple.

It's still a good idea though.

[tateu]

Sure, no problem. I've been around here for years, never really talked all that much, but I've learned a great deal and found a lot of useful info and programs just by reading through the forums. Programming is just a hobby and I've always been grateful to all the people that work on programs like avisynth and release the source code for people like me to study and learn from. I'm glad that I can finally offer something useful back.

I've used Avisynth in our production workflow for years. I've always been impressed with how well and how much it can do.

Integrating directly into VirtualDub was another option I had thought of and I think it's better than a standalone cli version. I'm just not sure I am capable of doing it. I use fccHandler's modified VirtualDub-MPEG2 and I've glanced at the code he wrote for directly importing Mpeg-2 in the hopes that I could alter it to use my quicktime and omf code. I haven't done much with it so far but I do have my own custom version that checks the file extension of the file you are trying to open and if it is mov, mp4, dv or omf it writes an avisynth script in the VirtualDub-Mpeg2 home directory and opens that file instead. I also think I really need to get quicktime audio import and export working before fully integrating it with VirtualDub.

I don't think Apple would have much to say about quicktime integrated into VirtualDub. Wasn't the asf code reverse engineered? Apple releases an official quicktime SDK so that support can be added to any program. I am not a lawyer (and license agreements seem to cause a large thumping like pain just over my right eye) but everything I have read of the license agreements deals with distributing the quicktime installer and/or logos with an application.

New version...
http://www.tateu.net/software/dl.php?f=QTSource
http://www.tateu.net/software/dl.php?f=QTSource_src

This update contains the dither=1 (rounding mode), raw 2Vuy output, removes some unnecessary code and contains a frame count check for QTOutput to keep it from writing more frames than the input clip has.

[Mug Funky]

just tried export out in FCP with a decklink card and it accepted it without asking to re-render. that's very good news

note i've only tried PAL with this... it's possible that NTSC is another story - to be honest i've yet to successfully make a 2Vuy NTSC file that would actually play out to the NTSC deck here. that's certainly not a priority though - i only ever needed to do it once, and the file was small enough to burn to disc, so that wasn't a problem. i'm pretty sure it's user error, too - nobody here has actually needed to output NTSC through SDI, so no-one's got any experience doing it.

another thing i noticed is this plugin has exposed some alignment oddities in ffdshow - it was returning yuy2, and avisynth was reading it fine, but on QT export the picture was skewed majorly. i'm guessing it's a pitch oddity, but i fixed it by telling ffdshow to output the closest matching colourspace (even though in both cases it appeared to be returning yuy2).

[tateu]

Are you talking about something like this: http://www.tateu.net/software/skew.jpg? That's what I get when I use QTInput outputting YUY2 on a 420x240 quicktime. I tried several different codecs (uncompressed, Sorenson, mjpa) all with the same result. If I change QTInput to output RGB32, it works correctly. RGB24 is also skewed, but not as much and it's in greyscale.

I then tried a 418x240 quicktime which is skewed in all three colorspaces. I'm not sure what's going on, but it must be somewhere in my code.

[Mug Funky]

well, interestingly enough, the clip i was using also failed with HC (it just spat out a 0 kb file and said it was finished), even though i'd converted it to yv12 beforehand.

i have a sneaking suspicion it's a pitch or alignment issue that doesn't get corrected in avisynth (which is fine because it saves a little time i guess). but the old crop(align=true) chestnut didn't seem to work for it, while changing ffdshow's decoding options fixed it.

perhaps you could work around it by only giving quicktime pixels that you know are in the frame. as i understand it avisynth can store pixels outside the frame in order to either save time with simple operations like cropping, or to align data to 32 bit chunks. having not looked at the source, i don't really know what is going one here, rather i've just read a few discussions about pitch and alignment and possible problems with both in filter design.

i've also noticed some plugins don't work with other plugins as far as the skewing thing goes (for example, interleaving motion-compensated frames via mvtools and then using tsp's medianblurt on them results in a skewed mess).

[tateu]

Audio input for Quicktime 7 now seems to be working. Tests were run on several different formats: 48KHz Stereo (uncompressed, Mace6:1, ALaw, ULaw, QDesgin2 and AAC), 32KHz, 11KHz, mono, etc. I do not have any sample files with 5.1 surround sound to test. The movies were opened in Quicktime Pro and with QTInput (VirtualDub) and exported to 32bit wav files. The wavs were then invert mixed together to check for differences. The difference files never rose above -78db.

It always returns 32bit float. The sample rate and number of channels returned are the same as the source movie.

Audio is still disabled by default. You must set audio=true to use it.

http://www.tateu.net/software/dl.php?f=QTSource
http://www.tateu.net/software/dl.php?f=QTSource_src

[Mug Funky]

awesome!

do you think the differences are due to random dither for 16-bit stuff? i had wondered why when i exported audio from an SDI capture that audition reported true 32-bit. dither would explain this (if it was lossless from tape to mov to wav, audition should have reported 20 bits deep, even in a 32-bit file. digibeta carries 4 discrete 20 bit channels at 48k). i can live with dither though

i'll try this build out when i'm done exporting this movie to tape (thanks to your plugin ). oyu might be interested to know that it worked flawlessly even when exporting an 83 minute feature. no crashes or memory problems i know about (at least not enough to crash a machine with 1 gig real and 2 gigs swapfile... i let it go overnight so i don't know how it performed).

[tateu]

If you grabbed v0.0.5, please download again to get v0.0.5a.
http://www.tateu.net/software/dl.php?f=QTSource
http://www.tateu.net/software/dl.php?f=QTSource_src
I broke raw YUV input in v0.0.5 when I added raw RGB input.

It's probably the dither. One of the main things I did to get audio working in this version is switch over to 32bit float from 16bit signed int. I'd prefer to return whatever format the source is in, but 32bit is the only mode I can get working correctly. All of my test files were 16bit. And I guess now I'll have to add a raw audio mode for uncompressed sources so that it doesn't pass through quicktimes decoders.

The memory errors didn't seem to cause a whole lot of problems, but they always crashed VirtualDub if I used F2 to reload the avs script multiple times. 83 minutes...that's good to know. The longest I have tried is only about 10 minutes.

[Mug Funky]

thanks for the update

btw, i tried some 10-bit stuff on it. it looked nice (not that i can see any differences... suffice to say it worked, looked good and was fast).

one thing though - a couple of short clips i have tend to crash before the end somehow. tried it on a short chunk out of Pom Poko (2Vuy), and a short clip from GITS SAC (v210). they are both around 250 frames long, and both disappear virtualdub at around frame 205. they were captured on a mac with FCP 4.5 (we have FCP 5 now, which in case anyone is interested makes it possible to capture onto a networked NTFS drive where before it would hang for 3 hours and bugger up after 2 mins of capture).

if i can find a nice big place to host these, i'll give it a shot. they're around 150 megs each.

[tateu]

I found a bug when opening files with more than 2 channels of audio (5.1, etc. such as an h264/aac trailer downloaded from apple.com)...it crashes . I'm currently testing a fixed version that mixes it down to stereo and will hopefully have it available in a day or two. I've modified the new version so that all audio is output as 16bit Signed Int. I'd like to be able to return the data in the same format as the source file (8bit, all 6 channels, etc.) but that is not yet working. The new version will also feature much faster h264 decoding (about 15x the current version). For h264, though, ffdshow via DirectShowSource is still a better option. In my tests, it's atleast 1.5x as fast. And I'm still trying to get QTInput to return YV12 data.

I finally got a few raw clips encoded with QTOutput to the Avid Meridien Uncompressed Codec(AVUI), imported into Avid Adrenaline (it converts to omf) and printed to DigiBeta. I had to modify the raw output settings when using AVUI. Adrenaline didn't like the 2Vuy (uyvy) encoded files because it uses AVUI (also uyvy) but it stores each frame of 720x486 data as 720x496 (10 lines of black at the top) and has a four byte footer. It uses something similar for PAL and I need to add that correction in as well.

[Mug Funky]

hmm strange about the extra 10 lines. maybe it's for CC, macrovision or teletext?

i eaagerly await the next release. btw, avisynth can handle multichannel audio, so you may be able to return the correct number of channels without a downmix. this'd be cool for anime tapes come with ch 1+2 = english, 3+4 = japanese, and capping all 4 at once would be cooool (just do getchannel(1,2) etc). not sure if quicktime allows that kind of thing though.

[hanfrunz]

@tateu:
the extra lines in the avid file are just the "nonactive" lines, i found that avid also grabs them. You can switch this off somewhere in the settings, but then the lines are just blanked. You can use this for getting vitc and keycode-data into the avid system.

hanfrunz

[bond]

Originally Posted by tateu

The new version will also feature much faster h264 decoding (about 15x the current version). For h264, though, ffdshow via DirectShowSource is still a better option. In my tests, it's atleast 1.5x as fast.

great stuff, how did you get 15x to work? i recently measured the speed of various qt wrappers with h.264 mp4 with the following results:
- intervideo dshow: 19.93 fps (yuy2)
- qtinput 0.0.1: 19.60 fps (yuy2)
- qtinput 0.0.5: 18.74 fps (yuy2)
- cyberlink dshow: 18.60 fps (rgb24)
- qtreader: 15.72 fps (rgb24)

so you can see that qtinput seems to be pretty much in a range of the other wrappers

i noticed that qtinput 0.0.1 was faster than 0.0.5, any idea what caused the slowdown?

btw i couldnt get mode 1 and 2 to work with h.264 mp4 files!?
whats the difference, advantage/disadvantage of those? which one does qt itself use during playback?

And I'm still trying to get QTInput to return YV12 data.

would be great. cant be that a yv12 h.264 stream cant be decoded to raw yv12 via qt
what colorspace does qt itself output during playback?

edit:

To get h264 mp4 decoding working in this latest version, I didn't actually use the new quicktime 7 API's, I just fixed an error in my old decoding method. My best guess is that my method causes quicktime to always start the decode process from the nearest ref frame, even when stepping sequentially through the file. I don't think there is a frame buffer for anything but the current frame. This means that it is decoding way more frames than it actually needs to.

hm this sounds not good
is this still the case in 0.0.5?

[tateu]

I still haven't fully tested the new decoding routines so I haven't put together a release of a new version yet but if you want to give them a try (quicktime 6 will not work with this version)...http://www.tateu.net/software/QTSource_test_h264.7z. 6 channel audio should also now be correctly decoded to 6 channel 16bit pcm. Just be aware that this version may (and probably does) have even more bugs than my normal release versions and may have broken some previously working functions.

To test the new h264 speeds, use mode=2.
QTInput("file.mp4", color=2, mode=2)


As far as I can tell, the quicktime player seems to only operate in RGB. Using QTInput in RGB mode produces colors that are a much closer match to what quicktime player puts out. I do not know what decoding method it uses, though.

The new speed increase is using mode=2. Mode=0 speeds are still just as slow in the new version. Mode=0 uses a generic quicktime API function called MoviesTask which does all of the decoding work behind the scenes. You jump to a specific frame number, then call MoviesTask and it returns the decoded frame. From what I can tell, this method redecodes frames several times, almost as if it has to start the decode process from the nearest I-Frame, for every single frame, even when stepping sequentially. To decode frame 1, it redecodes frame 0. To decode frame 2, it redecodes 0 and 1. To decode frame 3, it redecodes 0, 1 and 2. Watch the frame rate as the file plays back, it starts out fast then slows down more and more, then instantly speeds up (when it reaches an I-frame, I think) then gradually slows down again. This means that it is possibly decoding 10-20 times the amount of frames actually contained in the file and why it can give you decent results on a 0 b-frame file but terrible results on a 2 b-frame file.

For the new method in mode=2, when a movie is first loaded, QTInput uses a QTSampleTableRef from CopyMediaMutableSampleTable to grab a list of all video frames with their respective decode and playback times. The list is copied into two vector structs and one is sorted by playback order and the other is sorted by decode order. It also keeps track of the last frame number decoded, the nearest I-Frame and a small buffer of 6 or 7 recently decoded frames. When avisynth requests a specific frame, QTInput uses the vector structs to determine which frames to decode and which frame to return for playback or. If you step sequentially forward by 1 frame at a time, this new method only decodes each frame once, so you get a huge increase in speed over the old method. If you skip ahead by several frames, it starts the decoding either from the last decoded frame or the nearest I-Frame, whichever is closer.

Mode=1 only works for uncompressed sources. It is a raw copy method that bypasses all quicktime codecs and returns raw YUV or RGB data.

I'm not sure why v0.0.1 would be faster than v0.0.5. The 0.86 fps your results show is such a small amount and I've made a ton of changes since v0.0.1. What test method are you using to get your results? Do your sample files have b-frames? The only way I can get speeds with QTInput mode=0 that are even in the same neighborhood as other decoders is on files that don't have b-frames.

I reran some mplayer benchmark tests today using the sample file you posted on page 2 of this thread (quicktime.mp4) and also a trailer from apple's site (King Kong 480p)converted to mp4 using mp4box (mp4box.exe -add king_kong-tlr_h480p.mov#1 -add king_kong-tlr_h480p.mov#2 king_kong-tlr_h480p.mp4). Both of them contain b-frames. These tests show the new mode=2 to be about 10x as fast as mode=0 and DirectShowSource with various decoders to be as much as 2.25x as fast as mode=2. And DirectshowSource used about 1/4 the amount of memory used by QTInput. To play the King Kong trailer, CoreAvc through DirectShowSource never went above 100MB, whereas QTInput used almost 400MB. I'm sure atleast some of this memory increase in QTInput is due to bad programming on my part but I don't know if any of it is actually due to quicktime itself.

The tests were run on a Dual 3GHz Xeon with 2GBs of ram using Celtic Druid's mplayer2006.01.01.P4.7z, Celtic Druid's ffdshow-20051103.exe and avisynth v2.5.7.0 built on 2005-12-15.

quicktime.mp4 - 300 frames
- QTInput Mode 0: 19.921s = 15.06 fps
- QTInput Mode 2: 1.932s = 155.28 fps
- AviSynth DShow ffmpeg: 0.983s = 305.19 fps
- AviSynth DShow CoreAVC 0.0.4: 0.907s = 330.76 fps
- AviSynth DShow Moonlight: 0.858s = 349.65 fps
- AviSynth DShow Nero6: 1.150s = 260.86 fps
- QTInput v0.0.1: Failed to decode

MPlayer dev-CVS-060101-18:59-4.0.2 (C) 2000-2005 MPlayer Team
CPU: Intel Pentium 4/Celeron 4 Northwood; Pentium 4 EE/Xeon Prestonia, Galla Family: 15, Stepping: 9)
CPUflags:  MMX: 1 MMX2: 1 3DNow: 0 3DNow2: 0 SSE: 1 SSE2: 1
Compiled for x86 CPU with extensions: MMX MMX2 SSE SSE2

86 audio & 201 video codecs
Playing quicktime_Mode2.avs.
AVS file format detected.
VIDEO:  [YV12]  640x256  12bpp  25.000 fps    0.0 kbps ( 0.0 kbyte/s)
=====================================================
Opening video decoder: [raw] RAW Uncompressed Video
VDec: vo config request - 640 x 256 (preferred colorspace: Planar YV12)
VDec: using Planar YV12 as output csp (no 0)
Movie-Aspect is undefined - no prescaling applied.
VO: [null] 640x256 => 640x256 Planar YV12
Selected video codec: [rawyv12] vfm: raw (RAW YV12)
=====================================================

mplayer -vo null -benchmark quicktime_Mode0.avs
	QTInput("quicktime.mp4", color=2, mode=0)
mplayer -vo null -benchmark quicktime_Mode2.avs
	QTInput("quicktime.mp4", color=2, mode=2)
mplayer -vo null -benchmark quicktime_Dshow.avs
	DirectShowSource("quicktime.mp4", 25)
mplayer -vo null -benchmark quicktime_v0.0.1.avs
	QTInput("quicktime.mp4", color=2)

king_kong-tlr_h480p.mp4 - 4201 frames
- QTInput Mode 0: 271.514s = 15.47 fps
- QTInput Mode 2: 39.101s = 107.44 fps
- AviSynth DShow ffdshow: 29.684s = 141.52 fps
- AviSynth DShow CoreAVC 0.0.4: 23.679s = 177.41 fps
- AviSynth DShow Moonlight: 34.910s = 120.34 fps
- AviSynth DShow Nero6: 30.584s = 137.36 fps
- QTInput v0.0.1: 272.396s = 15.42 fps

VIDEO:  [YV12]  848x352  12bpp  23.976 fps    0.0 kbps ( 0.0 kbyte/s)
=====================================================
Opening video decoder: [raw] RAW Uncompressed Video
VDec: vo config request - 848 x 352 (preferred colorspace: Planar YV12)
VDec: using Planar YV12 as output csp (no 0)
Movie-Aspect is undefined - no prescaling applied.
VO: [null] 848x352 => 848x352 Planar YV12
Selected video codec: [rawyv12] vfm: raw (RAW YV12)
=====================================================

mplayer -vo null -benchmark king_kong-tlr_h480p.mp4_Mode0.avs
	QTInput("king_kong-tlr_h480p.mp4", color=2, mode=0)
mplayer -vo null -benchmark king_kong-tlr_h480p.mp4_Mode2.avs
	QTInput("king_kong-tlr_h480p.mp4", color=2, mode=2)
mplayer -vo null -benchmark king_kong-tlr_h480p.mp4_Dshow.avs
	DirectShowSource("king_kong-tlr_h480p.mp4", 23.976)
mplayer -vo null -benchmark king_kong-tlr_h480p.mp4_v0.0.1.avs
	QTInput("king_kong-tlr_h480p.mp4", color=2)

[foxyshadis]

Curious, but do you throw away decoded b-frames? If you have knowledge of the decode order you can throw away frames as soon as they are no longer referenced (or never referenced at all), and that might keep mem usage down. If quicktime lets you know the references, that is...

[bond]

Originally Posted by tateu

I still haven't fully tested the new decoding routines so I haven't put together a release of a new version yet but if you want to give them a try (quicktime 6 will not work with this version)...http://www.tateu.net/software/QTSource_test_h264.7z. 6 channel audio should also now be correctly decoded to 6 channel 16bit pcm. Just be aware that this version may (and probably does) have even more bugs than my normal release versions and may have broken some previously working functions.

To test the new h264 speeds, use mode=2.
QTInput("file.mp4", color=2, mode=2)

i tried it on a .mp4 file encoded with qt7 itself (1 b-frame) and i dont get a speedup at all
in fact it gets slower:
mode2: 17.74 fps
mode0: 18.91 fps

I'm not sure why v0.0.1 would be faster than v0.0.5. The 0.86 fps your results show is such a small amount and I've made a ton of changes since v0.0.1. What test method are you using to get your results? Do your sample files have b-frames? The only way I can get speeds with QTInput mode=0 that are even in the same neighborhood as other decoders is on files that don't have b-frames.

my sample file uses 1 b-frame (has been encoded with qt)
my speed measurement method is loading the avs via directshow into graphedit and than using the elecard chegepua renderer

I reran some mplayer benchmark tests today using the sample file you posted on page 2 of this thread (quicktime.mp4)

i tried this sample too with both modes and on this sample there is indeed a big difference
with mode0 i get like 1.5 fps, with mode2 i get like 25 fps

quicktime.mp4 has been encoded with x264 and aimed at only using settings the qt7 encoder also uses. it wasnt perfect tough (used adaptive b-frames and only 1 slice) so i made a new one with the following settings being equivalent to the settings the qt7 encoder uses:

x264 --bitrate 770 --threads 5 --bframe 1 --no-b-adapt --ref 1 --filter 0:0 --no-cabac --subme 5 --analyse p8x8,b8x8,i4x4 --progress --no-psnr -o quicktime_x264.mp4 input.avs

i also encoded with the same settings to raw .264 and muxed the stream into .mp4 with both mp4box and mp4creator to see if this makes a difference
you can get all these samples here

as you will see yourself, all those samples also play very slow with mode 0, but ok with mode 2

i think there are now two possibilities:
1) mode 0 takes care about buffering_period SEI the qt7 encoder writes into the h.264 bitstream, but x264 doesnt
2) the qt7 mp4 muxer does something different than the others which make mode 0 be able to play the file faster

ad 1) this is not the case, because when i remux an qt7 encoded h.264 stream with mp4box to .mp4, the resulting file also plays slow with mode 0

ad 2) i guess thats the reason for the speed difference between decoding qt7 produced mp4 files and others with mode 0

x264 uses the gpac lib for mp4 creation (as mp4box does too). mp4creator uses its own lib. i have to say i tested the libgpacs and mp4creators mp4 files extensively and i dont know any open issues. so, as both also show the same slow speed, i dont think they do anything incompliant
i also tried remuxing all the files via qt7 passthrough, which didnt work as qt crashed reporting an "unknown error"

comparing the same stream once created with qt7 and once remuxed with mp4box and mp4creator i see the following differences in the files:
- slightly different durations in the mvhd and tkhd atom (mp4box and mp4creator have the same)
- qt7 has an elst atom (under edts)
- qt7 has a sdtp atom with lots of 00 and 08 (under stbl atom)
- qt7 has far more entries in the stsc and stco atoms (under stbl)
- qt7 has two free atoms before the mdat (mp4box doesnt, mp4creator has the mdat at the beginning)


btw i also noticed that the mp4creator created mp4 doesnt work with mode 2 (it works with mode 0). i only get a green picture (maybe because the mdat is at the beginning, headers at the end?)
qt7 handles it, so i think qt7 itself doesnt use your mode 2 during playback

The new speed increase is using mode=2. Mode=0 speeds are still just as slow in the new version. Mode=0 uses a generic quicktime API function called MoviesTask which does all of the decoding work behind the scenes. You jump to a specific frame number, then call MoviesTask and it returns the decoded frame. From what I can tell, this method redecodes frames several times, almost as if it has to start the decode process from the nearest I-Frame, for every single frame, even when stepping sequentially. To decode frame 1, it redecodes frame 0. To decode frame 2, it redecodes 0 and 1. To decode frame 3, it redecodes 0, 1 and 2. Watch the frame rate as the file plays back, it starts out fast then slows down more and more, then instantly speeds up (when it reaches an I-frame, I think) then gradually slows down again. This means that it is possibly decoding 10-20 times the amount of frames actually contained in the file and why it can give you decent results on a 0 b-frame file but terrible results on a 2 b-frame file.

For the new method in mode=2, when a movie is first loaded, QTInput uses a QTSampleTableRef from CopyMediaMutableSampleTable to grab a list of all video frames with their respective decode and playback times. The list is copied into two vector structs and one is sorted by playback order and the other is sorted by decode order. It also keeps track of the last frame number decoded, the nearest I-Frame and a small buffer of 6 or 7 recently decoded frames. When avisynth requests a specific frame, QTInput uses the vector structs to determine which frames to decode and which frame to return for playback or. If you step sequentially forward by 1 frame at a time, this new method only decodes each frame once, so you get a huge increase in speed over the old method. If you skip ahead by several frames, it starts the decoding either from the last decoded frame or the nearest I-Frame, whichever is closer.

i noticed that the first time you decode some frames its slow. when you stop and restart playing the frames the already decoded ones are shown very fast and once you come to frames that havent been decoded yet it slows down a lot
this is with both mode 0 and 2, i assume this means qt somewhere buffers the already decoded frames with both modes?

edit: changed a lot with new findings

[bond]

actually i think the reason for the performance difference between qt7 created files and others is the sdtp atom

the specs say:

Independent and Disposable Samples Box

This optional table answers three questions about sample dependency:
1) does this sample depend on others (is it an I-picture)?
2) do no other samples depend on this one?
3) does this sample contain multiple (redundant) encodings of the data at this time-instant (possibly with
different dependencies)?

propably quicktime relies on this info during playback?

either way sdtp is not mandatory, so a player shouldnt need it for playback