Quicktime Import Filter - Test Version, Still Buggy - 2006-01-09

[tateu]

This is very much a test version, there are still quite a few bugs, especially when dealing with audio. You must have Quicktime installed. Quicktime 6 is supported for video input and output, Quicktime 7 is supported for audio/video input and video output. Audio input for Quicktime appears to be working now. The filter should be able to read video from any file that Quicktime player can open (mov, mp4, raw dv, etc.). Quicktime mov creation supports video only and should be able to create a Quicktime file using any Quicktime codec that you have installed. Advanced codec parameters are partially supported.

[UPDATE 2006-01-09]
v0.0.5a
http://www.tateu.net/software/dl.php?f=QTSource
Source Code http://www.tateu.net/software/dl.php?f=QTSource_src

usage - Read:
QTInput("FileName.mov", color = 2, quality = 100, audio = false, mode = 0, raw = "yuyv", info = 0, dither = 0)
QTInput("FileName.mov")

file - input filename
color - output colorspace (default = 2)
0 = RGB24
1 = RGB32
2 = YUY2
quality - decompression quality (default = 100)
There are really only two values for this setting:
0 = normal
> 0 = High quality
This parameter does not have an effect on all codecs but there is a noticeable drop in quality for some of them (dvc) if it is set to 0. It is recommended that you leave this value at it's default value of > 0.
audio - audio input (default = false)
false = audio disabled
true = audio enabled
This is only valid if you have Quicktime 7. Audio is completely disabled for Quicktime 6.
mode - Decode mode (default = 0)
0 = Uses Quicktime SDK function MoviesTask to decode video
1 = If source file is one of several YUV formats (listed below), this mode returns raw data
2 = Uses Quicktime SDK function DecompressSequence to decode video
raw - raw input colorspace (default = "yuyv")
If mode = 1, this sets the input file colorspace method. Choices are:
uyvy (2vuy), yuyv (yuy2), yvyu, vyuy, yuv2, rgb, argb
info - Overlays codec info on the video (default = 0)
0 = info feature is turned off
1 = info feature is turned on
dither - raw 10bit to 8bit dither mode (default = 0)
0 = off
1 = Uses the 2 least significant bits as a fraction for rounding the 10bit value to the nearest 8bit value.
Currently this really isn't a dither mode, but is in place for a future dither mode option.
This only has an effect when mode=1, raw="v210."


usage - Write:
QTOutput("FileName.mov", format = "", quality = 80, datarate = 1000, keyframe = 1, raw = "")
QTOutput("FileName.mov")

file - output filename
format - quicktime FourCC codec (default = "")
If you leave this value blank, a dialog box will open allowing you to choose which codec to use.
h264 compression is a little buggy. It doesn't work if "keyframe every" is checked. This means you have to use the dialog box for h264 and uncheck it
Other options for this depend on what codecs you have installed, some examples:
"dvc" = DV25 NTSC
"dvcp" = DV25 PAL
"rle" = Animation
"SVQ3" = Sorenson Video 3
"cvid" = Cinepak
"IV41" = Intel Indeo¨ Video 4.4
"cvid" = Cinepak
quality - compression quality (default = 100)
This value is divided by 20 and truncated to an integer, so a value of 25 is identical to a value of 20.
0 = Least
20 = Low
40 = Medium
60 = High
80 = Best
100 = Lossless (this is listed as lossless in the SDK, but is probably not always lossless, but rather just the highest setting for a given codec
333 = This is a special mode that will create a file called "Quicktime_CodecList.txt" in the same directory as the avs script. This file will contain a list of the FourCC codes installed on your system.
datarate - datarate in KBytes/sec (default = 1000)
keyframe - keyframe every n frames (default = 1)
raw - raw output colorspace (default = "")
This is a raw output mode. Instead of using a quicktime codec to encode data, this mode copies the data directly from YUY2 to the requested output mode.
You must set format to a valid quicktime FourCC.
Currently only format="2Vuy", raw="uyvy" is supported.
And you must have a codec installed that handles 2Vuy, such as the Blackmagic codec.


Version 0.0.5a, 2006-01-09
Fixed raw YUV input that was broken in 0.0.5.

Version 0.0.5, 2006-01-09
Added raw input mode for RGB and ARGB data.
Fixed (hopefully) audio input for Quicktime 7.
Fixed several memory errors that caused QTInput to sometimes crash on opening or closing a file.

Version 0.0.4, 2006-01-07
Added a dither (rounding) mode to raw 10bit input (v210).
Added a raw output mode for 2Vuy (uyvy).
Added a frame count check to QTOutput. This keeps it from writing more frames than the input clip has.
Removed some unnecessary code.

Version 0.0.3, 2006-01-05
Added 10Bit raw decoding of v210 uyvy video.
Fixed a bug with the way fps info was calculated for some movies.
Fixed a bug with the way frame count was calculated for h264 mp4 files.
Fixed part of a bug with the decoding of h264 mp4 files in mode=0. Still not 100% working, first frame is not decoded and it sometimes skips over a frame.
Added the default compression dialog box and two new compression parameters, datarate and keyframe.
h264 compression is a little buggy. It doesn't work if "keyframe every" is checked. This means you have to use the dialog box for h264 and uncheck it .

Version 0.0.2, 2005-12-20
Added two new decode methods: Quicktime SDK DecompressSequence and a raw mode for some types of YUV data.
Added info parameter to overlay codec info on top of video.

[peter100m]

Very nice tateu!
Have tried QTInput with different sources and compressions (uncompressed, dv, animation) and everything seems good so far (except for audio of course)!

Now we can finally choose colorspace and get YUY2 data from Quicktime in an easy way!

For testing purposes, is there a way to see which decoder is being used? I have several decoders for uncompressed video and I want to know which one that actually does the decoding.

Many thanks for this!

[tateu]

Don't jump for joy just yet. Even though Quicktime is returning YUY2, I am not sure that it is returning an exact copy of the uncompressed data. I'll have to setup up some comparisons against one of the rawsource filters.

If it is altering the data, I can try a call to Quicktime's GetMediaSample and return the raw data to AviSynth. I might run into the same problems as rawsource with this method, though. I did a quick test this morning on an Avid Meridian 1:1 encoded mov and GetMediaSample returned 720896 bytes of data, which would suggest a YUY2 frame size of 720x500 with an 896 byte header. The actual frame size of the file was 720x486, so my test was unsuccessful because of a buffer overflow. I'm not sure if I will be able to figure out the header and extra frame size data for all of the different uncompressed codecs.

As for showing you which decoder is being used, I'll put in an option to use subtitle to overlay some file and decoder info. The codec is chosen based on the FourCC value of the file. If you open the file in Quicktime player and select "Show Movie Info," it will show you the codec being used. My filter uses the same codec.

[bond]

wow, this might be the first way to measure decoding speed of qt's decoders

but i have a few questions:
- why does QTInput() not support YV12 output colorspace? doesnt qt7 support it? what colorspace is outputted in qt7 itself by default?
- i dont understand what output decompression quality should be? shouldnt every decoder output the same quality? does this affect the h.264 decoder?
- how fast/slow is QTInput()?

[peter100m]

I'll have to setup up some comparisons against one of the rawsource filters.

I actually did this prior to posting before. I took an uncompressed YUV-file of some colorbars and compared QTInput(color=2) with Rawsource. The colors were the same (checked with the Color Picker filter in Virtualdub, and color picker in Photoshop).

I was asking for decoder info because QT reports the Blackmagic codec as decoder for the file and I heard somewhere that the Blackmagic codec only did RGB output. I guess it at least can output YUY2 as well...
The option to overlay some info could be useful! Thanks again!

@bond:
regarding decompression quality, if you play a DV file in the Quicktime Player you will see a visually degraded picture because the player appearantly doesn't use "decompression quality=100"..if you use QTInput() with decompression quality=100 the picture is not degraded. Maybe this exists to reduce CPU load but is sure is annoying, not knowing what your file actually looks like.

[tateu]

Bond,
I can't seem to get it to output YV12 but I'm still trying. The Quicktime developer site has sample code to use for determining if a given codec can output to the desired format (it uses a codec's cpix resource). The code doesn't seem to work with 3rd party codecs but for the one's I've gotten it work on, kYUV420PixelFormat always returns false. kYUVSPixelFormat, k24BGRPixelFormat and k32BGRAPixelFormat return true.

For QTInput, there are really only two quality options: equal to 0 and greater than 0. Quality = 0 sets the decoder in a normal (lower quality) decoding mode, probably meant for a preview mode so you get a faster decode. It does not have an affect for all codecs but for some of them, DV25 for example, there is a large drop in quality when set to 0 (very blurry output). I have not tried it on an x264 file yet but if you leave it set at the default value of 100 you should get the best possible quality.

For QTOutput, quality sets the basic encoder compression parameters. If you open any program that can export to Quicktime you get a dialog box where you choose the codec, fps, depth and quality (least, low, medium, high, best). This quality setting is what gets set by the QTOutput quality param. This does not affect all codecs, but for mjpa this adjusts the amount of jpeg compression per frame. For an mjpa test encode I tried:
quality = 0, resulted in a data rate of 11.55 mbit/s
quality = 20, resulted in a data rate of 15.00 mbit/s
quality = 40, resulted in a data rate of 29.14 mbit/s
quality = 60, resulted in a data rate of 46.48 mbit/s
quality = 80, resulted in a data rate of 63.45 mbit/s
quality = 100, resulted in a data rate of 105.11 mbit/s

There are also sometimes more advanced compression options (data rate, interlaced, etc.). These are not yet available with my filter.

I haven't done any speed tests. Are you looking for speed compared to mencoder/ffmpeg quicktime decoding or to compared to avi vfw decoding?


peter100m,
That's good news. I'm still going to try and have an option to return raw data. This will also facilitate a planned direct stream copy mode from quicktime to avi.

[bond]

thx

what colorspace is outputted in qt7 itself by default?
eg mpeg-4 and h.264 streams are always YV12, so it would be interesting to know if qt7 does a colorspace conversion during playback and if thats avoidable with your plugin

i made a quick test on a h.264 sample with quality = 0 and 100 and it seems to output exactly the same picture

Originally Posted by tateu

I haven't done any speed tests. Are you looking for speed compared to mencoder/ffmpeg quicktime decoding or to compared to avi vfw decoding?

yes, till now there exists no way to test the max decoding speed a decoder in qt can achieve afaik

for getting an idea on how fast/slow your plugin is i now did the following:
i played an .avi with uncompressed YUY2 video (~3gb) via .avs and your qtinput() in directshow (so its decoded in qt7) and measured the decoding speed
than i played the same .avi directly in directshow without avs and your plugin

the result was:
qtinput: 22.63fps
normal: 39.94fps

so when going via your plugin it was played slower than realtime (the file has 25fps)
i noticed that at the beginning it played at normal speed, after seeking eg to the middle of the file i saw that things were slower (this is not the same in qt7)

i dunno if thats a bug in your plugin or maybe caused somehow by the big filesize or if your plugin is simply slower?

[IanB]

Source code?

[Mug Funky]

excellent.

one thing - i've got a couple of 2vuy clips hanging around (i can do a bit comparison of output if you like, because i have the exact same clips in uyvy avi files)

they don't load.. virtualdub gives me a clip of correct length (you can see the little ticks on the timeline), but before the image shows up it disappears.

[Leak]

Originally Posted by IanB

Source code?

Ummm...

Originally Posted by tateu

I'll try and get the source packaged sometime in the next few days.

[tateu]

A new version and the source code will be available later tonight, probably 8-10 hours from now. The new version doesn't fix any of the bugs, it just adds two new decoding modes and an info option which shows you the codec used by the decoder. Mode=0 decodes the video the same way that the current version does. Mode=1 (not 100% perfect yet, sometimes has trouble with the extra frame header data and interlaced separate field codecs) will copy the raw 8bit YUV (uyvy, yuyv, yvyu or vyuy) data directly from the file without decompression. Mode=2 will give you the exact same output as Mode=0 but it uses a different decompression method, which seems to be a little faster.


Mug Funky,
Hmm...do you feel like sending me one of the files that crashes? If you don't have a site to upload to, I can get an ftp link for you.


bond,
I'm not sure what the default colorspace for qt7 is. The way I understand quicktime: You build a GWorld, which is some magic quicktime construct that I don't fully understand, which is used for compressing and decompressing graphics. In the constructor for a GWorld, you tell it what pixel format you want (YUY2, 24BGR, etc.) and this determines the output colorspace when decompressing an image. When the decompresser does not support the desired colorspace the function fails. This is what happens when I try YV12.

I've done some tests using color bars as the source. I used Avisynth's ColorBars(720,486) to create a YUY2 avi with a luminance range of 16-235. I then converted that avi with Adobe After FX into several different types of quicktime movies and fed them all back into QTInput. For most codecs, it seems that QTinput returns a closer match to the original avi colors and luminance range when using RGB output. When using QTInput YUY2, most codecs converted the grey bar rgb(180,180,180) to rgb(170,170,170) and the white bar rgb(235,235,235) to rgb(232,232,232). When using QTInput RGB24, there was a much less noticeable change. I also opened the file in quicktime player and saved a still frame as a bmp. The bmp was a very close match to QTinput in RGB mode. The one exception I have found so far, is the Black Magic 2Yuv codec. It's output was a closer match when QTInput was set to YUY2. I have not run these tests on actual video material yet.

I have sometimes noticed inconsistent frame rates when playing my scripts in Media Player but haven't done a whole lot of testing with it. Nor have I tried anything larger than a 1GB mov. The 1GB DV25 file I just tried, seems to play at about 30fps the whole way through no matter how much I jump around. Maybe it's related to a specific codec?

I also did some speed tests of my own earlier today, with a different method, on DV25 and 2vuy material. I used ffmpeg's raw stream copy method as my speed test. I fed several different scripts and the mov file into ffmpeg with the command line "ffmpeg -an -i file.mov -vcodec copy -r 29.97 -f avi file.avi" and timed how long it took to complete. In this mode, ffmpeg read the raw data from the mov file and created a dv25 avi and a 2yuv avi. It read the raw data from the Avisynth scripts and created i420 avi files.

Results for a 30 second 2yuv mov:

QTInput Mode 0 = QTinput("2yuv.mov", color=2, mode=0)
QTInput Mode 1 = QTinput("2yuv.mov", color=2, mode=1, raw="uyvy")
QTInput Mode 2 = QTinput("2yuv.mov", color=2, mode=2)
AviSource("2yuv.avi") - via ffdshow

Mov DirectStream Copy	= 15.7s
QTInput Mode 0		= 42.4s
QTInput Mode 1		= 15.5s
QTInput Mode 2		= 38.9s
AviSource		= 39.4s

Results for a 8799 frame DV25 mov:

QTInput Mode 0 = QTinput("DV25.mov", color=2, mode=0)
QTInput Mode 2 = QTinput("DV25.mov", color=2, mode=2)
AviSource("DV25.avi")

Mov DirectStream Copy	= 107s
QTInput Mode 0		= 252s
QTInput Mode 2		= 228s
AviSource (CedocidaDV)	= 204s
AviSource (ffdshow DV)	= 240s

[tateu]

New version:
http://www.tateu.net/software/dl.php?f=QTSource

Source is a mess, has lots of incomplete and unused code but here it is anyway:
http://www.tateu.net/software/dl.php?f=QTSource_src

Added two new decode methods: Quicktime SDK DecompressSequence and a raw mode for some types of YUV data.
Added info parameter to overlay codec info on top of video.

[Mug Funky]

haha! i accidentally made it open an avi file, and it worked perfectly...?

it was an uncompressed uyvy avi though, and qt7 has no trouble opening these (which is quite useful)

[edit]

with the latest version you posted i have no trouble loading 2vuy files. in raw mode it gives identical results to loading the avi (not a pixel changed... very good). however, using blackmagic's codec with color set to 2 (default), there's quite a difference - about quarter/half a pixel shift for some reason and chroma is tinted. i was using subtract(qt, avi).coloryuv(autogain=true) to compare, so i couldn't say which direction the image shifted.

looks like i'll be using raw mode anyway, as it's hella fast. thankyou very much - i've been wanting to import 2vuy quicktimes _losslessly_ for a long time, and now i can

sure beats having to rip the decklink card out of the mac and shove it in a PC for capturing...

[tateu]

Looks like maybe it is quicktimes fault? I started with a Blackmagic quicktime file of NTSC color bars (15-234, for some reason). I opened the file in After Effects...rgb(179,179,179) was changed to rgb(187,187,187). rgb(234,234,234) was changed to rgb(236,236,236). rgb(16,15,179) was changed to rgb(23,21,189).

I then opened the same file in quicktime player, chose export and selected the Blackmagic codec. This should theoretically give me an identical file? It did not. After 9 generations of exports, the file was nowhere near its original state. I also tried the same export test using one of Avid's (supposedly) lossless yuv codecs (AV1x). It performed better than the Blackmagic codec, but still did not return an identical file.

Still Frame Source: http://www.tateu.net/software/Bars_Source.png
Still Frame Blackmagic Generation 9: http://www.tateu.net/software/Bars_BlackMagic_Gen9.png
Still Frame Avid AV1x Generation 9: http://www.tateu.net/software/Bars_Avid1.1x_Gen9.png

[Pookie]

tateu - Awesome. Thanks so much for working on this.

[bond]

tateu, so yv12 output doesnt work even on yv12 files (eg h.264 ones)?

[hanfrunz]

Originally Posted by tateu

I then opened the same file in quicktime player, chose export and selected the Blackmagic codec. This should theoretically give me an identical file? It did not. After 9 generations of exports, the file was nowhere near its original state. I also tried the same export test using one of Avid's (supposedly) lossless yuv codecs (AV1x). It performed better than the Blackmagic codec, but still did not return an identical file.

i had similar problems the other day. I imported a .yuv file with a avid mediacomposer. It had some areas of "super black" (y=13). I thought avid would import raw yuy2 data, but it looks like that the .yuv-import filter makes a conversion to RGB first. Because these areas were always clipped (i checked all import setings). maybe it is also a quicktime related problem. Who knows....

regards and thanks again for this cool filter,
hanfrunz

[tateu]

bond,
It doesn't work with the code I have written but I looked through the mplayer source code today and it looks like maybe they have been successful setting it to YV12. Hopefully I can make sense of their method and adapt it to work with my filter.


hanfrunz,
After seeing the results with quicktime, I was also curious about what happens when importing into an Avid system. When I get some free time, I plan to try it out on our Adrenaline.

[bond]

Originally Posted by tateu

bond,
It doesn't work with the code I have written but I looked through the mplayer source code today and it looks like maybe they have been successful setting it to YV12. Hopefully I can make sense of their method and adapt it to work with my filter.

mplayer can also wrap quicktimes decoders? if yes, how?

[morsa]

well there is the Quicktime Alternative project also.
I guess it is a directshow filter for quicktime.I use it with Avisynth and it works.It has some problems though, like getting a quarter resolution clip when its color space is YV12 or 4:1:1/4:2:0