Quicktime Import Filter - Test Version, Still Buggy - 2006-01-09

[tateu]

This is very much a test version, there are still quite a few bugs, especially when dealing with audio. You must have Quicktime installed. Quicktime 6 is supported for video input and output, Quicktime 7 is supported for audio/video input and video output. Audio input for Quicktime appears to be working now. The filter should be able to read video from any file that Quicktime player can open (mov, mp4, raw dv, etc.). Quicktime mov creation supports video only and should be able to create a Quicktime file using any Quicktime codec that you have installed. Advanced codec parameters are partially supported.

[UPDATE 2006-01-09]
v0.0.5a
http://www.tateu.net/software/dl.php?f=QTSource
Source Code http://www.tateu.net/software/dl.php?f=QTSource_src

usage - Read:
QTInput("FileName.mov", color = 2, quality = 100, audio = false, mode = 0, raw = "yuyv", info = 0, dither = 0)
QTInput("FileName.mov")

file - input filename
color - output colorspace (default = 2)
0 = RGB24
1 = RGB32
2 = YUY2
quality - decompression quality (default = 100)
There are really only two values for this setting:
0 = normal
> 0 = High quality
This parameter does not have an effect on all codecs but there is a noticeable drop in quality for some of them (dvc) if it is set to 0. It is recommended that you leave this value at it's default value of > 0.
audio - audio input (default = false)
false = audio disabled
true = audio enabled
This is only valid if you have Quicktime 7. Audio is completely disabled for Quicktime 6.
mode - Decode mode (default = 0)
0 = Uses Quicktime SDK function MoviesTask to decode video
1 = If source file is one of several YUV formats (listed below), this mode returns raw data
2 = Uses Quicktime SDK function DecompressSequence to decode video
raw - raw input colorspace (default = "yuyv")
If mode = 1, this sets the input file colorspace method. Choices are:
uyvy (2vuy), yuyv (yuy2), yvyu, vyuy, yuv2, rgb, argb
info - Overlays codec info on the video (default = 0)
0 = info feature is turned off
1 = info feature is turned on
dither - raw 10bit to 8bit dither mode (default = 0)
0 = off
1 = Uses the 2 least significant bits as a fraction for rounding the 10bit value to the nearest 8bit value.
Currently this really isn't a dither mode, but is in place for a future dither mode option.
This only has an effect when mode=1, raw="v210."


usage - Write:
QTOutput("FileName.mov", format = "", quality = 80, datarate = 1000, keyframe = 1, raw = "")
QTOutput("FileName.mov")

file - output filename
format - quicktime FourCC codec (default = "")
If you leave this value blank, a dialog box will open allowing you to choose which codec to use.
h264 compression is a little buggy. It doesn't work if "keyframe every" is checked. This means you have to use the dialog box for h264 and uncheck it
Other options for this depend on what codecs you have installed, some examples:
"dvc" = DV25 NTSC
"dvcp" = DV25 PAL
"rle" = Animation
"SVQ3" = Sorenson Video 3
"cvid" = Cinepak
"IV41" = Intel Indeo¨ Video 4.4
"cvid" = Cinepak
quality - compression quality (default = 100)
This value is divided by 20 and truncated to an integer, so a value of 25 is identical to a value of 20.
0 = Least
20 = Low
40 = Medium
60 = High
80 = Best
100 = Lossless (this is listed as lossless in the SDK, but is probably not always lossless, but rather just the highest setting for a given codec
333 = This is a special mode that will create a file called "Quicktime_CodecList.txt" in the same directory as the avs script. This file will contain a list of the FourCC codes installed on your system.
datarate - datarate in KBytes/sec (default = 1000)
keyframe - keyframe every n frames (default = 1)
raw - raw output colorspace (default = "")
This is a raw output mode. Instead of using a quicktime codec to encode data, this mode copies the data directly from YUY2 to the requested output mode.
You must set format to a valid quicktime FourCC.
Currently only format="2Vuy", raw="uyvy" is supported.
And you must have a codec installed that handles 2Vuy, such as the Blackmagic codec.


Version 0.0.5a, 2006-01-09
Fixed raw YUV input that was broken in 0.0.5.

Version 0.0.5, 2006-01-09
Added raw input mode for RGB and ARGB data.
Fixed (hopefully) audio input for Quicktime 7.
Fixed several memory errors that caused QTInput to sometimes crash on opening or closing a file.

Version 0.0.4, 2006-01-07
Added a dither (rounding) mode to raw 10bit input (v210).
Added a raw output mode for 2Vuy (uyvy).
Added a frame count check to QTOutput. This keeps it from writing more frames than the input clip has.
Removed some unnecessary code.

Version 0.0.3, 2006-01-05
Added 10Bit raw decoding of v210 uyvy video.
Fixed a bug with the way fps info was calculated for some movies.
Fixed a bug with the way frame count was calculated for h264 mp4 files.
Fixed part of a bug with the decoding of h264 mp4 files in mode=0. Still not 100% working, first frame is not decoded and it sometimes skips over a frame.
Added the default compression dialog box and two new compression parameters, datarate and keyframe.
h264 compression is a little buggy. It doesn't work if "keyframe every" is checked. This means you have to use the dialog box for h264 and uncheck it .

Version 0.0.2, 2005-12-20
Added two new decode methods: Quicktime SDK DecompressSequence and a raw mode for some types of YUV data.
Added info parameter to overlay codec info on top of video.

[peter100m]

Very nice tateu!
Have tried QTInput with different sources and compressions (uncompressed, dv, animation) and everything seems good so far (except for audio of course)!

Now we can finally choose colorspace and get YUY2 data from Quicktime in an easy way!

For testing purposes, is there a way to see which decoder is being used? I have several decoders for uncompressed video and I want to know which one that actually does the decoding.

Many thanks for this!

[tateu]

Don't jump for joy just yet. Even though Quicktime is returning YUY2, I am not sure that it is returning an exact copy of the uncompressed data. I'll have to setup up some comparisons against one of the rawsource filters.

If it is altering the data, I can try a call to Quicktime's GetMediaSample and return the raw data to AviSynth. I might run into the same problems as rawsource with this method, though. I did a quick test this morning on an Avid Meridian 1:1 encoded mov and GetMediaSample returned 720896 bytes of data, which would suggest a YUY2 frame size of 720x500 with an 896 byte header. The actual frame size of the file was 720x486, so my test was unsuccessful because of a buffer overflow. I'm not sure if I will be able to figure out the header and extra frame size data for all of the different uncompressed codecs.

As for showing you which decoder is being used, I'll put in an option to use subtitle to overlay some file and decoder info. The codec is chosen based on the FourCC value of the file. If you open the file in Quicktime player and select "Show Movie Info," it will show you the codec being used. My filter uses the same codec.

[bond]

wow, this might be the first way to measure decoding speed of qt's decoders

but i have a few questions:
- why does QTInput() not support YV12 output colorspace? doesnt qt7 support it? what colorspace is outputted in qt7 itself by default?
- i dont understand what output decompression quality should be? shouldnt every decoder output the same quality? does this affect the h.264 decoder?
- how fast/slow is QTInput()?

[peter100m]

I'll have to setup up some comparisons against one of the rawsource filters.
I actually did this prior to posting before. I took an uncompressed YUV-file of some colorbars and compared QTInput(color=2) with Rawsource. The colors were the same (checked with the Color Picker filter in Virtualdub, and color picker in Photoshop).

I was asking for decoder info because QT reports the Blackmagic codec as decoder for the file and I heard somewhere that the Blackmagic codec only did RGB output. I guess it at least can output YUY2 as well...
The option to overlay some info could be useful! Thanks again!

@bond:
regarding decompression quality, if you play a DV file in the Quicktime Player you will see a visually degraded picture because the player appearantly doesn't use "decompression quality=100"..if you use QTInput() with decompression quality=100 the picture is not degraded. Maybe this exists to reduce CPU load but is sure is annoying, not knowing what your file actually looks like.

[tateu]

Bond,
I can't seem to get it to output YV12 but I'm still trying. The Quicktime developer site has sample code to use for determining if a given codec can output to the desired format (it uses a codec's cpix resource). The code doesn't seem to work with 3rd party codecs but for the one's I've gotten it work on, kYUV420PixelFormat always returns false. kYUVSPixelFormat, k24BGRPixelFormat and k32BGRAPixelFormat return true.

For QTInput, there are really only two quality options: equal to 0 and greater than 0. Quality = 0 sets the decoder in a normal (lower quality) decoding mode, probably meant for a preview mode so you get a faster decode. It does not have an affect for all codecs but for some of them, DV25 for example, there is a large drop in quality when set to 0 (very blurry output). I have not tried it on an x264 file yet but if you leave it set at the default value of 100 you should get the best possible quality.

For QTOutput, quality sets the basic encoder compression parameters. If you open any program that can export to Quicktime you get a dialog box where you choose the codec, fps, depth and quality (least, low, medium, high, best). This quality setting is what gets set by the QTOutput quality param. This does not affect all codecs, but for mjpa this adjusts the amount of jpeg compression per frame. For an mjpa test encode I tried:
quality = 0, resulted in a data rate of 11.55 mbit/s
quality = 20, resulted in a data rate of 15.00 mbit/s
quality = 40, resulted in a data rate of 29.14 mbit/s
quality = 60, resulted in a data rate of 46.48 mbit/s
quality = 80, resulted in a data rate of 63.45 mbit/s
quality = 100, resulted in a data rate of 105.11 mbit/s

There are also sometimes more advanced compression options (data rate, interlaced, etc.). These are not yet available with my filter.

I haven't done any speed tests. Are you looking for speed compared to mencoder/ffmpeg quicktime decoding or to compared to avi vfw decoding?


peter100m,
That's good news. I'm still going to try and have an option to return raw data. This will also facilitate a planned direct stream copy mode from quicktime to avi.

[bond]

thx :)

what colorspace is outputted in qt7 itself by default?
eg mpeg-4 and h.264 streams are always YV12, so it would be interesting to know if qt7 does a colorspace conversion during playback and if thats avoidable with your plugin

i made a quick test on a h.264 sample with quality = 0 and 100 and it seems to output exactly the same picture

I haven't done any speed tests. Are you looking for speed compared to mencoder/ffmpeg quicktime decoding or to compared to avi vfw decoding?yes, till now there exists no way to test the max decoding speed a decoder in qt can achieve afaik

for getting an idea on how fast/slow your plugin is i now did the following:
i played an .avi with uncompressed YUY2 video (~3gb) via .avs and your qtinput() in directshow (so its decoded in qt7) and measured the decoding speed
than i played the same .avi directly in directshow without avs and your plugin

the result was:
qtinput: 22.63fps
normal: 39.94fps

so when going via your plugin it was played slower than realtime (the file has 25fps)
i noticed that at the beginning it played at normal speed, after seeking eg to the middle of the file i saw that things were slower (this is not the same in qt7)

i dunno if thats a bug in your plugin or maybe caused somehow by the big filesize or if your plugin is simply slower?

[IanB]

Source code?

[Mug Funky]

excellent.

one thing - i've got a couple of 2vuy clips hanging around (i can do a bit comparison of output if you like, because i have the exact same clips in uyvy avi files)

they don't load.. virtualdub gives me a clip of correct length (you can see the little ticks on the timeline), but before the image shows up it disappears.

[Leak]

Source code?
Ummm...

I'll try and get the source packaged sometime in the next few days.

[tateu]

A new version and the source code will be available later tonight, probably 8-10 hours from now. The new version doesn't fix any of the bugs, it just adds two new decoding modes and an info option which shows you the codec used by the decoder. Mode=0 decodes the video the same way that the current version does. Mode=1 (not 100% perfect yet, sometimes has trouble with the extra frame header data and interlaced separate field codecs) will copy the raw 8bit YUV (uyvy, yuyv, yvyu or vyuy) data directly from the file without decompression. Mode=2 will give you the exact same output as Mode=0 but it uses a different decompression method, which seems to be a little faster.


Mug Funky,
Hmm...do you feel like sending me one of the files that crashes? If you don't have a site to upload to, I can get an ftp link for you.


bond,
I'm not sure what the default colorspace for qt7 is. The way I understand quicktime: You build a GWorld, which is some magic quicktime construct that I don't fully understand, which is used for compressing and decompressing graphics. In the constructor for a GWorld, you tell it what pixel format you want (YUY2, 24BGR, etc.) and this determines the output colorspace when decompressing an image. When the decompresser does not support the desired colorspace the function fails. This is what happens when I try YV12.

I've done some tests using color bars as the source. I used Avisynth's ColorBars(720,486) to create a YUY2 avi with a luminance range of 16-235. I then converted that avi with Adobe After FX into several different types of quicktime movies and fed them all back into QTInput. For most codecs, it seems that QTinput returns a closer match to the original avi colors and luminance range when using RGB output. When using QTInput YUY2, most codecs converted the grey bar rgb(180,180,180) to rgb(170,170,170) and the white bar rgb(235,235,235) to rgb(232,232,232). When using QTInput RGB24, there was a much less noticeable change. I also opened the file in quicktime player and saved a still frame as a bmp. The bmp was a very close match to QTinput in RGB mode. The one exception I have found so far, is the Black Magic 2Yuv codec. It's output was a closer match when QTInput was set to YUY2. I have not run these tests on actual video material yet.

I have sometimes noticed inconsistent frame rates when playing my scripts in Media Player but haven't done a whole lot of testing with it. Nor have I tried anything larger than a 1GB mov. The 1GB DV25 file I just tried, seems to play at about 30fps the whole way through no matter how much I jump around. Maybe it's related to a specific codec?

I also did some speed tests of my own earlier today, with a different method, on DV25 and 2vuy material. I used ffmpeg's raw stream copy method as my speed test. I fed several different scripts and the mov file into ffmpeg with the command line "ffmpeg -an -i file.mov -vcodec copy -r 29.97 -f avi file.avi" and timed how long it took to complete. In this mode, ffmpeg read the raw data from the mov file and created a dv25 avi and a 2yuv avi. It read the raw data from the Avisynth scripts and created i420 avi files.

Results for a 30 second 2yuv mov:
QTInput Mode 0 = QTinput("2yuv.mov", color=2, mode=0)
QTInput Mode 1 = QTinput("2yuv.mov", color=2, mode=1, raw="uyvy")
QTInput Mode 2 = QTinput("2yuv.mov", color=2, mode=2)
AviSource("2yuv.avi") - via ffdshow

Mov DirectStream Copy = 15.7s
QTInput Mode 0 = 42.4s
QTInput Mode 1 = 15.5s
QTInput Mode 2 = 38.9s
AviSource = 39.4s

Results for a 8799 frame DV25 mov:
QTInput Mode 0 = QTinput("DV25.mov", color=2, mode=0)
QTInput Mode 2 = QTinput("DV25.mov", color=2, mode=2)
AviSource("DV25.avi")

Mov DirectStream Copy = 107s
QTInput Mode 0 = 252s
QTInput Mode 2 = 228s
AviSource (CedocidaDV) = 204s
AviSource (ffdshow DV) = 240s

[tateu]

New version:
http://www.tateu.net/software/dl.php?f=QTSource

Source is a mess, has lots of incomplete and unused code but here it is anyway:
http://www.tateu.net/software/dl.php?f=QTSource_src

Added two new decode methods: Quicktime SDK DecompressSequence and a raw mode for some types of YUV data.
Added info parameter to overlay codec info on top of video.

[Mug Funky]

haha! i accidentally made it open an avi file, and it worked perfectly...?

it was an uncompressed uyvy avi though, and qt7 has no trouble opening these (which is quite useful)

[edit]

with the latest version you posted i have no trouble loading 2vuy files. in raw mode it gives identical results to loading the avi (not a pixel changed... very good). however, using blackmagic's codec with color set to 2 (default), there's quite a difference - about quarter/half a pixel shift for some reason and chroma is tinted. i was using subtract(qt, avi).coloryuv(autogain=true) to compare, so i couldn't say which direction the image shifted.

looks like i'll be using raw mode anyway, as it's hella fast. thankyou very much - i've been wanting to import 2vuy quicktimes _losslessly_ for a long time, and now i can :)

sure beats having to rip the decklink card out of the mac and shove it in a PC for capturing...

[tateu]

Looks like maybe it is quicktimes fault? I started with a Blackmagic quicktime file of NTSC color bars (15-234, for some reason). I opened the file in After Effects...rgb(179,179,179) was changed to rgb(187,187,187). rgb(234,234,234) was changed to rgb(236,236,236). rgb(16,15,179) was changed to rgb(23,21,189).

I then opened the same file in quicktime player, chose export and selected the Blackmagic codec. This should theoretically give me an identical file? It did not. After 9 generations of exports, the file was nowhere near its original state. I also tried the same export test using one of Avid's (supposedly) lossless yuv codecs (AV1x). It performed better than the Blackmagic codec, but still did not return an identical file.

Still Frame Source: http://www.tateu.net/software/Bars_Source.png
Still Frame Blackmagic Generation 9: http://www.tateu.net/software/Bars_BlackMagic_Gen9.png
Still Frame Avid AV1x Generation 9: http://www.tateu.net/software/Bars_Avid1.1x_Gen9.png

[Pookie]

tateu - Awesome. Thanks so much for working on this.

[bond]

tateu, so yv12 output doesnt work even on yv12 files (eg h.264 ones)?

[hanfrunz]

I then opened the same file in quicktime player, chose export and selected the Blackmagic codec. This should theoretically give me an identical file? It did not. After 9 generations of exports, the file was nowhere near its original state. I also tried the same export test using one of Avid's (supposedly) lossless yuv codecs (AV1x). It performed better than the Blackmagic codec, but still did not return an identical file.


i had similar problems the other day. I imported a .yuv file with a avid mediacomposer. It had some areas of "super black" (y=13). I thought avid would import raw yuy2 data, but it looks like that the .yuv-import filter makes a conversion to RGB first. Because these areas were always clipped (i checked all import setings). maybe it is also a quicktime related problem. Who knows....

regards and thanks again for this cool filter,
hanfrunz

[tateu]

bond,
It doesn't work with the code I have written but I looked through the mplayer source code today and it looks like maybe they have been successful setting it to YV12. Hopefully I can make sense of their method and adapt it to work with my filter.


hanfrunz,
After seeing the results with quicktime, I was also curious about what happens when importing into an Avid system. When I get some free time, I plan to try it out on our Adrenaline.

[bond]

bond,
It doesn't work with the code I have written but I looked through the mplayer source code today and it looks like maybe they have been successful setting it to YV12. Hopefully I can make sense of their method and adapt it to work with my filter.mplayer can also wrap quicktimes decoders? if yes, how? :)

[morsa]

well there is the Quicktime Alternative project also.
I guess it is a directshow filter for quicktime.I use it with Avisynth and it works.It has some problems though, like getting a quarter resolution clip when its color space is YV12 or 4:1:1/4:2:0

[Mug Funky]

@ tateu:

fascinating. looks like QT/blackmagic has a dirty little secret? :) that chroma drift looks to have been created by multiple yuy2-rgb conversions (and it drifts to the side! that means even with 1 generation there's going to be a chroma offset... i mean come on, is it that hard to convert colourspaces without moving stuff around?)

that's good though - that means us avisynthers can make as many generations as we like without loss. though i fear for 10-bit files the same losses will occur (it'd be cool to dither 10-bit to 8-bit on import, but that's for another time. and when avs 3.0 comes along it'd be much nicer to just use it's 15 bit colourspace on 10 bit files).

looks like i'll be doing all my capping in 8 bit for now then :)

thanks heaps for your import plugin - it's nice to know there's more quality to be had from quicktime than what quicktime itself actually provides.

[tateu]

Yes, it seems strange that there would be that much of a shift. I expected lossless in, lossless out (I'm still hoping that someone proves me wrong and shows me some checkbox I forgot to select). As I delve deeper and deeper into quicktime and omf, I'm finding that I mostly prefer plain old AVIs, I seem to have fewer problems.

10bit...yes, I was looking for a way to do a raw import, but I don't know enough to write my own 10bit to 8bit function from scratch. If I can find an open source one I'll add it in.

The raw import still needs some work. I need to figure out how to accurately determine the frame header size so I can skip over it. Right now it's just a guess. A call to GetMediaSample returns the raw data and the size of the data in bytes. I know the height and width of the file which allows me to determine the actual size in bytes of 1 frame. I then assume the header size is equal to (Raw data size) - (actual frame size) and skip over that amount.

As an Example using 720x486, progressive, Avid AV1x codec:
GetMediaSampleSize = 720,896
YUY2 FrameSize = 699,840 = 720 * 486 * 2
Assumed Header Size = 21056 = 720,896 - 699,840

This method does not work on every file I have tried and it definitely does not work for separate field interlaced files because each field has it's own header data and GetMediaSample returns the entire frame, not each field. I'm still looking for a foolproof solution.

And I found a bug that shows up occasionally which sets the incorrect fps. I used the wrong values in my calculation. I used MovieTimeScale / GetMovieNextInterestingTimeDuration when it looks like it might be better to use MediaTimeScale / MediaSampleDuration. Quicktime doesn't have an fps parameter, it has timescale and sampleduration which seem to act kind of like fps.numerator and fps.denominator. The movie as a whole has a timescale and each piece of media (video track, audio track, etc.) has it's own timescale. I was calling GetMovieNextInterestingTime and using it's duration as my fps denominator but this failed on one file I tested. MovieTimeScale was 2997 and the first call to GetMovieNextInterestingTime was 64 resulting in 46.83fps. If I had used MediaTimeScale / MediaSampleDuration it would have been 600 / 20 = 30fps. I don't fully understand the concept of sampleduration, though, so even this method may not be 100% correct. One 30fps file that I have, has a MediaTimeScale of 600 so it seems that each frame should have a sample duration of 20, but the frames actually ranged in duration from 19-22. It wasn't meant to be a variable frame rate file, it was just a DV25 file rendered out of Final Cut Pro on a Mac.

December is always my busiest time of year so I won't have time to work on any of these issues until the beginning of next Jan.

[Mug Funky]

i wonder if it'd be possible to reverse-engineer output of some uncompressed formats?

because even though we can decode "lossless" (i'm growing more and more wary of this term when dealing with quicktime), we can't export back to it without loss.

with the example of blackmagic 2Vuy, my preferred method is to export via ffdshow a uyvy avi file. the quicktime blackmagic codecs can load this without fuss, so i save it out from there. in theory there's no yuy2 to rgb conversion happening, at least in theory.

now with the QTsource plugins, we can also export straight out of our scripts by just playing through. now this would seem an even safer way to preserve the colourspace throughout the chain.

however, i did a little test where i made a mov with both these techniques. they both loaded fine in QT and avs, but when subtracted from the source, it was clear there were differences.

the avs -> ffdshow uyvy -> mov file showed evidence of colourspace conversion (it was blurred and slightly shifted).

the avs -> QToutput -> mov file showed no blurring that i could tell, but a luma change of some kind (i haven't tracked down the numbers yet). it doesn't seem to be the standard 8-235 clamping.

i think if this is sorted, we might finally have a way of working with quicktime that's truly lossless.

[edit]

i think it's using different colour coefficients... the reds are sort of orange when i view it in QT player. that means it's almost certainly converting colourspace. bugger it.

[morsa]

one more of the hundred reasons why I insist on not using quicktime despite what Mac fans tell about about its "superior image quality" :confused:

Anyway I respect it as a container, problem is it is so closed that you can't never really tell what is going on inside it......

[hanfrunz]

10bit...yes, I was looking for a way to do a raw import, but I don't know enough to write my own 10bit to 8bit function from scratch. If I can find an open source one I'll add it in.


ITU-601 says:
To avoid confusion between 8-bit and 10-bit representations, the eight most-significant bits are considered to be an integer part while the two additional bits, if present, are considered to be fractional parts.

So just take the eight most-significant bits and you got your 8->10 bit conversion.

hanfrunz

[Mug Funky]

Anyway I respect it as a container, problem is it is so closed that you can't never really tell what is going on inside it......

hit the nail right on the head there. just another bit of ammunition for me to use on the mac fanboys at work :sly:

@ hanfrunz:
true, true. i'd like to see dither added too - it might help repress blocking when encoding to mpeg-2 (or anything else for that matter). certainly a nicer kind of noise than blockbuster gives.

[bond]

tateu: i have upload a .mp4 file with h.264 video here (http://home.pages.at/bond_/quicktime.mp4). it has been created with x264
it plays in quicktime (jerkily) but when i try to open it via your plugin it only shows me a green picture. the funny thing is i also cant open it via the cyberlink dshow qt source filter (coming with qt alternative)

any idea why it doesnt work?

[tateu]

Mug Funky,
This document, http://developer.apple.com/quicktime/icefloe/dispatch019.html has a lot of info on quicktime and yuv. There are a few ImageDescription Extensions (colr & nclc) which describe color matrixes and transfer functions. I'm muddling my way through and hoping to find something useful. I've also found that mencoder using quicktime codecs in YUY2 mode does a much better job than my filter of decoding to YUY2 with the correct color output (still not perfect and some filters seem to only work in RGB). Mencoder/mplayer uses yet another quicktime function for decoding frames (ImageCodecBandDecompress) that I have not tried yet. I'll have to look into it some more. I'm definately in agreement with morsa...Quicktime is really starting to get annoying...there are way too many functions available to decompress a frame of video (ImageCodecBandDecompress, DecompressSequenceFrameWhen, MoviesTask, DecompressImage, ICMDecompressionSessionDecodeFrame, DataCodecDecompress, FDecompressImage and probably others that I haven't found yet).


Bond,
Sorry, I remember typing (in notepad) a reply to your question about mplayer and quicktime but I guess I never actually logged on and posted it. Mplayer has a quicktime demuxer which reads the properties of the quicktime file (FourCC, height, etc.). It then has a set of codecs that it can use to decode the video. "Codecs.conf" is used to determine which codec to use for each FourCC. If you play a Sorenson Video 3 quicktime with mplayer, the demuxer will determine that the FourCC is SVQ3 and will search through "codecs.conf" for a compatible decoder. The first one listed is ffmpeg, so mplayer will grab compressed media samples from the quicktime file and feed them to the ffmpeg SVQ3 decoder. If you remove or comment out the ffmpeg decoder entry, it will try to use the next one in the list, "Win32/QuickTime SVQ3 decoder." I have not been successful using the quicktime decoders with mplayer but they seem to work with mencoder. I added an entry for Blackmagic's 10Bit codec (v210) and mencoder successfully re-encoded the file to mpeg4.

I tried "mplayer -vo directx SVQ3.mov" which crashes
and "mencoder SVQ3.mov -ovc lavc -lavcopts vcodec=mpeg4 -ffourcc XVID -o SVQ3.avi" which successfully encodes an mpeg4 avi.

As an example of a new mplayer/mencoder codec, here's the entry I added for the Blackmagic's 10Bit codec:
videocodec qtv210
info "Win32/QuickTime Blackmagic 10Bit"
status untested
fourcc v210
driver qtvideo
dll "BlackmagicCodec.qtx"
out BGR32

About h264...There are two problems there: 1) My method of returning the frame count doesn't seem to work with your sample file and always returns a value of 1. 2) It looks like I am using older APIs that cannot handle B-Frames, the error message that quicktime gives me is "kQTMediaHasDisplayOffsetsErr = -9461, /* Returned by old (non-B-frame-aware) media APIs that cannot cope with nonzero decode to display deltas on media that contain them. */." My guess is that the cyberlink filter has a similar problem.

I'll have to revert back to my first method of returning the framecount (GetMediaSampleCount) and figure out the newer quicktime 7 video APIs (ICMDecompressionSession).


hanfrunz,
Thanks for that info. I now have Blackmagic 10Bit (v210) raw decoding added in, using the 8 most significant bits.


I should hopefully have a new version available sometime in the next few days. It will add 10Bit decoding and fix the fps error. I think I may end up needing to start a new filter from scratch to clean everything up and try to get the colorspace and h264 issues resolved correctly. Right now, I just keep dumping code into the middle of an already messy project and I'm starting to get lost in it.

[bond]

Sorry, I remember typing (in notepad) a reply to your question about mplayer and quicktime but I guess I never actually logged on and posted it. Mplayer has a quicktime demuxer which reads the properties of the quicktime file (FourCC, height, etc.). It then has a set of codecs that it can use to decode the video. "Codecs.conf" is used to determine which codec to use for each FourCC. If you play a Sorenson Video 3 quicktime with mplayer, the demuxer will determine that the FourCC is SVQ3 and will search through "codecs.conf" for a compatible decoder. The first one listed is ffmpeg, so mplayer will grab compressed media samples from the quicktime file and feed them to the ffmpeg SVQ3 decoder. If you remove or comment out the ffmpeg decoder entry, it will try to use the next one in the list, "Win32/QuickTime SVQ3 decoder." I have not been successful using the quicktime decoders with mplayer but they seem to work with mencoder. I added an entry for Blackmagic's 10Bit codec (v210) and mencoder successfully re-encoded the file to mpeg4.hm, i have been able playing a sorenson3 mov file via mplayer after placing all the qtsystem files into the codecs folder of mplayer, but i dont seem to be able to get it to decode h.264 via qt's decoder. all i get is a black picture...

[tateu]

Pretty much the same here, mplayer crashes and I get nothing but black output from mencoder. I've tried the MPlayer-mingw32-dev-CVS-050928 build from mplayerhq.hu and CelticDruid builds from Aug, Nov and Jan 06. It could be the same problem as my filter: the quicktime function mplayer uses (ImageCodecBandDecompress) may not be able to handle B-Frames.

I'm using this as my config:

videocodec qth264
info "Win32/QuickTime h264"
status untested
fourcc H264,h264
fourcc avc1
driver qtvideo
dll "QuickTimeH264.qtx"
out YV12,I420,IYUV

[bond]

i tried this too an a mp4 without b-frames and it didnt work either :(

[tateu]

Ok, hopefully I didn't break anything with these changes...
http://www.tateu.net/software/dl.php?f=QTSource
http://www.tateu.net/software/dl.php?f=QTSource_src

A few bugs with fps and framecount were fixed. h264 mp4 files can now be partially decoded in mode=0 but it is still not 100% working. It returns a green frame for frame 0 and sometimes skips the decoding of a few frames. And it's really slow. Raw decoding of 10bit v210 files has been added. This is the format for the Blackmagic 10Bit codec. The last 2 bits of each component are dropped to convert to 8Bit YUY2.

For QTOutput, the standard compression dialog box has been added. This allows you to set a datarate for certain codecs. If you want to use the dialog box, leave format equal to "". If you do not want to use the dialog box, set format equal to a valid FourCC. Along with the dialog box, two new parameters have been added: datarate and keyframe. h264 compression is a little buggy. It doesn't work if "keyframe every" is checked. This means you have to use the dialog box for h264 and uncheck it.

[bond]

h264 mp4 files can now be partially decoded in mode=0 but it is still not 100% working. It returns a green frame for frame 0this might be caused by the delay introduced by b-frames?

and sometimes skips the decoding of a few frames.i noticed that too when playing avc in qt. i think thats caused if you use more than 1 b-frame

[tateu]

bond,
It's definately something with the b-frames. I took your sample mp4 file, converted it to YUY2 avi with mencoder and then encoded it to h264 mp4 using MeGui (mencoder, mp4box, CE-Quicktime profile) with 0, 1 and 2 b-frames.

The 0 b-frame version plays back with my filter at normal 25fps speed. The 1 and 2 b-frame versions behave exactly like your sample...stuttering around 2-6fps playback speed. They all play back relatively smoothly and at a decent speed in quicktime player, though.

To get h264 mp4 decoding working in this latest version, I didn't actually use the new quicktime 7 API's, I just fixed an error in my old decoding method. My best guess is that my method causes quicktime to always start the decode process from the nearest ref frame, even when stepping sequentially through the file. I don't think there is a frame buffer for anything but the current frame. This means that it is decoding way more frames than it actually needs to.

[anton_foy]

...WOW...I just need to say it...WOW!
Never thought this filter would ever come and now it is here, great work Tateu!

Is it possible to import mpeg-2 material with this filter and output to QT?

This would be a much faster and easier way than my present way:
importing my HDV material (mpg) in VDub, add a lot of avisynth filters, output in raw .avi then import the raw avi in AfterFX to put some more filters on and finally output it in Avid's DNxHD format (1080-25P 185 10-bit).

[tateu]

That's what it's supposed to do. No audio output, though....yet. And Avid's DNxHD codec seems to work in a test I just tried.

[Mug Funky]

Raw decoding of 10bit v210 files has been added. This is the format for the Blackmagic 10Bit codec. The last 2 bits of each component are dropped to convert to 8Bit YUY2.

weee! thanks for that. this will speed things up considerably with 10-bit sources (and increase the quality too).

one thing... would dithering be a possibility, or is there no point? i guess it would be too slow and would give a very small quality increase (if any).

[edit]

it just occured to me... would it be difficult to allow raw output as well? i'd like to avoid using QT's own stuff wherever possible :) something like the input filter where a fourcc (2Vuy for example) and pixel type (like uyvy) could be specified.

[tateu]

Raw output into a quicktime container? I don't think that will be overly difficult. I'll look into it this weekend.

It doesn't hurt to try dithering. If there's no quality increase or it's too slow then you don't have to use it. I don't know how to do it, though. I did a quick search this morning for a formula and didn't find anything that I understood. If I figure something out, I'll add it in.

One thing I did try (though I'm not sure how correct it is)...The current version completely ignores the 2 least-significant bits. I added an option which uses these bits for fractional rounding. Here's a version if you want to try it out...http://www.tateu.net/software/QTSource_10Bit_Round.zip. Add dither=1 to turn on rounding (only has an effect when using mode=1, raw="v210").

y10b = 1110 1010 10
y8b = 1110 1010 = 234
y_frac = 10 = 2

y8b_rounded = y8b + (y_frac > 1) = 235

[Mug Funky]

hehe... thanks for that :)

i don't have any 10-bit stuff on my machine right now (i'm at home for the weekend thankfully), but i'll try it out when i get to work.

i highly doubt there'd be a perceptible difference, but it could help to have a little bit of dither noise in there for the sake of the encoder - perfectly clean video will go blocky even at very low quants, but a little bit of noise (bonus if it's related to the signal like dither noise is) can help prevent blocking without hurting bitrate performance.

and if you get raw export working, even for 2Vuy, i would kiss you. i've got a really awesome looking "my neighbor totoro" in huffy that i want to export back to didge, and it'd suck if i had to bugger up the sharpness and colours by saving it with quicktime's routines. (seriously, anyone who gets the R4 totoro in march will have the best looking release in the world :). i'm real happy with how it turned out).

doing a simple subtraction between a resaved QT and the original is a real eye opener...

[edit]

i'm just reading up on dither for both audio and video. it looks like 1/2 the LSB of noise added before rounding will be fine. only thing is you might want to use the same seed each time, otherwise we will never get the same encode twice :)

there's also floyd-steinberg dither... i'm just looking for appropriate links

[edit 2]

here's a page with source for a floy-steinberg dither. i'm not sure if interlace needs to be taken into account for it though (probably) as it propagates error from 1 line onto pixels from the next (possibly unrelated) line.

http://www.home.unix-ag.org/simon/gimp/fsdither.html

[tateu]

I haven't looked at the dithering info yet, but here's a version with raw out to try...http://www.tateu.net/software/QTSource_RawOut.zip.

Avisynth must, of course, be outputing YUY2 and 2Vuy is the only setting currently supported (others, yuv2, etc., may be added later). You must have a codec installed that supports 2Vuy (Blackmagic) otherwise it will fail.

QTOutput("file.mov", format="2Vuy", raw="uyvy")

Raw out is running about 1.5 times as fast in my tests.

[Mug Funky]

with QT7 (the latest? i dl'd it a second ago) i'm getting "error -2048 - not a movie file" :(

but it's nice and fast :)

[edit]

the made file does load in QTinput without any problem.

btw, thanks heaps for this! it's a real nice new year's present, i tell you :)

[edit 2]

scratch that, the created file doesn't load with QTinput. i was accidentally loading the old one...

[edit 3]

scratch even that... what was happening was the input plugin was crashing on my sample at around frame 205, and that meant the movie was never finished writing... changing the input made QToutput make a usable file.

which is good news! :):):):):):):) this makes me so happy :) i'll crack a beer open for you.

[edit 4]

output is bit identical to input! party time!

[mode editing, omg]

writing the files seems to be tricky - i figured the safest way to make a file would be to load it in virtualdub and go "preview VBR" so it played the clip beginning to end. but when it finished, virtualdub went back to the first frame, which was then written to the end of my file... perhaps space needs to be allocated first and frames written to specific byte offsets in the output file as they're accessed? that could be tricky for large files (like an 80 minute movie...), but a simpler measure that would work for the "preview VBR" method is to not allow it to write more frames than in the source clip. it would just mean random access isn't possible (which i can live with of course).

[tateu]

writing the files seems to be tricky
Yeah, I know. I've been using it with VirtualDub pretty much the same way...My first version, several months ago, had all of the quicktime SDK functions called directly from within the avisynth methods. I quickly got rid of that code and created separate quicktime movie classes (CQTMovieDec and CQTMovieEnc) so that I could hopefully (and easily?) create a standalone quicktime program. The plan has always been to create a cli program that reads quicktime, avi, avs and can output to quicktime, avi, raw. I haven't started working on it yet, though.

As a quick fix, I like your idea of not allowing it to output more frames than the input clip has. I'll add that in and get it out sometime tomorrow.

The problem with your other method of preallocating space is that I am using quicktime functions to handle everything except encoding the data. Instead of encoded data, I feed the raw frames to AddMediaSample and I don't see a way of specifying offset locations with this method. The reason I do it this way is so that I don't have to figure out how to write the correct header and footer data.

[Mug Funky]

ah. that makes sense.

i really appreciate this by the way :) without the means to write my own plugins (or the time to learn a language), i have to harrass good people like you to make stuff :) it's really amazing how a free tool like avisynth can do stuff to such a high quality and speed that only the most expensive of pro equipment can rival it. in fact, your plugin is unique in that it's the only way of re-saving an uncompressed mov without any loss.

perhaps you could hook up with Avery Lee and maybe QT output could be hacked into Virtualdub? that would solve the frame access problem pretty definitively, and give people a much nicer alternative to QT pro. of course, if you're making your own program for similar things, then i'll let you do that too ;)

[Revgen]

perhaps you could hook up with Avery Lee and maybe QT output could be hacked into Virtualdub? that would solve the frame access problem pretty definitively, and give people a much nicer alternative to QT pro. of course, if you're making your own program for similar things, then i'll let you do that too ;)

Remember when Microsoft went after Avery because VirtualDub had support for .asf files?

I doubt he wants to repeat that experience with Apple.

It's still a good idea though.

[tateu]

Sure, no problem. I've been around here for years, never really talked all that much, but I've learned a great deal and found a lot of useful info and programs just by reading through the forums. Programming is just a hobby and I've always been grateful to all the people that work on programs like avisynth and release the source code for people like me to study and learn from. I'm glad that I can finally offer something useful back.

I've used Avisynth in our production workflow for years. I've always been impressed with how well and how much it can do.

Integrating directly into VirtualDub was another option I had thought of and I think it's better than a standalone cli version. I'm just not sure I am capable of doing it. I use fccHandler's modified VirtualDub-MPEG2 and I've glanced at the code he wrote for directly importing Mpeg-2 in the hopes that I could alter it to use my quicktime and omf code. I haven't done much with it so far but I do have my own custom version that checks the file extension of the file you are trying to open and if it is mov, mp4, dv or omf it writes an avisynth script in the VirtualDub-Mpeg2 home directory and opens that file instead. I also think I really need to get quicktime audio import and export working before fully integrating it with VirtualDub.

I don't think Apple would have much to say about quicktime integrated into VirtualDub. Wasn't the asf code reverse engineered? Apple releases an official quicktime SDK so that support can be added to any program. I am not a lawyer (and license agreements seem to cause a large thumping like pain just over my right eye) but everything I have read of the license agreements deals with distributing the quicktime installer and/or logos with an application.

New version...
http://www.tateu.net/software/dl.php?f=QTSource
http://www.tateu.net/software/dl.php?f=QTSource_src

This update contains the dither=1 (rounding mode), raw 2Vuy output, removes some unnecessary code and contains a frame count check for QTOutput to keep it from writing more frames than the input clip has.

[Mug Funky]

just tried export out in FCP with a decklink card and it accepted it without asking to re-render. that's very good news :)

note i've only tried PAL with this... it's possible that NTSC is another story - to be honest i've yet to successfully make a 2Vuy NTSC file that would actually play out to the NTSC deck here. that's certainly not a priority though - i only ever needed to do it once, and the file was small enough to burn to disc, so that wasn't a problem. i'm pretty sure it's user error, too - nobody here has actually needed to output NTSC through SDI, so no-one's got any experience doing it.

another thing i noticed is this plugin has exposed some alignment oddities in ffdshow - it was returning yuy2, and avisynth was reading it fine, but on QT export the picture was skewed majorly. i'm guessing it's a pitch oddity, but i fixed it by telling ffdshow to output the closest matching colourspace (even though in both cases it appeared to be returning yuy2).

[tateu]

Are you talking about something like this: http://www.tateu.net/software/skew.jpg? That's what I get when I use QTInput outputting YUY2 on a 420x240 quicktime. I tried several different codecs (uncompressed, Sorenson, mjpa) all with the same result. If I change QTInput to output RGB32, it works correctly. RGB24 is also skewed, but not as much and it's in greyscale.

I then tried a 418x240 quicktime which is skewed in all three colorspaces. I'm not sure what's going on, but it must be somewhere in my code.

[Mug Funky]

well, interestingly enough, the clip i was using also failed with HC (it just spat out a 0 kb file and said it was finished), even though i'd converted it to yv12 beforehand.

i have a sneaking suspicion it's a pitch or alignment issue that doesn't get corrected in avisynth (which is fine because it saves a little time i guess). but the old crop(align=true) chestnut didn't seem to work for it, while changing ffdshow's decoding options fixed it.

perhaps you could work around it by only giving quicktime pixels that you know are in the frame. as i understand it avisynth can store pixels outside the frame in order to either save time with simple operations like cropping, or to align data to 32 bit chunks. having not looked at the source, i don't really know what is going one here, rather i've just read a few discussions about pitch and alignment and possible problems with both in filter design.

i've also noticed some plugins don't work with other plugins as far as the skewing thing goes (for example, interleaving motion-compensated frames via mvtools and then using tsp's medianblurt on them results in a skewed mess).

[tateu]

Audio input for Quicktime 7 now seems to be working. Tests were run on several different formats: 48KHz Stereo (uncompressed, Mace6:1, ALaw, ULaw, QDesgin2 and AAC), 32KHz, 11KHz, mono, etc. I do not have any sample files with 5.1 surround sound to test. The movies were opened in Quicktime Pro and with QTInput (VirtualDub) and exported to 32bit wav files. The wavs were then invert mixed together to check for differences. The difference files never rose above -78db.

It always returns 32bit float. The sample rate and number of channels returned are the same as the source movie.

Audio is still disabled by default. You must set audio=true to use it.

http://www.tateu.net/software/dl.php?f=QTSource
http://www.tateu.net/software/dl.php?f=QTSource_src

[Mug Funky]

awesome!

do you think the differences are due to random dither for 16-bit stuff? i had wondered why when i exported audio from an SDI capture that audition reported true 32-bit. dither would explain this (if it was lossless from tape to mov to wav, audition should have reported 20 bits deep, even in a 32-bit file. digibeta carries 4 discrete 20 bit channels at 48k). i can live with dither though :)

i'll try this build out when i'm done exporting this movie to tape (thanks to your plugin :)). oyu might be interested to know that it worked flawlessly even when exporting an 83 minute feature. no crashes or memory problems i know about (at least not enough to crash a machine with 1 gig real and 2 gigs swapfile... i let it go overnight so i don't know how it performed).