new filter: import/export uncompressed videofiles

[tateu]

Anyone successfully using this with uncompressed OMF files or have another plugin that can read them? I'm having trouble figuring out what the offset values should be. I have NTSC 720x486 uncompressed OMF files created by an Avid Adrenaline and by After FX 6.5.

I've also downloaded what's left of the OMF Toolkit (did not come with documentation) and have started trying to write my own OMF Reader plugin. I'm not having a whole lot of luck with that either. I can successfully read a few frames before it crashes. I think it's the offset values again causing the major problems.

[hanfrunz]

hello tateu,

when i wrote this filter, i only tested PAL-files, so i can't help you with the offsets. But i can give some tipps to find them out.

-generate a clip with one black frame, then a white frame.
-use 8 bit files. 10bit arent supported
-use your favorite hexeditor to find out where the black frame starts.
-do the same with the white frame
-avid also stores some lines of video above the picture data (timecode etc.), so you have more than 480 lines/frame
-if i remeber right the fields are stored one after the other, they are not weaved

if you really start a real omf-import filter, tell me maybe i can hlp you.

regards,
hanfrunz

[tateu]

Thanks for the offer of help. I actually made some good progress tonight on my OMF import filter. I can successfully read all of the NTSC 1:1 sample OMF files that I have. I can read the 1:1 PAL samples too, but the field order is reversed. I haven't yet figured out how to determine the correct field order from the OMF.

The next steps will be to figure out the field order issue. So far, all of my NTSC files are lower field first and the PAL files are upper field first. If that turns out to be a rule set in stone I can easily change my filter to always read PAL files upper field first. Then, I'd like to be able to read audio data and compressed (DV, tiff, jpeg) video data and eventually create OMF files. And then, of course, fixing the millions of bugs that it probably has...Whether I can actually ever accomplish any of these things is another story. I'm leaving for vacation in a few days, though, so I probably won't be able to make any more progress on this filter for a few weeks.

To use it, you will need omfToolkit.dll in your system path, which can be downloaded from OMF.zip at
http://www.aafassociation.org/download/ about 1/3 of the way down the page in the link for "OMF developer libraries."

Here are two 1:1 OMF 2.0 files that I was using for testing purposes. They were created with After FX v6.5 and just cycle through 10 different solid colors over and over. They are 1:1 OMF 2.0, PAL and NTSC. I have also successfully tested OMF 1.0 files captured by an Avid Adrenaline.
http://www.tateu.net/software/dl.php?f=OMFColorsNTSC
http://www.tateu.net/software/dl.php?f=OMFColorsPAL

And here is the current source and compiled dll in case anyone wants to take a look:
http://www.tateu.net/software/dl.php?f=OmfSource

The syntax is:
OMFInput("Colors_v2_NTSC_1-1.omf")

[iC3]

Hi Hanfrunz,

Did you ever get anywhere with using Blackmagic Quicktimes?
I have just been having the same problem and have foung using the quicktimereader.vfp allows Blamagick HD Qt's to be imported of any length and size.
Playback is a little slow though for my liking. Id like to import HD quicktimes,
post process them and read them out as DPX frames.

iC3

[Mug Funky]

iC3 - if you've got tons of disk space you can use ffmpeg to rip the data out of the mov and whack it into an avi file. not an ideal solution i know...

also you can do future caps on a PC if you're allowed to rip the decklink card out of the mac it's in.

good to see another melbournite. you're in good company

[Mug Funky]

yo, i noticed that "dirtycineon" doesn't appear to work. says "file not found", and quotes the correct filename.

if i change "dirtycineon" to "cineon", the problem goes away.

...basically i've a sudden selfish interest in getting a dpx sequence into an OMF as quickly as possible (preferably as soon as they come off the scanner, but realistically i'll probably have to work on the previous roll while the current one is scanning).

[hanfrunz]

@Mug Funky
i have to take a a deep look in the latest sourcecode maybe i fixed this problem If i remember right "dirtycineon" just read the rgb values directly from the cineon file. "Cineon" uses the xnview-library...

[squid_80]

Working from your v1.5 sourcecode there seemed to be an if statement missing that was causing the file not found error to always be thrown. Also I tweaked the shifting code a bit since it was doing a signed right shift instead of unsigned:

dstp[w+2] =  source[dummy];
dstp[w+1] = (source[dummy+1] << 2) + ((unsigned char)source[dummy+2] >> 6);
dstp[w  ] = (source[dummy+2] << 4) + ((unsigned char)source[dummy+3] >> 4);

The colours come out a lot better now

I'm putting some finishing touches on some SSE2 code to do the bit shifting, let me know if you want me to post it here.

[squid_80]

Well that was easier than I thought: http://members.optusnet.com.au/squid...ressedfile.dll

Someone speed test this for me, I can't discern any difference here since I've only got 5 dpx files to play with. (Source code is of course available upon request, I just don't have the room to host it at the moment.)

[Wilbert]

Someone speed test this for me, I can't discern any difference here since I've only got 5 dpx files to play with. (Source code is of course available upon request, I just don't have the room to host it at the moment.)

I will host it if you send it to me: wilbertdijkhof at yahoo.com

[Mug Funky]

thanks heaps for this squid!

speed for 77 dpx files (1k - 1024x768, 10-bit log) is about 35fps as read from my SATA drive. that's fantastic. in fact, running rgblut("10 x 256 / ^ 64 * 64 -") on it didn't incur any noticable speed penalty (that's a hacky log-to-lin conversion that i pulled out of my bottom).

machine is AMD X2 4200+ @ 2.2GHz with a pretty vanilla SATA drive.

for some reason the machine at work running off the SAN is a lot slower (it'd otherwise be a much better testing bed, with access to nearly limitless dpx files of different sizes). i think that might be machine specific, as copying a sequence locally still gave me a rather flaccid 3-4fps. i'll have to track down a beefier machine that's not being used...

to give an indication of what's possible with this format, the grading machine at work is a similar spec machine (intel's equivalent i guess) and it's able to play out 2k (2048x1556) with 32 bit processing, ~4 layers of 3d luts and spline windows at a steady 25fps . it uses GPU for a lot of that though.

[morsa]

what is that machine using? Some kind of "lustre" or what?

[Mug Funky]

@ morsa: close but no discreet. it's a newish one called "scratch".

[jmac698]

can you add a feature to return >8bit files as two clips, with high byte and low byte? then we could instantly have HDR video feature (high dynamic range).
masktools should be enough to manipulate it, though it will be awkward to handle a carry. An HDR brightness function would need three steps.
low=omfimport("low","hdr.omf")
high=omfimport("high","hdr.omf")
bright=+20
carry=yv12lut(low,"x bright + 255 > 1 0 ?")
low1=yv12lut(low,"x bright +")
high1=yv12lutxy(high,carry,"x y +")

Well I won't write it now, but I think it's possible.

[Mug Funky]

that might be more a feature for avisynth 3... though it's not a bad idea to do a workaround in 8 bit, it could get unwieldy.

[jmac698]

hanfrunz,
this may help
http://aaf.sourceforge.net/
open media framework interchange, invented by Avid in 1998, finally gained accepted on a wider basis with microsoft, who then developed an SDK, with most of the code from the old Avid SDK. It's freely distributable including derived works.
It can import older file formats.
And could you make another dirtyimportlow that has the lower bits - if 10bit format, put them in the top two bits please.
I can write log converter just in script!

[Mug Funky]

@ jmac698:

you wont get much extra in those 2 bits, considering film grain works as a dither here (unless you're working with 70mm or Imax in 2k...). though it would be awesome to be able to do grading and compositing in avisynth

[Cooger]

I have my post here:
http://forum.doom9.org/showthread.php?t=137442

And I was refered by Wilbert to this post.

This is a very needed plugin, in my opinion. I am personally interested in feeding 10bit log DPX into avisyth and encode to bluray via meGUI.
(more of my problems in the post above)

I may be wrong here, but as I think, that Bluray encoding is 8 bit, the plugin could apply a rec709 LUT on the DPX.

That way, I would be able to have a quick- dailies renderout on bluray.
If this plugin is able to do that... OMG !!!

Is there a way I can contribute? I can generate various tiff, dpx and quicktime uncompressed sequences from the RED ONE footage, via the RED Alert program. Since its my own footage, and I would limit the lenght, there would be no royalty problems.

Jonas

www.jonasrejman.com/carved

[Mug Funky]

wow! who's asking for dailies on blu-ray? that's a bit extravagent isn't it?

(sry, i only say that because all i've got for dailies is an antique Ursa Gold w/ twigi. SD only)

btw, Cooger: Scratch is awesome with red stuff. 2k -> HDCAM-SR with grades... realtime.

[Cooger]

Hi there, again.

Well, clients are asking for that. The clear advantage is, that they can play it back on their home theatre systems.

SCRATCH or Speedgrade RX are grading tools. They do not help you with encoding and authoring to Bluray. Again, if you are able to afford SCRATCH with SDI and HDCAM deck, and are able to get the client another deck to playback, you are off the hook.

I was looking for an alternative without tapes. I hate tapes. ;-D

Jonas

www.jonasrejman.com