Any 16 bpp color support in Avisynth yet?

[trbarry]

Hi -

I've been away from here for awhile but when I left there was some talk about supporting say, 48 or 64 bit RGB, or other 16 bit per pixel color spaces.

Did that ever happen? Or is it planned?

The reason I ask is because I just made a modified version of RawSource that can read raw SGI format files with 16 bpp RGB. I needed this for something else but also added an option to output 16 bpp huge frames simply by lying about the frame width.

But if there is some official way to support this I'd do it. Is any such thing even planned? And if so, would the 16 bpp valules be in Intel little-endian format or bigendian?

Before I send RawSource back to the author I thought I should ask in case there is already a standard.

- Tom

[IanB]

The most that's happened is I have added some notes to my 2.6 to do list. But lack of free time always conspires.

I had sort of assumed internally the data format would be dictated by what was best in MMX/SSE i.e. 16 bit signed int using only the 0 to 32k range for PC levels and approx 2000 to 30000 for TV levels.

I am hopeing to at least add some infrastructure hooks in the 2.6 API change so when/if we get around to actually writing some 15 bit code the API won't have to be changed yet again. In theory the 2.6 engine should be able to route 15 bit channel data thru all the zero cost calls, like splice, trim and crop, and all the pixel processing calls throw a generic what's this pixel format exception.

The notes I have made are :- Reserve a pixel_type bit for 8/15 bit data, currently 0. Add a <Bytes/Bits>PerPixel<Channel or some better name> method that currently returns <1/8> and in the future return <2/15>. Define example RGB45 and YUV45 templates. Make sure all code correctly handles unknown pixel_type values.

In terms of pixel_type I am going to redefine the bits so for new the planar formats it describes the chroma subsampling, rather than the single bit per colour format we have in the interem.

Thorts?

[trbarry]

Ian -

I agree about the Intel little-endian byte order for convenient MMX/SSE coding. But I'm not sure why you would want only 15 bits instead considering them 16 bit unsigned shorts. That's the SGI format the SVT samples are (except byte order).

I wrote the recent RawSource changes in MMX using this format and didn't encounter any limitations except for the signed pcmpgtw command. I also wrote a matching Dither16to8Bit() filter as an experiment and for my own use in SSE2 and had no problems. It seems anyone needing unsigned 15 bit could use a single shift instruction and anyone wanting signed arithmatic could use a single psubw instruction first to convert to full 16 bit signed. (or just flip the hi bit)

But maybe I'm missing something here. I've left the 16 bit Rawsource and Dither filters as 16 for the moment but please point it out if there's something else I should be considering.

- Tom

[IanB]

I think it was Klaus or Richard who made the original 15 bit suggestion. My memory of it is mainly because of the occasional holes in the MMX instruction sets. pcmpgtw, packssdw and packuswb are 3 examples of missing unsigned int versions. Also pmulhuw is only available in iSSE and up. At this point I don't think it matters because I have no intentions beyond place holder hooks.

So how we spend our 16 bits can wait until code is being drafted. So far all the 16 bit MMX/SSE code I have ever done is signed for audio, until we start writing 16 bit unsigned in earnest I guess we won't find the pitfalls.

Is there a genuine need for full 16 bit precision over 15 bit? Most of the time I envisage 10 bit precision. Even with un-gamma-ing you only need 12 bit intermediates, so 15 bits is still 3 bits of luxury.

I also vaguely remember a case for only using 14 bits [0..16384] and allowing for +400% -0% or +200% -200% intermediate headroom.

[trbarry]

Not really a genuine need...

But I've written a lot of 8 bit filters, finally had some real 4k 16 bit source, and didn't want to leave anything lying on the table. Plus I was reading about dithering and did not want to start by truncating something. ;)

For my own purposes I'll leave it 16 bit for now. If 16 bit AVS ever becomes real it will be easy enough to change then to whatever becomes the standard.

- Tom

PS - what I really want is internal 32 bit float values. ;)

[Richard Berg]

If we design around a CPU architecture, it should probably be SSE2. Should be as common as iSSE by the time we actually code & ship anything ;) Plus it will make it easier to port old 8-bit MMX algorithms to 16-bit since you can fit the same # of pixels per register.

I thought it was interesting that Avery used floating point for his new filter SDK.

[IanB]

Notes to self :- Reserve 2 pixel_type bits for 8/15/32/resvd bit data, ...

[Prettz]

I thought it was interesting that Avery used floating point for his new filter SDK.
So is he really not concerned with filter performance at all?

[trbarry]

So is he really not concerned with filter performance at all?

Machines have gotten a lot faster since Avisynth was first written. Meanwhile, large amounts of storage to save high quality video has become effectively free and 1080p displays are becoming standard.

So sometimes it becomes easier to decide to go slower and get a higher quality image saved in a larger format.

Q: Why do dogs and cats lick their genitals?

A: Because they can. ;)

- Tom

[IanB]

Check the performance of the SSE packed single precision floating point instructions on current cpu cores. Very impressive! Also many algorithms when implemented in float lose a lot of "frilly bits" boundary code. On a P4, ResampleAudio with Floats is about the same speed as with Int16 audio data.

[trbarry]

Also, remember these days optimized assembler MMX/SSEx code spends much of its time just fetching stuff from memory and not actually doing any arithmetic. Granted, 32 bit float does fetch 4 times as much stuff from memory as 8 bit whatever format. But current machines are way more than 4 times faster than in the early days of Avisynth.

- Tom

[a uio]

Highest quality image processing with high dynamic range is often done in no less than 32-bit float these days, for video games, hollywood level movie processing, cgi, etc. Consumer level apps like Photoshop & Premiere both support 16-bit int & 32-bit float now.

I've written lots of 32-bit float code for filters to apply to my own films. Given the amount of time spent on a film overall, the filtering speed is not anywhere near as high on my list as quality is.

But in general, why wouldn't AviSynth have a goal to support 8 bit uint, 10bit uint, 16 bit uint and 32 bit float? And let the user choose speed over quality, thus being a great tool for the home video enthusiast and the professional film maker, as well as being exactly inline with the paths other tools are taking.

Cheers,
A

[smok3]

well, count me in, i would simply like to see something like:

imagereader (hdr sequence)
do some gamma, chroma, levels, brightnes, contrast
return 8bit()

[Leak]

But in general, why wouldn't AviSynth have a goal to support 8 bit uint, 10bit uint, 16 bit uint and 32 bit float?
Errr... 8, 16 and 32 bit I understand, but 10 bit? That's such an odd size for processing that it's guaranteed to be slower than 16 bit...

[squid_80]

Errr... 8, 16 and 32 bit I understand, but 10 bit? That's such an odd size for processing that it's guaranteed to be slower than 16 bit...

Yeah, there's a few capture cards/devices/codecs that use it though. See my comment here (http://forum.doom9.org/showthread.php?p=908373#post908373) (and IanB's follow-up).

[Mug Funky]

10 bit is quite common actually :)

digibeta (and anything carried through SDI cables) is 10-bit, as is HDCAM/SR, and the most commonly used flavour of cineon/dpx.

it's just a pain that it's not a multiple of 8...

[Leak]

it's just a pain that it's not a multiple of 8...
Errr... that's nothing a promotion from 10 to 16 bits can't fix. Just shift the 10 input bits 6 bits to the left and copy the top 6 bits into the zeroed lowest ones - that should make any processing afterwards much more pain-free, and it will map all values from 0 to 2^10-1 evenly onto the values from 0 to 2^16-1, as opposed to just shifting the input.

[a uio]

Mpeg defines 10-bit profile levels for high quality "professional" profiles. 10bits is sort of the minimum you need to not destroy your images through quantization round-off error accumulation in filter chains.

I think somes reasons that some people like 10-bit are that 3 channels fit in a 32bit int (so you can store and pull pixels from memory with fewer reads than 16bit) and 10bit lookup tables are still quite reasonably sized. But obviously if you have a 16bit number, you can round it to 10 bits for table lookups if you want.

Personally, all I would really care about are 16bit int and 32 bit float. But people often bring up the memory size/bandwidth issue...

Cheers,
Landrew

[freezer]

I second the call for higher bit depth support. It would be really usefull to improve the quality if time is not important.
It would be nice to see support for 10 bit log files, too.

[Mug Funky]

@ freezer: careful, Apple might buy up avisynth and we'll never see it again :). but it would be the height of awesome to see a free way to do DI grading, even if it would only be useful to very very few people (srsly, if you can get 10-bit log files then you at the very least have a stupidly expensive film scanner at your disposal).

that said, i'm currently trying to sort out a realistic way to use mvtools denoising on digital intermediates. 8 bit is a bummer, but for something as grainy as kodak 500T it'll do just fine.

for now, importuncompressedsequence(mode="dirtycineon") combined with some fancy RGBlutting will give you the best you can hope to get out of a scripting language (soft spline grading windows and 3d LUTs would be cool :))

[Didée]

3d LUTs would be cool :))
In v0.32, Manao has implemented mt_lutxyz. :)

Use sparsely, one instance of the filter takes up 3 * 256^3 -> 48MB RAM for the LUT.

[Mug Funky]

mt_lutxyz, huh?

i might have to take a look at it. certainly would be quite helpful for those selecting-the-best-from-3-clips operations.

but by 3D LUT i meant more along the lines of a "colour cube", sort of a freeform deformation of the entire colour space (imagine a cube/cylinder with X and Y being the same as X and Y on a vectorscope, and Z being luma). this is sort of the ultimate in getting a "film look", as you can make physical measurements and apply them as a 3D lut. you can also do things like select reds and make them blue by just smooshing the right part of the cube.

[Didée]

That's quite possible now, isn't it? Split the three channels of your clip into three clips' luma channel, apply lutxyz with an appropriate expression, then re-merge. The fun is building the expression. (Hint: Avisynth handles strings up to a length of 4096 chars, afaik...) :D

[Mug Funky]

ah, i see.

very ouch though. generally 3d luts are defined as 16x16x16 cubes, and interpolation is done during processing.

however, specific effects can still be done with lutxyz, so i might have a play when i've got some free time.

[foxyshadis]

Next step, then, is to create an ICM to lut converter. =p (Sounds painful.)

[signal]

Please pardon my naivety on this, but from what I'm reading here if I'm capturing 10bit uncompressed (via a Blackmagic Decklink HD Extreme) then I can expect some problems in AVIsynth?

[IanB]

@signal, Most of the AVI world is 8bit, currently Avisynth is.

10Bit YUV formats are used to accuratly represent 8bit RGB, but can obviously hold even greater precision for your data.

[jmac698]

Yes, the use of 10bit is for accurate conversion to another color format. I'd like to see 16bit support personally, for the quality and scientific uses. I've been waiting for this import filter, as I'm willing to write a set of functions/filters to do basic operations on 16bit data. I've looked up the Cineon formula and the LUT function of MTTools could be used almost immediately.
Just expose the data as two videos, high and low bits, I can work with it.

[Mug Funky]

just for fun i've decided to condense a few operations down to 1 mt_lutxyz:

it's messy as hell, but it does:
- simple offset/gain
- log-to-lin conversion
- convert RGB to rec.601 yv12

using 1 lut and some workarounds to keep my head from a splode

sequence = importuncompressedsequence("%06d.dpx","dirtycineon",183622,183699, 1024,768,25,8192,0 )

setmtmode(1)

offset=-10
gain=1.15

roff=offset+2
rgain=gain*1.0

goff=offset-10
ggain=gain*1.0

boff=offset-5
bgain=gain*1.0

rprimary="x "+string(rgain)+" * "+string(roff)+" +"
gprimary="y "+string(ggain)+" * "+string(goff)+" +"
bprimary="z "+string(bgain)+" * "+string(boff)+" +"


rexpr="10 "+rprimary+" 256 / ^ 64 * 64 -"
gexpr="10 "+gprimary+" 256 / ^ 64 * 64 -"
bexpr="10 "+bprimary+" 256 / ^ 64 * 64 -"

sequence

r=showred().converttoyv12()
g=showgreen().converttoyv12()
b=showblue().converttoyv12()

y=mt_lutxyz(r,g,b,yexpr="16 "+rexpr+" .2568 * "+gexpr+" .5041 * + "+bexpr+" .0979 * + +",y=3,u=0,v=0)
u=mt_lutxyz(r,g,b,yexpr="128 "+rexpr+" -.1482 * "+gexpr+" -.291 * + "+bexpr+" .4392 * + +",y=3,u=0,v=0).lanczosresize(width/2,height/2)
v=mt_lutxyz(r,g,b,yexpr="128 "+rexpr+" .4392 * "+gexpr+" -.3678 * + "+bexpr+" -.0714 * + +",y=3,u=0,v=0).lanczosresize(width/2,height/2)

mergechroma(y,ytouv(u,v))


speed is quite fast, but init time takes forever.

in theory you could do some quite sophisticated grading functions within a single lut call (well, 3 for now unless it gets modified for rgb space). you could do lift, gain, gamma, desat white, desat black, vector-selector (ie grab the reds and do HSV ops on only them), and probably some other stuff. i'm thinking of an interface with avsp, but i can see problems with having to rebuild the lut on every slider change...

[jollye]

Hi all,

If I'm not wrong there is a planned support for Y16 and RGB48 in AviSynth 2.6. Do you have any idea about when AviSynth 2.6 could be available?

Thanks for your answer(s).

[anton_foy]

@jollye

Well I want it NOW!!! But as far as I know there probably will take less time for Avisynth 2.6 than 3.0 if thats any help :P

Just started using Ubuntu...feeling the power searing through my fingers so I am gonna pick up programming and help AVS3.0...so if you need my programming abilities, contact me ten years from now :p

[Wilbert]

Well I want it NOW!!!
Well then, start coding.

v2.60 will have 16 bit support, but probably none of the filters will support it then. I hope IanB will add it to the resizers and color conversions, and then I will add it to other filters with some help. My idea is to add it to filters like RGBAdjust, ColorYUV, Levels and Tweak first. I don't think you will see any of this before v2.61, unless other people help implementing it.

[IanB]

Sorry folks, 2.6 will NOT have 16 bit support.

All it will have is the API infrastructure to allow 16bit support to be added without breaking the API and needing a recompile of all the filters in the future.

[Fizick]

So, it is planned for 2.7.x ?

[jollye]

Sorry folks, 2.6 will NOT have 16 bit support.

All it will have is the API infrastructure to allow 16bit support to be added without breaking the API and needing a recompile of all the filters in the future.

If I understand well this means that internal filters will not be ported but that we will be able to implement external filters using those color spaces. This also means that the methods used (inside filter's code) to determine what the color space of the clip is, get a write or read pointer... will be available. If this is what you mean I think it's a very good start. Do you confirm?

Also do you have any idea of when 2.6 could be available? (end of 2008, middle of 2009, some other date?).

Thanks for your reply