New Plugin - Sashimi (raw file access)

[PitifulInsect]

Hey there Doom9erinos!

History:
Many an age ago, I wrote my own raw reading and writing plugins for AVISynth because RawReader didn't do quite what I wanted.
Later, I needed to use raw reading of some files in a corporate environment, and software without clear licensing was banned from even being downloaded so I couldn't use RawReader! I dusted off my old plugins, made the first version of Sashimi, and made it GPLv2 (please note before commenting, that many business applications still have a big issue with GPLv3 because of the NAP clause, so I deliberately used v2 and for the next while at least I think that you should too if you'd like your software to be widely used and you don't plan to sue anyone).
Finally and more recently, I cleaned it up a lot and added some features, so now I'm putting it out there.

Actual Relevant Stuff:
In combination with the included .avsi scripts, it can read and write a whole host of raw formats, and I've used it myself a lot so it's relatively well tested but I'm sure that some major issues will be found as soon as others start using it too.
Let me know what you think! I'm keen to get feedback, and I plan to fix bugs if any are found, but I'm one of those people who sometimes takes ages to respond, so advance apologies if I'm slow answering any direct questions.

See here for the latest version on my site:
https://sites.google.com/site/ourenthusiasmsasham/soft (https://sites.google.com/site/ourenthusiasmsasham/soft#TOC-Sashimi)

But for simplicity, the plugin and source are also attached to this post.

UPDATE: I've fixed bugs and extended the abilities since the original post. Even though discussion and development continues, these attached files are up to date, get sashimi here.
Previous versions are available at my site and linked from within the documentation. See below for comments, and here for the files:

[b66pak]

thanks a lot...
_

[IanB]

...
But for simplicity, the plugin and source are also attached to this post:
Attached Files
File Type: zip sashimi_073.zip (http://forum.doom9.org/attachment.php?attachmentid=11104&d=1275124835) (146.9 KB, 20 views)
File Type: zip sashimi_073_src.zip (http://forum.doom9.org/attachment.php?attachmentid=11105&d=1275124847) (122.9 KB, 6 views)Seems the attachments are barseackwards :o

The 147KB file is the source, the 123KB file is the release.

[foxyshadis]

I like it, I could have used it last week! But it will be quick to switch to it from ImmaWrite/RawReader.

[PitifulInsect]

Cheers IanB, they were backwards, but I've changed them around to be the right way around while updating the version to 0.74.

Minor bugfix aside, what I did was add a massive pile of documentation and a heap of runnable examples. I'm going to brag here about what the plugin does in those examples, so that this thread can be found by anyone doing a text search for how to do those things:

Read and write raw files as GREY, Y8, and (interleaved) BGR, RGB, BGRA, ARBG, ABGR, and RGBA
Read and write raw planar files as RGB24, RGB32, YUV444, YUV422, YUV420-as-YV12, and YUV420-as-IMC2
Read and write interleaved YUV files as YUV, YUY2, UYVY, and AYUV
Read uncompressed YUV (YCbCr) format files created in ImageMagick without the need for a colourspace conversion
Has documentation on how to pass YUV (YCbCr) images into ImageMagick out of AVISynth without the need for a colourspace conversion
Read and write other raw exotic formats
Read and write all of the above as either single video files or as collections of raw images per-frame
and lots more! (well, I already mentioned the "exotic" thing, so this might be a small lie).


Cheers for the comments so far!

[Chikuzen]

＠PitifulInsect
Thanks!

btw,i have a question.
are you scheduled corresponding to reading yuv4mpeg2?

[18fps]

A question: could it be used to read an "exotic" RGB48 file?

[foxyshadis]

Just note that if you need to process YUV files in imagemagick, the 16-bit precision (Q16) version often crashes reading and writing them.

One minor bug I found: RawWriter(...,overwrite=true) will only open the file, write into it, and close, but it will not truncate it first. So you get a bunch of garbage at the end if the original file was longer than the new one.

[PitifulInsect]

are you scheduled corresponding to reading yuv4mpeg2?

I had to look it up - I hadn't come across it before.

Sorry Chikuzen, I wasn't planning to, but it should work just fine if you read the human-readable (as I understand it) headers and set the right offset, which would probably be good enough for a once-off solution. I would think that a half hour of hacking would make a Python script to parse the YUV4MPEG2 headers and write an appropriate AVISynth script (that sets the offset and dimensions, then calls the appropriate Sashimi function).

If you (or anyone) should write such a Python script, post it here and I'll include it in the next Sashimi release. If there are any YUV4MPEG2 pixel formats that Sashimi doesn't support (but AVISynth could), then please let me know. I won't immediately offer to do it myself, since I'm unfamiliar with the format or its usage, but I might wander back to this sometime - I'm intrigued!

I'm wondering if the headers could somehow be parsed directly from within a tricky AVISynth script without invoking anything else...

[PitifulInsect]

A question: could it be used to read an "exotic" RGB48 file?

16 bits per channel? Yes, but then what to do in AVISynth which only supports 8-bit formats?

Off the top of my head without testing (someone please test, debug, and re-post if you've used it - At the minimum I'll include it in the next release):
#
# Pass your double-frame-width frames through this sucker
# If you only want the low or high bits, then just use the appropriate
# first lines from this function, otherwise set bitshift between 1 and 7
#
function 16bit_to_8bit(clip c, int bitshift)
{
hibits = TurnLeft().SeparateFields().SelectEven().TurnRight()
lobits = TurnLeft().SeparateFields().SelectOdd().TurnRight()

hibits = hibits.Levels(0,1,255, 32*bitshift,255, false)
lobits = lobits.Levels(0,1,255, 0,32*bitshift-1, false)
return hibits.Overlay(lobits, mode="Add")
}

You lose precision by going via AVISynth, but at least you're able to process the images at all.

Thoughts or suggestions from someone with 16-bit experience? Is a bit-offset between 0 and 8 a sensible approach? Should it be a scaling factor between 1 and 255? Should it be a range (between 1 and 65535) to scale to 255? Is there some other common way to deal with this?

16-bit data is probably a common enough use-case scenario to warrant direct support. I'm tempted to throw in the towel on 12-bit data though - what a nuisance! :eek:

[PitifulInsect]

Just note that if you need to process YUV files in imagemagick, the 16-bit precision (Q16) version often crashes reading and writing them.
You mean ImageMagick's problem, right? Sadly, though I love what ImageMagick is and does, I fully agree that it's a temperamental creature and the documentation can be a bit of a maze.

If you spot a Sashimi problem though, let me know and I'll jump all over it.


One minor bug I found: RawWriter(...,overwrite=true) will only open the file, write into it, and close, but it will not truncate it first. So you get a bunch of garbage at the end if the original file was longer than the new one.

Hmmm - that's kinda deliberate, but I guess that it's non-obvious behaviour - my bad. The exotic file-writing example uses the fact that you can write a file in multiple passes, and I also imagined that someone might want to write headers to files then add content later. It wasn't adequately documented though, and I suppose that I should also add a "truncate" parameter and set it to true by default, but be able to set it to false for the examples that I just mentioned. Sound good? It makes the default behaviour as expected but retains the ability to be fancy.

Cheers for the comments!

[cretindesalpes]

16 bits per channel? Yes, but then what to do in AVISynth which only supports 8-bit formats?

I recently wrote a dithering script (http://forum.doom9.org/showthread.php?p=1386559#post1386559) (I'm currently converting it to a plug-in for faster processing) and modified several denoising plug-ins in order to extract the extra bits from already dithered data. You could use a compatible method : either stack vertically the MSB part and the LSB part, or interleave them.

[kemuri-_9]

I had to look it up - I hadn't come across it before.

Sorry Chikuzen, I wasn't planning to, but it should work just fine if you read the human-readable (as I understand it) headers and set the right offset, which would probably be good enough for a once-off solution. I would think that a half hour of hacking would make a Python script to parse the YUV4MPEG2 headers and write an appropriate AVISynth script (that sets the offset and dimensions, then calls the appropriate Sashimi function).

If you (or anyone) should write such a Python script, post it here and I'll include it in the next Sashimi release. If there are any YUV4MPEG2 pixel formats that Sashimi doesn't support (but AVISynth could), then please let me know. I won't immediately offer to do it myself, since I'm unfamiliar with the format or its usage, but I might wander back to this sometime - I'm intrigued!

I'm wondering if the headers could somehow be parsed directly from within a tricky AVISynth script without invoking anything else...

http://linux.die.net/man/5/yuv4mpeg is a fairly good description of the format.
the format has a file header in addition to frame headers.
so in addition to the initial offset for the file header, you'll need to offset more at the start of each of frame for the frame header.

for avisynth 2.5.x you'd only need need to worry about 4:2:0.
for 2.6.x you'd need to also support 4:1:1, 4:2:2, and 4:4:4.

you could also refer to x264 and/or ffmpeg as they have some specific demuxer/muxer related code for yuv4mpeg.

[Wilbert]

yuv4mpeg writing is supported in ImmaWrite too. Its format:


line terminator \n = 0x0A (in C/C++)

yuv4mpeg.h in mjpegtools_1.9.0rc3.zip

/************************************************************************
************************************************************************

Description of the (new!, forever?) YUV4MPEG2 stream format:

STREAM consists of
o one '\n' terminated STREAM-HEADER
o unlimited number of FRAMEs

FRAME consists of
o one '\n' terminated FRAME-HEADER
o "length" octets of planar YCrCb 4:2:0 image data
(if frame is interlaced, then the two fields are interleaved)


STREAM-HEADER consists of
o string "YUV4MPEG2"
o unlimited number TAGGED-FIELDs, each preceded by ' ' separator
o '\n' line terminator

FRAME-HEADER consists of
o string "FRAME"
o unlimited number of TAGGED-FIELDs, each preceded by ' ' separator
o '\n' line terminator


TAGGED-FIELD consists of
o single ascii character tag
o VALUE (which does not contain whitespace)

VALUE consists of
o integer (base 10 ascii representation)
or o RATIO
or o single ascii character
or o non-whitespace ascii string

RATIO consists of
o numerator (integer)
o ':' (a colon)
o denominator (integer)


The currently supported tags for the STREAM-HEADER:
W - [integer] frame width, pixels, should be > 0
H - [integer] frame height, pixels, should be > 0
C - [string] chroma-subsampling/data format
420jpeg (default)
420mpeg2
420paldv
411
422
444 - non-subsampled Y'CbCr
444alpha - Y'CbCr with alpha channel (with Y' black/white point)
mono - Y' plane only
I - [char] interlacing: p - progressive (none)
t - top-field-first
b - bottom-field-first
m - mixed -- see 'I' tag in frame header
? - unknown
F - [ratio] frame-rate, 0:0 == unknown
A - [ratio] sample (pixel) aspect ratio, 0:0 == unknown
X - [character string] 'metadata' (unparsed, but passed around)

The currently supported tags for the FRAME-HEADER:
Ixyz - framing/sampling (required if-and-only-if stream is "Im")
x: t - top-field-first
T - top-field-first and repeat
b - bottom-field-first
B - bottom-field-first and repeat
1 - single progressive frame
2 - double progressive frame (repeat)
3 - triple progressive frame (repeat twice)

y: p - progressive: fields sampled at same time
i - interlaced: fields sampled at different times

z: p - progressive: subsampling over whole frame
i - interlaced: each field subsampled independently
? - unknown (allowed only for non-4:2:0 subsampling)

X - character string 'metadata' (unparsed, but passed around)

************************************************************************
************************************************************************/

[foxyshadis]

Hmmm - that's kinda deliberate, but I guess that it's non-obvious behaviour - my bad. The exotic file-writing example uses the fact that you can write a file in multiple passes, and I also imagined that someone might want to write headers to files then add content later. It wasn't adequately documented though, and I suppose that I should also add a "truncate" parameter and set it to true by default, but be able to set it to false for the examples that I just mentioned. Sound good? It makes the default behaviour as expected but retains the ability to be fancy.

Cheers for the comments!

Sounds good to me. I had no idea you could even do that, didn't look deep enough into the docs. Cool!

I recently wrote a dithering script (http://forum.doom9.org/showthread.php?p=1386559#post1386559) (I'm currently converting it to a plug-in for faster processing) and modified several denoising plug-ins in order to extract the extra bits from already dithered data. You could use a compatible method : either stack vertically the MSB part and the LSB part, or interleave them.

Does dithering alone actually work? Most of the time 12+ bit formats require gamma correction before truncation/dither to 8/10 bit. (I've seen 10 bit go either way.) I guess as long as you aren't too heavy on denoising and don't try to convert to/from rgb, it'll work. On the other hand, resizing should be more accurate.

PitifulInsect: Avisynth 2.6 is supposed to get full support for a 16-bit format, but right now it's rudimentary and unusable. I wish I was good enough at optimization to help with that.

[Chikuzen]

If you (or anyone) should write such a Python script, post it here and I'll include it in the next Sashimi release.


#!/bin/env python
import sys
colorspace = None
plugin = r'c:\avisynth 2.5\plugins_32bit\'
print 'input y4m file path'
y4mpath = raw_input()

f=open(y4mpath,'r')
line1 = f.readline()
line2 = f.readline()
f.close()

header=line1.strip('\n').split(' ')
if header[0] != 'YUV4MPEG2':
print 'This file is not yuv4mpeg2. Please confirm and try again.'
sys.exit()

for param in header:
if param[0] == 'C':
colorspace = 'yuv' + param[1:4]
if param[0] == 'W':
width = param[1:]
if param[0] == 'H':
height = param[1:]
if param[0] == 'F':
fps = param.strip("F").split(":")
fps_num = fps[0]
fps_den = fps[1]

if colorspace == None:
colorspace = 'yuv420'

filepath = y4mpath.replace('/','\\')
filehead = str(len(line1))
framehead = str(len(line2))

loadPlugin = 'LoadPlugin(%(plugin)s+"Sashimi.dll")\n' % vars()
importavs = 'Import(%(plugin)s+"PlanarConversions.avsi")\n' % vars()
y4mreader = 'RawReadPlanar("%(filepath)s", "%(colorspace)s", %(width)s, %(height)s, %(filehead)s, %(framehead)s)\n' % vars()
assumefps = 'AssumeFPS(%(fps_num)s, %(fps_den)s)\n' % vars()

avs = open('./y4msource.avs','w')
for makeavs in [loadPlugin, importavs, y4mreader, assumefps]:
avs.write(makeavs)
avs.close()


This is python script that I wrote for the first time.
Please teach me when a mistake is found.

[jmac698]

Pitiful,
Could you update to include v210 format? I've started here:
UPDATED TO V0.5

#v210 Reader Ver 0.5 Feb/11 by jmac698
#Read v210 quicktime files with rawreader, by jmac698
#Requires Sashimi 0.75 http://sites.google.com/site/ourenthusiasmsasham/contributions/open-source-software#TOC-Sashimi
#also http://forum.doom9.org/showthread.php?p=1403600
#Masktools 2a48 http://manao4.free.fr/masktools-v2.0a48.zip
#In this experiment version, the QTSource is not used...
#QTSource http://www.tateu.net/software/ tested wtih Version 0.1.1 2011-02-19
#Quicktime 7: http://www.free-codecs.com/download/QT_Lite.htm qtlite 4.1 with quicktime 7.69.80.9
#FFMpegSource: http://code.google.com/p/ffmpegsource/downloads/list 2.15
#Ver 0.5 - reads all test videos in .mov given the right headersize, fixed pixel order
#Ver 0.4 - reads full image in 8bit greyscale, with 10bit easily accessible
#Ver 0.3 - reads fully to 8bit greyscale, now requires masktools 2a48
#Ver 0.2 - fixed wrapping bug. 83 pixels *8=664, that would be the trailing bytes
#Ver 0.1 - reads only greyscale, can only read 1/3 of the size, only reads upper 6 bits, wrapped by 83 pixels
#Errors and what to do
#Message: Unable to Initialize Quicktime Environment
# to fix: install quicktime components

Global deeplasthi=blankclip(pixel_type="YV12").tweak(bright=-255,coring=false).tweak(bright=16,coring=false)
Global deeplastlo=blankclip(pixel_type="YV12").tweak(bright=-255,coring=false).tweak(bright=0,coring=false)
Global format=0#0=YV12, 1=YUY2, 2=RGB

fn="E:\project001a\deepcolor\v210 samples\"
f1="radialramp_480p29.97_16bitRGB_v210.mov"#hdr=48, 1 frame
f2="radialramp_576p25_16bitRGB_v210.mov"#hdr=48, 1 frame
f3="radialramp_720p23.976_16bitRGB_v210.mov"#hdr=48, 1 frame
f4="radialramp_1080p29.97_16bitRGB_v210.mov"#hdr=48, 1 frame
f5="testchart720p29.97_v210.mov"#hdr=48, 2 frame
f6="ArriAlexa.PR422.BT709.v210.mov"#hdr=1896, 121 frames
f7="bm444-480p29.97_r210.avi"#can't be loaded by ffvideosource, but should be able to be read raw
f8="v210.mov"#parkjoy standard test video, hdr=20528, 3 frames
fn=fn+f5
#deepread(fn)#f3, f5, f7 don't work i.e. QTSource has compatibility problems
#qtinput(fn,color=2,audio=2,mode=4)#mode 0,3 opens all but no video, mode4 couldn't find vfw codec but when enabled access violation
readv210(fn)
#ffvideosource(fn)#f7 don't work


function every(clip v, int n, int offset) {#select every n bytes horizontally with offset, works on yv12 only
v
w=width
h=height
pointresize(v,w*2,h)
crop(offset*2,0,0,0).addborders(0,0,offset*2,0)#shift left offset pixels
pointresize(w/n,h)
}

function readv210(string fn) {#reads v210 file of a given dimensions, but need to provide a header size
#How to find the right header size: try starting at 48, and then count how many lines of garbage are at the top.
# then multiple rows of garbage by rowsize as calculated below (1920 for 480p/576p files, 5120 for 1080p files), and add this to headersize
# finally count the number of garbage pixels or shift at the left, divide by 3 and times 4, add this to headersize
# i.e. headersize=0. There were 36 pixels of garbage at the top left, so headersize=36/3*4=48
# you can also increment by byte until you see picture, after that you can increment by 4 at a time to shift left by 3 pixels
video=ffvideosource(fn)#opens the video however only as 8 bit, so we'll use the proper video properties from this to read raw
w=video.width
h=video.height
f=video.framecount
multiple48=w%48==0#is width a multiple of 48? If false, there's some padding at the end of each row
rowsize=ceil(w*8/3/128.0)*128#rows are padded to the nearest 48 pixels which is 128 bytes in v210 format
hdr=w==1920?48:w==1280?48:w==720?48:48#These only work for some test files and can be considered the minimum header
base=RawReader(fn, format="y8", width=rowsize, height=h, numframes=f, filehead=hdr, framehead=0)
#a 1080p v210 (f8) file needed 5120*1080=5529600 bytes per frame, 3 frames, and header size=20528, trailer=664. Filesize was 16609992
#a 1080p v210 (f4) file needed 5120*1080=5529600 bytes per frame, 1 frame, and header size=48, trailer=15880. Filesize was 5545528
#a 720p v210 (f3) file needed 3456*720=2488320 bytes per frame, 1 frame, and header size=48, trailer=16762 Filesize was 2505130
#a 480p v210 (f1) file needed 1920*480=921600 bytes per frame, 1 frame, and header size=48, trailer=15880 Filesize was 937528
#byte3 byte2 byte1 byte0
#xx987654 32109876 54321098 76543210
# Cr0 Y0 Cb0
#byte3 byte2 byte1 byte0
#xx987654 32109876 54321098 76543210
# Y2 Cb1 Y1
high4=base.every(4,2).every(2,0)#byte2, first row in diagram above or 3rd byte every 8 (so every(8,2) would work too), Y0 high 4 bits (in low 4 bits)
low6=base.every(4,1).every(2,0)
mt_lutxy(low6,high4,yexpr="y 15 &u 4 << x 4 >> +")#x 4 >> y 4 << +
y0=last
high2=base.every(4,1).every(2,1)
low8=base.every(4,0).every(2,1)
mt_lutxy(low8,high2,yexpr="y 3 &u 6 << x 2 >> +")#x 4 >> y 4 << +
y1=last
high6=base.every(4,3).every(2,1)#The 3rd byte in every 32bit word, and the 2nd word every 2 words, which is Y2 and Y5 pixels
low4=base.every(4,2).every(2,1)
mt_lutxy(low4,high6,yexpr="x 6 >> y 2 << +")
y2=last
weave3h(y0,y1,y2)
crop(0,0,-(last.width-w),0)#dimensions that are padded need to get cropped at the right, e.g. width=1280 is read at 1296 then cropped
}

function weave3h(clip a, clip b, clip c) {#horizontally weave 3 clips
a=a.turnright
b=b.turnright
c=c.turnright
interleave(a,b,a,c)#0 1 2 3->01 23->0213->213 or abc
assumefieldbased
assumetff
weave
assumefieldbased
assumetff
weave
pointresize(width,height*3/4)#deletes every 4th line offset 0
turnleft
}

function getlowbyte(clip vround, clip vtrunc) {#vround may have some pixels +1 compared to vtrunc; put the difference in the high bit
Overlay(vround,vtrunc, mode="Subtract", pc_range=true)#e.g. y:129-128=1, 128-128=0. u,v are 128 or 129
coloryuv(gain_y=127*256,off_u=-128,off_v=-128)
coloryuv(gain_u=127*256,gain_v=127*256)
}

function deepread(string fn) {
global deeplasthi=qtinput(fn,color=2,audio=2,mode=1,raw="v210")
global format=IsYUY2(deeplasthi)?1:0
qtinput(fn,color=2,audio=2,mode=1,raw="v210",dither=1)#9th bit
#low bit to high bit
getlowbyte(last,deeplasthi)
global deeplastlo=last
deeplasthi
}


I would like it if you store 10bit values as two bytes in two clips, then scripts can use 16bit video in a semi-compatible way.

[PitifulInsect]

Hey all! Apologies for the extended absence.

A friend suggested directly reading gzipped files, since gzip is also GPLv2, and it sounded like a good idea - for some synthetic sequences, you can save a lot of diskspace working directly with compressed files. Unfortunately I found that the interface to gzip is more than a little hairy, I fought a tonne of compatibility problems, and when I got a debug-only version working I discovered that the caching performance was appalling since reading and seeking within the file requires decompressing big big chunks or even the whole thing to RAM! If that sounds like and exhasperatedly long sentence, there's a reason!

So yeah, development stalled, and all of your wonderful comments went unanswered. Sorry! I'm putting the idea of live compression aside for now - just use XVid or AVC (.h264) lossless if you need something in your workflow :-)

On to the updates!

foxyshadis - I've added that truncate option (default true) as discussed. Thank you for that feedback!
Chikuzen, kemuri-_9, Wilbert - I'm having a think about how to best deal with yuv4mpeg. Thank you especially Chikuzen for the script!
18fps, foxyshadis, cretindesalpes - It looks like higher bit depths are a major use case alright!

For 16-bit, I have not yet, but will add some scripts for handling it in greyscale, and a native function for interleaved-16-bit RGB since no combination of TurnLeft(), Interleave() and SeparateFields() can interleave 3 things together.
This should also cover 24 and 48 bit values, handling them all as multiple planes of 8-bit internal values.
12-bit breaks a lot of assumptions though, I'll put my thinking hat on for that one.

jmac698 - Thank you! A piece of the puzzle for free! Could you post a link to a sample file in v210 format or an official description of the format? For now anyone looking for it has your original solution right here :-) I'll learn about it and extend Sashimi next update.


Meantime: Version 0.75 posted! See top of page.
Thanks all! More updates soon I hope.

[Gavino]

... no combination of TurnLeft(), Interleave() and SeparateFields() can interleave 3 things together.
Not sure if it's the same as what you want, but have you seen this thread?

[jmac698]

Yes, I've extracted the weave3, I've had to do this myself a few months ago, which also lead me to discover bug... err, issue with pointresize; the implementation is different for each colorspace, the point which is thrown away in downsizing is different.

w = width(Clip1)
h = height(Clip1)

Pair1 = Interleave(Clip1, Clip3).Weave
Pair2 = Interleave(Clip2, Clip3).Weave
QuadHeight = Interleave(Pair1, Pair2).Weave # Line sequence is Clip1, Clip2, Clip3, Clip3, repeating.
TripleHeight = PointResize(QuadHeight, w, h*3)


Pt: I wrote a working full 10bit file reader with 16bit color controls in deepcolor utilties. The official docs are http://developer.apple.com/quicktime/icefloe/dispatch019.html#v210
The full quicktime spec is huge; but getting a visible image is easy, just skip some bytes and interpret the packed pixel format. Some people posted samples in my other thread. http://forum.doom9.org/showthread.php?p=1467907#post1467907

[jmac698]

And here's my weave3h

w=32
h=128
clip1 = BlankClip(width=w,height=h,pixel_type="YUY2", color_yuv=$4164d4)#red
clip2 = BlankClip(width=w,height=h,pixel_type="YUY2", color_yuv=$70483a)#grn
clip3 = BlankClip(width=w,height=h,pixel_type="YUY2", color_yuv=$23d472)#blu
clip4 = BlankClip(width=w,height=h,pixel_type="YUY2", color_yuv=$9a388b)#yel
Global colorspace="rgb"#change this line to show the bug
weave3h(clip1.c, clip2.c, clip3.c, clip4.c)
function c(clip v) {lcase(colorspace)=="rgb"?v.converttorgb:v}
#Results: dark grey everything ok, yellowish means first line of 4 is not being deleted

bilinearresize(width,height/3)

function weave3h(clip a, clip b, clip c, clip x) {#horizontally weave 3 clips. tested only with Avisynth 2.58.
#Relies on implementation specific behaviour, may break in other versions.
#There will be color issues in yv12 vertically, and all yuv spaces when horizontal
a=a.turnright
b=b.turnright
c=c.turnright
x=x.turnright
#0 1 2 3->01 23->0213->213 or abc in rgb; 021 or xab in yuv
#x,b,a,c for rgb; a,c,b,x for yv12
#to fix use:
a.isrgb?interleave(x,b,a,c):interleave(a,c,b,x)
assumefieldbased
assumetff
weave
assumefieldbased
assumetff
weave
pointresize(width,height*3/4)#Deletes every 4th line offset 0 in rgb;
#deletes every 4th line offset 3 in yuv. This is the part that's colorspace/version dependant.
#turnleft#avoid color issues for this test
}

[cpul]

good article.tq

[jmac698]

The v210 article was great. I've updated my v210 reader; it reads all files in greyscale.

[Yellow_]

Would any of the exotic formats support a wide RGB gamut? Trying to get xvYCC out as RGB image sequences using yesgreys yCMS 3D LUT generator and Triticals td3lut plugin.

[PitifulInsect]

:thanks:
Gavino - Thank you for the Weave3 link! And jmac698 - cheers for the examples. Here is the cleaned up script that will be in the next version of Sashimi - but all yours if you want to use it now. By "cleaned up" I simply mean that I've coded it for what I think is maximum clarity and readability:

##
## Take an RGB clip and expand it to greyscale where the first column is R,
## the second G, the third B, repeating.
##
## You should never really need this function for writing to / reading to file
## for simple cases, just use the native support - ie: RawWriter(..., "RGB")
## This is provided as an example for assembling exotic formats.
##
function ExpandRGBTo3ChannelGrey(clip C)
{
C = C.ConvertToRGB32()
Red = ShowRed( C, "YV12")
Green = ShowGreen(C, "YV12")
Blue = ShowBlue( C, "YV12")
Alpha = ShowAlpha(C, "YV12")
RiBiGiA = Interleave(Red, Blue, Green, Alpha)

RiBiGiA = RiBiGiA.TurnRight()
RBiGA = RiBiGiA.AssumeFieldBased().AssumeTFF().Weave()
RGBA = RBiGA.AssumeFieldBased().AssumeTFF().Weave()
RGBA = RGBA.TurnLeft()
#
# Thanks to <Gavino> for pointing me to <pbristow>'s trick: PointResize
# can be used to skip lines for you!
# When you skip every fourth column of RGBB, you get RGB!
# http://forum.doom9.org/showthread.php?t=153737
#
return PointResize(RGBA, RGBA.Width()/4*3, RGBA.Height())
}


##
## Take a greyscale clip where the columns alternate between 3 channels and
## combine it to be an RGB clip, ie: The first column becomes the red channel,
## the second green, the third blue, repeating.
##
## You should never need this function for writing to / reading to file for
## simple RGB cases, just use the native support - ie: RawWriter(..., "Y8")
##
## This is provided as an example for assembling exotic formats.
##
function Collapse3ChannelToRGB(Clip C)
{
RGB = C.ConvertToYV12()

#PointResize repeats values to pad as necessary.
RRGB = PointResize(RGB, RGB.Width()/3*4, RGB.Height())

RRGB = RRGB.TurnRight()
RGiRB = RRGB.AssumeFrameBased().AssumeTFF().SeparateFields()
RiGiRiB = RGiRB.AssumeFrameBased().AssumeTFF().SeparateFields()
RiGiRiB = RiGiRiB.TurnLeft()
red = RiGiRiB.SelectEvery(4,0)
green = RiGiRiB.SelectEvery(4,1)
blue = RiGiRiB.SelectEvery(4,3)
return MergeRGB(red, green, blue)
}

EDIT: There were some issues first post, so I fixed it. I have attached the full tested script to this post with more functions than shown here.
See also:
http://forum.doom9.org/showthread.php?t=153737

[PitifulInsect]

To all who are interested in yuv4mpeg support - I wrote up a Python script based on Chikuzen's (exactly equal in functionality), and included it in Sashimi last version (0.75)... by accident - it's not yet tested! But it is there.
I need to just do some cleanup, quality control, and testing soon.

A question for those who want 16-bit (and other) support. How exactly do you want it? 16, 24, and 48-bit can all be imported as wide-frame grey and shuffled as previously discussed in this thread, but then what? What are the use-cases for 12 bit? I'm not sure what packing to expect - probably the tricks previously-mentioned in this thread would work, no?

I'm not purely waiting for answers - I've seen the specific comments and I have some ideas (eg: load as Y16 but then you have to provide arguments for shifting / scaling to 8 bit), but since I haven't met such formats myself, I'm not sure what would be most useful and need more spec. Post (small) example attachments, links to sample files, etc please!

PS: Did anyone else stop getting update emails recently, or is it just me? Checking manually is no problem, but so nice to see emails with immediate updates.

[PitifulInsect]

Would any of the exotic formats support a wide RGB gamut? Trying to get xvYCC out as RGB image sequences using yesgreys yCMS 3D LUT generator and Triticals td3lut plugin.

That depends - could you explain what you need?

Links would be helpful in the future. I Googled to find that the LUT generator gives the conversion info (http://forum.doom9.org/showthread.php?t=154719), and then it took me a little while to find your thread about td3lut (http://forum.doom9.org/showthread.php?t=139389), but I couldn't find what the plugin is or does. As far as searching shows, "td3lut" is mentioned just once, on page 31 of the thread and without context.

Sashimi isn't going to extend AVISynth's core ability though, it's just a file IO plugin for reading and writing raw data. If there's a way of picking video up off disk or putting it down that isn't currently supported but would be useful, please let me know!

[jmac698]

Pit,
Your Collapse3ChannelToRGB doesn't help me at all; I'm working in YV12 which is what v210 format is. Also my deepcolor tools will import deepcolor formats and do levels adjustments like Tweak on it. This is practically useful for bringing out darker scenes that would be black in 8bit import. I made up a format that's useful for scripts; the high byte in one clip and the low byte in another. That way you can work normally with the high byte but also incorporate the low bits where needed.
If you could support searching for a set of bytes in the raw file that would help me a lot.
Here's my quick way to finish my v210 reader; I load the file in ffmpegsource normally, read a few pixels, figure out how those would be stored as raw bytes, then search for those raw bytes in the file, now this gives me the header size, then I open again with my raw importer and read the full 10 bits.

[PitifulInsect]

If that doesn't immediately help, you can take what's there and expand upon it. As I said, higher bit depth support is coming. Please also check your PM from before.

I've just updated the Expand and Collapse functions (attached to the above post, attachment currently pending approval) to include 3 channels to and from YUV.

While we wait for approval, here's Collapse3ChannelToYUV:
##
## Take a greyscale clip where the columns alternate between 3 channels and
## combine it to be an YUV clip, ie: The first column becomes the Y channel,
## the second U, the third V, repeating.
##
## You should never need this function for writing to / reading to file for
## simple RGB cases, just use the native support - ie: RawWriter(..., "Y8")
##
## Unfortunately, AVISynth doesn't support non-subsampled YUV, so the U and V
## channels will be affected. If this is no good for you, then change your
## workflow. For example, you could do identical transformations to two clips,
## One for your Y channel (with half-width U and V), and one for your U and V
## channels (with double-width Y).
##
## This is provided as an example for assembling exotic formats.
##
function Collapse3ChannelToYUV(Clip C)
{
YUV = C.ConvertToYUY2()

#PointResize repeats values to pad as necessary.
YYUV = PointResize(YUV, YUV.Width()/3*4, YUV.Height())

YYUV = YYUV.TurnRight()
YUiYV = YYUV.AssumeFrameBased().AssumeTFF().SeparateFields()
YiUiYiV = YUiYV.AssumeFrameBased().AssumeTFF().SeparateFields()
YiUiYiV = YiUiYiV.TurnLeft()
Y = YiUiYiV.SelectEvery(4,0)
# Yes, resize with PointResize. This is actually the correct way!
# See http://msdn.microsoft.com/en-us/library/ms867704.aspx#yuvformats_yuvsampling
U = YiUiYiV.SelectEvery(4,1).PointResize(Y.Width()/2, Y.Height())
V = YiUiYiV.SelectEvery(4,3).PointResize(Y.Width()/2, Y.Height())
return YToUV(U, V, Y)
}

[PitifulInsect]

After sleeping on the problem, I'm thinking that the RawReader could accept a bit-packing string, allowing you to specify how the bits of the input are to be interpreted down (or up) to the 8-bits-per-sample used in AVISynth. It would be a sort of pre-filter before the format string picks up the resulting samples. Numbers below 8 become the high bits, numbers above will require that you specify how they are treated. I think that it allows a lot of flexibility and clarity.

Thinking aloud, not yet final:

For 16-bit Y, you could either load and de-interleave it yourself, or specify "Grey" format and bit-packing "16:15-0" to tell Sashimi that 16 bits make a sample, and bits 0-15 should be compressed into that (8-bit) sample. The bits after the colon are mandatory because that first number is higher than 8.
If you knew that in a particular video you didn't need the first two bits, you'd say "16:13-0". If you wanted to just take the high-byte: "16:15-8". If you wanted to discard everything after the first 6 bits: "16:15-9". If you wanted alternate pixels of high and low byte: "16:15-8;16:7-0"... the same as saying nothing at all but more expensive!
A format with 12 bits for Y and 6 each for U and V would be format "YUV", packing "12:11-0;6;6".
For something like 16-bit highcolour (http://en.wikipedia.org/wiki/Highcolor#16-bit_high_color), you'd load as "RGB" and say "5;6;5" for the packing.
That funky V210 format would be best read in as grey, 10-bit packing, 12 of which only 10 are used at the end to discard those padding bits, something like ("10:9-0;10:9-0;12:11-2"), and you'd untangle it from there (it's something like UYVY, right? I haven't read the full [long] spec).
For something truly exotic, read it in multiple passes, eg: extracting the Y channel from v210: "32:20-10;16:13-4;16:9-0;" and then you'd get repeating streams of Y0 Y2 Y1 Y3 Y5 Y4, and do similar for the Cr and Cb (U and V).
It would probably have to be a requirement that the width and packing were such that each row ended on an 8-byte boundary.


Opinions, thoughts, suggestions?
Does this allow everyone to load what they want?
(Can I implement this bug-free?)

[PitifulInsect]

Hey all!

I'm posting a Beta. It seems to work, but I haven't tried it on [m]any example files :-)

This may be the answer to all of your 10 and 12 bit dreams! In fact, it's general, so it could be the answer to your 17-bit dreams it you have any.

Let me know! Feedback appreciated:
https://sites.google.com/site/ourenthusiasmsasham/contributions/open-source-software#TOC-Sashimi

-- (They Call Me Mr.) Pitiful

[jmac698]

It's crashing on me with rawreader
File "avisynth.pyo", line 309, in GetPitch
ValueError: NULL pointer access

[PitifulInsect]

25th February 2011, 00:39

jmac698 - Thank you! A piece of the puzzle for free! Could you post a link to a sample file in v210 format


12th March 2011, 23:04 ...since I haven't met such formats myself, I'm not sure what would be most useful and need more spec. Post (small) example attachments, links to sample files, etc please!

13th March 2011, 03:22 could you explain what you need?
Links would be helpful in the future.

Yesterday, 00:43 It seems to work, but I haven't tried it on [m]any example files :-)

Looking online hasn't shown up anything, and I'm hoping that the hinting might pay off. Don't you think that making a sample of that data available would be a good idea? Please encode and upload somewhere a 1-frame video of the sample JPEG from Sashimi for example.

Other notes:

avisynth.pyo? Python Object? Are you running something odd or unusual?
Did you read the instructions? I changed the packing string format when I implemented it. It's documented.
I've just uploaded an updated beta, which does some checking on the packing string and gives some error feedback. It's functionally identical, but it will give better feedback about a typo or a mis-specified packing string, and it's possible that it will exit more gracefully than before if a bad string is supplied.

[jmac698]

http://www.mediafire.com/?78nid4qncerqjqm
v210 samples.7z
1080p, 480p, 576p, 720p, 29.97, 23.976, 25 fps yuv, rgb, should cover about everything.

The error came from AvsP, an avisynth editor. It gives a rough indication of why I crashed. Forget the .py reference, it's the avisynth function getpitch which caused the crash.

This is the line I used, does it work on your end?

base=RawReader(fn, format="y8", width=rowsize, height=h, numframes=0, filehead=21192-664, framehead=0)

[PitifulInsect]

v210 is a UYVY format by definition, I'm not sure what's with the files that you added "RGB" to the filename of. They look about right if you read them in using the same reading code as the v210 samples, but without the original, I can't confirm this.

In future, as a practicality as well as a politeness, I recommend that you provide an original reference if providing an encoded sample (http://www.belle-nuit.com/testchart.html for the benefit of others), that's why I asked for the sashimi image to be encoded. Similarly, all necessary context and variables are required if providing code-snippets that you ask others for help with or opinions on. You reference functions that don't exist over in another thread where you said that you had some results from running Sashimi with v210 files.

Since your values and a specific filename aren't provided in the snippet above, I can't tell you what my machine does when running the snippet. The file header is just 48 bytes by the way, not the number you have, whatever it's origin.

Anyway, I've been fiddling with it all, and I've had no issue with anything crashing, I can't replicate your problem. I did have one bugger of a time figuring out the format around the little-endianness of the v210 bytes, but I overcame that, and I seem to have a generic v210 reading function. My scripts are attached - I hope that they work for you!

Because there's a lot of value-shuffling in AVISynth, the attached scripts are quite slow, but there's no sensible way to move more of the processing into Sashimi (which is fast).

[jmac698]

Some of the samples include r210, which is like v210 but with rgb. I didn't do the encodes, and I have no way of encoding to this format. I don't necessarily know where they came from. The header size is not always 48, remember these are often just still frames or have no audio. The dimensions can be got from opening once in ffms. I think your plugin would be a lot faster and more useful if you could specify the packing string for 3 planes at once, then it could be read in one pass. I'll try your new binary.
A null pointer shouldn't be happening. My first guess is that I had a header size larger than the file.
Some of the files could be named rgb because they came from an 8bit rgb source and therefore don't have the full range of 10bit yuv values.

[PitifulInsect]

I think your plugin would be a lot faster and more useful if you could specify the packing string for 3 planes at once, then it could be read in one pass.
You already can - if I understand your idea correctly.

I've specified Y8 and read the v210 file in in passes in my example because different planes of images need to be combined using different levels adjustments, but if that wasn't necessary, you could do something like (with reference to OptimisedReadV210(), and note that you have to use RGB not YV12 or YUY2 because of chroma subsampling):
Highbits = RawReader(filename, "RGB", rwidth, height, filehead=fh, packing="32:14:2; 32:20:4; 32:26:6")
Cb0Y1Cr1Y4_H = Highbits.ShowRed("YV12")
Y0Cb1Y3Cr2_H = Highbits.ShowGreen("YV12")
Cr0Y2Cb2Y5_H = Highbits.ShowBlue("YV12")
If that's not what you meant though with the colour planes, let me know. I'm keen to make Sashimi more useful, but I'm trying to keep it general.

It's all of the overlaying, rotating, field-separating and weaving which makes that script slow though. I don't believe that the file-reads and Sashimi are the bottleneck. You'd probably need to benchmark it, specific to your machine, to trade off the cost of multiple Sashimi calls vs just two followed by separation as above before the Overlay() calls. It will be entirely down to processor speed vs. data access and read-caching.
BTW: From memory, and just for reference, be careful with Overlay() when working in RGB; you'll destroy your data if you miss a manual conversion. I've meant Sashimi to be used in scientific applications, so I've been careful to make sure that it's bit-perfect everywhere, with no exceptions that I yet know of.

I'll try your new binary. A null pointer shouldn't be happening. My first guess is that I had a header size larger than the file.
Hmmm - I'll double check that it's smart enough not to fall over a bad header length. It might not be. Do report back though if it's something else and can be replicated.

[PitifulInsect]

So as a separate post to keep things clear: How's the packing syntax? Some similar alternatives are obvious, but is anything obviously better or worse than what I have? People sometimes have strong opinions about these things.

Before pulling it out of beta, I should add row-end-padding, eg: rowpad=4. You specify width in pixels, and Sashimi will do the math based on packing and format, then pad to the nearest X bytes, where X defaults 0, is commonly 4 (32-bit boundaries are used often), and will be 48 in the v210 example discussed above. Since padding is interesting whether you specify packing or not, it will be a separate argument.

I've been toying with the idea of adding endian-ness correction in as a feature, since that's the big annoyance in the v210 example, and it is a fairly general thing that one might meet in the future...maybe? I can't decide. There is now a supplied demonstration (thanks to jmac698's v210 example) of how to work around that in a script, which makes it redundant, though it is a pain in the arse to do. Things that can be done in-script are always the clearest and most flexible, and therefore efficient to code and debug, if not the most efficient to run. If endian-ness is to join the party though, how to specify it? Something like "32] " at the start of a packing string? And is it really a sensible general thing or would I implement it for v210 and then never find another use for it ever again?

[Yellow_]

That depends - could you explain what you need?

Links would be helpful in the future. I Googled to find that the LUT generator gives the conversion info (http://forum.doom9.org/showthread.php?t=154719), and then it took me a little while to find your thread about td3lut (http://forum.doom9.org/showthread.php?t=139389), but I couldn't find what the plugin is or does. As far as searching shows, "td3lut" is mentioned just once, on page 31 of the thread and without context.

Sashimi isn't going to extend AVISynth's core ability though, it's just a file IO plugin for reading and writing raw data. If there's a way of picking video up off disk or putting it down that isn't currently supported but would be useful, please let me know!

Apologies for insufficient links.

I'm not sure whether Avisynths RGB image export is gamut independant or whether it writes image formats as sRGB.

I'm using yesgreys yCMS to do the color space conversion from YCbCr to RGB. yCMS will produce sRGB, AdobeRGB and it's own wide gamut RGB.

http://forum.doom9.org/showthread.php?p=1489309#post1489309

Is it possible to use your plugin to write images with those gamuts or a raw output as wide gamut as the source, that can be assigned the coresponding gamut outside of Avisynth.

Basically I would like to be sure that Avisynth isn't affecting the RGB data when writing to image formats.

Hope that makes sense.

[PitifulInsect]

Hey Yellow_,

As long as your sRGB is 8 bits per sample, any software (including AVISynth) will handle it, it just won't know that the RGB samples that are being processed have a different display interpretation. If you have such samples in a file, Sashimi can pick them up into AVISynth or if you have them in AVISynth it can put them down do disk, and doesn't manipulate their values at all - I really designed it for research and scientific uses, so I've taken care to make sure that it doesn't mess with data. So your answer is that Sashimi is completely agnostic to the data that it handles.

If you're curious: I have found that unfortunately AVISynth's regular ImageSource() and ImageWriter() functions do perform internal colour-space conversions, but I don't know the details and you can probably read more elsewhere, this isn't the thread for that. If you're processing any data internally, keep in mind that it no-longer means what the processing functions think that it means, so you won't know if they're treating it correctly. Some functions like Mask() and Layer() do internal conversions, so you know that they're not right - I always use Overlay() instead, which is colour-space native (YV12, YUY2, and RGB).

Only vaguely relevant, but interesting: The problem with using any nonlinear colour space of course is that things will be wrong when you try to manipulate the data in any way. Almost no PC software (almost certainly not AVISynth either) even rescales images correctly!: http://www.4p8.com/eric.brasseur/gamma.html. These sorts of problems get fixed and updated over time, but I think generally the best advice might be to just make sure that your colour space is appropriate for the software that you're using. Good luck!

[Gavino]

Some functions like Mask() and Layer() do internal conversions, so you know that they're not right - I always use Overlay() instead, which is colour-space native (YV12, YUY2, and RGB).
Mask() only accepts RGB32, so no conversions there.
Layer() accepts either RGB32 or YUY2, but both clips must be the same, so no conversions there either.
Overlay(), on the other hand, converts everything into an internal YUV 4:4:4 format, so there is a colorspace conversion for RGB and chroma resampling for YUY2 and YV12. In addition, there is a rounding bug in the RGB conversion in v2.58 - see this post, and posts #24 and #30 above and below it.

Almost no PC software (almost certainly not AVISynth either) even rescales images correctly!: http://www.4p8.com/eric.brasseur/gamma.html.
We now have PhrostByte's gamma-aware resizer.

[Wilbert]

If you're curious: I have found that unfortunately AVISynth's regular ImageSource() and ImageWriter() functions do perform internal colour-space conversions
Please elaborate in which cases this happens. Sure there is a conversion if you feed it with YCbCr or the image formats don't support RGB.

[Yellow_]

Hey Yellow_,

As long as your sRGB is 8 bits per sample, any software (including AVISynth) will handle it, it just won't know that the RGB samples that are being processed have a different display interpretation. If you have such samples in a file, Sashimi can pick them up into AVISynth or if you have them in AVISynth it can put them down do disk, and doesn't manipulate their values at all - I really designed it for research and scientific uses, so I've taken care to make sure that it doesn't mess with data. So your answer is that Sashimi is completely agnostic to the data that it handles.

Ok, I'll give it another go, thanks.

If you're curious: I have found that unfortunately AVISynth's regular ImageSource() and ImageWriter() functions do perform internal colour-space conversions, but I don't know the details and you can probably read more elsewhere, this isn't the thread for that. If you're processing any data internally, keep in mind that it no-longer means what the processing functions think that it means, so you won't know if they're treating it correctly. Some functions like Mask() and Layer() do internal conversions, so you know that they're not right - I always use Overlay() instead, which is colour-space native (YV12, YUY2, and RGB).

Only vaguely relevant, but interesting: The problem with using any nonlinear colour space of course is that things will be wrong when you try to manipulate the data in any way. Almost no PC software (almost certainly not AVISynth either) even rescales images correctly!: http://www.4p8.com/eric.brasseur/gamma.html. These sorts of problems get fixed and updated over time, but I think generally the best advice might be to just make sure that your colour space is appropriate for the software that you're using. Good luck!

That's ok, I'm going to RGB with yCMS and now Phrostbytes gamma aware resizer, then have been using Wilberts Imagemagick plugin. Thanks for the details.

[jmac698]

Pitiful,
That link to google sites is down. Is there a mirror for Sashimi hosting?
And I might have to finish my v210 import soon :)

[PitifulInsect]

Hi jmac,

The URL's just changed, apologies:
http://sites.google.com/site/ourenthusiasmsasham/soft
You should get an automatic redirect suggestion though.

In case of future trouble, I've been keeping the first post of this thread updated with the latest stable version of the plugin so that as long as you can see the thread, you can get a copy.

Either way, I'll update the links soon and put out the next version of Sashimi. No big changes to announce - bugfixes in the multi-byte support and updates of the documentation. It's all delayed because I developed a set of tests and examples for the multi-bit-depth stuff and found bugs. I've squashed two, but there is at least one more bug, and I want to find and kill it before releasing.

[18fps]

I'd like to read a headerless file that is 2048 * 2048 pixels, 16 bits per channel, RGB, 2 "channels", the second has to be debayered. How sould I use the filter to read it, maybe at last to get a b&w image? Thank you!

[jmac698]

# Read the content in from the file with 32 bits/pixel (to be sorted later)
raw=RawReader("C:\Myfiles\sensor.raw", "argb", 2048, 2048, filehead=0, framehead=0)
IR25uMhi8bits=raw.showalpha
IR50uMhi8bits=raw.showgreen#If this doesn't look right try showblue, showred etc.
black=blankclip(width=2048,height=2048,pixel_type="RGB")
black.mergergb(black,IR50uM,black)#Make the sensor data "green" to work with debayer (you have to know what your's doing here, the pixel layout of the sensor, if it's 2 channel green is like a checkerboard and should work)
demosaic(xoffset = 1, yoffset = 0, method = "bilinear")#I've assumed the 2nd channel is offset 1 pixel..

Get the debayer plugin from http://forum.doom9.org/showthread.php?p=1108152

[PitifulInsect]

Gee people are quick on here! I was halfway through typing my own answer when jmac698 beat me to it!
Anyway, there are a lot of unknowns, so I'll just post some notes.

You say RGB, then you say 2 channels, which is a little confusing, I'll assume you simply have a bayer-grid dump. If you don't and you need more help, please make a sample file and more detail available.

jmac698's code is one neat way to do it, an alternative is
# Double-width and height for the bayer pattern, double width for 16-bit
RawReader("file.raw", Y8, 2048*4, 2048*2)
# Take out every second byte to cut down to 8-bit (in case of failure, try SelectOdd() instead.
TurnRight().AssumeFrameBased().AssumeTFF().SeparateFields().TurnLeft().SelectEven()

The reason I offer this alternative way of thinking about it, is that if you get the most recent Beta version of Sashimi (I promise to post an update soon, keep an eye on this thread), you can simplify still further to:
# Double-width and height for the bayer pattern, 16-bit
RawReader("file.raw", Y8, 2048*2, 2048*2, packing="16")
The advantage of doing it this way is that if you find that your image is very low contrast, you can try dropping some of the higher bits by changing the packing from "16" to "16:1" or even "16:2" or "16:3" etc (dropping 1, 2, or 3 high-end bits).

Removing the Bayer pattern is the hardest thing to do well - it's a whole research area of its own. A quick Google for me though, showed that there are a lot of tools in astronomy and science that take "Y800" files and give you various de-bayer algorithms. Lucky that there's a de-facto standard! You can make a "Y800" AVI file by saving your successfully loaded greyscale file directly out of VirtualDubMod using the ffdshow encoder, after setting the ffdshow encoder to "Uncompressed", colorspace "Y800". Then you can play with existing debayer tools as you find them.

[PitifulInsect]

Ooops - forgot the good link that I found. Here:
http://www.theimagingsourceforums.com/showthread.php?323680-quot-New-quot-Debayering-Solution-Using-AviSynth

[PitifulInsect]

Good news! The last bug in bit-depth support turned out to be an old bug - sashimi has apparently always barfed on YUY2. It's now fixed (error message if you try to use YUY2 - use YV12 instead), and the new version of Sashimi is available, either from the first post of this thread (attachments have an approval delay, please be patient), or directly from my site:
https://sites.google.com/site/ourenthusiasmsasham/soft
Cheers!
-- PitifulInsect.