New Plugin - Sashimi (raw file access)

[jmac698]

@pitfiul..
lesson learned, never draw with me in a gunfight :)
Look forward to the new version..

and I gave him a debayer plugin, which might actually work for him..

[SAPikachu]

@jmac698
I have made a new script based on your v210 reader 0.4, it can now read v210 file into full-color 8bit/16bit clip. May be useful to you. :)

You can check it here:
http://forum.doom9.org/showthread.php?p=1526131#post1526131

[jmac698]

That's awesome!
@pit:
I just realized, if you could make a plugin like sashimi but to read data files directly into script variables, like filereader(int a, float b, string c)
tweak(bright=a, constrast=b)
subtitle(c)
I have many uses for parsing text files, for example to read the histogram file from colorlike, or the logs from dgindex, etc. Something a bit more powerful than conidtionalreader.

It would be a set of vars per frame, so above would parse repeated values during the clip.

[18fps]

jmac698 and jmac698: thank you for your fast answers! I'll try them today.

[jmac698]

@Pit
I finally got around to writing my own readv210 with your new version, and I found another way to do it in just 2 passes. Not sure if it's any faster or not though; the code certainly seems simpler to me.

#v210 Reader Ver 0.5 by jmac698
#Read v210 Quicktime files with Sashimi, by jmac698
#Ver 0.5 - reads grey image in a new way with sashimi 0.85
#Requirements - only Sashimi 0.85
fn="D:\project001a\deepcolor\v210 samples\v210.mov"
readv210new(fn, 48, 1920, 1080)

function readv210new(string fn, int file_head, int frame_width, int frame_height) {#greyscale v210 file
line_size = (frame_width * 16 / 6 + 127) / 128 * 128 # all lines are padded to 128 bytes boundary
LeftWords = RawReader(fn, "RGB", line_size/4, frame_height, filehead=file_head, packing="8:0:8; 6:0:6; 18:0:2")#3 bitfields in 32bits
RightWords = RawReader(fn, "RGB", line_size/4, frame_height, filehead=file_head, packing="20:16:4; 4:0:4; 8:2:6")
p2 = LeftWords.every(2,1)
p3 = RightWords.every(2,1)
p0 = LeftWords.every(2,0)
p1 = RightWords.every(2,0)
Cb0_7_0=p0.showred
Y0_5_0=p0.showgreen
Cb0_9_8=p0.showblue
Cr0_3_0=p1.ShowRed
Y0_9_6=p1.ShowGreen
Cr0_9_4=p1.showblue
Y1_7_0=p2.showred
Cb1_5_0=p2.showgreen
Y1_9_8=p2.showblue
Y2_3_0=p3.showred
Cb1_9_6=p3.showgreen
Y2_9_4=p3.showblue

Y0_9_2=CombineBits(Y0_9_6,Y0_5_0,4)
Y1_9_2=CombineBits(Y1_9_8,Y1_7_0,2)
Y2_9_2=CombineBits(Y2_9_4,Y2_3_0,6)
Weave3h(Y0_9_2,Y1_9_2,Y2_9_2)
}

function weave3h(clip a, clip b, clip c) {#horizontally weave 3 clips
a=a.turnright
b=b.turnright
c=c.turnright
interleave(a,b,a,c)#0 1 2 3->01 23->0213->213 or abc
assumefieldbased
assumetff
weave
assumefieldbased
assumetff
weave
pointresize(width,height*3/4)#deletes every 4th line offset 0
turnleft
}

function every(clip v, int n, int offset) {#select every n bytes horizontally with offset, works on yv12 only
v
w=width
h=height
pointresize(v,w*2,h)
crop(offset*2,0,0,0).addborders(0,0,offset*2,0)#shift left offset pixels
pointresize(w/n,h)
}

function CombineBits(clip left, clip right, int bits) {
#Combine first bitsl of left with first bitsr of right
#e.g. 9876xxxx and 5432xxxx with bitsl=4, bitsr=4 gives 98765432
bitsright=8-bits
pwr=pow(2,bitsright)
Overlay(left, right.Levels(0,1,256, 0,int(pwr), false), mode="Add")
}

[Aegwyn11]

Resurrecting an old very helpful thread :)

I modified the ReadV210 script included in Sashimi 0.85 to support Stack16 output, enabling use of tools like Dither. I also made it so you can define the fh in the function call, as this works better for me. I realize its a bit dirtier than jmac698's version, but that's the one I started with, it works, and I don't want to spend any more time on it :)

I double checked using Dither and my input file matched my output file bit for bit.

Checking that input file matches output file bit for bit:

Readv210(input.yuv, 1920, 1080)
Dither_quantize(10,reducerange=true)
Dither_out()

#Command to generate output.yuv
#avs2yuv -raw -csp I422 "input.avs" - | ffmpeg -y -f rawvideo -pix_fmt yuv422p10le -s 1920x1080 -r 60000/1001 -i - -c:v v210 "output.yuv"

Modified ReadV210:
#
# Example of reading a really difficult file format.
# See http://developer.apple.com/quicktime/icefloe/dispatch019.html#v210
#
# Peter Pakulski, 2011
#
# Stack16 output added by Aegwyn11, 2013 (requires AVIsynth 2.6)
#

function ReadV210(string filename, int width, int height, int "fh")
{
fh = Default (fh, 0)

#OldReadV210(filename, width, height, fh)
#OptimisedReadV210(filename, width, height, fh)
Readv210Stack16(filename, width, height, fh)
}

#
# A straightforwards interpretation of a curly file format.
#
function OldReadV210(string filename, int width, int height, int "fh")
{
#
# There seems to be a standard file-header length, and it's only 48 bytes.
# The rest of the specification (there's more) is after the file body.
#fh = 48
fh = Default (fh, 0)

# The Read-width of the image is the width padded up to the nearest 48-byte
# boundary as per the specification, and divided by 6 because of the
# effective interlacing you get from reading only one value per pass.
rwidth = (width/48 + (width%48==0 ? 0 : 1)) * 48 / 6

#
# What a NUISANCE! The v210 format is little-endian 32-bit words, 128 bits at
# a time. So from the specification, for example, starting with the first 32:
# _______ _______ _______ ______
# / Byte3 \/ Byte2 \/ Byte1 \/ Byte0\
# XX 9876543210 9876543210 9876543210
# \___Cr___/ \___Y0___/ \___Cb___/
#
# Is actually stored all mixed up, if reading it in, in byte order:
# ______ _______ _______ _______
# / Byte0\ / Byte1 \ / Byte2 \ / Byte3 \
# 76543210 543210 98 3210 9876 XX 987654
# \__Cb__/ \_Y0_/ Cb \Cr/ \Y0/ \_Cr_/
#
# This pattern repeats.
#
# So below: Extract the muddled pixel colour components, half at a time, in
# multiple reads. Assemble them, then assemble them into colour planes, and
# finally assemble the colour planes into a colour image.
#
# 4-byte word #0 - as above (Cr0, Y0, Cb0)
Cb0H = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="128:14:2")
Cb0L = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="128:0:6")
Cb0 = Cb0H.Overlay(Cb0L.Levels(0,1,256, 0,64, false), mode="Add")

Y0H = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="128:20:4")
Y0L = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="128:8:4")
Y0 = Y0H.Overlay(Y0L.Levels(0,1,256, 0,16, false), mode="Add")

Cr0H = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="128:26:6")
Cr0L = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="128:16:2")
Cr0 = Cr0H.Overlay(Cr0L.Levels(0,1,256, 0,4, false), mode="Add")

# 4-byte word #1 - Y2, CB1, Y1
Y1H = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="128:46:2")
Y1L = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="128:32:6")
Y1 = Y1H.Overlay(Y1L.Levels(0,1,256, 0,64, false), mode="Add")

Cb1H = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="128:52:4")
Cb1L = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="128:40:4")
Cb1 = Cb1H.Overlay(Cb1L.Levels(0,1,256, 0,16, false), mode="Add")

Y2H = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="128:58:6")
Y2L = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="128:48:2")
Y2 = Y2H.Overlay(Y2L.Levels(0,1,256, 0,4, false), mode="Add")

# 4-byte word #2 - Cb2, Y3, Cr1
Cr1H = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="128:78:2")
Cr1L = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="128:64:6")
Cr1 = Cr1H.Overlay(Cr1L.Levels(0,1,256, 0,64, false), mode="Add")

Y3H = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="128:84:4")
Y3L = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="128:72:4")
Y3 = Y3H.Overlay(Y3L.Levels(0,1,256, 0,16, false), mode="Add")

Cb2H = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="128:90:6")
Cb2L = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="128:80:2")
Cb2 = Cb2H.Overlay(Cb2L.Levels(0,1,256, 0,4, false), mode="Add")

# 4-byte word #3 - Y5, Cr2, Y4
Y4H = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="128:110:2")
Y4L = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="128:96:6")
Y4 = Y4H.Overlay(Y4L.Levels(0,1,256, 0,64, false), mode="Add")

Cr2H = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="128:116:4")
Cr2L = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="128:104:4")
Cr2 = Cr2H.Overlay(Cr2L.Levels(0,1,256, 0,16, false), mode="Add")

Y5H = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="128:122:6")
Y5L = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="128:112:2")
Y5 = Y5H.Overlay(Y5L.Levels(0,1,256, 0,4, false), mode="Add")

# Next, assemble the three planes of the image:
# Y (with components 0, 1, 2, 3, 4, and 5),
# and Cb and Cr (each with components 0, 1, and 2).
# Assemble padded to even numbers, then use PointResize to remove the padding.
Y0_3_2_5_1_4_p_p = Interleave(Y0, Y3, Y2, Y5, Y1, Y4, Y0, Y0).TurnRight()
Y03_25_14_pp = Y0_3_2_5_1_4_p_p.AssumeFieldBased().AssumeTFF().Weave()
Y0235_1p4p = Y03_25_14_pp.AssumeFieldBased().AssumeTFF().Weave()
Y012p345p = Y0235_1p4p.AssumeFieldBased().AssumeTFF().Weave()
Y012p345p = Y012p345p.TurnLeft()
Y012345 = Y012p345p.PointResize(Y012p345p.Width()*3/4, Y012p345p.Height())

Cb0_2_1_p = Interleave(Cb0, Cb2, Cb1, Cb0).TurnRight()
Cb02_1p = Cb0_2_1_p.AssumeFieldBased().AssumeTFF().Weave()
Cb012p = Cb02_1p.AssumeFieldBased().AssumeTFF().Weave()
Cb012p = Cb012p.TurnLeft()
Cb012 = Cb012p.PointResize(Cb012p.Width()*3/4, Cb012p.Height())

Cr0_2_1_p = Interleave(Cr0, Cr2, Cr1, Cr0).TurnRight()
Cr02_1p = Cr0_2_1_p.AssumeFieldBased().AssumeTFF().Weave()
Cr012p = Cr02_1p.AssumeFieldBased().AssumeTFF().Weave()
Cr012p = Cr012p.TurnLeft()
Cr012 = Cr012p.PointResize(Cr012p.Width()*3/4, Cr012p.Height())

Y012345 = Y012345.ConvertToYUY2()
Cb012 = Cb012.ConvertToYUY2()
Cr012 = Cr012.ConvertToYUY2()
YToUV(Cb012, Cr012, Y012345)

# Each row is padded out, so crop back to the actual size.
Crop(0, 0, width, height)
}

#
# This version as above, but optimised, taking some shortcuts.
# As is, it is very hard to read, which is why the above exists.
#
function OptimisedReadV210(string filename, int width, int height, int "fh")
{
#fh = 48
fh = Default (fh, 0)

rwidth = (width/48 + (width%48==0 ? 0 : 1)) * 48 * 2/3

Cb0Y1Cr1Y4_H = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="32:14:2")
Cb0Y1Cr1Y4_L = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="32:0:6")
Cb0Y1Cr1Y4 = Cb0Y1Cr1Y4_H.Overlay(Cb0Y1Cr1Y4_L.Levels(0,1,256, 0,64, false), mode="Add")

Y0Cb1Y3Cr2_H = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="32:20:4")
Y0Cb1Y3Cr2_L = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="32:8:4")
Y0Cb1Y3Cr2 = Y0Cb1Y3Cr2_H.Overlay(Y0Cb1Y3Cr2_L.Levels(0,1,256, 0,16, false), mode="Add")

Cr0Y2Cb2Y5_H = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="32:26:6")
Cr0Y2Cb2Y5_L = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="32:16:2")
Cr0Y2Cb2Y5 = Cr0Y2Cb2Y5_H.Overlay(Cr0Y2Cb2Y5_L.Levels(0,1,256, 0,4, false), mode="Add")

Cb0Y1Cr1Y4 = Cb0Y1Cr1Y4.TurnRight()
Cb0Cr1_Y1Y4 = Cb0Y1Cr1Y4.AssumeFrameBased().AssumeTFF().SeparateFields()
Y1Y4 = Cb0Cr1_Y1Y4.SelectOdd()
Cb0_Cr1 = Cb0Cr1_Y1Y4.SelectEven().AssumeFrameBased().AssumeTFF().SeparateFields()
Cb0 = Cb0_Cr1.SelectEven()
Cr1 = Cb0_Cr1.SelectOdd()

Y0Cb1Y3Cr2 = Y0Cb1Y3Cr2.TurnRight()
Y0Y3_Cb1Cr2 = Y0Cb1Y3Cr2.AssumeFrameBased().AssumeTFF().SeparateFields()
Y0Y3 = Y0Y3_Cb1Cr2.SelectEven()
Cb1_Cr2 = Y0Y3_Cb1Cr2.SelectOdd().AssumeFrameBased().AssumeTFF().SeparateFields()
Cb1 = Cb1_Cr2.SelectEven()
Cr2 = Cb1_Cr2.SelectOdd()

Cr0Y2Cb2Y5 = Cr0Y2Cb2Y5.TurnRight()
Cr0Cb2_Y2Y5 = Cr0Y2Cb2Y5.AssumeFrameBased().AssumeTFF().SeparateFields()
Y2Y5 = Cr0Cb2_Y2Y5.SelectOdd()
Cr0_Cb2 = Cr0Cb2_Y2Y5.SelectEven().AssumeFrameBased().AssumeTFF().SeparateFields()
Cr0 = Cr0_Cb2.SelectEven()
Cb2 = Cr0_Cb2.SelectOdd()

Y03_25_14_pp = Interleave(Y0Y3, Y2Y5, Y1Y4, Y0Y3)
Y0235_1p4p = Y03_25_14_pp.AssumeFieldBased().AssumeTFF().Weave()
Y012p345p = Y0235_1p4p.AssumeFieldBased().AssumeTFF().Weave()
Y012345 = Y012p345p.PointResize(Y012p345p.Width(), Y012p345p.Height()*3/4)
Y012345 = Y012345.TurnLeft()

Cb0_2_1_p = Interleave(Cb0, Cb2, Cb1, Cb0)
Cb02_1p = Cb0_2_1_p.AssumeFieldBased().AssumeTFF().Weave()
Cb012p = Cb02_1p.AssumeFieldBased().AssumeTFF().Weave()
Cb012 = Cb012p.PointResize(Cb012p.Width(), Cb012p.Height()*3/4)
Cb012 = Cb012.TurnLeft()

Cr0_2_1_p = Interleave(Cr0, Cr2, Cr1, Cr0)
Cr02_1p = Cr0_2_1_p.AssumeFieldBased().AssumeTFF().Weave()
Cr012p = Cr02_1p.AssumeFieldBased().AssumeTFF().Weave()
Cr012 = Cr012p.PointResize(Cr012p.Width(), Cr012p.Height()*3/4)
Cr012 = Cr012.TurnLeft()

Y012345 = Y012345.ConvertToYUY2()
Cb012 = Cb012.ConvertToYUY2()
Cr012 = Cr012.ConvertToYUY2()
YToUV(Cb012, Cr012, Y012345)

Crop(0, 0, width, height)
}

#
# This version as a modified version of OptimisedReadV210 to output Stack16.
# Using this, you can use Stack16 capable plugins (such as Dither) to process all 10 bits.
#
function Readv210Stack16(string filename, int width, int height, int "fh")
{
fh = Default (fh, 0)

rwidth = (width/48 + (width%48==0 ? 0 : 1)) * 48 * 2/3

Cb0Y1Cr1Y4_H = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="32:14:2")
Cb0Y1Cr1Y4_L = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="32:0:6")
Cb0Y1Cr1Y4_8 = Cb0Y1Cr1Y4_H.Overlay(Cb0Y1Cr1Y4_L.Levels(0,1,256, 0,64, false), mode="Add")
Cb0Y1Cr1Y4_10 = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="32:6:2")
Cb0Y1Cr1Y4 = StackVertical(Cb0Y1Cr1Y4_8,Cb0Y1Cr1Y4_10)

Y0Cb1Y3Cr2_H = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="32:20:4")
Y0Cb1Y3Cr2_L = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="32:8:4")
Y0Cb1Y3Cr2_8 = Y0Cb1Y3Cr2_H.Overlay(Y0Cb1Y3Cr2_L.Levels(0,1,256, 0,16, false), mode="Add")
Y0Cb1Y3Cr2_10 = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="32:12:2")
Y0Cb1Y3Cr2 = StackVertical(Y0Cb1Y3Cr2_8,Y0Cb1Y3Cr2_10)

Cr0Y2Cb2Y5_H = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="32:26:6")
Cr0Y2Cb2Y5_L = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="32:16:2")
Cr0Y2Cb2Y5_8 = Cr0Y2Cb2Y5_H.Overlay(Cr0Y2Cb2Y5_L.Levels(0,1,256, 0,4, false), mode="Add")
Cr0Y2Cb2Y5_10 = RawReader(filename, "Y8", rwidth, height, filehead=fh, packing="32:18:2")
Cr0Y2Cb2Y5 = StackVertical(Cr0Y2Cb2Y5_8,Cr0Y2Cb2Y5_10)

Cb0Y1Cr1Y4 = Cb0Y1Cr1Y4.TurnRight()
Cb0Cr1_Y1Y4 = Cb0Y1Cr1Y4.AssumeFrameBased().AssumeTFF().SeparateFields()
Y1Y4 = Cb0Cr1_Y1Y4.SelectOdd()
Cb0_Cr1 = Cb0Cr1_Y1Y4.SelectEven().AssumeFrameBased().AssumeTFF().SeparateFields()
Cb0 = Cb0_Cr1.SelectEven()
Cr1 = Cb0_Cr1.SelectOdd()

Y0Cb1Y3Cr2 = Y0Cb1Y3Cr2.TurnRight()
Y0Y3_Cb1Cr2 = Y0Cb1Y3Cr2.AssumeFrameBased().AssumeTFF().SeparateFields()
Y0Y3 = Y0Y3_Cb1Cr2.SelectEven()
Cb1_Cr2 = Y0Y3_Cb1Cr2.SelectOdd().AssumeFrameBased().AssumeTFF().SeparateFields()
Cb1 = Cb1_Cr2.SelectEven()
Cr2 = Cb1_Cr2.SelectOdd()

Cr0Y2Cb2Y5 = Cr0Y2Cb2Y5.TurnRight()
Cr0Cb2_Y2Y5 = Cr0Y2Cb2Y5.AssumeFrameBased().AssumeTFF().SeparateFields()
Y2Y5 = Cr0Cb2_Y2Y5.SelectOdd()
Cr0_Cb2 = Cr0Cb2_Y2Y5.SelectEven().AssumeFrameBased().AssumeTFF().SeparateFields()
Cr0 = Cr0_Cb2.SelectEven()
Cb2 = Cr0_Cb2.SelectOdd()

Y03_25_14_pp = Interleave(Y0Y3, Y2Y5, Y1Y4, Y0Y3)
Y0235_1p4p = Y03_25_14_pp.AssumeFieldBased().AssumeTFF().Weave()
Y012p345p = Y0235_1p4p.AssumeFieldBased().AssumeTFF().Weave()
Y012345 = Y012p345p.PointResize(Y012p345p.Width(), Y012p345p.Height()*3/4)
Y012345 = Y012345.TurnLeft()

Cb0_2_1_p = Interleave(Cb0, Cb2, Cb1, Cb0)
Cb02_1p = Cb0_2_1_p.AssumeFieldBased().AssumeTFF().Weave()
Cb012p = Cb02_1p.AssumeFieldBased().AssumeTFF().Weave()
Cb012 = Cb012p.PointResize(Cb012p.Width(), Cb012p.Height()*3/4)
Cb012 = Cb012.TurnLeft()

Cr0_2_1_p = Interleave(Cr0, Cr2, Cr1, Cr0)
Cr02_1p = Cr0_2_1_p.AssumeFieldBased().AssumeTFF().Weave()
Cr012p = Cr02_1p.AssumeFieldBased().AssumeTFF().Weave()
Cr012 = Cr012p.PointResize(Cr012p.Width(), Cr012p.Height()*3/4)
Cr012 = Cr012.TurnLeft()

Y012345 = Y012345.ConvertToYV16()
Cb012 = Cb012.ConvertToYV16()
Cr012 = Cr012.ConvertToYV16()
YToUV(Cb012, Cr012, Y012345)

#This is commented out because I don't think its necessary. ## This IS required for some resolutions, DUH.
Crop(0, 0, width, height*2)
}

[Aegwyn11]

Does anyone know how to read AVI and/or MOV files with Sashimi?

Using FFmpeg, I created test.avi with v210 video, then created test.yuv from the test.avi file (using -c:v copy). I then subtracted the resulting two file sizes and put that number as my file header. My frame of video gets chopped, with some of what should be on the left showing on the right side (the amount changes as you advance frames with the amount of "frame on the left" increasing with each frame). Also, it looks like the chroma channels are getting reversed frame to frame.

Are there frame headers as well as a file header? I can use FFmpeg to copy my files to .yuv files, but I'd rather be able to read the avi's and mov's directly (saves time, effort, and disk space).

[kolak]

Use patched ffms2 source plugin.

[tin3tin]

Strange that this thread was resurrected just as I needed it. :-)

I'm trying to import a 14 bit RAW file from a Canon 60D with Magic Lantern. 1280x720.

I've tried the various examples, but unfortunately there was no 14 bit example.

https://www.wetransfer.com/downloads/eeacf5f23acbb7fe498be86a96e04a2320130607200431/412272183bda689761feb86f4ae7de8820130607200431/4cf428

This shows some resemblance with the original image:
LoadPlugin("Sashimi.dll")
RawReader("M03-1651.RAW", "RGB", 1280,720,packing="14")

But I don't know the "packing" of Magic Lantern Raw. Would be great if Avi/Vapour-synth could import those files for further correction.

[Aegwyn11]

Strange that this thread was resurrected just as I needed it. :-)

I'm trying to import a 14 bit RAW file from a Canon 60D with Magic Lantern. 1280x720.

I've tried the various examples, but unfortunately there was no 14 bit example.


Unfortunate for you, camera RAW modes are NOT standard. And they vary from camera to camera, sometimes even within the same manufacturer. I could help show you how to read RGB from RGB16 files, but I don't think that would be helpful here.

http://en.wikipedia.org/wiki/Raw_image_format

[Aegwyn11]

Use patched ffms2 source plugin.

I get video track unseekable. I've moved away from ffms2 in general because I've had tons of issues with frame accuracy and it crashing randomly.

[kolak]

Try using version based on ffmbc.
ffms2 is not the most stable source filter, but for uncompressed files it should work.

In other case don't use AVI, use MOV and find header offset as per instructions.

[Aegwyn11]

Try using version based on ffmbc.
ffms2 is not the most stable source filter, but for uncompressed files it should work.

In other case don't use AVI, use MOV and find header offset as per instructions.

ffms2 based on ffmbc works for v210, but doesn't support UTvideo (which I use pretty heavily and ffms2/libav seems to handle fine).

I was hoping the header offsets would be constant for a given file format, but it looks like they aren't. I was able to get things working in a mov container with fh=48, but then when I did a bigger file, its different.

I guess I can stick the libav version of ffms2 in a different directory, then make a function called UTVideoSource in the plugins directory that calls the libav version of ffms2. That seems to work okay.

Thanks!

[Yellow_]

Aegwyn11, what would be involved in supporting 16bit image sequences, png or tif in the mod of Readv210? I believe Devil supports 16bit per channel import. http://forum.doom9.org/showthread.php?p=1553428#post1553428

[Aegwyn11]

Aegwyn11, what would be involved in supporting 16bit image sequences, png or tif in the mod of Readv210? I believe Devil supports 16bit per channel import. http://forum.doom9.org/showthread.php?p=1553428#post1553428

You might be able to use FFmpeg to convert your PNG or TIF files to raw RGB16, then read using Sashimi. If you do this, its easy enough to get into AVISynth...take a look at the first 7 lines of the script I wrote for processing RGB16 to YUV10 using AVIsynth: http://forum.doom9.org/showthread.php?t=167667

I don't think you could skip this intermediate step without something like a modded ImageSource.

[Yellow_]

Thanks for the pointers. Following your suggestions I used ffmpeg to do a 16bit tiff -> 16bit raw rgb with this:

ffmpeg -f image2 -i sixteenbittest.tif -pix_fmt rgb48le -vcodec rawvideo -an -f rawvideo rgb16.rgb

Then used Avisynth 2.6.0 Alpha:

#Build RGB Stack16
MSB = RawReader("rgb16.rgb", "RGB", 1920, 1088, packing="16:0:8")
LSB = RawReader("rgb16.rgb", "RGB", 1920, 1088, packing="16:8:8")
StackVertical(LSB, MSB)

return last

1088 because it's mod16 h264 1920x1088 HD source.

Which gives me the recognisable image above the LSB. Avisynth reports RGB24 output, so not sure if I'm doing it right? Should I have double height for each? Is the packing correct, I followed your example.

Will look at flash3kyuu_deband for stacked output and maybe yuv444p16le source to stacked is a better format to work with within than Avisynth than stacked 16bit RGB?

Thanks for your help.

[Aegwyn11]

Which gives me the recognisable image above the LSB. Avisynth reports RGB24 output, so not sure if I'm doing it right? Should I have double height for each? Is the packing correct, I followed your example.

Will look at flash3kyuu_deband for stacked output and maybe yuv444p16le source to stacked is a better format to work with within than Avisynth than stacked 16bit RGB?

Thanks for your help.

Stack16 is actually stored as 8 bit (AVISynth only knows 8 bit and Stack16 is a way to "trick" it into passing 16 bits of information to 16 bit aware plugins). Even Sashimi only knows 8 bits, which is why the MSBs and LSBs have to be opened separately, then combined into the Stack16. So AVISynth reporting it as RGB24 is correct, because as far as AVISynth knows, it is RGB24! If you need to output RGB48 out of AVISynth, take a look at Dither's help...I'm pretty sure its covered.

Speaking of, read the first paragraphs of Dither's help...it probably does a better job explaining Stack16 than I did.

As far as using YUV, if your source is RGB and you need RGB output, I'd avoid doing YUV conversions. They're lossy and possibly unnecessary. If you do need YUV output, you can use the script I wrote and linked to in the previous post...it will do a better job than FFmpeg. Plus the same AVISynth 8 bit limitations apply, so even if you have YUV source files that are >8 bit, you have to do some tricks to get them into the Stack16 format (actually, loading RGB48 as Stack16 is easier).

[Yellow_]

Aegwyn11, thanks for the answers, since posting I've been working through and got something together based on your posts. My first starting point before starting the thread was infact the Dither Docs and the Dither colorspace thread but that question on that thread still goes unanswered :-( , but in the Dither Docs I didn't quite understand the importing 16bit as 3 RGB planes, now based on your scripts it's more clear. :-) Using Avisynth 2.6.0 beta allows 4:4:4 YV24, so using ffmpeg:

ffmpeg -f image2 -i sixteenbittest.tif -pix_fmt yuv444p16le -vcodec rawvideo -an -f rawvideo yuv444p16le.yuv

Then:

#Build 444yuv Stack16
MSB = RawReadPlanar("yuv444p16le.yuv", "YUV444", 1920, 1088, packing="16:0:8").ConvertToYV24()
LSB = RawReadPlanar("yuv444p16le.yuv", "YUV444", 1920, 1088, packing="16:8:8").ConvertToYV24()
StackVertical(LSB, MSB)

So now think I have two workable 'lossless' approaches depending on RGB or YCC source and delivery needs and at 8, 10 or 16bit depending. Part of the problem I was having was getting stacked output that the Dither tools could handle, simple task of making it Planar. RawReadPlanar hopefully does the trick losslessly with regard to YCC input anyway.

Thanks again.

[Aegwyn11]

So now think I have two workable 'lossless' approaches depending on RGB or YCC source and delivery needs and at 8, 10 or 16bit depending. Part of the problem I was having was getting stacked output that the Dither tools could handle, simple task of making it Planar. RawReadPlanar hopefully does the trick losslessly with regard to YCC input anyway.

Thanks again.

From what I can tell, it looks like RawReadPlanar is outputting YUY2, which is 4:2:2, even if you flag your input as YUV444. So you're losing information. The ConvertToYV24() you added just changes it back to yuv444...it doesn't give you back the information you lost.

If you really want to process without losing information, your best bet will be to use RawReader to get it in as RGB48 per the code I linked to. Once its in and in the Stack16 format, Dither can get you the rest of the way.

For getting RGB48 out, look at Dither_convey_rgb48_on_yv12(). It tells you how to trick the whole system to do it. One note is that you'll have to split the RGB Stack16 at the top of my script to individual R,G, and B components, but you can use ShowRed(clip, pixel_type="RGB24") and the correlating ShowBlue/ShowGreen for that easy enough.

[PitifulInsect]

Just a note for the purpose of thread archeology: I tweaked the binary to allow reading little-endian 32-bit (4-byte) words. It makes the v210 stuff much neater.
I've updated the documentation and sample v210 script accordingly.

[Wilbert]

I approved your attachments in your first post.

[Aegwyn11]

So yesterday I was going to use Sashimi for the first time in quite a while to read some YUV444 planar files and ran into some issues. Basically, if I try to do any reading using the helper AVSI files in VirtualDub, I get Script error: Invalid arguments to function "RawReader".

Writing seems to work.

I even get this using the included example scripts that use the helper AVSI's for reading. These don't work:
AllInterleavedConversions_Read.avs
AllInterleavedYUVFormats_Read.avs
AllPlanarFormats_Read.avs
OtherBitDepths_Read.avs

These do work (since they don't use the helper AVSI files):
AllNativeFormats_Read.avs
ExoticInMultipleCalls_Read.avs
Simple_Read.avs

My PC is Windows 10 64 bit, AVISynth is version 2.6, VirtualDub is version 1.10.4.

Please help! Thanks.