flash3kyuu_deband 1.5.1 / 2.0pre2 with native VapourSynth support [2012-12-03]

[SAPikachu]

Released 1.2.0.

[ryrynz]

Thank you SAPikachu, looking forward to the 16bit internal precision.

[SAPikachu]

Released 1.3.0. Added x64 version.

[mp3dom]

VERY very nice!

[kolak]

f3kdb_dither, but how to feed 10bit file :)


Andrew

[SAPikachu]

f3kdb_dither, but how to feed 10bit file :)


Andrew

You need a source filter that can output high bit-depth data. But I don't know any h264 source filter with high output at this time (Actually f3kdb_dither is originally made for raw source filters, not 10bit h264). Maybe someone can modify ffms2 to add this functionality.

[mp3dom]

I think kolak means not to input a 10bit AVC file but rather a raw 10bit (like v210 avi/mov). AVS plugins actually manages 8bit (truncated or dithered) from a 10bit source and BlackMagic codecs or AE internally made a colorspace conversion to RGB. What we needs is something like a v210 input plugins that outputs a (16bit?) stacked clip so flash and other dither tools can then be used to dither a 10bit clip directly and output a final 8bit clip.

[kolak]

Yes- I would like to feed v210 for dithering to 8bit.

[SAPikachu]

Sorry that I misunderstood the problem. As far as I know you need RawReader (Sashimi) and mt_lut to load the file as correct format, like http://forum.doom9.org/showthread.php?t=158985&highlight=v210 . I will try to figure it out later today.

[SAPikachu]

Just completed the script. I tested it on two samples from http://samples.mplayerhq.hu/V-codecs/v210/ (the 720p one need to be converted to mov container first). Script needs avisynth 2.6.

file_head needs to be manually figured out for other files. Open the file in a hex editor, find last "mdat" tag in ASCII view (won't be too far from start), file_head is offset of the immediately followed byte.

EDIT: There are special cases, see http://forum.doom9.org/showthread.php?p=1526877#post1526877 .


# modified from http://forum.doom9.org/showthread.php?p=1469679#post1469679

function readv210(string fn, int file_head, int frame_width, int frame_height, bool "flip") {

line_size = (frame_width * 16 / 6 + 127) / 128 * 128 # all lines are padded to 128 bytes boundary

base=RawReader (fn, format="y8", width=line_size, height=frame_height, numframes=0, filehead=file_head, framehead=0, flip=default(flip,false))


p0=base.every(4,0)
p1=base.every(4,1)
p2=base.every(4,2)
p3=base.every(4,3)

lsb0=mt_lut(p0,yexpr="x 6 <<u 255 &u")
msb0=mt_lutxy(p0,p1,yexpr="x 2 >>u y 6 <<u |u 255 &u")
out0=StackVertical(msb0, lsb0)

lsb1=mt_lut(p1,yexpr="x 2 >>u 6 <<u 255 &u")
msb1=mt_lutxy(p1,p2,yexpr="x 4 >>u y 4 <<u |u 255 &u")
out1=StackVertical(msb1, lsb1)

lsb2=mt_lut(p2,yexpr="x 4 >>u 6 <<u 255 &u")
msb2=mt_lutxy(p2,p3,"x 6 >>u y 2 <<u |u 255 &u")
out2=StackVertical(msb2,lsb2)

weave3h(out0,out1,out2)

y=last.every(2,1)
u=last.every(4,0)
v=last.every(4,2)

# return last
YToUV(u, v, y)

Crop(0,0,frame_width,0)

f3kdb_dither(stacked=true)


}

function every(clip v, int n, int offset) {#select every n bytes horizontally with offset, works on yv12 only
v
w=width
h=height
pointresize(v,w*2,h)
crop(offset*2,0,0,0).addborders(0,0,offset*2,0)#shift left offset pixels
pointresize(w/n,h)
}

function weave3h(clip a, clip b, clip c) {#horizontally weave 3 clips
a=a.turnright
b=b.turnright
c=c.turnright

interleave(a,c,b,c) # a c b c -> ac bc -> abcc
assumefieldbased
assumetff
weave
assumefieldbased
assumetff
weave

# From http://avisynth.org/mediawiki/Advanced_Scripting_Tips:

# In RGB, PointResize deletes the first of each group of 4 lines.
# However, in YUV modes, it deletes the last of each group of 4 lines.
pointresize(width,height*3/4)
turnleft
}


LoadPlugin("flash3kyuu_deband.dll")

# readv210("output.mov", 36, 640, 480, flip=true)

# readv210("coff3.mov", 1103, 176, 144)

readv210("v210_720p.mov", 36, 1280, 720)

[kolak]

Thanks a lot! I will test it (once have a bit of free time) and let you know how it behaves with v210 files.
File has to be be v210 mov- yes?

[mp3dom]

Thanks!! Now one final question... is it possible to convert a v210 AVI into MOV without recompress/bit-depth lost/colorspace conversion? (a Direct-stream copy changing only the container). Thanks.

[kolak]

FFmpeg can do it- just use vcodec copy option:)

[mp3dom]

Doh! Thanks! :)

[kolak]

Just completed the script. I tested it on two samples from http://samples.mplayerhq.hu/V-codecs/v210/ (the 720p one need to be converted to mov container first). Script needs avisynth 2.6.

file_head needs to be manually figured out for other files. Open the file in a hex editor, find last "mdat" tag in ASCII view (won't be too far from start), file_head is offset of the immediately followed byte.



It works ! :)

I had some 5sec v210 sample- some footage from Arri Alexa.

Does it do dithering inside function?
f3kdb_dither(stacked=true) - can I customise settings here?
If I remove f3kdb_dither(stacked=true) does it mean I can pass data to other tools which support 10bit in stacked mode?

Why does it have to be MOV?

Going to do some tests :)

Thanks a lot,
Andrew

[kolak]

I found some small problem on v210 file generated by AE (my other file is fine).

http://img196.imageshack.us/img196/6905/unledlfk.png

Top left corner has few bad pixels- files is 1920x1080.

Any idea why- is it AE?

Andrew

[poisondeathray]

I found some small problem on v210 file generated by AE (my other file is fine).

Top left corner has few bad pixels- files is 1920x1080.

Any idea why- is it AE?




No, I don't think it's AE

I noticed this in avspmod or vdub (just the readv210 function), but the defect was on bottom right for me

I used the greyscale v210 ramps I prepared earlier in the v210 thread
http://www.mediafire.com/?cxc8bmw5haceclo

[kolak]

I'm trying with AVIs and it shows picture, but it all gets shifted. Is it due to some padding in AVI container? RIFF chunk list shows data and some padding between chunks- is this padding causing problems? I also notice that some software will create AVIs with constant padding (so there is pattern) and some with one which changes. All bit of black magic for me :)

Does MOV stores all data in one go (more like raw yuv file)?

[kolak]

No, I don't think it's AE

I noticed this in avspmod or vdub (just the readv210 function), but the defect was on bottom right for me

I used the greyscale v210 ramps I prepared earlier in the v210 thread
http://www.mediafire.com/?cxc8bmw5haceclo

Hmmm- AE creates problematic AVIs- I had strange problems- some software don't like these streams. My other sample (coming from MAC I assume- no it comes from Adobe software also!) works fine.

[poisondeathray]

Hmmm- AE creates problematic AVIs- I had strange problems- some software don't like these streams. My other sample (coming from MAC I assume) works fine.


But the samples I posted were MOV wrapped :)


What v210 decoder are you using in your avs script for your screenshot ? (or are you using the script in this thread?)

[kolak]

Yes- script from this thread and Vdub.
I use my own samples- some work, some (one from AE) have this small artefacts- not sure why. They work fine in other apps which can read v210.

AVIs don't work.

[poisondeathray]

Yes- script from this thread and Vdub.
I use my own samples- some work, some (one from AE) have this small artefacts- not sure why. They work fine in other apps which can read v210.

AVIs don't work.



For AE, I notice it depends on the decoder. Sometimes internal, vs. BM, vs. AJA, vs. Drastic will give different results, it's unpredictable . When you hover over the asset in the clip bin, it will tell you which decoder it is using. You can uninstall that decoder and use another (restart AE)

But that defect in your screenshot I'm fairly certain is from the avs script

[kolak]

IT's other way around- in AE I don't have problems. With files (AVIs mainly) created by AE I have sometimes strange problems- something to do with headers which AE creates. It always puts lots of metadata there. Re-savin file in Vdub fixes this- strange.

[mp3dom]

Tried to convert v210 AVI to MOV with ffmpeg but using the above script I get all messed colors (pink become cyan, reds become blu, light yellow become light green and so on).

[poisondeathray]

Tried to convert v210 AVI to MOV with ffmpeg but using the above script I get all messed colors (pink become cyan, reds become blu, light yellow become light green and so on).


Did you use the correct file_head value with the hex editor ?

I found the same results as you when using the incorrect values

The only problem I still see is the defective corner pixels

[SAPikachu]

@kolak @poisondeathray
I tried the grayscale ramp sample but I don't get any defective pixels, I used the following line:


readv210("radialramp_1080p29.97_16bitRGB_v210.mov", 48, 1920, 1080)


Maybe file_head is wrong?

@kolak
Yes, f3kdb_dither does Floyd-Steinberg dithering by default. You can see readme.txt for parameter details. But only change "mode" and "keep_tv_range" as other parameters are to specify source data format. To get stacked 16bit clip just remove the f3kdb_dither line.

Need to be mov because avi may add variable-length data between frames. Actually raw v210 bitstream may also work, but I haven't test that.

[poisondeathray]

@kolak @poisondeathray
I tried the grayscale ramp sample but I don't get any defective pixels, I used the following line:


readv210("radialramp_1080p29.97_16bitRGB_v210.mov", 48, 1920, 1080)


Maybe file_head is wrong?


Thanks for reply,

yes, file_head was wrong. I used 64

Can you describe how you got "48" ? I don't know much about hex editors... I highlighted "mdat" and the value next to it was 64

[SAPikachu]

I use WinHex. Here is an demo:

https://lh5.googleusercontent.com/-mnbVS0B7Nas/TnFKHbDepEI/AAAAAAAAAcA/72Is0h6_Ek0/s800/hex.JPG

(Note the offset value is hex number, need to be converted to decimal for parameter)

EDIT: There are special cases, see http://forum.doom9.org/showthread.php?p=1526877#post1526877 .

[poisondeathray]

Thank you for explanation and screenshot, I understand now :)

So "30" from the "offset" is expressed in hexadecimal, but is "48" in decimal value

[SAPikachu]

Yes, that's right. :)

[mp3dom]

Did you use the correct file_head value with the hex editor ?

I found the same results as you when using the incorrect values

The only problem I still see is the defective corner pixels

I've used the right value, I think... the byte next to 'mdat' is at location 1C and I've used 28 as offset. The image doesn't have any corruption, just wrong colors.

Edit: OK, I get it... after a bit of 'brute forcing', using 36 was successfully. But 36 (hex: 24) it's not immediately after mdat. The previous bytes contains '10'.
Anyway, If I want to deband the clip with 16bit precision, can I use flash3kyuu? Should I put it just before the call to f3kdb?

[SAPikachu]

I've used the right value, I think... the byte next to 'mdat' is at location 1C and I've used 28 as offset. The image doesn't have any corruption, just wrong colors.

Edit: OK, I get it... after a bit of 'brute forcing', using 36 was successfully. But 36 (hex: 24) it's not immediately after mdat. The previous bytes contains '10'

36 is not correct. If you don't see corrupted pixel, 28 is probably right value. Try adding SwapUV after loading the file?

EDIT: I was wrong. Please see http://forum.doom9.org/showthread.php?p=1526877#post1526877 for details.

[SAPikachu]

.
Anyway, If I want to deband the clip with 16bit precision, can I use flash3kyuu? Should I put it just before the call to f3kdb?

It is not possible with f3kdb now. I will add high bit-depth input in next version. Right now you can use the dither package. Just remove f3kdb_dither and treat it as stacked 16bit clip.

[kolak]

@kolak @poisondeathray
I tried the grayscale ramp sample but I don't get any defective pixels, I used the following line:


readv210("radialramp_1080p29.97_16bitRGB_v210.mov", 48, 1920, 1080)


Maybe file_head is wrong?

@kolak
Yes, f3kdb_dither does Floyd-Steinberg dithering by default. You can see readme.txt for parameter details. But only change "mode" and "keep_tv_range" as other parameters are to specify source data format. To get stacked 16bit clip just remove the f3kdb_dither line.

Need to be mov because avi may add variable-length data between frames. Actually raw v210 bitstream may also work, but I haven't test that.

I'm not talking about ramp file- I have created file in AE (which is actually also ramp file- hehe) and I have issue on this file. I have also other file and this one is fine.

I will post this file, so we can verify- I believe my ofset is correct, but we can double check.

AVIs- yes, I've noticed that some v210 have not constant dummy data, but some do have constant. I can also post such a sample.

Andrew

[qyot27]

Are there certain hardware requirements (CPU, RAM, etc.) that need to be met? I noticed that flash3kyuu_dither has a base requirement of an SSE2-capable processor, but there's no such warning in the README's flash3kyuu_deband section.

Even with opt=0 set, I'm getting an "Evaluate: System exception - Illegal instruction" error pointing to the line where flash3kyuu_deband gets called, and I can only see that error when trying to load the script into VirtualDub - if I try to play the script in WMP6.4 the error text doesn't show up at all. Commenting it out allows the script to load as usual.

I wouldn't be surprised if it's just that my hardware is way too old (Coppermine-based Celeron = no SSE2 at all), but I would have thought that since the opt parameter defaults to autoselecting the right instruction set level and opt=0 disables optimizations, that it wouldn't error out like that, and just be a lot slower.

[kolak]

I've tried radialramps (http://www.mediafire.com/?cxc8bmw5haceclo) 1080 file and this works fine for me also.

I came back to my problematic file and.... works fine:)
Sorry for wrong report- it was an user error :)

I also tried yet another file- ProRes from Alexa converted to AJA v210 in QT- also works fine :)

[kolak]

readv210 works with RAW v210 files also :) (ofset=0)


Andrew

[SAPikachu]

@kolak
No problem, it is great to get it working. :)

If the AVI file has constant frame header, it is possible to read it by setting framehead in the script. But I think it is still easier to convert the file first.

@qyot27
That's weird, I checked the disassembly, C version don't have SSE-related code (except memset, but it will check CPU at runtime to select instruction set), and should be runnable on Coppermine... I don't have that CPU so I can't find the root cause, but I suspect it is an ICC problem. Can you try this build (http://nmm.me/x), with and without opt=0, and see if it works? Thanks.

[mp3dom]

No problem, it is great to get it working. :)

Yes, definitely! It's great to have the possibility to correctly manage 10bit in AVS and use dither/debanding plugins without colorspace conversion!
Thanks for your efforts!

[qyot27]

@qyot27
That's weird, I checked the disassembly, C version don't have SSE-related code (except memset, but it will check CPU at runtime to select instruction set), and should be runnable on Coppermine... I don't have that CPU so I can't find the root cause, but I suspect it is an ICC problem. Can you try this build (http://nmm.me/x), with and without opt=0, and see if it works? Thanks.
The new build worked in both instances, with or without opt=0. Thanks for taking a look.

[SAPikachu]

The new build worked in both instances, with or without opt=0. Thanks for taking a look.

Got it. :) So ICC generates instructions that older CPU can't support, I need to maintain a msvc version for them.

[kolak]

mp3dom- don't use it on files which are re-wrapped from AVIs- it looks like it does not work. Padding data is not constant, so there are distortions- different on different frames.
You can always force ffmpeg to re-encode video and than put to MOV container- I think it can do v210 to v210 properly.


Andrew

[mp3dom]

Tried -vcodec copy and -vcodec v210 on ffmpeg and both options outputs the exact same file. It's bit per bit identical. Regarding my offset (28) using SwapUV I see correct colors but also some corruption (probably already there, not due to SwapUV). So it seems that 36 is the real offset (watching almost all frames, seems correct) but it's not immediately after 'mdat'

Just for reference, I have this:
http://thumbnails52.imagebam.com/14973/11ca0e149720767.gif (http://www.imagebam.com/image/11ca0e149720767)
Red arrow: What I should use
Blue arrow: What I'm using

Edit: kolak, I know hex numbers, 0x1c=28, 0x24=36

[kolak]

Make sure you understand numbers. These are HEX, so you can convert in windows calc to decimal. There is only one valid value (I believe), so another is wrong :)

28 HEX is 40 dec and this is what my file needed.

[poisondeathray]

@mp3dom - SAPikachu said to use the offset number (as in the screenshot in the previous page), so in your screenshot the offset for the next byte is "10" and converted is "16" ?

[kolak]

@mp3dom - SAPikachu said to use the offset number (as in the screenshot in the previous page), so in your screenshot the offset for the next byte is "10" and converted is "16" ?

No- ofset is 1C, so it's 28 dec. Use 28 in v210reader.

Andrew

[poisondeathray]

No- ofset is 1C, so it's 28 dec. Use 28 in v210reader.



OK I see , that makes sense . I was reading from the column, not the actual offset

Then why is he getting swapped colors ?

[mp3dom]

I don't know, probably a strange v210 AVI. I'm start to think that it can be a mac conversion from v210 QT to v210 AVI. Unfortunately the files comes from an external company, not from our capture so I don't have a lot of infos about it.

[kolak]

OK I see , that makes sense . I was reading from the column, not the actual offset

Then why is he getting swapped colors ?

It looks like re-wrapping from AVI to MOV- I and the same problem and it was also after re-wrapping.

[mp3dom]

Makes sense, but I have recompressed it again to uncompressed mov but the file is exactly the same (unless -vcodec v210 doesn't mean 'recompress to v210')