Color banding and noise removal

[TheProfosist]

is there a way to dither when going to 16bit i.e.: Dither_convey_yuv4xxp16_on_yvxx () ?

[cretindesalpes]

is there a way to dither when going to 16bit i.e.: Dither_convey_yuv4xxp16_on_yvxx () ?

This is a planned feature. In the meantime… here is a quick and dirty hack:

Edit: rewritten, and made a function of it (but it's still a suboptimal hack)

# n = target bitdepth (8–14)
# outn = true : output is scaled to the specified bitdepth
# = false: output is kept in 16 bits, the lowest bits are set to 0 after dithering.
# other parameters: transmitted to DitherPost().
#
# Example:
#
# .avs:
# ...
# GradFun3(lsb=true) # Stack16 clip
# fslg_dither_to_n_bits (10, outn=true)
# Dither_convey_yuv4xxp16_on_yvxx ()
#
# .bat:
# avs2yuv -raw script.avs -o - | x264-10bit --demuxer raw --input-depth 10 --input-res 1280x720 --fps 24000/1001 --output video.mkv -

Function fslg_dither_to_n_bits (clip o, int n, bool "outn", int "mode", int "ampo", int "ampn", bool "dyn")
{
outn = Default (outn, false)

o
msb = Dither_get_msb ()
lsb = Dither_get_lsb ()
hr = 2

m = String (Round (Pow (2, 16 - hr - n)) - 1)
l = msb.mt_lut ("x "+m+" &u", y=3, u=3, v=3)
h = msb.mt_lut ("x 255 "+m+" - &u", y=3, u=3, v=3)

s1 = String (n - 8)
ofs = String (Round (Pow (2, 7 - hr)))
a = StackVertical (l, lsb)
a = a.Dither_lut16 ("x "+s1+" <<u "+ofs+" 256 * +", y=3, u=3, v=3)

b = a.DitherPost (mode=mode, ampo=ampo, ampn=ampn, dyn=dyn)

(outn)
\ ? fslg_dither_to_n_bits_out_n (b, h, n, hr, s1, ofs, m)
\ : fslg_dither_to_n_bits_out_16 (b, h, n, hr, s1, ofs)
}

Function fslg_dither_to_n_bits_out_16 (clip b, clip h, int n, int hr, string s1, string ofs)
{
s2 = String (16 - n)
ofs2 = String (Round (Pow (2, 15 - hr - n)))
b_msb = b.mt_lut ("x "+s1+" >>u 255 &u "+ofs2+" - 128 +", y=3, u=3, v=3)
b_lsb = b.mt_lut ("x "+ofs+" - "+s2+" <<u 255 &u", y=3, u=3, v=3)

c = mt_adddiff (h, b_msb, y=3, u=3, v=3)

StackVertical (c, b_lsb)
}

Function fslg_dither_to_n_bits_out_n (clip b, clip h, int n, int hr, string s1, string ofs, string m)
{
ofs2 = String (Round (Pow (2, 7 - (n-8) - hr)))
b2 = b.mt_lut ("x "+s1+" >>u "+ofs2+" - 128 + 255 "+m+" - &u", y=3, u=3, v=3)
h2 = mt_adddiff (h, b2, y=3, u=3, v=3)
msb = h2.mt_lut ("x "+String(16-n)+" >>u", y=3, u=3, v=3)

m2 = String (Round (Pow (2, 8 - hr)) - 1)
lsb = mt_lutxy (h2, b, "x "+s1+" <<u 255 "+m2+" - &u y "+ofs+" - "+m2+" &u +", y=3, u=3, v=3)

StackVertical (msb, lsb)
}

Note: by sending 10-bit content to x264 (instead of 8- or 16-), you make sure that it will encode the exact values instead of scaling and dithering them again. Thus, you can encode safely with a vanilla 10-bit x264 instead of having to use a patched build.

Note 2: scaling issue mentioned in the previous note is now solved in the official x264 build (from r2164).

[Yellow_]

Dither 1.9.5 (http://forum.doom9.org/showthread.php?p=1386559#post1386559) released:

Bug fixed in Dither_resize16(), displaying green bars when SSE2 optimisations are disabled. Better check of stack16 clip dimensions.


TV range is all about keeping a headroom for the ringing causing by some filters (resizers, sharpeners…), so with PC range this headroom is lost. But this is not a big difference, and the bottom headroom is cleared by the gamma/linear conversion anyway.


Yes.


GammaYUV -> linearYUV -> linearRGB will give you wrong colors. Inverse colorspace conversions should be done in the same gamma/linearity as the forward conversion. For gamma YUV to linear RGB conversion you can use:

Dither_convert_yuv_to_rgb (output="rgb48y")
Dither_y_gamma_to_linear (tv_range_in=false, tv_range_out=false)
Dither_convey_rgb48_on_yv12 (
\ SelectEvery (3, 0),
\ SelectEvery (3, 1),
\ SelectEvery (3, 2) )


You're right, it appears that BT709 doesn't use the same transfer curve as sRGB, the slope at 0 in sRGB is much steeper than in BT709.

If you want to linearize the R'G'B' components converted from the BT709 Y'CbCr, apply on each one : R = R'/4.5 if R' < 0.081 or R = ((R' + 0.099) / 1.099) ^ (1/0.45) if R' ≥ 0.081.The formula are the same as in sRGB, but with different values. I will add the BT709 mode to Dither conversion functions in a future release.

Could you clarify, do you mean with Imagemagick via avs2yuv as this will be 48bit RGB or can this be done with the stacked in avisynth?

[Zero1]

I downloaded and placed the dither package in the plugins directory and all the 16 bit stuff seems to be working fine, however I can not get gradfun3 to work (in normal YV12). I simply just get a green screen. I've deleted a bunch of plugins in an attempt to troubleshoot and this is what I'm left with:

15/05/2011 21:14 192,512 dfttest.dll
21/12/2008 22:46 26,624 DirectShowSource.dll
20/07/2011 22:03 44,994 dither.avsi
21/07/2011 08:18 233,472 dither.dll
21/06/2010 11:12 1,527,650 libfftw3f-3.dll
31/12/2010 18:13 966,144 mt_masktools-25.dll
31/12/2010 18:12 966,656 mt_masktools-26.dll
19/03/2006 20:56 430,080 mt_masktools.dll
11/09/2010 14:39 2,787 mt_xxpand_multi.avsi
20/01/2005 21:58 212,992 MVTools.dll
11/05/2011 22:09 753,664 mvtools2.dll
01/08/2005 00:08 65,536 RemoveGrainS.dll

I'm using avisynth 2.5.8. Any ideas?

[wOxxOm]

remove mt_masktools-25.dll, mt_masktools.dll, MVTools.dll and make sure that mt_masktools-26.dll is alpha48

[cretindesalpes]

Could you clarify, do you mean with Imagemagick via avs2yuv as this will be 48bit RGB or can this be done with the stacked in avisynth?
I meant this can be done with avisynth. It's just a few coefficients to change in Dither_y_gamma_to_linear and Dither_y_linear_to_gamma.

remove mt_masktools-25.dll, mt_masktools.dll, MVTools.dll and make sure that mt_masktools-26.dll is alpha48
Zero1 wrote he was using 2.5.8, so it's the other way round: keep mt_masktools-25 (alpha 48) and remove mt_masktools-26.dll

[Zero1]

Thanks for the info. I think I have inadvertantly found out what is going wrong. I tried to run dfttest() earlier and it returns and error saying it cannot load libfftw3f-3.dll.
When I moved it to another directory and used loadplugin, it tells me that it is not an Avisynth 2.5 pluginr?!

Edit
I googled and found that I needed to put that in my syswow64 dir, so now dfttest works. Also after deleting the numerous masktools dlls, gradfun3 appears to work too. Much thanks guys, these are really going to come in handy.

[Yellow_]

I meant this can be done with avisynth. It's just a few coefficients to change in Dither_y_gamma_to_linear and Dither_y_linear_to_gamma.

I've looked through dither.avsi at the above functions and quite simply I'm lost, so thanks anyway, think I'll just look out for the BT709 mode in some future version of Dither and find an alternative route for the linearizing bit in the meantime.

[MADAJ]

Hi,

I fed an 8-bit source to x264-10bit, and I converted the 8-bit to 16 bit (using dither scripts) then fed it to x264-10bit.
However color wise, I noticed that the 1st encoding is fine and looks like the 8-bit source, while the later encoding is overlayed with greenish color.
So which one is correct?

[cretindesalpes]

Is this related with this issue (http://forum.doom9.org/showthread.php?p=1517620#post1517620)? But the difference should be hardly noticeable. If you really have a green tint, first check your masktools version (must be alpha 48 or later), then post your script and encoding command.

[atra dies]

How do you do the pfclip? I want to use blur() but any other examples are welcome. I finally saw this in the dither doc and it looks pretty good. It says invalid arguments to function blur. I also tried it x 4 and as a variable.

o = last
Dither2Pre ("TTempSmoothf(maxr=2,lthresh=40,cthresh=40,strength=2,scthresh=32,pfclip=last.blur())",
overlap=0, tvopt=true)

[MADAJ]

Is this related with this issue (http://forum.doom9.org/showthread.php?p=1517620#post1517620)? But the difference should be hardly noticeable. If you really have a green tint, first check your masktools version (must be alpha 48 or later), then post your script and encoding command.

Yeah, that issue. I have the mentioned masktools version.

Script

gradfun3 #modified version where (result = last) instead of DitherPost()

Interleave (Dither_get_lsb (), Dither_get_msb ())
TurnRight ()
AssumeFieldBased ().AssumeTFF ()
Weave ()
TurnLeft ()


encoding command

avs2yuv -raw "script.avs" -o - | x264-10bit --OTHER_SETTINGS --demuxer raw --input-depth 16 --input-res 1280x720 --fps 24000/1001 --output "out.mkv" -


I just wanted to compare the two scenarios (8bit-10bit and 8bit->16bit-10bit) and see which one gives better result, but I can't judge because of the green tint in the second case.
Theoretically, which one do you think is better?

I hope the bug get solved soon.

[cretindesalpes]

How do you do the pfclip? I want to use blur() but any other examples are welcome.
Technically, it's possible to use a wrapper function around TTempSmooth and Blur, but I tried and it gave massive artifacts. This is one of the Dither1/2Pre() limitations.

but I can't judge because of the green tint in the second case.
Theoretically, which one do you think is better?
If your 8-bit source is untouched (no use of GradFun3), both methods are (should be) equivalent. If you use GradFun3, direct 16->10 bits encoding should give better results than 16->8->10 via DitherPost().

[atra dies]

Nevermind, I'm going with mdegrain sharp=2 if anything (and maybe pel=4).

I assume the output of dither 8 to 16 is supposed to have a plain green LSB(should be black?)?

Can 8 bit filters be used on msb and lsb separately (I remember you said something about processing luma separately) or is the lsb too sensitive (you mentioned protecting it with unsharp mask)? Or is it better to sharpen outside in 8 bit cause of artifacts. I'm thinking of limitedsharpenfaster here.


By the way for anyone wandering in and you don't know like me a few seconds ago, this is how you generate 16 bit with mdegrain, add lsb=true at the end. Does this process it in 16 bits or just output 16 bits? <-May be a stupid question like most of them.


super = MSuper(pel=4, sharp=2)
backward_vec2 = MAnalyse(super, blksize=4,isb = true, delta = 1, overlap=0,temporal=true)
forward_vec2 = MAnalyse(super, blksize=4,isb = false, delta = 1, overlap=0,temporal=true)
MDegrain1(super, backward_vec2,forward_vec2,thSAD=400,lsb=true)

or

super = MSuper(pel=4, sharp=2)
backward_vec2 = MAnalyse(super, isb = true, blksize=4,delta = 2, overlap=0,temporal=true)
backward_vec1 = MAnalyse(super, isb = true, blksize=4,delta = 1, overlap=0,temporal=true)
forward_vec1 = MAnalyse(super, isb = false, blksize=4,delta = 1, overlap=0,temporal=true)
forward_vec2 = MAnalyse(super, isb = false, blksize=4,delta = 2, overlap=0,temporal=true)
MDegrain2(super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400,lsb=true)

I tried all the block sizes and overlaps and temporal = true. This filter is driving me nuts but removes a lot of noise without blurring it like FFT. Still seems soft and motion looks a little blurred but not as bad as ttempsmooth. Still high thSAD settings only cause artifacts beyond 400 instead of removing more noise.

Can ditherpost or some other step convert to yuy2 or is it fine for avisynth to do that. I thought I saw that there was a bug with that command or was that just for interlaced true?

Also I'm seeking opinions on whether users remove 16mm grain mpeg2 blocking or leave as is since it is considered resolution by some.

[cretindesalpes]

I assume the output of dither 8 to 16 is supposed to have a plain green LSB(should be black?)?
Exactly.

Can 8 bit filters be used on msb and lsb separately (I remember you said something about processing luma separately) or is the lsb too sensitive (you mentioned protecting it with unsharp mask)?
It's possible in some cases but not recommended. If you want to mix 8 and 16 bit processing, see post #250 (http://forum.doom9.org/showpost.php?p=1518953&postcount=250).

this is how you generate 16 bit with mdegrain, add lsb=true at the end. Does this process it in 16 bits or just output 16 bits?
Input is 8 bits, output 16 bits and the internal processing is done in 32 bits.

Can ditherpost or some other step convert to yuy2 or is it fine for avisynth to do that.
You can do it with ConvertToYUY2(), but the conversion will probably cancel the dithering benefits on the chroma plane (luma will stay fine).[/QUOTE]


・ ・ ・ ・・・————◇◆◇————・・・ ・ ・ ・

Dither 1.10.0 (http://forum.doom9.org/showthread.php?p=1386559#post1386559) released:
Added Dither_addborders16.
Added the BT 709 curve to Dither_y_gamma_to_linear and Dither_y_linear_to_gamma, as well as the plane selectors.
Added the possibility to have static noise in DitherPost.
Added Dither_add_grain16, generating stabilized noise.
Added a dif parameter to Dither_add16.
Added the lsb and lsb_in parameters to GradFun3 to deal with 16-bit data. The function also accepts smaller radii than before.

[Yellow_]

cretindesalpes, excellent update. :-)

I have a bit of a query regarding spiking luma using Dither_y_gamma_to_linear function and conversion to RGB. :-)

I have two test video's, ones a h264 AVC off a Canon HDSLR and the other off a HDV camera both sources converted with Dither 1.10 and piped through avs2yuv to ImageMagick Q16 hdri build. Writing 16bit tifs & exr's.

Using a 709 curve with the h264AVC (BT601 Color Matrix, full range luma, 709 transfer curve) gives me spiked luma histogram, using the srgb curve doesn't.

But with the HDV video source (BT709 Color Matrix, full range luma, BT709 transfer curve) whether I use srgb or 709 curve both give smooth luma histogram and definite variation in 'exposure' which I anticipated.

The basis of the scripts is:

Dither_convert_8_to_16 ()

Dither_convert_yuv_to_rgb(matrix="709", tv_range=false, cplace="MPEG2", chromak="bicubic", lsb_in=true, output="rgb48y")

Dither_y_gamma_to_linear (tv_range_in=false, tv_range_out=false, curve="srgb")

Dither_convey_rgb48_on_yv12 (SelectEvery (3, 0),SelectEvery (3, 1),SelectEvery (3, 2) )

Obviously I'm changing the matrix="" to suit colour matrix of source and adjusting pixel dims to avoid any scaling ie: 1920x1088 and 1440x1080.

What could be causing the spiking luma with the h264? I thought the problem lay with Imagemagick colorspace handling but that appears to be ok, considering HDV source works fine.

Thanks again for a great update.

[cretindesalpes]

Using a 709 curve with the h264AVC (BT601 Color Matrix, full range luma, 709 transfer curve) gives me spiked luma histogram, using the srgb curve doesn't.

But with the HDV video source (BT709 Color Matrix, full range luma, BT709 transfer curve) whether I use srgb or 709 curve both give smooth luma histogram and definite variation in 'exposure' which I anticipated.
Do you have source samples showing these spiked histograms? How does look the spikes? 8 to 16 bit conversions always give large histogram holes, just because we spread only 256 discrete values into 65536 slots. Some additional slots are filled with overshoot, ringing, bluring or other artifacts caused by the chroma resampling, but probably not enough to cover the whole 16-bit histogram continuously.

[Yellow_]

Do you have source samples showing these spiked histograms?

I'll post a link to some examples once I get back to my machine later.

I wonder why I'm seeing this spikes when using rec709 inverse curve with h264 but not HDV, could this be more to do with compression type HDV vs h264, I don't see spikes with HDV so could this be luma deadzone quantisation in the h264 maybe?

The h264 bitrate is about 44Mbps compared to 20Mbps for the HDV, both consumer 4:2:0.

[SSH4]

Added the possibility to have static noise in DitherPost.

Great! But i think one thing missed - seed like in AddgrainC that really neded for static noise. (Imagine encoding per chapters...)

[cretindesalpes]

In static mode, the noise will remain the same and depends only on the frame size. The seed of random number generator is reset at the same value at the beginning of each plane. Therefore setting the seed explicitly is not needed.

[SilaSurfer]

Added Dither_add_grain16, generating stabilized noise.


Nice cretindesalpes to see you are working on your functions. Dither_add_Grain 16 - is that to be used for banding prevention or is it for usage to add grain to the whole frame?

[Yellow_]

I'll post a link to some examples once I get back to my machine later.

I wonder why I'm seeing this spikes when using rec709 inverse curve with h264 but not HDV, could this be more to do with compression type HDV vs h264, I don't see spikes with HDV so could this be luma deadzone quantisation in the h264 maybe?

The h264 bitrate is about 44Mbps compared to 20Mbps for the HDV, both consumer 4:2:0.

Scrub this, I see spiking in the HDV's as well at closer inspection.

Once I apply gamma correction (rec709 or srgb) after compositing (32bit float), histogram smooths again. :-)

I guess some sort of levels smoothing would happen using other Dither functions, a denoiser or such?

[SSH4]

cretindesalpes, is it possible made 1bit dithered image from gray with any of your functions from dither.dll?

small correction. what the fastest way for made this.

[cretindesalpes]

Dither_add_Grain 16 - is that to be used for banding prevention or is it for usage to add grain to the whole frame?
There is no specific purpose, it was mainly a wOxxOm request. You can use it to add visible grain or to replace the internal noise generator in DitherPost(). It was much easier to write a separate function than changing DitherPost().

cretindesalpes, is it possible made 1bit dithered image from gray with any of your functions from dither.dll?
small correction. what the fastest way for made this.
From an RGB picture in grey shades:
ShowRed ("YV12") # Or ConvertToYV12(matrix="PC.709")
StackVertical (last.BlankClip (color_yuv=0), last)
DitherPost (mode=7, u=1, v=1)
mt_binarize (threshold=0, u=-128, v=-128)Output will be full range. For YV12 TV-range, replace the mt_binarize with mt_lut ("x 219 * 16 +", u=-128, v=-128).

EDIT:
For much more accurate results, dither in linear light by replacing the StackVertical() with:
Dither_convert_8_to_16 ()
Dither_y_gamma_to_linear (tv_range_in=false, tv_range_out=false, curve="srgb", u=1, v=1)
Dither_lut16 ("x 256 /", u=1, v=1)

[SSH4]

Thanx! I thought about StackVertical-DitherPost-mt_binarize(or mt_lut) too.

Oh my god, i missed this - Dither_lut8, Dither_lutxy8, Dither_lutxyz8, Dither_lut16! :)

cretindesalpes may be possible made full 16bit (MSB/LSB) mod of masktools?

[Dogway]

Hello cretindesalpes, Im having problems with a source. Im trying to convert yuv to rgb32 skipping chroma planes all along, (output is greyscale). The code Im more confident with is the next but I tried many variations with no success:
SmoothGrad (thr=0.25, radius=16, stacked=true,u=1,v=1)

Dither_convert_yuv_to_rgb(lsb_in=true,tv_range=true,output="rgb48y")
ditherpost(mode=6,ampn=0.5,y=3,u=-128,v=-128)
MergeRGB(SelectEvery(3,0),SelectEvery(3,1),SelectEvery(3,2))

Main problems are first smoothgrad, there is no way to skip chroma and get correct results(screen is green), next is Dither_convert_yuv_to_rgb, I miss the u and v parameters, the output in this case is not 100% greyscale, you can find slight variations like 127,128,128, etc. Here ditherpost doesn't solve anything since mergergb creates these variations afterwards...

[xackphair]

I am getting a ghosting problem that I do not remember seeing before. I am just using the basic

dfttest (lsb=true)
DitherPost ()

I will post images when my lossless is done. However, if you have an idea of what is going on, please let me know. Thanks!

[cretindesalpes]

I am getting a ghosting problem that I do not remember seeing before.
It's most likely a dfttest related problem. Dfttest is not protected against ghosting, therefore you have to MCompensate/Interleave the previous and next frames first, and SelectEvery the result. Check the Dither documentation, there is an example using the MCompensate functions. You can also check dfttestMC (http://forum.doom9.org/showthread.php?t=147676) for a more complex example.


may be possible made full 16bit (MSB/LSB) mod of masktools?
I don't think so. It's a huge amount of work. Moreover, straight multi-dimension 16-bit look-up tables wouldn't work because their size would be squared (hundreds of terabytes for a single lut_xyz16).


Main problems are first smoothgrad, there is no way to skip chroma and get correct results(screen is green)
This is normal. u=1 and v=1 means "don't care about the chroma planes", and the output can be any value, including invalid combinations with the luma plane. Green screen most likely means: chroma is all 0, and the luma could be anything. Just add a GreyScale() afterwards if you want to check the luma (but don't process a stack16 clip after a GreyScale, the chroma LSB will be wrong).
next is Dither_convert_yuv_to_rgb, I miss the u and v parameters, the output in this case is not 100% greyscale, you can find slight variations like 127,128,128, etc. Here ditherpost doesn't solve anything since mergergb creates these variations afterwards...
The RGB conversion has no means to skip planes, however you are processing here the trashed chroma channels, so the conversion is definitely wrong. But if you want greyscale output, there is no need an RGB conversion. Just copy the Y channel directly to the R, G and B channels. You may also have to extend it to full range.
SmoothGrad (thr=0.25, radius=16, stacked=true,u=1,v=1)
Dither_lut16 ("x 4096 - 219 / 255 *", u=1, v=1) # TV range -> Full range
ditherpost(mode=6,ampn=0.5,y=3,u=1,v=1)
MergeRGB (last, last, last)

[Dogway]

Yeah, that was the problem, I couldn't use greyscale to LSB, besides LSB also stores chroma modification values. I also tried with u and v = -128 but output was also green. Also tried copying luma to the r,g,b channels but I missed the tv range thing, so THANKS a lot, this is something I already asked in the smoothadjust thread (equation of range conversion).

Im also interested in your opinion on my previous post (http://forum.doom9.org/showthread.php?p=1518483#post1518483)
The main problem was floyd dithering on non mdegrain'ed areas (where there is motion) EX01. I came up with the EX04 workaround so ordered dithering is always used, does the logic work or you see any flaws in the code? I have in sight modifying my smdegrain mod upon your observations...

[cretindesalpes]

I also tried with u and v = -128 but output was also green.
If the output clip is 16 bits, filling values have to be 16 bits too, so it's u and v = -128*256

Im also interested in your opinion on my previous post (http://forum.doom9.org/showthread.php?p=1518483#post1518483)
The main problem was floyd dithering on non mdegrain'ed areas (where there is motion) EX01. I came up with the EX04 workaround so ordered dithering is always used, does the logic work or you see any flaws in the code? I have in sight modifying my smdegrain mod upon your observations...
Oh I haven't realized you updated your post. I don't know if the MDegrain difference is reliable enough to build the mask. Intuitively, I would have used a combination of the MMask() SAD results, but your solution looks fine too. Maybe it's possible to get rid of the second MDegrain and compare ms2.DitherPost(mode=-1) to la2? But I suppose you probably had good reasons to use a second MDegrain. Anyway, it seems that differences between EX01 and EX04 are minor.

[Dogway]

Thanks for the input, normally I try to elaborate from your hints on dither usage, but mmask seems more reasonable, Im not sure which will be more correct or faster but my thinking was more towards "not all areas that can be mdegrain'ed (masked) are definitively going to be degrained"
What I was thinking was; mode=6 is nice to feed for further degraining/denoising, but not as left over, so what wasn't degrained over mode=6, then use the mode=0

I experimented with another source in a fast panning frame:

RAW
http://i212.photobucket.com/albums/cc35/Dogway/Misc/th_raw-2.png (http://i212.photobucket.com/albums/cc35/Dogway/Misc/raw-2.png)

with smdegrain:
20.3s (encoding time)
11.08Mb (http://www.mediafire.com/?7f0vnbykm2s8o2b)
http://i212.photobucket.com/albums/cc35/Dogway/Misc/th_sm.png (http://i212.photobucket.com/albums/cc35/Dogway/Misc/sm.png)
dfttest(tbsize=1,sigma=10,lsb=true)
ditherpost(mode=6)

smdegrain(tr=3,thSAD=400,pel=2,sharp=2,lsb=true,overlap=4)

with the EX04 code
29.7s (encoding time)
11.07mb (http://www.mediafire.com/?36t8wcrj4uqh4w8)
http://i212.photobucket.com/albums/cc35/Dogway/Misc/th_ex04.png (http://i212.photobucket.com/albums/cc35/Dogway/Misc/ex04.png)
o=dfttest(tbsize=1,sigma=10,lsb=true)
la2=o.DitherPost(mode=6)
la1=o.DitherPost(mode=-1)
over=4
super = la2.MSuper(pel=2, sharp=2)
b3vec = super.MAnalyse(isb = true, delta = 3, overlap=over, blksize=8, search=4)
b2vec = super.MAnalyse(isb = true, delta = 2, overlap=over, blksize=8, search=4)
b1vec = super.MAnalyse(isb = true, delta = 1, overlap=over, blksize=8, search=4)
f1vec = super.MAnalyse(isb = false, delta = 1, overlap=over, blksize=8, search=4)
f2vec = super.MAnalyse(isb = false, delta = 2, overlap=over, blksize=8, search=4)
f3vec = super.MAnalyse(isb = false, delta = 3, overlap=over, blksize=8, search=4)

ms=la1.MDegrain3(super, b1vec, f1vec, b2vec, f2vec , b3vec, f3vec, thSAD=400, limit=255, limitc=255,lsb=true)
ms2=la2.MDegrain3(super, b1vec, f1vec, b2vec, f2vec , b3vec, f3vec, thSAD=400, limit=255, limitc=255,lsb=true)

msk=DitherBuildMask(la1,ms.ditherpost(mode=-1))
Dither_merge16_8(o,ms2,msk)

DitherPost(mode=0)

with the mmask workaround:
Not sure if the mask matches that used in EX04...
8.9s (encoding time)
11.05mb (http://www.mediafire.com/?mk1iujah2d9a5mb)
http://i212.photobucket.com/albums/cc35/Dogway/Misc/th_mmask3.png (http://i212.photobucket.com/albums/cc35/Dogway/Misc/mmask3.png)
o=dfttest(tbsize=1,sigma=10,lsb=true)
la2=o.DitherPost(mode=6)

gam=1.0
SCD1=400
SCD2=150
over=4
super = la2.MSuper(pel=2, sharp=2)
b3v = super.MAnalyse(isb = true, delta = 3, overlap=over, blksize=8, search=4)
b2v = super.MAnalyse(isb = true, delta = 2, overlap=over, blksize=8, search=4)
b1v = super.MAnalyse(isb = true, delta = 1, overlap=over, blksize=8, search=4)
f1v = super.MAnalyse(isb = false, delta = 1, overlap=over, blksize=8, search=4)
f2v = super.MAnalyse(isb = false, delta = 2, overlap=over, blksize=8, search=4)
f3v = super.MAnalyse(isb = false, delta = 3, overlap=over, blksize=8, search=4)
SADbv3 = MMask(b3v,kind=1,gamma=gam,thSCD1=SCD1,thSCD2=SCD2)
SADbv2 = MMask(b2v,kind=1,gamma=gam,thSCD1=SCD1,thSCD2=SCD2)
SADbv1 = MMask(b1v,kind=1,gamma=gam,thSCD1=SCD1,thSCD2=SCD2)
SADfv1 = MMask(f1v,kind=1,gamma=gam,thSCD1=SCD1,thSCD2=SCD2)
SADfv2 = MMask(f2v,kind=1,gamma=gam,thSCD1=SCD1,thSCD2=SCD2)
SADfv3 = MMask(f3v,kind=1,gamma=gam,thSCD1=SCD1,thSCD2=SCD2)

SAD6 = average(SADbv3,1./6,SADbv2,1./6,SADbv1,1./6,SADfv1,1./6,SADfv2,1./6,SADfv3,1./6)
ms2=la2.MDegrain3(super, b1v, f1v, b2v, f2v , b3v, f3v, thSAD=400, limit=255, limitc=255,lsb=true)

Dither_merge16_8(ms2,o,sad6)

DitherPost(mode=0)

with the EX04 revised, getting rid of one mdegrain:
27.8s (encoding time)
11.01mb (http://www.mediafire.com/?6myzolcdkf3brr8)
http://i212.photobucket.com/albums/cc35/Dogway/Misc/th_ex04r.png (http://i212.photobucket.com/albums/cc35/Dogway/Misc/ex04r.png)
o=dfttest(tbsize=1,sigma=10,lsb=true)
la2=o.DitherPost(mode=6)

over=4
super = la2.MSuper(pel=2, sharp=2)
b3vec = super.MAnalyse(isb = true, delta = 3, overlap=over, blksize=8, search=4)
b2vec = super.MAnalyse(isb = true, delta = 2, overlap=over, blksize=8, search=4)
b1vec = super.MAnalyse(isb = true, delta = 1, overlap=over, blksize=8, search=4)
f1vec = super.MAnalyse(isb = false, delta = 1, overlap=over, blksize=8, search=4)
f2vec = super.MAnalyse(isb = false, delta = 2, overlap=over, blksize=8, search=4)
f3vec = super.MAnalyse(isb = false, delta = 3, overlap=over, blksize=8, search=4)

ms2=la2.MDegrain3(super, b1vec, f1vec, b2vec, f2vec , b3vec, f3vec, thSAD=400, limit=255, limitc=255,lsb=true)

msk=DitherBuildMask(la2,ms2.ditherpost(mode=-1))
Dither_merge16_8(o,ms2,msk)

DitherPost(mode=0)

I don't understand how the mmask workaround is so much faster than smdegrain. I don't know if it is a reliable test since the internal mask used in smdegrain and EX04 could be different. The EX04r with one mdegrain workaround compressed the best, I liked the idea but I was just unsure if comparing areas of mode=6 of dfttest and mode=-1 of mdegrain was legit.

The funny thing is here, I overlayed the mask from mmask and the mask from the difference of dfttest and mdegrain of EX04r. One would think that there is going to be 0 changes in the areas enclosed in the mmask mask (because its where there is more movement), but as you see in the DitherBuildMask, changes are everywhere.
EX04r mask .....................EX04 mask
http://i212.photobucket.com/albums/cc35/Dogway/Misc/th_mask-1.png (http://i212.photobucket.com/albums/cc35/Dogway/Misc/mask-1.png)http://i212.photobucket.com/albums/cc35/Dogway/Misc/th_maskex04.png (http://i212.photobucket.com/albums/cc35/Dogway/Misc/maskex04.png)

Sorry for the bulky post

[Dogway]

Ok I will implement finally the EX04r code, it compresses the best (tested on another anime source) and is not the slowest. The thinking has logic, and ssim compared to ex04 is the same, and probably better than smdegrain running after ditherpost(mode=6)

The only thing I still want to test is using the mmask workaround. But it's difficult to know what mask produces manalyse internally. When I try to match with the exact same settings the mmask code with the smdegrain (excluding dfttest), I get totally different results. Probably this is more a question for the mvtools thread, and even this workaround has its flaws, like passing mode=6 through static areas finally not denoised by mdegrain.

I encourage anyone who wants to test EX04r (or my 1.5d version of smdegrain below) over (old) smdegrain code to see if it compresses better under different sources and codecs.
I have run some encoding tests and ffv1 and h.264 (CRF) take less space than not using the EX04r code.

Here it is smdegrain 1.5d (http://www.mediafire.com/?4928ukpuvj85jgz) with "fake" lsb_in option. Despite compressing better in ffv1 and in same CRF in h.264, quality is slightly dropped in h.264 whether it is CRF or 2 pass, so I still consider this option experimental.

lsb_in=true against using ditherpost(mode=6) before smdegrain with lsb_in=false.

[ajp_anton]

How do I get 16-bit 4:4:4 video into x264?
(Dithered) 8-bit YV24 works directly, and 16-bit YV12 works via avs2yuv, but 16-bit YV24 gives weird results...

[cretindesalpes]

I experimented with another source in a fast panning frame
From your posted pictures, I'd say that the MMask version looks a bit more faithful to original than the others. There is a couple of fine shading that get blurred (mainly the line between the bottom of the robot and its shadow, under the sunlight halo) in the SMDegrain/EX04 versions.

How do I get 16-bit 4:4:4 video into x264?
(Dithered) 8-bit YV24 works directly, and 16-bit YV12 works via avs2yuv, but 16-bit YV24 gives weird results...
I couldn't make it work neither. However I don't think avs2yuv supports YV24 output, you should try avs2pipe26mod instead. But I've got this from x264 r2074:
resize [warning]: converting from yuv444p16le to yuv420p16le
[swscaler @ 003e53e0] full chroma interpolation for destination format 'yuv420p16le' not yet implemented

———


Updated Dither (http://forum.doom9.org/showthread.php?p=1386559#post1386559) to v1.11.0. Actually this is more a MVTools update:

MDegrainN added.
MAnalyse: Added the "multi" mode for MDegrainN.
MAnalyse: Documented the negative delta values and fixed some functions accordingly.
MFlowInter: Fixed the YUY2 planar mode.


With MDegrainN, filtering should be simpler, a bit like the MVDegrainMulti in Josey Wells' version:
tr = 6 # Temporal radius
super = MSuper ()
vmulti = MAnalyse (super, multi=true, delta=tr)
MDegrainN (super, vmulti, tr, thSAD=400, thSAD2=150, lsb=true)
DitherPost ()
I added a SAD threshold bell curve (with thSAD2) to allow the filter to be more selective with the furthest reference frames.

You can also extract single vectors from the multi-vect:
vb1 = vmulti.SelectEvery (tr * 2, 0)
vf1 = vmulti.SelectEvery (tr * 2, 1)
vb2 = vmulti.SelectEvery (tr * 2, 2)
vf2 = vmulti.SelectEvery (tr * 2, 3)
MDegrain2 (super, vb1, vf1, vb2, vf2)

I had to change more things than expected in the MVTools code to implement the multi-vector mode so I hope I didn't break anything…

[Dogway]

Thanks for the feedback!

I see the no-blurring thing more as a fault of the mmask example than an achievement. All in all what I expect is a result very near to the smdegrain example, but with some ordered dithering in a few places instead of the error diffusion. This only difference doesn't make areas to "sharpen/no-blur" as in the mmask ex. I think the same blurring could happen by just running a simple degrain script, one using the vectors from manalyse and another using mmask to mask out the mdegrain. I need to know how to match these two procedures with identical results. edit: probably Id need to be using kind=0 instead of 1 in mmask(), not sure.

I will have a look at your mvtools mod update for using it in smdegrain : P


It has a small issue I think, when I render frame in avspmod the image gets full of artifacts, they get away if I frameskip forth and back.
Also is there a way I can use MRecalculate with the vmulti vectors?

[cretindesalpes]

Indeed, and it's not restricted to AvsPmod. It's a serious issue IMHO. Don't use this version until I fix the bug! Sorry for the trouble.

[cretindesalpes]

Fixed, hopefully (Dither 1.11.1)

[Dogway]

Thanks! looks ok.
mrecalculate is not modified for vmulti, right?

[cretindesalpes]

Yes. I'll modify it later.

[Dogway]

ah, ok. No prob! I just have smdegrain v1.1d with your updates waiting in the drawer for when its ready

[cretindesalpes]

Dither 1.12.0 (http://forum.doom9.org/showthread.php?p=1386559#post1386559) is an MVTools update again:
MRecalculate can now process multi vector clips (don't forget to set the new tr parameter).
Added the -Vit-'s fix that could improve the multithreading stability.

[Yellow_]

cretindesalpes, excellent update. :-)

I have a bit of a query regarding spiking luma using Dither_y_gamma_to_linear function and conversion to RGB. :-)

I have two test video's, ones a h264 AVC off a Canon HDSLR and the other off a HDV camera both sources converted with Dither 1.10 and piped through avs2yuv to ImageMagick Q16 hdri build. Writing 16bit tifs & exr's.

Using a 709 curve with the h264AVC (BT601 Color Matrix, full range luma, 709 transfer curve) gives me spiked luma histogram, using the srgb curve doesn't.

But with the HDV video source (BT709 Color Matrix, full range luma, BT709 transfer curve) whether I use srgb or 709 curve both give smooth luma histogram and definite variation in 'exposure' which I anticipated.

The basis of the scripts is:

Quote:
Dither_convert_8_to_16 ()

Dither_convert_yuv_to_rgb(matrix="709", tv_range=false, cplace="MPEG2", chromak="bicubic", lsb_in=true, output="rgb48y")

Dither_y_gamma_to_linear (tv_range_in=false, tv_range_out=false, curve="srgb")

Dither_convey_rgb48_on_yv12 (SelectEvery (3, 0),SelectEvery (3, 1),SelectEvery (3, 2) )
Obviously I'm changing the matrix="" to suit colour matrix of source and adjusting pixel dims to avoid any scaling ie: 1920x1088 and 1440x1080.

What could be causing the spiking luma with the h264? I thought the problem lay with Imagemagick colorspace handling but that appears to be ok, considering HDV source works fine.

Thanks again for a great update.

Do you have source samples showing these spiked histograms? How does look the spikes? 8 to 16 bit conversions always give large histogram holes, just because we spread only 256 discrete values into 65536 slots. Some additional slots are filled with overshoot, ringing, bluring or other artifacts caused by the chroma resampling, but probably not enough to cover the whole 16-bit histogram continuously.

I realise now why I'm seeing spiked histogram with linearising BT601 matrix HD video, where as BT709 Matrix is smooth histogram, BT601 & 709 luma levels are slightly different, so assume Dithers linearising is 709 only. Any chance of adding a 601 linearising option at some point, if you agree that could be the issue.

[cretindesalpes]

BT.601 and BT.709 use the same formula to correct the gamma of the RGB signals, so you can use curve="709". I could add a "601" curve, but it will just map to the same formula.

[Yellow_]

Ok, thanks, I'll try again to find what might be causing the spikes, only does it when using BT601 matrix going to RGB. :-(

[SilaSurfer]

cretindesalpes Thanks for the update. Looking forward on using MdegrainN. One request if I may, would it be possible to add to your function Dither_Add_Grain_16 to be able to add 3 layers of grain like what one would get with Grainfactory3 from Didée? (hope he's well) Thanks.

[Davjd]

Hello Dear Doom9 users,
Is-it possible to input YUY2 4:2:2 (ConvertToYV16 ?)
and output with RGB48 for ImageMagick ? if yes, how ?
I try with

y = ConvertToYV12 ()
u = UToY ().ConvertToYV12 ()
v = VToY ().ConvertToYV12 ()
YToUV (u, v, StackVertical (y, y))

GradFun3 (mask=0)

Dither_convert_yuv_to_rgb(matrix="709", tv_range=false,
cplace="MPEG2", chromak="bicubic", lsb_in=true, output="rgb48y")


But it's only YV12 which is 4:2:0 ?

I'm sorry but I try to understand how everything works but I'm still lacking knowledge about all this color things.

Yellow_, how do you input your canon footage ?

Thank you.

[Yellow_]

@DavJd

FFmpegSource2("MVI_0000.MOV")

Dither_convert_8_to_16 ()

Dither_convert_yuv_to_rgb(matrix="601", tv_range=false, cplace="MPEG2", chromak="bicubic", lsb_in=true, output="rgb48y")

#Dither_y_gamma_to_linear (tv_range_in=false, tv_range_out=false, curve="709")

Dither_convey_rgb48_on_yv12 (SelectEvery (3, 0),SelectEvery (3, 1),SelectEvery (3, 2) )


I've hashed out the linearising line, in case you don't want that, if you're writing to EXR then should linearise first. It's linearised based on inverse of 709 curve so output is Scene Linear.

The above is 16bit, ie to 10bit x264 or 16bit image sequences via IM. You'll need to dither back to 8bit if you want 8bit output.

Then use Avs2yuv or probably better to use Avs2pipemod to either x264 or IM like this from Dither.html that comes with the plugin. ;-)

avs2yuv -raw "script.avs" -o - | x264-10bit --demuxer raw --input-depth 16 --input-res 1280x720 --fps 24 --output "out.mp4" - [Delete the - at the end]

avs2yuv -raw "script.avs" -o - | convert -depth 16 -size 1280x720 rgb:- "out-%06d.tif"

[cretindesalpes]

You could use YV16 on avisynth 2.6 instead of YUY2. But for whatever reason, passing YV16 to Dither_convert_yuv_to_rgb doesn't work. Before I fix this issue, you can convert to YV24 instead:
ConvertToYV24 ()

GradFun3 (mask=0, lsb=true)

Dither_convert_yuv_to_rgb(matrix="709", tv_range=false,
\ cplace="MPEG2", chromak="bicubic", lsb_in=true, output="rgb48y")
GradFun3 outputs 8 bit, you must specify manually that you want 16 bit output.

Edit:

Bug found. dither.avsi, lines 245 and 254, replace && with ||.

[ajp_anton]

I couldn't make it work neither. However I don't think avs2yuv supports YV24 output, you should try avs2pipe26mod instead. But I've got this from x264 r2074:
resize [warning]: converting from yuv444p16le to yuv420p16le
[swscaler @ 003e53e0] full chroma interpolation for destination format 'yuv420p16le' not yet implementedI get that too, and the output is still messed up.
So there's no way to encode "real" 10-bit 4:4:4 videos?

edit:
Nevermind. Of course I figured it out just after I posted that. You have to add "--input-csp i444" to x264. "--input-csp yv24" looks like the chroma channels are swapped or something, but i444 works fine.

[cretindesalpes]

No it doesn't work. It looks OK, but the 4:4:4 colorspace is converted to 4:2:0. Or did you use a specific option to force the encoding in 4:4:4 colorspace? Anyway it should be possible to do it, this sample (http://www.cccp-project.net/beta/test_files/H.264-444-10bit-sample.mkv) looks like it has been encoded with x264.