Color banding and noise removal

[real.finder]

Originally Posted by cretindesalpes

No, this is not a gamma function, and actually it makes the data going even further from the linear-light scale. The purpose is to amplify the scale of the dark parts relative the the other parts. So it acts like a luma-based multiplier on the SAD. With a slope s0 = 2.0, thSAD for the darkest parts is implicitly divided by 2. The c parameter indicates the length of the transition between the dark and normal SAD.

Here are some graphs to show how the curve looks for different s0 and c values:



Blue = original luma, green = mapped luma

Note that for high c or s0 values, the brightest parts have also their thSAD modified (in the other direction), because the slope of the curve near 1 tends to flatten.

the image no longer there, can someone upload it somewhere?

edit: got it form Dogway

[TheSkiller]

Okay, this may be a dumb question, but how do I make Dither convert RGB32 or RGB24 to RGB48y?

If I try to use dither_convert_8_to_16() when the clip is RGB24 or RGB32 I get this error message:

"Avisynth open failure:
BlankClip: color_yuv only valid for YUV color spaces
dither.avsi, line 1045
dither.avsi, line 1052"

[colours]

Originally Posted by TheSkiller

Okay, this may be a dumb question, but how do I make Dither convert RGB32 or RGB24 to RGB48y?

Interleave(ShowRed("y8"), ShowGreen("y8"), ShowBlue("y8")).Dither_convert_8_to_16()

[~ VEGETA ~]

I want to discuss something with you, which is masking the details of the video out and leave only flat areas. I didn't do much tests and I am using vapoursynth now I didn't achieve flexibility yet.

My method is this: make an edge mask and make it strong so that it includes all edges and details. Since details are somehow similar to edges. Playing around with the mask (I guess it was sobel or something) makes better results, although I didn't know how to make it expand in VS at that time.

Now that I have the details, I just inverse the result to get the no-details areas, hence, flat ares. Now I can use smdegrain without fear.

I do mask the dark areas (using levels) to safely apply f3kdb the way I want, I almost never apply filters to the whole video. However, when I want to use gradfun3, I take extra care because sometimes its results are bad... like masking out the dark areas and the extremely bright ones, then apply it to that area.

As for add_grain or GrainFactory3, sometimes I like them but they are not suitable for normal anime and I haven't used them on actual encode.


What do you think of this way? what is your way?

[sneaker_ger]

Originally Posted by ~ VEGETA ~

My method is this: make an edge mask and make it strong so that it includes all edges and details. Since details are somehow similar to edges. Playing around with the mask (I guess it was sobel or something) makes better results, although I didn't know how to make it expand in VS at that time.

I wonder if there's any plugin that works like the activity masking from Daala and could be useful for your idea. Scroll down to "Activity Masking: Putting the P in PVQ":
https://people.xiph.org/~jm/daala/pvq_demo/

[~ VEGETA ~]

Originally Posted by sneaker_ger

I wonder if there's any plugin that works like the activity masking from Daala and could be useful for your idea. Scroll down to "Activity Masking: Putting the P in PVQ":
https://people.xiph.org/~jm/daala/pvq_demo/

Seems interesting enough. What do you use to achieve what I posted?

thanks

[sneaker_ger]

I never bothered with masking. I just filter everything.

[~ VEGETA ~]

Originally Posted by sneaker_ger

I never bothered with masking. I just filter everything.

oh but what filters do you use?

[feisty2]

Originally Posted by ~ VEGETA ~

I want to discuss something with you, which is masking the details of the video out and leave only flat areas. I didn't do much tests and I am using vapoursynth now I didn't achieve flexibility yet.

My method is this: make an edge mask and make it strong so that it includes all edges and details. Since details are somehow similar to edges. Playing around with the mask (I guess it was sobel or something) makes better results, although I didn't know how to make it expand in VS at that time.

Now that I have the details, I just inverse the result to get the no-details areas, hence, flat ares. Now I can use smdegrain without fear.

I do mask the dark areas (using levels) to safely apply f3kdb the way I want, I almost never apply filters to the whole video. However, when I want to use gradfun3, I take extra care because sometimes its results are bad... like masking out the dark areas and the extremely bright ones, then apply it to that area.

As for add_grain or GrainFactory3, sometimes I like them but they are not suitable for normal anime and I haven't used them on actual encode.


What do you think of this way? what is your way?

details = mt_makediff(last, last.removegrain(20).removegrain(20).removegrain(20))
last.removegrain(20).removegrain(20).removegrain(20)
#blah blah blah to flat components
last.mt_adddiff(details)

[GMJCZP]

How do I convert a video from 12 bit to 8 bit with X264? Do I need to use Dither Tools to avoid banding?

[kgrabs]

Originally Posted by GMJCZP

How do I convert a video from 12 bit to 8 bit with X264? Do I need to use Dither Tools to avoid banding?

With --input-depth 12 in the command line, x264 will use Sierra-2-4A error diffusion to dither to the target depth.

Otherwise, with 12 bit interleaved format, this should be mostly identical to ditherpost defaults (ordered dither):

f3kdb(0,0,0,0,0,0, keep_tv_range=true, dither_algo=2, input_mode=2, input_depth=12, output_mode=0, output_depth=8)

[real.finder]

Originally Posted by kgrabs

With --input-depth 12 in the command line, x264 will use Floyd Steinberg error diffusion to dither to the target depth.

Otherwise, with 12 bit interleaved format, this should be mostly identical to ditherpost defaults (ordered dither):

f3kdb(0,0,0,0,0,0, keep_tv_range=true, dither_algo=2, input_mode=2, input_depth=12, output_mode=0, output_depth=8)

or

f3kdb_dither(mode=0, stacked=false, input_depth=12)

[GMJCZP]

Thank you very mucho to both.

I am getting a distorted image with many colors when using both formulas with f3kdb.

I do not work neither version 1.5.1 nor f3kdb-rev410.
With the f3kdb-rev410 32 bit version does not deserve the function f3kdb_dither (mode = 0, stacked = false, input_depth = 12).

[real.finder]

Originally Posted by GMJCZP

Thank you very mucho to both.

I am getting a distorted image with many colors when using both formulas with f3kdb.

I do not work neither version 1.5.1 nor f3kdb-rev410.
With the f3kdb-rev410 32 bit version does not deserve the function f3kdb_dither (mode = 0, stacked = false, input_depth = 12).

what avs you use? and by what source filter and it's version?

[GMJCZP]

Avisynth sET avisynth_20150220; ffms2-2.23.1-msvc.

Edit: I already tried with Avisynth 2.6.0 RC3 (AviSynth_150419) and same result.

[real.finder]

Originally Posted by GMJCZP

Avisynth sET avisynth_20150220; ffms2-2.23.1-msvc.

Edit: I already tried with Avisynth 2.6.0 RC3 (AviSynth_150419) and same result.

the output in your case is always 8 bit, you must use avs+, and avs+ already has bits converter too

[GMJCZP]

Originally Posted by real.finder

the output in your case is always 8 bit, you must use avs+, and avs+ already has bits converter too

I understand. The alternative is pass directly the source in x264 with the option --input-depth 12 and make and lossless x264 8 bit output for future modifications.

[sneaker_ger]

Originally Posted by kgrabs

With --input-depth 12 in the command line, x264 will use Floyd Steinberg error diffusion to dither to the target depth.

x264's dithering is based on Sierra-2-4A.

[GMJCZP]

How strange, in Simplex264 I put --input-depth 12 and I get the video with the strange colors.

[sneaker_ger]

What exactly is your script and the command Simplex264 uses? If AviSynth outputs 8 bit and you tell x264 the input is 12 bit that's gotta result in disaster, naturally.