Color banding and noise removal

Originally Posted by TheProfosist

im getting crazy slower speed with smode=1 as well

smode=0 1.87FPS
smode=1 0.12FPS
this is with 1080p

Seems I wasnt the only one who had this problem. I guess Ill just have to wait it out and hope it gets faster/fixed in time.

[cretindesalpes]

Originally Posted by Alek93j

so, it would be:
super = MSuper (src,levels=1)
to get the same result with a little increase of the velocity?

Yes, you're right. No need for all levels in MDegrain and MCompensate.

Originally Posted by mswaino2

Seems I wasnt the only one who had this problem. I guess Ill just have to wait it out and hope it gets faster/fixed in time.

Have you run the test in post #211? How does it compare to my results? I don't know where your problem comes from, so I need a bit more information.

Originally Posted by Dogway

hey I just needed to use addborders inside stacked 16b. Thought on sharing the function (primitive but useful).
[...]
btw Im also looking forward the v&c implementation

Thank you. I added it to the next Dither release, but with the LSB part being a true 0.
What do you mean "v&c"?


SilaSurfer & mandarinka: thank you for your kind words.

[Dogway]

Originally Posted by cretindesalpes

What do you mean "v&c"?

Void & Cluster algo. I also recall reading something related to it a few months ago, but I can't talk since I have no idea on dither algorithms. Did you test it?

[cretindesalpes]

OK. No, I haven't tested it yet.

[Dogway]

I had some time to look at making work nnedi with dither. I also wanted to contrasharp after dfttest and before nnedi so I mixed both workarounds. Can you confirm it is correct?
I also added a mask for nnedi because it was introducing some banding/artifacts at flat/gradient areas...

This is ultimately going to be processed by smdegrain(lsb=true) so I dither it down to mode=6. Is this ok, or should I use mode=-1?

raw=last

predf=dfttest(sigma=10,tbsize=1,lsb=true)
post=predf.ditherpost(mode=-1)
contr=Contrasharpening(post,raw).LSFmod(defaults="slow",strength=30,edgemode=1,soothe=true,ss_x=1.0,ss_y=1.0)

spl=contr.Spline36Resize(640,360)
nn=contr.nnedi3_rpow2(rfactor=2,cshift="spline64resize",nns=4,qual=2,pscrn=4)
hop=mt_edge (mode="prewitt", thY1=10, thY2=30).mt_expand.BilinearResize(640,360)
nn=mt_merge(spl,nn,hop, luma=true, y=3)

mask=DitherBuildMask(contr, post)
sharped=Dither_merge16_8(predf,contr.Dither_convert_8_to_16,mask).Dither_resize16(640,360,kernel="spline36")

mask2=DitherBuildMask(nn,spl)
Dither_merge16_8(sharped,nn.Dither_convert_8_to_16,mask2)

DitherPost(mode=6,prot=true)

[SilaSurfer]

Error diffusion modes (6,7,8) are awasome when using high bitrates, otherwise mode (0) 8-bit ordered dither + noise (Bayer matrix) accept that I don't use ampn setting but rather ampo (~0.5) when using higher compression.

[SSH4]

v&c looks great but "This technique is covered by Ulichney’s U.S. patent 5535020 and the specific implementation we showed is partly covered by Epson’s U.S. patent 6088512."
But I hope you don't care about this.

[atra dies]

I'm testing some images from here:
http://www.4p8.com/eric.brasseur/gamma.html

ImageReader("C:\gamma_colors.jpg")

Dither_convert_rgb_to_yuv(matrix="601",tv_range=false,lsb=false)
Dither_convert_8_to_16()
Dither_y_gamma_to_linear(false,false)
Dither_resize16(128,192)
Dither_y_linear_to_gamma(false,false)
DitherPost(mode=-1)

What am I doing wrong? The page claims they come out wrong but the one at the bottom says the scaling software rules. Also the NASA big lights from space image comes out with the lights less yellow than the original.

I modified the linear to gamma function to process chroma but that didn't change anything. Is chroma on the power scale too? Is this plugin built just for yv12 or should it work for rgb the same way? It could be something I don't know about because I haven't read much about it.

[cretindesalpes]

Originally Posted by Dogway

Can you confirm it is correct?

Yes at the first glance it looks correct, but this becomes a bit complicated to follow... Anyway, I would have extracted the "hop" edge mask from the nnedi result rather than from the original picture, although I don't know if it makes a significant difference.

This is ultimately going to be processed by smdegrain(lsb=true) so I dither it down to mode=6. Is this ok, or should I use mode=-1?

Definitely not mode=-1, you'd lose all the benefits of this complex script. Mode 6 looks like the most appropriate dither if you're going to filter the clip again, because you need the maximum quality at this intermediate stage and you don't want the motion estimation to be tricked by the regular patterns of an ordered dithering.

Originally Posted by atra dies

What am I doing wrong? The page claims they come out wrong but the one at the bottom says the scaling software rules. Also the NASA big lights from space image comes out with the lights less yellow than the original.

This is the same problem as with the colorspace conversions mentioned a few posts above. Ideally, for a color sRGB picture, linear/gamma scaling should be done on the RGB components, not on the converted Y signal only. This modified piece of code should work as expected:

ImageReader ("gamma_colors.jpg")

Interleave (ShowRed ("YV12"), ShowGreen ("YV12"), ShowBlue ("YV12"))
Dither_convert_8_to_16()
Dither_y_gamma_to_linear(false,false) # u=1, v=1
Dither_resize16(128, 192, u=1, v=1)
Dither_y_linear_to_gamma(false,false) # u=1, v=1
DitherPost(mode=-1, u=1, v=1)
MergeRGB (SelectEvery (3, 0), SelectEvery (3, 1), SelectEvery (3, 2))

I modified the linear to gamma function to process chroma but that didn't change anything. Is chroma on the power scale too?

Chroma depends on something on a power scale. The correct conversion is related to the initial RGB->YUV matrix. Keeping the chroma untouched is technically wrong, but the formula to fix it is complicated!

[wOxxOm]

How about an option to have a static ampn noise? e.g. a negative value = static, or better an additional parameter like ampn_temp_avg=0..100 (similar to GrainFactory3) with -1 for a completely static noise.

[pandy]

is there any way to reduce bit depth with dither to less than 8 bits? (like 5 or 4 for embedded applications - quite frequently only 4 or 5 bits depth per component is supported)?

[cretindesalpes]

Originally Posted by wOxxOm

How about an option to have a static ampn noise? e.g. a negative value = static, or better an additional parameter like ampn_temp_avg=0..100 (similar to GrainFactory3) with -1 for a completely static noise.

You'll have static noise in the next Dither release. For a more fancy noise, I think it's better to provide a noise clip produced by an external source. I might implement that too.

Originally Posted by pandy

is there any way to reduce bit depth with dither to less than 8 bits?

Yes. Reduce the signal before dithering and amplify it afterwards.

bits = 4    # Valid range: 0 - 8
mul = String (Pow (2, 8 - bits))

Dither_convert_8_to_16()
Dither_lut16 (expr="x "+mul+" /")
DitherPost ()
mt_lut ("x "+mul+" *", y=3, u=3, v=3)

[SilaSurfer]

Hello guys

cretindesalpes what would be better in your opinion. I'm doing some heavyweigth filtering Mvtools style on 1080p Bluray source that is going to be encoded to x264. But my processor wont handle that kind of stress (time to get I7 I guess ). I was thinking of doing a lossless pass and then going to x264 but should I use your dithering before the lossless pass or later on lossless source before feeding it to x264? Thanks in advance

[cretindesalpes]

Originally Posted by SilaSurfer

should I use your dithering before the lossless pass or later on lossless source before feeding it to x264? Thanks in advance

I think it doesn't matter much. DitherPost() is quite light on resource, compared with 1080p MC filtering. But if the output of your first lossless pass has a 16-bit depth, the generated file will be huge (compression of the LSB part is inefficient).

[SilaSurfer]

Originally Posted by cretindesalpes

But if the output of your first lossless pass has a 16-bit depth, the generated file will be huge (compression of the LSB part is inefficient).

No I was thinking of staying in 8-bits and then on lossless as source doing:

Dither_convert_8_to_16 ()
Smoothgrad if needed
DitherPost()

[atra dies]

Originally Posted by mswaino2

Seems I wasnt the only one who had this problem. I guess Ill just have to wait it out and hope it gets faster/fixed in time.

I am getting super slow speeds with smode 1 and 2 (I think 2 is supposed to be the highest quality, so I would try that one instead) but then I tried lowering the radius from 13 to 1, got an error saying valid values are 2-64, wouldn't accept anything below 6, then it works at twice the speed so I would try lowering the radius and seeing if you get acceptable results.

I am using it to smooth horrible compression artifacts, bands of blocks for flat animation colors and nothing removes it so completely as gradfun3 .66 strength and I tried a lot of filters and pp settings. For non-animation I have found that smoothd works wonders at removing lots of noise but blurs the picture, set adaptive mode to 1. Seems promising if someone used that principle for another filter let me know. (Edit: sorry, should have tried mdegrain2, it is great but slow and one of the mod16 filters.)

Now I have to figure out where or if levels or regular denoisers can be done on linear gamma for most accurate results?

[Dogway]

Originally Posted by cretindesalpes

Definitely not mode=-1, you'd lose all the benefits of this complex script. Mode 6 looks like the most appropriate dither if you're going to filter the clip again, because you need the maximum quality at this intermediate stage and you don't want the motion estimation to be tricked by the regular patterns of an ordered dithering.

I kept wondering... smdegrain, actually any mdegrain that uses motion vectors is going to denoise only on static, or semistatic areas. This way moving areas are going to keep the floyd & steingberg dithering which consumes lots of bitrate. Do you think manalyse is really affected by the ordered dithering in the case I used mode=0 over mode=0?
Anyway this is about the easy way. I guess I can do something like, process motion analysis over mode=6 dummy, and degrain over mode=0, implement this inside the smdegrain function as a "fake" lsb_in.


In my last post I also wanted to tell you about something I observed. I didn't fully test so I was unsure to comment, but maybe you may want to check the fh and fv parameters of Dither_resize16 for next version. I think that in some occasions I couldn't get sharper results, maybe with small sources, downsizing or upsizing, can't remember. Can I assume >1.0 = sharper is always true?

@atra dies: Lately I discovered how effective dfttest can be at deblocking, this is a known fact, but the tricky thing I found is that deblocking works more effective when tbsize=1 (no temporal). try dfttest(tbsize=1,sigma=10,lsb=true).ditherpost. Im very surprised (using it every day). It blurs a little but this is unavoidable, although you can (contra)sharpen afterwards. sigma=10 looks like a fixed value, you won't likely deblock more with higher values.

[Heaud]

Originally Posted by wOxxOm

How about an option to have a static ampn noise? e.g. a negative value = static, or better an additional parameter like ampn_temp_avg=0..100 (similar to GrainFactory3) with -1 for a completely static noise.

For now you can achieve static noise on the post dithered content with addgrainc(constant=true). Set the strength to 0.8 as a good starting point and then adjust to taste.

[wOxxOm]

Originally Posted by Heaud

For now you can achieve static noise on the post dithered content with addgrainc(constant=true). Set the strength to 0.8 as a good starting point and then adjust to taste.

Actually for some types of content (and for most cartoons similar to anime) this is undesirable since the constant grain over the whole frame will be perceived as physical dust on a display device, visible everywhere except very bright areas during any panorama shift or character movement.

Adding of grain should be performed only in areas altered by DitherPost, and proportionally of course, so that there wouldn't be any abrupt grain density changes. Currently I do so by using a motion-mask, so that moving areas get moving grain applied and still areas get constant grain, with the final result applied only to areas altered by GradFun3.

[SilaSurfer]

Originally Posted by wOxxOm

Adding of grain should be performed only in areas altered by DitherPost, and proportionally of course, so that there wouldn't be any abrupt grain density changes. Currently I do so by using a motion-mask, so that moving areas get moving grain applied and still areas get constant grain, with the final result applied only to areas altered by GradFun3.

wOxxOm could you post a sample of this script? I'm interested of trying it out.