Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 Doom9's Forum Color banding and noise removal
 Register FAQ Calendar Search Today's Posts Mark Forums Read

20th October 2011, 16:11   #321  |  Link
cretindesalpes
͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿

Join Date: Feb 2009
Location: No support in PM
Posts: 610
Quote:
 Originally Posted by SSH4 how about this kind dither LSB from 8bit to 2bit (16bit to 10bit)
Yes it works too, but I think you omitted a StackVertical() at the end. Actually I developed the more complex solution to ensure the quantized values can spread across the 2-bit levels, if required (but it's not perefect). Extreme example using ampo=4. Also, significant improvement can be done by shifting the LSB before dithering. Example showing the amplified difference between a 16-bit gradient directly dithered to 8 bits and the same gradient using the 10-bit intermediate. Here is a fixed version:
Code:
```a=Dither_get_lsb()
b=Dither_get_msb()
c1 = a.mt_lut("x 6 >>",u=3,v=3)
c2 = a.mt_lut ("x 2 << 255 &u", u=3, v=3)
DitherPost (c1,c2,mode=6, u=3, v=3)
mt_lut("x 6 <<",u=3,v=3)
StackVertical (b, last)```
Quote:
 and why ampn not work on mode 2-5 ?
Because the modes 2–5 use a piece of code different from mode 0–1 and modes 6–8, and I was too lazy to add the noise stuff to it. Moreover, I now consider these modes obsolete. The patterns for values +1/4 and +3/4 look less resilient to compression than ordered dithering, although they may be perceptually a bit better (but this is questionable). I might remove them later to simplify further code maintenance, if nobody complains.

Edit:

Quote:
 And what can give better encoding result (quality/size on same CRF) on x264-10bit encode: • dithered (ordered or floid steinberg + static noise) 8-bit source; • 16-bit (MSB/LSB) • 10-bit dithered (probably with static noise too) from 16-bit ?
I haven't a firm answer to this question, one should conduct tests with different types of content. I would think noise/grain of very low level could play a key role between choosing dithering to 8 or 10/16 bits. Compression size drops drastically when the picture looks almost stable and clean after dithering, with distinct patterns on flat areas, compared to more obvious grain.
__________________
dither 1.27.2 for AviSynth | avstp 1.0.3 for AviSynth development | fmtconv r19 for Vapoursynth | trimx264opt segmented encoding

Last edited by cretindesalpes; 20th October 2011 at 16:35.

 20th October 2011, 17:47 #322  |  Link SSH4 Registered User   Join Date: Nov 2006 Posts: 90 nice. Code: ```#put here any source with banding :) n=6 # bit dithering o=String(8 - n) r=String(n) a=last d=a.mt_lut("x " + o + " >>",u=3,v=3) a.mt_lut("x " + r + " << 255 &u", u=3, v=3) b=DitherPost (d,last,mode=6,ampn=1,staticnoise=true, u=3, v=3).mt_lut("x " + o + " <<",u=3,v=3) Interleave(a.Subtitle("orig"),b.Subtitle("dither"))``` Last edited by SSH4; 20th October 2011 at 19:11.
 23rd October 2011, 18:30 #323  |  Link SSH4 Registered User   Join Date: Nov 2006 Posts: 90 oops. why src_left not work right for me (on all version dither.dll)? http://i30.fastpic.ru/big/2011/1023/...465.png?noht=1 Dither_resize16 (1920,1080,src_left=9,src_width=-7,kernel="bicubic",a1=0,a2=0.33) while BicubicResize(1920,1080,0,0.33,src_left=9,src_width=-7) work right on already dithered source.
 23rd October 2011, 23:31 #324  |  Link cretindesalpes ͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿     Join Date: Feb 2009 Location: No support in PM Posts: 610 You're right, I spotted the bug a few days ago and it is now fixed, along with some other bugs related to Dither_resize16 and extreme horizontal/vertical resizing ratios. I need to do a bit more testing. I'll make a new release as soon as possible. __________________ dither 1.27.2 for AviSynth | avstp 1.0.3 for AviSynth development | fmtconv r19 for Vapoursynth | trimx264opt segmented encoding
 26th October 2011, 20:21 #325  |  Link SSH4 Registered User   Join Date: Nov 2006 Posts: 90 May i be a beta tester? One my project stalled because i can't out for this moment 1080p in 16/10bit. And here i made some tests with 16/10/8 bit output and encoding in 10-bit x264: BD source VC1 "opening" filtered with dfttest (16bit mod) with motion compensation. Three output: 8 bit (DitherPost(mode=6,ampn=1,staticnoise=false) 10 bit (MSB+LSB dithered with DitherPost(mode=6,ampn=1,staticnoise=false) 16 bit LSB+MSB encoded with hq settings with 2-pass in 5000K/s and 3000K/s 16 bit 5Mb/s - http://www.mediafire.com/?3bxu032qsbcyxdx 10bit 5Mb/s - http://www.mediafire.com/?oodq8xyrz6brmcj 8 bit 5Mb/s - http://www.mediafire.com/?62jlpy7m3gclbtf 16 bit 3Mb/s - http://www.mediafire.com/?1sj5iavs85xryux 10 bit 3 Mb/s - http://www.mediafire.com/?cf9l942zmc1ecaz 8 bit 3 Mb/s - http://www.mediafire.com/?yo3m33c5mrf7m8o And dumn i can't understand what look better. 8bit sometime looks better because grain a little bit more visible than 10Bit or 16. Sometime 10 or 16 looks better than 8
 26th October 2011, 21:08 #326  |  Link Keiyakusha 契約者     Join Date: Jun 2008 Posts: 1,567 SSH4 Just a random thought. Where is the source we can compare these results to? Also are you sure you not throwing away extra information in 10-16 bit files? For example you can't open them in avisynth like any other 8bit ones. What decoders you use - also matters. EDIT: also it doesn't sounds right that in 8 bits grain looks better, preserving grain and dither is the point of 10 bit. Last edited by Keiyakusha; 26th October 2011 at 21:15.
 26th October 2011, 22:18 #327  |  Link SSH4 Registered User   Join Date: Nov 2006 Posts: 90 hmm, you want me upload luseless HUFFYUV LSB/MSB avi in >10Gb?? avs2yuv rawpipe to x264-10bit avs2yuv raw dither_convey_... MSB/LSB 16bit pipe to x264 raw input 16bit and as we talk later in http://forum.doom9.org/showthread.ph...08#post1533008 10bit (2bit dithered LSB) and MSB/LSB 16bit pipe to x264 raw input 16bit.... Just check results. what result look better than other, not what result better or more like original. I understand that 16 bit without dither vs 10 bit (2-3bit dithered) or 8 bit (3-4 bit dithered) not the same sources. and probably we must use different psy-rd, tree and other settings for compare results... But anyway. with identical encode settings in same bitrate what file looks better? Because if i try encode in CRF 10/16 bit source beat 8bit (4-5Mb/s vs 8-10Mb/s)
26th October 2011, 22:49   #328  |  Link
Keiyakusha

Join Date: Jun 2008
Posts: 1,567
Quote:
 Originally Posted by SSH4 hmm, you want me upload luseless HUFFYUV LSB/MSB avi in >10Gb??
Eh, no. I was referring to unprocessed VC1 stream. Or better to say small part of it. I just realized that you didn't tell how exactly you filtered stuff so I won't be able to replicate it anyway. But whatever.
By the way I misunderstood that by 8bit you only mean 8bit source but output from x264 is still 10 bit.
To me 16 and 10 bits looks better than 8 bits. 8 bits has more artifacts, that's all. Between 10 and 16 bits my choice is 10, because 16 is an overkill, none of them is better they just different.
EDIT: however, personally I use different deband/dither so I'm not sure which one (10 or 16) bits consumes less resources in your case. If that's 16bit one, then my choice is 16bit.
BTW, I'm talking about 5mbps samples, all 3mbps looks not good to even compare.

Last edited by Keiyakusha; 26th October 2011 at 23:11.

 26th October 2011, 23:20 #329  |  Link SSH4 Registered User   Join Date: Nov 2006 Posts: 90 I think you understand why x264-10 bit can made better result on 8bit stream (specially on gradients) than 8bit x264... This is not about filtering, this question about 8/10/16 bit So my test is compare similar source but in 8/10/16 bit (3-4bit/2-3bit/0 bit dither) and encode it in 10bit x264 only. ok, 10bit looks more better for me too. on 8bit sometime looks like grain can help protect a little bit more textures than in 10/16bit. But mostly clean source better in 10/16 bit because it made less artifacts. But... clean source "CLI" ^___^ looks not real. So my choice use 10bit with add small grain in MSB for more realistic result. And i hope i can test it on my Mushishi "remaster"
26th October 2011, 23:42   #330  |  Link
Keiyakusha

Join Date: Jun 2008
Posts: 1,567
Quote:
 Originally Posted by SSH4 I think you understand why x264-10 bit can made better result on 8bit stream (specially on gradients) than 8bit x264...
Sure.
Yet again, If you filter stuff in 16 bit, I don't see reason to go to 8 as you seems to be doing in (8 bit (DitherPost(mode=6,ampn=1,staticnoise=false)).
Quote:
 Originally Posted by SSH4 But... clean source "CLI" ^___^ looks not real.
I wonder if it was really meant to look real :Р

 27th October 2011, 00:15 #331  |  Link cretindesalpes ͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿     Join Date: Feb 2009 Location: No support in PM Posts: 610 Dither 1.12.1:Fixed a bug with Dither_resize16, occuring with positive values for src_left or src_top. Fixed other issues in Dither_resize16, related to huge differences between vertical and horizontal resizing ratios. It's not perfect yet, but still much better than before. Dither_convert_yuv_to_rgb now supports RGB24 output colorspace. Simplified Dither_crop16 for pure horizontal cropping. __________________ dither 1.27.2 for AviSynth | avstp 1.0.3 for AviSynth development | fmtconv r19 for Vapoursynth | trimx264opt segmented encoding
27th October 2011, 09:19   #332  |  Link
SSH4
Registered User

Join Date: Nov 2006
Posts: 90
cretindesalpes Thank you!

Quote:
 Originally Posted by Keiyakusha Sure. Yet again, If you filter stuff in 16 bit, I don't see reason to go to 8 as you seems to be doing in (8 bit (DitherPost(mode=6,ampn=1,staticnoise=false)).
Yep, i think you right.

But:
• x264 just clip 6 bites from 16 bit, without any dither.
• 8bit data because of rounding in the encoding process take 10bit values.
• Most of sources i ever seen have or need some grain.

So i try find best workflow for me.
Original style avisynth 8bit, or avialable with dither.dll 16bit LSB/MSB, but more difficult to implement.

 30th October 2011, 18:12 #333  |  Link Dogway Registered User   Join Date: Nov 2009 Posts: 981 is it possible to add u and v to dither_merge16_8? Also is there a way to extract a plane of a 16 bit clip? I suspect this involves using luts functions...
 30th October 2011, 23:21 #334  |  Link cretindesalpes ͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿     Join Date: Feb 2009 Location: No support in PM Posts: 610 Yes, I'll probably rewrite this function as a native plugin, and then I will add the y, u and v plane selectors. But what do you mean "extract a plane"? __________________ dither 1.27.2 for AviSynth | avstp 1.0.3 for AviSynth development | fmtconv r19 for Vapoursynth | trimx264opt segmented encoding
 31st October 2011, 00:14 #335  |  Link Dogway Registered User   Join Date: Nov 2009 Posts: 981 The idea came from the Color Vibrance thread Vtoy(),Utoy() and YToUV(). I didn't know how to do that in 16 bit. There might be more uses for these functions than I can think now, maybe it's not important, but I felt a bit limited when I thought about it. Same for mergechroma and mergeluma. edit: To put an example: 16 bit input ... o=ditherpost(mode=6) a=Dither_lutxy8(o,o.SwapUV(),"x 128 - 2 ^ y...",y=2,u=3,v=3) ... 16 bit output (but luma comes from a ditherpost stage) So now I need to mix in 16 bit the luma previous to the ditherpost stage with the chroma of the dither_lutxy8(). My initial thoughts are: Dither_merge16(last,a,mt_lut("65535",u=-65535,v=-65535),y=2,u=4,v=4) Then I guess this can become sort of a mergeluma, mergechroma workaround for 16bit. Greyscale or how to get Y plane: dither_lut16("x",y=2,u=-128,v=-128) Ditherpost if applied in this case dithers chroma too despite being flat, so one gets 129 values in UV planes. About UtoY, VtoY, YtoUV, I have no idea, nor I know if they would be of any use in 16 bit. I'm just thinking loud I guess, so sorry for that... Last edited by Dogway; 31st October 2011 at 08:13.
 31st October 2011, 08:58 #336  |  Link cretindesalpes ͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿     Join Date: Feb 2009 Location: No support in PM Posts: 610 You can use all these functions (UToY, VToY, MergeXXX, …) with 16 bit clips. Of course, for functions with multiple clip arguments, all clips must be of the same bitdepth. __________________ dither 1.27.2 for AviSynth | avstp 1.0.3 for AviSynth development | fmtconv r19 for Vapoursynth | trimx264opt segmented encoding
 5th November 2011, 07:29 #337  |  Link redfordxx Registered User   Join Date: Jan 2005 Location: Praha (not that one in Texas) Posts: 791 Hi I am little bit confused with so many parameters with the Dither_resize16, so I ask plainly. What should I do, when I need to do on the stacked 16bit clip exactly same thing as following does on 8bit: Code: `mt_convolution(horizontal=h,vertical=v,total=t,...)` Thank you
 5th November 2011, 18:50 #338  |  Link cretindesalpes ͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿     Join Date: Feb 2009 Location: No support in PM Posts: 610 Currently, there is no direct mapping between mt_convolution and Dither_resize16. You'll have to scale the kernel coefficients manually (dividing them by t, if required). Also, it's not possible to use a different kernel for vertical and horizontal directions, so you'll have to do it in 2 passes: Code: ```Dither_resize16 (Width(), Height()/2, kernel="impulse "+h, fh=-1, center=false) Dither_resize16 (Width(), Height()/2, kernel="impulse "+v, fv=-1, center=false)``` I'll add some parameters in a next release to make Dither_resize16 more compatible with mt_convolution. __________________ dither 1.27.2 for AviSynth | avstp 1.0.3 for AviSynth development | fmtconv r19 for Vapoursynth | trimx264opt segmented encoding Last edited by cretindesalpes; 6th November 2011 at 01:59.
 5th November 2011, 19:04 #339  |  Link vampiredom Registered User   Join Date: Aug 2008 Posts: 233 One simple thing I have done is to use FFT3dFilter, with fairly strong sigma, then blend that with the original clip to a certain percentage, like: Code: ```orig = last filt = FFT3DFilter(sigma=2) Merge(filt, orig, weight=0.25)``` This seems to do a fairly good job of reducing visible noise while maintaining detail and avoiding banding (depending on the source, of course).
5th November 2011, 19:52   #340  |  Link
redfordxx
Registered User

Join Date: Jan 2005
Location: Praha (not that one in Texas)
Posts: 791
Quote:
 Originally Posted by vampiredom One simple thing I have done is to use FFT3dFilter, with fairly strong sigma, then blend that with the original clip to a certain percentage, like:
My personal experience:
dfttest keeps more detail than dctfilter

 Tags color banding, deblocking, noise reduction