SmoothAdjust [v3.20 - Update 2015/11/15] - Page 13

kostik · 7th June 2011, 20:50

How can I use 16bit ? what command should be put in avisynth?
I get strange colors when using :
ConvertToYv16()
SmoothCurve16("0-65535;32768-32768;65535-0")

can someone write a code to get 16bit ?
thanks

Dogway · 8th June 2011, 02:05

hey kostik, I like your code but I think that is not going to work.
download dither and write the next code:

Code:

Dither_convert_8_to_16()
SmoothCurve16("0-0;32768-36768;65535-65535")
ditherpost()

Play with the values but that is a good starting point.

kostik · 8th June 2011, 08:16

thanks!

mandarinka · 11th June 2011, 22:47

I want to try processing in 16bit. I assume the filter doesn't support working from 8bit input to 16bit output, right?
Would this be correct to get the MSB/LSB stacked clip, or do I need to use ditherXpre (even though I don't need any actual dithering at that point)?

Code:

video = last
stackvertical(video,blankclip(video,color_yuv=$000000))
# and smoothtweak...
smoothtweak16(contrast=0.84)

cretindesalpes · 11th June 2011, 23:08

Yes, Dither_convert_8_to_16() does exactly that.

Anyway DitherXPre doesn't do any dithering. Dithering is the inverse operation (from 16 back to 8 bits).

mandarinka · 11th June 2011, 23:32

Oh, I see. To be honest I got a bit lost in all the functions in Dither and wasn't sure what does what (or doesn't) at all...

Thanks for the tip.

jmac698 · 17th June 2011, 01:10

Interesting, I'll have to add support for your stacked kludge into my HDR tools. I use separate clips for msb/lsb because to me it didn't make sense to stack them; most filters wouldn't produce valid results if applied the same way to both bytes.
If you need to pass a highbit video in one clip, that's the only way of course.

Yellow_ · 29th June 2011, 21:38

Ok I have a bit of a problem getting SmoothAdjust to co operate with writing out 16bit image sequences via avs2yuv and imagemagik.

Basically to get SmoothAdjust 16bit functions to work it appears I need to add 'Dither_convert_8_to_16()' otherwise I get the same split frame as Kostick as posted above.

BUT adding 'Dither_convert_8_to_16()' works so far until I try to use:

Dither_convert_yuv_to_rgb(matrix="601", tv_range=false, cplace="MPEG2", chromak="bicubic", output="rgb48yv12")

And then pipe out via avs2yuv to Imagemagick I get this error:

Dither_resize16: Cannot allocate buffer memory.

And no images get written.

But if I remove the: 'Dither_convert_8_to_16()' and hash out the SmoothAdjust stuff I can write 16bit image sequences.

Is SmoothAdjust msb/lsb stack not be implemented the same as Dither 1.9?

cretindesalpes · 29th June 2011, 22:20

Your problem comes from Dither_convert_yuv_to_rgb() running very quickly out of memory (it uses Dither_resize16() internally). I'm currently working on this issue, to make Dither_resize16 less memory hungry.

Yellow_ · 29th June 2011, 22:40

Ok, thanks for the explanation, wasn't sure if I was using the functions correctly.

Dogway · 30th June 2011, 08:52

It would be cool if you added lsb_in feature for inputting 16bits stacked clips. Besides correcting chroma in debug, in case it is a bug.

edit: Also there seems to be a bug in UV planes with smoothcurve16. It is less red than default smoothcurve. Maybe it is related to dither, who knows...

LaTo · 3rd July 2011, 15:18

Quote:

Originally Posted by Dogway

It would be cool if you added lsb_in feature for inputting 16bits stacked clips. Besides correcting chroma in debug, in case it is a bug.

edit: Also there seems to be a bug in UV planes with smoothcurve16. It is less red than default smoothcurve. Maybe it is related to dither, who knows...

Can you post the curve that you used? I will try to reproduce the problem...

Dogway · 3rd July 2011, 16:57

Code:

#~ SmoothCurve(\
#~ ycurve="0-9;16-30;128-128;233-220;255-255",\
#~ ucurve="0-16;255-240",\
#~ vcurve="0-16;255-240",\
#~ interp=50,smooth=10,show=false,limiter=false)

SmoothCurve16(\
ycurve="0-2304;4096-7680;32768-32768;59648-56320;65535-65535",\
ucurve="0-4096;65535-61440",\
vcurve="0-4096;65535-61440",\
interp=50,smooth=10,show=false,limiter=false)

LaTo · 3rd July 2011, 17:10

Quote:

Originally Posted by Dogway

Code:

#~ SmoothCurve(\
#~ ycurve="0-9;16-30;128-128;233-220;255-255",\
#~ ucurve="0-16;255-240",\
#~ vcurve="0-16;255-240",\
#~ interp=50,smooth=10,show=false,limiter=false)

SmoothCurve16(\
ycurve="0-2304;4096-7680;32768-32768;59648-56320;65535-65535",\
ucurve="0-4096;65535-61440",\
vcurve="0-4096;65535-61440",\
interp=50,smooth=10,show=false,limiter=false)

255 in 16 bit is 65280, so your conversion will never be the same as 8 bit.

If you want exactly the same, add 65280-65280 before 65535-65535.
It works?

PS: Why "Besides correcting chroma in debug, in case it is a bug."? What is the problem?

Dogway · 3rd July 2011, 17:24

Im just doing what the readme suggests:
For example: SmoothCurve("0-255;128-128;255-0") --> SmoothCurve16("0-65535;32768-32768;65535-0")

If I change it to 65280 I get an error: *curve have not a correct synthax!

The debug graph for UV planes is different from yours.

yours

LaTo · 3rd July 2011, 18:29

Quote:

Originally Posted by Dogway

Im just doing what the readme suggests:
For example: SmoothCurve("0-255;128-128;255-0") --> SmoothCurve16("0-65535;32768-32768;65535-0")

If I change it to 65280 I get an error: *curve have not a correct synthax!

Try:

Code:

SmoothCurve16(\
ycurve="0-2304;4096-7680;32768-32768;59648-56320;65280-65280;65535-65535",\
ucurve="0-4096;65535-61440",\
vcurve="0-4096;65535-61440",\
interp=50,smooth=10,show=false,limiter=false)

Quote:

Originally Posted by Dogway

The debug graph for UV planes is different from yours.

yours

It's the debug histogram of avisynth 2.6.x

Dogway · 4th July 2011, 02:02

Thank you, but result is the same (in other sources too). Does it work with you?

edit: smoothcurve16 doesn't have any benefit over default smoothcurve, right (quality wise)? I realised I dont need it since after that Im processing in 8b. In any case it's still worth a look on why that happens.

LaTo · 4th July 2011, 12:20

Quote:

Originally Posted by Dogway

Thank you, but result is the same (in other sources too). Does it work with you?

Oups, sorry... Try:

Code:

SmoothCurve16(\
ycurve="0-2304;4096-7680;32768-32768;59648-56320;65280-65280;65535-65280",\
ucurve="0-4096;65280-61440;65535-61440",\
vcurve="0-4096;65280-61440;65535-61440",\
interp=50,smooth=10,show=false,limiter=false)

Quote:

Originally Posted by Dogway

edit: smoothcurve16 doesn't have any benefit over default smoothcurve, right (quality wise)? I realised I dont need it since after that Im processing in 8b. In any case it's still worth a look on why that happens.

Yes, it has no benefit if your processing chain is 8bits.

Dogway · 4th July 2011, 15:06

Quote:

Originally Posted by LaTo

Oups, sorry... Try:

Are you theorizing, or are you getting instead right results? It doesn't work for me either.
You know the insights of SmoothAdjust so you'll probably want to start matching smoothcurve internal dither algorithm with Dither's one, and start from there for testing, and discard either SmoothAdjust or Dither. Although I have just tested a simple resize with Dither vs Spline36 and chroma is alright, so it doesn't look like its fault. (It could be the way it "records"/deals chroma to LSB, cretindesalpes should have more to say on this...)

By the way, overriden parameter in smoothcurve16 are.... "dither" only? Maybe it would be a good idea to delete that option.

LaTo · 4th July 2011, 17:53

Quote:

Originally Posted by Dogway

Are you theorizing, or are you getting instead right results? It doesn't work for me either.

It *should* work, but I don't use 16 bits... So I can't verify.

Quote:

Originally Posted by Dogway

You know the insights of SmoothAdjust so you'll probably want to start matching smoothcurve internal dither algorithm with Dither's one, and start from there for testing, and discard either SmoothAdjust or Dither. Although I have just tested a simple resize with Dither vs Spline36 and chroma is alright, so it doesn't look like its fault.

I haven't checked Dither's code because I don't care (*joke*)

Seriously, I don't think that the problem is the dithering.
(Have you tried with "dither=-1"?)

Quote:

Originally Posted by Dogway

(It could be the way it "records"/deals chroma to LSB, cretindesalpes should have more to say on this...)

Maybe, in SmoothAdjust I use that:

Code:

 msb[x] = (BYTE)( smooth[x]>>8 );
 lsb[x] = (BYTE)( smooth[x]    );

It should be correct...

Quote:

Originally Posted by Dogway

By the way, overriden parameter in smoothcurve16 are.... "dither" only? Maybe it would be a good idea to delete that option.

7th June 2011, 20:50	#241 \| Link
kostik Registered User Join Date: Jul 2007 Posts: 161	How can I use 16bit ? what command should be put in avisynth? I get strange colors when using : ConvertToYv16() SmoothCurve16("0-65535;32768-32768;65535-0") can someone write a code to get 16bit ? thanks Attached Images

8th June 2011, 02:05	#242 \| Link
Dogway Registered User Join Date: Nov 2009 Posts: 2,361	hey kostik, I like your code but I think that is not going to work. download dither and write the next code: Code: Dither_convert_8_to_16() SmoothCurve16("0-0;32768-36768;65535-65535") ditherpost() Play with the values but that is a good starting point. __________________ i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread

11th June 2011, 22:47	#244 \| Link
mandarinka Registered User Join Date: Jan 2007 Posts: 729	I want to try processing in 16bit. I assume the filter doesn't support working from 8bit input to 16bit output, right? Would this be correct to get the MSB/LSB stacked clip, or do I need to use ditherXpre (even though I don't need any actual dithering at that point)? Code: video = last stackvertical(video,blankclip(video,color_yuv=$000000)) # and smoothtweak... smoothtweak16(contrast=0.84)

11th June 2011, 23:08	#245 \| Link
cretindesalpes ͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿ Join Date: Feb 2009 Location: No support in PM Posts: 712	Yes, Dither_convert_8_to_16() does exactly that. Anyway DitherXPre doesn't do any dithering. Dithering is the inverse operation (from 16 back to 8 bits). __________________ dither 1.28.1 for AviSynth \| avstp 1.0.4 for AviSynth development \| fmtconv r30 for Vapoursynth & Avs+ \| trimx264opt segmented encoding

29th June 2011, 22:20	#249 \| Link
cretindesalpes ͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿ Join Date: Feb 2009 Location: No support in PM Posts: 712	Your problem comes from Dither_convert_yuv_to_rgb() running very quickly out of memory (it uses Dither_resize16() internally). I'm currently working on this issue, to make Dither_resize16 less memory hungry. __________________ dither 1.28.1 for AviSynth \| avstp 1.0.4 for AviSynth development \| fmtconv r30 for Vapoursynth & Avs+ \| trimx264opt segmented encoding

8th June 2011, 08:16	#243 \| Link
kostik Registered User Join Date: Jul 2007 Posts: 161	thanks!

11th June 2011, 23:32	#246 \| Link
mandarinka Registered User Join Date: Jan 2007 Posts: 729	Oh, I see. To be honest I got a bit lost in all the functions in Dither and wasn't sure what does what (or doesn't) at all... Thanks for the tip.

17th June 2011, 01:10	#247 \| Link
jmac698 Registered User Join Date: Jan 2006 Posts: 1,867	Interesting, I'll have to add support for your stacked kludge into my HDR tools. I use separate clips for msb/lsb because to me it didn't make sense to stack them; most filters wouldn't produce valid results if applied the same way to both bytes. If you need to pass a highbit video in one clip, that's the only way of course.

29th June 2011, 21:38	#248 \| Link
Yellow_ Registered User Join Date: Sep 2009 Posts: 378	Ok I have a bit of a problem getting SmoothAdjust to co operate with writing out 16bit image sequences via avs2yuv and imagemagik. Basically to get SmoothAdjust 16bit functions to work it appears I need to add 'Dither_convert_8_to_16()' otherwise I get the same split frame as Kostick as posted above. BUT adding 'Dither_convert_8_to_16()' works so far until I try to use: Dither_convert_yuv_to_rgb(matrix="601", tv_range=false, cplace="MPEG2", chromak="bicubic", output="rgb48yv12") And then pipe out via avs2yuv to Imagemagick I get this error: Dither_resize16: Cannot allocate buffer memory. And no images get written. But if I remove the: 'Dither_convert_8_to_16()' and hash out the SmoothAdjust stuff I can write 16bit image sequences. Is SmoothAdjust msb/lsb stack not be implemented the same as Dither 1.9?

29th June 2011, 22:40	#250 \| Link
Yellow_ Registered User Join Date: Sep 2009 Posts: 378	Ok, thanks for the explanation, wasn't sure if I was using the functions correctly.

30th June 2011, 08:52	#251 \| Link
Dogway Registered User Join Date: Nov 2009 Posts: 2,361	It would be cool if you added lsb_in feature for inputting 16bits stacked clips. Besides correcting chroma in debug, in case it is a bug. edit: Also there seems to be a bug in UV planes with smoothcurve16. It is less red than default smoothcurve. Maybe it is related to dither, who knows... __________________ i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread Last edited by Dogway; 3rd July 2011 at 10:19.

3rd July 2011, 17:24	#255 \| Link
Dogway Registered User Join Date: Nov 2009 Posts: 2,361	Im just doing what the readme suggests: For example: SmoothCurve("0-255;128-128;255-0") --> SmoothCurve16("0-65535;32768-32768;65535-0") If I change it to 65280 I get an error: *curve have not a correct synthax! The debug graph for UV planes is different from yours. yours __________________ i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread

4th July 2011, 02:02	#257 \| Link
Dogway Registered User Join Date: Nov 2009 Posts: 2,361	Thank you, but result is the same (in other sources too). Does it work with you? edit: smoothcurve16 doesn't have any benefit over default smoothcurve, right (quality wise)? I realised I dont need it since after that Im processing in 8b. In any case it's still worth a look on why that happens. __________________ i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread Last edited by Dogway; 4th July 2011 at 07:10.

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode