Understanding Levels in Firesledge's Dither

j7n · 27th June 2014, 22:40

I am using Dit her 1.19.0 and AviSynth 2.5.8 to downscale, deband and add dither noise to videos. I got Dither from here a long time ago, and have been happy with the results.

However, I am still using basic Levels() from AviSynth, even though I realize increased precision would be needed for this task more than anywhere else. I find the level adjustment in Dither difficult to understand.

The documentation mentions working with curves and converting sample range between TV and PC. So the plug-in is most certainly capable of basic level adjustment.

The docs mention "lut", which I understand stands for look-up table, as well as "reverse polish notation". These terms appear rather intimidating.

I'd like to insert level adjustment, while the picture is still in 16-bits, and avoid installing any new plug-ins to prevent conflicts with my existing stuff.

Could someone please give me a few examples of shifting the histogram up or down, or multiplying it to stretch it? No linear light, or gamma, or anything like that, yet.

colours · 28th June 2014, 06:41

RPN is not actually as intimidating as it may sound. Instead of writing a mathematical operator in between the operands (as in "1 + 1"; aka infix notation), you write it after the operands (as in "1 1 +"). RPN has the benefit of not requiring brackets since the operator precedence is simply based on where in the expression the operators are, instead of some arbitrarily decided hierarchy people came up with centuries ago.

dither_lut16("x 100 +") increases the luma of every pixel by 100, dither_lut16("x 100 -") decreases the luma likewise, and dither_lut16("x 2 *") doubles the luma of every pixel.

Keep in mind that with 16-bit video the range of valid luma values is from 4096 to 60160 for TV range and 0 to 65535 for full range; small changes like ±1 don't have any visible effect unlike with 8-bit video.

If you really don't want to learn RPN, you can still use infix notation and the mt_polish helper function to convert from infix to RPN, like dither_lut16(mt_polish("((x/65535)^2)*65535")).

raffriff42 · 28th June 2014, 08:00

SmoothAdjust plugin: SmoothLevels16
http://forum.doom9.org/showthread.php?t=154971

Your 0-255 range is now 0-65535, otherwise it's (closely) compatible with Levels:

Code:

  - SmoothLevels16( input_low, gamma, input_high, output_low, output_high, chroma, limiter, TVrange, preset,
                  Lmode, darkSTR, brightSTR, Ecurve, Ecenter, protect,
                  smooth, dither, useMT, useOPT, show, screenW, screenH, scale )

bxyhxyh · 28th June 2014, 16:19

Levels(coring=false) is

Code:

dither_lut16("x input_low - input_high input_low - / 1 gamma / ^ output_high output_low - * output_low +")

I still couldn't get Levels(coring=true)'s formula.

Also
16 (8bit) and 4096 (16bit) is not equal in avisynth!
255 (8bit) white is equal to 65535 (16bit) white.

Hence 16bit_value = 8bit_value * 257

Therefore 16 (8bit) is equal to 4112 (16bit).

Bloax · 28th June 2014, 16:38

It's 8bit_value * 256, because 255 and 65535 being the max values is simply an artifact of the values being 0 to (2^bits)-1, instead of 1 to 2^bits.

The 256x multiplier is explained by 2^16 being 2^(16-8) times bigger than 2^8. (And 2^8 just happens to equal 256)

bxyhxyh · 28th June 2014, 16:48

Quote:

Originally Posted by Bloax

It's 8bit_value * 256, because 255 and 65535 being the max values is simply an artifact of the values being 0 to (2^bits)-1, instead of 1 to 2^bits.

Yes, you're right.
In math
0 = 0
1 = 256
.
.
.
255 = 65280
256 = 65536
This means 65535 (16bit) > 255 (8bit)

But for avisynth, 255 (8bit) is pure white and 65535 (16bit) is pure white. Therefore in avisynth, 255 (8bit) = 65535 (16bit).
So 16bit_value = 8bit_value * 257.

cretindesalpes · 28th June 2014, 18:15

If you’re using TV-range, multiply the values by 256, so black and white are represented by 16<<16 and 235<<16 . With full range, multiply them by 257 so 0 and 255 are mapped to 0 and 65535.

StainlessS · 28th June 2014, 18:36

Quote:

Originally Posted by cretindesalpes

represented by 16<<16 and 235<<16 .

Perhaps 16<<8 and 235<<8.

cretindesalpes · 28th June 2014, 18:57

Oh yes you’re right.

j7n · 29th June 2014, 01:12

This makes sense now. Thank you for the pointers. I've used another scripting language where the operators were separated (Cakewalk), and I can understand it now.

My installation of AviSynth appears to have a problem or a conflict, which prevented me from figuring out the syntax through trial and error. If I use dither_lut* with any "expr", luma always becomes black and chroma remains untouched.

Problem also occurs in a simple script of three lines. AVISource("124.avi").dither_lut8().ditherPost()

I have updated Dither to v1.24 (1.26 didn't work on XP), with no change. Resize, dither and gradfun work fine. What could be the reason?

If I can't solve this, I'll try SmoothAdjust.

cretindesalpes · 29th June 2014, 03:04

Quote:

Originally Posted by j7n

If I use dither_lut* with any "expr", luma always becomes black and chroma remains untouched.

Problem also occurs in a simple script of three lines. AVISource("124.avi").dither_lut8().ditherPost()

With dither_lut8, "x" on input is scaled to 8 bits (0–255, 128 for neutral chroma), but must be scaled to 16 bits on output (0–65535, 32768). Therefore without argument, dither_lut8 will keep the 8-bit values, reducing the contrast by 256.

About the chroma, the dither_lut* functions work like their Masktools equivalent. If you don’t specify anything for u and v, the planes will not be processed and will contain random data, depending how the frame buffers are internally recycled by Avisynth. It could be something like the input frame content, but it could be anything else. To process the chroma, set u=3, v=3.

Quote:

Originally Posted by j7n

I have updated Dither to v1.24 (1.26 didn't work on XP)

I didn’t know that, and after a quick search it seems it is related to the switch to VS 2012. I changed the compilation target to XP for the 32-bit versions, so the next build should be compatible with XP again (same with all other projects I’m working on).

j7n · 30th June 2014, 11:30

The problem of the function not working was caused by a loaded build 36 of mt_masktools, which I had installed as a dependency for another plugin. Replacing that with build a48, as required in the documentation, made dither_lut16 work correctly. I stand corrected about what dither_lut8 does. I incorrectly assumed it was just a shorthand for convert_8_to_16 and lut16.

I have a working Levels16 based on bxyhxyh's formula.

SmoothAdjust appears to work with Dither's stacked clip format. But it currently doesn't load on the older of my two PCs, which is inconvenient. A good choice otherwise, since it's just a single DLL.

27th June 2014, 22:40	#1 \| Link
j7n Registered User Join Date: Apr 2006 Posts: 137	Understanding Levels in Firesledge's Dither I am using Dit her 1.19.0 and AviSynth 2.5.8 to downscale, deband and add dither noise to videos. I got Dither from here a long time ago, and have been happy with the results. However, I am still using basic Levels() from AviSynth, even though I realize increased precision would be needed for this task more than anywhere else. I find the level adjustment in Dither difficult to understand. The documentation mentions working with curves and converting sample range between TV and PC. So the plug-in is most certainly capable of basic level adjustment. The docs mention "lut", which I understand stands for look-up table, as well as "reverse polish notation". These terms appear rather intimidating. I'd like to insert level adjustment, while the picture is still in 16-bits, and avoid installing any new plug-ins to prevent conflicts with my existing stuff. Could someone please give me a few examples of shifting the histogram up or down, or multiplying it to stretch it? No linear light, or gamma, or anything like that, yet.

28th June 2014, 08:00	#3 \| Link
raffriff42 Retried Guesser Join Date: Jun 2012 Posts: 1,373	SmoothAdjust plugin: SmoothLevels16 http://forum.doom9.org/showthread.php?t=154971 Your 0-255 range is now 0-65535, otherwise it's (closely) compatible with Levels: Code: - SmoothLevels16( input_low, gamma, input_high, output_low, output_high, chroma, limiter, TVrange, preset, Lmode, darkSTR, brightSTR, Ecurve, Ecenter, protect, smooth, dither, useMT, useOPT, show, screenW, screenH, scale )

28th June 2014, 16:19	#4 \| Link
bxyhxyh Registered User Join Date: Dec 2011 Posts: 354	Levels(coring=false) is Code: dither_lut16("x input_low - input_high input_low - / 1 gamma / ^ output_high output_low - * output_low +") I still couldn't get Levels(coring=true)'s formula. Also 16 (8bit) and 4096 (16bit) is not equal in avisynth! 255 (8bit) white is equal to 65535 (16bit) white. Hence 16bit_value = 8bit_value * 257 Therefore 16 (8bit) is equal to 4112 (16bit). Last edited by bxyhxyh; 28th June 2014 at 16:26.

28th June 2014, 18:15	#7 \| Link
cretindesalpes ͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿ Join Date: Feb 2009 Location: No support in PM Posts: 712	If you’re using TV-range, multiply the values by 256, so black and white are represented by 16<<16 and 235<<16 . With full range, multiply them by 257 so 0 and 255 are mapped to 0 and 65535. __________________ dither 1.28.1 for AviSynth \| avstp 1.0.4 for AviSynth development \| fmtconv r30 for Vapoursynth & Avs+ \| trimx264opt segmented encoding

28th June 2014, 18:57	#9 \| Link
cretindesalpes ͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿ Join Date: Feb 2009 Location: No support in PM Posts: 712	Oh yes you’re right. __________________ dither 1.28.1 for AviSynth \| avstp 1.0.4 for AviSynth development \| fmtconv r30 for Vapoursynth & Avs+ \| trimx264opt segmented encoding

28th June 2014, 06:41	#2 \| Link
colours Registered User Join Date: Mar 2014 Posts: 308	RPN is not actually as intimidating as it may sound. Instead of writing a mathematical operator in between the operands (as in "1 + 1"; aka infix notation), you write it after the operands (as in "1 1 +"). RPN has the benefit of not requiring brackets since the operator precedence is simply based on where in the expression the operators are, instead of some arbitrarily decided hierarchy people came up with centuries ago. dither_lut16("x 100 +") increases the luma of every pixel by 100, dither_lut16("x 100 -") decreases the luma likewise, and dither_lut16("x 2 ") doubles the luma of every pixel. Keep in mind that with 16-bit video the range of valid luma values is from 4096 to 60160 for TV range and 0 to 65535 for full range; small changes like ±1 don't have any visible effect unlike with 8-bit video. If you really* don't want to learn RPN, you can still use infix notation and the mt_polish helper function to convert from infix to RPN, like dither_lut16(mt_polish("((x/65535)^2)*65535")).

28th June 2014, 16:38	#5 \| Link
Bloax The speed of stupid Join Date: Sep 2011 Posts: 317	It's 8bit_value * 256, because 255 and 65535 being the max values is simply an artifact of the values being 0 to (2^bits)-1, instead of 1 to 2^bits. The 256x multiplier is explained by 2^16 being 2^(16-8) times bigger than 2^8. (And 2^8 just happens to equal 256)

29th June 2014, 01:12	#10 \| Link
j7n Registered User Join Date: Apr 2006 Posts: 137	This makes sense now. Thank you for the pointers. I've used another scripting language where the operators were separated (Cakewalk), and I can understand it now. My installation of AviSynth appears to have a problem or a conflict, which prevented me from figuring out the syntax through trial and error. If I use dither_lut* with any "expr", luma always becomes black and chroma remains untouched. Problem also occurs in a simple script of three lines. AVISource("124.avi").dither_lut8().ditherPost() I have updated Dither to v1.24 (1.26 didn't work on XP), with no change. Resize, dither and gradfun work fine. What could be the reason? If I can't solve this, I'll try SmoothAdjust.

30th June 2014, 11:30	#12 \| Link
j7n Registered User Join Date: Apr 2006 Posts: 137	The problem of the function not working was caused by a loaded build 36 of mt_masktools, which I had installed as a dependency for another plugin. Replacing that with build a48, as required in the documentation, made dither_lut16 work correctly. I stand corrected about what dither_lut8 does. I incorrectly assumed it was just a shorthand for convert_8_to_16 and lut16. I have a working Levels16 based on bxyhxyh's formula. SmoothAdjust appears to work with Dither's stacked clip format. But it currently doesn't load on the older of my two PCs, which is inconvenient. A good choice otherwise, since it's just a single DLL.