AVS Softlight

[Argaricolm]

Example (https://imgsli.com/MjIyMzA1)
Brightness example (https://imgsli.com/MjM3MTEx)

Realization of CUDA soflight negative average blend.

Plugin is x64 (CUDA toolkit 12.8 & 11.8)

You could see on Youtube videos about removing color cast using Photoshops softlight blend of negative average. This is a CUDA realization of it that process every frame.
Input should be in PC color range (output will be too)! Use mode 8 & 9 to convert to full range and back.
And I suggest to remove noise from input before processing.

Parameters:

Softlight(mode, formula, skipblack, yuvin, yuvout, rangemin, rangemax, changerange)

All parameters are optional.

mode = 0-12 (0 is default)

Can be used like this: Softlight() same as Softlight(0)

Mode 0 (default):

YUV->RGB conversion
Calculates sums of all pixels in R,G,B planes (for each plane).
Get average from these sums (sum / number of pixels).
Get negative from this sum (255 - sum)
Use softlight blend of each plane with above negative. After this step we have same as photoshop does. But brightness of frame will be changed. To have brightness intact we need to restore it to original. That what other steps do.
We get HSV planes. V plane from orignal image (RGB => V). And HS from result after softlight. Then we do HS(changed) + V(original) -> RGB -> YUV
So first mode will neutralize only colors (hue + saturation) in frame and not brightness (volume).

Also keep in mind that you better remove black bars in video for correct processing (if there are any). Or they will affect average sum.

1 mode: Same as mode 0 but planes S & V restored to their original values. So this mode only normalizes lightness/brightness and does not change colors.

2 mode: Same as mode 0. But plane S is also boosted (softlight is done for each pixel with itself). So it neutralises colors and boost contrast.

3 mode: Same as mode 0 but without brightness restoration. Use it if you want to make brigtness also average (makes dark frames brighter).

4 mode: Same as mode 3 but each of RGB planes are boosted using softlight (contrast boost).

5 mode: YUV->RGB->softlight each RGB plane with itself->YUV (color/contrast boost).

6 mode: YUV->RGB->HSV->boost S->RGB->YUV (boost saturation).

7 mode: Limited color range clamping. Some videos with limited color range contain values < 16 and > 235. This mode change them to 16 & 235. This mode is not needed after mode 9.

8 mode: TV to PC color range conversion (use it on videos where you see no total black and only grays). Or check video using ShowChannels plugin (if minimum in Y is 16 or 15 - then your source is in limited range). You can change input levels using rangemin & rangemax params. They are used only in this mode. If they are not specified or wrong (rangemin>=rangemax) then default will be used.

9 mode: PC to TV color range conversion

10 mode: Grayscale.

For RGB32 - this mode uses RGB -> YUV444 -> RGB cuda conversion. U & V planes are set to 128 (and 512 on 10 bit).

For YUV - just U & V planes are set to 128 (or 512 for 10bit) without cuda.

11 mode: OETF function is applied to each pixel.

12 mode: EOTF function is applied to each pixel.

You can use 3 different softlight formulas:

formula = 0,1,2

0 - pegtop

1 - illusions.hu

2 - W3C

In my opinion - pegtop fomula is the best.

Also mode 1 & mode 3 are my favourite.

Photoshop formula was removed because of discontinuity of local contrast.

Formulas are explained here: https://en.wikipedia.org/wiki/Blend_modes

rangemin & rangemax These parameters are used for TV2PC color range conversion. If not specified, then default 16-235 (8 bit) and 64 - 963 (10 bit) will be used.

changerange Previously named "fullrange". But not only name was changed - now it works different. By default it is 0. When 0 it will treat YUV as limited range and RGB as full range. This means that for YUV it will rerange it to full before processing and for RGB it will not rerange. Else if it is 1, then YUV will not be reranged and RGB will be reranged. So, for example, if your source is RGB but in limited range (that is not normal) you should do:

softlight(3,changerange=1).

This will rerange RGB to full range, process it, and rerange to limited back. But if your RGB source is normal (full range), then

softlight(3)

will not rerange anything.

Example for range conversion outside for YUV source:

softlight(8) - we change YUV to full range

softlight(3,changerange=1) - we process "not normal" YUV without reranging it

softlight(9) - we make it back to "normal" limited range YUV

This will be slower, than when conversion is done inside. Because data will go back and forth from RAM to VRAM with each mode call.

Usage in AviSynth:

Softlight() same as SoftLight(0,0,0) same as SoftLight(mode=0,formula=0,skipblack=0,yuvin=0,yuvout=0)

Usage in VapourSynth:

video = core.Argaricolm.Softlight(video) or core.Argaricolm.Softlight(video,mode=0,formula=0,skipblack=0)

Skipblack option is a new enhancement for averate calculation. By default skipblack = 0 and it means it is activated.

To disable it - set it to anything not zero (like 1).

What it does is calculates how many plane (channel) elements are zero. Then they will not be counted in average calculation.

Example:

Original: (1 + 2 + 0) / 3 = 1 average

With skipblack enabled: (1 + 2 + 0) / 2 = 1.5 average

Color modes supported so far:

Avisynth:

Planar YUV 420 8 bit and 10 bit (YUV420P8, YUV420P10)
Planar YUV 444 8 bit and 10 bit (YUV444P8, YUV444P10)
Not planar RGB32 (BGR32) - this one is default you get by using ConvertToRGB() or ConvertToRGB32()
Planar RGB 8 bit and 10 bit (you get it by using ConvertToPlanarRGB()
Same for VapourSynth except BGR32 (Fredrik "asked" not to implement it in VapourSynth plugins)

yuvin & yuvout options are used for modes where yuv <-> rgb conversion is used and they define formula used for decode and encode

0 = Default is Rec.709.

Or you can select 601, 709, 2020. Like Softlight(yuvin=601,yuvout=601)

About OETF & EOTF functions.

They are added just to play with. EOTF is a reverse of OETF.

Try OETF function when your source is converted to PC range. To convert to PC range use Softlight(8). If result after OETF lacks of contrast then try to change black level higher than 16 like so:

Softlight(8, rangemin=16, rangemax=235)
Softlight(11)

Download at github (https://github.com/ArturAlekseev/AVS_SoftLight/releases)

[StainlessS]

Would not do any harm to post a few Mode before/after example images.

Postimages.org allows to embed images in your post, without needing Postimages.org account
(and dont need to wait for mods approval)

Postimages.org:- https://postimages.org/
Use, "thumbnail" or "image" for forum, modes. [copies url to clipboard, just paste in your post]

EDIT: If you do post images, I'll try remember to delete this post.

[Selur]

@StainlessS: here's an example: https://imgsli.com/MTg4MTEz
script used: ClearAutoloadDirs()
SetFilterMTMode("DEFAULT_MT_MODE", MT_MULTI_INSTANCE)
LoadPlugin("F:\Hybrid\64bit\Avisynth\avisynthPlugins\LSMASHSource.dll")
Import("F:\Hybrid\64bit\Avisynth\avisynthPlugins\mtmodes.avsi")
LoadPlugin("c:\Users\Selur\Desktop\Softlight.dll")
# loading source: G:\TestClips&Co\files\MPEG-4 H.264\Canon 5D RAW.mp4
# color sampling YV12@8, matrix: bt709, scantyp: progressive, luminance scale: limited
LWLibavVideoSource("G:\TestClips&Co\files\MPEG-4 H.264\Canon 5D RAW.mp4",cache=false,format="YUV420P8", prefer_hw=0)

org=last
Softlight(mode=X)

Interleave(org.Subtitle("Original"), last.Subtitle("Softlight(mode=X)"))
# current resolution: 1920x1080
PreFetch(16)
# output: color sampling YV12@8, matrix: bt709, scantyp: progressive, luminance scale: limited
return last

@Argaricolm: Any plans for a Vapoursynth version?
Any plans to also allow RGB input and high bit depth support?

Cu Selur

[StainlessS]

Cheers Selur, nice comparison method.

[Argaricolm]

@Argaricolm: Any plans for a Vapoursynth version?
Any plans to also allow RGB input and high bit depth support?

Cu Selur

RGB input is easy. I can do it fast.
Vapoursynth - never compiled for it. If much needed I can do it.
For high bit depth I'm not sure. If I will be able to change softlight code for it - then possible.

[Selur]

More supported color spaces are always better, since it give more freedom.
Vapoursynth would be great, since I mainly use Vapoursynth. https://forum.doom9.org/showthread.php?t=182961 might help with supporting both Avisynth and Vapoursynth.

Cu Selur

[tormento]

Am I the only one who doesn't understand what it does? :)

[Argaricolm]

Am I the only one who doesn't understand what it does? :)

It does this (https://www.youtube.com/watch?v=m5V2zuhGr4U). But is not changing brightness.

[Frank62]

Interesting. Has this something to do with what they did with the colours on "Moby Dick"? Or later with the BluRay version of "French Connection"?

[Argaricolm]

Interesting. Has this something to do with what they did with the colours on "Moby Dick"? Or later with the BluRay version of "French Connection"?

Don't know. I'v got an idea to do it with video. Possibly someone got same idea. But it's possible only using CUDA/GPU. Because summing each frame on CPU is very slow.

[wonkey_monkey]

But it's possible only using CUDA/GPU. Because summing each frame on CPU is very slow.

How are you defining "possible" and "very slow"?

[Argaricolm]

How are you defining "possible" and "very slow"?

Possible on CPU too. But it will be around some fps. While cuda version 10x more faster.

[StainlessS]

Because summing each frame on CPU is very slow.
Any good for plain C version ? [should be ok for 8K+, req slight mods for 16 bit, probably dont need altscan, or crop coords]

double __cdecl PVF_AverageLuma_Planar(const PVideoFrame &src,const int xx,const int yy,const int ww,const int hh,const bool altscan) {
const int ystep = (altscan) ? 2:1;
const int pitch = src->GetPitch(PLANAR_Y);
const int ystride = pitch*ystep;
const BYTE *srcp = src->GetReadPtr(PLANAR_Y) + (yy * pitch) + xx;
__int64 acc = 0;
unsigned int sum = 0;
const int yhit = (altscan) ? (hh +1)>>1 : hh;
const unsigned int Pixels = (ww * yhit);

if(ww == 1) { // Special case for single pixel width
for(int y=yhit ; --y>=0;) {
sum += srcp[0];
srcp+= ystride;
}
} else {
const int eodd = (ww & 0x0F);
const int wm16 = ww - eodd;
for(int y=yhit; --y>=0 ;) {
switch(eodd) {
case 15: sum += srcp[wm16+14];
case 14: sum += srcp[wm16+13];
case 13: sum += srcp[wm16+12];
case 12: sum += srcp[wm16+11];
case 11: sum += srcp[wm16+10];
case 10: sum += srcp[wm16+9];
case 9: sum += srcp[wm16+8];
case 8: sum += srcp[wm16+7];
case 7: sum += srcp[wm16+6];
case 6: sum += srcp[wm16+5];
case 5: sum += srcp[wm16+4];
case 4: sum += srcp[wm16+3];
case 3: sum += srcp[wm16+2];
case 2: sum += srcp[wm16+1];
case 1: sum += srcp[wm16+0];
case 0: ;
}
for(int x=wm16; (x-=16)>=0 ; ) {
sum += (
srcp[x+15] +
srcp[x+14] +
srcp[x+13] +
srcp[x+12] +
srcp[x+11] +
srcp[x+10] +
srcp[x+ 9] +
srcp[x+ 8] +
srcp[x+ 7] +
srcp[x+ 6] +
srcp[x+ 5] +
srcp[x+ 4] +
srcp[x+ 3] +
srcp[x+ 2] +
srcp[x+ 1] +
srcp[x+ 0]
);
}
if(sum & 0x80000000) {acc += sum;sum=0;} // avoid possiblilty of overflow
srcp += ystride;
}
}

acc += sum;
double dacc = double(acc);
return dacc / Pixels;
}

EDIT: From RT_Stats, v2.0 Beta 13 [8 bit CS only].
Not that slow really. Similar method for other colorspace in "PVF_ ... " files.

EDIT: The switch stuff only accounts for ww pixel width, does not take srcp memory alignment stuff into account, so could be improved
to better use compiler vectorization type stuff {probably require additional switch thingy for end cases}.
If always full frame {no coords}, then could take some shortcuts. {Avisynth+ frames LHS always aligned, not so for Avs standard 'in place' cropping}

EDIT: Might be handy, [from here:- https://forum.doom9.org/showthread.php?p=1935661#post1935661 ]

Function PitchTortureTest(clip c) { # IanB:- https://forum.doom9.org/showthread.php?p=1628159#post1628159
c
A=SelectEvery(4, 0)
B=SelectEvery(4, 1).AddBorders(0,0,8,0).Crop(0,0,-8,0)
C=SelectEvery(4, 2).AddBorders(0,0,16,0).Crop(0,0,-16,0)
D=SelectEvery(4, 3).AddBorders(2,0,22,0).Crop(2,0,-22,0)
Interleave(A,B,C,D)
}

Could probably be improved if modified to take cropping granularity of colorspace into account.

EDIT: from OP,
1. YUV->RGB conversion
2. Calculates sums of all pixels in R,G,B planes (for each).
3. Get average from these sums (sum / number of pixels).
4. Get negative from this sum (255 - sum)
Check intent of -ve method.

From posted link for IanB thingy thread, here:- https://forum.doom9.org/showthread.php?p=1935616#post1935616


MyAverage, v2.6+

A simple average filter for Avisynth v2.60 standard colorspaces, only.

Returns a clip where each return frame is a single color average of input frame, same size and colorspace as input.
Does an invert on result if Bool Invert==true.


ColorSpace, YV12, YV16, YV24, YV411, Y8, YUY2, RGB24, RGB32, only.

Return clip Y, U and V, or R, G and B, will be channel averages, unless Invert==True, where channels averages will be inverted.

MyAverage(clip c, Bool "Invert"=false,Bool "TV_YUV"=False,Bool "MyYV24"=False)

Invert, Default false == sampled average. Otherwise Inverted average.
TV_YUV, Default false, If True(And YUV), then photo negative invert around TV levels mid Y(125.5), rather than 127.5.
MyYV24, If true and YV24 (only YV24), then process Y,U,V, together, else by planes.

Returns clip same colorspace and size as input.



if(invert) { // invert ?
if(tvy) { // tv levels invert ? [ TV levels center is 125.5 not 127.5, ie (16 + 235)/2 ]
ave = int(-(ave_D - 125.5) + 125.5 + 0.5); // TV_YUV Y mid = 125.5, invert, and Round
} else {
ave = int(ave_D + 0.5) ^ 0xFF; // PC_YUV Y mid = 127.5, symmetrical about 127.5 [EDIT: ADDED, or ave = 255 - int(ave_D + 0.5)]
}
aveU ^= 0xFF ;
aveV ^= 0xFF;
} else {

Also
In this snippet from posted source,

if(invert) { // invert ?
if(tvy) { // tv levels invert ? [ TV levels center is 125.5 not 127.5, ie (16 + 235)/2 ]
ave = int(-(ave_D - 125.5) + 125.5 + 0.5); // TV_YUV Y mid = 125.5, invert, and Round
} else {
ave = int(ave_D + 0.5) ^ 0xFF; // PC_YUV Y mid = 127.5, symmetrical about 127.5
}
aveU ^= 0xFF ;
aveV ^= 0xFF;
} else {
ave = int(ave_D + 0.5);
}
ave = max( min( ave, 255) ,0);

Stuff in BLUE ain't exactly correct, xor with $FF would invert $80 U,V center to $7F,
but avoids problem where source U,V == 0 would invert to $100, we invert to $FF in 8 bit range,
also method adopted will arrive back to original source value if inverted twice.
As it is, it aint quite right but method I chose. Maybe I should invert and clip, there should be no source of $00 anyway.
EDIT: Source $00 equivalent to center - 128, and source $FF equiv to center + 127, ie not symmetrical about center 128.

[StainlessS]

Further to above,

W = 3 * 1280 # 3840
H = 3 * 720 # 2160
STATICFRAMES = False
SECONDS = 5 * 60
###
FRAMES = Round(29.97 * SECONDS)

# SECONDS seconds @ 29.97 FPS. STATICFRAMES, If set to false, generate all frames. Default true (one static frame is served)
Colorbars(Width=W,Height=H,pixel_type="YV12",staticframes=STATICFRAMES).Trim(0,-FRAMES)

# Comment one of below out
Return Scriptclip("AverageLuma() return last") # Avs+ builtin. Always full frame.
#Return Scriptclip("RT_AverageLuma() return last") # RT_Stats, RT_AverageLuma. Used to be faster than AVS 2.60 Standard.


4K : AVS+ AverageLuma : STATICFRAMES = FALSE

c:\Z>avsmeter64 test.avs

AVSMeter 3.0.9.0 (x64), (c) Groucho2004, 2012-2021
AviSynth+ 3.7.3 (r3996, master, x86_64) (3.7.3.0)

Number of frames: 8991
Length (hh:mm:ss.ms): 00:05:00.000
Frame width: 3840
Frame height: 2160
Framerate: 29.970 (30000/1001)
Colorspace: YV12
Audio channels: 2
Audio bits/sample: 32 (Float)
Audio sample rate: 48000
Audio samples: 14399985


Frames processed: 8991 (0 - 8990)
FPS (min | max | average): 554.7 | 890.6 | 843.1
Process memory usage (max): 104 MiB
Thread count: 16
CPU usage (average): 7.9%

Time (elapsed): 00:00:10.664


4K : RT_AverageLuma : STATICFRAMES = FALSE

c:\Z>avsmeter64 test.avs

AVSMeter 3.0.9.0 (x64), (c) Groucho2004, 2012-2021
AviSynth+ 3.7.3 (r3996, master, x86_64) (3.7.3.0)

Number of frames: 8991
Length (hh:mm:ss.ms): 00:05:00.000
Frame width: 3840
Frame height: 2160
Framerate: 29.970 (30000/1001)
Colorspace: YV12
Audio channels: 2
Audio bits/sample: 32 (Float)
Audio sample rate: 48000
Audio samples: 14399985

Frames processed: 8991 (0 - 8990)
FPS (min | max | average): 220.2 | 382.7 | 329.0
Process memory usage (max): 104 MiB
Thread count: 14
CPU usage (average): 8.0%

Time (elapsed): 00:00:27.328




4K : AVS+ AverageLuma : STATICFRAMES = TRUE

c:\Z>avsmeter64 test.avs

AVSMeter 3.0.9.0 (x64), (c) Groucho2004, 2012-2021
AviSynth+ 3.7.3 (r3996, master, x86_64) (3.7.3.0)

Number of frames: 8991
Length (hh:mm:ss.ms): 00:05:00.000
Frame width: 3840
Frame height: 2160
Framerate: 29.970 (30000/1001)
Colorspace: YV12
Audio channels: 2
Audio bits/sample: 32 (Float)
Audio sample rate: 48000
Audio samples: 14399985

Frames processed: 8991 (0 - 8990)
FPS (min | max | average): 2335 | 3244 | 3082
Process memory usage (max): 81 MiB
Thread count: 16
CPU usage (average): 6.9%

Time (elapsed): 00:00:02.917


4K : RT_AverageLuma : STATICFRAMES = TRUE

c:\Z>avsmeter64 test.avs

AVSMeter 3.0.9.0 (x64), (c) Groucho2004, 2012-2021
AviSynth+ 3.7.3 (r3996, master, x86_64) (3.7.3.0)

Number of frames: 8991
Length (hh:mm:ss.ms): 00:05:00.000
Frame width: 3840
Frame height: 2160
Framerate: 29.970 (30000/1001)
Colorspace: YV12
Audio channels: 2
Audio bits/sample: 32 (Float)
Audio sample rate: 48000
Audio samples: 14399985

Frames processed: 8991 (0 - 8990)
FPS (min | max | average): 511.8 | 859.5 | 801.5
Process memory usage (max): 81 MiB
Thread count: 16
CPU usage (average): 7.9%

Time (elapsed): 00:00:11.218


Clearly, Pinterf did some improvements to Avs+ AverageLuma,
Suggest steal some of his code [I will not tell him].

Also note, Scriptclip would slow it quite a bit compared with GetFrame() in plugin.
We did not assign AverageLuma to variable in scriptclip, as that would likely greatly affect results.

However, FPS for the RT_AverageLuma aint so very bad for 4K, and you would likely want pure C version anyway.

EDIT: Numbers above on i7-8700.
[No Prefetch {also Scriptclip single thread}, so I assume fully single core numbers]
EDIT: Yep, Resource meter seems to show single core in use.

[Selur]

Vapoursynth - never compiled for it. If much needed I can do it.
:) A Vapoursynth version would be really nice.

[Selur]

btw. since yuv->rgb is used internally: How about also supporting RGB input?

[Argaricolm]

btw. since yuv->rgb is used internally: How about also supporting RGB input?

I'm planning to add vapoursynth, YUV444, RGB support soon.
So far a new release.

[Selur]

Thanks! Looking forward to it!

[Argaricolm]

Added YUV 444 support and VapourSynth support. And a small bugfix.

[Selur]

Nice! Thanks!
Did some quick tests, seems to work fine in Vapoursynth.
About color space support:
Would be cool if you could also add 10, 16, 32bit support, if possible. :D
(in detail: RGBS, RGBH, RGB48,RGB30, YUV420P10, YUV420P16, YUV420PS, YUV420PH, YUV444PH, YUV444PS, YUV444P10, YUV444P16)


Cu Selur

[Argaricolm]

Nice! Thanks!
Did some quick tests, seems to work fine in Vapoursynth.
About color space support:
Would be cool if you could also add 10, 16, 32bit support, if possible. :D
(in detail: RGBS, RGBH, RGB48,RGB30, YUV420P10, YUV420P16, YUV420PS, YUV420PH, YUV444PH, YUV444PS, YUV444P10, YUV444P16)


Cu Selur

Redownload 1.11 release. I'v fixed it a little. It contained wrong check in avisynth version.
Next I will add RGB32 support.

[Selur]

Will do. Thanks!

[Argaricolm]

A new update.
Added new mode 1 (other numbers changed).
Added RGB32. But only for avisynth (vapoursynth strangely does not support RGB32).

[Selur]

Using the latest version in Vapoursynth, when using:
# adjusting color space from YUV420P8 to RGB24 for Softlight
clip = core.resize.Bicubic(clip=clip, format=vs.RGB24, matrix_in_s="470bg", range_s="limited")
# color adjustment using Softlight
clip = core.Argaricolm.Softlight(clip)
I get just a black output, while using:
# adjusting color space from YUV420P8 to YUV444P8 for Softlight
clip = core.resize.Bicubic(clip=clip, format=vs.YUV444P8, matrix_in_s="470bg", range_s="limited")
# color adjustment using Softlight
clip = core.Argaricolm.Softlight(clip)
works as expected.

Doesn't matter whether I use "CUDA 12.3/SoftLight.dll" or "CUDA 11.8/SoftLight.dll".
=> seems like v1.12 broke RGB24 support for Vapoursynth.


Cu Selur

Ps.: Do you prefer if I post such stuff here or on github?

[Argaricolm]

"Added RGB32. But only for avisynth (vapoursynth strangely does not support RGB32)."

There is no RGB24 support (so far).
And strangely vapoursynth does not support RGB32. That should be faster in memory because of 4 bytes addressing.

[Selur]

afaik. Vapoursynth handles alpha channels in a separate 'stream/clip'.

[Argaricolm]

afaik. Vapoursynth handles alpha channels in a separate 'stream/clip'.

Well for RGB32 i just use R,G,B bytes and skip alpha one.
So it's just like I use RGB24 but in RGB32 adressing.
In avisynth memory 4th byte is automatically set to FF (255) when 24bit (8*3) content is converted to RGB32.

So it's not realy a correct RGB32 support.
Maybe I need to change it to RGB24.

[Selur]

Yeah, sound like it should be RGB24 not RGB32 if the alpha channel isn't used.

[Argaricolm]

Yeah, sound like it should be RGB24 not RGB32 if the alpha channel isn't used.

New version 1.13 (https://github.com/ArturAlekseev/AVS_SoftLight/releases/tag/v1.13).
Now RGB will work in Vapoursynth. It is RGB24 planar.
Also I'v updated CUDA toolkit to 12.4 version.

[wonkey_monkey]

Am I missing something?

colorbars(pixel_type="rgb24", width = 3840, height = 2160).softlight

doesn't seem to do anything (same with a real image source). AvsMeter can't time the script because it's too fast, which kind of suggests the filter isn't doing anything. I tried it on two different computers (laptop and desktop) with Nvidia cards.

Does it just pass through the original clip if there is a CUDA issue?

---

Edit: RGB24 interleaved doesn't work, RGB32/YV12/presumably others does

Further edit: doesn't seem to work at all on my desktop computer, just returns unaltered clip...

Edit: Having looked at the code there is zero error handling/reporting, even for CUDA failures. You might want to add some!

[Selur]

RGB24 works in Vapoursynth here.

[Argaricolm]

Does it just pass through the original clip if there is a CUDA issue?

It does nothing if CUDA is not supported or not supported input format.

[wonkey_monkey]

It does nothing if CUDA is not supported or not supported input format.

Error throwing would be very helpful to avoid confusion, e.g.:

if (cudaStatus == cudaSuccess) {
...
} else {
env->ThrowError("SoftLight: CUDA failed");
}


and similar when none of the conditions in GetFrame are met (although testing should be in the constructor, ideally).

Going back in time a little:

Possible on CPU too. But it will be around some fps. While cuda version 10x more faster.

I've investigated my scepticism and although it will obviously vary depending on CPU, GPU, colourspace etc, for a YV12 input I found only a 1.4x-1.7x speed increase over CPU AVX code implementing mode=3 (pegtop).

For interleaved RGB input, AVX code was 1.3x faster than CUDA, even with AviSynth+ colourspace conversion overheads. Multithreading might give another 25%-50% boost.

[DTL]

Because summing each frame on CPU is very slow.

To sum all samples of the frame at SIMD there are possible several ways:

1. Sum at integer - require unpacks of 8..16 samples to 32bits and use summing of standard SIMD full width * superscalar factor of sum dispatch ports first for all samples of a line.

Because it looks 32bit integer can not hold UHD frame samples number * 256..65535 samples values sum without overflow - it is possible to make intermediate division of intermediate sums for each line and accumulate normalized sums of the all lines of a frame. It is more complex to program in compare with float32 processing but maybe visibly faster for SD 8bit and some HD frame sizes.

2. Make unpack and convert to float32 and perform all of 1 in float32 domain.

So best performance implementation can have different processing engines inside for different frame sizes. At least 1920x1080 with 8bit still can be processed with integer full frame summing without 32bit accumulator overflow. Also with SIMD word summing programmer anyway have partial sums at the final SIMD word ready to partial normalizing with some more overflow protection (AVX2 SIMD word of 8 32bit integers provides additional +3bits to overflow so total capacity is 32+3=35bits) and without significant precision loss.

Method 2 can process any frame sizes in single engine but expected to be slower at non-UHD frame sizes.

CPU SIMD is not very slow - but algorithm requires at least 2 full frame passes: first analisys pass of sum and second is correction pass of adjustment so performance will depend on frame size fitting in availavle CPU caches (our lovely Xeon MAX with HBM onboard will be nice performer here).

[Argaricolm]

A new release - 1.14 (https://github.com/ArturAlekseev/AVS_SoftLight/releases/tag/v1.14-release).

Also a question for video gurus here:
As I see nearly ALL content that is released now on blurays or streamed through streaming services are encoded in limited color range (16-235). The question is why it is so?
Old TVs that had such limitations are already all in junk. And new TV's can't determine automatically that source is limited color range.
This results that we watch limited color range without convertion to full range. But we should watch limited range converted to full.
I understand that streaming services long ago could do this to make streaming smaller in size (limited color range take fewer space).
But now, when we have fast internet speeds nearly everywhere it is just ridiculous.
And for blurays I don't understand it at all.

It looks like some conspiracy to mock on people eyes.

[DTL]

" The question is why it is so?"

It is industry standard to keep more quality with limited number of bits (until possible changing to float32 or at least float16 samples values encoding). But 8bit-narrow (limited) works good enough so it is unlikely industry will change to float16/32 any fast.

"This results that we watch limited color range without convertion to full range."

Physical display converts 16..255 Y code values to 0..PHYmax brightness values. So you not lost 236..255 code values encoded in 8bit narrow range. You can test it with 16..255 Y values test pattern. If display clips 236..255 to PHYmax it is broken and need repair or adjustment.

235 code value only marks position of nominal white - not max PHY white. Display hardware may treat 236..255 range very differently (depending on the processor cost and AI algoriphms included) - either continue to track system transfer function or make HDR-expansion of any type.

[Selur]

@Argaricolm: posted in the issue tracker over at GitHub, 10bit does not work in Vapoursynth

[Argaricolm]

" The question is why it is so?"

It is industry standard to keep more quality with limited number of bits (until possible changing to float32 or at least float16 samples values encoding). But 8bit-narrow (limited) works good enough so it is unlikely industry will change to float16/32 any fast.

8 bit is not about limited color range.

Limited color range is 8 bit 16-235 levels of brightness (220 from 256).

When you view it on tv as it is - nothing is converted to full range.
You see incorrect colors and contrast.
But you have seen it for years now. So you think that it is "normal".
Here is example (https://imgsli.com/MjYxMzA0).
In limited color range there is no 0 and thats why in frames with a lot of black/dark you dont see black. You see only nearly black. This results in fewer contrast. And incorrect colors (because of contrast). For example you should see red color, but you will see light red.

Physical display converts 16..255 Y code values to 0..PHYmax brightness values.

It should do so. But how can it find out that it should?
For example - that batman 2022 video from above is surely limited color range but it does not have any info inside about limited color range. And I'v checked pixels. You can find values 0-15 inside any limited color range video. Its just they are fewer in numbers than should be. So I don't see any easy way for TV to determine - is video with limited color range or not.

[DTL]

"You can find values 0-15 inside any limited color range video"

It is also correct - footroom in narrow range mapping is to hold filter undershoots to display better sharpness (visible in the PHY range above zero). See https://forum.doom9.org/showthread.php?p=2000687#post2000687

[wonkey_monkey]

When you view it on tv as it is - nothing is converted to full range.
You see incorrect colors and contrast.
But you have seen it for years now. So you think that it is "normal".
Here is example.
In limited color range there is no 0 and thats why in frames with a lot of black/dark you dont see black. You see only nearly black.

Are you saying all TVs have been getting it wrong since forever?

Because I don't think that is the case.

[DTL]

"Here is example."

Computer displays and OS (bitmap processing) were designed for RGB full range mapping. Typically for static imaging like photo. So to watch industry standard encoded moving pictures with narrow range mapping you need special software (or software + driver for video card to support all required conversions and levels re-mapping) and you will got all your blacks correct and some not clipped and not very bad super-whites. It is topic for '(software) video players' section of forum - https://forum.doom9.org/forumdisplay.php?f=15

[Julek]

When you view it on tv as it is - nothing is converted to full range.


That's just wrong.
There is metadata for this, check a WEBDL with mediainfo for example.

And if you can't see 100% black on your TV, maybe it's because your TV isn't OLED, in which case it's physically incapable of making true black.

[wonkey_monkey]

So to watch industry standard encoded moving pictures with narrow range mapping you need special software

I wouldn't say you need special software. I've never seen a video player that doesn't expand from limited range by default.

[DTL]

"And new TV's can't determine automatically that source is limited color range."

It may be broken TV or badly configured from defaults. After RCA/SCART analog cunsumers connections between Disk Players and Display devices new standard is HDMI. And typical display should expect HDMI data in narrow range by default. Some displays have control how to treat HDMI data - narrow or full (for the case of Computer connection via HDMI). Also HDMI may have some metadata signalling on range mapping used (is some version and depends on transmitter and receiver compatibility ?).

So in the case you use standard consumer Disk Player and standard consumer Media Display Device and connect via RCA/SCART or HDMI everything expected to runs fine with correct blacks and super-whites.

If you trying to use Media Display Device with media file playback - there are many points of failure like badly created file rip or wrong display playback firmware (or incompatible with some hand-crafted file rip etc). If you try to use general purpose consumer Computer to playback some file rip there are even more places to fail.

If you use some network streaming it is also may or may not be correctly decoded in playback device (depend on codec/protocol/etc settings at source side and firmware at playback device). So it is the subject to post issues to streaming provider or manufacturer of playback device about possible errors in range mapping treatment.

With AVS it is possible to change levels mapping using Levels so simulate Computer Playback transform of narrow range RGB like Levels(16,255,1,0,255) (and in system transform domain or linear (?)) into Computer full range RGB 0..255. Yes - in 8bit it will adds some quantization noise (banding) so may be good to add some dithering after this range mapping (expansion) if source natural noise levels are too low to do self-dithering.

It is not one and always correct range remapping - it only some example to keep super-white unclipped. To get more contrast with clipping of possible super-whites you can use Levels(16,235,1,0,255) or you can go into RGBPS and apply some LUT or AI/NN plugin to do some nice super-whites expansion to HDR and convert result back into some standard HDR transfer domain to feed to HDR-capable display device.

[Argaricolm]

Well I have a cheap TV Skyworth. And it does not have any switch between TV / Full color range and looks like it shows everything in full range.
Yes it has some feature called "adaptive brightness control", but it is adaptive. Its is not a static convertion from tv to full range.
Also I watch some content from TV box Beelink, that is flashed with android tv custom firmware. It also does not have any switch for TV / Full range. It has only settings to choose between YUV444, YUV422, RGB. But this switch does not affect color range.
And also I'm not some guru, so as a normal user when I see some settings about full range or limited range I think that "full" is better than "limited". And I think most other normal TV users think like so.
And in result we see limited color range without convertion. So we watch data in its intermediate state designed to be converted to full range before output. And I think it's a strange design for our days when we have fast transfer speeds and space - no need for limited color range anymore.

And here is example (https://imgsli.com/MjYyNTYz) of what we see and what we should see.

And especially strange to see such content on youtube. For example most tv records are published there without convertion to full range. And we watch it on our tablets/computers also without any convertion. And we think that it is "normal".

[Julek]

And here is example (https://imgsli.com/MjYyNTYz) of what we see and what we should see.

Can you post the script used to convert limited->full, you seem to be doing it the wrong way, when you convert the YUV video to RGB it is already adjusted to full, so there should be no difference, and your “full” is clipping dark areas.

[wonkey_monkey]

It might be worth pointing out here that humans have a tendency to automatically associate brighter/louder/higher contrast/higher saturation with "better". That doesn't mean "full" is always the proper choice.

[Argaricolm]

Can you post the script used to convert limited->full, you seem to be doing it the wrong way, when you convert the YUV video to RGB it is already adjusted to full, so there should be no difference, and your “full” is clipping dark areas.

Before I was using YUV <-> RGB conversion formula from here (https://learn.microsoft.com/en-us/windows/win32/medfound/recommended-8-bit-yuv-formats-for-video-rendering).
But result was always YUV with limited range.

Now I use formula from here (https://www.mikekohn.net/file_formats/yuv_rgb_converter.php).

It does not care for color space resulting the same range.
Example: RGB(5,5,5) <=> YUV(5,128,128)
So from limited YUV I get limited RGB and then limited YUV back (or from full I get full).

Softlight(8) rerange limited range to full this way:

1. YUV is converted to RGB (if input is YUV)
2. RGB is reranged
3. Back to YUV.

Rerange is done this way:

(R - 16) / 220 * 255 + 0.5

So each level from 16 to 235 will become from 0 to 254.
When it is converted to YUV - I get full range YUV.

So far the best combination I use for myself is:
ConvertToRGBNV() <- this is from ImageSourceNV plugin
softlight(8)
softlight(3)
ConvertToYUVNV()

Here I use BGR32 input because I use one plugin between softlight(8) and softlight(3) that requires BGR32 input.

If you use only softlight combination then its better to convert to RGB planar input. This way softlight functions will not convert YUV <-> RGB each time.
In above my example they convert BGR32 <-> RGB planar in each softlight call (but using CUDA).

Think I will add just convertion functions the next release.

[Argaricolm]

It might be worth pointing out here that humans have a tendency to automatically associate brighter/louder/higher contrast/higher saturation with "better". That doesn't mean "full" is always the proper choice.

Better choice to view surely.
Looks more 3D and with higher saturation.
Yes you see fewer details in dark areas. But I think its a small tradeoff not to see some dark details for deeper 3D and not brightened colors (and that's how you should view it anyway - if your hardware/software will correctly identify source as limited).

[wonkey_monkey]

Better choice to view surely.
Looks more 3D and with higher saturation.
Yes you see fewer details in dark areas. But I think its a small tradeoff not to see some dark details for deeper 3D and not brightened colors.

That's your personal choice. It doesn't mean your TV is doing something wrong just because you can override it to a setting that you think looks better.

[Argaricolm]

That's your personal choice. It doesn't mean your TV is doing something wrong just because you can override it to a setting that you think looks better.

In my case I can't override it.
Also i'v not mentioned it here:
If you watch video on PC using MPC-HC - its default renderer "MPC Video Renderer" will output video in limited range even if your video is full range and even if video has full range tag.
To watch full range video you need to select EVR renderer in options.

Also if you want to process limited range input and get limited range output - I think its better to convert it to full before and then back to limited.
Because neutralization functions are designed for full range. They get negative average sum this way: 255 - average.
I have not tested it with (235 - average) for limited input. And it will be more complicated, because limited input technically can have noise levels > 235. So for proper average calculations I need to cut those levels before sum.
Will try it later but not soon.

[hello_hello]

And also I'm not some guru, so as a normal user when I see some settings about full range or limited range I think that "full" is better than "limited". And I think most other normal TV users think like so.

TVs generally expect limited range for YUV and expand it to full range to display it. Likewise they should/could expect RGB to be full range and therefore shouldn't expand it.

My old Samsung TV doesn't have a setting labelled full/limited range either. Samsung decided to label it "HDMI Black Level", where "Normal" means the TV expects full range and "Low" means it expects limited range. It only applies when the input is RGB over HDMI though. Any other time it's greyed out as it expects YUV to be limited range, and it expects RGB at it's VGA input to be full range.

The LCD monitor connected to this PC does have a setting labelled RGB full/limited range for it's HDMI inputs.

And in result we see limited color range without convertion. So we watch data in its intermediate state designed to be converted to full range before output. And I think it's a strange design for our days when we have fast transfer speeds and space - no need for limited color range anymore.

As long as the output and input match in respect to levels the picture should look correct, and any input/output settings probably only apply to RGB over HDMI anyway. Here's a screenshot showing how to change the levels in the Nvidia control panel. https://i.redd.it/x44a5pwwned51.png

And here is example (https://imgsli.com/MjYyNTYz) of what we see and what we should see.

I'd place a bet on the limited range version being the correct one, although if your TV has a really horrible black level I can see why the full range picture might look better.

It's fairly easy to check when you're using a PC with MPC-HC, as it has pixel shaders for changing the levels. If you play a video with black borders encoded and you tell MPC-HC to expand the levels with a pixel shader.... if the picture gets darker but the borders don't then they were already black and the levels were already being expanded. Like this:

https://imgur.com/PrVqOYK.png

https://i.ibb.co/W0W98WB/1.jpg

https://i.ibb.co/VMNM2CR/2.jpg

However if the black borders do get darker then the levels are wrong.

Here's a small YV12 video you can try. There's a histogram on top.
The first 500 frames have the correct levels. Black=16.
The next 500 frames have those levels expanded. If it's being displayed correctly the black borders won't/can't get any darker even though the picture will.
For the last 500 frames a PC to TV conversion was applied so the black borders should be dark grey rather than black and the picture will look a bit washed out. If the borders still look black rather than dark grey it means the levels are being expanded unnecessarily, maybe due to a mismatched input/output.

Black Test.mkv (https://files.videohelp.com/u/210984/Black%20Test.mkv)

[Argaricolm]

I'm making an update with support for lossless yuv<-> rgb convertion. And also I'v added rec 601 & rec 709 so far (not published).
What will be better:
1) Use based on height: Rec601 when height <= 576; Rec709 for 577-1080; Rec2020 for > 1080
2) Or use rec 709 always by default.
It will be possible to select input rec and output rec separately.

[Argaricolm]

Black Test.mkv (https://files.videohelp.com/u/210984/Black%20Test.mkv)

Tried this video from TV and from TVbox.
My TV shows it same way as on PC.
I see limited range in first part (with bad blacks), PC range in middle (ideal) and bright last.
No settings in TV about color range, no settings in any android video player about it. Only "adaptive lightness control" in TV - but it does a little different work and it change full range too.

Looks like I'm stuck with converting video or playing it from PC.

[DTL]

I'm making an update with support for lossless yuv<-> rgb convertion. And also I'v added rec 601 & rec 709 so far (not published).
What will be better:
1) Use based on height: Rec601 when height <= 576; Rec709 for 577-1080; Rec2020 for > 1080
2) Or use rec 709 always by default.
It will be possible to select input rec and output rec separately.

For fail-safe operation it may be better to disable any auto-estimation and force user to provide required param.

You can set init value to -1 for example and if it is left as default at class constructor - throw an error about required param not set.

Also if you want to try new AVS+ features you can read frame/clip property of the colour matrix (if set). So possible ways are:
1. Force user input only.
2. Auto-estimate based on the frame height.
3. Read input frame/clip properties.

[Argaricolm]

Example (https://imgsli.com/MjY4NzIz)of OETF function.
It makes all movie bright like on the Sun...

[Argaricolm]

Version 1.15 (https://github.com/ArturAlekseev/AVS_SoftLight/releases/tag/v1.15) is out.

[Selur]

yuvin & yuvout options are used for modes where yuv <-> rgb conversion is used and they define formula used for decode and encode 0 = Default is Rec.709. Or you can select 601, 709, 2020. Like Softlight(yuvin=601,yuvout=601)
okay, 0 = Rec.709 What are 601 and 2020?

[DTL]

It looks current version uses direct numbering - https://github.com/ArturAlekseev/AVS_SoftLight/blob/a9f046fd8a60d187dc9364197601bced443ca4dd/kernel.cu#L2892

So 0 and 709 are the same and others are 601 and 2020 integers for rec.601 and rec.2020 matrices. Any other integer numbers except 601 and 2020 will cause default to rec.709 matrix ?

[Selur]

@DTL: Thanks.
Any other integer numbers except 601 and 2020 will cause default to rec.709 matrix ?
yes.

[Argaricolm]

Anyone knows something about OETF and EOTF functions?
The interesting thing here for me is:
From their description in docs I'v found that OETF is used to encode original light and EOTF is reverse.
So first I thought that if I use EOTF on data I will get "original lightness of the scene". But EOFT function makes everything MUCH darker. And OETF makes brighter. It can't be that original scene lightness was so dark.
Or it is just a misunderstanding because of "light" differs from "electric signal" - so this functions are not designed to be used such way (maybe they are designed to be used only inside camera and inside display hardware).
But then it is strange that OETF function gives great results on some content.
For example Divergent 2 and 3 blurays processed by it (Rec.709 limited -> full range -> OETF) results it bright picture but with good contrast. While I'v tested on other BDs and result are just bright picture with lost contrast (its possible to get better contrast if limited -> full conversion is used twice before OETF, but this burns a lot of darks). So it looks like for some content there should be another OETF function. But same OETF is specified for Rec 601 and Rec 709.

[DTL]

"But EOFT function makes everything MUCH darker. And OETF makes brighter. It can't be that original scene lightness was so dark."

Your display device doing EOTF for you (and it is at current PCs typically rec.709/sRGB EOTF). So if you apply EOTF to data in the distribution domain and feed to standard display - you got EOTF applied twice and darker displaying.

To check linear light values you need to understand how digital values maps to physical light. And inspect digital values directly (or switch your E->O display device to linear EOTF if possible).

" looks like for some content there should be another OETF function."

Most of cine-titles (not direct live broadcasts) are mastered not with real scene light but after complex colour-grading process for artistic reasons and by choice of product director. So usase of standard EOTF and standard display only guarantees you will see same optical displaying of the content as was expected at production.

For live broadcasts depending on camera settings you can get real OETF only tracking and can restore linear light using EOTF transform. But real OETF tracking for physical production camera rarely usable (only in some 100% controlled studio lighting) so for real ENG and other open air shooting cases some (unknown) non-linearity added like KNEE (and AUTO KNEE) control to soft-compress highlights and this data not sent as metadata with encoded content so you can not restore full linear light with EOTF only.

Only for special closed digital imaging like medical highly linear path may be used from camera to display so doctor can see as much real image as possible. Most of other digital imaging are somehow distorted at production to have 'more nice look'. Same and much more changes applied to colours view.

You can take some digital camera with RAW output and look how awful is it looks in both colour and tone in most of real life scenes.

[Argaricolm]

I'v tested my beelink android TV box and MPC player again. This time with my own specially created files.
Results:
TV box does not care about HEVC flag - full or limited color space. If video is in full space but flagged limited - it will show it full anyway.
But looks like it cares for range itself. So I'v created image with half RGB(16,16,16) and half RGB(235,235,235). And then it shows it as black and white (really white not some gray nearly white).
So looks like TV box can fail in limited color space determinations if first frames will contain values < 16.

MPC player:
* MPC Video Renderer cares about full or limited color space flag and displays correctly.
* MadVR is same. (But with decoding AV1 its faster and does not freeze). So I better use it.
* EVR Renderer does not care about it. It always rerange YUV from limited to full. If video is already full - it will rerange it again.
Same in Staxrip (avisynth) - if source is YUV - preview will always be reranged to full.

[Argaricolm]

What bothers me now is that why twice reranged video to full looks more like it should be so:
Limited -> Full (https://imgsli.com/MjcyMzg0)
Full -> Twice Full (https://imgsli.com/MjcyMzg1)

[DTL]

Content mastered at master control monitor. See EBU Tech 3320 and 3325 for requirements and testing as example.

https://tech.ebu.ch/docs/tech/tech3320.pdf
https://tech.ebu.ch/docs/tech/tech3325.pdf

These documents also list EOTF and test methods and requirements for different tiers of quality. To see how it was mastered you need to simulate master control monitor and viewing environment (different for SDR and HDR too). Also you can tweak as you want with local display controls - it will be your local version of image.

[Argaricolm]

New release v1.16 (https://github.com/ArturAlekseev/AVS_SoftLight/releases/tag/v1.16).

Now you can easily get limited range back like so:
softlight(8) = full range
softlight(3) = average
softlight(9) = back to limited

You can change default range parameters for limited range like so: Softlight(8, rangemin = 16, rangemax = 235)
Its needed on some sources to play with OETF function (it needs black to be absolute - or contrast will be lost).

Also added softlight(7) mode. Some video sources with limited range contain values < 16 and > 235. This mode will change them to 16 and 235. This will ensure correct range identification by your decoding hardware (seems it looks at first frame for values outside limited range to identify video as full range). Plus it will allow to compress your video better.

[Argaricolm]

OETF makes magic in Alien 1979 without any levels correction.
Just > full range > OETF and much more details are visible without contrast lost. Example (https://imgsli.com/MjcyOTE0)

[DTL]

It looks like you do not understand why users need some levels and colour correction plugins. The studio mastered titles are always perfectly mastered for colour and tone so do not need any corrections. You can read also the article from Ken Rockwell how studio shootings always get perfect colour and tone - https://www.kenrockwell.com/tech/highlight-shadow.htm
But the job of studio colour grading costs a lot.

On some days (like 19xx..201x) there were home end users video unexperienced shooting with bad natural lighting on cheap hardware and bad auto-exposure and bad auto-colour etc. This result in bad colour and tone in most cases. So users typically want to make old home and amateur recordings better using some free tools. Thus to show how the tool can help end users with badly recorded home video it is better to use the real samples - not perfectly mastered studio titles. Some real content may be downloaded from forum posts at videohelp forum where users show some bad footages for repair.

[Selur]

I'm a bit confused what mode 8 (tv2pc) does.
8 mode: TV to PC color range conversion (use it on videos where you see no total black and only grays).
I expected it to do the same as:
clip = core.std.Levels(clip=clip, min_in=16, max_in=235, min_out=0, max_out=255) scaling tv to pc scale.
but:
clip = core.std.Levels(clip=clip, min_in=16, max_in=235, min_out=0, max_out=255)
clip = core.Argaricolm.Softlight(clip, mode=11, yuvin=601, yuvout=601)

gives me:
https://i.ibb.co/KLsy80t/grafik.png (https://ibb.co/YydjVR5)
while:
clip = core.Argaricolm.Softlight(clip, mode=8, yuvin=601, yuvout=601)
clip = core.Argaricolm.Softlight(clip, mode=11, yuvin=601, yuvout=601)
gives me:
https://i.ibb.co/Jym72CH/grafik.png (https://ibb.co/xfXjG36)

I also looked at:
clip = core.resize.Bicubic(clip, range_in_s="limited", range_s="full")
clip = core.Argaricolm.Softlight(clip, mode=11, yuvin=601, yuvout=601)
https://imgsli.com/MjcyOTUw

=> any inside on this? What does tv2pc do?

Cu Selur

[Argaricolm]

TV2PC changes range this way for 8 bit (in RGB for each pixel):
(R/G/B - 16) / 219 * 255
if input is yuv - it is converted to rgb for processing (without changing range)

As for core.std.Levels maybe you need to rerange channels in yuv differently (from vp docs):
clip = std.Levels(clip, min_in=16, max_in=235, min_out=0, max_out=255, planes=0)
clip = std.Levels(clip, min_in=16, max_in=240, min_out=0, max_out=255, planes=[1,2])

As for examples. I see difference here too.
AviSynth and VapourSynth give different results (https://imgsli.com/MjczNTQx).
Btw you should rerange your result back to limited range using softlight(9)
Otherwise you will have full range in yuv. If you preview it in staxrip (and maybe other software) you will see it twice reranged to full (for me preview is always reranged from limited to full - if I already have full - in preview I see it even more dark). To not have such issue you should try to work in RGB space.

As for different result in VapourSynth looks like a bug in VapourSynth (thou its strange).
Here what I'v found.

Using this script:
import vapoursynth as vs
from vapoursynth import core
core.std.LoadPlugin(path='c:\\..\\softlight.dll')
clip = core.std.BlankClip(width=640,height=480, format=vs.YUV420P8, length=500, fpsnum=2997, fpsden=125, color=[16, 128, 128])
clip = core.Argaricolm.Softlight(clip, mode=8)
clip = core.Argaricolm.Softlight(clip, mode=11)
clip.set_output()

When I open in VirtualDub - mode 8 is never executed.
Y = 16 is passed to mode 11 resulting in 55
Then 55 is passed again to mode 11 resulting 115
So it does mode 11 twice without doing mode 8.
I don't see how it can be from my code.

If only mode 8 is in script - it is executed correctly resulting 0 in Y.

If I do mode 8 then 11 then 9 - only mode 9 is executed 3 times.
I'll try to update VapourSynth (have R65).

[Selur]

Here what I'v found.
Okay, if multiple calls to Softlight (in Vapoursynth) atm. result in just the last mode called multiple times, I will stick to calling it only once.

I'll try to update VapourSynth (have R65).
I'm using R68.

[Argaricolm]

Fixed it.
v1.17 (https://github.com/ArturAlekseev/AVS_SoftLight/releases/tag/v1.17)

The problem was in different parameters passing in VapourSynth than in AviSynth.
AviSynth uses different objects for each call - so there I use object variables to store parameters.
But VapourSynth uses same object for different calls.
It first calls filterCreate for each call (in same object). So last call replaced previous parameters. And after that it starts to call filterGetFrame.
So I made it to pass parameters for each vsapi->createVideoFilter call.
Looks like it is not a bug. Strange, that filter skeleton does not have parameters passing example.

[Argaricolm]

The studio mastered titles are always perfectly mastered for colour and tone so do not need any corrections.
I can understand that they master color & tone perfectly. Or it may be director vision of the picture.
What I don't understand is that why nearly all latest titles are so dark.
Same goes for old titles too.
I don't think the problem was with the lack of light.

[DTL]

To display dark scenes - the lower code values of standard 8bit encoding used. The sRGB and rec.709 dynamic range with 8bit encoding is about 1000:1 (as ratio from first non-zero code value of 17 to nominal white of code value 235)so for daylight scenes higher part used and for night scenes - lower part. The total range is fixed in relative lighting (not as possilble with Dolby HDR with dymanic metadata ?) so it is easy enough to show dark scenes as dark and bright as bright. But this only work best in required viewing conditions - see viewing conditions for HDTV ITU-R BT.2022 https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.2022-0-201208-W!!PDF-E.pdf and ITU-R BT.2035 https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.2035-0-201307-I!!PDF-E.pdf and also check your display gamma-tracking for rec.709 EOTF first and also display nominal white level (or peak luminance) if you do not see dark scenes correctly.

Typically for end-users HDTV and rec.709 -
(BT.2022)
1.1 General viewing conditions for subjective assessments in a laboratory environment
The assessors’ viewing conditions should be arranged as follows:
a) Room illumination: low
b) Chromaticity of background: D65
c) Peak luminance1: 70-250 cd/m2
(See § 1.7.2)
d) Monitor contrast ratio: < 0.02 (See § 1.7.1)
e) Ratio of luminance of background behind picture monitor to
peak luminance of picture:
~ 0.15
1.2 General viewing conditions for subjective assessments in a home environment
a) Environmental illuminance on the screen (incident light from
the environment falling on the screen, should be measured
perpendicularly to the screen): 200 lux
b) Peak luminance1
: 70-500 cd/m2
(See § 1.7.2)
c) Ratio of luminance of inactive screen to peak luminance
monitor contrast ratio:
< 0.02 (See § 1.7.1)

(BT.2035)
1.1 Viewing environment for subjective assessment
a) Room illumination: 10 Lux
b) Chromaticity of background: D65 (optionally D93 in some
regions)
c) Ratio of luminance of background behind picture
monitor to peak luminance of picture:
≈ Between 10% ±2% of reference white value

10 Lux of room illumination is really very dim lighting (indoor and after sunset typically in room with windows).

"I don't think the problem was with the lack of light."

It may be with too many light in your viewing environment and too low display contrast (or too low display peak brightness too). Too much external lighting shifts viewer's brightness of adaptation to high levels and viewer lost ability to see low brightness levels (if even display tracks EOTF completely perfect).

[Selur]

Fixed it.
v1.17

Thanks, I can confirm it works fine now.

Cu Selur

[Argaricolm]

Another question.
Does anyone know why some sources have incorrect range?
For example this:
Solo A Star Wars Story.
This is a frame from bluray.
As you can see on the frame with a lot of blacks the lowerst value is 22. But should be 16 (at least some pixels should be absolute black).

For comparison same frame reranged:
softlight(8,rangemin=22,rangemax=235)
softlight(9)

Screenshots are from YUV (so you see both of them as they will appear on TV).

And this is a "new" movie. So this is not a result of something wrong. By unknown reason this range was intended.
I'v already seen some of such sources where lowerst range limit is ~22 instead of 16.

Example (https://imgsli.com/Mjc3ODYx)

By the way. If you want normal blacks after softlight(1 or 3 or 11) you should rerange such sources this way.

[Emulgator]

But should be 16 (at least some pixels should be absolute black)
I would not assume that for every frame, given that the lighting intent of your sample frame might well have been "close to dark, but just not sunk into blackness"
Watched on a OLED it makes sense, watched on a TFT you may want to lower that black because TFT blacks are poor to begin with.
But tying that to 16 for all audiences means loss of shadow detail, and crushed blacks on any slighty misadjusted monitor.
Just had such a source where this was attempted, the real blacks were sitting around 8. Was hard to reverse without banding.

[Argaricolm]

I'v found that some sources are twice converted to limited range (full -> limited -> limited).
To determine such issue you should check first black frames of video for levels (can do so using ShowChannels plugin). Normally first black frame should be (Y = 16, U = 128, V = 128). It may differ a little due to compression.
And normally each video should contain such totally black frame or frames at the begining. This signals hardware that video is in limited range.
When your hardware decodes video and displays - it always converts limited range to full before displaying.
But in case of twise converted range - hardware converts twise limited range to limited range and displays limited range to you.
For example (https://imgsli.com/MjkyNzc3)is the first season of Rick & Morty that has Y = 30 in first key frames. I have not checked all seasons, but 6 & 7 do not have such issue.

[Argaricolm]

New version 1.19 (https://github.com/ArturAlekseev/AVS_SoftLight/releases/tag/v1.19-release2).
Critical fix in TV2PC 10 bit color range conversion function. It was working wrong.
Added fullrange option.
By default functions will treat source as limited color range. It will be converted inside. Only OETF & EOTF functions are same.

So now you can do just:

softlight(3)

instead of:

softlight(8)
softlight(3)
softlight(9)

If you want to do same as before:
softlight(8)
softlight(3,fullrange=1)
softlight(9)

This is so only for YUV color space, because it is treated as limited color range by default.
I'm thinking to make same changes for RGB color space (add ability to change color space inside call). But by default it is treated as full range. Maybe I should do it reverse way (do not change it for RGB by default, but change it if fullrange = 1).

[Selur]

My 2 cents about this:
when:

fullrange = 0, SoftLight should assume input is 16-235 and its output should be 16-235 (unless TV->PC conversion is used)
fullrange = 1, SoftLight should assume input is 0-255 and its output should be 0-255 (unless PC->TV conversion is used)

[Argaricolm]

Made a new release. Fullrange is changed to "changerange". By default (0) it will treat YUV as limited range (and will rerange for processing) and RGB as full range (and will not rerange it). OETF and EOTF functions don't use this param and will treat any source as full range (you need to rerange it to full and back yourself using 8 & 9).
This param will correspond with my plugin ImageSourceNV where I make CUDA conversion functions. Will publish it's updated version soon.

[Selur]

So, if I make sure that my source is full range for all modes != 8 and limited range when using mode = 8, can simply skip the 'changerange' parameter?

[Argaricolm]

If your source is YUV and it is in full range (that is not normal), then you should use changerange=1. Otherwise it will rerange it from limited to full and back to limited.
If your source is YUV and it is in limited range (that is normal), then you should not use changerange.
If your source is RGB and it is in limited range (that is not normal), then you should use changerange=1.
If your source is RGB and it is in full range (that is normal), then you should not use changerange.

Simply said YUV is treated as limited range and RGB is treated as full range by default. And if it is not so, then you need to use changerange.
8,9,10,11,12 modes don't use this parameter.

[Selur]

okay,... how about adjusting the filter, so that the user, just reports whether the source should be treated as limited or full and the filter does rest and output the same range it was fed?
Current handling seems to be unnecessarily complicated.

8,9,10,11,12 modes don't use this parameter.
Does this mean they work without any requirement, that they always require limited or full range, or what?

Cu Selur

[Argaricolm]

Well, it's hard to determine is input full range or limited range.
Ideally limited range should not have values from 0-15 and from 236 to 255 (in 8 bit). I'v also read that values near 0 were used for some sort of TV signal syncronization.
But in real life limited range source can have such values by unknown reasons. Maybe related to compression. Maybe related to buggy encoding software. Some sort of noise.
And the only way I'v found to determine range is to look at it with eyes and see.
Mostly you will not find source in full range, because nearly all content I see is encoded in YUV and it is by standard that YUV is limited range. Thou technically it can contain full.

Plugin does same as you said, just for default it does not need range to be specified. YUV -> limited, RGB -> full.
And if by some reason you have somethig weird like YUV full range, then you use changerange.
So normally you don't need to use it :)

Example when I use it myself:
I have avisynth plugin that works only for RGB source. And I just like to use it on source without range conversion.
So I use my ImageSourceNV plugin to convert YUV to RGB without range conversion. So I get RGB in limited range (that is weird). Then I use this plugin. And then I use ImageSourceNV to convert limited RGB back to limited YUV.

For 8-12 modes. They just work same way no matter what input range you give.
If you use 8 on full range input - it will be treated as limited. So 0-15 and 236-255 levels will be just lost (cut) and middle levels will be reranged.
Same for mode 9. If you give limited range input to it you will get twice limited output. The weird thing is that I'v already found a lot of video sources with twice limited range. And the even more weird thing is that such sources are not from some home video. They are from TV, they are from streaming services and they are even from blurays!
I'v even found some cartoons, that some studios do in twice limited range. And they stream them in such way in streaming services. I don't get it why.
Twice limited range video has its lowerst blacks near 30 (instead of 16).
Example (https://imgsli.com/MzUxMzUz). <- and it is from bluray!

But also there are some sources that looks like twice limited, but they are not.
Such example is "Spy x Family" anime. Its levels look like twice limited. If you will use mode 8 on it you will get colors closer to primary. But on dark scenes you will see that too much visual information is not seen. So cartoon was designed to be in that range. And it's not some fault.

For 11 & 12 source should be full range. Otherwise result will be more weird. Image will be very bright after mode 11 if source is limited range. But you should not use 11 & 12 in some production. It just to play with. Mostly it gives weird results. But I'v found some sources where mode 11 do cool results:
5to7 (https://imgsli.com/MzUxMzQ4)
Insurgent (https://imgsli.com/MzUxMzQ5)
Insurgent (https://imgsli.com/MzUxMzUx)
But if you use mode 11 on whole movie you will get terrible result.

And mode 10 just removes colors. Range is not changed.

And for automatic detection:
I'v found that video sources mostly use first black frames to show in which range video is encoded. That's why most sources have black frames at start. But I'v also found sources in twice limited range that had these key black frames in 1 time limited range. So it's not very reliable. Much easier to do it by eyes and hands.

[Argaricolm]

I'm thinking to change mode 1 (already changed for myself and now testing). I want your thoughts about it.
So far in mode 1 after all RGB channels are processed by softlight - Saturation and Volume (from HSV) are restored to original. So hue (colors) and saturation are not changed, but only Volume. This causes image to be mostly brighter (in most cases source is darker). But it also causes result image to become oversaturated. Because original saturation is for darker image and it looks like for brighter image saturation should be lower. I have changed it to not restore saturation. So now it is also changed. It shows results like mode 3, but because Hue is original results have original colors (no greeny faces).
The question is - do you need original mode 1 or I can replace it with a change?

[DTL]

.
But in real life limited range source can have such values by unknown reasons. Maybe related to compression. Maybe related to buggy encoding software. Some sort of noise.


There are lots of reasons to have code values +-nominal black value in narrow range encoded files. One is to save from false DC offset from nominal black by cutting out 'negative' code values added by real noise or dithering process or something else.

Example:

For samples sequence

14, 16, 18

Average value is 16 and it is good black.

If you hard limit 14 under-black to 16 the output will be

16, 16, 18

Average is (16+16+18)/3=16.6 rounded to 17. So we got lost (damage) of contrast by increasing average black to about +1 code value.

[Selur]

Well, it's hard to determine is input full range or limited range
okay, then simply let the user set whether the input is full or limited and do the rest internal instead of having the user jump through hoops.

[Argaricolm]

Technically value 14 in limited color range should not exist at all. Because it is LIMITED to 16-235. And I'm talking about RGB. As it is limited in RGB. Because 0-255 range is reranged to 16-235. So 0 will become 16.

Limited range is not something that exist by itself. It is a middle in transformation.
Record Full -> Limited -> Stream -> Limited -> Full for Viewer.
And Full RGB can't be correctly converted to limited to have values < 16 or > 235. Because RGB can't be lower than 0. And 0 will be 16.
Because rerange from full to TV is R / 255 * 219 + 16.

And Y will not be < 16 in this case. Because minimum RGB in limited range is 16 16 16.
It can become lower only because of rounding problems (or maybe also because of YUV compression/decompression). Rounding problems like here (https://www.mikekohn.net/file_formats/yuv_rgb_converter.php). Where he uses Math.floor for rounding and gets Y = 15 for RGB (16,16,16).

On another part TV to Full conversion must cut all values <16 or >235. Or result will be incorrect.
This way:
(X - 16) / 219 * 255
And in the end we do clamping to 0,255 (just to be sure no negative values). Because in RGB negative can't exist.
So if you have 14 in limited range. You get 0 in full. You get 0 for 0-16 (limited) and 255 for 235-255 (limited).
You can test it using photoshop + imagesource in avisynth.
I'v created image.mkv (https://disk.yandex.ru/d/PAbbpzvGJGp3NQ) for this test.
First 5 frames are full RGB 16 16 16. To be key frames for hardware to see it is in limited range. Other frames has lines paint in photoshop that is of color (10,10,10).
If I open it in mpc-hc (MadVR decoder) it treats video as limited range and all frames are complete black. You can pause at any frame and save image as bmp. All bytes will be 0 (except only few some that are 01) - must be because of compression/decompression.
But I'v also opened it in my android tv device in Kodi (android tv version). And there it is different.
I can see lines on TV and they are black. So Kodi somehow use 10 as the lowerst limit of range (10 becomes absolute black). Maybe Kodi is not the cause because it uses hardware here for AV1 decoding.
This is example why <16 should not exist in limited color range video. Because this noise may be treated as color info and it will ruin contrast and colors because of wrong rerange. What should be black is treated as gray.

[DTL]

"Because it is LIMITED to 16-235."

It is not hard limited - it is narrow range. Where 16 is nominal black (physical zero) and 235 is nominal white. The digital domain is not completely equal to physical light domain so it can have 'negative' non-exist in positive light physics code values. But they are required for many math operations in digital domain to keep quality. So valid code values for narrow 8 bit digital are at least from 1 to 254 if compatible with SDI or from 0 to 255 if compatibility with SDI interface is not required.

"Because in RGB negative can't exist."

You still mix digital RGB and physical light RGB. The digital domain RGB can be any signed value (negative too). Only if you go at the end of digital chain to physical light RGB you make cutting out (clamping to zero) negative non-possible to display RGB power values.

"You can pause at any frame and save image as bmp. All bytes will be 0"

Typically personal computers are end-users devices to playback to the RGB physical display and they like to use 'monitor-RGB' (usually sRGB) with black at code value of 0 and no negative RGB encoded. But it was from old days when PCs where too slow for moving images processing and process only static images encoded in different from moving pictures digital. sRGB really encodes black at zero.
Nowdays if you run some software player of digital moving images (encoded in narrow YUV) it make decode and scale to PC monitor resolution and send data in sRGB domain with zero at 0 (ready to display at physical monitor without scaling). If you grab bitmap from operating system in RGB (PC sRGB) you got black at zero and no under-blacks.

If your display support YUV narrow or RGB narrow you can try to grab this data from frame buffer and see the negative under-blacks in it. It is when you connect some device like TV set/panel via HDMI and feed YUV 4:2:2 (or 4:4:4) or RGB narrow to the device.

If you work with digital moving images you typically use industry standards like ITU bt.601/709/2020 and they uses different physical levels to code values mapping (with negative Y and RGB values too). If you create software for moving pictures processing at computers it is good to start from learning industry standards for digital levels encoding and colour space conversions. They are free and open for download from ITU website.

"it's hard to determine is input full range or limited range."

You can try simple statistical analysis:
1. Count number of samples equal to 0 to Cnt_0 and equal to 16 to Cnt_16.
2. If Cnt_0 > Cnt_16 - clip mostly probably uses 'full' levels maping scheme. Else - 'narrow'.

You can do counting in the single pass using SIMD of comparison with constant 0 and constant 16 to mask and masked addition of 1 to counters.

It is not best way for very low noise content. For low noise levels it need some changes (like addition of nosie with average delta value about 16/2 ?).

From the math point of view the task is simple enough - you need to create computationally cheap and SIMD-friendly algoriphm to discriminate between 2 possible distributions of random value in the range 0..16+7 :
1. If the max number of samples are located in the code values range 0..8 - result if 'full'
2. If the max number of samples are located in the code values range 16+-7 (9 to 23) - result is 'narrow'

May be it is enough to calculate some math average in the range 0..16+7 and compare with middle value of 8. If it is below 8 - the result is 'full'. Where average may be some of mean or median or mode (see in the wiki how they are calculated - the 'mode' implementation with SIMD available in vsTTempSmooth plugin in Asd-g github - https://github.com/Asd-g/AviSynth-vsTTempSmooth/blob/master/src/vsTTempSmooth_pmode1_AVX2.cpp ).

Some description for 'math center' calculations - https://www.khanacademy.org/math/statistics-probability/summarizing-quantitative-data/mean-median-basics/a/mean-median-and-mode-review

Mean, median, and mode are different measures of center in a numerical data set. They each try to summarize a dataset with a single number to represent a "typical" data point from the dataset.
Mean: The "average" number; found by adding all data points and dividing by the number of data points.
Example: The mean of 4, 1, and 7 is (4+1+7)/3 = 12/3 = 4.
Median: The middle number; found by ordering all data points and picking out the one in the middle (or if there are two middle numbers, taking the mean of those two numbers).
Example: The median of 4, 1, and 7 is 4 because when the numbers are put in order 1, 4, 7, the number 4 is in the middle.
Mode: The most frequent number — that is, the number that occurs the highest number of times.
Example: The mode of {4, 2, 4, 3, 2, 2} is 2 because it occurs three times, which is more than any other number.

The 'mode' analysis allows to get some data about reliability of result - if there is 'no mode' in samples dataset it mean the detection is not reliable and other dataset (frame or set of frames area) need to be checked. Example sequence 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23 have 'no mode'.