Problem with adding grain

[olle83]

I was modifying the existing GrainFactory3 avs-script. Removing stuff I don't need etc.
But I noticed that with my script the grain was added to completely black frames/areas (like end credits).
I'd like to prevent this.

I'm thinking it could be done with mt_merge/mt_lut, but I don't know how to code it.
The original script had code like ".mt_merge(grainlayer2, o.mt_lut("x "+th1str+" < 0 x "+th2str+" > 255 255 "+th2str+" "+th1str+" - / x "+th1str+" - * ? ?"), U=1,V=1)"
If someone with more knowledge with Avisynth coding could help me?
Also, I'd like to the code to work with SDR and HDR videos and with all bitdepths. I'm using Avisynth+.

function AddGrain(clip clp,float "g1str",int "g1shrp",float "g1size")
{

g1str = default( g1str, 5 ) # [ 0 - ???] strength of grain
g1shrp = default( g1shrp, 66 ) # [ 0 - 100] sharpness of grain (NO EFFECT when g1size=1.0 !!)
g1size = default( g1size, 1.50 ) # [0.5 - 4.0] size of grain

#-----------------------------------------------------------------------------------

o = clp
ox = o.width
oy = o.height
sx1 = m4(ox/float(g1size))
sy1 = m4(oy/float(g1size))
sx1a = m4((ox+sx1)/2.0)
sy1a = m4((oy+sy1)/2.0)

b1 = g1shrp/(-50.0) + 1.0
b1a = b1/2.0
c1 = (1.0-b1)/2.0
c1a = (1.0-b1a)/2.0

#-----------------------------------------------------------------------------------

grainlayer1 = blankclip(o, width=sx1, height=sy1, color_yuv=$808080) .AddGrainC(g1str, 0,0,0)

grainlayer1 = (g1size == 1.0 || sx1==ox && sy1==oy) ? grainlayer1
\ : (g1size > 1.7) ? grainlayer1.bicubicresize(sx1a,sy1a, b1a,c1a).bicubicresize(ox,oy, b1a,c1a)
\ : grainlayer1.bicubicresize(ox,oy, b1,c1)

#-----------------------------------------------------------------------------------

result = o.mt_makediff(grainlayer1, U=2,V=2)

return( result ) }

#-----------------------------------------------------------------------------------
function m4(float x) {return( x<16?16:int(round(x/4.0)*4)) }
#===================================================================================

[Arx1meD]

Have you tried using AddGrainC (http://avisynth.nl/index.php/AddGrainC)?
It works fine with any color formats and bitdepths. He's also fast.
If you need a lot of grains, then apply this filter several times. And that is all.

[Boulder]

Checking the original function, could you set g1str to 0 and adjust th1 and th2 so that the darkest parts don't get any grain?

[real.finder]

what about AdaptiveGrainFactory https://github.com/realfinder/AVS-Stuff/blob/Community/avs%202.5%20and%20up/sAddGrainCT.avsi it work with any bitdepths

it's new and still in test status

[olle83]

Looks like I found the solution myself after doing some Googling and finding couple of helpful guides.

By adding this code I can control how bright pixels get the grain or not.
Further tweaking is still needed to make it work automatically with all bitdepths.

blanklayer = blankclip(o, color_yuv=$808080)

grainlayer = blanklayer.mt_merge(grainlayer1, o.mt_binarize(threshold=16*256, paramscale="i16"), U=1,V=1)

[FranceBB]

Adjusting the brightness doesn't automatically make it work with all bit depth.
8bit, 10bit, 12bit, 14bit, 16bit planar and 32bit float can easily be SDR 100nits with three different colour matrices each and with no colour curve applied, therefore the number of nits scales linearly with the bit value up to 100 nits, but you can have other kind of curves for HDR. In HLG, Hybrid Log Gamma you start like a normal SDR and the you slowly curve in the opposite direction up to 1000 nits so that the curve plotted looks like a sigmoid, while with PQ you start as logarithmic and you keep going. Same goes with different logs that have different stops/nits like Slog3 or Clog3 etc. The worse part is that Avisynth does a pretty bad job on passing this information from filter to filter after getting it from indexers and many plugins are just not HDR aware so you would have to add an option to the user to select a certain mode manually...

As to AddGrainC, it works great but it's SDR only, remember that! Using it on an HDR source, like in PQ for instance, would introduce lots of 10'000 nits spikes at each grain "object" on an otherwise lower nit video, thus making it not only wrong but painful to watch and incredibly challenging to display for everything but OLED-like technologies...
(I mean an LCD would hate such a signal)

[real.finder]

But I noticed that with my script the grain was added to completely black frames/areas (like end credits).
I'd like to prevent this.

try this https://github.com/Asd-g/AviSynthPlus-Scripts/blob/master/AdaptiveGrain.avsi

Adjusting the brightness doesn't automatically make it work with all bit depth.
8bit, 10bit, 12bit, 14bit, 16bit planar and 32bit float can easily be SDR 100nits with three different colour matrices each and with no colour curve applied, therefore the number of nits scales linearly with the bit value up to 100 nits, but you can have other kind of curves for HDR. In HLG, Hybrid Log Gamma you start like a normal SDR and the you slowly curve in the opposite direction up to 1000 nits so that the curve plotted looks like a sigmoid, while with PQ you start as logarithmic and you keep going. Same goes with different logs that have different stops/nits like Slog3 or Clog3 etc. The worse part is that Avisynth does a pretty bad job on passing this information from filter to filter after getting it from indexers and many plugins are just not HDR aware so you would have to add an option to the user to select a certain mode manually...

As to AddGrainC, it works great but it's SDR only, remember that! Using it on an HDR source, like in PQ for instance, would introduce lots of 10'000 nits spikes at each grain "object" on an otherwise lower nit video, thus making it not only wrong but painful to watch and incredibly challenging to display for everything but OLED-like technologies...
(I mean an LCD would hate such a signal)

I think it's better to convert to SDR and then do filtering and then back to HDR, but I think that will need some method to add back the HDR data, or maybe it better to use XYZ?

[FranceBB]

I think it's better to convert to SDR and then do filtering and then back to HDR, but I think that will need some method to add back the HDR data


Uhm... is it even doable?
I mean, if you convert to SDR and you applied some kind of linear transformation like with a LUT which maps points of a curve into another, there's no way that such a transformation is going to be an isomorphism, which means that you're gonna be mapping more points of the input curve to the very same point in the output curve, hence you lose info.

https://qph.fs.quoracdn.net/main-qimg-d72f6e33450ce667acb6bbba4688d63a

In order not to lose info a transformation has to be both 1-1 and onto, hence an isomorphism.

https://upload.wikimedia.org/wikipedia/commons/thumb/a/af/Gen_bijection.svg/400px-Gen_bijection.svg.png

TL;DR if you get 1000 nits in a logarithmic curve like PQ and you map them to a linear curve like the one used in Linear BT2020 100 nits SDR, you can't go back to PQ at 1000 nits without "inventing" values as you have "lost" them when you mapped.


maybe it better to use XYZ?

Using XYZ is actually recommended for HDR post-processing and it's also one of the reason why Jean Philippe uses it and wrote a whole paper about his plugins.
Unfortunately, though, in Avisynth XYZ support is lacking and it's done through a fake RGBPS... :(
I would really not like it to end up like stacked/interleaved which took years of people building around them before eventually introducing support, but looks like right now that's the only way... :(

[real.finder]

well, I didn't work with HDR before and don't have enough information about it, but maybe if it convert to float SDR (that has out of range info than 0..1) maybe it will not loss info?

Using XYZ is actually recommended for HDR post-processing and it's also one of the reason why Jean Philippe uses it and wrote a whole paper about his plugins.
Unfortunately, though, in Avisynth XYZ support is lacking and it's done through a fake RGBPS... :(
I would really not like it to end up like stacked/interleaved which took years of people building around them before eventually introducing support, but looks like right now that's the only way... :(

maybe we need avs+ devs to add basic XYZ aka CIE 1931 and CIE Lab (YUV like) support

[olle83]

I'm using this only for BD and UHD-BD sources.

I did some testing and it worked alright with HDR content, though I'm only adding a very small amount of grain to it (strength was at 0.92).

I used DGDecNV to load video, then added the noise in 16bit and finally used z_ConvertFormat to crop, downscale and get to 10 bits. Which I then piped using Avs2YUV to x265 in commandline.

[FranceBB]

Can you show us the waveform before and after adding grain?

I'll try to put something together to show you what I mean anyway when I get to the office... as soon as my body realises that I have to get out of bed, get dressed and go to work to survive even if I'm tired...

[olle83]

I don't know anything about waveforms, how do I check them?

[FranceBB]

TurnRight.Histogram.TurnLeft #luma
Histogram("color2") #chroma

Or using my VideoTek() (https://forum.doom9.org/showthread.php?p=1832846)

But... I found something weird...
Ok, so this is the original waveform of the HDR PQ stream, which peaks at a relatively low number of nits: Link (https://i.imgur.com/hITIYOb.png)

Now what I thought was going to happen when adding grain was to find some dots (corresponding to the grain) every here and there, way higher in the waveform, so at an higher level, which would have ended up being far too bright, but it didn't happen! Instead, this is what it looks like: Link (https://i.imgur.com/7UwB3HJ.png)

This is because I was thinking about grain as completely synthetic rather than "composed" from neighbouring objects. I guess the "luck" here is that AddGrain is actually using a sort of mask to make sure that grain isn't totally synthetic but matches the levels of the source and, since the source is a logarithmic curve, even thought the filter has no clue about what it is, it's taking into account the neighbors which are totally logarithmic, therefore the deviation from the curve is made by points close to the curve, hence it's almost negligible.
Yes, you can see it being out of range in the lower part, but on the other hand I exaggerated the values to show you the difference, so...
Anyway, I'm positively surprised to see that adding Grain with AddGrainC is actually better than I thought, even if it's not HDR aware...

[feisty2]

An isomorphism is a structure-preserving bijective mapping. A bijective function is not automatically an isomorphism unless it preserves the mathematical structure of its input object. Don’t sprinkle a mathematical term around if you don’t know its meaning.

[FranceBB]

An isomorphism is a structure-preserving bijective mapping.


Right


A bijective function is not automatically an isomorphism unless it preserves the mathematical structure of its input object.


Ok, right, it's true, my bad, but the point here is that you can't map back and forth without losing anything as you would be literally mapping more than one point to the very same point and once you've done that you can't really go back, which is what real.finder was suggesting at first. But I give you that, I mean, even if we were to map each single point to individual others and we can go back, there's no guarantee that the structure would be preserved, so, yes, you're right, having bijective function doesn't necessarily mean that we would have an isomorphism. Still, I think you knew very well what I meant in this very scenario and in this very context...


Don’t sprinkle a mathematical term around

Sharp as always, Feisty eheheh but that's alright, I know you may sound "hard" but you do it for the sake of the community and it's alright, if something is "imprecise" it should be pointed out, especially here in the public 'cause it will stay forever and people shouldn't come here reading things that are not precise. :)
So... thank you for the clarification. ;)

[feisty2]

I brought it up because there are video-related bijective transformations which are not isomorphism, the obvious example being "transfer characteristics" (gamma curve).
it is bijective (ignoring precision loss), you can convert linear light to a gamma compressed space and vice versa, but it is not structure preserving.
if "transfer characteristics" were an isomorphism over, say, convolution, we would have the following theorem: LinearToBT709(Convolution(x)) = Convolution(LinearToBT709(x)), which is obviously not true because otherwise we wouldn't have things like gamma aware resizing which are dedicated to solve this problem.

[olle83]

Here are my test pictures.
Nothing added: http://www.imagebam.com/image/c783b61372562097
Grain added: http://www.imagebam.com/image/8daa821372562094

Judging from the histogram it's clear that grain here is added as an offset to original pixel luma values. Which is how mt_makediff is supposed to work like. Some pixels remain original, some get -2, +1, -1, etc... offsets.

[StainlessS]

I've no idea what I'm talkin' bout [and no idea what you fella's is talkin bout either],
but I think I saw somewhere in this thread that grain is first rendered to grey blankclip [relative Y=128].
Just thought I'de point out that TV_Levels Y mid point is 125.5 [ 126 usually used, Round((16+235)/2.0) ].
Maybe not an issue though.
EDIT: 8 bit Subtract() result is relative 126, not 128.