Dogway's Filters Packs

[Dogway]

I will have a look now. Thanks for the call snippets!


EDIT: Test now, only need to update TransformsPack (https://github.com/Dogway/Avisynth-Scripts/blob/master/TransformsPack.avsi).

[real.finder]

After the update I will give it a try, your point made me curious. Also, I meant to ask but I always forget:
The ancient QTGMC version doesn't use vinverse, your does, can you please explain what is the function of vinverse in QTGMC?

I'm working on my own bob, starting from scratch I got good results with less shimmering and blending, but it still has some problems in particular cases which QTGMC doesn't have, so I thought to just mix it with QTGMC and see if get better.

I haven't delved too deep into QTGMC internal logic, you might ask real.finder for that. My work there was to optimize filters, expression and small blocks of code, and some logic flow besides adding the customary HBD and frameprops support. I plan to update MVTools related settings with the Zopti outcomes but no further.

I recommend you to check initial versions of QTGMC to understand the core principle and have a look at the deshimmer filter. For a simple antialiased dumb bob I implemented nnedi3 into ex_bob(), might help for ideas.

For deep_resize() I'm now adding a regrain option as a cheap hallucination option and fixing some low ratio resize bugs.

it happened here http://forum.doom9.net/showthread.php?p=1921844#post1921844 (read until #72 post) and it's not mandatory :)

[mp3dom]

EDIT: Test now, only need to update TransformsPack (https://github.com/Dogway/Avisynth-Scripts/blob/master/TransformsPack.avsi).
Yes, it works now, thank you!
However, I'm facing another issue. Using colorbars as a source "reference" I see this:
yuv444p16/p10/p8 sources -> output is OK
yuv422p16/p10 sources -> lower part of the bars seems wrong
yuv422p8 sources -> output is OK
yuv420p16/p10/p8 sources -> output is OK

[Dogway]

Thanks, yes a typo, 'src_width' and 'src_height' should default to 0.0 I think. Fixed!

[Ceppo]

I still get an error:
https://i.ibb.co/ydVsBg0/error.jpg

colorbars(width=3840,height=2160,pixel_type="yv16")
deep_resize(720,576)


line 828:
Y = Eval("Y." + resampler + cplaceY + (noring ? Ynr : ""))
line 832:
}
line 1072:
} }

@real.finder, thanks!

[mp3dom]

DPID is an additional (external) DLL, but even with that you still need z.lib DLL (avsresize)

@Dogway: Thanks!

[Ceppo]

@mp3dom

Thanks, it works now!

[Ceppo]

@Dogway, I need your opinion:

Source (https://i.ibb.co/SmHTXV7/source.png)
deep_resize(720,480).spline36resize(1920,1080) (https://i.ibb.co/pxZ1x4d/deep-resize-720-480.png)
CResize(720,480).spline36resize(1920,1080) (https://i.ibb.co/4fjr4vx/CResize-720-480.png)

Default deep_resize is too aliased for my tastes, is possible to tune it to get it aliasing free like CResize, and save more details?

[Dogway]

Use a better upscaler:

deep_resize(720,480)
deep_resize(1920,1080,grain=0)


---------------------------------------

By the way, I updated ScenesPack v1.0 final. I merged SceneRange() and SceneStats() so performance is not harmed. I also included 'mode', as I added an option for '_SceneMotion' frameprops. Now you can do crazy things like the following:


Level normalization
SceneStats() or ReadStats("D:\Stats.log")

ScriptClip(function [] () {
sts = propGetAsArray("_SceneStats")
ex_levels(max(16,sts[0]),1.0,min(235,sts[1]))
} )

Higher Temporal Radius for calm scenes
SceneStats("Motion") or ReadStats("D:\Stats.log")

ScriptClip(function [] () {
mo = propGetFloat("_SceneMotion")
mo = -5*mo+0.85
SMDegrain(tr= round(mo*10))
} )


I see a rise in ScriptClip abusing, sadly all filters need to pass through ScriptClip as frame properties at frame#0 are not dynamic, details here (https://forum.doom9.org/showthread.php?p=1966333#post1966333) by pinterf.

[Ceppo]

I can't see why it is related to the upscale filter, it is just to make easier the comparison, and spline36 is pretty much neutral, if I upscale with like nnedi it would end up fixing the aliasing, which is pointless cuz TVs don't use nnedi or deep_resize for the upscaling, at least not mine.

[Dogway]

I don't know what scalers SmartTVs use but I thought they were some kind of edge directed upscalers which would be similar to nnedi3. A good upscaler is one that antialiases so this is not a problem of downscaling but upscaling.

A reference "cheap" upscaler is typically lanczos4 not spline36 (although I do like bicubic "precise" also). lanczos antialiases a bit better than spline36 and 4 taps adds more sharpness at the cost of almost no additional halo because the 4th lobe is negative which in gamma encoded images is almost invisible.

I mean I didn't invent DPID, you can read the paper. All "recognized" downscalers would be pretty aliased in your terms; DPID, SSIM_downsample, "Didée", "Zopti"...

CResize discards too much information by blurring, it can be nearly replicated with RobiSoft:

bicubicresize(720,480,0.679623,0.160189)
spline36resize(1920,1080)

[Ceppo]

Thanks for your answer.

I did try your suggestion, but all of them seem to output too much aliasing on the character's edges. Then I just did CResize(720,480,aastr=1) and got a close result with the details and less aliasing on the edges.

Do you have some idea to better detect "visible" aliasing?

What I do now, is to just use a sharp resize and a soft one then I make the difference. I apply antialiasing to the difference then I add the difference to the soft one to recover details. Sangnom2 and nnedi3 seem too strong for the job, so I had to make CAntialiasing for it, but it still doesn't cut it, as you said.

[Dogway]

To detect visible aliasing use your preferred antialias.

AA = SantiagMod(strh=1,strv=1) # My favorite (nnedi3 at its core)
ex_makediff(AA) # diff centered at range_half

Your current method is what is commonly used to remove ringing, the difference of Hermite vs your kernel.

[Ceppo]

Thanks :)

[anton_foy]

No rush just eagered for the Zopti ootimization of SMDegrain, when is it going to be released approximately? And is this optimization going to affect modes such as temporalsoften aswell?

[Dogway]

No rush just eagered for the Zopti ootimization of SMDegrain, when is it going to be released approximately? And is this optimization going to affect modes such as temporalsoften aswell?

I said I would start end of this week, I noticed small issues in SceneStats so I will work on it tomorrow and probably start with Zopti correlations on Monday.

[anton_foy]

I said I would start end of this week, I noticed small issues in SceneStats so I will work on it tomorrow and probably start with Zopti correlations on Monday.

Sorry I missed that okay I will await with great anticipation :)

[DTL]

"what scalers SmartTVs"

For cheap end-users displays it mostly probably something close to bicubic (with low kernel size). Because it is most universal and cheap and not show too much ringing at badly conditioned content. For more expensive end-user displays it may be also some non-linear and if user need best result - the downscaler need to be tuned for given display's scaler.

[kedautinh12]

I think they just use fastest and don't show too much ringing scalers in SmartTVs

[Dogway]

I think for most SmartTVs and for some years now (since 4K implementation) they include a dedicated chip for upscaling even for mid tier models (Quantum Processer 4K for Samsung, LG Tru Engine 4K for LG, etc).
LG (https://www.lg.com/us/support/video-tutorials/ultra-hd-4k-tv-upscaling-uhd-tv-video-CT10000018-1432737109927) and Samsung (https://www.samsung.com/us/televisions-home-theater/tvs/qled-4k-tvs/55-q70a-qled-4k-smart-tv-2021-qn55q70aafxza/#benefits)

[Dogway]

I finished ScenesPack to what I consider a stable version so please give it a try. I know there are some false positives/negatives scene changes, but overall it should give more matches than SCSelect, or other SC filters. There are ways to improve the SC algo but I'm done here for now (will only update as soon as pinterf adds 'average' stat to PlaneMinMaxStats() )
I will start today with the Zopti correlations by porting CoS to Python and doing the correlations there. It will take me some time though. Then I will try to draw some non-linear functions that correlate different parameters for MVTools.

[tormento]

The remastered Godfather box arrived me since a couple of days and even if they recovered the tint, that was yellowish in the last set I had, they did a terrible, horrible work with compression and perhaps some denoising.

Just look at the clusters of noise in these two images:

https://i.lensdump.com/i/rhPyVK.md.png (https://lensdump.com/i/rhPyVK)

https://i1.lensdump.com/i/rhP5vZ.md.png (https://lensdump.com/i/rhP5vZ)

@Dogway: here you can find some clips of the movies.

Godfather_desk (https://krakenfiles.com/view/cVyqTvw78p/file.html)
Godfather_doorface (https://krakenfiles.com/view/zied2pS3bt/file.html)
Godfather_greenwall (https://krakenfiles.com/view/97v5hUlb2M/file.html)
Godfather_greysuit (https://krakenfiles.com/view/bcip6D1e3T/file.html)
Godfather_hug (https://krakenfiles.com/view/L2v5zDArmu/file.html)
Godfather_phone (https://krakenfiles.com/view/53T9Ju2V6S/file.html)
Godfather_picture (https://krakenfiles.com/view/x5Do3vQkaX/file.html)
Godfather_shades (https://krakenfiles.com/view/sQi35igzPo/file.html)
Godfather_table (https://krakenfiles.com/view/UJjUlrmRSh/file.html)

I have tried to play a little with SMDegrain switches but I can't find consistent settings that can be applied to a whole movie recode. The worse parts are in flat areas of colors and in dark scenes.

To get rid of the noise on the desk of Godfather_desk, I would lose tiny details in the lighter scenes. I was thinking: as darker scenes suffers from higher film grain noise (perhaps higher ISO film?), wouldn't be possibile to adapt denoising strenght to the general darkness of a scene?

In the mean while, what kind of script could I use on such a difficult movie?

[Dogway]

Woow, what happened to the doorface scene! looks like they copy-pasted the face from another source without postpro of any kind.

In these old films grain typically shows when a high ISO stock is used, for example for dark scenes, so you can run per-scene stats to detect them and perform more aggressive denoising like explained in a previous post (https://forum.doom9.org/showthread.php?p=1966366#post1966366).

My recommendation for performance is to do a stats prepass with:
SceneStats("Range+Stats+StdErr",lookahead=90,path="D:\mystats.log")

Then load the stats with:
ReadStats(path="D:\mystats.log")

In any case I found SceneStats() has issues with long scenes (over 20 seconds) as it's very slow.
I was waiting for pinterf to update PlaneMinMaxStats(), but I will upload what I have now and try to check feedback to see if I can speed things up.

You can detect dark scenes with

ScriptClip(function [] () {

sts = propGetAsArray("_SceneStats")

avg = sts[4]
med = sts[5]
skw = sts[6]

dark = avg < 80 && med < 60 || skw > 0.8
} )

[Julek]

You might be interested in this:
https://github.com/dnjulek/jvsfunc/blob/main/jvsfunc/expr.py#L10-L55

I did ccd using only Expr, I saw that AVS does not yet have a native plugin (https://forum.doom9.org/showthread.php?p=1947846#post1947846) and I think this might help.

To help with vs-expr: https://github.com/AkarinVS/vapoursynth-plugin/wiki/Expr#translating-avs-expr-expressions

[Dogway]

Thanks for the note. I might need to read a bit more about CCD denoise, is it some kind of debayer denoise?
Looks like it can be implemented, as for the rest we lack (https://forum.doom9.org/showthread.php?p=1946915#post1946915) 'sort' operator in Expr. I'm curious to know the performance gap compared to my sorting network approach.

Can you compare:
ExtractY()
ex_median("median5")
Prefetch(6) # for 4C/8T or tune according to CPU
to
y_clip = core.std.ShufflePlanes(planes=0, colorfamily=vs.YUV)
medianblur(y_clip,2)

And also with "median7" and n=3


---------------


As for SceneStats() code wise it is correct now, but it suffers a lot with long scenes/shots (>20s). This is mainly because due to lack of a loopback, for each frame we do a lookahead and a lookback to set start and end scene frames. Ideally we should only do this once at start of scene and then copy _SceneRange props from previous frame (here's what we need a loopback/feedback for) if current frame is not a SC.

[Julek]

Thanks for the note. I might need to read a bit more about CCD denoise, is it some kind of debayer denoise?
You can find a more complete explanation here: https://github.com/DomBito/VapourSynth-CCD/blob/main/README.md


Tests:


BlankClip(length=5000, width=1920, height=1080, pixel_type="Y16")

ex_median("median5")
Prefetch(12)
Result: https://i.imgur.com/VcISpEt.png

from jvsfunc import medianblur
import vapoursynth as vs
core = vs.core

src = core.std.BlankClip(None, 1920, 1080, vs.GRAY16, 5000)
blur = medianblur(src, 2)

blur.set_output()
Result: https://i.imgur.com/IUCU4cZ.png

BlankClip(length=5000, width=1920, height=1080, pixel_type="Y16")

ex_median("median7")
Prefetch(12)
Result: https://i.imgur.com/zab2gdQ.png

from jvsfunc import medianblur
import vapoursynth as vs
core = vs.core

src = core.std.BlankClip(None, 1920, 1080, vs.GRAY16, 5000)
blur = medianblur(src, 3)

blur.set_output()
Result: https://i.imgur.com/Tc8hpLI.png

CPU: AMD 3700X

#EDIT: now that I noticed the note about Prefetch, I will redo the tests, as I don't use AVS I'm not familiar with the tool
#EDIT2: updated tests

[Dogway]

Wow, about 2.5x the performance, could be very worth it, not sure if it makes use of internal optimizations with LLVM or something though.

[Dogway]

I updated ScenesPack to deal with issues on long scenes, now it only performs lookahead on scene start frames (not every frame as previous versions). I also optimized other code blocks and generally performs about 30% faster. There are still some issues with StdErr mode, so avoid that mode or 'ALL'.

Later I realized that the culprit for long scenes was a low memory assignment, so for good measure I recommend going no less than setmemorymax(2048*3).

I might ask you now what would you consider a "dark" frame, ideally 'dark' as synonymous of high ISO film stock. We might agree that a low scene's average and median would depict a dark frame. But "dark scenes" could also show a bright spotlight. I didn't find any paper talking about ISO or dark scene detection but would you consider a frame/scene with 90% below 8-bit 80 avg/median value and 10% <200 bright a dark frame/scene?

In other words:
dark = avg < 80 && med < 60
or
dark = avg < 80 && med < 60 && mx < 150

Now I also feel confident to finish the SAD analysis tool, it's not an unsupervised tool though nor a fast one, but can help on defining scene's grain density.


I also implemented reading '_SceneStats' on several filters like ex_retinex, softlimiter, greyworld, and made ex_autolevels. Will be uploading soon.

[anton_foy]

I updated ScenesPack to deal with issues on long scenes, now it only performs lookahead on scene start frames (not every frame as previous versions). I also optimized other code blocks and generally performs about 30% faster. There are still some issues with StdErr mode, so avoid that mode or 'ALL'.

Later I realized that the culprit for long scenes was a low memory assignment, so for good measure I recommend going no less than setmemorymax(2048*3).

I might ask you now what would you consider a "dark" frame, ideally 'dark' as synonymous of high ISO film stock. We might agree that a low scene's average and median would depict a dark frame. But "dark scenes" could also show a bright spotlight. I didn't find any paper talking about ISO or dark scene detection but would you consider a frame/scene with 90% below 8-bit 80 avg/median value and 10% <200 bright a dark frame/scene?

In other words:
dark = avg < 80 && med < 60
or
dark = avg < 80 && med < 60 && mx < 150

Now I also feel confident to finish the SAD analysis tool, it's not an unsupervised tool though nor a fast one, but can help on defining scene's grain density.


I also implemented reading '_SceneStats' on several filters like ex_retinex, softlimiter, greyworld, and made ex_autolevels. Will be uploading soon.

Now this is interesting! But did you take into consideration the red channel? With cmos red tint combined with low exposures makes for noisy footage. I mean high ISO usually gets more noise when light is on the red side. Blue light (daylight balanced LED lights for example) is less noisy. EDIT: Sorry I read wrong (tired) you said film stock and I talked about high ISO video.

[Dogway]

Thanks, yes that is an idea, probably for grain detection.
I was thinking a bit and researching, and made up a 2D matrix for evaluating scene exposure.
Raw exposure takes the scene's median and classifies it within the 5 blocks/zones (ymax/5)
Contrast does the same but takes the range (th_max - th_min) and classifies it in 5 zones as well. It acts as a bias for Raw exposure.

| \ | | | | | |
| \Med| V.Dark | Dark | Mid | Bright |V.Bright|
|Rng \ | | | | | |
|-------|--------|--------|--------|--------|--------|
| | | | | | |
| Flat | 0 | 1 | 2 | 3 | 4 |
| | | | | | |
|-------|--------|--------|--------|--------|--------|
| | | | | | |
| Mid | 0 | 1 | 2 | 3 | 4 |
| | | | | | |
|-------|--------|--------|--------|--------|--------|
| | | | | | |
| Cont | 1 | 2 | 3 | 4 | 4 |
| | | | | | |
|-------|--------|--------|--------|--------|--------|
| | | | | | |
| V.Cont| 2 | 3 | 4 | 4 | 4 |
| | | | | | |
|-------|--------|--------|--------|--------|--------|
| | | | | | |
| Bimod | 3 | 4 | 4 | 4 | 4 |
| | | | | | |
|-------|--------|--------|--------|--------|--------|


I think I should decouple 'th' from the calculation of th_min and max and lock it at 1%. I also built a grid to show what 'th' represents in pixels in 'show=true' mode. Uploading in a moment.


EDIT: Updated to v3.2 to fix Exposure expression. Also converted divisions to multiplication for performance.
Here's a snapshot of the 'show' panel where it shows all the possible stats.
Each square of the grid represents the percentage of 'th' so you can figure out the size of discarded pixels for 'th_min' and 'th_max' (in this case 1%).
Pearson's skewness wasn't accurate for Exposure detection, as you can have negative skewness (meaning bright) in the lower portion of the histogram, and it was sensitive to outliers.

https://i.imgur.com/Qd4Lbdkh.png (https://i.imgur.com/Qd4Lbdk.png)

[Shinkiro]

broken flatmask
Script error: ex_retinex does not have a named argument "tv_range"
(C:/AviSynth+/plugins64+/MasksPack.avsi, line 161)
(New (3), line 24)

[Dogway]

Thanks, uploaded now. I will be uploading now all my left over internal commits.

[Fjord]

MasksPack v6.2 error - line 580, column 149: missing comma after dct=0

[Dogway]

MasksPack v6.2 error - line 580, column 149: missing comma after dct=0

Thanks! Was a bit busy and couldn't finish my uploads. Updates done now.

[Boulder]

Do you have any recommendations for a subtle sharpener to apply after a 1080p -> 720p or a 4k -> 1440p/1080p downscale? This would be just to restore a bit of sharpness lost during downscaling.

[Dogway]

Do you mean edge sharpness? Did you try deep_resize() with some of the recommended downscalers? I found them to work quite good. For example yesterday I had to resize down a book cover and DPID didn't work as fine as I thought but SSIM2 did. Bicubics though added a bit of overshoots. ex_unsharp() also works fine for subtle sharpening all around.

This reminds me that I need to update TransformsPack because my last internal fix was indeed fixing SSIM2 mode.
I have been a bit busy with ScenesPack (finished already) and backing up my 870 EVO that I use for all my programs' cache which started failing just one year after buy! It's in warranty so I hope they replace it back.

[Boulder]

Do you mean edge sharpness? Did you try deep_resize() with some of the recommended downscalers? I found them to work quite good. For example yesterday I had to resize down a book cover and DPID didn't work as fine as I thought but SSIM2 did. Bicubics though added a bit of overshoots. ex_unsharp() also works fine for subtle sharpening all around.

This reminds me that I need to update TransformsPack because my last internal fix was indeed fixing SSIM2 mode.
I have been a bit busy with ScenesPack (finished already) and backing up my 870 EVO that I use for all my programs' cache which started failing just one year after buy! It's in warranty so I hope they replace it back.

I briefly tested deep_resize yesterday at default settings, but it produced a clearly blurrier result than a Spline36 + FineSharpPlus(0.1) and didn't look at it any further then. I usually use Zopti to process the Didée Bicubic idea of negative b and positive c using MDSI or GMSD, but in this case the BDs are very grainy (but mostly very high quality for an 80s TV show) and the bitrate goes through the roof in encoding. For these cases, I'm looking for a middle way solution between Spline36 and a full Didée :)

[Dogway]

Well, that's very strange since DPID captures the most important features for downscaling, it's a bit like downscaling in linear light, and yes it can be a bit blurry in some cases but I hard coded it to lambdaY=0.5 which gave me the most natural look after upscaling back with reference upscaler. In this case you can use SSIM2 as it will give you slightly sharper output.

I'm curious to see if there's value in edge/flat mixing for downscale, as you say your source is grainy so you might want a sharp edge and a soft flat, maybe share a sample so I can have a look. I implemented your 'Zopti' coefficients as scaling preset so you can also try deep_resize(1280,720,edge="Zopti2"), that's -0.6,0.4. I found -0.6,0.3 ("Zopti") to be more neutral though, maybe with softer 'flats'.

[Boulder]

Well, that's very strange since DPID captures the most important features for downscaling, it's a bit like downscaling in linear light, and yes it can be a bit blurry in some cases but I hard coded it to lambdaY=0.5 which gave me the most natural look after upscaling back with reference upscaler. In this case you can use SSIM2 as it will give you slightly sharper output.

I'm curious to see if there's value in edge/flat mixing for downscale, as you say your source is grainy so you might want a sharp edge and a soft flat, maybe share a sample so I can have a look. I implemented your 'Zopti' coefficients as scaling preset so you can also try deep_resize(1280,720,edge="Zopti2"), that's -0.6,0.4. I found -0.6,0.3 ("Zopti") to be more neutral though, maybe with softer 'flats'.

Here's a short sample:
https://drive.google.com/file/d/1mITC71QTGeJyYk4MyUl8EWIvcJr-wH2y/view?usp=sharing

It might be a good idea to use a softer flat area resizer as the grain is rather heavy and needs a little damping down. Zopti or Zopti2 are a bit sharper than DPID, especially if you take a look after upscaling back to the original resolution.

By the way, edge="SSIM2" gives an error "CombinePlanes: source has no such plane U"

[Dogway]

I see now. Yes DPID didn't yield good results either on this source, I think it was trained with clean sources, I will change default downscaling kernel to SSIM2, it represents source accurately and it's sharp, so for this case it also enhances grain. I think I will implement downscaling edge/flat mixing but not as default.

I did some tests upscaling back with deep_resize() (defaults settings).
I found spline36+finesharp a bit plasticky in the look (like a median filter), spline36 plain works fine as well for sources of these kind without too much detail but I found Zopti2 to be sharper. So in this case SSIM2 for accuracy (probably not worth it) and Zopti2 for speed, both edge limited after I update the script.

source.....................................................Spline36+finesharp.....................................................Zopti2.....................................................SSIM2
http://i.imgur.com/GCib3Khm.png (https://imgur.com/GCib3Kh.png)http://i.imgur.com/kkrXoHnm.png (https://imgur.com/kkrXoHn.png) http://i.imgur.com/eTTBq6Qm.png (https://imgur.com/eTTBq6Q.png) http://i.imgur.com/SAEZpqAm.png (https://imgur.com/SAEZpqA.png)


The error, yes as I said above it's fixed internally but haven't uploaded yet, I will between today and tomorrow after I also update SoftLimiter() for SceneStats.


EDIT: Edited deep_resize(), it didn't look good though, no good settings avoid the image from looking broken. grain sneaks in and the transition to flat->edge is not smooth.
deep_resize(958.0, 720.0,edge="SSIM2",flat="SoftCubic100",show=false,th=0.6,elast=9)
http://i.imgur.com/ADC77hmm.png (https://imgur.com/ADC77hm.png)

In my opinion I would just do Zopti and if needed compression run SPresso or some subtle degraining, before or after resize.

[Boulder]

EDIT: Edited deep_resize(), it didn't look good though, no good settings avoid the image from looking broken. grain sneaks in and the transition to flat->edge is not smooth.
deep_resize(958.0, 720.0,edge="SSIM2",flat="SoftCubic100",show=false,th=0.6,elast=9)
http://i.imgur.com/ADC77hmm.png (https://imgur.com/ADC77hm.png)

In my opinion I would just do Zopti and if needed compression run SPresso or some subtle degraining, before or after resize.

Yes, I agree that it's just a bit too difficult to process so that it won't start looking strange. I'm probably resorting to a simple Spline36 downscale, I already do a combination of a very light SPresso and a light MDegrain + contrasharpening before the resizing part.

[Shinkiro]

I used this expression mt_lut("x "+string(mthr)+" <= x 1 >> x 1 << ?", U=1, V=1), tell me how to write it correctly for ex_lut()?

[Shinkiro]

I can't figure out what I'm not doing wrong, seem all the clips YUV420P16
mthr = 34

ConvertBits(16)
Source=last
mask1b=last.ex_edge("kirsch",6,10)
mask2b=last.ex_edge("kirsch",4,7)
mask=last.ConditionalFilter(mask1b, mask2b, "AverageLuma()",">","50").mt_lut("x "+string(mthr)+" <= x 1 >> x 1 << ?", U=1, V=1).RemoveGrain((980>960) ? 20 : 11, -1)

deb = Source.neo_f3kdb(sample_mode=2, Y=68, Cb=68, Cr=68, grainy=54, grainC=40, range=15, dynamic_grain=true)
ex_merge(deb ,Source ,mask, luma=true, Y=3, UV=3)
returns an error

why does he say that it is not so, or does ex_merge not know how to handle 10-16bit?
I found the culprit of this error, it turned out to be a script SeparateResize_v1.6.avsi (https://github.com/dsamahentai/AviSynth_Filters/blob/master/SeparateResize_v1.6.avsi)
Everything works as it should without it

[kedautinh12]

I used this expression mt_lut("x "+string(mthr)+" <= x 1 >> x 1 << ?", U=1, V=1), tell me how to write it correctly for ex_lut()?

maybe:
bi = BitsPerComponent(input)
mthrHBD = ex_bs(mthr, 8, bi, true)
ex_lut("x {mthrHBD} <= x 1 >> x 1 << ?", UV=1)

[Shinkiro]

maybe:
bi = BitsPerComponent(input)
mthrHBD = ex_bs(mthr, 8, bi, true)
ex_lut("x {mthrHBD} <= x 1 >> x 1 << ?", UV=1)
I get this error
Expr: Failed to convert '{mthrHBD}' to float
(C:/AviSynth+/plugins64+/ExTools.avsi, line 66)

[Dogway]

Use Format() like:
bi = BitsPerComponent(input)
mthrHBD = ex_bs(mthr, 8, bi, true)
ex_lut(Format("x {mthrHBD} <= x 1 >> x 1 << ?"), UV=1)

I didn't know Expr() supported bitshifts operators...

[Shinkiro]

Now such a error
Expr: Failed to convert '>>' to float
(C:/AviSynth+/plugins64+/ExTools.avsi, line 66)

[Dogway]

Ah, then it doesn't support them, do manually:
ex_lut(Format("x {mthrHBD} <= x 0.5 * x 2 * ?"), UV=1)

[Shinkiro]

Ah, then it doesn't support them, do manually:
ex_lut(Format("x {mthrHBD} <= x 0.5 * x 2 * ?"), UV=1)

It works that way, thank you!

[kedautinh12]

It works that way, thank you!

Can you share script you were change to expr??