Dogway's Filters Packs [Archive] - Page 29

DTL

11th August 2022, 14:56

SMDegrain(tr=2,"

Max possible temporal denoising amplitude is about doubled with about 4x of tr-value. So tr=2 with 4 ref frames may only about half of noise amplitude. To decrease noise more it is required something like 2x4=8 or 2x16=32 (about 4 times less noise in lucky case of all ref blocks found and used).

With too big noise level and too high thSAD it is good to set thSCD1 not below thSAD value.

I think it is visibly faster to increase tr-value with single pass. Also if any prefiltering used - all th-values require re-adjustment. Because prefiltering typically make SAD values lower.

kedautinh12

13th August 2022, 01:49

Hi, i report that deep_resize had problem when downscale from 4k to 1080p
I tried with this script:
deep_resize(1920,edge="Zopti1080")
But in video, her hair is wrong color
My sample: https://youtu.be/49R_nMEW7X8

Dogway

13th August 2022, 10:37

Thanks, updated TransformsPack - Main with a fix and some cosmetics.

LeXXuz

13th August 2022, 14:53

Dogway I noticed some faint distortions in my encodes, barely noticeable.
So I played around with the preview mode a little and saw a lot of erroneous frames of the prefiltered clip. See pics below:

https://abload.de/thumb/2022-07-291tjjv3.png (https://abload.de/image.php?img=2022-07-291tjjv3.png)https://abload.de/thumb/2022-07-292b4kjm.png (https://abload.de/image.php?img=2022-07-292b4kjm.png)https://abload.de/thumb/2022-07-2931lknh.png (https://abload.de/image.php?img=2022-07-2931lknh.png)https://abload.de/thumb/2022-07-29452j8t.png (https://abload.de/image.php?img=2022-07-29452j8t.png)https://abload.de/thumb/2022-07-2958qk14.png (https://abload.de/image.php?img=2022-07-2958qk14.png)https://abload.de/thumb/2022-07-290uk5b.png (https://abload.de/image.php?img=2022-07-290uk5b.png)

EDIT: I turned off DGDenoise and the errors are gone. Tried with no prefilter (-1) and BM3DCUDA (6) with no errors.

Just a heads up regarding this issue. Sorry for the late reply.
It doesn't seem to be SMDegrain related. My old statement that DGDenoise() works fine is not valid. I get the very same errors when using DGDenoise() alone and denoising the chroma planes. With cstrength=0 the problem seems to be gone.

Will report this to DG. Maybe he can reproduce that error.

Dogway

14th August 2022, 13:02

@LeXXuz, thanks for reporting back. Just retested with ConvertBits(16).DGDenoise(cstrength=2) and I don't get the issue so as it looks it's driver/GPU related. I'm still on Win7 so I use the latest 400 branch driver version.

ChaosKing

14th August 2022, 14:18

I'm on the lastest driver win10x64 - 3070TI, no issues here.
Post your script.

LeXXuz

15th August 2022, 07:32

Problem is solved. And it was much simpler and stupider than expected. I downloaded a new mtmodes.avsi on that machine a few weeks back and some smart person thought it would be a good idea to set DGDenoise() as a NICE_FILTER. Well, you can guess the rest, I think. :rolleyes:

tormento

20th August 2022, 19:40

Could you please suggest me some good quality resizer to resample from 720p to 1080p (1.5x)?

Unfortunately Waifu2x accepts 1.5 only. Any workaround or better to use deepresize or nnedi3resize?

How to use them with nnedi3cl? What about the parameters? I always hope you find the time to write some documentation. :)

Dogway

20th August 2022, 20:00

ResizersPack was updated with NNEDI3CL by default, you can switch to nnedi3 cpu with gpuid=-1 (read dependencies)

Yes, deep_resize is the way to go for quality resizing (like nnedi3_resize16() was before), it mixes different kernels for edge/flat/chroma areas. By default it's set to optimal kernels (check examples) but you can play with other ones if you want FCBI, etc.
I haven't implemented waifu2 yet.

tormento

20th August 2022, 20:10

Yes, deep_resize is the way to go for quality resizing
Any hint for anime? As usual 1080p are 720p with bicubic. I inverted the process and now I want to rescale to proper resolution.
I haven't implemented waifu2 yet.
I have tried w2xncnnvk by Asd-g but, as I told, it supports only 2x.

DTL

20th August 2022, 20:30

With typical non-linear resamplers of 2x you make processibg in 2 steps - make 2x resize to closer upper size and downsize with second variable scale method to required output size.

tormento

20th August 2022, 20:39

With typical non-linear resamplers of 2x you make processibg in 2 steps - make 2x resize to closer upper size and downsize with second variable scale method to required output size.
Can you suggest me a good downsizer for anime (lineart)?

Dogway

20th August 2022, 20:46

I have tried w2xncnnvk by Asd-g but, as I told, it supports only 2x.

Then it's the same behavior than nnedi3, loop waifu2 until closest target dimensions then use alternative scaler to downscale or upscale. Didee bicubic coeffs are very good for small downscales.

tormento

21st August 2022, 00:00

@Dogway where can I find some explanation of the deep_resize parameters? Some are obscure to me, even reading the avsi.

Dogway

21st August 2022, 02:31

What do you need to know, it's pretty straight forward.

Read the comments for the settings. In any case everything is already tuned for best results by default depending on up/dw scaling and the ratio of it.

rth = Default(th_rat, 1.25) # threshold for mix mode. Lower than this (target_size/source_size) will use only 'kernel1' kernel
krn1 = Default(edge, rat<=rth*0.95 ? "Didee" : /* # Resize kernel for the edge area. AI upscalers recommended here: nnedi3, FSRCNN, FCBI, SuperResXBR... or for downscaling: DPID, SSIM2, Zopti */
\ rw<1 && rh<1 ? isUHD ? "Zopti1080" : "SSIM2" : /* # By default, for small upscales -> "blackman" (define *small* with 'th_rat' value) */
\ rat<=rth ? "Blackman" : "nnedi3") /* # By default, for big upscales -> "nnedi3"
# By default, for small downscales -> "Didée"
# By default, for big downscales -> "Zopti1080"
# By default, for mid downscales -> "SSIM2" (DPID also fine for down to 720p) (Beware!! for UHD to HD against all logic I recommend going with Zopti1080 (b=-0.99,c=0.06) ) */
krn1c = Default(chroma,rw<1 && rh<1 ? "Zopti" : "Blackman") # Chroma resizing kernel (no edge/flat discretion) *Remember you can directly call "kernel + taps". ie. Blackman6, Lanczos4
krn2 = Default(flat, rw<1 && rh<1 ? krn1 : isHD && isUHD2 ? "Sinc" : "Lanczos" ) # Resize kernel for the flat area (noop when downscaling). Ringing kernels are ok here (specially when up ratio is <1.7) like SincLin, Blackman, Lanczos, Sinc... ('Sinc' low taps -ie. 4- is worth testing, creates some faints halos specially for SD to HD so see whether it's faint enough for you)
etp = Default(e_taps, krn1 =="Blackman" ? 6 : krn1 =="Lanczos" ? 4 : 0) # taps for edge kernel (0 for undefined)
ftp = Default(f_taps, krn2 =="Blackman" ? 16 : krn2 =="Lanczos" ? 8 : 0) # taps for flat kernel (0 for undefined)
ctp = Default(c_taps, krn1c=="Blackman" ? 6 : krn1c=="Lanczos" ? 4 : 0) # taps for chroma kernel (0 for undefined)
thr = Default(th, 1.0) # 0-255 (normally up to 10)
ela = Default(elast, 4.0) # 1-255 (normally up to 10)
qual = Default(qual, 1)
sh = Default(show, false)
gpuid = Default(gpuid, 0) # Set to -1 for "cpu" nnedi3
dec = Default(deconv, 0) # 0~10 (0 is disabled, raise to apply deconvolution when upscaling a very sharpened downscale -anything sharper than Catmull-Rom-)
gr = Default(grain, rw>1 && rh>1 && rat>rth ? 0.6 : 0 ) # Grain strength. >0 Add grain to big upscales as a mean of cheap hallucination

Boulder

21st August 2022, 09:34

DGSource("test.dgi",ct=44,cb=44,cl=0,cr=0)
a=deep_resize(1920, 1040)
interleave(bicubicresize(1920, 1040, b=-0.5, c=0.25).propshow, a.propshow)

I was testing deep_resize with this simple script on a regular UHD (HDR) source, and I noticed that the colors of the deep_resize version are a bit warmer than in the source. Also the chroma location property seems to go missing.

Dogway

21st August 2022, 17:09

Yes, the chroma loc property was already fixed internally yesterday, and about the warmer tones I don't get them, but as I said I made some changes and fixes to TransformsPack, will release tomorrow probably.

tormento

21st August 2022, 19:02

about the warmer tones I don't get them
Dunno if related but have a look in the fmtc thread.

Seems planes and spaces sometimes don't mix well. :p

Dogway

21st August 2022, 23:15

fmtc hard codes 32-bit to full range so it's going to confuse you a little bit.
I have been refactoring a few things in ConvertFormat() to make it more robust, fixed quite a bit. With luck the next version after this one will finally be v1.0.
Also added waifu2xresize() a wrapper for w2xncnnvk. Quality is very good as expected, sharper than nnedi3 but very slow (~1fps on my card)

tormento

22nd August 2022, 09:50

fmtc hard codes 32-bit to full range so it's going to confuse you a little bit.
I "solved" the issue from 32 bits to lower, with some obscure Converbits(32) switches. Now it's the planar to YUV that's bugging me.

tormento

22nd August 2022, 14:01

Also added waifu2xresize()
Crawling here too.

In the mean time, people, which one (https://imgsli.com/MTIyMDQw) do you prefer?

The title of the image says it all. (i.e. downsample to 720p, waifu2x to 1440p and then downsample again to 1080p.

DTL

22nd August 2022, 15:12

" people, which one do you prefer?"

It is better to compare scaling in real movie running. Because if you going to have as sharp as possible image also using non-linear processing inbetween it it easy to fall into gibbs/aliasing issues so it may be not (clearly) visible at static frame but start to show moire at textures or flickering at small objects at movie frames running.

The 'sharpest/detailed possible' non-linear upscaler for static images may be not best for moving images. Moving-images non-linear upscaler need to reconstruct the object's shape and preform new moving_pictures_camera_AA processing for moving images to be safe from gibbs/aliasing. It is much less important for static images upscalers.

So some 'reference' or 'enough fail-safe' non-linear upscaling is sort of

MyLovelyNonLinearUpscaler(width*16, height*16) (for power-2 upscalers)
UserDefined2ResizeMT(output_width, output_height, b=__, c=__), where output_width is about input_width*2 (so ratio of NLUp/downscale is about 5 or more, 10 may be close to ideal limit) and b,c depend on 'image makeup type video/film' required.

The physical idea of processing:
1. MyLovelyNonLinearUpscaler(width*16, height*16) - create virtually infinite resolution image dataset as free of aliasing as possible and as detailed as possible (really the upscale factor depends on 'Luma range', so for HDR best value may be much more 10).
2. UserDefined2ResizeMT(target_width, target_height,makeup_params) - condition input infinite resolution image dataset to target resolution with adjustment of 'makeup/look' (acts as virtual moving pictures first transform camera) and protection from aliasing/gibbs. Also may convert low precision (8bit) to high presicion (8+bit) data to decrease banding or other defects. 1+2 is real 'digital image data mining process' - require lots of computing power :) .

The single frame intermediate datapath from 1 to 2 may be very RAM-non-friendly - so better solution is using sliced-processing of a small pieces or frame. Or some moving-picture data 'conditioner' need to be integrated in the NL_upscaler engine so user may download ready to use frame with target resolution from accelerator.

Dogway

22nd August 2022, 19:38

Crawling here too.

In the mean time, people, which one (https://imgsli.com/MTIyMDQw) do you prefer?

The title of the image says it all. (i.e. downsample to 720p, waifu2x to 1440p and then downsample again to 1080p.

08 looks best to me. What DTL said is right, maybe you can get away with waifu for anime (using model=0) but for live action there are better upscalers like basicvsrpp or PP-MSVSR.

I thought on adding a limiter to give more temporal coherence, I think I will add it as default when model!=0 is used. Something like Soothe() or so.

tormento

22nd August 2022, 19:44

you can get away with waifu for anime
I wish I had a brand new 4090 TI for free.

On my 1660 SUPER, I get 1 fps BEFORE encoding. :(

Dogway

22nd August 2022, 20:41

In any case I mostly only use waifu for vector art, logos, etc. For real anime there are much better models in ESRGAN. I don't know how fast that is in VS.

The good news is that waifu2x with model=1 already blows NGU out of the water.

EDIT: By the way, w2xncnnvk() has a bug which causes a green tint when matrixed back to YUV

tormento

22nd August 2022, 22:05

I don't know how fast that is in VS.
Is there for AVS+?

Reel.Deel

22nd August 2022, 22:16

Is there for AVS+?

Nope. Although the ncnn-vulkan version is not available in VapourSynth, at least there is some hope: https://github.com/xinntao/Real-ESRGAN-ncnn-vulkan

It uses the same framework as rife-ncnn-vulkan and waifu2x-ncnn-vulkan.

tormento

22nd August 2022, 22:58

Nope. Although the ncnn-vulkan version is not available in VapourSynth
So what program is Dogway talking about?

StvG

22nd August 2022, 23:09

EDIT: By the way, w2xncnnvk() has a bug which causes a green tint when matrixed back to YUV

Did you try with clamped w2xncnnvk output within [0,1]?

Dogway

23rd August 2022, 09:49

So what program is Dogway talking about?
vs-realesrgan, I think you already use it?

Did you try with clamped w2xncnnvk output within [0,1]?
I tried that and several settings. I think it has to do with discrepancies in RGB to YUV conversion as I get the same one (ConvertFormat) compared to fmtconv or avsresize, but mines does a roundtrip and I think it has to do with the coeff derivations. I don't know if waifu does a YUV conversion internally but it looks so (or modelled after a conversion), the problem for whatever that is is it doesn't roundtrip.

EDIT: Updated TransformsPack

tormento

23rd August 2022, 11:02

vs-realesrgan, I think you already use it
I tried but I don't find VS "friendly". I can't get proper control over threads and I keep on saturating my computer and graphic memory.

P.S: Do you have some fmtc laying around in your scripts? I think you saw the thread about tint changing. How are your conversion function speeds and can they be competitive to fmtc?

Boulder

23rd August 2022, 15:24

EDIT: Updated TransformsPack

Looks like the color issue I observed is fixed :)

Dogway

23rd August 2022, 19:59

Looks like the color issue I observed is fixed :)
Thanks for report!

I tried but I don't find VS "friendly". I can't get proper control over threads and I keep on saturating my computer and graphic memory.

P.S: Do you have some fmtc laying around in your scripts? I think you saw the thread about tint changing. How are your conversion function speeds and can they be competitive to fmtc?

Yeah, I find Python too verbose, like writting a letter, I want to type and read less while doing more. More expressive languages. Too bad I want to make Android apps.

What thread is that? I don't make use of fmtconv to minimize on dependencies, I just tested and performance seems on par for a YUV to RGB conversion. I think ConvertFormat() benefits when you add more things into the mix like upscaling, etc but cretindesalpes is making Convert() so maybe that can make things faster (and more user friendly).
TransformsPack is my personal notebook, it has a very different approach like color models, color matching functions, different tristimulus matrices, derived coefficients... It's not perfect but helps me to learn about it and code better (hopefully).

tormento

23rd August 2022, 22:27

What thread is that?
CPU and GPU threads ;)

DTL

25th August 2022, 07:34

One of practical usage of neural-networks 2x upscalers may be higher quality RAW debayer. Nowdays some enduser hardware can write RAW bayer stream (like some smartphones cameras setups) and require debayer before starting of 'standard' processing.
So for debayer the grey/bayer-RAW input may be separated into RGB low-sized planes, 2x upscaled and combined to full-size standard 4:4:4 RGB format.

Dogway

25th August 2022, 13:52

CPU and GPU threads ;)

You said "thread about tint changing".

One of practical usage of neural-networks 2x upscalers may be higher quality RAW debayer. Nowdays some enduser hardware can write RAW bayer stream (like some smartphones cameras setups) and require debayer before starting of 'standard' processing.
So for debayer the grey/bayer-RAW input may be separated into RGB low-sized planes, 2x upscaled and combined to full-size standard 4:4:4 RGB format.

I read about it last week on this paper (https://www.eecs.yorku.ca/~mbrown/ICCV19_Tutorial_MSBrown.pdf#page=226). In page 128 there's a simple "edge aware" algo for demosaicing. But it's also being considered for gray world (auto white balance), as it's one of the harder tasks for cameras.

I personally would like it used for color science: gamut mapping, CMFs, etc

tormento

25th August 2022, 14:00

You said "thread about tint changing".
That was the fmtc one.

tormento

25th August 2022, 18:40

Ok, I did some tests with an anime (Knights of Sidonia) with both deep_resize and w2xncnnvk.

The second, while giving better results, is really too slow (~1.5 fps on a 1660 SUPER) to be feasible. And I would have to encode it too.

So I had to play with deep_resize only.

The script I did is like

SetMemoryMax()
SetFilterMTMode("DEFAULT_MT_MODE", 3)
LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource("F:\In\Knights of Sidonia\Knights 01A.dgi")
ConvertBits(32)
DeBicubicResizeMT(target_width=1280,target_height=720,threads=1)
BM3D_CUDA(sigma=3, radius=2)
BM3D_VAggregate(radius=2)
deep_resize(1920,flat="nnedi3",grain=0,qual=2)
fmtc_bitdepth (bits=10,dmode=8, fulld=true)
neo_f3kdb(range=15, Y=65, Cb=40, Cr=40, grainY=0, grainC=0, sample_mode=2, blur_first=true, dynamic_grain=false, mt=false, keep_tv_range=true)
Prefetch(4)

but I am not getting impressive results on aliased lines.

Need some help. If you want to try, I put a small untouched clip here (https://krakenfiles.com/view/fJTGN2gqEw/file.html).

Dogway

26th August 2022, 10:39

For some reason I'm not liking w2xncnnvk here, on model=0, but also tested the other models. It creates jaggies, and some lineart thinning. It also accentuates ringing.

I find nnedi3 more pleasant, yes it's on the soft side so I played a bit with sharpening and result is rather good.

While at it I spotted two issues, one bug on Soothe() for 32-bit, and another one in LSFplus() for Lmode=2 at 32-bit (actually ex_edge() in 32-bit). Will upload the fixes (done already) today along a new argument (sharpness) for deep_resize() and nnedi3resize().

EDIT: oh I forgot to say, I don't get jaggies at all after nnedi3. But your debicubic is too strong, not sure if you measured the upscaling correctly. Are you aware of VapourSynth GetNative()? If there's no similar one in AVS+ I might give a chance at it, I won't port it, just create my own version following what I did for OOTF() last week.

tormento

26th August 2022, 10:56

deep_resize() and nnedi3resize()
What is the difference between the two?[/QUOTE]
EDIT: oh I forgot to say, I don't get jaggies at all after nnedi3
Please post your script.
not sure if you measured the upscaling correctly
I did use GetNative:

Bicubic b 0.33 c 0.33 AR: 1.78 Steps: 1
Native resolution(s) (best guess): 720p

Isn't my debicubic correct? What should I use?

You can try on the same mkv I posted in my previous message.

Dogway

26th August 2022, 11:33

nnedi3resize() is a nnedi3 all around upscaler, as simple as that.
deep_resize() is a power user oriented high quality scaler (up and down), with appropriate defaults, and post-processing options (grain, sharpening, deconvolution)

My script is the same as yours. I haven't tested GetNative because I don't use VS, was just wondering, but I will create an AVS+ version. I don't promise ultra performance (GetNative claims 13s).

EDIT: I updated ResizersPack.

The correct use in this case with edge=flat="nnedi3" is to use nnedi3resize, because the sharpening is performed in (usually) super sampling mode.

So something like:
nnedi3resize(1920,qual=2,sharpness=30)

tormento

26th August 2022, 13:24

My script is the same as yours.
But you wrote
I don't get jaggies at all after nnedi3. But your debicubic is too strong
So... if the same script how can I have different results? ;)
So something like
Thanks!

P.S: I can't really find any difference between quality 1 and 2. What are those values supposed to do?

Boulder

26th August 2022, 14:23

Dogway

27th August 2022, 14:03

Dogway, I strongly suggest considering some other kernel than SSIM2 for downscaling, at least in 1080p -> 720p it causes considerable changes in the image colors near edges. Probably due to local contrast being changed? I've been using Zopti720 which sharpens the image slightly but doesn't have other issues, or at least they are not so apparent.

I did a comparison and while Zopti720 is sharper it also creates some aliasing compared to SSIM2.
Using ex_makediff(metrics="All") SSIM2 showed better scores for all metrics compared to Zopti720.
Analyzing the diff map you can pin point such aliasing issues.
A good alternative I think would be to run SSIM2 for edges and Zopti720 for flat.

deep_resize(1280, edge="SSIM2", flat="Zopti720")

I didn't spot any change in color near borders, can you show an example?

Boulder

27th August 2022, 14:53

I did a comparison and while Zopti720 is sharper it also creates some aliasing compared to SSIM2.
Using ex_makediff(metrics="All") SSIM2 showed better scores for all metrics compared to Zopti720.
Analyzing the diff map you can pin point such aliasing issues.
A good alternative I think would be to run SSIM2 for edges and Zopti720 for flat.

deep_resize(1280, edge="SSIM2", flat="Zopti720")

I didn't spot any change in color near borders, can you show an example?
This is a good sample clip for the issue, see the computer panel in the background.

https://drive.google.com/file/d/1U2LcJlQ1bN19uDYnBsfI4YoPIYJS9X--/view?usp=sharing

I tested using those different kernels and it seems to work fine with this clip so it probably is safe to adjust things like that. Is it possible to see visually which parts are marked as edge and flat?

Dogway

27th August 2022, 15:20

This is the diff map with no augmentation (augment didn't show anything relevant either).
https://i.imgur.com/I1eRPSJl.png (https://i.imgur.com/I1eRPSJ.png)

It actually isn't an edge mask as it uses the nnedi3_resize16() original method of limitdif "masking". This is faster as it processes the whole thing in one call.
I could have implemented a one call mask+merge limiter as well but wanted to keep the rationale as in nnedi3_resize16() and the elast and thr args.
The internal mask actually is "filtered - original", typically you will get more differences around edges so it will naturally create an edge mask tailored with the elast and thr values.
I can try to implement a show mask arg to ex_limitdif().

tormento

28th August 2022, 11:33

I own both the BD and the UHD version of Akira movie.

I am using it to make some tests with LUTs and tone mapping, as the BD version is too saturated and I'd like to recover a nicer SDR version from the UHD.

I have some problems with the luma level after PQ to SDR conversion, i.e. luminosity is a bit low, even in its higher value.

This is the original SDR:

https://i.lensdump.com/i/194vVF.md.png (https://lensdump.com/i/194vVF)

This is the converted SDR (Akira PQ to SDR 26nit grey - gamma 2.40):

https://i1.lensdump.com/i/194MG0.md.png (https://lensdump.com/i/194MG0)

I'd like to "expand" the luminosity thru the usueful range (limited, of course) seeing that there is a nice clipped part to use as reference.

Expand because I'd like to leave the black where it is and fill just the upper part.

Should I touch a bit some color level too?