Vapoursynth

[Txico]

Hello there,
As with any new Ubuntu release, I compile VapouSynth and plug-in from scratch. That's a new compiler versions used so it might help (or not).
With a newly installed Ubuntu 22.10 and after compiling VapourSynth and plug-ins I get an error every time I try a vpy script:
Script evaluation failed:
Python exception: attempted relative import with no known parent package

Traceback (most recent call last):
File "src/cython/vapoursynth.pyx", line 2819, in vapoursynth._vpy_evaluate
File "src/cython/vapoursynth.pyx", line 2820, in vapoursynth._vpy_evaluate
File "example.mp4.vpy", line 2, in <module>
import havsfunc as haf
File "/usr/local/lib/python3.10/site-packages/havsfunc.py", line 69, in <module>
import mvsfunc as mvf
File "/usr/local/lib/python3.10/site-packages/mvsfunc.py", line 59, in <module>
from ._metadata import __version__
ImportError: attempted relative import with no known parent package

I know, I know. It suck to be me but, can anyone at least point me in the correct direction?

[Selur]

Have you installed mvsfunc like mentioned over at https://github.com/HomeOfVapourSynthEvolution/mvsfunc or did you just copy the mvsfunc.py file?
If you did the latter, it would explain the issue. (since _metadata.py is missing which includes the __version__-variable)

Cu Selur

[Txico]

Have you installed mvsfunc like mentioned over at https://github.com/HomeOfVapourSynthEvolution/mvsfunc or did you just copy the mvsfunc.py file?
If you did the latter, it would explain the issue. (since _metadata.py is missing which includes the __version__-variable)

Cu Selur

I have uninstalled mvsfunc with pip3 uninstall mvsfunc and after that pip3 install git+https://github.com/HomeOfVapourSynthEvolution/mvsfunc as specified in the Git Hub page.
Perhaps there some reboot needed in between? Any cache that needs to be redone? Any other package that needs to be installed like this that I need to know?

Thank you for your quick answer Selur.
Txico

[Txico]

Hello again,

After you pointed me to the mvsfunc issue I have uninstalled it again and then searched for any other mvsfunc file in my filesystem, and guess what... I found one where there should be none.
After deleting it and looking for it again without results this time, installing mvsfunc using pip again gets everything working. Flabbergasted.

Thank you, Cu Selur. That solved the issue.

Txico

[Selur]

Can someone tell me how to translate:
clip.Mt_Convolution(Horizontal=" 1 1 1 ", vertical = " 1 ", u=1, v=1)
my guess atm. is 'clip.std.Convolution(matrix=[ 1 1 1 ], planes= [ 0 ])'
and
MT_Luts(Diff, Diff, mode="med", pixels = " 0 0 1 0 -1 0 " , expr = " X Y - X Y - X Y - abs 1 + * X Y - abs 1 + "+THR+" 1 >= "+THR+" 0.5 ^ "+THR+" ? + / - 128 +", u=1,v=1) (no clue)
and
Mt_AddDiff(Blurred, ReconstructedMedian)
to Vapoursynth? (Diff, Blurred, ReconstructedMedian are vs.Videonode and THR is a string-representation of an int)

Thanks!

Cu Selur

[Selur]

Okay,
Avisynth: Mt_AddDiff(Blurred, ReconstructedMedian)
<>
Vapoursynth: core.std.MergeDiff(Blurred, ReconstructedMedian))

Avisynth:
MT_Luts(Diff, Diff, mode="med", pixels = " 0 0 1 0 -1 0 " , expr = " X Y - X Y - X Y - abs 1 + * X Y - abs 1 + "+THR+" 1 >= "+THR+" 0.5 ^ "+THR+" ? + / - 128 +", u=1,v=1)
<>
Vapoursynth:
partial_expr = lambda M, N: f" x x[{M},{N}] - x x[{M},{N}] - x x[{M},{N}] - abs 1 + * x x[{M},{N}] - abs 1 + {THR} 1 >= {thr_s} 0.5 pow {THR} ? + / - 128 + "
medianDiff = core.akarin.Expr(diff, [partial_expr(0,0) + partial_expr(1,0) + partial_expr(-1,0) + "sort3 drop swap drop", ""])



=> But how to convert,
Avisynth
Mt_Convolution(Horizontal=" 1 1 0 0 1 0 0 1 1 ", vertical = " 1 ", u=1, v=1)
to Vapoursynth?

Cu Selur

[Selur]

got it,
clip.std.Convolution(matrix=[ 1, 1, 0, 0, 1, 0, 0, 1, 1 ], mode='v', planes=[0])
works. (the mode='v' was the main issue)
Uploaded it to: https://github.com/Selur/VapoursynthScriptsInHybrid/blob/master/fromDoom9.py

Cu Selur

[Myrsloik]

R61-RC1 (https://github.com/vapoursynth/vapoursynth/releases/tag/R61-RC1) is out. Bug fixes only.

[Selur]

Thanks :)

[Myrsloik]

Thanks :)

R61 released now. No changes from RC1.

[Selur]

Thanks again. :)

[lansing]

I'm testing out the VSGAN filter in vsedit2, how do I free the gpu ram usage after running it? vsapi->freeFrame() doesn't free it.

[Selur]

I get totally different decimated output when using:
# Imports
import vapoursynth as vs
# getting Vapoursynth core
core = vs.core
# Loading Plugins
core.std.LoadPlugin(path="i:/Hybrid/64bit/vsfilters/DeinterlaceFilter/TIVTC/libtivtc.dll")
core.std.LoadPlugin(path="i:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
# source: 'C:\Users\Selur\Desktop\sample_from_DVD.mkv'
# current color space: YUV420P8, bit depth: 8, resolution: 720x480, fps: 29.97, color matrix: 470bg, yuv luminance scale: limited, scanorder: telecine
# Loading C:\Users\Selur\Desktop\sample_from_DVD.mkv using LWLibavSource
clip = core.lsmas.LWLibavSource(source="C:/Users/Selur/Desktop/sample_from_DVD.mkv", format="YUV420P8", stream_index=0, cache=0, prefer_hw=0)
# Setting color matrix to 470bg.
clip = core.std.SetFrameProps(clip, _Matrix=5)
clip = clip if not core.text.FrameProps(clip,'_Transfer') else core.std.SetFrameProps(clip, _Transfer=5)
clip = clip if not core.text.FrameProps(clip,'_Primaries') else core.std.SetFrameProps(clip, _Primaries=5)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# making sure frame rate is set to 29.97
clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=0)
# cropping the video to 704x480
clip = core.std.CropRel(clip=clip, left=6, right=10, top=0, bottom=0)
# Deinterlacing using TIVTC
clip = core.tivtc.TFM(clip=clip, mode=4)
clip = core.tivtc.TDecimate(clip=clip)# new fps: 23.976
# make sure content is preceived as frame based
clip = core.std.SetFieldBased(clip, 0)
# adjusting output color from: YUV420P8 to YUV420P10 for x265Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, range_s="limited")
# set output frame rate to 23.976fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001)
# Output
clip.set_output()
compared to:
# Imports
import vapoursynth as vs
# getting Vapoursynth core
core = vs.core
# Loading Plugins
core.std.LoadPlugin(path="i:/Hybrid/64bit/vsfilters/DeinterlaceFilter/TIVTC/libtivtc.dll")
core.std.LoadPlugin(path="i:/Hybrid/64bit/vsfilters/SourceFilter/DGDecNV/DGDecodeNV.dll")
# source: 'C:\Users\Selur\Desktop\sample_from_DVD.mkv'
# current color space: YUV420P8, bit depth: 8, resolution: 720x480, fps: 29.97, color matrix: 470bg, yuv luminance scale: limited, scanorder: telecine
# Loading C:\Users\Selur\Desktop\sample_from_DVD.mkv using DGSource
clip = core.dgdecodenv.DGSource("G:/Temp/mkv_6971e61a600461af6ca6ccf15732fb4f_853323747.dgi",fieldop=2)
# Setting color matrix to 470bg.
clip = core.std.SetFrameProps(clip, _Matrix=5)
clip = clip if not core.text.FrameProps(clip,'_Transfer') else core.std.SetFrameProps(clip, _Transfer=5)
clip = clip if not core.text.FrameProps(clip,'_Primaries') else core.std.SetFrameProps(clip, _Primaries=5)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# making sure frame rate is set to 29.97
clip = core.std.AssumeFPS(clip=clip, fpsnum=30000, fpsden=1001)
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=0)
# cropping the video to 704x480
clip = core.std.CropRel(clip=clip, left=6, right=10, top=0, bottom=0)
# Deinterlacing using TIVTC
clip = core.tivtc.TFM(clip=clip, mode=4)
clip = core.tivtc.TDecimate(clip=clip)# new fps: 23.976
# make sure content is preceived as frame based
clip = core.std.SetFieldBased(clip, 0)
# adjusting output color from: YUV420P8 to YUV420P10 for x265Model
clip = core.resize.Bicubic(clip=clip, format=vs.YUV420P10, range_s="limited")
# set output frame rate to 23.976fps
clip = core.std.AssumeFPS(clip=clip, fpsnum=24000, fpsden=1001)
# Output
clip.set_output()
And the only difference between it is the source filter.
With dgdecodenv.DGSource I get wrong decimation hits. :(
In Avisynth I would use "clip = clip.PreRoll(Int(clip.FrameRate())" to handle this, but what to do in Vapoursynth?
I encountered this with the source from https://forum.videohelp.com/threads/407942-29-97-to-23-976-interlace-dilemma

Okay, scratch that, using PreRoll in Avisynth doesn't help either, only switching away from DGDecNV.
The question whether there is an alternative to PreRoll still remains.

Cu Selur

[poisondeathray]

And the only difference between it is the source filter.
With dgdecodenv.DGSource I get wrong decimation hits. :(


I get similar results between avs and vpy versions of simple TIVTC script using DGSource (frames match up, but not bit identical, I think libtivtc might be taking a slightly different field on some matches, but still resulting in the same frame visually)

[Selur]

Is there another SourceFilter than DGDecNV which has 'soft telecine' (ForceFilm) handling?

[poisondeathray]

Is there another SourceFilter than DGDecNV which has 'soft telecine' (ForceFilm) handling?

d2vsource ? RFF = False ? or DGIndex FF ?


Unlike DGDecode, it's up to the user to apply RFF flags as they see fit, by passing rff=True to the source function, or by passing rff=False and using core.d2v.ApplyRFF(clip, d2v=r'C:\path\to\my.d2v') after calling the source function. Unless you know your source is 100% FILM, you probably want to apply these. DGDecode's traditional "Force FILM" mode isn't really present in this plugin, but if your source or part of it is 100% FILM, which is the only time you should be Force FILMing anyway, you can simply set Force FILM in DGIndex, which will set the framerate properly in the D2V file, and then not apply RFF flags. It's also feasible to trim away any non-FILM frames and still Force FILM.


Maybe ffms2 ... but rffmode might not be present in the vapoursynth version of ffms2 (or maybe it's always mode=0 ?)

[Selur]

Issue was resolved over at: https://www.rationalqm.us/board/viewtopic.php?f=8&t=1220
It was my mistake, I misunderstood how fieldop in DGSource source worked.
Instead of fieldop=2, I should have used fieldop=0. :/
(so not a bug in DGDecNV, but in my mind :))

Cu Selur

[_Al_]

About _ColorRange prop,
I have to specify it this way: RANGE = {0:'limited', 1:'full'}, I checked Zimg and it has it the same way.
But manual has it backwards, http://www.vapoursynth.com/doc/apireference.html#reserved-frame-properties
Could it be fixed in that documentation?

[Julek]

About _ColorRange prop,
I have to specify it this way: RANGE = {0:'limited', 1:'full'}, I checked Zimg and it has it the same way.
But manual has it backwards, http://www.vapoursynth.com/doc/apireference.html#reserved-frame-properties
Could it be fixed in that documentation?


Frame prop and zimg arg are different things.
1 is limited in _ColorRange prop and full in zimg arg.

[_Al_]

I have to define range and matrix using strings, otherwise things are off:

print(vs.__api_version__)
clip = core.lsmas.LibavSMASHSource('YUV.mp4')
rgb1 = clip.resize.Bicubic(format=vs.RGB24, matrix_in_s='709', range_in_s='full')
rgb2 = clip.resize.Bicubic(format=vs.RGB24, matrix_in_s='709', range_in=0)
>>>VapourSynthAPIVersion(api_major=3, api_minor=6)
and rgb1 and rgb2 are not the same, is it an old bug? I do not have portable tkinter in portable API4 right now to quick check

Oh, I get it now, it is swapped, so if wanting to set 'limited' in props, we have to use:

API3:
clip = clip.std.SetFrameProp(prop='_ColorRange', intval=1)
API4:
clip = clip.std.SetFrameProps(_ColorRange=1)
but if wanting to specify limited range for YUV before conversion, we have to:
rgb = clip.resize.Bicubic(format=vs.RGB24, matrix_in_s='709', range_in=0)
Wow, I wonder how many headaches this caused, I certainly was confused for a long while ... :-)

[Selur]

Trying to port CQTGMC (https://forum.doom9.org/showthread.php?t=183823) to Vapoursynth, I started with:
def CQTGMC(clip: vs.VideoNode, Sharpness: float=0.25, thSAD1: int=192, thSAD2: int=320, thSAD3: int=128, thSAD4: int=320) -> vs.VideoNode:

flip = core.std.FlipHorizontal(clip)
flip = Padding(clip=clip,top=clip.width%64)
flip = Padding(clip=clip,right=clip.height%64)
padded = core.std.StackHorizontal([clip,flip])
# todo: optional use nnedi3CL
bobbed = core.znedi3.nnedi3(clip=padded, field=2)
denoised = core.rgvs.RemoveGrain(clip=bobbed,mode=12)
denoised = core.fmtc.resample(clip=denoised, kernel="gauss", w=denoised.width, h=denoised.height, scaleh=denoised.width+0.0001, interlaced=False, interlacedd=False)
denoised = core.resize.Bicubic(clip=denoised, format=bobbed.format, dither_type="error_diffusion") # adjust format after fmtc
denoised = core.std.Merge(clipa=denoised, clipb=bobbed,weight=0.25)

srchClip = denoised
csuper = core.mv.Super(clip=denoised);
bvec = core.mv.Analyse(csuper,isb=True,blksize=64,overlap=32)
fvec = core.mv.Analyse(csuper,isb=False,blksize=64,overlap=32)
Comp1 = core.mv.Compensate(denoised,csuper,bvec,thsad=thSAD2)
Comp2 = core.mv.Compensate(denoised,csuper,fvec,thsad=thSAD2)
denoised = core.std.Interleave([Comp1,denoised,Comp2])
csuper = core.vs.Super(clip=denoised)
bvec = core.mv.Analyse(csuper,isb=True,blksize=64,overlap=32)
fvec = core.Analyse(csuper,isb=False,blksize=64,overlap=32)
Inter = core.mv.FlowInter(csuper,bvec,fvec,blend=False)
# global CQTGMC_A = Inter.SelectEvery(3,0)
# global CQTGMC_B = Inter.SelectEvery(3,1)
# srchClip
# ScriptClip("""
# P = YDifferenceFromPrevious()
# N = Trim(1,0).YDifferenceFromPrevious()
# N < P ? CQTGMC_A : CQTGMC_B
# """)
# super = MSuper()
# bVec1 = MAnalyse(super,isb=true,overlap=4,delta=1)
# fVec1 = MAnalyse(super,isb=false,overlap=4,delta=1)
# bobbed
# super = MSuper(levels=1)
# bComp1 = MCompensate(super,bVec1,thSAD=thSAD3)
# fComp1 = MCompensate(super,fVec1,thSAD=thSAD3)
# Interleave(\
# SeparateFields(bobbed).SelectEvery(4,0),\
# SeparateFields(fComp1).SelectEvery(4,1),\
# SeparateFields(bComp1).SelectEvery(4,2),\
# SeparateFields(bobbed).SelectEvery(4,3))
# Weave()
# # super = MSuper(levels=1)
# bComp1 = MCompensate(super, bVec1,thSAD=thSAD4)
# fComp1 = MCompensate(super, fVec1,thSAD=thSAD4)
# tMax = mt_logic(fComp1,"max",U=3,V=3).mt_logic(bComp1,"max",U=3,V=3)
# tMin = mt_logic(fComp1,"min",U=3,V=3).mt_logic(bComp1,"min",U=3,V=3)
# MDegrain1(super,bVec1,fVec1,thSAD=thSAD1)
# sharpen = mt_adddiff(mt_makediff(Removegrain(20),u=3,v=3),u=3,v=3)
# mt_clamp(sharpen,tMax,tMin,Sharpness,Sharpness,3,3,3)
# super = MSuper(levels=1)
# MDegrain1(super,bVec1,fVec1,thSAD=thSAD1)
# Crop(0,0,-(input.width()%64),-(input.height()%64))
# return last

# from havsfunc
def Padding(clip: vs.VideoNode, left: int = 0, right: int = 0, top: int = 0, bottom: int = 0) -> vs.VideoNode:
if not isinstance(clip, vs.VideoNode):
raise vs.Error('Padding: this is not a clip')

if left < 0 or right < 0 or top < 0 or bottom < 0:
raise vs.Error('Padding: border size to pad must not be negative')

width = clip.width + left + right
height = clip.height + top + bottom

return clip.resize.Point(width, height, src_left=-left, src_top=-top, src_width=width, src_height=height)

but now I'm stuck.
Can someone tell me how to convert this part:
global CQTGMC_A = Inter.SelectEvery(3,0)
global CQTGMC_B = Inter.SelectEvery(3,1)
srchClip
ScriptClip("""
P = YDifferenceFromPrevious()
N = Trim(1,0).YDifferenceFromPrevious()
N < P ? CQTGMC_A : CQTGMC_B
""")
to Vapoursynth?

Thanks!

Cu
Selur

[kedautinh12]

SelectEvery(3,0) -> std.SelectEvery(clip=clip, cycle=2, offsets=0)
SelectEvery(3,1) -> SelectEvery(clip=clip, cycle=2, offsets=1)
YDifferenceFromPrevious() -> core.std.PlaneStats(clip, clip[0] + clip)

[Selur]

Okay, but why:
SelectEvery(3,0) -> std.SelectEvery(clip=clip, cycle=2, offsets=0)
shouldn't cycle stay 3 ?

[kedautinh12]

Okay, but why:
SelectEvery(3,0) -> std.SelectEvery(clip=clip, cycle=2, offsets=0)
shouldn't cycle stay 3 ?

Lol, i'm forgot change that, sr

[Selur]

Okay. Thanks :)
Opened a separate thread for the porting. https://forum.doom9.org/showthread.php?p=1981140

Cu
Selur

[~ VEGETA ~]

I've been trying to use fvf insertsign to overlay a lagarith .avi with alpha on top of my video. however, it does not work.

how can i specify it right so that it takes the alpha channel properly?

I used the same files with avs and it worked... i think i am missing something related to telling vs about the alpha channel?

the logo appears but without being overlayed properly

[_Al_]

havsfunc could be used:
import havsfunc
clip=havsfunc.Overlay(clip1,clip2,mask=alpha)
https://github.com/HomeOfVapourSynthEvolution/havsfunc/blob/master/havsfunc.py#L5960

[Julek]

I've been trying to use fvf insertsign to overlay a lagarith .avi with alpha on top of my video. however, it does not work.

how can i specify it right so that it takes the alpha channel properly?

I used the same files with avs and it worked... i think i am missing something related to telling vs about the alpha channel?

the logo appears but without being overlayed properly

https://github.com/Irrational-Encoding-Wizardry/fvsfunc/pull/12

[~ VEGETA ~]

havsfunc could be used:
import havsfunc
clip=havsfunc.Overlay(clip1,clip2,mask=alpha)
https://github.com/HomeOfVapourSynthEvolution/havsfunc/blob/master/havsfunc.py#L5960

overlay succeeded but the result was bad, the alpha overlay is not good. it didn't get completely transparent as it should be.

also, it does not have options to specify ranges.

also:

video1= lvf.misc.overlay_sign(clip=video1,overlay=fixed_logo,frame_ranges=[8508,8655])

didn't work and put this: https://pastebin.com/GSZRYLqi

first it didn't accept 2 colorspaces as alpha clip in RGBA, so I did:

video1= lsmash...
fixed_logo = core.ffms2.Source(source=path..., alpha=True)
fixed_logo = core.resize.Bilinear(fixed_logo, format=vs.YUV420P8, matrix_s="709")
video1= lvf.misc.overlay_sign(clip=video1,overlay=fixed_logo,frame_ranges=[8508,8655])

but it still showed the error above.

On insertsign side, I still cannot use it even after modifying it with the mentioned commit manually in scripts folder.

[_Al_]

overlay succeeded but the result was bad, the alpha overlay is not good. it didn't get completely transparent as it should be.Make mask a grayscale clip with only values 0 or 255 (for 8bits). You can use Levels or Expr to make it from your logo.

[~ VEGETA ~]

Make mask a grayscale clip with only values 0 or 255 (for 8bits). You can use Levels or Expr to make it from your logo.

like converting to yuv then shuffleplanes first plane?

levels needs inputs and outputs 0-255, how to do your idea?

[_Al_]

I'd check alpha first, it needs to be 255 to have full transparency to see logo over clip, and zero values to see clip without any transition. In a previewer using color pickers. The borders would be some grayscale for smooth transitions. And it looks like mask has to be specifically used as a keyword, havsfunc.Overlay(clip, logo, mask=alpha)

oh those levels, ..., if alpha values do not reach 255, then using something like, for grayscale clip (plane does not have to be specified):
core.std.Levels(max_in=230, max_out=255) to get it to 255, that number 235 you test in previewer, could be a bit higher or lower

[~ VEGETA ~]

I'd check alpha first, it needs to be 255 to have full transparency to see logo over clip, and zero values to see clip without any transition. In a previewer using color pickers. The borders would be some grayscale for smooth transitions. And it looks like mask has to be specifically used as a keyword, havsfunc.Overlay(clip, logo, mask=alpha)


thanks but i managed to make insertsign work by just using the string of file location as input rather than ffms2.

[~ VEGETA ~]

I'd like to ask about better ways to sharpen anime rather than using regular sharpeners (like LSFmod). this is regardless of descale or not to descale.

I found AI or NN upscalers with nice results in terms of denoise, deblock, and sharpening but are they used on actual fansub encodes (modern anime)? I am only interested in making the encodes slightly more sharp (no halos) but not too sharp.

Doing one of these upscalers then SSIM downsample to 1080p... then doing maskedmerge using an edge mask to only get sharp edges from the upscaled clip. is this a good approach? I have Ryzen 7900X CPU, 3060TI GPU, 32G DDR5 5600MHz PC so I think it can handle high work load.

what do you think?

[Selur]

You might also want to look into vsgan models from https://upscale.wiki/wiki/Model_Database, most of them are trained on animes.

[~ VEGETA ~]

You might also want to look into vsgan models from https://upscale.wiki/wiki/Model_Database, most of them are trained on animes.

i will dig into this soon, to learn the syntax of using them in VS.

however, what do you think about the approach I explained above?

also, besides vsgan, what other similar tools which are used in encoding anime real releases not just for testing.

I feel like such tools can do some damage to backgrounds or some unwanted features like denoise and deblock. thus i pointed out the masks.

[Selur]

Yes, your approach might work too.
Anime release groups usually use tons of masking and rarely use a filter as is. :) (+ they often filter per scene)
You might also want to check out the 'Enhance Everything!' discord channel (https://discord.com/invite/cpAUpDK) and the 'Irrational Encoding Wizardry' (https://discord.gg/qxTxVJGtst) channel.

i will dig into this soon, to learn the syntax of using them in VS.
Here's a simple example:
# resizing using VSGAN
from vsgan import ESRGAN
vsgan = ESRGAN(clip=clip,device="cuda")
model = "I:/Hybrid/64bit/vsgan_models/1x_BroadcastToStudioLite_485k.pth"
vsgan.load(model)
vsgan.apply()
clip = vsgan.clip
the VSGAN (https://github.com/rlaphoenix/VSGAN)-site also has some good documentation.

Cu Selur

[~ VEGETA ~]

Yes, your approach might work too.
Anime release groups usually use tons of masking and rarely use a filter as is. :) (+ they often filter per scene)
You might also want to check out the 'Enhance Everything!' discord channel (https://discord.com/invite/cpAUpDK) and the 'Irrational Encoding Wizardry' (https://discord.gg/qxTxVJGtst) channel.


Here's a simple example:
# resizing using VSGAN
from vsgan import ESRGAN
vsgan = ESRGAN(clip=clip,device="cuda")
model = "I:/Hybrid/64bit/vsgan_models/1x_BroadcastToStudioLite_485k.pth"
vsgan.load(model)
vsgan.apply()
clip = vsgan.clip
the VSGAN (https://github.com/rlaphoenix/VSGAN)-site also has some good documentation.

Cu Selur

thanks for this my friend.

what other tools exist for this task rather than vsgan\esrgan? how can we compare their results and see which is best for anime?

I know these stuff do many enhancements but i am only interested in getting lines sharper, the line art itself not background or texture.

so I don't want to do these silly "4k upscaled anime" releases or even do very sharp everything... i think you got what i mean.

I know fansub releases do many masking, me included. However, i didn't see a vs script for a release which has stuff like vsgan in it which made me wonder why it is not used.

[Selur]

afaik. most release groups use normal filters or script collections like lvsfunc&co and no magic tools.
Why it's not used is probably easy:
a. requires up-to-date hardware, especially if you want to encode tons of content. Encoding with on multiple machines to speed up the processing is hard if each of them require a 3000+ NVIDIA GPU to not totally suck and are not really fast even then.
b. assuming you don't need to do restoration, you can archive the stuff most ai filtering does through other filters when you spend enough effort.
c. you don't have much control aside from different types of masking to control what the ai stuff does (assuming you didn't train the models yourself)
d. folks filtering anime often want to keep artifacts which they perceive as details of the source. (there is also the discussion of which release of anime xy has the colors 'right', often it has to be the one that came out earlier,...)
=> to get to the bottom of things, you probably will need to go to the discord channels of the groups and ask them and some might actually reply honestly and not simply say 'ai bad => you bad' *gig*

Cu Selur

[~ VEGETA ~]

I get this error: https://pastebin.com/Kj8QBQDi

I tried doing many tools but could not get the release to be as sharp as another good one. talked to them but didn't give many details how they achieved it. thus I thought of this method.

I have Ryzen 7900X and 3060Ti so I guess my PC qualifies.

I get it that AI upscale is bad for anime releases but i don't plan to use it that way.. I only want the good sharp lines. ALL other stuff are exactly as they were.

[Selur]

btw. can someone port EZDenoise to Vapoursynth?
Original:
function EZdenoise(clip Input, int "thSAD", int "thSADC", int "TR", int "BLKSize", int "Overlap", int "Pel", bool "Chroma", bool "out16")
{
thSAD = default(thSAD, 150)
thSADC = default(thSADC, thSAD)
TR = default(TR, 3)
BLKSize = default(BLKSize, 8)
Overlap = default(Overlap, 4)
Pel = default(Pel, 1)
Chroma = default(Chroma, false)
out16 = default(out16, false)

Super = Input.MSuper(Pel=Pel, Chroma=Chroma)
Multi_Vector = Super.MAnalyse(Multi=true, Delta=TR, BLKSize=BLKSize, Overlap=Overlap, Chroma=Chroma)

Input.MDegrainN(Super, Multi_Vector, TR, thSAD=thSAD, thSAD2=int(float(thSAD*0.9)), thSADC=thSADC, thSADC2=int(float(thSADC*0.9)), out16=out16)
}


Cu Selur

[Selur]

@~ VEGETA ~: No clue. First time I see that error. :/
I can send you a link to my current 'torch-AddOn' for Hybrid which basically is a folder with a portable Vapoutsynth, which also includes vsgan (and tons of other stuff).

Cu Selur

[~ VEGETA ~]

Hello

I am trying to use this https://github.com/YomikoR/GetFnative with base dimensions of 1920x1080p and fractional height of 847.047 (or so, don't remember now).

However, I'd like to use SSIM downsampler instead of regular spline36 if possible. I tried so but could not because it always produces a shifted image.

I tried different manual values but couldn't do it properly as the original code example shown in linked page.

any tips?

[DTL]

btw. can someone port EZDenoise to Vapoursynth?
Cu Selur

The magic is inside mvtools - not in this simple mvtools usage function. First is good to port to vapoursynth all new features of todays mvtools (post 2.7.45 builds to the end of 2022 or with some new planned features to 2023 like finally fixing quality issue of https://github.com/pinterf/mvtools/issues/59 for non-4:4:4 sources).

At first VS may still not support simple MDegrainN from old 2.7.45 era. As in latest commit in 2023 https://github.com/dubhater/vapoursynth-mvtools/commit/b5d58cb7ca1cfe27bdcb30fbcff67254580b7ab9 it is only start to support fixed-tr to 6. It is lightyears behind latest end of 2022 MDegrainN already having 'spatial' multi-pass blending modes and going to go into 'temporal' multi-pass blending as a next step.

Also the most useful features like several interpolated overlap modes (at the MDegrain stage - running MAnalyse in max speed non-overlapped or using hardware accelerator of MVs from MPEG encoder chip) including new quality/performance balanced mode of 'diagonal' overlap having only 2x number of blocks and giving close to blksize/2 4x overlap mode quality in old mvtools overlap design. Also runtime sub-sample shifting allowing to save host RAM traffic and expensive onboard CPU caches trashing with pre-calculated subsample refined planes. The required for correct pel=4 UV processing in 4:2:0 granularity is pel/8. So if designed in old way via MSuper it will bug memory subsystem even more.

Also running MVs analysis in 'large' tr of 10 and more opens better possibility to perform intermediate linear or non-linear MVs grading in time axis after MAnalyse and before MDegrain (MVLPF current implemented feature of MDegrainN). Using too few tr for 1..6 gives too low timed samples of MVs to make good FIR convolution linear LPF.

[Selur]

Okay, so in conclusion with https://github.com/dubhater/vapoursynth-mvtools EZDenoise can't be ported atm. .
Sad news, but I will have to live with it.
Thanks.

[mastrboy]

Myrsloik, can we get a weight parameter for std.MaskedMerge like there is for std.Merge?

[Myrsloik]

Myrsloik, can we get a weight parameter for std.MaskedMerge like there is for std.Merge?

What? Tbe weight is already in the mask

[Selur]

He, probably, wants a weight since he isn't using a grayscale, but a binary mask,...

[mastrboy]

What? Tbe weight is already in the mask

Might just be some lack of knowledge on my understanding of how masks works...

Consider the following mask/merge:
edge_mask = core.tedgemask.TEdgeMask(video, link=1, threshold=5.0).std.Maximum(threshold=128)
edge_merged = core.std.MaskedMerge(clipa=video, clipb=filtered, mask=edge_mask, planes=[0, 1, 2], first_plane=True)
output = core.std.Merge(clipa=video, clipb=edge_merged, weight=[0.45])

If I can control the weight in the mask itself, how would that code snippet look if I wanted to replace the "core.std.Merge(clipa=video, clipb=edge_merged, weight=[0.45]" part by reducing the mask weight 45%?

[Selur]

@masterboy: TEdgeMask by default creates a binary mask, but if you set threshold to 0 you can set scale,... doesn't that do what you are aiming for?