Avisynth woes - ffmpeg hanging, denoising

[ENunn]

I'm having issues getting my script encoded. When I start encoding with ffmpeg, I get around 6fps. Could be better but, eh. I left it on overnight, the speed has now dropped to 0.8fps and has hanged. No CPU or GPU is being used. I've been plagued by this issue for months.

I thought the initial fix was to add setmemorymax(63448) to the top of my script but now it's not helping. I don't know what's causing it. I know my script is a bit heavy, but I can wait it out at 6fps, not when it's frozen.

Does anyone know how to fix the issue?

Script (Source file is FFV1):
setmemorymax(63448)
v = lwlibavvideosource("i:\virtualdub\tape transfers\angelcam g4 lot\tape 37.mkv", fpsnum=30000, fpsden=1001) #atrack=0,
a = lwlibavaudioSource("g:\recordings\angelcam tape 37.w64")
AudioDub(v,a)
delayaudio(-0.150)
getleftchannel()
assumetff().converttoyuv422(matrix="rec601", interlaced=true).convertbits(10)
Levels(20, 1, 1045, 0, 1023, coring=false,dither=true).tweak(hue=-3, coring=false, dither=true).convertbits(8)
#turnRight().Histogram().TurnLeft()
Trim(68, 615963)
Trim(54, 81752)
normalize(0.8912)
CCD(interlaced=true, u=10, v=1)
qtgmcp(Preset="slow", Tuning="DV-SD", EZKeepGrain=1.0, sourcematch=3, tr0=2, tr1=2, tr2=3, lossless=2, MatchEnhance=1, NoiseProcess=1, NoiseRestore=1.0, Sigma=2.0, SLMode=4, subpel=4)
MergeChroma(aWarpSharp2(depth=20,threads=4))MergeChroma(aWarpSharp2(depth=10,threads=4))
santiag(strv=0, nns=4, nsize=5, threads=4)
SMDegrain(str=0.8, refinemotion=true)
cropresize(indar=4.0/3.0, resizewo=true)
ConvertToYv24(matrix="rec601",interlaced=false)
Crop(8, 0, -5, -15)
SuperRes(matrixin="rec601", 2, .43, 0, """edi_rpow2(2, nns=4, cshift="Spline16Resize")""", matrixout="rec709")
x = float(width) / float(height)
spline36resize(round(x * 2164),2164)
converttoyv12(matrix="rec709", interlaced=false)
prefetch(6)

ffmpeg command:
-c:v hevc_nvenc -gpu any -profile:v rext -tier high -g 30 -rc vbr -cq 12 -qmin 12 -qmax 12 -b:v 0K -preset p7 -c:a copy

Specs:
ASUS Prime Z390-A
Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz
Zotac NVIDIA GeForce RTX 2060 6 GB
G.SKILL TridentZ RGB Series 64GB DDR4 3600MHz
Samsung 960 EVO 500GB M.2
2x WD Black 4TB

I have also tired to use h264_nvenc to no avail. CPU encoding gives me with ~3fps x264 slow and ~0.9fps with x265 slow. Expected, but I want to take advantage of the full speed of the script, which is why I've been using NVENC. Final output looks fine to me.

[Boulder]

Do you happen to have avstp.dll anywhere in your path? If so, remove it.

[Reel.Deel]

@ENunn

No offense here but your script is doing some questionable things. Try to streamline the script a bit.

For example:

If your source is YUV420 do not do any chroma subsampling conversions;
MergeChroma(aWarpSharp2(depth=20,threads=4))MergeChroma(aWarpSharp2(depth=10,threads=4)) can be called like this aWarpSharp2(depth=20,threads=4,chroma=2).aWarpSharp2(depth=10,threads=4,chroma=2) and it might be better to disable internal multithreading since AVS+ is already doing it;
Try to combine deinterlacing and denoising in one step with QTGMC;
Odd that you need additional anti-aliasing after denoising;
SuperRes with EEDI3 I bet is slow, try using NNEDI3 or maybe W2xncnnvk;
This has no effect on speed but ConvertToXX(matrix="rec709") or any other matrix has no effect on YUV, only on YUV<->RGB conversions.


Do you have a short sample you can share?

[FranceBB]

Do you happen to have avstp.dll anywhere in your path? If so, remove it.

Boulder is right, go to your plugins folder and delete avstp.dll as this is almost definitely caused by this known bug: https://github.com/pinterf/mvtools/issues/46

[kedautinh12]

QTGMC() will be faster than qtgmcp() in 8 bit
https://github.com/realfinder/AVS-Stuff/blob/Community/avs%202.6%20and%20up/QTGMC.avsi

[ENunn]

Do you happen to have avstp.dll anywhere in your path? If so, remove it.
Boulder is right, go to your plugins folder and delete avstp.dll as this is almost definitely caused by this known bug: https://github.com/pinterf/mvtools/issues/46

It's been removed. Thanks for the heads up!

@ENunn
If your source is YUV420 do not do any chroma subsampling conversions;

Source is YUV422, I convert to 10-bit to fix levels without any major banding, then convert to YV24 to do pixel-accurate cropping, then convert to YV12 for the final output. I don't switch back and forth between color spaces.

MergeChroma(aWarpSharp2(depth=20,threads=4))MergeChroma(aWarpSharp2(depth=10,threads=4)) can be called like this aWarpSharp2(depth=20,threads=4,chroma=2).aWarpSharp2(depth=10,threads=4,chroma=2) and it might be better to disable internal multithreading since AVS+ is already doing it;

Removed any mention of threads, thanks. I did see a slight difference in changing the line, and I'm not sure if I prefer it.

MergeChroma(aWarpSharp2(depth=20,threads=4))MergeChroma(aWarpSharp2(depth=10,threads=4))
https://i.ibb.co/C2cF32w/image.png

aWarpSharp2(depth=20,threads=4,chroma=2).aWarpSharp2(depth=10,threads=4,chroma=2)
https://i.ibb.co/wc1qHmy/image.png

Try to combine deinterlacing and denoising in one step with QTGMC;
That could work, but I do prefer SMDegrain's denoising over QTGMC's.

Odd that you need additional anti-aliasing after denoising;

I don't use aWarpSharp for AA, I use it to fix chroma/color errors, and remove some streaks in my tape transfers.

Without MergeChroma(aWarpSharp2):
https://i.ibb.co/vcFsN9z/image.png

With MergeChroma(aWarpSharp2):
https://i.ibb.co/C2cF32w/image.png

I use Santiag for AA. Looks so much better than without it.

Without Santiag:
https://i.ibb.co/dWqzw79/image.png

With Santiag:
https://i.ibb.co/9pyfFDM/image.png

SuperRes with EEDI3 I bet is slow, try using NNEDI3 or maybe W2xncnnvk;

Changed it to NNEDI3. Thanks!

This has no effect on speed but ConvertToXX(matrix="rec709") or any other matrix has no effect on YUV, only on YUV<->RGB conversions.

Ahh, OK. Just wanted to make sure everything was converted properly from 601 to 709.

Do you have a short sample you can share?

My source? I'll send as soon as I can (got work), you probably won't have any issue with a small clip of a 40 minute source.

QTGMC() will be faster than qtgmcp() in 8 bit
https://github.com/realfinder/AVS-Stuff/blob/Community/avs%202.6%20and%20up/QTGMC.avsi

Interesting. I'll get that replaced.

Thanks for the optimizations everyone! Will speed also decrease if my source is using FFV1? I know it's pretty slow to decode (wish there was a GPU accelerated version out there :P)

Unfortunately these usually take about 7hrs a piece, so I won't know if it's properly encoded until then. I'll post back once this is all done.

[kedautinh12]

About Santiag for AA. I think your sample same blocking not AA. Try Deblock scrips like Deblock_QED combine with SM_Degrain
https://github.com/Dogway/Avisynth-Scripts/blob/master/EX%20mods/DeblockPack.avsi#L9

Example:


# Remove blocking and smooth out jagged edges (only applies where motion match fails)
mb=Deblock_QED(quant1=30, quant2=40,UV=1)
mb=mb.ex_smooth(1, mode="SG", limit=true, sharp=true)

# LFR to recover some smearing. limits=false to don't sharpen motion areas
SMDegrain(6, 400, prefilter=6, mfilter=mb, ContraSharp=true, RefineMotion=true, plane=4, LFR=300, limits=false, DCTFlicker=false)

[Reel.Deel]

My source? I'll send as soon as I can (got work), you probably won't have any issue with a small clip of a 40 minute source.


I'm not trying to test the encoding time of a full clip, it's more to see what the actual output of your script looks like and if can be made faster by means of streamlining your script. Things start to become slow when you use many filters, albeit sometimes needed, but there are times when you can achieve the same results with less.

And maybe try running the script in AVSMeter with SetLogParams("stdout", 4) at the very beginning of your script. The logging facility in AVS+ will automatically log errors, and will issue warnings and notes about potential problems, buggy plugins, suboptimal settings, etc.

[ENunn]

About Santiag for AA. I think your sample same blocking not AA. Try Deblock scrips like Deblock_QED combine with SM_Degrain
https://github.com/Dogway/Avisynth-Scripts/blob/master/EX%20mods/DeblockPack.avsi#L9

Example:

Not sure what I like better to be honest.

mb=Deblock_QED(quant1=30, quant2=40,UV=1)
mb=mb.ex_smooth(1, mode="SG", limit=true, sharp=true)
SMDegrain(6, 400, prefilter=6, mfilter=mb, ContraSharp=true, RefineMotion=true, plane=4, LFR=300, limits=false, DCTFlicker=false)
https://i.imgur.com/eBEAc8l.png

santiag(strv=0, nns=4, nsize=5).SMDegrain(str=0.8, refinemotion=true)
https://i.imgur.com/5u2BQY1.png

I'm not trying to test the encoding time of a full clip, it's more to see what the actual output of your script looks like and if can be made faster by means of streamlining your script. Things start to become slow when you use many filters, albeit sometimes needed, but there are times when you can achieve the same results with less.

Ahhhh, OK. Here's a sample.
(https://mega.nz/file/DglVgSwS#EkHwJSQOWf6p1HLfsX9rEVsSKVrhD3_xGIyObg26Ktw)
And maybe try running the script in AVSMeter with SetLogParams("stdout", 4) at the very beginning of your script. The logging facility in AVS+ will automatically log errors, and will issue warnings and notes about potential problems, buggy plugins, suboptimal settings, etc.

Ran it, I pasted the log in a text file, that's attached.

I did a test encode after removing threads/avstp and changing QTGMCp to QTGMC, I'm now encoding at 9fps. Definitely better than 6.

[Reel.Deel]

Ran it, I pasted the log in a text file, that's attached.

I did a test encode after removing threads/avstp and changing QTGMCp to QTGMC, I'm now encoding at 9fps. Definitely better than 6.

3 fps increase, that's a start :)

Paste your log on pastebin.com or something similar. Attachments here take forever and a day to get approved.

[ENunn]

Paste your log on pastebin.com or something similar. Attachments here take forever and a day to get approved.

Here you go. (https://pastebin.com/gxrg5J9s)

[kedautinh12]

copy this scripts to file mtmodes.avsi and place this file in your plugins folders
https://publishwith.me/ep/pad/view/ro.rDkwcdWn4k9/latest

[Reel.Deel]

@ENunn

I took a look at the sample, along with the script and to be honest I think some of your settings for QTGMC are way overkill for the quality of the source. The chroma with some bands and the luma has some type of pattern on it. I think with the settings you use for QTGMC accentuates those artifacts and thus needing to use santiag afterwards to mitigate them. Is your intent to keep the video noisy?

Does something like this seem acceptable to you? https://imgsli.com/MTM0MDkw

[ENunn]

copy this scripts to file mtmodes.avsi and place this file in your plugins folders
https://publishwith.me/ep/pad/view/ro.rDkwcdWn4k9/latest
Looks like I already had this, I replaced it with the one you sent. Thanks.

I took a look at the sample, along with the script and to be honest I think some of your settings for QTGMC are way overkill for the quality of the source. The chroma with some bands and the luma has some type of pattern on it. I think with the settings you use for QTGMC accentuates those artifacts and thus needing to use santiag afterwards to mitigate them. Is your intent to keep the video noisy?

Nah, not at all. I need to find a good denoising settings for my ES15 transfers. I've been using the same settings for all my transfers, and they've all looked fine except for most of my ES15 captures.

I have to change my VCR to edit mode because with the TBC off, there's visual errors with it set to norm. Sadly this introduces a lot of noise. Doesn't help that the source is a 2nd gen tape (1st gen is probably lost and taped over by now).

Does something like this seem acceptable to you? https://imgsli.com/MTM0MDkw

Are those new settings or my script? Looks definitely better than what I saw with my script.

An update for encoding: it took five hours but it eventually completely rendered! Thanks for the help in getting my encodes to stop hanging!

[kedautinh12]

Add SelectEven() beside QTGMC() will be faster. Example: QTGMC(Preset="Slow", Sharpness=0).SelectEven() but may cause error in trans screen. Only suggestion Not test with your sample

[Reel.Deel]

Add SelectEven() beside QTGMC() will be faster.

Adding SelectEven() does not make QTGMC faster, the processing speed still remains the same for any processes after/before SelectEven(). It's only 'faster' because there is half the frames to process, not sure if that is what he wants.

Are those new settings or my script? Looks definitely better than what I saw with my script.


It's my own frankenstein script (borrowed snippets from a few other scripts), I can post it later if you want. Unless you're done with this clip.

Here's the script that I used in case you're interested. No cropping or upscaling there. The script is a modification of a script Didée wrote a long time ago: https://forum.doom9.org/showthread.php?p=1107122#post1107122

FFVideoSource("tape 37-trim.mkv")
src = last
BWDIF(field=-2)#QTGMC(preset="fast")
o = last

mds()
spline36resize(o.width(),o.height()/2/4*4)
mds()
spline36resize(o.width(),o.height()/6/4*4)
mds()
spline36resize(o.width(),o.height())

o.mergechroma(last)
nostripes2 = last.minblur(1)#LSFplus(ss_x=1.0,ss_y=1.0,strength=200,soft=100,overshoot=5)

srchbase = nostripes2.fft3dfilter(sigma=6,sigma2=4,sigma3=3,sigma4=2,bt=5,bw=16,bh=16,ow=8,oh=8,plane=4).fluxsmootht()
pre = Merge(srchbase, o, weight=.45)

BLK = 16 # 8
ME1 = 5
ME2 = 2 # 8
_DCT = 5

sup1 = srchbase.MSuper(hpad=16, vpad=16, pel=2, levels=0,sharp=1)
sup2 = pre.MSuper(hpad=16, vpad=16, pel=2, sharp=0)
bv2 = MAnalyse(sup2,delta=2,truemotion=false,global=true,blksize=BLK,overlap=BLK/2,search=ME1,searchparam=ME2,isb=true, dct=_DCT)
bv1 = MAnalyse(sup2,delta=1,truemotion=false,global=true,blksize=BLK,overlap=BLK/2,search=ME1,searchparam=ME2,isb=true, dct=_DCT)
fv1 = MAnalyse(sup2,delta=1,truemotion=false,global=true,blksize=BLK,overlap=BLK/2,search=ME1,searchparam=ME2,isb=false,dct=_DCT)
fv2 = MAnalyse(sup2,delta=2,truemotion=false,global=true,blksize=BLK,overlap=BLK/2,search=ME1,searchparam=ME2,isb=false,dct=_DCT)

mdegrain2(sup1,bv1,fv1,bv2,fv2,thsad=200,thSCD1=256,thSCD2=96)
QTGMCp(src, EdiExt=last)

Interleave(o,last)
PointResize(width*2, height*2)



# helper function: vertical-only MinBlur with vertical radius 2

function mds(clip clp)
{
a = clp.mt_luts(clp,mode="median",pixels="0 -2 0 -1 0 0 0 1 0 2",Y=2,U=3,V=3)
b = clp.mt_convolution("0 1 0","1 2 1",y=2,U=3,V=3).mt_convolution("0 1 0","1 2 1",Y=2,U=3,V=3)
aD = mt_makediff(clp,a,Y=2,U=3,V=3)
bD = mt_makediff(clp,b,Y=2,U=3,V=3)
DD = mt_lutxy(aD,bD,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?",Y=2,U=3,V=3)
clp.mt_makediff(DD,Y=2,U=3,V=3)
return(last)
}


Needs MinBlur (http://avisynth.nl/index.php/MinBlur)

[kedautinh12]

Yes, i mean faster is half the frames to process :D

[Reel.Deel]

Yes, i mean faster is half the frames to process :D

QTGMC().SelectEven() is your solution to everything. Third time I see you suggest that, in the last week :p

[kedautinh12]

Because it's faster :D

[ENunn]

It's my own frankenstein script (borrowed snippets from a few other scripts), I can post it later if you want. Unless you're done with this clip.

Here's the script that I used in case you're interested. No cropping or upscaling there. The script is a modification of a script Didée wrote a long time ago: https://forum.doom9.org/showthread.php?p=1107122#post1107122

FFVideoSource("tape 37-trim.mkv")
src = last
BWDIF(field=-2)#QTGMC(preset="fast")
o = last

mds()
spline36resize(o.width(),o.height()/2/4*4)
mds()
spline36resize(o.width(),o.height()/6/4*4)
mds()
spline36resize(o.width(),o.height())

o.mergechroma(last)
nostripes2 = last.minblur(1)#LSFplus(ss_x=1.0,ss_y=1.0,strength=200,soft=100,overshoot=5)

srchbase = nostripes2.fft3dfilter(sigma=6,sigma2=4,sigma3=3,sigma4=2,bt=5,bw=16,bh=16,ow=8,oh=8,plane=4).fluxsmootht()
pre = Merge(srchbase, o, weight=.45)

BLK = 16 # 8
ME1 = 5
ME2 = 2 # 8
_DCT = 5

sup1 = srchbase.MSuper(hpad=16, vpad=16, pel=2, levels=0,sharp=1)
sup2 = pre.MSuper(hpad=16, vpad=16, pel=2, sharp=0)
bv2 = MAnalyse(sup2,delta=2,truemotion=false,global=true,blksize=BLK,overlap=BLK/2,search=ME1,searchparam=ME2,isb=true, dct=_DCT)
bv1 = MAnalyse(sup2,delta=1,truemotion=false,global=true,blksize=BLK,overlap=BLK/2,search=ME1,searchparam=ME2,isb=true, dct=_DCT)
fv1 = MAnalyse(sup2,delta=1,truemotion=false,global=true,blksize=BLK,overlap=BLK/2,search=ME1,searchparam=ME2,isb=false,dct=_DCT)
fv2 = MAnalyse(sup2,delta=2,truemotion=false,global=true,blksize=BLK,overlap=BLK/2,search=ME1,searchparam=ME2,isb=false,dct=_DCT)

mdegrain2(sup1,bv1,fv1,bv2,fv2,thsad=200,thSCD1=256,thSCD2=96)
QTGMCp(src, EdiExt=last)

Interleave(o,last)
PointResize(width*2, height*2)



# helper function: vertical-only MinBlur with vertical radius 2

function mds(clip clp)
{
a = clp.mt_luts(clp,mode="median",pixels="0 -2 0 -1 0 0 0 1 0 2",Y=2,U=3,V=3)
b = clp.mt_convolution("0 1 0","1 2 1",y=2,U=3,V=3).mt_convolution("0 1 0","1 2 1",Y=2,U=3,V=3)
aD = mt_makediff(clp,a,Y=2,U=3,V=3)
bD = mt_makediff(clp,b,Y=2,U=3,V=3)
DD = mt_lutxy(aD,bD,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?",Y=2,U=3,V=3)
clp.mt_makediff(DD,Y=2,U=3,V=3)
return(last)
}


Needs MinBlur (http://avisynth.nl/index.php/MinBlur)

Hoooo boy. This helped out a lot!!! I modified and added stuff to the script, now I'm getting at least 12 fps and it looks so much better.

Old script:
https://i.imgur.com/QfVGevg.png

New script:
https://i.imgur.com/GkXIg10.png

Easily compare:
https://imgsli.com/MTM0MDk4

Thanks so much Real! Now to see what looks good for non ES15 tapes.

[Reel.Deel]

I'm glad you like it and improved the overall speed. Here's a more tweaked script, I got rid of the superfluous denoising step and other things like use NNEDI3CL and neo_fft3d. I also incorporated the upscale, I used NNEDI3CL for that, not SuperRes (didn't check the differences) but the source has very low detail so I don't think the difference will be night and day. z_ConvertFormat resizes to the final size, converts it to YUV420, corrects the shift caused by NNEDI3 upscaling and also changes the Colorimetry all in one go :). Script now also needs LSFplus.

I don't know the quality of your other tapes but this one here works well for low quality VHS with chroma bands and very noisy. Definitely overkill for higher quality footage. Enjoy, and if you come across another hard to denoise clip, please share ;)

FFVideoSource("tape 37-trim.mkv")
Crop(8,0,-6,-16)
ConvertBits(10)
Levels(20, 1, 1045, 0, 1023, coring=false, dither=true)
Tweak(hue=-3, coring=false, dither=true)
ConvertBits(8)
src = last
NNEDI3CL(3)
o = last

mds().Spline36Resize(o.width(),o.height()/2/4*4)
mds().Spline36Resize(o.width(),o.height()/6/4*4)
mds().Spline36Resize(o.width(),o.height())

o.MergeChroma(last)
nostripes2 = last.MinBlur(1).LSFplus(ss_x=1.0, ss_y=1.0, strength=100, soft=100, overshoot=5)
srchbase = neo_fft3d(nostripes2,sigma=6, sigma2=4, sigma3=3, sigma4=2, bt=5, bw=16, bh=16, ow=8, oh=8).FluxSmoothT()
pre = MergeLuma(srchbase, o, weight=.25)

QTGMCp(src, EdiExt=pre)

NNEDI3CL(1, dh=true, dw=true)
NNEDI3CL(1, dh=true, dw=true)
NNEDI3CL(1, dh=true, dw=true)

z_ConvertFormat(2880, 2160, pixel_type="YV12", colorspace_op="470bg:601:170m:full=>709:709:709:full", \
resample_filter="bicubic", filter_param_a=-0.5, filter_param_b=0.25, src_left=-3.5, src_top=-3.5)



# helper function: vertical-only MinBlur with vertical radius 2

function mds(clip clp)
{
a = clp.mt_luts(clp,mode="median",pixels="0 -2 0 -1 0 0 0 1 0 2",Y=2,U=3,V=3)
b = clp.mt_convolution("0 1 0","1 2 1",y=2,U=3,V=3).mt_convolution("0 1 0","1 2 1",Y=2,U=3,V=3)
aD = mt_makediff(clp,a,Y=2,U=3,V=3)
bD = mt_makediff(clp,b,Y=2,U=3,V=3)
DD = mt_lutxy(aD,bD,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?",Y=2,U=3,V=3)
clp.mt_makediff(DD,Y=2,U=3,V=3)
return(last)
}

[StainlessS]

RD,
Not sure of your intent there, but
mds().Spline36Resize(o.width(),o.height()/2/4*4)
is same as
mds().Spline36Resize(o.width(),o.height()/2)

as intended ?

maybe something like
mds().Spline36Resize(o.width(),Round(o.height()/2.0/4.0)*4)

[Reel.Deel]

StainlessS, it's a copypasta from a script from Didée (https://forum.doom9.org/showthread.php?p=1107122#post1107122), I didn't dare to challenge his logic :D

Edit: to below - Round() would work, the script is meant for progressive video so mod2 should be enough.

[StainlessS]

I bet it should have been as suggested (or eg Int() instead of round), same for the 6/4/4.

EDIT: Whether or not you should use Round() or Int(), it is intended that result height should be mod4. [suggest Round()]

[StainlessS]

Edit: to below - Round() would work, the script is meant for progressive video so mod2 should be enough.
This lot would seem to require reduction to 1/2 size (mod4),
and then to reduction to 1/6 size (mod4).
Then resize back to original height.
So downsize to 1/12 height, then back up.
I imagine that the big DD had reason for double downsize, and maybe even the modulo stuff between and ending.
(dont think anything to do with Progressive/Interlaced)

mds().Spline36Resize(o.width(),o.height()/2/4*4) # /2
mds().Spline36Resize(o.width(),o.height()/6/4*4) # /6 : total /12
mds().Spline36Resize(o.width(),o.height()) # back to original

EDIT:
so maybe

mds().Spline36Resize(o.width(),Round(o.height()/2.0/4)*4) # /2
mds().Spline36Resize(o.width(),Round(o.height()/6.0/4)*4) # /6 : total /12
mds().Spline36Resize(o.width(),o.height()) # back to original

EDIT: And instead of Int() if required, just divide by int instead of float

mds().Spline36Resize(o.width(),(o.height()/2/4)*4) # /2
mds().Spline36Resize(o.width(),(o.height()/6/4)*4) # /6 : total /12
mds().Spline36Resize(o.width(),o.height()) # back to original

[Reel.Deel]

This lot would seem to require reduction to 1/2 size (mod4),
and then to reduction to 1/6 size (mod4).
Then resize back to original height.
So downsize to 1/12 height, then back up.
I imagine that the big DD had reason for double downsize, and maybe even the modulo stuff between and ending.
(dont think anything to do with Progressive/Interlaced)

mds().Spline36Resize(o.width(),o.height()/2/4*4) # /2
mds().Spline36Resize(o.width(),o.height()/6/4*4) # /6 : total /12
mds().Spline36Resize(o.width(),o.height()) # back to original

EDIT:
so maybe

mds().Spline36Resize(o.width(),Round(o.height()/2.0/4)*4) # /2
mds().Spline36Resize(o.width(),Round(o.height()/6.0/4)*4) # /6 : total /12
mds().Spline36Resize(o.width(),o.height()) # back to original

EDIT: And instead of Int() if required, just divide by int instead of float

mds().Spline36Resize(o.width(),(o.height()/2/4)*4) # /2
mds().Spline36Resize(o.width(),(o.height()/6/4)*4) # /6 : total /12
mds().Spline36Resize(o.width(),o.height()) # back to original


Gotcha. If I decide to make a VHS band reducer script, I will incorporate those changes. Thanks StainlessS

[ENunn]

Thanks for the updated script.

I tried it on one of my typical VHS captures. VCR set to norm. Relatively low noise to begin with but I do want to slightly denoise before scaling. Not sure what I prefer.

Source:
https://i.imgur.com/RK9Hsc7.png

Old script:
https://i.imgur.com/TJxgdDM.png

New script (with RF's first settings, modified)
https://i.imgur.com/7oxKMxJ.png

New script #2 (with RF's second settings, modified)
https://i.imgur.com/0wN3jiz.png]

No scaling applied yet. All look clean though. I'll send a sample once I'm off work. If only I could improve text clarity to make it on-par with the source.

[johnmeyer]

Do you have the edit switch on your VCR turned on? Those captures look pretty soft to me.

If your are unfamiliar with the edit switch, consult your VCR's manual to see the settings they recommend for dubbing to another tape deck. You want to use those settings. Turning that switch ON turns OFF the built-in noise filtering. You do NOT want the VCR to filter noise because it kills details in order to reduce all those little dots. You will never get back that detail once lost. Once you have your capture, the digital noise filters available today are 100x better than the simple analog circuits used in a VCR. The final result will be more detailed and better in every way.

[ENunn]

Do you have the edit switch on your VCR turned on? Those captures look pretty soft to me.

I honestly prefer Norm over Edit. I'm not a fan of all that added noise. I find Norm to be more natural to me. I only use Edit if I use my ES15.

I'll take your word for it though :P I'll do a capture with Edit on and try it out.

[johnmeyer]

You are absolutely correct: setting that switch to Edit will make the picture noisier. If you were just watching the video on your TV, that might be what you want, but it is not what you want when digitizing video.

To convince yourself, capture a scene which has a lot of detail (leaves, grass, striped shirt, a shag rug, etc.). Do that with Edit on and then do it again with Edit off. Then do an A/B between your videos while they are running (I do this by putting them above each other in my NLE and then muting/unmuting one track). You can also look at stills captured from each. Ignore the noise, but look at the detail. You should be able to resolve more detail, and even the detail that is already there should reveal more information when Edit is on (i.e., noise reduction is turned off).

There are dozens of AVISynth plugins that can very effectively remove most of the noise without killing the detail. The VCR approach is to just apply a low pass filter which removes the noise and also the details. It has no way to "look" at pixels (because there are no pixels in analog video) and then compare those pixels from one frame to the next, something a temporal denoiser does. It is this frame-to-frame comparison that permits a digital temporal denoiser to get rids of all those dancing dots without removing detail.

BTW, a similar comment applies to removing noise with AVISynth: resist the urge to get rid of ALL the noise because if you do that, you often remove detail.

Remember, "NR" stands for noise reduction, not noise removal.

[ENunn]

Thanks for the help John! I did my first edit mode transfer in a long time, results do look nice, albeit very noisy.

I ran the capture through the script, there's definitely a difference but I do want a little better.

Original:
https://i.imgur.com/cSXw0es.png

RealFinder's settings (modified by me):
https://i.imgur.com/NTkN0KC.png

There's definitely a difference, but I dunno if it's on-par with what my VCR can do. At least detail isn't lost :P

When I try to modify fft3dfilter, MSuper, and mdegrain2, nothing seems to happen. I do want the noise reduced to the point that detail isn't lost.

[kedautinh12]

TemporalDegrain2 can do it but speed very slow

[Reel.Deel]

@ENunn
Post a small sample.

I'm starting to get a little confused. I saw you wrote the name RF earlier and now RealFinder. Are you confusing me with RF? :D

[kedautinh12]

Maybe he think you're a copy ver of RealFinder :D

[ENunn]

TemporalDegrain2 can do it but speed very slow

Yeah, I used it before I went with SMDegrain. Really slow.

@ENunn
Post a small sample.

Got it. (https://mega.nz/file/C9chTQxJ#lXJLCCktA09sNPLzguL4pJ3lgjNSEDb-HhprLyn5DBM)

I'm starting to get a little confused. I saw you wrote the name RF earlier and now RealFinder. Are you confusing me with RF? :D

Yeah :p Little bit confusing on my part.

Maybe he think you're a copy ver of RealFinder :D

It's true, they're the exact same person!!!!!!! It's true!!!1one

[Reel.Deel]

RF and I are most definitely not the same person lol

I took a look at your second sample, and in terms of noise it's much better than the first sample. The luma channel has much less noise and no crosshatch patterns. But the chroma is still noisy, not the same as before but enough to use the same script as post #21. I only made one change:

pre = MergeLuma(srchbase, o, weight=.75) # was .45

This way it only does a little bit of denoising on the luma channel and helps retain details.

Here are the results:

https://imgsli.com/MTM0NTYz
https://imgsli.com/MTM0NTY0

Images are 2x point resampled and crop, levels, tweak, and upscaling parts of script were commented out.

By the way, I think I recognize some of those commercials. Definitely the Neutrogena one. I used to watch WGN in my younger days ;)

[ENunn]

Sorry for the late response.

I took a look at your second sample, and in terms of noise it's much better than the first sample. The luma channel has much less noise and no crosshatch patterns. But the chroma is still noisy, not the same as before but enough to use the same script as post #21. I only made one change:

pre = MergeLuma(srchbase, o, weight=.75) # was .45

This way it only does a little bit of denoising on the luma channel and helps retain details.

Here are the results:

https://imgsli.com/MTM0NTYz
https://imgsli.com/MTM0NTY0


Thanks for this, there's definitely a difference but I do have one issue with it. I don't know how to word it. There's a shift when there's an area of red. I think it's due to the smoothing, but I'd like to alleviate that if possible.

Original:
https://i.imgur.com/Xn7SMjN.png

Script:
https://i.imgur.com/jZ9l17P.png

I posted another screenshot of the "problem", screenshot 4 on post 27. (https://forum.doom9.org/showthread.php?p=1978300#post1978300)

Here's a sample. (https://mega.nz/file/6ts2FDyR#nyWnBXIJ3E9gH1WKkA6EDy97y3biZXB5_LcqzonnB70)

It's probably not that big of a deal though. I just saw it and it looked a tad off to me.

[Emulgator]

If you consider the brighter, but narrow red stripes weighed into it seems resonable.

[ENunn]

Hey there. Wanted to post here again, I recently got myself a Panasonic AG-1980P after my old JVC stopped taking tapes. I initially transferred my tapes with the picture switch set to DETAIL. However, there's chroma ghosting. I have now set the switch to EDIT, and the ghosting is gone. There's a lot more noise, however, and I'm seeing if I can filter it out without destroying too much detail.

Original-ish
AVISource("J:\virtualdub\tape transfers\wizards vs bulls 1-4-02.avi")
assumetff().converttoyuv422(matrix="rec601", interlaced=true).convertbits(10)
#Crop(8, 4, -24, -6)
Levels(50, 1,915, 0, 1020, coring=false,dither=true).tweak(bright=0, cont=1.00, hue=-0, sat=1.00, coring=false, dither=true).convertbits(8)
Trim(1084, 277267)
QTGMC(Tuning="DV-SD", ezdenoise=0.0, EZKeepGrain=1.0, sourcematch=3, tr0=2, tr1=2, tr2=3, lossless=2, MatchEnhance=1, NoiseProcess=1, NoiseRestore=1.0, Sigma=2.0, SLMode=4, subpel=4)
selectodd()
https://i.imgur.com/d7YiT3E.png

I initially used the script in post #21, but with the modifications mentioned in post #36 and other modifications.


AVISource("J:\virtualdub\tape transfers\wizards vs bulls 1-4-02.avi")
assumetff().converttoyuv422(matrix="rec601", interlaced=true).convertbits(10)
#Crop(8, 4, -24, -6)
Levels(50, 1,915, 0, 1020, coring=false,dither=true).tweak(bright=0, cont=1.00, hue=-0, sat=1.00, coring=false, dither=true).convertbits(8)
Trim(1084, 277267)
src = last
NNEDI3CL(3)
o = last

mds().Spline36Resize(o.width(),o.height()/2/4*4)
mds().Spline36Resize(o.width(),o.height()/6/4*4)
mds().Spline36Resize(o.width(),o.height())

o.MergeChroma(last)
nostripes2 = last.MinBlur(1).LSFplus(ss_x=1.0, ss_y=1.0, strength=100, soft=100, overshoot=5)
srchbase = neo_fft3d(nostripes2,sigma=6, sigma2=4, sigma3=3, sigma4=2, bt=5, bw=16, bh=16, ow=8, oh=8).FluxSmoothT()
pre = MergeLuma(srchbase, o, weight=.75)

QTGMC(src, EdiExt=pre, Tuning="DV-SD", EZKeepGrain=1.0, sourcematch=3, tr0=2, tr1=2, tr2=3, lossless=2, MatchEnhance=1, NoiseProcess=1, NoiseRestore=1.0, Sigma=2.0, SLMode=4, subpel=4)

# helper function: vertical-only MinBlur with vertical radius 2

function mds(clip clp)
{
a = clp.mt_luts(clp,mode="median",pixels="0 -2 0 -1 0 0 0 1 0 2",Y=2,U=3,V=3)
b = clp.mt_convolution("0 1 0","1 2 1",y=2,U=3,V=3).mt_convolution("0 1 0","1 2 1",Y=2,U=3,V=3)
aD = mt_makediff(clp,a,Y=2,U=3,V=3)
bD = mt_makediff(clp,b,Y=2,U=3,V=3)
DD = mt_lutxy(aD,bD,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?",Y=2,U=3,V=3)
clp.mt_makediff(DD,Y=2,U=3,V=3)
return(last)
}
selectodd()
https://i.imgur.com/MUGkKDh.png

It got rid of some noise, but not most of it. So I tried the first script I was given on post #16, also slightly modified.

AVISource("J:\virtualdub\tape transfers\wizards vs bulls 1-4-02.avi")
assumetff().converttoyuv422(matrix="rec601", interlaced=true).convertbits(10)
#Crop(8, 4, -24, -6)
Levels(50, 1,915, 0, 1020, coring=false,dither=true).tweak(bright=0, cont=1.00, hue=-0, sat=1.00, coring=false, dither=true).convertbits(8)
Trim(1084, 277267)
#turnRight().Histogram().TurnLeft()
src = last
#BWDIF(field=-2)#QTGMC(preset="fast")
qtgmc(Preset="slow", Tuning="DV-SD", EZKeepGrain=1.0, sourcematch=3, tr0=2, tr1=2, tr2=3, lossless=2, MatchEnhance=1, NoiseProcess=1, NoiseRestore=1.0, Sigma=2.0, SLMode=4, subpel=4)
o = last
mds()
spline36resize(o.width(),o.height()/2/4*4)
mds()
spline36resize(o.width(),o.height()/6/4*4)
mds()
spline36resize(o.width(),o.height())

o.mergechroma(last)
nostripes2 = last.minblur(1)#LSFplus(ss_x=1.0,ss_y=1.0,strength=200,soft=100,overshoot=5)

srchbase = nostripes2.fft3dfilter(sigma=6,sigma2=4,sigma3=3,sigma4=2,bt=5,bw=16,bh=16,ow=8,oh=8,plane=4).fluxsmootht()
pre = Merge(srchbase, o, weight=.45)

BLK = 16 # 8
ME1 = 5
ME2 = 2 # 8
_DCT = 5

sup1 = srchbase.MSuper(hpad=16, vpad=16, pel=2, levels=0,sharp=1)
sup2 = pre.MSuper(hpad=16, vpad=16, pel=2, sharp=0)
bv2 = MAnalyse(sup2,delta=2,truemotion=false,global=true,blksize=BLK,overlap=BLK/2,search=ME1,searchparam=ME2,isb=true, dct=_DCT)
bv1 = MAnalyse(sup2,delta=1,truemotion=false,global=true,blksize=BLK,overlap=BLK/2,search=ME1,searchparam=ME2,isb=true, dct=_DCT)
fv1 = MAnalyse(sup2,delta=1,truemotion=false,global=true,blksize=BLK,overlap=BLK/2,search=ME1,searchparam=ME2,isb=false,dct=_DCT)
fv2 = MAnalyse(sup2,delta=2,truemotion=false,global=true,blksize=BLK,overlap=BLK/2,search=ME1,searchparam=ME2,isb=false,dct=_DCT)

mdegrain2(sup1,bv1,fv1,bv2,fv2,thsad=200,thSCD1=256,thSCD2=96)
#QTGMc(src, EdiExt=last)
#qtgmc(src, EdiExt=last, Preset="slow", Tuning="DV-SD", EZKeepGrain=1.0, sourcematch=3, tr0=2, tr1=2, tr2=3, lossless=2, MatchEnhance=1, NoiseProcess=1, NoiseRestore=1.0, Sigma=2.0, SLMode=4, subpel=4)

Interleave(o,last)

# helper function: vertical-only MinBlur with vertical radius 2

function mds(clip clp)
{
a = clp.mt_luts(clp,mode="median",pixels="0 -2 0 -1 0 0 0 1 0 2",Y=2,U=3,V=3)
b = clp.mt_convolution("0 1 0","1 2 1",y=2,U=3,V=3).mt_convolution("0 1 0","1 2 1",Y=2,U=3,V=3)
aD = mt_makediff(clp,a,Y=2,U=3,V=3)
bD = mt_makediff(clp,b,Y=2,U=3,V=3)
DD = mt_lutxy(aD,bD,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?",Y=2,U=3,V=3)
clp.mt_makediff(DD,Y=2,U=3,V=3)
return(last)
}
selectodd()
selectodd()
https://i.imgur.com/1hPZ6TK.png

It got rid of a lot of noise, and it did retain quite a bit of detail, but I'm not sure if it looks fine or not. Does it look OK? I'm open to trying out another script if needed.

sample video (https://mega.nz/file/P8kT2TZL#tx9RsQdndpXk513qPhfeTQSdzt813BkqU3qOp2y-pHI)

I'm not worried about speed this time, 6fps is probably the best I'm gonna get with deinterlacing/denoising/upscaling.

[Emulgator]

Without looking at your sample (mobile limited data at the moment):
What I found more detailed in 2024: QTGMCp.
Denoising without destroying detail in 2024: You may consider trying SMdegrain with prefilter=8 (BM3D).

[ENunn]

Without looking at your sample (mobile limited data at the moment):
What I found more detailed in 2024: QTGMCp.
Denoising without destroying detail in 2024: You may consider trying SMdegrain with prefilter=8 (BM3D).

I tried prefilter=8, and it gave me an error.
https://i.imgur.com/0ONBg1q.png

[ENunn]

Nevermind that, I had a really old version of SMDegrain lol

It definitely retained a bit more detail than the last script I used.

AVISource("J:\virtualdub\tape transfers\wizards vs bulls 1-4-02.avi", true)
assumetff().converttoyuv422(matrix="rec601", interlaced=true).convertbits(10)
#Crop(8, 4, -24, -6)
Levels(50, 1,915, 0, 1020, coring=false,dither=true).tweak(bright=0, cont=1.00, hue=-0, sat=1.00, coring=false, dither=true).convertbits(8)
Trim(1084, 277267)
Trim(32502, 33402)
normalize(0.8912)
QTGMC(tuning="DV-SD", EZKeepGrain=1.0, sourcematch=3, tr0=2, tr1=2, tr2=3, lossless=2, MatchEnhance=1, NoiseProcess=1, NoiseRestore=1.0, Sigma=2.0, SLMode=4, subpel=4)
smdegrain(prefilter=8)
selectodd()
https://i.imgur.com/YoLrgWy.png

I'm very pleased with the results, but I forgot how slow this is. I should've clarified in my last post. I used to get 5-6fps on my last script, now I'm getting 3.5 with my full script (deinterlacing/denoise/filters/upscale). I know I said I don't care about speed, but what I actually meant was that I was fine with the speed I had, and if there was a way to get a speed boost, then that's good. Sadly SMDegrain is just slow. Is there any other scripts I can use that's somewhat faster and looks pretty good?

I don't think anything can be fixed, but here's my full script, just in case:
AVISource("J:\virtualdub\tape transfers\wizards vs bulls 1-4-02.avi", true)
assumetff().converttoyuv422(matrix="rec601", interlaced=true).convertbits(10)
#Crop(8, 4, -24, -6)
Levels(50, 1,915, 0, 1020, coring=false,dither=true).tweak(bright=0, cont=1.00, hue=-0, sat=1.00, coring=false, dither=true).convertbits(8)
Trim(1084, 277267)
Trim(32502, 33402)
normalize(0.8912)
QTGMC(tuning="DV-SD", EZKeepGrain=1.0, sourcematch=3, tr0=2, tr1=2, tr2=3, lossless=2, MatchEnhance=1, NoiseProcess=1, NoiseRestore=1.0, Sigma=2.0, SLMode=4, subpel=4)
spotless()
finedehalo()
smdegrain(prefilter=8)
MergeChroma(aWarpSharp2(depth=20))MergeChroma(aWarpSharp2(depth=10))
santiag(strv=0, nns=4, nsize=5)
cropresize(indar=15.0/11.0, resizewo=true)
#cropresize(indar=4.0/3.0, resizewo=true)
converttoyv24(matrix="rec601", interlaced=false)
SuperRes(matrixin="rec601", 2, .43, 0, """nnedi3_rpow2(4, nns=4, cshift="Spline16Resize")""", matrixout="rec709")
#SuperRes(2, .43, 0, """edi_rpow2(2, nns=4, cshift="Spline16Resize")""").SuperRes(2, .43, 0, """edi_rpow2(2, nns=4, cshift="Spline16Resize")""")
x = float(width) / float(height)
spline36resize(round(x * 2160),2160)
converttoyv12(matrix="rec709", interlaced=false)
prefetch(8)

[anton_foy]

I dont know if its possible to re-use motion vectors from QTGMC for spotless and smdegrain.

https://forum.doom9.org/showthread.php?p=1991979#post1991979

[DTL]

QTGMC can export MVs via globals. But that MVs may be not best because SMDegrain use prefilter to fix incoming source somehow (depending on prefilter used) and new generations of MVs in the SMDegrain may (or may not) be better in comparison with exported from QTGMC. Also the tr for MVs must be enough (like using same tr in both QTGMC and SMDegrain).

But because SMDegrain also has internal MDegrain (up to MDegrain6) - it may be faster to use only internal degrain in the QTGMC using MVs generated inside QTGMC.

If user want to make new MVs after some new interfiltering to use in the next MDegrain - it is not possible to reuse MVs exported from QTGMC.