Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 28th May 2021, 14:10   #21  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,275
Quote:
Originally Posted by Dogway View Post
That makes sense now. I thought it only was the convolution types. I also noticed chaining removegrain (or Dither_boxfilter) is very fast, whereas Expr() suffers a lot, probably due what you explained. So processing in 32-bit float would tell another story.
These masktools2 filters has no simd versions, probably because they do generic stuff.
mt_convolution, mt_gradient, mt_mappedblur, mt_hysteresis
pinterf is offline   Reply With Quote
Old 28th May 2021, 14:16   #22  |  Link
GMJCZP
Registered User
 
GMJCZP's Avatar
 
Join Date: Apr 2010
Location: I have a statue in Hakodate, Japan
Posts: 741
I am trying to substitute Overlay for ex_blend in my RescueFrame script, and regardless of whether I use 8 bits or not, I am getting the following error message:

"Expr: All inputs must have the same number of planes and the same dimensions, subsampling included."

If I use the only ConverttoYv12 that works I only get a black image.
__________________
By law and justice!

GMJCZP's Arsenal
GMJCZP is offline   Reply With Quote
Old 28th May 2021, 14:40   #23  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,338
@pinterf: mt_hysteresis was one I doubted to include or not. I will have a look.

@GMJCZP: I checked it but what is it blending? it looks like blending with opacity of 1.0, basically frame replacing. There are tools already for that like ReplaceFramesSimple or Prune by StainlessS. ex_blend needs clips a and b to be the same format or it will error out.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread
Dogway is offline   Reply With Quote
Old 28th May 2021, 18:04   #24  |  Link
GMJCZP
Registered User
 
GMJCZP's Avatar
 
Join Date: Apr 2010
Location: I have a statue in Hakodate, Japan
Posts: 741
Don't Overlay and ex_blend have identical behavior?
__________________
By law and justice!

GMJCZP's Arsenal
GMJCZP is offline   Reply With Quote
Old 30th May 2021, 19:49   #25  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,338
Updated ExTools to improve performance for ex_xxpand() by decoupling the thresholding (now just 6% shy from mt_expand() and with only SSSE3!!!) .
Also improved ex_blur() performance when rad>1 by precomputing final pixel weights.
Finally added the LUT expr family for easier porting the code and UV plane handling.

I also had time to sanitize SMDegrain, still not complete sanitation but near (v3.2.0d RC1). Removed old avs, Dither and YUY2 support. Ported masktools2 to ExTools in a "MIX" fashion and fixed a few small issues. Finally updated code to new avs+ grammar and properly formatted it.

By the way, Transforms Pack is also near to v1.0 final, it's advancing nice, 32-bit support, 2020CL and NCL conversion, and bugfixes.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread

Last edited by Dogway; 30th May 2021 at 19:51.
Dogway is offline   Reply With Quote
Old 31st May 2021, 00:09   #26  |  Link
kedautinh12
Registered User
 
Join Date: Jan 2018
Posts: 2,082
Thanks
kedautinh12 is offline   Reply With Quote
Old 31st May 2021, 09:45   #27  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,439
Quote:
Originally Posted by Dogway View Post
I also had time to sanitize SMDegrain, still not complete sanitation but near (v3.2.0d RC1). Removed old avs, Dither and YUY2 support. Ported masktools2 to ExTools in a "MIX" fashion and fixed a few small issues. Finally updated code to new avs+ grammar and properly formatted it.
Wow!

Considering it is just the script I use plain always, big thanks!
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 31st May 2021, 13:17   #28  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,338
Yes, it should be a bit faster. In my tests (check post#2) it performs 0.150fps better which translates into shaving 30 mins in encoding speed for a full feature. In any case the major bottleneck in SMDegrain is mvtools itself. I also plan to add an UHD mode, to try to make it faster in such cases.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread
Dogway is offline   Reply With Quote
Old 31st May 2021, 14:33   #29  |  Link
GMJCZP
Registered User
 
GMJCZP's Avatar
 
Join Date: Apr 2010
Location: I have a statue in Hakodate, Japan
Posts: 741
Quote:
Originally Posted by GMJCZP View Post
Don't Overlay and ex_blend have identical behavior?
Your function does not accept ConvertToYuv24(), which does accept Overlay, and could help mitigate at least the effects of conversions.
__________________
By law and justice!

GMJCZP's Arsenal
GMJCZP is offline   Reply With Quote
Old 31st May 2021, 15:08   #30  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,338
Sorry I don't understand well. It works with YV24 on my side, how is your script?
It's not a 1:1 port of Overlay(), my blending modes matches Photoshop's.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread
Dogway is offline   Reply With Quote
Old 31st May 2021, 16:48   #31  |  Link
GMJCZP
Registered User
 
GMJCZP's Avatar
 
Join Date: Apr 2010
Location: I have a statue in Hakodate, Japan
Posts: 741
Quote:
Originally Posted by Dogway View Post
Sorry I don't understand well. It works with YV24 on my side, how is your script?
In my signature.

Quote:
Originally Posted by Dogway View Post
It's not a 1:1 port of Overlay(), my blending modes matches Photoshop's.
Therein lies the problem, I cannot achieve 100% emulation. RescueFrame calls Overlay to replace one or a group of frames with an image and so far I have not succeeded with Ex_blend.
__________________
By law and justice!

GMJCZP's Arsenal
GMJCZP is offline   Reply With Quote
Old 2nd June 2021, 10:57   #32  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,439
Quote:
Originally Posted by Dogway View Post
In any case the major bottleneck in SMDegrain is mvtools itself. I also plan to add an UHD mode, to try to make it faster in such cases.
As I wrote in other thread, I dream of MVTools2 on CUDA.
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 2nd June 2021, 16:48   #33  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,439
Quote:
Originally Posted by Dogway View Post
Yes, it should be a bit faster.
My current syntax with realfinder version is

SMDegrain (tr=4, thSAD=400, refinemotion=false, n16_out=true, mode=0, contrasharp=false, PreFilter=4, truemotion=true, plane=4, chroma=true)

to have a 16 bit output from 8 bit input.

Do you plan to introduce a similar flag or is it sufficient to delete it and add ConvertBits(16) before SMDegrain?

There is mode=0 too, about dithering, that is useless if working with a 16 bit chain. Have I to ignore it or simply use -1 as argument?
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 2nd June 2021, 17:16   #34  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,338
Yes, ConvertBits(16) beforehand for 16-bits.

I had a look at SVPFlow but it only supports 1 frame delta, also it was slower in my tests... Maybe when mvtools2 from Neo gets ported back, but pinterf said he isn't doing it so someone else needs to be up for the task.

I'm currently working on further SMDegrain cleanup, in my internal version mode is removed also, might upload today or tomorrow since I added quite a few changes.


On another note, I have been advancing with Transforms Pack, now in RC8. Internally I also added linear_in and linear_out arguments (when dealing with linear rgb image sequences for example) and a convert format only option, so you can convert from YUV420 to YV444 without any gamma nor gamut processing, it also uses a lazy mode input, so easy to use.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread
Dogway is offline   Reply With Quote
Old 2nd June 2021, 17:31   #35  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,439
Quote:
Originally Posted by Dogway View Post
Yes, ConvertBits(16) beforehand for 16-bits.
Would it be possible to introduce some depan/stabilize while denoising? We already have some vectors ready, perhaps it could be almost computation free.
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 2nd June 2021, 17:58   #36  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,338
Normally you want to denoise after stabilization not the other way around, in any case Stab() uses DepanEstimate for the vectors. What are you using that uses MAnalyse mv?
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread
Dogway is offline   Reply With Quote
Old 2nd June 2021, 18:22   #37  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,439
Quote:
Originally Posted by Dogway View Post
in any case Stab() uses DepanEstimate for the vectors. What are you using that uses MAnalyse mv?
Nothing, I think

I did some real world benchmarks.

SMDegrain-3.1.2111~realfinder.avsi with script:
PHP Code:
SetMemoryMax()
SetFilterMTMode("DEFAULT_MT_MODE"2)
LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource("F:\In\2_00 13 assassini\assassini.dgi",ct=140,cb=140,cl=0,cr=0)
CompTest24(1)
SMDegrain (tr=4thSAD=400refinemotion=falsen16_out=truemode=0contrasharp=falsePreFilter=4truemotion=trueplane=4chroma=true)
Prefetch(6
SMDegrain-3.2.0d~Dogway.avsi with script:
PHP Code:
SetMemoryMax()
SetFilterMTMode("DEFAULT_MT_MODE"2)
LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource("F:\In\2_00 13 assassini\assassini.dgi",ct=140,cb=140,cl=0,cr=0)
ConvertBits(16)
CompTest24(1)
SMDegrain (tr=4thSAD=400refinemotion=falsecontrasharp=falsePreFilter=4truemotion=trueplane=4chroma=true)
Prefetch(6
x264.exe --crf 20 --preset slow --aq-mode 2 --colorprim bt709 --colormatrix bt709 --transfer bt709 --range tv --input-depth 16 --output "F:\In\2_00 13 assassini\assassini_4_400_temp\assassini_4_400_out.h264" "F:\In\2_00 13 assassini\assassini_4_400_temp\assassini_4_400.avs"

realfinder: encoded 3624 frames, 10.39 fps, 2904.75 kb/s, 50.20 MB
Dogway: encoded 3624 frames, 13.36 fps, 2903.99 kb/s, 50.18 MB

x265.exe --crf 22 --preset slow --output-depth 10 --colorprim bt709 --colormatrix bt709 --transfer bt709 --range limited --input-depth 16 --dither --output "F:\In\2_00 13 assassini\assassini_4_400_temp\assassini_4_400_out.hevc" "F:\In\2_00 13 assassini\assassini_4_400_temp\assassini_4_400.avs"

realfinder: encoded 3624 frames in 968.31s (3.74 fps), 1740.06 kb/s, Avg QP:25.89
Dogway: encoded 3624 frames in 986.45s (3.67 fps), 1739.71 kb/s, Avg QP:25.90

The speed increase in x264 encoding is massive. I hope there aren't some tradeoffs.
__________________
@turment on Telegram

Last edited by tormento; 2nd June 2021 at 18:26.
tormento is offline   Reply With Quote
Old 2nd June 2021, 18:49   #38  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,338
Wow 13fps, what CPU do you have? I always use Refinemotion though, would also like to have a variable blocksize option in mvtools like in BM3D.
Normally I see a 4% to 5% speed increase, I swapped masktools2 calls with Expr() based ex_tools calls which run faster for HBD, also replaced removegrain blurs with faster alternatives, and process the motion vectors in 8-bit. And expression optimization in luma_rebuild.

Currently I'm trying to optimize for UHD by serving a half size pelclip to MSuper, for that I'm setting pel=1, it runs but I'm not sure if this is correct code-wise maybe if someone can confirm.
Also found that SimpleResize is faster than bilinear for downscaling, so I might use that but doesn't support HBD though.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread

Last edited by Dogway; 2nd June 2021 at 22:03.
Dogway is offline   Reply With Quote
Old 3rd June 2021, 10:03   #39  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,439
Quote:
Originally Posted by Dogway View Post
Wow 13fps, what CPU do you have?
i7-2600k @ 4.6 GHz

I have found that working with 16bit in AVS+ script and leaving all the dithering to x264 (proper HBD aware build from jpsdr) gives a lot of speed increase and better visual results too.

Unfortunately my CPU hasn't enough cores to use AVS+ script plus x265 efficiently.

Can you confirm me that your revised version of SMDegrain is bitdepth transparent, i.e. the output has the same bitplane of the input at any stage of the avsi?
__________________
@turment on Telegram

Last edited by tormento; 3rd June 2021 at 10:06.
tormento is offline   Reply With Quote
Old 10th June 2021, 20:24   #40  |  Link
Dogway
Registered User
 
Join Date: Nov 2009
Posts: 2,338
Ok, seems that converting vector bitdepth back to input bitdepth was strangely not ideal (defaults back to mode=1). In any case client filters can work with vectors of different bitdepth as declared in 2.7.25 of the changelog and also first line of 2.7.19.22.

With the new mvtools2 version DCT is now much faster, so I defaulted DCT to 5, if everyone agrees.

@Tormento, only 8-bit is used for motion vectors, MDegrain uses a MSuper of original bitdepth, and everything else is processed in original bitdepth.
__________________
i7-4790K@Stock::GTX 1070] AviSynth+ filters and mods on GitHub + Discussion thread

Last edited by Dogway; 10th June 2021 at 20:36.
Dogway is offline   Reply With Quote
Reply

Tags
avisynth, dogway, filters, hbd, packs

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 08:44.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2023, vBulletin Solutions Inc.