Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 30th November 2013, 13:49   #1  |  Link
TurboPascal7
Registered User
 
TurboPascal7's Avatar
 
Join Date: Jan 2010
Posts: 270
Avisynth+ plugin modernization efforts

As you might know, a part of Avisynth+ core team is working on plugins modernization, adding x64 support and other features. This thread is dedicated to discussion of those plugins. Some new plugins are also included.

General changes/rules
MMX/ISSE optimization paths are removed everywhere, replaced with SSE2/newer routines. C versions are added too unless specified otherwise.

YUY2 support is also removed if any work is required to support it. You can still use these plugins by converting your YUY2 clip to YV16, processing it with planar-only filters and converting back to YUY2 if you need. This conversion is lossless and fast.

Right now both Avisynth+ and Avisynth 2.6 are supported. Avisynth 2.6 support will be dropped at some point when Avs+ specific features are added - multithreading and >8bit colorspaces being obvious examples. Avisynth 2.5.8 is not supported.

All plugins are dynamically linked to vc110 runtime. Avisynth+ installer provides it for you.

VapourSynth, maybe?
Right now the answer is no. VapourSynth support is considered and might be added at some point in the future. For now - feel free to fork/take the code from anything you want, most plugins are/will be licensed under MIT.

How can I help?
Usual stuff - testing, discussing, updating some plugins that aren't included yet. Feel free to join our IRC channel for faster communication. Writing documentation would be very helpful too, especially for RgTools.

What's next?
Current "roadmap" can be found here. It's somewhat messy and does not mean that these plugins will be ported in any particular order, but it does show what's considered right now and what's not.


Plugins listed in random order. Only those hosted under the AviSynth organization on GitHub are included.

Vinverse
Source / binaries. Opt parameter is removed. New Vinverse2 function provides a fast implementation of this script.

TMaskCleaner
Source / binaries. A really simple plugin, never released before. It is basically mt_hysteresis with all areas less than length pixel discarded. Useful for some mask cleaning.

TColorMask
Source / binaries. A really fast implementation of ColorKeyMask for YUV colorspaces. Never released before.

SangNom2
Source / binaries / thread. Same "old" SangNom2. Additional info can be found in its thread. Does not have a C version yet.

MSharpen
Source / binaries. Same old MSharpen. Output for planar colorspaces is not exactly identical because of a bit different blurring routine. YUY2 support removed, all planar routines optimized for SSE2, a few old bugs fixed. External file configuration is no longer supported.

RgTools
Source / binaries. Replacement of some kassandro's plugins. 24 modes of Removegrain, 24 modes of Repair, both modes of VerticalCleaner and all variations of Clense are implemented. No more dlls mess - a single dll with dynamic dispatch is provided. Some routines might be slightly less efficient than original, some are faster. Output of some modes of RemoveGrain is not exactly identical to the original because of some rounding difference. Other functions should be identical.

Deblock
Source / binaries. Simple deblocking filter. No significant changes, simple refactoring. Should still be noticeably faster thanks to better compilers. YUY2 support removed, useless mmx and isse parameters too. This filter is licensed under GPL.

MedianBlur2
Source / binaries. Simple median filter similar to MedianBlur (function name is still MedianBlur). Theoretical runtime complexity per pixel is constant, meaning that the filter has same performance on any radius. Additional info here.

Checkmate
Source / binaries. Update of the old checkmate plugin, used for dot crawl removal. Better performance, less bugs, a bit lower precision. The difference is hardly noticeable when you're looking for it.

Masktools2
Source / binaries. A fork of masktools2. The only plugin compatible with avisynth 2.5.8 for now. More info in this post.

Average
Source / binaries. The good old Average plugin but a bit faster and with some additional sanity checks.

Last edited by TurboPascal7; 21st December 2013 at 15:14. Reason: Average filter added
TurboPascal7 is offline   Reply With Quote
Old 30th November 2013, 13:49   #2  |  Link
TurboPascal7
Registered User
 
TurboPascal7's Avatar
 
Join Date: Jan 2010
Posts: 270
Reserved
__________________
Me on GitHub | AviSynth+ - the (dead) future of AviSynth
TurboPascal7 is offline   Reply With Quote
Old 30th November 2013, 13:50   #3  |  Link
innocenat
Registered User
 
innocenat's Avatar
 
Join Date: Dec 2011
Posts: 77
As mentioned in roadmap above, I am currently working on tivtc/tdeint port.

Last edited by innocenat; 30th November 2013 at 13:53.
innocenat is offline   Reply With Quote
Old 30th November 2013, 14:56   #4  |  Link
Guest
Guest
 
Join Date: Jan 2002
Posts: 21,923
Quote:
MSharpen
Source / binaries. Same old MSharpen. Output for planar colorspaces is not exactly identical because of a bit different blurring routine. YUY2 support removed, all planar routines optimized for SSE2, a few old bugs fixed. External file configuration is no longer supported. The only plugin currently licensed under GPL, but that will most likely change in the near future.
I can't see any reference/credit to the original author. Do you think that is appropriate? And for the other plugins?

Last edited by Guest; 30th November 2013 at 15:01.
Guest is offline   Reply With Quote
Old 30th November 2013, 15:06   #5  |  Link
TurboPascal7
Registered User
 
TurboPascal7's Avatar
 
Join Date: Jan 2010
Posts: 270
Quote:
Originally Posted by neuron2 View Post
I can't see any reference/credit to the original author. Do you think that is appropriate? And for the other plugins?
It will be added in the readme in a few days.
EDIT: here, done for msharpen.
__________________
Me on GitHub | AviSynth+ - the (dead) future of AviSynth

Last edited by TurboPascal7; 30th November 2013 at 15:09.
TurboPascal7 is offline   Reply With Quote
Old 30th November 2013, 15:45   #6  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 1,045
I downloaded all of them, I'll try them nearest opportunity

Waiting for the rest

Thank you
real.finder is offline   Reply With Quote
Old 30th November 2013, 17:04   #7  |  Link
Keiyakusha
契約者
 
Keiyakusha's Avatar
 
Join Date: Jun 2008
Posts: 1,567
RgTools breaks existing scripts
Avisynth open failure:
Script error: RemoveGrain does not have a named argument "planar"
Keiyakusha is offline   Reply With Quote
Old 30th November 2013, 21:36   #8  |  Link
TurboPascal7
Registered User
 
TurboPascal7's Avatar
 
Join Date: Jan 2010
Posts: 270
Quote:
Originally Posted by Keiyakusha View Post
RgTools breaks existing scripts
Avisynth open failure:
Script error: RemoveGrain does not have a named argument "planar"
Yes, that parameter is removed because it does not make sense.
__________________
Me on GitHub | AviSynth+ - the (dead) future of AviSynth
TurboPascal7 is offline   Reply With Quote
Old 30th November 2013, 22:14   #9  |  Link
Keiyakusha
契約者
 
Keiyakusha's Avatar
 
Join Date: Jun 2008
Posts: 1,567
Quote:
Originally Posted by TurboPascal7 View Post
Yes, that parameter is removed because it does not make sense.
Okay, I'll stick to kassandro's plugins.
Keiyakusha is offline   Reply With Quote
Old 30th November 2013, 22:15   #10  |  Link
TurboPascal7
Registered User
 
TurboPascal7's Avatar
 
Join Date: Jan 2010
Posts: 270
Quote:
Originally Posted by Keiyakusha View Post
Okay, I'll stick to kassandro's plugins.
Just for reference: can you provide an example where this parameter is actually useful?
__________________
Me on GitHub | AviSynth+ - the (dead) future of AviSynth
TurboPascal7 is offline   Reply With Quote
Old 30th November 2013, 22:18   #11  |  Link
Keiyakusha
契約者
 
Keiyakusha's Avatar
 
Join Date: Jun 2008
Posts: 1,567
Quote:
Originally Posted by TurboPascal7 View Post
Just for reference: can you provide an example where this parameter is actually useful?
No. Until now I never knew this parameter existed at all and can only guess what it does by it's name. Perhaps those who use it in complex scripts can answer that.
If you want some example, I guess QTGMC will be one of the more or less known ones.

Last edited by Keiyakusha; 30th November 2013 at 22:26.
Keiyakusha is offline   Reply With Quote
Old 30th November 2013, 22:28   #12  |  Link
Sapo84
Registered User
 
Join Date: May 2008
Posts: 41
Quote:
Now, in order to prohibit unexperienced users from applying the above filters to interleaved color spaces, we have added the boolean variable planar to RemoveGrain, Repair and TemporalRepair. If planar=false (the default value), then these filters reject YUY2, RGB24 and RGB32 frames. Only if planar=true, then these filters accept YUY2, RGB24 and RGB32 input, but always assume that the data are organised in a planar way. If you use planar=true with ordinary interleaved input, you get garbage.
I'd say it's never actually useful and I agree with tp7 decision of removing it entirely.
If I set planar=false I'm expecting the filter to work with non planar color spaces, not rejecting YUY2, RGB24 and RGB32.
Sapo84 is offline   Reply With Quote
Old 30th November 2013, 22:28   #13  |  Link
Stephen R. Savage
Registered User
 
Stephen R. Savage's Avatar
 
Join Date: Nov 2009
Posts: 337
"Planar" is required to work on YUY2 inputs. You repack the image to YV16-in-YUY2 by using the Interleaved2Planar function. MaskTools also has support for this hack. In AVS 2.6, this hack should be replaced with YV16.
Stephen R. Savage is offline   Reply With Quote
Old 30th November 2013, 22:52   #14  |  Link
TurboPascal7
Registered User
 
TurboPascal7's Avatar
 
Join Date: Jan 2010
Posts: 270
Mkay, I just released a new version of RgTools. Planar parameter is added, should work as before now. You should feel really bad if you use this feature though and I do hope to remove it at some point in the future.
__________________
Me on GitHub | AviSynth+ - the (dead) future of AviSynth
TurboPascal7 is offline   Reply With Quote
Old 1st December 2013, 01:07   #15  |  Link
Guest
Guest
 
Join Date: Jan 2002
Posts: 21,923
Quote:
Originally Posted by TurboPascal7 View Post
MSharpen ... The only plugin currently licensed under GPL, but that will most likely change in the near future.
Please feel free to re-license it with any suitable less restrictive license.
Guest is offline   Reply With Quote
Old 1st December 2013, 02:49   #16  |  Link
TurboPascal7
Registered User
 
TurboPascal7's Avatar
 
Join Date: Jan 2010
Posts: 270
Quote:
Originally Posted by neuron2 View Post
Please feel free to re-license it with any suitable less restrictive license.
Thanks.
__________________
Me on GitHub | AviSynth+ - the (dead) future of AviSynth
TurboPascal7 is offline   Reply With Quote
Old 4th December 2013, 01:25   #17  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,381
Nooooo, now how am I going to use my dusty old Athlon XP without ISSE!? Haha.

Thank you so much for rewriting RgTools. It's always been a bit of a pain setting up the whole suite of DLLs since the 1.0pre split. It looks like the T versions aren't included, is that on the roadmap? With any luck a rewrite might not require avsrecursion functionality.
__________________
There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order. ~ Ed Howdershelt
foxyshadis is offline   Reply With Quote
Old 4th December 2013, 01:30   #18  |  Link
TurboPascal7
Registered User
 
TurboPascal7's Avatar
 
Join Date: Jan 2010
Posts: 270
Quote:
Originally Posted by foxyshadis View Post
It looks like the T versions aren't included, is that on the roadmap? With any luck a rewrite might not require avsrecursion functionality.
Not right now. Kassandro's stuff takes a lot of effort and time since it's essentially reverse-engineering and I'd like to get some other more useful plugins done first.
__________________
Me on GitHub | AviSynth+ - the (dead) future of AviSynth
TurboPascal7 is offline   Reply With Quote
Old 8th December 2013, 08:33   #19  |  Link
TurboPascal7
Registered User
 
TurboPascal7's Avatar
 
Join Date: Jan 2010
Posts: 270
New filter: MedianBlur2 (function is called MedianBlur).
It's really similar to the old MedianBlur. Has only 3 arguments: radii of Y, U and V plane filters. No other functions from original MedianBlur are implemented and most likely won't. Maximum radius is limited to 127. It can be changed but I'm not sure anyone uses too large medians.

Unlike the old MedianBlur, this implementation has constant runtime complexity, meaning that theoretical performance is same for any radius. Implementation is based on this paper, but includes some additional optimizations.
1) When SSE2 is available and radius is 1 or 2, special routines are used. They're a lot faster than the generic one for these radii.
2) For 2 < radius < 8, generic approach with 8-bit bin size is used. Using 8-bit bins makes histogram addition/subtraction faster with SSE2.
3) For large radii, 16-bit bins are used, as described in the paper.

In other words, you can expect huge performance drop when going from 1 to 2, not so huge but still large from 2 to 3 and a noticeable slowdown from 7 to 8. Between them the fps should be constant and it actually might get a bit faster with larger radius.
Performance with radius > 2 also depends on the actual frame content. Processing colorbars() is a lot faster than addgrainc(10000).
It is quite possible that you'll get worse performance for radius = 3 with this plugin compared to the original MedianBlur. All other radii should be (a lot) faster.

P.S. A few days ago I also added Deblock but forgot to mention it. No significant changes in there though.
__________________
Me on GitHub | AviSynth+ - the (dead) future of AviSynth

Last edited by TurboPascal7; 8th December 2013 at 10:09.
TurboPascal7 is offline   Reply With Quote
Old 9th December 2013, 12:52   #20  |  Link
TurboPascal7
Registered User
 
TurboPascal7's Avatar
 
Join Date: Jan 2010
Posts: 270
A small update to MedianBlur2: new function called MedianBlurTemp. Requested by Tempter57.

It partially implements the old MedianBlurT function but doesn't have MVTools-related parameters (hance the function name change). If you don't use mo-comp part of it, you should be able to just change the name used without any other changes. I might implement MedianBlurT as a script wrapper around MVTools/Masktools/MedianBlurTemp later.

Please do note that for temporal-only processing this filter is extremely inefficient (just like the original). For any reasonable radius (<= 15) pure-temporal median can be calculated a lot faster in SIMD. If the demand for a function like this is high, I might add it later.

Meanings of some parameters between two modes are different, just like they were in original filter. Setting spatial radius to 0 will just disable spatial filtering, setting it to -1 will copy the plane and all other values will set the plane to absolute value of radius. Yes, there is no way to set the plane to 1.

Two questions:
1) I don't really like the MedianBlurTemp name. Any better ideas? MedianBlurT is a no-go because of the reasons described above.
2) Better interface ideas? I.e. something that would allow you to set the plane to 1 and maybe be more consistent with the pure-spatial version of the filter.

Get the binaries in the usual place. And stay tuned.
__________________
Me on GitHub | AviSynth+ - the (dead) future of AviSynth

Last edited by TurboPascal7; 9th December 2013 at 12:55.
TurboPascal7 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 09:07.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.