Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 10th October 2017, 18:04   #3661  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,079
@davidhorman

See here for more info. http://avisynth.nl/index.php/Avisynthplus/Developers

There's a few post in this thread with a bit more info, unfortunately I'm away from my home computer ATM.
Reel.Deel is offline   Reply With Quote
Old 10th October 2017, 18:39   #3662  |  Link
davidhorman
I'm the Doctor
 
Join Date: Jan 2004
Posts: 1,325
I've read that, but I'm still none-the-wiser as to what actually happens. I am 99% sure my current filter can never be MT-friendly, at least...
__________________
My AviSynth filters / I'm the Doctor
davidhorman is offline   Reply With Quote
Old 10th October 2017, 22:56   #3663  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 1,991
Quote:
Originally Posted by davidhorman View Post
I'm guessing that MT_NICE_FILTER requests multiple frames in multiple threads, but only one instance of the filter, and that MT_MULTI_INSTANCE makes multiple instances, so it's a bit like calling the same plugin multiple times on trimmed/selecteveryed subclips and then joining or interleaving them? Or does it crop frames up into smaller frames then stack them back together, or something? (probably not) And that MT_SERIALIZED is the same as plain old single-threaded AviSynth?
Yes, if you're using Prefetch(8), MT_NICE_FILTER has 1 instance and 8 threads calling that one instance. That means the code must be thread-safe and you can't have class-level writable variables and buffers. Any work buffer must be initiated and destroyed within each GetFrame call.

MT_MULTI_INSTANCE creates 8 instances, and I'm not sure the thread creating the class is always the same as the thread calling GetFrame, in fact I think it can be any thread but always only 1 call at once.

MT_SERIALIZED, it will have 1 instance and only 1 call at once. However, there is currently a bug in the latest AVS+ where it can actually receive 2 calls at once in certain cases. Pinterf is working on fixing that, but for now, MT_SERIALIZED is broken.
MysteryX is offline   Reply With Quote
Old 11th October 2017, 13:13   #3664  |  Link
davidhorman
I'm the Doctor
 
Join Date: Jan 2004
Posts: 1,325
Quote:
but for now, MT_SERIALIZED is broken.
Which is the only one my filter can work under. Oh well! It's working for now on whatever version of AVS+ I'm using.
__________________
My AviSynth filters / I'm the Doctor
davidhorman is offline   Reply With Quote
Old 11th October 2017, 14:29   #3665  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 753
Quote:
Originally Posted by MysteryX View Post
MT_SERIALIZED, it will have 1 instance and only 1 call at once. However, there is currently a bug in the latest AVS+ where it can actually receive 2 calls at once in certain cases. Pinterf is working on fixing that, but for now, MT_SERIALIZED is broken.
It's not broken, fixed since r2502

Code:
20170602 r2502
  - fix: (Important!) MT_SERIALIZED mode did not always protect filters (regression since r2069)
    Such filters sometimes were called in a reentrant way (like being MT_NICE_FILTER), which
    possibly resulted in using their internal buffers parallel.
[...]
pinterf is offline   Reply With Quote
Old 11th October 2017, 14:42   #3666  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 753
Quote:
Originally Posted by raffriff42 View Post
It does look like a bug. The problem seems to be in ConvertTo8bit; its output is aprox. 1 step darker after each pass, probably a rounding issue.
While YUV 8<->16 conversion is a simple bit-shift, RGB conversion is full-scale, e.g. 8->16 bit conversion is something like x*65535/255. 255 becomes 65535. Similarly for 16->8 bits the formula is x*255/65535. Anyway, I'll check it (Sorry, I still have limited time for a couple of weeks).

Remark: do not worry about rounding errors in 8 bit conversions and in 8 bit filters in general. Most of the 8 bit functions work internally with 13-15 bits precision (like in classic Avisynth)
pinterf is offline   Reply With Quote
Old 11th October 2017, 18:09   #3667  |  Link
bxyhxyh
Registered User
 
Join Date: Dec 2011
Posts: 333
Quote:
Originally Posted by pinterf View Post
While YUV 8<->16 conversion is a simple bit-shift, RGB conversion is full-scale, e.g. 8->16 bit conversion is something like x*65535/255. 255 becomes 65535. Similarly for 16->8 bits the formula is x*255/65535. Anyway, I'll check it (Sorry, I still have limited time for a couple of weeks).
Is it bit-shifting or that (something like x*65535/255) arithmetic formula?
Bit shifting wouldn't work correct since 255 bit-shifted is not 65535.
But in avisynth they are equal white.

I'm pretty sure you know it, just pointing to it.
bxyhxyh is offline   Reply With Quote
Old 11th October 2017, 18:48   #3668  |  Link
davidhorman
I'm the Doctor
 
Join Date: Jan 2004
Posts: 1,325
Quote:
Is it bit-shifting or that (something like x*65535/255) arithmetic formula?
You can bitshift, and then OR in the original bits as well. It would be equivalent to the arithmetic formula (since 65535/255 = 257).

Code:
(x << 8) | x
Where does the convention that only a bitshift is used for YUV come from?
__________________
My AviSynth filters / I'm the Doctor
davidhorman is offline   Reply With Quote
Old 12th October 2017, 11:39   #3669  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 962
Quote:
Originally Posted by davidhorman View Post
Can someone give me a dumbed-down summary of the different MT modes? I've read this:

http://avisynth.nl/index.php/AviSynt...orrect_MT_mode

But it's not very clear to me. It doesn't really state how the modes work, and especially the bit for MT_NICE_FILTER isn't clear at all on when it should be used, instead focusing on when it shouldn't.

I'm guessing that MT_NICE_FILTER requests multiple frames in multiple threads, but only one instance of the filter, and that MT_MULTI_INSTANCE makes multiple instances, so it's a bit like calling the same plugin multiple times on trimmed/selecteveryed subclips and then joining or interleaving them? Or does it crop frames up into smaller frames then stack them back together, or something? (probably not) And that MT_SERIALIZED is the same as plain old single-threaded AviSynth?
I wrote this a while ago. May or may not help.

The tl;dr is that as a rule of thumb:
  • if your filter's GetFrame is threadsafe (basically, does not attempt to write to any memory that is not explicitly associated with this specific frame request) => MT_NICE_FILTER
  • if your GetFrame does depend on shared state of some kind, but your filter supports things like interleave(yourfilter().selecteven(), yourfilter().selectodd()) in vanilla Avisynth => MT_MULTI_INSTANCE
  • if you are a source filter, or rely on frame request order (i.e. stuff with side effects outside Avisynth such as writing to a file), or rely on internal state that changes with every frame request (i.e. pattern-tracking decimation filters) => MT_SERIALIZED

It also may or may not be relevant to note that all GetFrame calls that need to happen to produce a given output frame at the end of the script happen in the same single thread, regardless of what MT modes are in use. You probably shouldn't rely on this for anything, though.

Last edited by TheFluff; 12th October 2017 at 11:46.
TheFluff is offline   Reply With Quote
Old 12th October 2017, 14:20   #3670  |  Link
davidhorman
I'm the Doctor
 
Join Date: Jan 2004
Posts: 1,325
Thanks TheFluff - that helps. I'm certain my filter will never be able to be MT'd. It's internally multi-threaded though.
__________________
My AviSynth filters / I'm the Doctor
davidhorman is offline   Reply With Quote
Old 18th October 2017, 15:37   #3671  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 1,597
Is the avs+ api stable now? I really don't want to waste time adding compatibility for it if I'm going to have to change things later.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 18th October 2017, 15:54   #3672  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 753
I'm not planning to change it.
pinterf is offline   Reply With Quote
Old 18th October 2017, 16:11   #3673  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 799
Quote:
Originally Posted by pinterf View Post
I'm not planning to change it.
what about add support for vs api in avs+ to load vs plugins (dll) in avs+?

We discussed this in irc years ago and Myrsloik said it's possible
__________________
My Avisynth Stuff
real.finder is offline   Reply With Quote
Old 18th October 2017, 16:13   #3674  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 753
Short developer news, I hope I'll have less busy months from now (finished a marathon with 2:55, this year I was preparing on this race instead of coding)

- "Levels" now allows 32 bit float inputs
- today I have successfully ported Expr filter from the VapourSynth project, with some additions (masktools syntax of built-in constants and some of the scaling helper functions).
pinterf is offline   Reply With Quote
Old 18th October 2017, 16:18   #3675  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 753
Quote:
Originally Posted by real.finder View Post
what about add support for vs api in avs+ to load vs plugins (dll) in avs+?

We discussed this in irc years ago and Myrsloik said it's possible
If he says it's possible then it's the question of someone's time isn't it? Not planning in the near future either, to tell the truth I cannot imagine the magnitude of the effort at the moment.
pinterf is offline   Reply With Quote
Old 19th October 2017, 11:07   #3676  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 799
Quote:
Originally Posted by pinterf View Post
Short developer news, I hope I'll have less busy months from now (finished a marathon with 2:55, this year I was preparing on this race instead of coding)

- "Levels" now allows 32 bit float inputs
- today I have successfully ported Expr filter from the VapourSynth project, with some additions (masktools syntax of built-in constants and some of the scaling helper functions).
good news

Expr will be part of internal avs+ functions?
__________________
My Avisynth Stuff
real.finder is offline   Reply With Quote
Old 19th October 2017, 11:31   #3677  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 753
Quote:
Originally Posted by real.finder View Post
good news

Expr will be part of internal avs+ functions?
Yes, it was more convenient and faster for me than putting it in a separate dll (masktools).
pinterf is offline   Reply With Quote
Old 19th October 2017, 20:00   #3678  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 1,991
and Expr really has nothing to do with masks, it's more of a core feature
MysteryX is offline   Reply With Quote
Old 21st October 2017, 16:59   #3679  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 1,597
What's the proper way to detect if a plugin is compiled for the avs 2.6 or avs+ api? By whether or not it calls SetFilterMTMode() on init? Is there some other way? I want to make sure 2.6 plugins don't get exposed to avs+ only formats.

And speaking of formats... are only formats with a pre-combined constant allowed or can I have some 16bit yuv with insanely high subsampling?

Btw, do any filters have meaningful planar YUVA or RGBA support or can I skip implementing it properly for now?

Oh, and is the threadpool and jobcompletion stuff actually documented somewhere?
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet

Last edited by Myrsloik; 21st October 2017 at 17:02.
Myrsloik is offline   Reply With Quote
Old 21st October 2017, 19:17   #3680  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 167
Maybe AVSMeter (src included) can help you for the detection part: https://forum.doom9.org/showthread.php?t=174797
__________________
Search and denoise

Last edited by ChaosKing; 21st October 2017 at 19:22.
ChaosKing is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 13:41.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2017, vBulletin Solutions Inc.