Internaly multi-threaded resampling functions

[Groucho2004]

jpsdr, I sent you a PM.

[jpsdr]

I've seen, let me time to get back home... :p

[ultim]

My next next step, creating a plugin package with my 3 filters, but the filters in standalone version will continue and will not be dropped.


If you are so eager to add internal multithreading to all your plugins, you might really try IScriptEnv2 now. Yes if it breaks you'll have to recompile, but if you plan to switch to it later you'd have to recompile anyway+rewrite everything, you you might be better off to do it now. Users will have to upgrade both core and your plugins if that time comes. EDIT: I am planning to break the threadpool API at most once in the future, if at all, at the time it stabilizes. So you won't see repeated problems.

The basic usage is:
1) You call IScriptEnv2->NewCompletion() in your plugin constructor. The capacity arg is the number of jobs you'll enqueue at once in parallel.
2) In your GetFrame() you call IJobCompletion->Reset(), then call IScriptEnv2->ParallelJob() for each task you want to execute in the threadpool. The 1st arg is your function to execute, the 2nd is data that'll be passed to your function, and the 3rd is the compleiton object you created in step 1.
3) After all tasks are queued for a frame, you call IJobCompletion->Get to receive the result of the i'th queued job.
4) In your plugin destructor, you call IJobCompletion->destroy(). Do not call delete on this object yourself.

This will of course also reduce coding burden on you since proper thread-safe queueing, thread management, and in general everything associated with the thread pool will be managed and maintained by the core, and you won't have to maintain duplicate code for it in your plugin.

[ultim]

One more thing: if you use the core's threadpool API, then ofc you can keep your plugins in separate packages, and they will still share the same threadpool. So that is an upside too.

[jpsdr]

I'll try, out of curiosity to see if i see the performance drop i've noticed when i try to put the thread pool in an external DLL.
But it will complexify the code (a lot... not so much...?) to have in the same time my and your threadpool, because there is one thing not to forget : I want my plugins working multi-threaded an all avs+ and 2.6.x standard version, not only on the avs+ MT version.
But, i'll give a try, i'm curious...

[ultim]

Well, if you want to stay compatible with avs 2.6, than I can understand if you don't want to use avs+-specific APIs. May I ask the reasons for wanting to support 2.6? Is it a technical reason, like an area where avs+ doesn't work as great as 2.6 does, or is it simply because avs+ is not that commonly spread as 2.6 yet? I think the latter situation will improve once a new stable version of avs+ is released, which I'm planning on in the foreseeable future.

[jpsdr]

It's just my way... to want to be compatible with the most possible (reasonable) versions, if it's not something if think too hard. In fact, i often hate the opposite : being obliged to stick with a specific version, when i think it could have been otherwise just with a little work. (By reasonable i mean i've dropped 2.5.x ... :p ). No realy technical reasons, just a less restriction philosophy... ;)
And for those who still use 2.6.x for any reasons, they'll have something upgraded.
After, as i said, it's just my way, for now. What future will be...?
But, i'm still curious to give a try to your api, so, maybe not right now, but very soon, i'll made a specific version using it to make some tests.

[jpsdr]

New version, see first post.

[jpsdr]

Ooops... :eek:
Wait next release tomorrow, there is a tricky deadlock. It doens't show if you open a script with VDub and seek within it, it doesn't show if you run a script with avsmeter, but, it shoes if you're creating several jobs in batch in VDub and want to run them.
Fix is done, but it's late, so new builds an push on github tomorrow.

[jpsdr]

New version fixed, see first post.

[jpsdr]

New version, see 1rst post. I'll not say final, because there may be update in the functions of the core in the future i'll try to follow, but for now, i think i'm done with the threadpool part.

[jpsdr]

Never said "At last, final version"... :p
New version, see 1rst post.

[bilditup1]

With latest avs+ mt r2172 & your nnedi3 r26, using this yields a 1.5fps increase using this script (http://forum.doom9.org/showpost.php?p=1779369&postcount=2385) on a (for now) stock 4770K. Sometimes the encode appears to halt entirely but I'm not sure if this is an avs+ issue or a ResampleMT/nnedi3 issue.

[jpsdr]

New version, see first post.
I've added several parameter to allow to tune the created threadpool. I've made several testbench, and set the default settings correspond to the best result i have on my PC. But, it doesn't mean it will be also what produce the best result on others PC, so, the parameters added will allow to eventualy each one tune according his hardware configuration.

Check the ReadMe file for precise information.

[real.finder]

I did edit to ResizeX to make it work with this plugin

edit : add edi_rpow2_v1.0.zip and maa2 (http://forum.doom9.org/showpost.php?p=1725500&postcount=69)

new edit ResizeX (https://pastebin.com/Jkhvw9Yr) and edi_rpow2 (http://pastebin.com/jpYhcqee) will work in new avs+ colour format

edit: another ResizeX edit here https://forum.doom9.org/showpost.php?p=1823545&postcount=5 and use it with this https://pastebin.com/8kcg2MtJ edi_rpow2

[jpsdr]

New version (see 1rst post).
Can be used also with prefetch without issue, and can also be used with the last nnedi3 version and with prefetch also.

[jpsdr]

New version, see first post.

[real.finder]

I wonder how this filter work, it's depends on avs Resizers or have an edit of them?

I ask because I didn't see any mention on what colour format that this filter supports

[jpsdr]

It uses the code of the internal avs+ resamplers. About supported format, it supports all the formats supported by avs+ at the time the filter has been build.
I'm working on the new version to support the new formats, but for now it crashes in both avisynth and avs+ and i don't had time to investigate futher. But i'll probably be able to resume work on it soon.

[real.finder]

It uses the code of the internal avs+ resamplers. About supported format, it supports all the formats supported by avs+ at the time the filter has been build.
I'm working on the new version to support the new formats, but for now it crashes in both avisynth and avs+ and i don't had time to investigate futher. But i'll probably be able to resume work on it soon.

after I update to last ResampleMT and NNEDI3 from ResampleMT 1.0.2 and NNEDI3 0.9.4.24 I see speed lost, from 1.6 fps to 1.3 fps, is this because threadpool?

why not try using avstp?

and can z.lib (http://forum.doom9.org/showthread.php?t=173986) make it faster?

[jpsdr]

after I update to last ResampleMT and NNEDI3 from ResampleMT 1.0.2 and NNEDI3 0.9.4.24 I see speed lost, from 1.6 fps to 1.3 fps, is this because threadpool?

Honestly, i don't know, but it's not impossible, even more possible if you're also multi-threading again using prefetch. Is it the case ?
As there is only one threadpool shared instead of several (before, one was created for each instance filter) => result in slowdown when shared resources required.
For optimal, i suggest if you're also prefetching to reduce the number of threads and make tests. The values i suggest for testing are first threads=1, and second threads=CPU/2. And if you're using ResampleMT within NNEDI3, don't forget also to put the value on the threads_rs parameter.


why not try using avstp?

Not interested, and using an external dll for the threadpool make it even slower (i've tested with my threadpool).


and can z.lib (http://forum.doom9.org/showthread.php?t=173986) make it faster?

No idea. But if it uses directly a square kernell instead of doing two passes, multi-threading by splitting the image may be more difficult.

[jpsdr]

New version, check first post.

[jpsdr]

New version, check first post.

[jpsdr]

New version, check first post.

[real.finder]

range -
This parameter specify the range the output video data has to comply with.
Limited range is 16-235 for Y, 16-240 for U/V. Full range is 0-255 for all planes.
Alpha channel is not affected by this paramter, it's always full range.
Values are adjusted according bit depth of course. This parameter has no effect
for float datas.
0 : Automatic mode. If video is YUV mode is limited range, if video is RGB mode is
full range, if video is greyscale (Y/Y8) mode is Y limited range.
1 : Force full range whatever the video is.
2 : Force limited Y range for greyscale video (Y/Y8), limited range for YUV video,
no effect for RGB video.
3 : Force limited U/V range for greyscale video (Y/Y8), limited range for YUV video,
no effect for RGB video.

Default: 0


do we really need this? we can do it in avs Limiter if we have to, but having range in resize or nnedi make no sense and if someone work in full range will have to set it to 1 every time, and it's kinda annoying

[jpsdr]

I don't agree, standard YUV is not full range, so it results that nnedi and resize can produce incorrect value output. On the oposite, someone who want to work with standard value has to put a limiter...?
No, that's not a proper behavior for me, it's the contrary, you have to specify "something" when you want to work outside standard values.

[real.finder]

I don't agree, standard YUV is not full range, so it results that nnedi and resize can produce incorrect value output. On the oposite, someone who want to work with standard value has to put a limiter...?
No, that's not a proper behavior for me, it's the contrary, you have to specify "something" when you want to work outside standard values.

and what about scripts that do everything with Y8? at least you should make the default is 3 for Y/Y8 so that chroma will be correct ((if there is RGB support (by Y8 is scripts) then it will fall, but I don't care for RGB, if you care then Y/Y8 should have range default 1)), for both nnedi and this

edit: and scripts that did TV to PC range internally for some specific purpose like SMDegrain will fall too, so default should be 1 everywhere, that reminds me of keep_tv_range bool in flash3kyuu_deband, it's false by default

[jpsdr]

at least you should make the default is 3 for Y/Y8 so that chroma will be correct
But luma will not be, so...

edit: and scripts that did TV to PC range internally
A script/plugin can do whatever it wants internaly, it have to put back correct levels on its output, otherwise, it screws you things without telling you.

keep_tv_range bool in flash3kyuu_deband, it's false by default
Yes, it should be the oposite, when i use it on YUV data, i always have to put it to true.

[real.finder]

A script/plugin can do whatever it wants internaly, it have to put back correct levels on its output, otherwise, it screws you things without telling you.


it do tv to pc for motions search for example, your change will make that fall or useless

anyway, I will stick with the older versions, thank you for the old change efforts :) it was cool

[jpsdr]

it do tv to pc for motions search for example, your change will make that fall or useless

You just have now to specify that range is not anymore the standard range for the actual format, and it will not break anything.
Of course, if a metadata tag existed, it would have been perfect...

[real.finder]

You just have now to specify that range is not anymore the standard range for the actual format, and it will not break anything.
Of course, if a metadata tag existed, it would have been perfect...

for nnedi3, I don't have time for edit like 10 scripts just for that, and some of them still has 2.5 support (with 2.6 and plus) and that will make edit things way annoying, so I will pass

but if you kindly make range = 1 then I will taking the range into account for some scripts in future

[pinterf]

I don't think either that defaulting to 16-240 is a good idea, neither on YUV, especially nor for greyscale. There are many scripts out there that rely on the old behaviour and use clip converted at the beginning to PC range. Or use Y as an extracted channel from an RGB clip, these scripts (or plugins that invoke nnedi3_rpow2) may break now.
Many camcorders and video capable cameras now are using 16-255 that contain extra highlight information (superwhites) to be retreived later on processing. You can even display this range on a capable device (there are such projectors).
Now this extra highlight information is lost by the new defaults.
I think it is not the resizers task to police over the range. People just blindly download and use latest version and see that something went wrong.

[jpsdr]

I think it is not the resizers task to police over the range.
Personnaly, i think, not specificaly resizers, that any filter has to produce "in specs" output levels.
YUV "alone", without more indications or informations is by default/standard TV limited range, this is the proper behavior.
So, any filter processing YUV has to produce a proper "in spec" output.
You can have of course full range YUV, but it's not a standard behavior, so you have to specify that you're oustide the default behavior. Not the oposite, it's not when you're within the default/normal behavior that you have to specify it.
As there is not such information as the range in the avs "clip" (otherwise it could have been used), the standard prevail, and the standard for YUV for example, is that YUV is limited range.

Thanks for the 16-255 range, i didn't know about it, i'll add it in a future version.

[pinterf]

Just imagine what would happen if we had to provide every plugin and avisynth filter an extra parameter not to screw up the processing, removegrains, mvtools functions, any. I maintain that until the range cannot be extracted with 99.99% confidence from a clip or frame property, this auto-clamp feature is dangerous. Unless nnedi3 is a brand new filter that is not used widespreadly and behaves as such from the beginning.

[jpsdr]

I understand this point of view, i've thought about it before doing this.

But the only fact that with the actual behavior, something like this :

AVISource("My SD YV12 file",false,"YV12")
Spline36Resize(1280,720)

can produce out of spec/range output...
No... i don't like it.

And having a parameter to have to specify that your file is in the standard spec, and you want your output stay in standard spec...
Not again, i don't like it either.

I personnaly think that the actual behavior is wrong (and so my filters were wrong), i even didn't realise it until now, i'm a little pissed over myself of not noticing sooner something finaly so obvious, and i personnaly don't want to continue/keep this way.

I want that with my filters, a simple script like described before work properly when everything is within the standard specs, without having to tell when everything is "normal".
This is the behavior i'll stick with now.

[shekh]

Another possible convention is that all intermediate results are allowed to be out of range, this may simplify things (values must be clamped on output and before colorspace change, but not anywhere else).

[jpsdr]

Do you mean avisynth will automaticaly clamp values before outputing datas according the format of the video ? (Full range for RGB, limited for Y/YUV) ?

[shekh]

No, I don`t know current set of rules, just speculating.

[raffriff42]

Resampling/resizing is not affected by "range" at all, except in the case of overshoot. It should not affect a resizer AT ALL.

Handling overshoot without clipping is the whole point of so-called "limited range."
("so-called" because it actually stores a WIDER range of colors, unless it is misguidedly restricted to 16-235)

[pbristow]

Another possible convention is that all intermediate results are allowed to be out of range, this may simplify things (values must be clamped on output and before colorspace change, but not anywhere else).

Yes, this, please! It is crazy to place the burden of range-checking on every single filter in the chain, and crazy to expect novices have to figure out for each and every filter what option they should be choosing if they suspect the default might not be right. You're just adding needless complexity to the debugging chain.

Levels/ranges should only ever be policed explicitly, and at a place in the chain chosen by the user to suit their needs. They shouldn't have to work their way through the whole chain of filters trying to work out which one is messing up the range, and whether its because they've chosen the wrong option or because the filter itself has a bug!

What I'd much rather see is for someone to create a nice, user friendly "levels advisor" plugin. Something that looks at the incoming clip, checks the ranges of values, and says:
"This clip is using Y/U/V values above/below the standard YUV minimum/maximum. If you wish to store this data in a standard YUV format, consider using the Levels filter to either (a) re-scale the data to fit within range; (b) clip or "hard limit" the range. If you do not think the data should be exceeding the standard range at this point in your script, trying using LevelsAdvisor earlier in your script to trace where things are going wrong."

It should of course also display the actual maximum and minimum values found in each channel, and what the standard limits are. If the actual values are only ever one LSB out of range, that suggests trivial rounding errors are at fault, and hard limiting the range is the appropriate option. If the overshoot is bigger, that suggests re-scaling (or filter chain debugging) is needed.

[Wilbert]

Personnaly, i think, not specificaly resizers, that any filter has to produce "in specs" output levels.
YUV "alone", without more indications or informations is by default/standard TV limited range, this is the proper behavior.
So, any filter processing YUV has to produce a proper "in spec" output.
You can have of course full range YUV, but it's not a standard behavior, so you have to specify that you're oustide the default behavior. Not the oposite, it's not when you're within the default/normal behavior that you have to specify it.
As there is not such information as the range in the avs "clip" (otherwise it could have been used), the standard prevail, and the standard for YUV for example, is that YUV is limited range.

Thanks for the 16-255 range, i didn't know about it, i'll add it in a future version.
Like other people say it's a bad idea. Not only because you break backwards-compatibility, but also because there is no proper "in spec" output. Both are allowed in the h.264/h.265 specs for example and that's even within the same standard. Although one is more common than the other obviously.

[jpsdr]

For x264, the parameter is :

--input-range <string> Specify input color range ["auto"]
- auto, tv, pc

Unable to find what "auto" is doing. Maybe it changes the range "on the fly" during the encode process if it founds a value outside the TV range...?
All docs on the net are obsolete, they talk of a "fullrange" parameter with "off" by default ;), but this parameter doesn't exist anymore.
I never said that full range is not possible, i said that standard/default range of YUV, without more information than just only saying "it's YUV", is limited range.

Edit : More likely, the "auto" probably choose tv/pc according the input data format, and/or the encoded data format, and probably behave according the standard (YUV -> TV, RGB->PC).
Well, exactly what i'm doing.

[real.finder]

For x264, the parameter is :

--input-range <string> Specify input color range ["auto"]
- auto, tv, pc

Unable to find what "auto" is doing. Maybe it changes the range "on the fly" during the encode process if it founds a value outside the TV range...?
All docs on the net are obsolete, they talk of a "fullrange" parameter with "off" by default ;), but this parameter doesn't exist anymore.
I never said that full range is not possible, i said that standard/default range of YUV, without more information than just only saying "it's YUV", is limited range.

Edit : More likely, the "auto" probably choose tv/pc according the input data format, and/or the encoded data format, and probably behave according the standard (YUV -> TV, RGB->PC).
Well, exactly what i'm doing.

it's see the max and the min, with auto I encode full range anime EP (Trickster) from Japanese TV and it was fine with no clamp in auto

[jpsdr]

Well... Even if i realy don't like it, i'll set back the default range on 1 on next releases, as everyone seems to want to have improper behavior, i'll put the range in my scripts...

[pbristow]

If the actual values are only ever one LSB out of range, that suggests trivial rounding errors are at fault, and hard limiting the range is the appropriate option. If the overshoot is bigger, that suggests re-scaling (or filter chain debugging) is needed.

... *OR* that in fact neither is needed, because *at this particular point in the script or filter chain*, out-of-standard-range values are entirely to be expected, and nothing to worry about!

[pbristow]

Just seen this, after posting my addendum:

Well... Even if i realy don't like it, i'll set back the default range on 1 on next releases, as everyone seems to want to have improper behavior, i'll put the range in my scripts...

Thankyou.

Side-note: It's not "improper behaviour" we want (who in the world wants that?). It's "behaviour that we think is proper, and you don't". =;o}

[SALUTES YOU FOR TACKLING THE MULTI-THREADING OPPORTUNITIES IN THE BEST, MOST LOGICAL AND EFFECTIVE PLACE]

[jpsdr]

"Improper behavior" means thinking default YUV is full range when it's limited. You want Out of range value ? No problem, but you have to tell that you don't want normal output, that's all.
Why when you want "not out or range value", you have to specify it ? Proper behavior should be the oposite !
Well, as i said, it will be like this back again in next releases...

[jpsdr]

New version, see first post.

[jpsdr]

New version, see first post.

[FranceBB]

@jpsdr

ResampleMT 1.5.3 x86 XP SSE4.2 doesn't work on Windows XP x86, although the normal XP executable works fine (the one without assembly optimizations).
My CPU is an Intel i7 6700HQ, which supports SSE4.2, in fact your NNEDI3 XP SSE4.2 build works fine.

Oh, and by the way, thank you for your work and for supporting XP ;)