Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 5th November 2017, 22:11   #1221  |  Link
Groucho2004
 
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
Quote:
Originally Posted by johnmeyer View Post
Thanks for the help but it looks like I'll have to try another avenue. I tried to install the 340.52 driver and it wouldn't install (I got an error message from the nVidia installer that it could not install). I did a little research and found that this is a common problem, so I manually installed the driver. Unfortunately I ended up with a 4-bit 640x480 display using the generic Windows driver.
I bet you used the "Express Install". Bad idea since it installs a pile of crap (with .NET 4 dependencies) that you don't need.

1. Uninstall your old driver
2. Start the 340.52 installer and select "Custom".
3. De-select everything except the display driver
4. Install
__________________
Groucho's Avisynth Stuff
Groucho2004 is offline   Reply With Quote
Old 5th November 2017, 22:32   #1222  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,691
Groucho,

Yup, that's what I did. I did think of unchecking all that other stuff, because I never use it.

When I get back to the XP boot drive, I'll do as you suggest.

Back to my main problem. I've made real progress. I've switched over to the Win7 64-bit boot drive and after a lot of stupidity (forgot to copy the MT AVISynth+ version after installation of the old 2014 version), I have AVISynth+ running and KNLMeansCL is working without crashing. I've tuned the parameters and like the results.

However ...

My one remaining problem is that I can't get MT to work. I have all the SetMT calls in an AVSI file that is loading at startup, and I have the Prefetch(4) at the end of the script. I've used MT many times in the month since I first installed AVISynth+ on my XP partition, so I'm getting somewhat familiar with the syntax. I'm not sure what is going on. I only get 2 fps without it (I have to use d=3 to get decent noise reduction on this 1981 VHS dub of someone's wedding inside a dark church, which I've brightened to within an inch of its life). The result is remarkably good, far better than I've seen with the old fft3dfilter, MDegrain2, and other spatial, temporal, and fft denoisers I've used.

But because of the multiple frames that must be processed, it is pretty slow.

So, I am now really close and all I have to do is figure out how to speed it up. I've tried the script with and without the "requestlinear" statement. I don't think it is needed in my script.

Here's the script I'm using. For troubleshooting, I've temporarily removed the MtModes.avsi file and since there is only one plugin, I'm setting the MT mode explicitly within the script.

[edit]I just did a quick test, and MT is not working for any script. So, it is not this script and it is not KNLMeansCL. [/end edit]

Code:
loadPlugin("E:\Documents\My Videos\AVISynth\AVISynth Plugins\plugins\KNLMeansCL.dll")
setmemorymax(1200)

SetFilterMTMode("knlmeanscl", MT_SERIALIZED, true)

d_input=3     #number of past/future frames default=1; 0=1 frame; 1=3 frames
a_input=1     #radius of search window; default=2; 0=1 pixel; 1=9 pixels
s_input=4     #radius of similarity; default=4
h_input=10.0  #filter strength
wmode_input=0 #weighting function; default=0

source=AVISource("E:\fs.avi").KillAudio().Requestlinear()

separatefields(source)
even=selecteven()
odd=selectodd()
den_even = KNLMeansCL(even,d=d_input,a=a_input,s=s_input,h=h_input,wmode=wmode_input)
del_odd  = KNLMeansCL(odd, d=d_input,a=a_input,s=s_input,h=h_input,wmode=wmode_input)
interleave(den_even,del_odd)
output=weave()

return output

#Enable MT
Prefetch(5)

Last edited by johnmeyer; 5th November 2017 at 22:37.
johnmeyer is offline   Reply With Quote
Old 5th November 2017, 22:40   #1223  |  Link
Groucho2004
 
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
Quote:
Originally Posted by johnmeyer View Post
return output

#Enable MT
Prefetch(5)
Try placing the Prefetch call before "return output" (everything after a return statement is ignored).
__________________
Groucho's Avisynth Stuff

Last edited by Groucho2004; 6th November 2017 at 01:18.
Groucho2004 is offline   Reply With Quote
Old 6th November 2017, 00:36   #1224  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,691
Quote:
Originally Posted by Groucho2004 View Post
Try placing the Prefetch call before "return output" (everything after a return statement is ignored).
Well, if anyone would know, it would be you. However, that sure isn't what the AVISynth+ doc says, which is why my script is written that way. Here's the quote from the AVISynth+ Wiki:

"You enable MT by placing a single call to Prefetch(X) at the *end* of your script, where X is the number of threads to use."

I didn't put the asterisks on either side of "end," so whoever wrote that really want to emphasize, I thought, that it should be the very last statement in the script, period.

I'll try it when I get back to the Win7 boot partition and let you know (I'm back into the XP partition to try your suggestion about how to update the GPU driver). Thanks!

Last edited by johnmeyer; 6th November 2017 at 00:37. Reason: added thanks for the GPU driver suggestion
johnmeyer is offline   Reply With Quote
Old 6th November 2017, 01:28   #1225  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,691
I followed the directions exactly for installing just the nVidia driver, but the installatio program gave me the same error message. I'm not going to waste anymore of either my time, or anyone elses' time on this. While I still use XP a lot, it obviously isn't supported any more, not just by MS, but by any other company. Add to that the general incompetence of video graphic software programmers (I can't believe that the bugs and installation nonsense that started back in the 1980s is still going on today), and its a quagmire not worth entering.

However, thanks once again for the help.

No to go back to the Win7 64-bit and see if I can get MT to work.
johnmeyer is offline   Reply With Quote
Old 6th November 2017, 01:57   #1226  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,691
Moving prefetch to the line before the return statement did not do anything.

I'm going to take a break and see if I can figure out the error in my ways ...

[edit]Actually, I'm going to give up trying to get it to use MT. I just read through a huge portion of this thread, especially the long discussion between Groucho and MysteryX about using MT with this filter, with the KNLMeansCL author also making posts. My take from that is that MT may not even be possible with this filter, and at best, it is only going to provide a few percent improvement, not a 2x or 3x speed-up.

Last edited by johnmeyer; 6th November 2017 at 02:15.
johnmeyer is offline   Reply With Quote
Old 6th November 2017, 02:19   #1227  |  Link
Groucho2004
 
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
Quote:
Originally Posted by johnmeyer View Post
Moving prefetch to the line before the return statement did not do anything.
How do you determine this? Did you compare the number of threads used?

The script you posted above uses very few CPU cycles so depending on how you measured you may not have noticed a difference.
__________________
Groucho's Avisynth Stuff
Groucho2004 is offline   Reply With Quote
Old 6th November 2017, 03:33   #1228  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,691
Quote:
Originally Posted by Groucho2004 View Post
How do you determine this? Did you compare the number of threads used?

The script you posted above uses very few CPU cycles so depending on how you measured you may not have noticed a difference.
I viewed the AVISynth script output in VirtualDub. I used VD's "Run Video Analysis Pass" and watched the fps readout. I then moved the Prefetch call to a place both before and after the return statement. There was 0.00% difference: I got 2.04 fps for 30 seconds in both cases (I aborted after that amount of time). I also tried different thread settings from 2 all the way up to 8 (my computer's number of cores). No difference.
johnmeyer is offline   Reply With Quote
Old 6th November 2017, 08:14   #1229  |  Link
Groucho2004
 
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
Quote:
Originally Posted by johnmeyer View Post
I viewed the AVISynth script output in VirtualDub. I used VD's "Run Video Analysis Pass" and watched the fps readout. I then moved the Prefetch call to a place both before and after the return statement. There was 0.00% difference: I got 2.04 fps for 30 seconds in both cases (I aborted after that amount of time). I also tried different thread settings from 2 all the way up to 8 (my computer's number of cores). No difference.
Using your script, I get these results:

"Prefetch(5)" placed after "return output":
Code:
FPS (min | max | average):      22.66 | 23.51 | 23.41
Memory usage (phys | virt):     79 | 97 MiB
Thread count:                   23
CPU usage (average):            25%
"Prefetch(5)" placed before "return output":
Code:
FPS (min | max | average):      22.65 | 23.52 | 23.41
Memory usage (phys | virt):     80 | 97 MiB
Thread count:                   28
CPU usage (average):            25%
As I mentioned, the CPU load is very low and the GPU filter is the bottleneck.
If you use a filter that's heavy on the CPU such as MDegrain you will see a difference in speed and CPU usage.
__________________
Groucho's Avisynth Stuff
Groucho2004 is offline   Reply With Quote
Old 6th November 2017, 08:19   #1230  |  Link
Groucho2004
 
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
Quote:
Originally Posted by johnmeyer View Post
[edit]Actually, I'm going to give up trying to get it to use MT. I just read through a huge portion of this thread, especially the long discussion between Groucho and MysteryX about using MT with this filter, with the KNLMeansCL author also making posts. My take from that is that MT may not even be possible with this filter, and at best, it is only going to provide a few percent improvement, not a 2x or 3x speed-up.
As I mentioned several times, multi-threading a GPU filter makes little sense. That doesn't mean that you can't multi-thread the rest of your filters. That of course only makes sense if you're using filters that are heavy on the CPU.
__________________
Groucho's Avisynth Stuff
Groucho2004 is offline   Reply With Quote
Old 6th November 2017, 08:31   #1231  |  Link
Khanattila
Registered User
 
Khanattila's Avatar
 
Join Date: Nov 2014
Posts: 440
Quote:
Originally Posted by Groucho2004 View Post
As I mentioned several times, multi-threading a GPU filter makes little sense. That doesn't mean that you can't multi-thread the rest of your filters. That of course only makes sense if you're using filters that are heavy on the CPU.
Really, this is a topic that we have really talked about many times.
__________________
github.com
Khanattila is offline   Reply With Quote
Old 6th November 2017, 10:09   #1232  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 1,100
Quote:
Originally Posted by johnmeyer View Post
Moving prefetch to the line before the return statement did not do anything.

I'm going to take a break and see if I can figure out the error in my ways ...
Well. You explicitly set the KNLMeansCL MT mode to MT_SERIALIZED at the top of the script, which means that it - and everything upstream of it up to and including the source filter - will run single-threaded and synchronously, just like in vanilla Avisynth. You've thereby intentionally bottlenecked the script, so what are you expecting to see here, really? The only filters that are running multithreaded in your script are interleave() and weave().

You can definitely multithread KNLMeansCL by running multiple instances of it (MT_MULTI_INSTANCE, and I hope you have a ton of RAM) but the main benefit of this isn't that it improves the performance of KNLMeansCL itself, but that it allows filters upstream of it to run in parallel. Avisynth's parallelism is brain damaged, news at 11.

Last edited by TheFluff; 6th November 2017 at 10:13.
TheFluff is offline   Reply With Quote
Old 6th November 2017, 21:01   #1233  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,691
Quote:
Originally Posted by TheFluff View Post
Well. You explicitly set the KNLMeansCL MT mode to MT_SERIALIZED at the top of the script, which means that it - and everything upstream of it up to and including the source filter - will run single-threaded and synchronously, just like in vanilla Avisynth. You've thereby intentionally bottlenecked the script, so what are you expecting to see here, really? The only filters that are running multithreaded in your script are interleave() and weave().

You can definitely multithread KNLMeansCL by running multiple instances of it (MT_MULTI_INSTANCE, and I hope you have a ton of RAM) but the main benefit of this isn't that it improves the performance of KNLMeansCL itself, but that it allows filters upstream of it to run in parallel. Avisynth's parallelism is brain damaged, news at 11.
You are right that in post #1232 I did include MT_SERIALIZE, but in my earlier post (#1223), I didn't have that. Believe me, I've tried all the various MT modes.

Your idea of MT_MULTI_INSTANCE is very intriguing. I didn't know about that. I'll pursue that and see what happens. I do happen to have a "ton of RAM," although whether AVISynth+ running KNLMeansCL will use it, or not, I don't know. It should be fun to try it and find out. Thanks!
johnmeyer is offline   Reply With Quote
Old 8th November 2017, 19:17   #1234  |  Link
RieGo
Registered User
 
Join Date: Nov 2009
Posts: 59
i'd like to share a little story in case you got the same problem.
so lately my knlmeanscl didn't work. it always threw some missing dependency errors.
i tried everything mostly reinstalling ms vs runtimes, but i couldn't figure out the problem.
my graphics driver was up to date - that's what i thought.

here's what actually happened: i got a new Win 10 Upgrade lately. what i didn't expect/know is that it replaces my graphics driver with a worse one - which doesn't have opencl support(?)
completely reinstalling the nvidia driver fixed everything and KNLMeansCL is working great again
RieGo is offline   Reply With Quote
Old 8th November 2017, 21:59   #1235  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,691
Quote:
Originally Posted by RieGo View Post
i'd like to share a little story in case you got the same problem.
so lately my knlmeanscl didn't work. it always threw some missing dependency errors.
i tried everything mostly reinstalling ms vs runtimes, but i couldn't figure out the problem.
my graphics driver was up to date - that's what i thought.

here's what actually happened: i got a new Win 10 Upgrade lately. what i didn't expect/know is that it replaces my graphics driver with a worse one - which doesn't have opencl support(?)
completely reinstalling the nvidia driver fixed everything and KNLMeansCL is working great again
I think you are right about the graphics driver, and a few posts further up, Groucho tried to help me upgrade my XP driver, but the nVidia installer failed and I eventually gave up. My Win7 installation (same computer, different boot drive) has the last nVidia driver for Win7 and it will run KNLMeansCL.

As for having something replaced by a background upgrade that I don't authorize, I turn off ALL upgrades for everything, including Windows. Neither my Win XP or Win7 installation has been updated in seven years. As a result, I never have had the pleasure of turning on the computer, only to find that it doesn't behave like it did yesterday. As a result, I get a lot of work done because I don't spend any time in the "care and feeding" of my computer.

I partition my hard drives so that only the O/S and programs are on the boot partition. I can do an image backup in less than five minutes. So, if something bad does happen (people always worry that I'll get a virus or something like that, especially since I don't run any anti-virus programs), I can restore the computer in less than ten minutes. I've had to do this a few times when I've installed a new program that screws things up.

So: no background updates; no anti-virus software; no background programs (task manager only lists three processes in XP after I boot); and partition the boot drive. I highly recommend this to anyone who wants a fast, trouble-free computer.
johnmeyer is offline   Reply With Quote
Old 15th November 2017, 22:50   #1236  |  Link
edcrfv94
Registered User
 
Join Date: Apr 2015
Posts: 84
The result is different, which one is correct?

Code:
	SetMemoryMax(3000)
	
	ImageSource("1.png", end=0).Dither_convert_rgb_to_yuv(lsb=true,output="YV12").DitherPost(mode=6)
	
	ConvertBits(bits=16)
	
	KNLMeansCL(d=2, a=4, s=4, h=0.4, channels="Y", stacked=false, device_type="GPU")
	
	ConvertToStacked().DitherPost(mode=6, ampo=1)
Code:
	SetMemoryMax(3000)
	
	ImageSource("1.png", end=0).Dither_convert_rgb_to_yuv(lsb=true,output="YV12").DitherPost(mode=6)
	
	ConvertBits(bits=16)
	ConvertToStacked()
	
	KNLMeansCL(d=2, a=4, s=4, h=0.4, channels="Y", stacked=true, device_type="GPU")
	
	DitherPost(mode=6, ampo=1)
edcrfv94 is offline   Reply With Quote
Old 22nd November 2017, 06:47   #1237  |  Link
yup
Registered User
 
Join Date: Feb 2003
Location: Russia, Moscow
Posts: 854
Hi Khanattila!
Please give me link for original paper. I want understand how work all parameters (mainly a and s) Your plugin.
yup.
yup is offline   Reply With Quote
Old 22nd November 2017, 14:33   #1238  |  Link
videoFred
Registered User
 
videoFred's Avatar
 
Join Date: Dec 2004
Location: Terneuzen, Zeeland, the Netherlands, Europe, Earth, Milky Way,Universe
Posts: 689
I realy would like to test this filter. Unfortunately I'm getting the following error message:

Quote:
no compatible opencl platforms available
My graphics card is Nvidia GeForce GT 420, drivers are updated.
I'm on Win10, opencl.dll is there, both in system32 and in syswow64.

Any suggestions please?


Fred.
__________________
About 8mm film:
http://www.super-8.be
Film Transfer Tutorial and example clips:
https://www.youtube.com/watch?v=W4QBsWXKuV8
More Example clips:
http://www.vimeo.com/user678523/videos/sort:newest
videoFred is offline   Reply With Quote
Old 22nd November 2017, 14:42   #1239  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,806
Quote:
Originally Posted by videoFred View Post
I realy would like to test this filter. Unfortunately I'm getting the following error message:



My graphics card is Nvidia GeForce GT 420, drivers are updated.
I'm on Win10, opencl.dll is there, both in system32 and in syswow64.

Any suggestions please?


Fred.
Your GPU does not support OpenCL 1.2 !
https://www.geforce.com/hardware/not...specifications
Atak_Snajpera is offline   Reply With Quote
Old 22nd November 2017, 14:52   #1240  |  Link
Groucho2004
 
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
Quote:
Originally Posted by videoFred View Post
My graphics card is Nvidia GeForce GT 420, drivers are updated.
I'm on Win10, opencl.dll is there, both in system32 and in syswow64.

Any suggestions please?


Fred.
As mentioned, your card does not support OpenCL 1.2. However, you can use an older version of KNLMeansCL (0.6.11).
__________________
Groucho's Avisynth Stuff
Groucho2004 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 16:55.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.