Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 6th November 2017, 07:31   #1241  |  Link
Khanattila
Registered User
 
Khanattila's Avatar
 
Join Date: Nov 2014
Posts: 432
Quote:
Originally Posted by Groucho2004 View Post
As I mentioned several times, multi-threading a GPU filter makes little sense. That doesn't mean that you can't multi-thread the rest of your filters. That of course only makes sense if you're using filters that are heavy on the CPU.
Really, this is a topic that we have really talked about many times.
__________________
github.com
Khanattila is offline   Reply With Quote
Old 6th November 2017, 09:09   #1242  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 1,071
Quote:
Originally Posted by johnmeyer View Post
Moving prefetch to the line before the return statement did not do anything.

I'm going to take a break and see if I can figure out the error in my ways ...
Well. You explicitly set the KNLMeansCL MT mode to MT_SERIALIZED at the top of the script, which means that it - and everything upstream of it up to and including the source filter - will run single-threaded and synchronously, just like in vanilla Avisynth. You've thereby intentionally bottlenecked the script, so what are you expecting to see here, really? The only filters that are running multithreaded in your script are interleave() and weave().

You can definitely multithread KNLMeansCL by running multiple instances of it (MT_MULTI_INSTANCE, and I hope you have a ton of RAM) but the main benefit of this isn't that it improves the performance of KNLMeansCL itself, but that it allows filters upstream of it to run in parallel. Avisynth's parallelism is brain damaged, news at 11.

Last edited by TheFluff; 6th November 2017 at 09:13.
TheFluff is offline   Reply With Quote
Old 6th November 2017, 20:01   #1243  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,179
Quote:
Originally Posted by TheFluff View Post
Well. You explicitly set the KNLMeansCL MT mode to MT_SERIALIZED at the top of the script, which means that it - and everything upstream of it up to and including the source filter - will run single-threaded and synchronously, just like in vanilla Avisynth. You've thereby intentionally bottlenecked the script, so what are you expecting to see here, really? The only filters that are running multithreaded in your script are interleave() and weave().

You can definitely multithread KNLMeansCL by running multiple instances of it (MT_MULTI_INSTANCE, and I hope you have a ton of RAM) but the main benefit of this isn't that it improves the performance of KNLMeansCL itself, but that it allows filters upstream of it to run in parallel. Avisynth's parallelism is brain damaged, news at 11.
You are right that in post #1232 I did include MT_SERIALIZE, but in my earlier post (#1223), I didn't have that. Believe me, I've tried all the various MT modes.

Your idea of MT_MULTI_INSTANCE is very intriguing. I didn't know about that. I'll pursue that and see what happens. I do happen to have a "ton of RAM," although whether AVISynth+ running KNLMeansCL will use it, or not, I don't know. It should be fun to try it and find out. Thanks!
johnmeyer is offline   Reply With Quote
Old 8th November 2017, 18:17   #1244  |  Link
RieGo
Registered User
 
Join Date: Nov 2009
Posts: 57
i'd like to share a little story in case you got the same problem.
so lately my knlmeanscl didn't work. it always threw some missing dependency errors.
i tried everything mostly reinstalling ms vs runtimes, but i couldn't figure out the problem.
my graphics driver was up to date - that's what i thought.

here's what actually happened: i got a new Win 10 Upgrade lately. what i didn't expect/know is that it replaces my graphics driver with a worse one - which doesn't have opencl support(?)
completely reinstalling the nvidia driver fixed everything and KNLMeansCL is working great again
RieGo is offline   Reply With Quote
Old 8th November 2017, 20:59   #1245  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,179
Quote:
Originally Posted by RieGo View Post
i'd like to share a little story in case you got the same problem.
so lately my knlmeanscl didn't work. it always threw some missing dependency errors.
i tried everything mostly reinstalling ms vs runtimes, but i couldn't figure out the problem.
my graphics driver was up to date - that's what i thought.

here's what actually happened: i got a new Win 10 Upgrade lately. what i didn't expect/know is that it replaces my graphics driver with a worse one - which doesn't have opencl support(?)
completely reinstalling the nvidia driver fixed everything and KNLMeansCL is working great again
I think you are right about the graphics driver, and a few posts further up, Groucho tried to help me upgrade my XP driver, but the nVidia installer failed and I eventually gave up. My Win7 installation (same computer, different boot drive) has the last nVidia driver for Win7 and it will run KNLMeansCL.

As for having something replaced by a background upgrade that I don't authorize, I turn off ALL upgrades for everything, including Windows. Neither my Win XP or Win7 installation has been updated in seven years. As a result, I never have had the pleasure of turning on the computer, only to find that it doesn't behave like it did yesterday. As a result, I get a lot of work done because I don't spend any time in the "care and feeding" of my computer.

I partition my hard drives so that only the O/S and programs are on the boot partition. I can do an image backup in less than five minutes. So, if something bad does happen (people always worry that I'll get a virus or something like that, especially since I don't run any anti-virus programs), I can restore the computer in less than ten minutes. I've had to do this a few times when I've installed a new program that screws things up.

So: no background updates; no anti-virus software; no background programs (task manager only lists three processes in XP after I boot); and partition the boot drive. I highly recommend this to anyone who wants a fast, trouble-free computer.
johnmeyer is offline   Reply With Quote
Old 15th November 2017, 21:50   #1246  |  Link
edcrfv94
Registered User
 
Join Date: Apr 2015
Posts: 77
The result is different, which one is correct?

Code:
	SetMemoryMax(3000)
	
	ImageSource("1.png", end=0).Dither_convert_rgb_to_yuv(lsb=true,output="YV12").DitherPost(mode=6)
	
	ConvertBits(bits=16)
	
	KNLMeansCL(d=2, a=4, s=4, h=0.4, channels="Y", stacked=false, device_type="GPU")
	
	ConvertToStacked().DitherPost(mode=6, ampo=1)
Code:
	SetMemoryMax(3000)
	
	ImageSource("1.png", end=0).Dither_convert_rgb_to_yuv(lsb=true,output="YV12").DitherPost(mode=6)
	
	ConvertBits(bits=16)
	ConvertToStacked()
	
	KNLMeansCL(d=2, a=4, s=4, h=0.4, channels="Y", stacked=true, device_type="GPU")
	
	DitherPost(mode=6, ampo=1)
edcrfv94 is offline   Reply With Quote
Old 22nd November 2017, 05:47   #1247  |  Link
yup
Registered User
 
Join Date: Feb 2003
Location: Russia, Moscow
Posts: 854
Hi Khanattila!
Please give me link for original paper. I want understand how work all parameters (mainly a and s) Your plugin.
yup.
yup is offline   Reply With Quote
Old 22nd November 2017, 13:33   #1248  |  Link
videoFred
Registered User
 
videoFred's Avatar
 
Join Date: Dec 2004
Location: Gent, Flanders, Belgium, Europe, Earth, Milky Way,Universe
Posts: 663
I realy would like to test this filter. Unfortunately I'm getting the following error message:

Quote:
no compatible opencl platforms available
My graphics card is Nvidia GeForce GT 420, drivers are updated.
I'm on Win10, opencl.dll is there, both in system32 and in syswow64.

Any suggestions please?


Fred.
__________________
About 8mm film:
http://www.super-8.be
Film Transfer Tutorial and example clips:
https://www.youtube.com/watch?v=W4QBsWXKuV8
More Example clips:
http://www.vimeo.com/user678523/videos/sort:newest
videoFred is offline   Reply With Quote
Old 22nd November 2017, 13:42   #1249  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 6,958
Quote:
Originally Posted by videoFred View Post
I realy would like to test this filter. Unfortunately I'm getting the following error message:



My graphics card is Nvidia GeForce GT 420, drivers are updated.
I'm on Win10, opencl.dll is there, both in system32 and in syswow64.

Any suggestions please?


Fred.
Your GPU does not support OpenCL 1.2 !
https://www.geforce.com/hardware/not...specifications
Atak_Snajpera is offline   Reply With Quote
Old 22nd November 2017, 13:52   #1250  |  Link
Groucho2004
Registered User
 
Groucho2004's Avatar
 
Join Date: Mar 2006
Location: A wretched hive of scum and villainy
Posts: 4,373
Quote:
Originally Posted by videoFred View Post
My graphics card is Nvidia GeForce GT 420, drivers are updated.
I'm on Win10, opencl.dll is there, both in system32 and in syswow64.

Any suggestions please?


Fred.
As mentioned, your card does not support OpenCL 1.2. However, you can use an older version of KNLMeansCL (0.6.11).
__________________
Groucho's Avisynth Stuff
Groucho2004 is offline   Reply With Quote
Old 22nd November 2017, 14:25   #1251  |  Link
videoFred
Registered User
 
videoFred's Avatar
 
Join Date: Dec 2004
Location: Gent, Flanders, Belgium, Europe, Earth, Milky Way,Universe
Posts: 663
Thank you both for the fast advice!
Will try 0.6.11 as suggested by Groucho!

Fred.
__________________
About 8mm film:
http://www.super-8.be
Film Transfer Tutorial and example clips:
https://www.youtube.com/watch?v=W4QBsWXKuV8
More Example clips:
http://www.vimeo.com/user678523/videos/sort:newest
videoFred is offline   Reply With Quote
Old 22nd November 2017, 21:01   #1252  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,179
Quote:
Originally Posted by videoFred View Post
I realy would like to test this filter. Unfortunately I'm getting the following error message:

My graphics card is Nvidia GeForce GT 420, drivers are updated.
I'm on Win10, opencl.dll is there, both in system32 and in syswow64
.

Any suggestions please?
I had the same problem, but now have everything working on Win7 64-bit, with AVISynth+ (r2504) and the 11/5/2017 version of KNLMeansCL. I went back to my notes to see what changes I made to get it to work, but cannot remember what I did (sorry!). I most definitely could NOT get it to work under XP where, for various reasons, I've been unable to install a later driver for my old nVidia GeForce 9800GT. Under Windows 7 I did not have that problem and have what I think is the last supported driver for that O/S, version 341.44.

My opinion of the filter is quite varied: on some material it performs brilliantly, clearly doing a better job than MDegrain2. This is especially true for "blotchy noise" that results from brightening really dark video. However, on other material, despite several people's claims that it doesn't destroy details, it absolutely ruined the video. This wasn't minor, but was really bad. The problems were most apparent on things like water glasses on a banquet table, where the subtle shadings on the glass disappeared. On another video, the wavy folds on a dress shirt also disappeared. So, take a hard look at any contoured surfaces that have no "high frequency" detail and instead simply have subtle changes in shading.

The filter is also dog slow without any GPU assist, and MT doesn't seem to do much (several people have said in this thread that MT and GPU end up working at odds, although without a GPU, I would think MT could still do something, but what do I know).

For me, any filter that cannot get above 1 fps is a unusable. I simply don't have multiple days to devote to any individual film or video. Life is full of "engineering trade-offs," and performance does matter.

Last edited by johnmeyer; 24th May 2019 at 20:21. Reason: change "about" to "above"
johnmeyer is offline   Reply With Quote
Old 23rd November 2017, 12:54   #1253  |  Link
Khanattila
Registered User
 
Khanattila's Avatar
 
Join Date: Nov 2014
Posts: 432
Quote:
Originally Posted by yup View Post
Hi Khanattila!
Please give me link for original paper. I want understand how work all parameters (mainly a and s) Your plugin.
yup.
B. Goossens, H.Q. Luong, J. Aelterman, A. Pizurica, and W. Philips,
"A GPU-Accelerated Real-Time NLMeans Algorithm for Denoising Color Video Sequences",
in Proc. ACIVS (2), 2010, pp.46-57.
__________________
github.com
Khanattila is offline   Reply With Quote
Old 23rd November 2017, 16:18   #1254  |  Link
VS_Fan
Registered User
 
Join Date: Jan 2016
Posts: 91
Quote:
Originally Posted by Khanattila View Post
B. Goossens, H.Q. Luong, J. Aelterman, A. Pizurica, and W. Philips,
"A GPU-Accelerated Real-Time NLMeans Algorithm for Denoising Color Video Sequences",
in Proc. ACIVS (2), 2010, pp.46-57.
You could also find useful the papers referred by Kevin Stone (aka ‘Tritical’) when he released 'TNLMeans' 10 years ago. Those are still available at the AviSynth Wiki

EDIT: A functional snapshot from 2010 of tritical's web page, provided by wayback machine, where you will find the referred papers.

There are some other references at Wikipedia

Last edited by VS_Fan; 23rd November 2017 at 16:47.
VS_Fan is offline   Reply With Quote
Old 24th November 2017, 17:45   #1255  |  Link
Groucho2004
Registered User
 
Groucho2004's Avatar
 
Join Date: Mar 2006
Location: A wretched hive of scum and villainy
Posts: 4,373
Quote:
Originally Posted by johnmeyer View Post
However, on other material, despite several people's claims that it doesn't destroy details, it absolutely ruined the video. This wasn't minor, but was really bad. The problems were most apparent on things like water glasses on a banquet table, where the subtle shadings on the glass disappeared. On another video, the wavy folds on a dress shirt also disappeared.
Could you share the KNLMeansCL settings you used in these cases and possibly a clip to reproduce?
__________________
Groucho's Avisynth Stuff
Groucho2004 is offline   Reply With Quote
Old 25th November 2017, 00:17   #1256  |  Link
pcroland
Registered User
 
Join Date: Mar 2014
Location: Hungary
Posts: 59
Hi!

I recently find knlmeanscl and it works pretty great but I have some questions:

Why do I get glitches at the start of the video sometimes? And how should I get rid of them?
Here's an example: https://mega.nz/#F!mUpyTIII!IXp43ePL1PETUGyVMYNUUg (source and encode are both ~1min long)
Code:
DirectShowSource("D:\raccoons\DVD1\VIDEO_TS\VTS_01_1.VOB")
QTGMC(Preset="Fast")
SelectOdd()
Crop(14,2,-16,-0)
FillMargins(1,0,1,1)
bbmod(0,2,4,6)
knlmeanscl(D=2, A=3, h=3, device_type="GPU")
DeHalo_Alpha(rx=2.0, ry=2.0, darkstr=0.4, brightstr=1, lowsens=50, highsens=50, ss=1.5)
GradFun3(thr=0.35, radius=12, elast=3, mask=2, mode=0, smode=0, debug=0, lsb=False, lsb_in=False, staticnoise=False, y=3, u=3, v=3)
Spline36ResizeMod(688,538,1,0,-1,-1)
The other question is how can I speed up the encodeing process? knlmeanscl is the bottleneck right now. I have an i7-4790 but I don't have a dedicated GPU but knlmeanscl still runs faster with device_type="GPU". Should I get a GPU?
Thanks in advance,
pcroland.
pcroland is offline   Reply With Quote
Old 25th November 2017, 17:12   #1257  |  Link
Shirtfull
Registered User
 
Join Date: May 2007
Posts: 54
Quote:
Originally Posted by pcroland View Post
Hi!

Why do I get glitches at the start of the video sometimes? And how should I get rid of them?
Could be your source filter, try dgmpgdec
Shirtfull is offline   Reply With Quote
Old 25th November 2017, 17:43   #1258  |  Link
Groucho2004
Registered User
 
Groucho2004's Avatar
 
Join Date: Mar 2006
Location: A wretched hive of scum and villainy
Posts: 4,373
Quote:
Originally Posted by pcroland View Post
Why do I get glitches at the start of the video sometimes? And how should I get rid of them?
Here's an example: https://mega.nz/#F!mUpyTIII!IXp43ePL1PETUGyVMYNUUg (source and encode are both ~1min long)
Code:
DirectShowSource("D:\raccoons\DVD1\VIDEO_TS\VTS_01_1.VOB")
QTGMC(Preset="Fast")
SelectOdd()
Crop(14,2,-16,-0)
FillMargins(1,0,1,1)
bbmod(0,2,4,6)
knlmeanscl(D=2, A=3, h=3, device_type="GPU")
DeHalo_Alpha(rx=2.0, ry=2.0, darkstr=0.4, brightstr=1, lowsens=50, highsens=50, ss=1.5)
GradFun3(thr=0.35, radius=12, elast=3, mask=2, mode=0, smode=0, debug=0, lsb=False, lsb_in=False, staticnoise=False, y=3, u=3, v=3)
Spline36ResizeMod(688,538,1,0,-1,-1)
There are a bunch of potential culprits for the glitches. First and foremost, DirectShowSource is the worst choice for your source. Use DGIndex/DGDecode for VOB.
If that doesn't solve the glitch problem, test each of the filters/functions separately.

Quote:
Originally Posted by pcroland View Post
The other question is how can I speed up the encodeing process? knlmeanscl is the bottleneck right now. I have an i7-4790 but I don't have a dedicated GPU but knlmeanscl still runs faster with device_type="GPU". Should I get a GPU?
Yes, get a decent card (GTX1060/70/80).
__________________
Groucho's Avisynth Stuff
Groucho2004 is offline   Reply With Quote
Old 25th November 2017, 17:49   #1259  |  Link
pcroland
Registered User
 
Join Date: Mar 2014
Location: Hungary
Posts: 59
Quote:
Originally Posted by Groucho2004 View Post
There are a bunch of potential culprits for the glitches. First and foremost, DirectShowSource is the worst choice for your source. Use DGIndex/DGDecode for VOB.
If that doesn't solve the glitch problem, test each of the filters/functions separately.

Yes, get a decent card (GTX1060/70/80).
I tried DGIndex but there were still some glitches, for example the luma frame changed as it should but the chroma frames remained the same. I got an ASUS 1050 2GB, the encoding time is now 8fps and for some reason the glitches disappeared also
pcroland is offline   Reply With Quote
Old 25th November 2017, 22:34   #1260  |  Link
videoh
Registered User
 
Join Date: Jul 2014
Posts: 776
Earlier versions of the nVidia driver had a relevant MPEG2 bug. Maybe when you installed the 1050 you upgraded to the new driver, which has been fixed.

Still, there shouldn't be any problem with DGIndex. I'll have a look with your stream.

Last edited by videoh; 25th November 2017 at 22:43.
videoh is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 10:06.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.