Bilateral filter on GPU for avisynth

[s.virojanadara]

I just wrote the bilateral filter on GPU for avisynth.

For more information please visit http://gpubilateral.sourceforge.net/ .

Best regards,
Sompon Virojanadara

PS: If i post in the wrong thread fell free to move this post to the right thread.

[Fizick]

1. can it be adopted for pixel shader 2.0 video cards?
2. how its algo is similar to non-gpu bilater plugins (TBilateral by tritical)?

[MfA]

Tritical's filter by default works only on the luminance plane, and supports a couple more options, this filters RGB planes independently (something I personally wouldn't do, you are going to create false colors that way).

[s.virojanadara]

1.can it be adopted for pixel shader 2.0 video cards?

Unfortuantely, No. Because restriction on instruction slot impose on pixel shader 2.0.
However, If you have pixel shader 2.0a
You may try HLSL + avishader at http://forum.doom9.org/showthread.php?s=&threadid=87295
However the approximation algorithm doesn't quite accurate. Well, it's approximation anyway. Moreover, the full kernel bilateral is restricted to 5x5.

2. how its algo is similar to non-gpu bilater plugins (TBilateral by tritical)?
Tritical's filter by default works only on the luminance plane, and supports a couple more options, this filters RGB planes independently (something I personally wouldn't do, you are going to create false colors that way).

I doesn't have time to look throughly in TBilateral by tritical source code yet (Well i just googling about it when i saw your post). But at the first glance, Mfa is quite right. The false color will be fixed in the next release.

Indepth explanation about false color can be found at Experiments with Color Images in
http://www.soe.ucsc.edu/~manduchi/Papers/ICCV98.pdf

For the option that Tritical's filter has, well i want to keep my filter to be as simplest to use as possible. However if you want any option to be implement don't hestitate to post your request here or email me at s.virojanadara@gmail.com.

[s.virojanadara]

This is not a quite as my plan to do. The GPU CPU synchronization development takes longer than I expected. Thus, i decide to release this "half-developed" release. For more information visit http://gpubilateral.sourceforge.net/.

PS. "False color" elimination still not yet implement in this release please be patient. It should be implement in next release.

Best regards,
Sompon Virojanadara

[s.virojanadara]

For more information visit http://gpubilateral.sourceforge.net/index.php?option=com_content&view=article&id=7&Itemid=7 .

Next release should have support for planar type video (YUV, YV12). However, i have to study that first :p.

Best regards,
Sompon Virojanadara

[s.virojanadara]

For more information visit http://gpubilateral.sourceforge.net/

Add native support for planar type video (YUV, YV12) as promised ;).
Major stability fixed. (This is why it's so late released :mad:)

[Keiyakusha]

For more information visit http://gpubilateral.sourceforge.net/

Add native support for planar type video (YUV, YV12) as promised ;).
Major stability fixed. (This is why it's so late released :mad:)

Yes! Yes! :thanks:

[Adub]

Damn! I may have to look at this some more tonight! Thanks s.virojanadara!

[yup]

Hi folk!
I have Radeon HD 2600 Pro AGP card. I get error
Card not support glDeleteFencesNV
My card support shader 4.0. Why?
May be plugin support only Nvidia cards?
Please advice.
yup.

[Dreassica]

Judging from the NV in the error, Nvidia only yes. Not working on my 4870x2 either

[yup]

Fortunately MSU denoising work
http://compression.ru/video/denoising/index_en.html
If use manual preset, exist low, medium and high bilateral. colorspace RGB24. Work realtime even on my card.
Competing war between ATI and NVidia not very good for user:D.
yup.

[s.virojanadara]

Hi folk!
I have Radeon HD 2600 Pro AGP card. I get error
Code:
Card not support glDeleteFencesNV
My card support shader 4.0. Why?
May be plugin support only Nvidia cards?
Please advice.
yup.


:thanks:, for feedback yup (Should i call you this? Sorry if my question is rude :confused:). That surprise me too. I never thought that ATI cards not fully support Open GL 2.0 specification. Well, regretably I don't have an ATI card to test it myself :p.

Fortunately MSU denoising work
http://compression.ru/video/denoising/index_en.html
Yeah, great to have an alternative.

If use manual preset, exist low, medium and high bilateral. colorspace RGB24. Work realtime even on my card.
Well you are comparing Radeon HD 2600 Pro with my poor on board GeForce 8200. That is 144 GFlops vs 21.6* GFlops. :devil:

Competing war between ATI and NVidia not very good for user.
I couldn't agree more!

Ps. Right now I working on other project. If I have free time again I will try to solve this matter.

Pps. Hope, reader know by now that my filter not support ATI card yet :p

* Ppps. I never found my Geforce 8200 mGPU specification that 21.6 figure i take it from Geforce 8300 pci-e card. :p:p:p

Sincery yours,
s.virojanadara

[Leak]

:thanks:, for feedback yup (Should i call you this? Sorry if my question is rude :confused:). That surprise me too. I never thought that ATI cards not fully support Open GL 2.0 specification. Well, regretably I don't have an ATI card to test it myself :p.
If you read the OpenGL 2.0 specification (http://www.opengl.org/documentation/specs/version2.0/glspec20.pdf) you'll find that neither glDeleteFencesNV nor the word "Fence" is in it. So it's a proprietary extension from nVidia...

np: The Matthew Herbert Big Band - The Yesness (There's Me And There's You)

[s.virojanadara]

If you read the OpenGL 2.0 specification you'll find that neither glDeleteFencesNV nor the word "Fence" is in it. So it's a proprietary extension from nVidia...

Hmm.., well you are right. Where did I get that idea from...? :confused: Well, nevermind. Thanks to Leak for enlighten me :).

After i carefully read the OpenGL 2.0 specification :p. It's seems there is no "Fence" equivalent instruction. Only synchronization OpenGL 2.0 Core specification provided is Finish command (Correct me if I am wrong. Thanks in advanced). Even worse there is no fence extension (or equivalent) from ATI in OpenGL® Extension Registry (http://www.opengl.org/registry/). (hmm, I wonder how do they synchronize CPU with ATI's card)

Thus, my solution for next version is. Detected weather the card support NV_fence or not if not ignore "wait for GPU routine". That's mean implicit GL_finish when read pixel back to memory will block the CPU time :mad:.

(And also replace all NV_ extension from my program if possible :p)

Sincery yours,
S.virojanadara

[MfA]

Even worse there is no fence extension (or equivalent) from ATI in OpenGL® Extension Registry (http://www.opengl.org/registry/). (hmm, I wonder how do they synchronize CPU with ATI's card)
GL_finish is a synhcronization mechanism, it just makes it hard to keep the GPU busy. Looking around, double buffering with PBOs should work fine for you.

[s.virojanadara]

GL_finish is a synhcronization mechanism, it just makes it hard to keep the GPU busy. Looking around, double buffering with PBOs should work fine for you.

Thanks for your comment MfA. However, my program already utilize the PBOs and FBOs. The problem is GL_finish always keep CPU busy by do nothing but wait for GPU to finish. Thus, i use Fence to query the status of GPU first. If GPU is still busy let CPU do something else. That make my program CPU utilization relatively low. Thus, your PC can process other CPU intensive task in the mean time.

However, as reader may already know ATI card do not have mean to query the GPU "busy or not" status in OpenGL (Please, correct me here. I really want to be wrong here :p). Thus, my next version (will be release soon, right now at the final test stage) will hopefully support ATI card (If someone who have ATI card like to test it for me. I would really appreciate). Nevertheless, the CPU utilization on PC with ATI card will be relatively higher than NVIDIA card due to the reason I mentioned above.

Sincery yours,
S.virojanadara

[MfA]

Thanks for your comment MfA. However, my program already utilize the PBOs and FBOs. The problem is GL_finish always keep CPU busy by do nothing but wait for GPU to finish.
That's implementation dependent, I doubt many drivers use a polling loop to implement glfinish though ... what almost certainly happens is that glfinish internally uses a blocking call (the win32 WaitForSingleObject call for instance) which suspends the process until the driver signals based on a hardware interrupt.

[s.virojanadara]

Alright, I just release gpubilateral filter version 1.2a. Now support windows Vista. Hopefully, it will support ATI GPUs. Try it at http://gpubilateral.sourceforge.net/.
Any feedback would be appreciated.

[vucloutr]

hi, it works fine for me with Avisynth (AvsP) in RGB32 colorspace but i get this error with YV12 colorspace:
Traceback (most recent call last):
File "AvsP.pyo", line 5819, in OnMenuVideoToggle
File "AvsP.pyo", line 8925, in ShowVideoFrame
File "AvsP.pyo", line 9467, in PaintAVIFrame
File "pyavs.pyo", line 322, in DrawFrame
File "pyavs.pyo", line 301, in _GetFrame
File "avisynth.pyo", line 277, in GetFrame
WindowsError: exception: access violation reading 0xFFFFFFFF
Traceback (most recent call last):
File "AvsP.pyo", line 7123, in OnPaintVideoWindow
File "AvsP.pyo", line 9467, in PaintAVIFrame
File "pyavs.pyo", line 322, in DrawFrame
File "pyavs.pyo", line 301, in _GetFrame
File "avisynth.pyo", line 277, in GetFrame
WindowsError: exception: access violation reading 0xFFFFFFFF

[McCauley]

Added to the wiki (http://avisynth.org/mediawiki/External_filters#Spatial_Denoisers).
I found nothing in description that it works temporal, is that right?

Regards
McCauley

[yup]

Hi s.virojanadara!
This one more step!
Now script loaded without error, but video output wrong
http://img6.imageshack.us/img6/3355/gpubilateralerroryu9.th.png (http://img6.imageshack.us/my.php?image=gpubilateralerroryu9.png)
Script from doc:
LoadPlugin("BilateralFilter.dll")
AVISource("sel.avi")
Telecide(guide=2,post=0)
clip=ConvertToRGB32()
clip.BilateralFilter(1,9,5.0)

yup.

[Dreassica]

Nope, it is still not working here.
Now i get Card not support glDeleteFencesNV

Again something with NV in it.

[vucloutr]

with a Spartial size of 3 it seems as if the picture doesn't get processed.
even with 5 itereations and a Intensity range of 300 i can't see any difference to the original picture.

[s.virojanadara]

:thanks: alot guys. For your feedback. Ok, let's start!
hi, it works fine for me with Avisynth (AvsP) in RGB32 colorspace but i get this error with YV12 colorspace:

May I ask for the GPU brand and model you are using please?


Added to the wiki.
:D Thank you very much McCauley, for add my little program to the wiki.

I found nothing in description that it works temporal, is that right?

Regards
McCauley
You are absolutely right. At the moment my filter still work on 2D domain. Planning to expand to temporal domain (3D) on version 2.0.

This one more step!
Now script loaded without error, but video output wrong

LoadPlugin("BilateralFilter.dll")
AVISource("sel.avi")
Telecide(guide=2,post=0)
clip=ConvertToRGB32()
clip.BilateralFilter(1,9,5.0)

Thanks again for your feed back yup:D. I try to replicate your test. However, what is Telecide(guide=2,post=0)? My avisynth 2.5.8 saids
' There is no function named "Telecide" '

Nope, it is still not working here.
Now i get Card not support glDeleteFencesNV

Again something with NV in it.
Please try version 1.2.b The glDeleteFenceNV instruction was remove from mandatory list since version 1.2.a.
Note: version progress for my program is 1.2. -> 1.2.a -> 1.2.b
My version system will be like (Major feature).(Minor feature).(Bug fix No.)


with a Spartial size of 3 it seems as if the picture doesn't get processed.
even with 5 itereations and a Intensity range of 300 i can't see any difference to the original picture.
Thanks to vucloutr! You got me big time. I forgot to check the loop counter! Clumsy me again :p.

Ok, I correct it in version 1.2.b. Download it at http://gpubilateral.sourceforge.net/

Note: From version 1.2.b on spatial size figure has been corrected. That's mean spatial size in version 1.2.b is equal to spatial size from previous version -2. I.e. spatial size in version 1.2.b = 9 will equal to spatial size in previous version = 11.You may need to change spatial size accordingly to preserve previous result and speed.

Sorry for your inconvenience
s.virojanadara

[Dreassica]

Sorry, but I still get the same message even with .2b

[vucloutr]

would it be possible to have a even smaller spatial size ?
"BilateralFilter(1,3,3.0)" seems a little strong or is it just me zooming to much ? i'm not sure.. :D

May I ask for the GPU brand and model you are using please?
It's a Nvidia 8800 GTS 512 (G92), driver is the 181.22 x64.

[s.virojanadara]

Sorry, but I still get the same message even with .2b

Hmm, that's strange. Could you please tell me the size (in bytes) of GPUBilateralEngine.exe at the Virtualdub folder?

would it be possible to have a even smaller spatial size ?
"BilateralFilter(1,3,3.0)" seems a little strong or is it just me zooming to much ? i'm not sure..

I think you zooming too much :D. May be you might want to try Lanczos4Resize.
Except you're working on very dark scene. That is the flaw of bilateral algorithm. However, i planning to implement the adaptive bilateral filter in the future.

Note: Try zoom after bilateralfilter for better performance with similar result.

[Dreassica]

Sorry, stupid user error on my end, i replaced it in the avs plugin dir, totally forgot thet exe had to go into the vdub dir.
Works now yea, but garbled output at 720x480. resolutions of 640x480 and 1280x720 do work, but of course i want to filter before any resizing.

[yup]

s.virojanadara!
Telecide(guide=2,post=0)
http://neuron2.net/decomb/decombnew.html
It is plugin for deinterlacing. You can remove this string from script.
yup.

[yup]

s.virojanadara!
Do not work on 2 computer,
first ATI 2600 AGP 512MB, wrong output old Catalyst Driver, second ATI 3850 1GB, 9.1 driver wrong output all white, loading GPU near 20%.
yup

[s.virojanadara]

Sorry, stupid user error on my end, i replaced it in the avs plugin dir, totally forgot thet exe had to go into the vdub dir.
Works now yea, but garbled output at 720x480. resolutions of 640x480 and 1280x720 do work, but of course i want to filter before any resizing.


s.virojanadara!

Code:
Telecide(guide=2,post=0)http://neuron2.net/decomb/decombnew.html
It is plugin for deinterlacing. You can remove this string from script.
yup.

Thanks, alot guys! You just give me the big clue! Look like it might be the size restriction on ATI's GL_TEXTURE_RECTANGLE_ARB or FBOs or something else. Hmm, i have to do some research more on this.

Thus, thanks to Dreassica. Now we know. In PC that equip with ATI card, my filter only work on video size equal to standard screen resolution.

Sincery yours,
s.virojanadara

[yup]

s.virojanadara & Dreassica!
May be for ATI card resolution could be ratio to 32?
yup.

[knutinh]

Thanks for this nice plugin.

Howver, I get an error when trying to run:
Card not support glBindBuffer

I am using an Nvidia 8800 GTS on Windows XP SP3 and gpu bilateral filter v 1.2b

GPU caps viewer is reporting:
OpenGL version string 2.1.2
GLSL version string: 1.20 NVIDIA via Cg compiler

[TheProfosist]

i decided on this: BilateralFilter(1,9,3.0) for anime do you think i choose well?