AviShader (Hardware assisted avisynth plugin)

[Antitorgo]

One thing I really like about the DDT algorithm is that it helps on sharp angled edges. Her eyelashes start getting blocking w/ lanczos, with the DDT it keeps the edge. (Although, this DDT only does 45 degree angles, a more advanced algorithm would use multiple angles and make things even less blocky). [Edit: Also, I'd like to some day do the DDT with something besides linear interpolation on the triangles, doing lanczos on the triangles would be interesting, perhaps when I finish a lanczos pixel shader I can implement this and we can compare again.]

It actually has me thinking on a new way to do compression that is inherently non-blocky (its trianguly). But then again, I am half asleep as I type this so I could be totally, totally out in left field...

[State of Mind]

Your left field probably has greener grass than mine does. Hehehe.

[708145]

Great idea this HW assisted filtering!

It would be even greater if it worked for me but I posted that in the usage thread already.

Development related I just want to post a list of filters I usually use and that would profit from HW acceleration:

fft3dfilter
limitedsharpen
mftoon
warpsharp
lanczos4resize


Additionally a kind of spp filter would be great.

Don't get me wrong. These are just suggestions in case you don't know what to implement after SSX_sharpen :sly:

I'll watch the progress closely and am already considering a GPU upgrade.

bis besser,
Tobias

[Antitorgo]

I've already started work on some of these. Since ssxsharpen requires me to write lanczos as a filter, it is holding me up a bit...

fft3dfilter - I'm not too sure how feasible this is, but would be waaaay down on my list since it involves FFTs and my brain doesn't want to think about FFTs for a while (although someone could probably look at using some of the FFT stuff from the Mersenne folks www.mersenne.org who have a super-high performance hand tuned FFT that seems to kick major ass over all the other FFT algorithms out there).

LimitedSharpen - Already done and working (with unsharp mask), it really is waiting for me to fix some other bugs and add speed improvements (Especially the YV12 stuff).

mftoon - would be fairly easy to implement.

warpsharp - I dunno, is there source for this?

Lanczos - this is what I'm currently working on (for the ssx stuff), I have bicubic and Lanczos3 working right now, but there seems to be a major bug on ATI hardware (since someone tested it on their nVidia 6800 and that bug wasn't there). So until I write up a test program to send to ATI, I'm holding off a bit on any sort of release (plus it required some rejiggering that I forked off into a seperate filter for testing).

[tsp]

well I', nearly done with a GPU version of fft3dfilter. It's working somewhat now (only bt=1 is implementet at the moment and there is still some bug hunting left.) I'm getting about the same speed as the cpu version with an Asus GeForce 6800GT(this card has slower memory than an ordinary GT) and an athlon xp 2400 MHz, but I will try to increase the speed when I have a working version without to many bugs. The filter will requere Hardware dx 9.0 support.
So it seems as if most of the filter you are requesting is in the making.

[708145]

@tsp: Great :D

Originally posted by Antitorgo
I've already started work on some of these. Since ssxsharpen requires me to write lanczos as a filter, it is holding me up a bit...

fft3dfilter - I'm not too sure how feasible this is, but would be waaaay down on my list since it involves FFTs and my brain doesn't want to think about FFTs for a while (although someone could probably look at using some of the FFT stuff from the Mersenne folks www.mersenne.org who have a super-high performance hand tuned FFT that seems to kick major ass over all the other FFT algorithms out there).

LimitedSharpen - Already done and working (with unsharp mask), it really is waiting for me to fix some other bugs and add speed improvements (Especially the YV12 stuff).

mftoon - would be fairly easy to implement.

warpsharp - I dunno, is there source for this?

Lanczos - this is what I'm currently working on (for the ssx stuff), I have bicubic and Lanczos3 working right now, but there seems to be a major bug on ATI hardware (since someone tested it on their nVidia 6800 and that bug wasn't there). So until I write up a test program to send to ATI, I'm holding off a bit on any sort of release (plus it required some rejiggering that I forked off into a seperate filter for testing).

Good to hear that lanczos4, mftoon and Limitedsharpen are no big deal.

Can you do DCT in a shader? That would enable spp and SmoothD.

bis besser,
Tobias

[tsp]

if you can do a FFT in a shader you can also do a DCT.

[Antitorgo]

DCT is on the same order of magnitude of FFT... These are possible, but I'm not sure about them on two levels.

1) The time required
2) If any of the filters require precision, there could be a problem.

Actually, from the FFT front, ATI has a demo of an FFT/iFFT being done in a shader and it runs pretty fast (I want to say 15-20fps, but I could be way off).

TSP: You're using Brook aren't you? I'm a little curious as to how that is working out.

On another note, I think I've finally decided to do my YV12 as uploading 4 frames in a texture at once using one color channel per frame. I think this will be infinitely easier than me packing/unpacking bits, that just added too much complexity... Once this is done, I expect to see a decent speedup in framerate (although non-realtime filters will give a "jumpy" framerate). Plus, it gives the CPU things to do while the GPU is rendering and adds to my CPU/GPU parallelism.

[tsp]

Antitorgo: Yes I'm using brook and it works (sort of) well. It's easier to code but hard to debug. Also my FFT algorithm is fast enough to do ~32000 16x16 FFT and iFFT per sec.

[tsp]

would someone with an ati radeon 9500 or better please confirm that this (http://www.tsp.person.dk/fft3dgpu.zip) version of fft3dgpu produces strange artifacts. I suspect that they could be caused by a NVIDIA driver bug. Usage:
fft3dGPU(float sigma,float beta,int bw,int bh)
bw and bh should be a power of 2 (8,16,32,64,128,256 etc)
sigma and beta has the same meaning as in fft3dfilter
also this version defaults to bt=1.

[TheJudge]

This produces some kind of strange "line-displacement" on a radeon 9800.

used line: fft3dGPU(2,1,32,32)

image removed

JD

[Antitorgo]

Hmm... if I set bh=8 it all looks fine, anything else gives me the same banding that TheJudge is getting.

Also, I plugged it into my avs file I had set up for debugging AviShader because it was conveniently open, and I noticed that D3D9 was throwing some warnings like:
Direct3D9: (WARN) :Can not render to a render target that is also used as a texture. A render target was detected as bound, but couldn't detect if texture was actually used in rendering.
and:
Direct3D9: (WARN) :Device that was created without D3DCREATE_MULTITHREADED is being used by a thread other than the creation thread.

Both of which I've encountered before. The first being that you the drivers may not support rendering your render target to itself. I'm guessing that you don't control this and it is something in Brook...

The second is that you either need to create your D3D device as D3DCREATE_MULTITHREADED or you need to do your initialization in the first GetFrame() call vs. in the constructor since AviSynth apparently calls the constructor w/ one thread and GetFrame in a totally different one...

Anyways, neither of those seems to account for the banding problem. I think that is more likely to be a rounding error (I commonly get the rounding problem on NP2 textures) or you have your array index being calculated wrong or something...

[tsp]

Thanks Antitorgo you just solved the problem. After adding D3DCREATE_MULTITHREADED in brook and change the shader that was rendering to the source texture the lines disappeared. I have updated the file above so please try it and tell me about the speed compaired to fft3dfilter with bt=1 and the same bh and bw.

[Antitorgo]

Okay, tried it again and everything worked! Hooray.

Speed comparison was using:
fft3dFilter(2,1,0,32,32,1,2,0,true) - 7.0fps @ 100% CPU
fft3dGPU(2,1,32,32) - 6.0 fps @ 100% CPU

I noticed that fft3dGPU was still throwing off a bunch of render target warnings, but again, that might not affect anything...

I'm not sure if you are doing anything like pre-loading frames or anything like that, but you might want to look into it...

Also, just because internally, the graphics card doesn't support rendering to 8-bit targets, I think brook does packing/unpacking which slows things down, you might want to look at using my trick of loading 4 luma frames into a texture and letting the shader operate across all 4 at once (in a 32-bit texture), that should save the packing/unpacking operation, plus the added benefit of parellelizing the CPU/GPU a bit more with the CPU loading/unloading textures while things are rendering on the GPU.

[tsp]

Antitorgo i fixed the last rendertargets warnings. Also the filter does use all 4 channels. It works like this.
The cpu mirrors the border to process the hole frame(instead as only working on the center as fft3dfilter) then the 1 channel 8 bit image is uploaded to the GPU. The GPU then creates a 4 channels floating point texture containing both the image and the shiftet image multiplied with a factor to avoid borderartifacts. This texture is then FFT'ed (so both the shifted and non-shifted image is transformed in the same passes.) To calculate this FFT the filter does 2 bitreverse passes and log2(bw)+log2(bh) Butterfly passes with renders to two textures and log2(bw)+log2(bh) collect passes that combines the 2 textures to 1. Between the Horizontal FFT and vertical FFT there is 1 pass to convert the complex fft to real fft.
Then the resulting Texture is filtered in 1 pass and then iFFT'ed that requeres the same number of passes as the FFT. The last step is to add the two images in the texture to 1 4 channel 8 bit texture that are downloaded to the cpu. I think it would be possible to move the combine stage to the butterfly stage. This would maybe increase the speed some. But for now I need to add a 3D FFT to the filter for support for bt=2 and 3.

What should I look for in the brook sourcecode if I want to place some sleep commands to reduce cpu load. Is it before the textures are downloaded or after the shaders is done?

[modsoul]

hi all.
I have question. i read in this post.
Brilliant ideas. The ability to do everything in AviSynth with your graphics card. Once you get a better/newer card, likely better filtering results, eh?
I compared the images and had to put my eyes an inch from the screen and look verrry closely to be the oh-so-smallest difference, but the difference was sharper quality with the Lancsoz resizes, though you have to be looking as close as I did to notice it.
My video card is an ATI Radeon 9800 Pro. I look forward to the progression of the development of your filter. Of, of, sheesh...

that with this plugin we can do anything avisynth does in the gpu. Now my question is can i do de-interlacing using my gpu. this is the command i normally use to do de-interlace.
directshowSource("D:\organised\anime\fate stay night\mgs_complete\mgs2\MGS Sons Of Liberty CD2.avi", fps=25)
ConvertToYV12()
#interlacing
edeintted =last.AssumeBFF().SeparateFields().SelectEven().EEDI2(field=-1)
TDeint(order=0,full=false,edeint=edeintted)
Trim (500,0)

now i am not too good with avisynt. please tell me how i can do the de-interlacing on my gpu cuz currently its working at <10 fps.

i'll be happy to post my results..simply as a way of contributing back.
[edit]
it seems google has a nasty habit of digging up old threads.
is work still active on this. if so where. if not then ok.

[tsp]

Currently there are no way to run TDeint on the GPU. Most graphics card can do hardware accelerated deinterlacing(using directx Video Acceleration) but I don't know if there are any avisynth filters that takes advantages of this yet .

[Fizick]

Do you really think, that you can do EVERYTHING with this plugin? Funny man. :)

Your current script is a result of advanced algorithms of plugin writers (tritical), it can not be effective ported to GPU.

[LigH]

What a pity I missed this thread for years... so now:

http://www.blosser.org/d9/dlAviShader01.rar => HTTP Error 404.0 - Not Found

I hope the avishader_25_dll_20041228.zip hostet at WarpEnterprises (http://avisynth.org/warpenterprises/) is about the same?

[Wilbert]

Looking at the dates they are the same. It's a pity that the source is not included.

[naoan]

What a pity I missed this thread for years... so now:

http://www.blosser.org/d9/dlAviShader01.rar => HTTP Error 404.0 - Not Found

I hope the avishader_25_dll_20041228.zip hostet at WarpEnterprises (http://avisynth.org/warpenterprises/) is about the same?

found seemingly newer version of avishader (0.42) here : http://www.avisynth.info/?%A5%A2%A1%BC%A5%AB%A5%A4%A5%D6

it's in japanese, but you could search "avishader" and it'll be a direct link. :)

[leeperry]

that's the latest build I have(1/12/2005): http://www.mediafire.com/?i8ltgj0dt11si9j

it's a CPU hog, but it works wonders w/ this script: http://www.avsforum.com/avs-vb/showthread.php?t=912720

[SubPixie]

Speaking about using external shaders, does anyone know how -if possible- I could adapt MAME CRT emulation HLSL Shader (http://forums.bannister.org/ubbthreads.php?ubb=showflat&Number=73673&page=1) to be used with AviShader ?

http://i44.tinypic.com/24osy75.png

I already tried something like :
AviShader("D:\Emu\MAME\hlsl\post.fx", "ScanMaskTechnique")

It works... I mean something is going on and the image is actually modified, it looks like a "burned out" image with a strong contrast... But I wonder how I could apply scanlines and shadow mask like in the above example... I quickly gave a look at the source code this afternoon at work, but I don't really HLSL language...

If anyone can help me, maybe...

[MysteryX]

AviShader allows to use a shader in AviSynth.

What would it take to use another shader, SuperRes, in AviSynth?
https://github.com/zachsaw/MPDN_Extensions/blob/master/Extensions/RenderScripts/Shiandow.SuperRes.cs