fft3dGPU 0.8.2

[tsp]

FrEEwilL: Hmm try installing directx version 9.0c it's the version I use. I think that it might be the cause of the error. because fft3dgpu stops executing when trying to setup directx.

[hartford]

For the record: DirectX 9.0c causes problems for many people. I was fortunate in being able to remove it and revert to 9.0b.

Perhaps you could be explicit in the requirements of your filter that
DX 9.0c is required.

(I have Win2000 SP4, DX 9.0b, dual amd processor, ATI 9700. Your filter doesn't work for me :( )

[MacAddict]

9.0c is running flawlessly on all 4 of my XP and Win2K machines here. I'm not a gamer in the least so maybe thats why I havent seen issues.

@tsp,

I only use XviD with the same settings, source, AVS and CQM's while testing. I overclocked the 9700 to obtain the previous mentioned speeds, still shocked me though compared to this FX5200!

[tsp]

hartford: I didn't know until now what 9.0c was required. I could try to compile a version for directx 9.0b although it wouldn't support pixelshader 3.0.

[hartford]

Well, I don't know what the implications would be wrt shader 3.0.

I do not expect you to code for those that cannot use DX 9.0c. I'm just
asking that you inform those that want to use your filter that DX 9.0c is required, that's all.

[tsp]

hartford: I will do what also I will include an error message in the next version if directx 9.0c isn't installed.

[hartford]

Originally posted by MacAddict
9.0c is running flawlessly on all 4 of my XP and Win2K machines here. I'm not a gamer in the least so maybe thats why I havent seen issues.



Well, good for you. I don't understand how "gamming" introduces a problem.

I do see where 2 cpu's can introduce a problem.

What changed from DX 9.0b to 9.0c?

I don't know. Many others don't know. 2 programs to remove DX 9.0c would not be in existance if there were no problems, whether 1 or 2 cpu's.

Bah. Until MS fixes this there will be many unhappy people.

[hartford]

Originally posted by tsp
hartford: I will do what also I will include an error message in the next version if directx 9.0c isn't installed.

Thanks. I'm sure that this will prevent many curses ;)

[tsp]

I have just released version 0.43
It mainly contains memory leaks fixes and the geforce fx fix.
Also new is a version compiled for DirectX 9.0b so I expect hartford to come with a full report about how this version works on a SMP machine ;D
I also added more comments to the sourcecode if anyone wants to take a look.

[FrEEwilL]

@tsp
oh jesus.. dx9b was the cause. yesterday i've installed 9c and finally the filter worked! :P thanks for your help.

[hartford]

Originally posted by tsp
I have just released version 0.43
It mainly contains memory leaks fixes and the geforce fx fix.
Also new is a version compiled for DirectX 9.0b so I expect hartford to come with a full report about how this version works on a SMP machine ;D
I also added more comments to the sourcecode if anyone wants to take a look.

I'm sorry to say that I get this error (using FFT3dGPU9b):

Avisynth read error: Avisynth: caught an access violation at 0X0291cc61, attempting to read from 0x00000008

[tsp]

Originally posted by hartford
I'm sorry to say that I get this error (using FFT3dGPU9b):

Avisynth read error: Avisynth: caught an access violation at 0X0291cc61, attempting to read from 0x00000008

:( ok I'm installing win2k + SP4 +dx9.0b + visual c++ .NET + dx 9.0b SDK on a WMware virtual machine so I think it should be posible for me to make a working version soon. (Another thing about 15 sec after I installed win2k and connected to the internet there were 3 Trojan horses /WORMS installed from the internet. Total insane!!!)

[edit]
Could you try try this (http://www.tsp.person.dk/FFT3dGPU9b.zip) version. It cantains two files: fft3dgpu9b.dll and fft3dgpu9b_log.dll. try fft3dgpu9b.dll first and if it doesn't work try fft3dgpu9b_log.dll. It will generate a file called c:\FFT3dGPU_log.txt so if you could post the content of this file it would be easier to see what's wrong.
This build works in win2k with SP4+directx 9.0b and the reference (software) render. Vmware doesn't support hardware accelerated directx 9 so I couldn't test that.

[MacAddict]

Anyone successfully encoded 2passes with 0.43 version yet on an entire movie? My 2nd pass stopped almost immediately indicating the DirectX device was no longer available:confused:

[tsp]

MacAddict: Is the exact error message: "Direct3D device lost. Please restart the application"?
if that is the case do you have any full-screen program/games/screensavers running. That can cause it.
How many frames can you encode before it happens?

[MacAddict]

@tsp

Thats the exact message I got. I'll have more details in a few hours when I arrive home. Guessing right now but I dont think anymore than a thousand frames on the 2nd pass were encoded.

I can tell you now that no games are installed on this particular machine, all screensavers are disabled. Only thing else running at the time was 8rda-vcore monitoring voltages and temps in the system tray. Now that I think about it I cant be sure I didn't also launch ATItool to check my GPU speed momentarily:rolleyes:
http://www.techpowerup.com/atitool/

I'll restart the 2pass again to see if in fact ATItool was the culprit for that mishap:) Report back later, thanks for the tip!

[tsp]

MacAddict: It could be ATItool. Is this the first time it happens? I must amit that it is posible to recover the the Direct3D device but I'm a little lazy so it is not implementet yet (but I will se what i can do about it). As for what can cause it here is sentence from the directx docs:

By design, the full set of scenarios that can cause a device to become lost is not specified. Some typical examples include loss of focus, such as when the user presses ALT+TAB or when a system dialog is initialized. Devices can also be lost due to a power management event, or when another application assumes full-screen operation. In addition, any failure from IDirect3DDevice9::Reset puts the device into a lost state.

losing the focus shouldn't be a problem because this only applies to full-screen applications.

[MacAddict]

I'm almost positive now that at one point during the encode I did launch ATItool. I suspect the tool polled for D3D devices and somehow interrupted fft3dgpu. This is the first time I've attempted a full 2-pass encode on a DVD source. I'll start the encode up again tonight and report back soon.

FYI- No fullscreen apps, windows or screensavers were used.

[hartford]

Could you try try this (http://www.tsp.person.dk/FFT3dGPU9b.zip) version.


This version loads without error. I'm not certain at the moment if it works, ie, I'm using default and it doesn't seem to clean.

My script:

loadplugin("d:\plugins\FFT3dGPU9b.dll")

avisource("d:\test.avi").ConvertToYV12()

fft3dGPU()



Now that it loads I can adjust parameters.

Will report again.

[hartford]

Ok, I increased Sigma from default and it does work ;) nice!

Thankyou!


I'll post FPS when I capture a clip longer than 10 seconds.

[hartford]

Results from analog capture (analog capture to Huffyuv v2.1.1 CCESP Patch v0.2.5):

FPS: about 17


Did a 3min 45sec capture.

Used this script:

loadplugin("d:\plugins\FFT3dGPU9b.dll")
loadplugin("d:\plugins\Decomb521.dll")

avisource("d:\test2.avi")

Telecide(order=1,Post=0,Guide=1,nt=25)
Decimate(Cycle=5,Mode=0,Quality=3)

ConvertToYV12()

fft3dGPU(sigma=8)

(Recompressed to Huffyuv via VirtualDub)

--

I'm not commenting on the results of Sigma=8 (did cause banding), just
the FPS at that setting.

[Zetto]

Originally posted by hartford
What changed from DX 9.0b to 9.0c?

I don't know. Many others don't know. 2 programs to remove DX 9.0c would not be in existance if there were no problems, whether 1 or 2 cpu's.

Bah. Until MS fixes this there will be many unhappy people.

Major change with DS9c was the introduction of shaders v3.0 for the GF6x00s. As for the DX uninstallers, there are problems with ALL versions of DX , as there were with DX9b (like that major bug with capture cards ;) )

TSP, keep up the good work. Thanks for the help over emails :D Any plans to add a GUI for ur filter? I wanna see my 6800 work ;) Right now I have to settle for monitoring it's status by noise (fan speeds up) and by temp in the control panel

[tsp]

I have released version 0.44 of fft3dgpu, it now includes support for recovering a lost device and hopefull the directx 9.0b version works this time.

Zetto: There are already support for monitering the work done by using nvperfHUD(link in readme). To run nvPerfHUD do it like this:

set NVperf=true and used this commandline to run it:

code:
"PATH TO NVPerfHUD\NVPerfHUD.exe" "PATH TO VIRTUALDUBMOD\virtualdubmod.exe" "PATH TO AVS\test.avs"

and enabled "force NON PURE device"


hartford: Good to hear that it works now. Try setting mode=1 to reduce banding or lower the sigma value. Can you get more than 50% cpu utilization with fft3dgpu I ask because the code is multithreaded.

MacAddict: the new version should solve your problem

[MacAddict]

@tsp

I was able to reproduce the lost device issue:) It turned out to be the tightVNC service that was running in the background which I use for remote viewing purposes. My 2nd test with a full 2-pass encode worked flawlessly once I disable tightVNC.

Thanks for the new version. I'll give it a shot with tightVNC;)

[Zetto]

Thx, the nvperf_hud looks pretty neat. I have to complain tho about compatibility with HT cpus - still crashing with HT enabled (single cpu is rock solid). Both .43 and .44 are crash-prone (stops encoding, try to cancel and exit - crashes with memory access violation or somethin of that sort). Surprisingly, the .43 u have sent to me in email (is not the same as u have released here) is most stable. It works if I start first with single cpu enabled and later one I add the affinity for the second one. Dunno what to do about it. Without HT, encoding is very slow, and but when enabled, it crashes every now and then.

[tsp]

Zetto: The problem with hyperthreading is to locate where the error lies. Is it the nvidia driver that doesn't support multiple threads accesing the card at the same time(all games I know of only have 1 thread doing all the directx stuff) or is it somewhere in fft3dgpu there happens to be a lockup or maybe avisynth doesn't like multiple threads accesing the same frame at the same time or it could be virtualdub there is something wrong with. You can try downloading this (http://www.tsp.person.dk/test.zip) version of fft3dgpu 0.44. The only difference is that it will produce a log file (C:\FFT3dGPU_log.txt) so if you could post the last 100 lines from thise file when virtualdubmod locks up(with HT enabled). The filesize can be 10-45 MB if the lockups first come after a couples of hours.

[Zetto]

Hehe, first pass and it crashes :( I've attached the last 100 lines or so of the log file. Let me know what seems to be the problem :D From my layman perspective it has something to do with GPU.. driver perhaps? I'm using 71.84.

.....

Getframe 5481
time: 16782156
SetSamplerState...
done
download from GPU...

[tsp]

Zetto: From reading in the directx newsgroups it seems as the problem is that directx doesn't like two threads sending commands to the same GPU at the same time very much. So I made a new version where only one thread at a time can use the GPU. So please try if this (http://www.tsp.person.dk/fft3dgpu_0443.zip) version solves the problem.

[Zetto]

Yay, tsp, u rule :D I've tried new version but now it locked up with kernel use (red bar in task manager) thru the roof (didn't happen before). With other versions, vdub crashes but doesn't use CPU at all but now it takes up 50% on HT CPU. Hmmm...

Thanks for the effort anyways. Hopefully we'll be able to track this thing down :D

[tsp]

:angry: Why does it have to be so hard to make a SMP stable version. :(
Could you try virtual dub 1.6 and see if that change anything. I have a new log producing version ready here (http://www.tsp.person.dk/fft3dgpu_0443.zip) so if you could repeat the test and PM the last 100 lines of the log to me I will see what I can get out of it.

[Revgen]

I use a Geforce 6800 Ultra using WinXP pro, DirectX9c, and 67.02 drivers.

I always get this error message "Avisynth: caught an access violation at 0x01e5cfe1, attempting to read from 0x00000008" instead of a picture whenever I play this script in media player.

Here is the script:

LoadPlugin("E:\TomsMoComp.dll")
LoadPlugin("E:\fft3dgpu\FFT3dGPU.dll")
directshowsource("E:\ds000.avi")
TomsMoComp(1,10,1)
FFT3dGPU(sigma=5,beta=1,bt=3,plane=0)
FFT3dGPU(sigma=5,beta=1,bt=3,plane=1)
FFT3dGPU(sigma=5,beta=1,bt=3,plane=2)

But it's okay when I use this script:

LoadPlugin("E:\TomsMoComp.dll")
LoadPlugin("E:\fft3dgpu\FFT3dGPU.dll")
directshowsource("E:\ds000.avi")
TomsMoComp(1,10,1)
FFT3dGPU(sigma=5,beta=1,bt=3,plane=0)
#FFT3dGPU(sigma=5,beta=1,bt=3,plane=1)
#FFT3dGPU(sigma=5,beta=1,bt=3,plane=2)

I even disabled the luma plane and left the chroma planes active and it didn't work, but it didn't give an error message that time.

Apparently this plugin doesn't want to process chroma.

The ds000.avi file is a video capture from an old vhs tape. I captured it at 720x480 resolution with a special YV12 version of Huffyuv.

Is this a problem on my end? I'm not as advanced a user as some of you are, so I'm not sure.

Any help would be appreciated. Thanks.

[tsp]

Revgen: I must admit I changed the way the plane parameter works compaired to fft3dfilter. You only need plane=1 to process both chroma planes(U and V)(I made it like this because how many times do you only filter U or V plane?).
But I also found a bug in the code. I forgot to change a U to a V. I will release a new version shortly until then use mode=1.

Another thing if anyone is able to run this filter on a radeon 9700 pro and windows xp/SP2 with this options: fft3dGPU(sigma=3.0,bt=3,bh=64,bw=64,mode=1)
without getting artifacts like this (http://www.bausoft.de/bir0/radeon9700pro.jpg) could you post what driver version you use.

[MacAddict]

tsp,

With a Radeon 9700 clocked to 400/315Mhz using XP SP2 I'm not getting that artifact with the parameters above. Using the latest Cat 5.4 driver suite as well.

I'm also happy to report the D3D device isnt lost anymore when using other applications like tightVNC, ATItool, etc:) Nice work!

[tsp]

New version ready that fixes the chromaplane bug.

MacAddict: Thanks and good to hear that the lost driver issue is gone.

[Zetto]

Hey TSP, I think u should add test (loggin) version with each new release ;) I wanna test .45, I totally didn't use chroma, wanna give a shot :D

[tsp]

Zetto: the only difference between 0.45 and the test version is the chroma bugfix (and of course the log). So I don't hope you mind being my beta tester :p

[Zetto]

Check ur pm TSP... now it crashed during 2nd pass, with kernel use again up to 50% on one CPU... Gah! Single affinity works without a hitch ;) but fairly slow (about 10hours instead of 8 :( )

BTW, TSP, point out a single person here who is NOT a beta-tester :D ;)

PS
Is it my imagination or does FFT3d use up a lot of kernel time from ver .443 forward (speed the same give or take)? What's changed?

[tsp]

Zetto: new test version ready 0.45.1 (http://www.tsp.person.dk/fft3dgpu_0451.zip).
just pm the log like last time. Also try reduceCPU=false if reduceCPU=true crashes. It seems as if the filter is stuck in an infinite loop so I added a timeout after 10 secs.
The kernel time did increase but the speed is the same when encoding XviD (or maybe a little faster). The filter is not multithreaded anymore(in the version from 0.40 to 0.44 the next frame was cached in a seperate thread) and I take care of only thread at a time acces the directx device so the directx driver don't have to handle it.



BTW, TSP, point out a single person here who is NOT a beta-tester :D ;)

well okay you are my alpha tester :) but the time is close where I might create a thread in Avisynth Usage. I think most of the bugs are fixed.

[Revgen]

I wouldn't post this filter into the Avisynth Usage thread until the WMP memory leaks are fixed. After I play an Avisynth file using fft3dgpu I always have to open up the task bar and shut down WMP manualy.

Although memory leaks don't occur when encoding a file when using Gordian Knot or Virtual Dub, I still believe that WMP should be able to work with it without any issues.

[tsp]

Revgen: What version of Windows media player do you use and are you using the same avs file as in your first post? Also that is not a memory leak unless you have a steady increasing memory utilization during playback. I guess why this error hasn't turned up is because this filter is to slow for real time playback.

[Zetto]

Giving .451 a spin right now. Loads up CPU pretty heavy, about 5-10% higher on HT CPU, speed remains the same tho (with default reduceCPU setting, will try to disable it if it crashes). What happes if there is a timeout? :confused:

[Revgen]

I use WMP 6.4.

It plays back fine for me. The only problem is that when I exit WMP the WMP program still stays in memory. It only happens when I use your filter. I just like playing the file to preview what the image quality is like before I encode it. I have an Athlon 64 3500+ Socket 939 CPU so it's not too bad as far as playback speed goes.

Here is the script I use. I now use mode=1 like you suggested. So it plays back fine.

LoadPlugin("E:\TomsMoComp.dll")
LoadPlugin("E:\fft3dgpu\FFT3dGPU.dll")

directshowsource("E:\ds000.avi")

TomsMoComp(1,10,1)
FFT3dGPU(sigma=2,beta=1,bw=32,bh=32,bt=3,plane=0,mode=1)
FFT3dGPU(sigma=2,beta=1,bw=32,bh=32,bt=3,plane=1,mode=1)

[tsp]

Zetto: The loop where the lockups happends asks the GPU if it is done rendering if what is not the case FFT3dGPU sleeps so other threads/XviD can use the cpu. If the rendering is done or a timeout occurs the next thing happening is the result is downloaded from the GPU back to main memory. I don't know if the filter stops here or can process with the downloading but we will see what happens.

Revgen: I reproduced the lockup with WMP 6.4. It happends because the thread that initialize the invisible window FFT3dGPU uses for rendering is different from the thread destroying it when WMP closes down so fft3dgpu is stuck in an infinite loop waiting to close the window down. I will fix this tomorrow also you can safely use mode 0 from version 0.45 and onwards. It seems as if you finds a lot of bugs in my filter please continue that and it will soon be bugfree :D

[Zetto]

It chewed through 4 files (40 min each), actually, 2 files x 2 passes. No freezes. I am worried though that if the lockup has occurred, fft3dgpu stopped filtering (is that possible?) I'll review the files and see if they have problems with filtering.

EDIT: Seems like the v .451 have solved the problem with HT... I'll let it run overnight, see if anything breaks :D My comp been running 24x7 for a while now, hope nothing breaks...:eek:

Still I am curious: how does FFT3fGPU handle those exceptions when GPU locks up in the loop... does it try to redo the frame it got stuck at? or just moves on to the next one? If did I misread the hole idea?

[tsp]

version 0.46 is done. Includes two bugfixes so WMP 6.4 should work now and also Pentium 4 with HT enabled and Geforce 6800 Ultra.

Zetto: Take a look at this code snip where the lockups happens:

//While GPU working sleep
DWORD endtick=GetTickCount()+10000;//This is the end time this assures that we don't sleep for more than 10 sec.

LOG("sleep while query not flushed...")
//if GPU command queue is not empty and the time out has not occured sleep to allow other thread to work
while((S_FALSE == pQuery->GetData( NULL, 0, D3DGETDATA_FLUSH ))&&(GetTickCount()<endtick))
Sleep(0);
LOG("done")
}

//download texture to system memory texture
LOG("GetRenderTargetData...")
result=_pDevice->GetRenderTargetData(_pVideoSurface,_pShadowSurface);
LOG("done")

as you can see if the loop times out(because a lock-up occured, maybe because the nvidia driver that handles the Query isn't threadsafe) fft3dgpu just continues to the next step that is to download the texture(in this case frame) to the main memory just as it would if the GPU signaled it was done rendering. So it just continues working on the frame as nothing had happend. The only time FFT3dGPU redo's the frame is if the Direct3D device(GPU) is lost while calculating the frame(this can happend if a screensaver or another fullscreen 3D program/game are active.). That happend for MacAddict.

[Zetto]

So, if I understand correctly, when GPU locks up, the FFT3dGPU just takes whatever GPU has (or hasn't) done to the frame and loads it into main memory for further processing? Can it potentially lead to corrupted frames? That is, if the lock-up occurs while GPU is processing a frame, and does not complete the procedure, would FFT3dGPU load up whatever incompletely filtered frame from GPU into main memory for further processing? Would such crashes/lockups also speed up FFT3dGPU because it won't be processing ALL frames, since GPU would lock up on some of those?

I'm not saying I saw any corrrupted frames but I yet have to check the material I have produces so far. The good news is that the .451 worked for 10 hours straight with HT enabled and didn't crash. I just hope I didn't get garbage as a result ;)

On totally unrelated note, how do u check for artifacts? :D

[tsp]

Zetto: It is not as much the GPU that locks up that it is one specific command to check if the GPU is done rendering. I don't think there are many games if any that utilize this command so that might be why it locks up once in a while.
So I don't think it would cause any corrupted frames but if it is only 1 out of every 30000 frames I don't think you would notice it. The only way to see if there is any corruption would be to watch the movie. Also every time a lockup occur FFT3dGPU would have to wait 10 seconds before the loop times out so if there are many lockups the speed would only decrease.

Could you try version 0.46 and see if that one also works.

The usual way I check for artifacts is to look at the filtered frame :) or use something like this to compaire it with the output fft3dfilter produce:

src=avisource("c:\test.avi").convertoyv12()
a=src.fft3dfilter(bw=32,bh=32,ow=32,oh=32,sigma=3)
b=src.fft3dgpu(bw=32,bh=32,ow=32,oh=32,sigma=3,mode=1,usefloat16=false)
subtract(a,b)

[Revgen]

Surprise! Surprise! Another bug!:eek:

I was trying to encode my clip with Gordian Knot 0.35.0 using the new 0.46 version with FP32 instead of the default FP16. Unfortunately GK gave me this error.

http://img219.echo.cx/img219/6347/fft3dgpufp32gkerror10fq.th.jpg (http://img219.echo.cx/my.php?image=fft3dgpufp32gkerror10fq.jpg)

Here is the script I use.

LoadPlugin("E:\TomsMoComp.dll")
LoadPlugin("E:\fft3dgpu\FFT3dGPU.dll")
directshowsource("E:\ds000.avi")
TomsMoComp(-1,10,0)
FFT3dGPU(sigma=8,beta=1,bw=32,bh=32,bt=3,plane=0,mode=1,useFloat16=false)
FFT3dGPU(sigma=8,beta=1,bw=32,bh=32,bt=3,plane=1,mode=1,useFloat16=false)



This was the first time I ever tried to use FP32, so I decided to try it out on the VirtualDubMod 1.5.10 program without using Gordian Knot as a frontend and it worked fine.

So I decided to go back to using the "useFloat16=true" parameter and Gordian Knot didn't give an error.

This error comes up despite whether I'm doing a compressibility check or a straight encode.

[tsp]

Revgen: It is not a bug just me forgetting to include the explanation of this error in the readme.txt (Maybe I should include a FAQ in the readme). It just means that fft3dgpu needs more memory on the graphics card (yes I know you have 256 mb but that is not enough. Infact I think FFT3dGPU is the only reason why someone would have to buy a Geforce 6800 Ultra with 512 MB ram :) ). The only work around is to use mode 0 or 2 (or FP16). They only uses half the memory.

[Revgen]

Are you sure?

Why would it work well in VirtualDub and not in Gordian Knot?

[tsp]

hmm good question. It might be because Gordian knot allocates some memory on the GPU for it's own use or even worse it initialize two instances of the avs file(and when uses twice the memory). I will try downloading gordian knot and investigate what happens.