SEt's Avisynth 2.5.8 MT compiled for *X86_64*, Latest Build 4/16/2010

[JoshyD]

@Stephen

Ah, I know why it happens in both . . . the resize code for RGB is pretty much left stock by myself, just pulled out into an asm file and written for all local register usage. It's never been performance optimized by anyone, to my knowledge. RGB resize doesn't come up that often I guess?

Found the problem, as well, I think. I'm uploading a new build for you to run through: Here's the test build. (http://avisynth64.googlecode.com/files/avisynth64_4-16-10.rar)

Slowly stepping through the interleave appears to be improved, but the original problem was subtle upon first inspection by my eyes. It's not completely gone . . .

32bit difference map:
http://www.mediafire.com/imgbnc.php/9fa6392282897d41706e823b120ce7642g.jpg


64bit difference map:
http://www.mediafire.com/imgbnc.php/3e7d1e79e38c6e26115da965894368f52g.jpg


Red means that the pixels are strictly different between adjacent frames of the image stream, using your script in VDub32 and VDub64. As you can see, it's better, but not perfect.

I've been playing around with the code seeing how much performance I could eek out of the core . . . let me know if anything breaks before I go update the first post.

For the devs, from the 2.58 source directly (and in 2.6 apparently):
// RGB24 is not recommended. 75% of all pixels are not aligned.
int y = vi.height;
int w = vi.width * 3;
int fir_filter_size = pattern_luma[0];
int* pattern_lumaP1 = pattern_luma+1 - fir_filter_size;
static const __int64 xFF000000 = 0xFF000000;
__asm {
push ebx
mov esi, srcp
mov edi, dstp
pxor mm2, mm2
movq mm4, xFF000000
align 16
yloop24:
xor ecx, ecx
mov edx, pattern_lumaP1 ;cur - fir_filter_size
align 16
xloop24:
mov eax, fir_filter_size
lea edx, [edx+eax*4] ;cur += fir_filter_size
mov ebx, [edx]
lea ebx, [ebx+ebx*2] ;ebx = ofs = *cur * 3
add edx, 4 ;cur++
pxor mm0, mm0 ;btotal, gtotal
pxor mm1, mm1 ;rtota
lea edx, [edx+eax*4] ;cur += fir_filter_size
add ebx, esi ;ebx = srcp + ofs*3
lea eax, [eax+eax*2] ;eax = a = fir_filter_size*3
align 16
aloop24:
sub edx, 4 ;cur--
sub eax, 3
movd mm7, [ebx+eax] ;mm7 = srcp[ofs+a] = 0|0|0|0|x|r|g|b
punpcklbw mm7, mm2 ;mm7 = 0x|0r|0g|0b
movq mm6, mm7
punpcklwd mm7, mm2 ;mm7 = 00|0g|00|0b
punpckhwd mm6, mm2 ;mm6 = 00|0x|00|0r
movd mm5, [edx] ;mm5 = 00|co (co = coefficient)
packssdw mm5, mm2
punpckldq mm5, mm5 ;mm5 = co|co
pmaddwd mm7, mm5 ;mm7 = g*co|b*co
pmaddwd mm6, mm5 ;mm6 = x*co|r*co
paddd mm0, mm7
paddd mm1, mm6
jnz aloop24
pslld mm0, 2
pslld mm1, 2 ;compensate the fact that FPScale = 16384
packuswb mm0, mm1 ;mm0 = x|_|r|_|g|_|b|_
psrlw mm0, 8 ;mm0 = 0|x|0|r|0|g|0|b
packuswb mm0, mm2 ;mm0 = 0|0|0|0|x|r|g|b
pslld mm0, 8
psrld mm0, 8 ;mm0 = 0|0|0|0|0|r|g|b
movd mm3, [edi+ecx] ;mm3 = 0|0|0|0|x|r|g|b (dst)
pand mm3, mm4 ;mm3 = 0|0|0|0|x|0|0|0 (dst)
por mm3, mm0
movd [edi+ecx], mm3

add ecx, 3
cmp ecx, w
jnz xloop24

add esi, src_pitch
add edi, dst_pitch
dec y
jnz yloop24
emms
pop ebx

Put the rounder in those registers, also, specifically why not change:


movd mm5, [edx] ;mm5 = 00|co (co = coefficient)
packssdw mm5, mm2
punpckldq mm5, mm5 ;mm5 = co|co


to


movd mm5, [edx] ;mm5 = 00|co (cocoefficient)
pshufw mm5, mm5,0xCCh


You're only interested in the low word anyhow, high double is zeroed on a movd.

[Stephen R. Savage]

Good work! Let's hope this update gets included in the main Avisynth trunk. I don't see any red on flat areas anymore, so I'm guessing it's about as fixed as it can get. Out of curiosity, what was the source of the bug?

Also, I hate to pester, but any chance of updating MaskTools to v2.0a37? The changes should be trivial.

Edit: Also, what did you use to make the difference map?
Edit2: Also, any chance you can port the Average() plugin (or write your own)?

[JoshyD]

@Stephen
The source of the problem is those two lines of highlighted inline ASM for 2.58, or in my case similar lines of code living in my .asm file of a comparable function. Basically, the resizers are working with an intent of doing the calculation of the new pixel value as a floating point value, and then converting it back to an integer quantity. For this, it's best to adopt some sort of rounding methodology, where I noted above. The rounding value is used in YV12 and YUY2 calculations, it was just omitted from the RGB functions, probably because they're pretty well neglected.

The masktools v2.0a37 changes are pretty trivial, I'll see what I can cook up in the next few days.

The program I was using is from an image analysis pack called ImageMagick. (http://www.imagemagick.org/script/binary-releases.php#windows) I'll just go back to their compare function to give some basic statistical analysis of changes I make to speed up the algorithms. Essentially, to weigh speed vs quality.

This error was a blatant error and actually was rather trivial to find, I was going to submit a SourceForge bug to the Avisynth guys, but apparently the site is down at the moment.

As for average, I'm not getting any hits from the filter collection or the web in general, is this something
a) in 2.58 that I didn't port and am totally drawing a blank on
b) in 2.6 that can be added
c) has the source tucked away in a location you are familiar with
d) closed source

If it falls under those categories a-c, then probably feasible, I just need to know where to find the basis. If it falls under the dreaded category d (nnedi2!) then well, I can try to recreate it, but ugh, I've been busy. I was considering writing a comparable plugin to nnedi2, but this conversion may be easier in the short term.

The busy factor also means I have limited time to set up an RSS feed, document well, etc. The downside of OSS is that it's generally poorly documented . . . I can see why now. Please, if you get a chance, put the posted build through the paces, it has a number of small tweaks which generate a measurable performance difference. I've been playing with a lot of instruction ordering and code size variability, but can only test it with my machine. It may throw up on itself under another environment. This would be good to know in advance to prevent another total mess like last time.

[Stephen R. Savage]

Average plugin: http://www.wilbertdijkhof.com/mg262/Average_v11.zip

It's actually very simple. It takes an arbitrary number of clips and weights, then calculates the weighted average (w_a * a + w_b * b + w_c * c [...]).

Edit:
@Avs developers: Why is the YV12 luma channel centered on 126?

@JoshyD: I can't tell a difference between the Y-channel differences for RGB and YUV resizing now. I think this problem is fixed for certain. Good work.

Edit2:

@JoshyD: Also, what happened to the TCPDeliver/Source plugin that ships with normal Avisynth?

[manoj4986]

Guys also need following 64 bit plugins
1)HDRAGC
2)dctfilter
3)medianblur
4)nnedi2

[Stephen R. Savage]

Guys also need following 64 bit plugins\
4)nnedi2

Good luck with that.

[squid_80]

pshufw isn't used because it's an SSE instruction, not MMX.

[turbojet]

JoshyD: Yeah you can use the benchmark results wherever you want. It was done on an Athlon II 620 at 3.4 ghz. The deadlock didn't display anything, x264.exe was using 25% cpu with no status updates. During the benchmark it seemed to me the goal with avisynth mt is hitting 90%+ CPU with the fewest number of threads with a truly multi-threaded encoder (x264 with fairly fast settings). Deciding between MT() and SetMTMode() seems to be a crap shoot and both may result in a deadlock. With a lot of research and some coding it might be possible to automatically set the optimal setmtmode or mt when it's helpful and works.

Stephen: For update notifications you can get an rss feed from the googlecode download page.

[levi]

I did some benchmarks of the common filters I use and a few slower filters here's the numbers (messy I know but I couldn't think of a better way). Trends and notes below may make more sense

Could you update your post with the full .avs scripts you used?

During the benchmark it seemed to me the goal with avisynth mt is hitting 90%+ CPU with the fewest number of threads with a truly multi-threaded encoder (x264 with fairly fast settings).

Agreed. I found the same in my testing.

Deciding between MT() and SetMTMode() seems to be a crap shoot and both may result in a deadlock. With a lot of research and some coding it might be possible to automatically set the optimal setmtmode or mt when it's helpful and works.

That would be awesome! It would also be nice to have a repository to submit & compare speeds among various filters. This database could be helpful in manually and/or programatically selecting the optimal setting.

[Blue_MiSfit]

I'm getting some nasty crashes in virtualdub x64 or x264 x64 when using the 4/12 release on an MDegrain2 script...


setmtmode(2,0)

directshowsource("E:\BluRay Rips\Extract\Extract.mkv", audio=false, fps=23.976)

assumefps("ntsc_film")

spline36resize(1280,720)
md2(150,300)

distributor


MD2 is just a little wrapper for MDegrain2... the two parameters are thSAD and thSADC respectively.

Has anyone had luck getting something like this working with the current x64 releases?

Thanks
~MiSfit

[Manao]

Hey guys

Anyone care to test the x64 version of masktools in my sig ? If it doesn't crash with mt_invert and mt_edge, then all the asm should work with 64bits asm.

[Stephen R. Savage]

I receive an unrecognized exception with both mt_edge() and mt_invert(), Manao. I tested both the -25 and -26 variants.

[Manao]

I was afraid of that :(

Can you test mt_lut("x", chroma="process") ? (that one doesn't have asm).

Oh, and just to be sure, you tested the -25-x64 and -26-x64 variant, didn't you ?

[Stephen R. Savage]

I still receive an exception with your mt_lut("x", chroma="process") command. I tried mt_masktools-25-x64.dll and mt_masktools-26-x64.dll.

[JoshyD]

For what it's worth, here's the hack job that I did on MaskTools 2.0a36. (http://www.mediafire.com/?txjumzey0jg)

It's far from pretty, but maybe can provide some basis or at least an idea or two.

FYI, there were some templates for functions that got shuffled around because ICC is a bit more strict on where you declare templates, and some other junk. I don't think I mangled the names of any of the functions, and I did some decent editing on the stack / heap management macros as well as the generic computation macros. There's just so many differences in the code it's hard to take a straight directory diff and merge the two sources selectively. One of the main reasons I haven't attempted this yet. If we're getting an officially maintained x64 version, that would be very cool.

[Manao]

You'll be interested in the modifications I made. The assembly rewrite was two fold : lots of cosmetics, and use of the new stack macros - which (ought to) supports win32, lin64 and win64. I hardly changed anything else. Now if it could only work...

I'll see tonight if I have some ideas.

Meanwhile, if you want to try and build it yourself, just be advised you'll need yasm 1.0 in order to do so.

[trevaaar]

Just tried Dehalo_alpha with the 4/12 build and output is seriously broken. The crashing with MCTemporalDenoise I said about earlier in the thread is gone now though.

[Stephen R. Savage]

Just tried Dehalo_alpha with the 4/12 build and output is seriously broken. The crashing with MCTemporalDenoise I said about earlier in the thread is gone now though.

I see no problems with DeHalo_alpha. I assume you are using a MaskTools2 modified version. In case you aren't:

function DeHalo_alpha(clip clp, float "rx", float "ry", float "darkstr", float "brightstr", float "lowsens", float "highsens", float "ss")
{
rx = default( rx, 2.0 )
ry = default( ry, 2.0 )
darkstr = default( darkstr, 1.0 )
brightstr = default( brightstr, 1.0 )
lowsens = default( lowsens, 50 )
highsens = default( highsens, 50 )
ss = default( ss, 1.5 )

LOS = string(lowsens)
HIS = string(highsens/100.0)
DRK = string(darkstr)
BRT = string(brightstr)
ox = clp.width()
oy = clp.height()

x = ox/rx
y = oy/ry
m4x = x<16?16:int(round(x/4.0)*4)
m4y = y<16?16:int(round(y/4.0)*4)
ssx = ox*ss
ssy = oy*ss
m4ssx = ssx<16?16:int(round(ssx/4.0)*4)
m4ssy = ssy<16?16:int(round(ssy/4.0)*4)

halos = clp.bicubicresize(m4x,m4y).bicubicresize(ox,oy,1,0)
are = mt_lutxy(clp.mt_expand(),clp.mt_inpand(),"x y -")
ugly = mt_lutxy(halos.mt_expand(),halos.mt_inpand(),"x y -")
so = mt_lutxy( ugly, are, "x", "y x - y 0.001 + / 255 * "+LOS+" - y 256 + 512 / "+HIS+" + *" )
lets = mt_merge(halos,clp,so)
remove = (ss==1.0) ? clp.repair(lets,1,0)
\ : clp.spline36resize(m4ssx,m4ssy)
\ .mt_logic(lets.mt_expand().bicubicresize(m4ssx,m4ssy),"min")
\ .mt_logic(lets.mt_inpand().bicubicresize(m4ssx,m4ssy),"max")
\ .spline36resize(ox,oy)
them = mt_lutxy(clp,remove,"x","x y < x x y - "+DRK+" * - x x y - "+BRT+" * - ?",U=2,V=2)

return( them )
}

[trevaaar]

Yep, that's the script I'm using. It's not an error running the script, it's garbled output. On further inspection, it only appears to do so if width is not mod8. Alignment problem somewhere?

[jpsdr]

Here's the source I'm building from. (http://www.mediafire.com/?o23z43nmj1x) If you do a wholesale diff on the directory with something like WinMerge, it's pretty easy to see where it's been hacked apart to work with x64, just look for #ifndef _AMD64_ for most of the 64bit only mods.



I'm totaly lost indeed... I'm absolutely not used with the way the asm files are writen, and i understand nothing...
I don't have (and will not install) things like WinMerge.
Can you tell me what files i need to take a look for the IEEE 1180 reference ?
I've already identify the idctref.cpp file, but it can't be the only one...

[moviefan]

I set up a fresh Windows 7 x64, SEt's Avisynth 2.5.8 MT and the required x64 plugins for GradFun2DBMod and LSFMod (RemoveGrain, AddGrainC, Repair, gradfun2db, masktools) + ffms2 all taken from the first page of this thread. I started encoding with x264.1542.x64 build from x264.nl but the encode always crashes after 1000-1500 frames. I removed the filters step by step and it still crashes when only loading the video with FFVideoSource without any filtering afterwards (ths time after 100 frames). Am I doing something wrong or is there a severe bug?

[Stephen R. Savage]

Yep, that's the script I'm using. It's not an error running the script, it's garbled output. On further inspection, it only appears to do so if width is not mod8. Alignment problem somewhere?

@ JoshyD
I have traced the problem to mt_merge(). It does not correctly process clips if they are not aligned to mod8. All other mt functions used provided correct output. You can verify the problem with the following script:


ColorBars()
ConvertToYV12()
mod4 = Crop(4, 0, 0, 0).mt_test().AddBorders(4, 0, 0, 0).Subtitle("mod4")
mod8 = Crop(8, 0, 0, 0).mt_test().AddBorders(8, 0, 0, 0).Subtitle("mod8")
Interleave(mod4, mod8)

function mt_test(clip input)
{
a = Invert(input)
b = FlipVertical(input)
return mt_merge(input, a, b)
}


Incidentally, I wonder why most programming languages don't let you set variables equal to the names of functions and data types like Avisynth does.

[kemuri-_9]

I've looked into the non-monotonic PTS warnings that x264 generates when SetMTMode is off and a resizer is used (i was using a 1920x1080 source -> Lanczos4Resize(1440,1080))
xmm6 through xmm14 are not zero and it's causing a situation a condition surrounding a double that should be false to be flagged as true and invalidating the input PTS.

this is indicating that the resizer asm code is violating the win64 calling convention by not preserving registers properly.

[JoshyD]

@kemuri-_9
You've got my number on that one.

Both MS and Intel compilers never ever conform to the published ABI originally set forth by AMD. Floats should be passed in those xmm0-xmm3 registers, but, in my experience I've never seen either compiler put this into play. So, I was sitting around watching xmm8-xmm15 being completely unused in any C++ code. I got into the habit of only coding around what the compiler I was using was going to produce. It's easy to change, it just makes the heuristics of the previous testing I've done invalid. End result, very very foolish.

I was just blown away by the fact that the compiler wasn't following spec, and as most of the interactions with Avisynth64 are from code compiled by either MS or Intel's compiler, I was greedy with my registers, not thinking that I'd even see this problem until ICC (insert ridiculously far off built) or something of the like became prevalent. Intel's all about having a great compiler to support their "awesome" silicon, and they were still shipping a product that minimally made use of x64 . . . and that can come back on them (and me).

I tucked away the code that follows spec in expectation of the day that a commercial compiler would simply comply to spec. I tested a number of cases to make sure that the registers weren't in use when the function call was made before committing to using the extra lot. My ABI compliant source is a bit out of sync with my current build, so I'll need some cycles to iterate on what's going to produce the best performance, but hopefully will have a nice little fix for the weekend. Register usage should be relatively simple to cut, some constants are kept in the registers at the top range (or lazily in xmm8 in earlier code that needs updating anyhow), a memory reference here and there shouldn't be too painful.

If you didn't research this, I personally am not familiar enough with the build practices of GCC (and derivatives like MinGW) nor the x264 source itself, it would have taken some time to realize the root cause. It's interesting from an academic standpoint. My familiarity of the x264 source stops at what is integrated into MVtool2s, and I had never seen the devs go beyond 8 XMM registers (in the functions borrowed in MVTools2) at the asm level until r1531 apparently. Is the double from the compiler or assembly?

As an aside, I've never broken spec on the other ASM. Performance impact of not explicitly defining the extra register use at the machine code level will likely be mitigated by the an out of order execution engine on silicon anyhow (unless you're encoding on Atom, which is just silly) as well as the fairly large / efficient caches on modern architectures . I just want to "get it right" before packaging it up all nice and pretty.

@Stephen
The mod 8 problem is most likely a problem with a direct conversion of the routines that relied on a parameter being mod 4, but when you move to x64, it gets changed to 8 . . . and you loose granularity. Manao's code is REALLY flexible, and was written with an incredible amount of reuse in mind. Unfortunately, there's going to be corner cases that don't get seen for a bit. If he's going forth with an official x64 build, he may have a better idea of where the underlying problem, and have a particular way he'd like to solve it. I'm willing to help at any turn though.

Oh, and the compiler rant above happens to coincide with your question of setting the function name the same as a variable. Compilers have to take what is known about the language syntax, assign variables in their workspace to various logic functions, do logic minimization, see what resources can be shared, where there will be constraints, etc.

Now, compilers are written by humans. I guess the main goal would be efficiency. If you name instantiate your class "Cat" as "Cat" it introduces confusion and overhead into process. Consider naming your cat "cat" in the real world, when speaking to other humans (the english language being our syntax in this case), you're going to run into cases where if not known a priori that you had a pet cat named "cat" they would have no idea how to distinguish the two in conversation.

The difference being that I can learn that your cat is named cat, compilers have no memory, and lack a good method of learning. We can build in some heuristics and some neat tricks, but we scorn the programmer who wants us to complicate our compiler to understand the difference between variable cat and function/class cat. We can mangle the names at compile time, but that's just unneeded overhead. Just name your variable "instance_of_cat," or similar.

Compilers really are quite dumb, amazingly so in some instances. The more explicit you code in a higher level language, the easier it is to produce a quality executable. Assembly becomes attractive when the compiler balks at the code. There's a good ~3 post discussion of mt_impand/mt_expand (or similar function) in the current masktools thread. They're written in straight up C++. When you execute TGMC, these two make up ~90% of the cpu time spent in MaskTools2, which accounts for ~30% CPU time of all processes combined. That's when you say "yes please" to some good ol' assembly.

@jpsdr
I actually fixed the IEEE 1180 about two weeks ago, and forgot to post The build. (http://www.mediafire.com/?ztinlyzjjz3) I hope that's the right one, will double check and update the first post accordingly.

It's been a long week, cheers to everyone keeping an active interest in the project.

[Mr VacBob]

Windows doesn't use the x86-64 ABI, so compilers shouldn't be expected to respect it.

[Stephen R. Savage]

@JoshyD: Do you think rewriting your code to conform to the Win64 ABI will cost performance? I had heard D_S mentioning in other threads that the Win64 ABI seriously limits the effectiveness of the extra ADM64 registers.

[kemuri-_9]

If you didn't research this, I personally am not familiar enough with the build practices of GCC (and derivatives like MinGW) nor the x264 source itself, it would have taken some time to realize the root cause. It's interesting from an academic standpoint. My familiarity of the x264 source stops at what is integrated into MVtool2s, and I had never seen the devs go beyond 8 XMM registers (in the functions borrowed in MVTools2) at the asm level until r1531 apparently. Is the double from the compiler or assembly?

line 1475 from x264.c (r1563) is the aforementioned double that was exhibiting the problem, so this would be GCC/MinGW.

I haven't been maintaining my ICL patch to x264 lately, so i haven't been able to see if it similarly shows issues.

As an aside, I've never broken spec on the other ASM. Performance impact of not explicitly defining the extra register use at the machine code level will likely be mitigated by the an out of order execution engine on silicon anyhow (unless you're encoding on Atom, which is just silly) as well as the fairly large / efficient caches on modern architectures . I just want to "get it right" before packaging it up all nice and pretty.

don't feel too bad about it as the problem was found and you are willing to fix it, unlike the situation with ffmpeg where they know they violate the win64 ABI and no one really wants to fix the problem it seems.

[Manao]

For the problem regarding mod8 + mt_merge, it might actually be my fault. Try the same script with masktools 2.0a36 win32, you should (i think) get the same garbled output. And it should be fixed in the win32 dlls contained here :

http://manao4.free.fr/masktools-v2.0a39.zip

Now, this package also contains another attempt at a 64bits masktools dll. If somebody could launch DebugView on his computer, then try with this dll mt_merge, mt_invert, mt_lut, mt_edge and mt_logic, then make the resulting log file available to me, it would be great. Alternatively, if somebody could give me RDP access to a win64 machine with avisynth 64, vdub 64 and debugview installed, it would be even better.


-

[noee]

Manao:
Testing with veedub64 (AMD1.9.9) your latest x64 masktools, I get a unrecognized exception on line 493 in LSFMod() (appears to be the MT_Luxy call). My script is:

loadCplugin("C:\Program Files (x86)\AviSynth 2.5\plugins64\ffms2.dll")
ffvideosource("E:\output\test.mkv")
lsfmod()

the mkv contains SD AVC source. Works fine with the mt_Masktools posted on the first page. I have a debugview log, where could I send it? I would gladly allow RDP, but I'm dial-up. :(

[manoj4986]

Guys also need following 64 bit plugins
1)HDRAGC
2)dctfilter
3)medianblur


Anyone working on these 3 filters to bring 64 bit version?

Regards,
manoj

[mp3dom]

If I can, I would like to request a porting of the "BlockBuster" filter to x64. Thanks!

[Stephen R. Savage]

For the problem regarding mod8 + mt_merge, it might actually be my fault. Try the same script with masktools 2.0a36 win32, you should (i think) get the same garbled output. And it should be fixed in the win32 dlls contained here :

http://manao4.free.fr/masktools-v2.0a39.zip

Now, this package also contains another attempt at a 64bits masktools dll. If somebody could launch DebugView on his computer, then try with this dll mt_merge, mt_invert, mt_lut, mt_edge and mt_logic, then make the resulting log file available to me, it would be great. Alternatively, if somebody could give me RDP access to a win64 machine with avisynth 64, vdub 64 and debugview installed, it would be even better.


-
Unfortunately, I can not arrange for RDP access, as I am behind a firewall.

I tested the following script using your new test version:

...
a = Invert()
b = FlipVertical()
mt_merge(a, b)

I get an exception for mod8 and mod16 but not mod2 and mod4.

DebugView log for mt_merge, mod4: http://pastebin.com/RJw1RGq6
DebugView log for mt_merge, mod16: http://pastebin.com/3xFjqCUg

mt_invert() passes.

mt_lut("x 5 -") fails with an unknown exception. Log here: http://pastebin.com/FiD3Krs0

mt_edge() crashes VirtualDub. Log here: http://pastebin.com/HaxPekiK

I don't know how to use mt_logic, so you'll have to give me instructions here.

[noee]

Okay, yeah, I forgot about pastebin. Here's mine (http://pastebin.com/3jf9uC29) for the LSFMod() (mt_luxy fail, line 493):

[Manao]

OK, thanks guys. Stephen, can I assume that all your logs and observations on mt_merge were made with the 64bits version ?

Apart from that, I have no idea why mt_lut raises an exception while doing float operations. I guess I'm doing something that makes my dll incompatible with JoshyD's avisynth. The working of mt_invert is a good sign, and the crash in mt_edge can only be my fault. I'll get back to you later.

As a side note, I've put a win xp 64bits in a virtualbox, and i tried vdub64 + avisynth64, but it refuses to load even a simple script. Is that dll compatible with xp64 ?

[kemuri-_9]

As a side note, I've put a win xp 64bits in a virtualbox, and i tried vdub64 + avisynth64, but it refuses to load even a simple script. Is that dll compatible with xp64 ?

I use xp x64 and JoshyD's avs works fine outside of known bugs. though his install script is broken as it uses bcdedit which is only on vista and later systems. also, my avs setup is a bit different than most people's so the install bat fails horribly for me even after that.

[turbojet]

Thanks for reporting the issue of the install script on XP 64. I took a look at it and fixed it among other things. Full package with the latest dll's from the first post can be found at Mediafire (http://www.mediafire.com/?j0dncmnyzi0) or http://www.multiupload.com/4PY7ZLPSCZ

Changes:
- Checks for dll's before installing, if one is missing it informs the user
- Fixed XP x64 install (asks for user/pass if the user doesn't have admin rights)
- Fixed Vista/7 with UAC disabled for a user with no admin rights (run as administrator doesn't work with batch files in this case. now it asks for user/pass)
- Notifies Vista/7 users to run as administrator when UAC is enabled

[Manao]

As for my problem with XP 64, I just had to install all the updates. A stock XP64 SP1 wasn't working with avs64 (while it did with avs32).

I can, at last, debug :)

[Frank K Abbott]

I tried using the MT.dll from the file in hope that it would correct the horizontal "shadow line" type issue which appears right in the sentral part of the video but it still hasn't been fixed yet :( And that faint horizontal line shows up with a lot of filters when using MT with them.

[jpsdr]

Apart from that, I have no idea why mt_lut raises an exception while doing float operations.

Just in case, a little notice, if by mischance you didn't know, (even if it's more unlikely), but the x87 stack register must be preserved if the calling function may alter it (as you mention float operations).

[Manao]

Here (http://manao4.free.fr/masktools-v2.0a40.zip) is another attempt. Everything seems to work on my virtualbox. Float registers are preserved (they already were), the exception wasn't float related (I thought so at first, but it actually was in the stl)

I checked mod8 mt_merge and it worked ok. Stephen, can you confirm it ?

[Stephen R. Savage]

mt_merge works in mod4 and mod8 now. mt_invert and mt_lut now pass.

mt_edge does not appear to work with custom kernels:
mt_edge("5 10 5 0 0 0 -5 -10 -5 4", 0, 255, 0, 255)
just produces a white frame.

Edit: Incidentally, could you make MaskTools produce a more useful error when fed RGB than "unknown exception"?

[Manao]

Now that's ironic. I disabled the asm function for custom kernels because it looked wrong to me. As it happens, it was working, while the C version itself didn't work (both for win32 & win64). I'll fix that ASAP, and try to remove the assertion too.

[JoshyD]

Wow, so much development . . . very cool, trying to digest it all

@Stephen and Manao
The error handling is probably MY fault. Avisynth32 has some custom code that passes the exceptions up the chain. Unfortunately, it doesn't play nice with x64's idea of not needing segmentation. There are cases where the error simply doesn't make it to the top of the chain because the custom error passing has been killed and a suitable replacement was never coded. I just never found myself running into quirky errors that weren't a result of code that I was currently in the process of debegging.

I was also the only filter writer at that time, so, not including descriptions to go with errors I knew the cause of didn't seem like a huge deal. Now that more people are playing with the source / targeting the build it seems pertinent to re-introduce this functionality, with a more future-proof method of error handling. From the top of my head, I think Avisynth 2.6 still attaches their custom handler to a segment register, which (I don't think) plays well with x86-64.

@Manao specifically
Any chance of getting a repository for masktools set up somewhere? I may be behind the times here, but I think most of the source is still hosted by you personally. I'm in the process of offloading all my personal filter mods into my repository, as hardly any of the source is easy to find for the modified filters. Say the word, and I think I can get you access / space to save your source. At the very least, data redundancy can be your friend.

@Turbojet
On top of things as always, thanks for maintaining the installer so vigilantly. I'm mirroring it over to the main link whenever I get done cleaning my repository / adding the filters.

@kemuri-_9
That's a wonderful bit of irony on the compiler. The one compiler that adheres to the ABI is open source and screws the code from the commercial compilers. You see this happen all too often, if people would just follow the spec set forth in the first place, we wouldn't be dumbfounded when we try and use code from ___ amount of time ago and realize it's non-functional. Fixing is in progress . . .

@Stephen
The abuse of the registers in the first place only added ~0.5% to the overall performance. I was just being a kid in a candy shop with my registers. There are few things more painful to a programmer than looking at their own code from months ago, and I'm in the midst of this right now. With some smart optimization, the removal of their usage may improve performance by reducing code size and just allowing the processor to register-rename at the hardware level. It's all a matter of how the architecture handles stalls due to data dependency. Intel got really smart about this (was forced to learn a lot) when they tried to make the Netburst execution pipeline ridiculously long. Now that I'm free to target architectures that benefit from their lessons, I'm seeing very few places where the processor balks at my compliant code.

[Manao]

JoshyD : The error handling was my fault (a division by zero that started occurring when I added Y8 / YV16 / YV24 support some times ago). And if you can provide write access to a repository, I'm all for it :)

[Stephen R. Savage]

Incidentally, Avisynth64 presents a good excuse to use the try:catch statement as it was originally intended.

Say, you wanted to share a common script between 32-bit and 64-bit Avisynth without putting all your plugins in autoload. You could use try:catch to load 32-bit plugins if the 64-bit plugins aren't recognized. This is obvious, but still useful.


try { LoadPlugin("64-bit.dll" }
catch(err_msg) { LoadPlugin("32-bit.dll") }

CrossPlatformFunction()


On another note, it appears that LoadPlugin() returns either a string corresponding to a description of the plugin or the Undefined() value. The return value of Import() when last is not defined in the script is the same. Can any AVS developer clarify on how the return value of LoadPlugin() works?

[Manao]

Stephen : masktools in my signature should fix your last issue.

[Stephen R. Savage]

Stephen : masktools in my signature should fix your last issue.

I can confirm that mt_masktools 2.0a41 is stable with all scripts tested. I believe they cover mt_lut, mt_lutxy, mt_edge, mt_merge, mt_inpand/expand, and mt_logic.

Congratulations on being the second (?) developer to port one of his projects to Avs64.

Edit @JoshyD:
I just finished my first non-trivial encode (dfttest, MaskTools, MVTools, aWarpSharp, GradFun2DB, AddGrainC) using Avisynth64. It completed 10% faster than previous encodes using the 32-bit release. Good work!

Also, could you link to Manao's mt_masktools64 on the first post of this thread?

[Gavino]

On another note, it appears that LoadPlugin() returns either a string corresponding to a description of the plugin or the Undefined() value. The return value of Import() when last is not defined in the script is the same.
LoadPlugin returns the string returned by the plugin's initialisation function AvisynthPluginInit2(), or an Undefined() value if the string is null or the plugin cannot be loaded. Conventionally, this string is a description provided by the plugin writer.

The return value of Import follows the same rule for function bodies and scripts in general. See here (http://avisynth.org/mediawiki/The_full_AviSynth_grammar#Closing_Remarks).

[Stephen R. Savage]

@JoshyD: There is a bug in the horizontal resizer code for the non-sinc resizers (point, bilinear, bicubic).

The following command will trigger an exception:

PointResize(last.width, last.height, src_left=2)

[Hiritsuki]

Can anyone help me build dedup x64 ver. ?
=> http://akuvian.org/src/avisynth/dedup/