SEt's Avisynth 2.5.8 MT compiled for *X86_64*, Latest Build 4/16/2010

[jpsdr]

Two things :
1 :
In the first page, you said that for Dgindex, not all the idct mode are present. Can you specify which ones are present actualy ?
2 :
I've tested the automaptt3 avs script wich use the the masktools 2.0a with x64, result is a VDub crash (32bit version works fine).

Otherwise, thanks for your work, and good luck for the following.

[trevaaar]

Any chance you could post the avisynth.h/avisynth.lib from your port? I'm interested in compiling hqdn3d so TemporalDegrain will work. Only have VS2008, not ICL.

Also, I'm getting a crash with MCTemporalDenoise(settings="low",deblock=false) in Veedub64. It opens up alright but as soon as you try to step forward a frame or save the results it dies. I'm on an i7 920 if that makes any difference.

[JoshyD]

@Stephen R. Savage
As always very useful feedback, thanks for taking the time to create a simple test case. I've got some idle cycles today, and hopefully should get that shored up for you in no time. I was constantly playing with the way masktools handles the stack when compiling the 64 bit version, there's a chance it's already fixed (lucky me).

@Trevaaar
You can find my code and changes here. (http://code.google.com/p/avisynth64) If you just need the header, it's in trunk/src No real major changes were made save for some new CPUID's and compliance with any new operator wanting to be "size_t" instead of unsigned int when compiling for x64. I just looked at the source for hqdn3d, and it's a really easy compile. No inline asm or other funny business. I figured I already had it built so I'm just going to post it here. (http://www.mediafire.com/?gyvmmzx0v4z) Visual inspection looks like it's working . . .

Also, with the crash, what function are you using to open the source? I still haven't had time to sufficiently play with MCTemporalDenoise, hopefully I'll get it all ironed out soon, it's a relatively popular script.

@jpsdr
I'm not familiar with automaptt3 script, but perhaps Stephen's bug is related to some of the functionality employed in both cases. We'll see if I can't get this all straightened out.

[jpsdr]

Well, the automaptt3 script was just to report, as i'm not using it for now, it was just a test i've made out of curiosity. i'll post it here later if you want to test also.
Nevertheless, i'm greatly more interested to know what IDCT are implemented in DGDecode, because this one i intend to use it ! (i've begin to port my VDub filters to x64 for this)

[Stephen R. Savage]

Alright, I've gotten a DgDecode 1.5.8 built and running correctly on my system. It will only decode with SSE2 or SKALSSE algorithms for now, I'll work on getting the others back. Link is on the main page. It's not exactly x64 optimized, but at least it's the latest build.

This post. SKALSSE corresponds to idct=6. Not sure about the other. It doesn't matter, because all IDCT modes have an SSIM higher than 0.999.

[jpsdr]

Ok, thanks. I'll wait for the IDCT IEEE ... don't remember exactly.

[JoshyD]

The IDCT modes that are working are 6 and 3, the IEEE full floating point spec is written in some nasty floating point inline assembler. I've been kind of putting it off, but I guess if there's some demand . . . ;)

[Mr VacBob]

3 should be the fastest IDCT, no point in using anything else.

(If you replaced Skal with the SSE2 Skal in ffmpeg it would be much faster than that, though)

[Stephen R. Savage]

The IDCT modes that are working are 6 and 3, the IEEE full floating point spec is written in some nasty floating point inline assembler. I've been kind of putting it off, but I guess if there's some demand . . . ;)

On the other hand, wouldn't this one benefit the most from porting to 64-bit (and not just because of registers)?

[JoshyD]

DgDecode is using sse2 skal from somewhere, probably ffmpeg as you mentioned.

The thing about it's highest precision mode, is it does 80-bit precision floating point calculations, which means we're still tied down to the floating point registers (which are architecturally the mmx registers, plus some bits that mmx doesn't use). I'm not too well versed on assembly before all floating point calcs were done in either mmx or sse registers, but it definitely appears the highest precision mode of dgdecode is using the 80 bit extended registers. It's not exactly going to see any speed gains from vectorized instructions or that kind of thing.

On a lighter note, I got your bug ironed out, and finally wrote the horizontal resize code to work with the SSE registers instead of MMX. I'm going to poke it and prod it a bit, but should be up to tinker with this weekend.

[trevaaar]

Thanks for the build of hqdn3d!

The source I'm using in the script that crashes is AviSource with a huffyv12-through-ffdshow AVI. Works fine on its own.

[jpsdr]

I personnaly seek for the most precise and accurate, so, i'm only using this mode.
Can you send me the source code you've already ported, or put link on first page ?

[Stephen R. Savage]

I personnaly seek for the most precise and accurate, so, i'm only using this mode.
Can you send me the source code you've already ported, or put link on first page ?

If you wanted the most accurate output, you would use one of the MMX or SSE transforms and not the IEEE reference. This is because nothing is encoded with the IEEE defined DCT, and therefore you will lose approximately 0.0005 SSIM by using the IEEE.

[JoshyD]

I personnaly seek for the most precise and accurate, so, i'm only using this mode.
Can you send me the source code you've already ported, or put link on first page ?

Here's the source I'm building from. (http://www.mediafire.com/?o23z43nmj1x) If you do a wholesale diff on the directory with something like WinMerge, it's pretty easy to see where it's been hacked apart to work with x64, just look for #ifndef _AMD64_ for most of the 64bit only mods.

I'm sure everyone would be very appreciative if you could bring the floating point assembly up to 64bit compatibility, or just improve the source in general. A word of caution, it is inline asm, and can't be built with MSVS's C++ compiler. If you want to rewrite it, I'll more than happily compile it with Intel's compiler, which still happily accepts 64bit inline asm. As of now, I'm just not building with the fp source files at all.

They could be converted to C++ using the "long double" type apparently, that's one idea. Another idea is that Intel offers 30 day evals of their compilers, you could use that if you really want to compile the inline assembly.

I'm having a bit of trouble finding good documentation on the 8087+ fp implementation. It seems that through the years the 80bit format has been mostly dropped in favor of 64bit double precision computations.

Wikipedia articles, while not being the most credible sources have some insight into fp standards without being an IEEE member. The last IEEE754 spec appears to have been released in 2008.

Anyhow this may give insight into 80 bit computation (http://en.wikipedia.org/wiki/Extended_precision) and this gives an idea of how quad precision (http://en.wikipedia.org/wiki/Quadruple_precision) fp is supposed to work. However, current hardware implementations apparently do not support quad precision calculations directly (at least according to wikipedia!)

I'm fairly certain that the AMD64/EMT64 ABI's don't allow manipulation of things like the fp control word and will throw an exception when trying to even use _control87() in C++. It would appear that the powers that be are trying to move everything to a power of 2 standard (32, 64, or 128 bit fp computations).

Corrections, information, and ideas on these concepts is definitely appreciated.

If you wanted the most accurate output, you would use one of the MMX or SSE transforms and not the IEEE reference. This is because nothing is encoded with the IEEE defined DCT, and therefore you will lose approximately 0.0005 SSIM by using the IEEE.

Interesting point, and food for thought. I'm pretty sure straight up 80 bit calculations went the way of the dinosaur many moons ago. The DgDecode website lists IEEE 1180 encode/decode as the most accurate, but your point makes definite sense. Who encodes with 80 bit fp calcs? Most programming conventions I'm aware of obscure this format even from the assembly programmer. There are no modern MMX/SSE instructions that even attempt to support this, and to my knowledge, GPU's skipped it entirely. It's from the 8087, which was err, something like 30 years ago? If you debug C at the assembly level, modern compilers do all their FP work in the SSE registers, at least from what I've seen. I've never personally run into disassembled code that still makes reference to the old FP stack way of doing things.

Sticking with 64bit or 32bit calcs also allows offloading them to the GPU anyhow, this may be what DgNVDecode does. An open sourced DirectX or OpenGL / CL version of these calcs may be of more interest / use at the moment.

[jpsdr]

My thoughts are, but i may be wrong :
- IEEE (ect...) it strictly compliant to the MPEG-2 spec, and ways of doing things. Encoders are supposed to be strict compliant to the spec too.

Something else : A long time ago, i've made convertions with mpeg2avi. This SW propose around the same options of decoding than DGIndex.
At that time, i was wondering why my second convertions looked worse than my first, before realising that my 2nd convertions where done with mmx mode, and 1rst with IEEE... But the mmx was faster.

I'll take a look out of curiosity. For now, i'm porting my own VDub filters under x64, but, after this, i may take a look at this part, as i'm intersted in it.
I've in my youth made my own math library under DOS (turbo pascal) in ASM using FPUx87 instructions, so, i'ill probably recall old memories when i'll take a look.

Thanks.

[Stephen R. Savage]

MaskTools 2.0a37 is now available. I'm looking forward to an updated 64-bit version!

[levi]

using avisynth64_4-10-10 I still get warnings:


mpeg2source("my.d2v")
tfm(d2v="my.d2v")
tdecimate()
crop(16,8,1884,1072)
Lanczos4Resize(1280,720)



avs [info]: 1280x720p 0:0 @ 24000/1001 fps (cfr)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.2
x264 [info]: profile Main, level 3.1
x264 [warning]: non-strictly-monotonic pts at frame 1 (0 <= 0)
x264 [warning]: non-strictly-monotonic pts at frame 2 (0 <= 1)
x264 [warning]: non-strictly-monotonic pts at frame 3 (0 <= 2)
x264 [warning]: too many nonmonotonic pts warnings, suppressing further ones

[Stephen R. Savage]

My thoughts are, but i may be wrong :
- IEEE (ect...) it strictly compliant to the MPEG-2 spec, and ways of doing things. Encoders are supposed to be strict compliant to the spec too.

Quite the opposite. In fact, the MPEG-2 standard doesn't define the DCT at all! What this results in is every encoder using its own arbitrary-precision implementation of the DCT, which is also likely not the same one the decoder uses. Since it is incredibly unlikely that any encoder would use the very slow IEEE transform, there is no benefit in using it to decode. It was not until H.264 that we actually saw one specific DCT implementation mandated by the standard.

Edit: Also, JoshyD, could you move the plugins to the googlecode page and clean up some of the old cruft from the first post? It's getting hard to parse with the layers of outdated information. Perhaps you could copy the relevant information to documentation and changelogs on the googlecode project.

[osgZach]

That's my fault... dragging my feet on his web site. I've been sick for a week, just getting over it.

But good news!! Lost even more hours and pay at one of my jobs, so I will have more free time to get working on it! :D


edit:

I have started preleminary work on building up the site pages. I got WordPress 2.92 installed and put up a rather boorish "About" page.. Still have some work to do though, gotta find the right plugins and perhaps and spruce up the default theme and layout a little bit.. JoshyD isn't very web server savvy I think, so I need to come up with an easy way for him to add and update downloadable files for the main releases and x64 filter conversions, etc...

I'm still recommending you go with a main install-pack that gets the guts put in, and then a basic plugins pack that has all the plugins inside it. Then you can do staged update packs in the future, or have individual plugin pages as warranted for ongoing popular things.. We'll get it all sorted out in any case ;)

http://joshyd.oldskoolgames.com

[jpsdr]

Here the script of automttap3


#modified LanczosPlus using Masktools 2.0
#LanczosPlus is now LanczosmtPlus()
#autotap3 is now automttap3
# Optimisations by: wizboy11 and Didée
function automttap3(clip clp, int "dx", int "dy", int "mtaps3", int "thresh"){

c = clp
#g = clp.converttoy8()

dx = default(dx, c.width*2)
dy = default(dy, c.height*2)
mtaps3 = default(mtaps3, 1)
thresh = default(thresh, 256)

t1 = lanczosresize(clp, dx, dy, taps=1)
t2 = lanczosresize(clp, dx, dy, taps=2)
t3 = lanczosresize(clp, dx, dy, taps=3)
t4 = lanczosresize(clp, dx, dy, taps=4)
t5 = lanczosresize(clp, dx, dy, taps=5)
t6 = lanczosresize(clp, dx, dy, taps=9)
t7 = lanczosresize(clp, dx, dy, taps=36)


m1 = mt_makediff(clp, lanczosresize(t1, clp.width, clp.height, taps=1), u=1, v=1)
m2 = mt_makediff(clp, lanczosresize(t2, clp.width, clp.height, taps=1), u=1, v=1)
m3 = mt_makediff(clp, lanczosresize(t3, clp.width, clp.height, taps=1), u=1, v=1)
m4 = mt_makediff(clp, lanczosresize(t4, clp.width, clp.height, taps=2), u=1, v=1)
m5 = mt_makediff(clp, lanczosresize(t5, clp.width, clp.height, taps=2), u=1, v=1)
m6 = mt_makediff(clp, lanczosresize(t6, clp.width, clp.height, taps=3), u=1, v=1)
m7 = mt_makediff(clp, lanczosresize(t7, clp.width, clp.height, taps=6), u=1, v=1)


cp1 = mt_merge(t1.blur(1.42), t2, lanczosresize(mt_lutxy(m1, m2, "x 128 - abs y 128 - abs - "+string(thresh)+" *", u=1, v=1), dx, dy, taps=mtaps3))
m100 = mt_makediff(clp, bilinearresize(cp1, clp.width, clp.height), u=3, v=3)

cp2 = mt_merge(cp1, t3, lanczosresize(mt_lutxy(m100, m3, "x 128 - abs y 128 - abs - "+string(thresh)+" *", u=1, v=1), dx, dy, taps=mtaps3))
m101 = mt_makediff(clp, bilinearresize(cp2, clp.width, clp.height), u=3, v=3)

cp3 = mt_merge(cp2, t4, lanczosresize(mt_lutxy(m101, m4, "x 128 - abs y 128 - abs - "+string(thresh)+" *", u=1, v=1), dx, dy, taps=mtaps3))
m102 = mt_makediff(clp, bilinearresize(cp3, clp.width, clp.height), u=3, v=3)

cp4 = mt_merge(cp3, t5, lanczosresize(mt_lutxy(m102, m5, "x 128 - abs y 128 - abs - "+string(thresh)+" *", u=1, v=1), dx, dy, taps=mtaps3))
m103 = mt_makediff(clp, bilinearresize(cp4, clp.width, clp.height), u=3, v=3)

cp5 = mt_merge(cp4, t6, lanczosresize(mt_lutxy(m103, m6, "x 128 - abs y 128 - abs - "+string(thresh)+" *", u=1, v=1), dx, dy, taps=mtaps3))
m104 = mt_makediff(clp, bilinearresize(cp5, clp.width, clp.height), u=3, v=3)

cp6 = mt_merge(cp5, t7, lanczosresize(mt_lutxy(m104, m7, "x 128 - abs y 128 - abs - "+string(thresh)+" *", u=1, v=1), dx, dy, taps=mtaps3))
m105 = mt_makediff(clp, bilinearresize(cp6, clp.width, clp.height), u=3, v=3)

return(mergechroma(cp6, spline36resize(clp, dx, dy), 1))}

###
###
###

function Lanczosmtplus( clip c, int "dx", int "dy", int "thresh", int "thresh2", int "athresh",
\ float "sharp1", float "sharp2", float "blur1", float "blur2",
\ int "mtaps1", int "mtaps2", int "ttaps", int "ltaps", bool "preblur",
\ int "depth", float "wthresh", float "wblur", int "mtaps3"){

dx = default(dx, c.width*2)
dy = default(dy, c.height*2)
thresh = default(thresh, 0)
thresh2 = Default(thresh2, ((thresh+1)*(thresh+1))*64/(thresh+1))
sharp1 = default(sharp1, 1)
sharp2 = default(sharp2, 4)
blur1 = default(blur1, 0.33)
blur2 = default(blur2, 1.25)
mtaps1 = default(mtaps1, 1)
mtaps2 = default(mtaps2, 1)
ttaps = default(ttaps, 1)
ltaps = default(ltaps, 1)
depth = default(depth, 2)
wthresh = default(wthresh, 0.90)
wblur = default(wblur, 2)
mtaps3 = default(mtaps3, 1)
preblur = default(preblur, false)
athresh = default(athresh, 256)

c
s = c

fd1 = lanczosresize(c, dx, dy, taps=mtaps1)
fre1 = fd1.lanczosresize( c.width, c.height, taps=mtaps1)
fre2 = fre1.automttap3( ((c.width/16)*8 >= 144 ? (c.width/16)*8 : 144), ((c.height/16)*8 >= 144 ? (c.height/16)*8 : 144), \
mtaps3=mtaps3, thresh=athresh).automttap3( c.width, c.height, mtaps3=mtaps3, thresh=athresh)
m1 = mt_lutxy(fre1, s, "x y - abs "+string(thresh)+" - "+string(thresh2)+" *", u=1, v=1)
m2 = lanczosresize(lanczosresize(frfun7(m1, 2.01, 256, 256), (c.width/16)*8, (c.height/16)*8, taps=ttaps), \
dx, dy, taps=ttaps)

d = preblur ? mt_merge(c, fre2, m1) : c
d2 = d.automttap3(dx, dy, mtaps3=mtaps3, thresh=athresh)
d3 = lanczosresize(lanczosresize(d, c.width, c.height, taps=ttaps), \
d2.width, d2.height, taps=ttaps)
d4 = mt_merge( mt_lutxy(d2, d3 , "x y - "+string(sharp1)+" * x +" , u=1, v=1), \
mt_lutxy(d2, d3 , "y x - "+string(blur1)+" * x +" , u=1, v=1), m2)
d5 = d4.automttap3(d.width, d.height, mtaps3=mtaps3, thresh=athresh)

e = mt_merge(d5, c, m1).automttap3(dx, dy, mtaps3=mtaps3, thresh=athresh)
e = e.awarpsharp(depth=depth, cm=0, blurlevel=wblur, thresh=wthresh)
e = e.awarpsharp(depth=depth, cm=0, blurlevel=wblur, thresh=wthresh)
e = e.awarpsharp(depth=depth, cm=0, blurlevel=wblur, thresh=wthresh)
e = e.awarpsharp(depth=depth, cm=0, blurlevel=wblur, thresh=wthresh)


fd12 = lanczosresize(e, ((e.width*(dx/c.width))/16)*16, ((e.height*(dy/c.height))/16)*16, taps=mtaps2)
fre12 = fd12.lanczosresize( e.width, e.height, taps=mtaps2)
fre22 = fre12.lanczosresize( (e.width/16)*8, (e.height/16)*8, taps=mtaps2).lanczosresize( e.width, e.height, taps=mtaps2)
m12 = mt_lutxy(fre12, e, "x y - abs "+string(thresh)+" - "+string(thresh2*1)+" *", u=1, v=1)
m12 = m12.lanczosresize( (e.width/16)*8, (e.height/16)*8, taps=mtaps2).lanczosresize( e.width, e.height, taps=mtaps2)


e2 = lanczosresize(lanczosresize(e, c.width, c.height, taps=ltaps), \
e.width, e.height, taps=ltaps)
e2 = e2.awarpsharp(depth=depth, cm=0, blurlevel=wblur, thresh=wthresh)
e2 = e2.awarpsharp(depth=depth, cm=0, blurlevel=wblur, thresh=wthresh)
e2 = e2.awarpsharp(depth=depth, cm=0, blurlevel=wblur, thresh=wthresh)
e2 = e2.awarpsharp(depth=depth, cm=0, blurlevel=wblur, thresh=wthresh)

e3 = mt_merge( mt_lutxy(e, e2 , "y x - "+string(blur2)+" * x +" , u=1, v=1), \
mt_lutxy(e, e2 , "x y - "+string(sharp2)+" * x +" , u=1, v=1), \
m12)
e3 = e3.awarpsharp(depth=depth, cm=0, blurlevel=wblur, thresh=wthresh)
e3 = e3.awarpsharp(depth=depth, cm=0, blurlevel=wblur, thresh=wthresh)
e3 = e3.awarpsharp(depth=depth, cm=0, blurlevel=wblur, thresh=wthresh)
e3 = e3.awarpsharp(depth=depth, cm=0, blurlevel=wblur, thresh=wthresh)


mt_merge(d4, e3, m2)

return(last.mergechroma(spline36resize(c, dx, dy), 1))}

[noee]

What is "frfun7" in the script above (automttap3, line 90)? That appears to be a missing component for 64bit.

Edit:

Okay, it appears to be a function in hqdn3d.dll, which I have added to the plugins64 folder. I still get an error when trying to run "Lanczosmtplus" above, saying "no function named frfun7".

[kemuri-_9]

What is "frfun7" in the script above (automttap3, line 90)? That appears to be a missing component for 64bit.

Edit:

Okay, it appears to be a function in hqdn3d.dll, which I have added to the plugins64 folder. I still get an error when trying to run "Lanczosmtplus" above, saying "no function named frfun7".

frfun7 is separate filter: a quick search lead to this thread (http://forum.doom9.org/showthread.php?t=110200) and this binary (http://avisynth.org/warpenterprises/files/frfun_25_dll_20060510.zip)
haven't seen the source lying around anywhere so won't be able to be ported...

[Stephen R. Savage]

Build 4/10/2010 displays highly incorrect results for horizontal resizing in YUV 4:2:0. No artifacts are seen in RGB. I tested a resize of 1920x1080 -> 704x1080 and 1920x1080 -> 704x480. Both show strong tearing artifacts in all resizers.

[ryrynz]

To be fair Stephen, this isn't an official release.

I'm sure JoshyD tests all he can given he most likely has other projects.

You shouldn't really ask that sort of a question from someone who is doing this in their free time, free of charge for the benefit of the community.

[kemuri-_9]

It's also impossible for one person to test every scenario that can occur with avisynth.

[jpsdr]

What is "frfun7" in the script above (automttap3, line 90)? That appears to be a missing component for 64bit.


Very strange... For me, when i load a script using automttap3,
VDub crash, nothing is telling me that there is an unknow function.
I'll check for the hqdn3d.dll in my 32 bits version, but i don't remember having this, so, i need checking.

[osgZach]

You're a knowledgeable fellow Stephen, feel free to volunteer helping him out with stuff...

[Zep]

Do you actually test your builds before releasing them? I don't mean to be rude, but this seems to be blatantly obvious.

But you are being rude. How about you simply point out problem and say thanks.

[turbojet]

I did some benchmarks of the common filters I use and a few slower filters here's the numbers (messy I know but I couldn't think of a better way). Trends and notes below may make more sense

1080p lanczosresize(1280,720) --ref 4 --psy-rd 1.0:0.2 2pass
ver 32bit 32mt 64bit 32sm2 64sm2 32sm4 64sm4 32mt2 64mt2
2.57 73.71 70.79 72.23 58.60 - 40.43 - 70.75 -
2.58 74.47 70.41 79.35 60.25 61.14 43.34 41.10 71.43 76.41
2.60 81.11 74.73 - 60.53 - 43.84 - 72.16 -

1080p --ref 4 --psy-rd 1.0:0.2 2pass
ver 32bit 32mt 64bit 32sm2 64sm2 32sm4 64sm4 32mt2 64mt2
2.57 45.60 45.68 47.44 40.30 - 31.36 - - -
2.58 45.06 45.77 46.80 40.83 40.66 37.54 38.33 - -
2.60 43.48 45.79 - 41.02 - 37.10 - - -

1080p --preset medium
ver 32bit 32mt 64bit 32sm2 64sm2 32sm4 64sm4 32mt2 64mt2
2.57 17.27 17.64 18.39 16.14 - 14.84 - - -
2.58 17.76 17.60 18.14 16.35 16.22 14.94 15.32 - -
2.60 17.89 17.89 - 16.13 - 15.72 - - -

1080p lanczosresize(1280,720) --preset medium
ver 32bit 32mt 64bit 32sm2 64sm2 32sm4 64sm4 32mt2 64mt2
2.57 35.49 34.32 35.36 26.75 - 24.23 - 33.86 -
2.58 35.90 34.20 37.27 27.76 29.98 21.66 21.89 34.29 36.29
2.60 37.13 35.33 - 28.71 - 22.14 - 34.65 -

480i TFM().TDecimate() --preset medium
ver 32bit 32mt 64bit 32sm2 64sm2 32sm4 64sm4 32mt2 64mt2
2.57 86.62 87.03 86.97 80.27 - 72.92 - 84.59 -
2.58 86.95 86.62 89.69 81.01 86.97 68.85 84.41 85.14 86.97
2.60 86.82 86.70 - 80.16 - 73.56 - 85.06 -

480i TDeint() --preset medium
ver 32bit 32mt 64bit 32sm2 64sm2 32sm4 64sm4 32mt2 64mt2
2.57 78.07 77.94 87.51 83.57 - 76.54 - 86.92 -
2.58 77.26 78.04 89.70 83.88 89.70 72.44 78.00 44.07 44.30
2.60 77.49 77.91 - 83.21 - 71.94 - 39.70 -

480i LeakKernelDeint(order=1) --preset medium
ver 32bit 32mt 64bit 32sm2 64sm2 32sm4 64sm4 32mt2 64mt2
2.57 103.0 103.7 105.5 71.94 - 51.66 - 79.23 -
2.58 102.1 102.1 108.7 69.80 78.00 50.42 52.00 79.81 78.00
2.60 102.6 103.2 - 70.57 - 49.58 - 79.31 -

480i TGMCmod_beta1(eedi2) --preset medium
ver 32bit 32mt 64bit 32sm2 64sm2 32sm4 64sm4 32sm6 64sm6 32mt2 64mt2 32mt4 64mt4
2.57 3.62 3.88 *1 4.56 - 7.97 - 11.98 - 7.76 - 11.86 -
2.58 3.87 3.86 4.96 4.19 5.83 8.30 10.66 12.63 16.16 7.46 9.36 11.48 14.11
2.60 3.69 3.87 - 4.18 - 8.02 - 12.03 - 7.11 - 10.92 -

480i LSFMod --preset medium
ver 32bit 32mt 64bit 32sm2 64sm2 32sm4 64sm4 32mt2 64mt2
2.57 42.12 41.81 *1 *2 *2 *2 - 64.23 -
2.58 41.62 41.76 62.95 *2 *2 *2 *2 64.59 70.35
2.60 50.93 42.56 - *2 - *2 - 64.19 -

Legend
32bit: 32 bit sourceforge version
32mt: 32 bit MT version
64bit: 64 bit (2.57 isn't MT, 2.58 is MT, no 2.60 build)
32sm2: 32 bit SetMTMode(2,2)
64sm2: 64 bit SetMTMode(2,2)
32sm4: 32 bit SetMTMode(2,4)
64sm4: 64 bit SetMTMode(2,4)
32sm6: 32 bit SetMTMode(2,6)
64sm6: 64 bit SetMTMode(2,6)
32mt2: 32 bit MT(2,2)
64mt2: 64 bit MT(2,2)

*1: temporalsoften unrecognized function in 64 2.57 MT
*2: SetMTMode().LSFMod() deadlocks with 25% CPU usage (only 1 thread serving?)

Trends:
- avisynth.dll from mt is slower when resizing
- Without slower filters MT() is slow SetMTMode() is slower
- with slower filters SetMTMode is very fast, MT() is fast
- 64 2.58 MT is faster except when resizing with fast x264 settings (loss from MT? coreavc?)

Notes:
- 2.58 MT and 2.60 MT have issues with MT.dll and TDeint, 2.57 MT doesn't
- 2.57 MT is often faster but sometimes slower then 2.58 MT, 2.60 MT
- 2.60 MT randomly caused 3 BSODs

All sources were AVC decoded by CoreAVC 2.0 on a quad core cpu. All used 95%+ cpu except SetMTMode.TGMC: 1 thread 25% 2=33% 4=66% 6=99%
MT(TGMC): 1 thread 30% 2=55% 4=90% 5=85% 6=85%
MT(LSFMOd): 1 thread=60% 2=90%

[turbojet]

Concerning 64 bit plugins, some organization with them would be nice. Alphabetically? Category+Alphabetically?

On the other hand, if you want to include the dll's in a separate directory with new versions the install script could easily move them for auto loading. Or just one big pack, downloading them individually is kind of annoying to begin with but easier updating I guess.

[JoshyD]

@Stephen
I noticed the artifacts last night myself, I'm currently looking into this. I did test with a variety of resizes before releasing, but apparently missed some corner cases. I'll revert the code for now, and come back soon with something more robust. It's hard to write the code and then test with all possible input variations. It really is hard to code and then extensively test all cases in the free time I do have. I'm sorry I screwed up this time, but, if I don't put out the code I do have, I'll never find some of the oddities that arise from code changes. I got excited on this one and jumped the gun.

For perspective, taking an already large project and just understanding it well enough to make minor modifications to it is difficult. There's not a million comments through the code, and you have to use a little intuition and some educated guesses as to the intended end result of the function. There's still oddities in the original source for 2.5.8 (Temporal::Soften accumulate_line_mode2 is commented as having funny results regardless of variable choice for the inline asm) and there were definitely times I found myself looking at the source and saying "what??". There's actually a pretty funny comment in squid_80's 2.5.7 64 bit port in this vein.
void Assign(const AVSValue* src, bool init) {
if (src->IsClip() && src->clip)
src->clip->AddRef();
if (!init && IsClip() && clip)
clip->Release();
// make sure this copies the whole struct!
/* UGH! what the- ?
((__int32*)this)[0] = ((__int32*)src)[0];
((__int32*)this)[1] = ((__int32*)src)[1];
*/
this->clip = src->clip;
this->type = src->type;
this->array_size = src->array_size;
}

I did the same thing when originally looking at the code, then compared the function to squid's conversion, and realized, yes, this was weird.

As per usual, older builds that do function as they should remain available. If I screw it up, roll back for the time being. It's a crappy answer, but it's the best I can do.

I guess kind of take this as the community took 2.6alpha2. That was released kind of as a community test. When you're just a few guys (the avisynth team) or just one guy (me) having the code put through the paces by the community at large expedites the process of bug finding / squashing.

As for personal testing methodology, I have a few core files I work from, and a few scripts that use a variety of filters, both internal and external. They usually turn up my blunders, but this time, they failed to do so. Apparently, I need to vary my sources / tests more.

@Turbojet
Thank you for your time running the comparisons . . . they're a nice insight into the good and bad of the various builds, as well as the pro's and cons of threading. As a point of reference, can you list the machine specs, I may tabularize these in a wiki page, if that's alright.

The problem with the way avisynth's cache was hacked around is that single threaded performance inherently suffers. The added complexity to a core component of the program eats away at overall performance.

As to the thread deadlock, I'll see if I can make it happen. I've had it run out of memory on me before, which was the root of the problem, but I'm guessing something popped up complaining about a deadlock? I have had this happen on occasion when I've done something stupid/illogical in the code.

The filters *DO* need to be organized, eventually I may just redirect to the google code hosted project, and format a wiki page there, for now, a clean-up of the first post is in order . . . after I figure out why the resizer is eating it. I'm fairly positive it's the horizontal part that's going awry, as the vertical code didn't change.

@levi
The monotonic points problem appears to be related to the caching. I get the error when trying to run through code with the regular cache, but it's all rosy when using SetMTMode(). It's definitely the frame rate information getting lost in translation. For now though, it's taking the number 2 spot on the todo list, so I can shore up the resize code.

Only so many hours in the day, I'll come back with something more stable soon.

EDIT: First post in alphabetical order, cleaned, easier to read . . . and the resize functionality is working properly, at least on my end, the clip that was showing artifacts last night no longer does.

[Andrey /MAG/]

Hello.
Still reminding about 'blending bug' in MVTools.
Maybe I can do something to solving this problem?
What the source of the problem and how can I help to fix it?

[Delerue]

Hello.
Still reminding about 'blending bug' in MVTools.
Maybe I can do something to solving this problem?
What the source of the problem and how can I help to fix it?

The interesting thing is that I can't reproduce the problem you're talking about. Everything works like it should be (i.e. equal to the x86 version). Here's the script I'm using:


setMTMode(2,12)
LoadPlugin("%path%\mvtools2 x64.dll")
source = ffdshow_source()
super = source.MSuper(pel=1)
backward_vec = MAnalyse(super, blksize=8, dct=10, overlap=2, isb = true, search=3, searchparam=2)
forward_vec = MAnalyse(super, blksize=8, dct=10, overlap=2, isb = false, search=3, searchparam=2)
source.MFlowFps(super, ThSCD1=350, blend=false, backward_vec, forward_vec, \
num=2*FramerateNumerator(source), den=FramerateDenominator(source))
distributor()


I've tried to set blend to 'true' and also remove the blend parameter. It still works flawlessly. Can you show your script so we can help you?

[tormento]

Has anybody compiled a x64 version ov MVTools 1.x?

[Andrey /MAG/]

I can't reproduce the problem
The problem is in specific means of function parameters in script. See details (source, script and results) in zip-file: http://www.sendspace.com/file/4u8s0t

This fact is submitted by JoshyD in his post (http://forum.doom9.org/showpost.php?p=1387856&postcount=286).

[Delerue]

The problem is in specific means of function parameters in script. See details (source, script and results) in zip-file: http://www.sendspace.com/file/4u8s0t

This fact is submitted by JoshyD in his post (http://forum.doom9.org/showpost.php?p=1387856&postcount=286).

There seems to be a problem with your 'blksize' value. Here, if I change it to '16' instead of '32', it works like it should. Also the 'searchparam' value 6 looks like an overkill. I did a lot of tests here with MFlowFPS (including your sample), and beyond 'searchparam' value 3 there's no visible gain, but the CPU usage increases a lot. I can guarantee you that the script I told you before gives a better final result. Of course you can tweak some parameters in order to get a better performance. What you think? :)

[Blue_MiSfit]

I just wanted to say thanks to everyone who is working hard on this project. This is a substantial development, and is making huge strides in AviSynth performance.

THANK YOU!!!

~MiSfit

[7ekno]

I just wanted to say thanks to everyone who is working hard on this project. This is a substantial development, and is making huge strides in AviSynth performance.


+1, Awesome work guys, keep it up! Viva la 64bit!

Am sure there will be hiccups along the way and the more information provided here (as opposed to critisism) will help all x64 users as a whole!

7

[Stephen R. Savage]

Artifacts confirmed fixed. Thanks a lot, JoshyD. By the way, I didn't notice the new build when it was originally uploaded, as you did not make a post about it. This is not a problem, but is there any chance of getting a RSS feed or similar to follow progress?

Also, could you add the original version numbers for the plugins listed on the first page?

@tormento: Only mvtools2 is up, but why do you need mvtools 1.x? If you have a custom script that requires it, perhaps I could help you migrate your script to mvtools2.

@Avs developers: I notice that resizing RGB images causes the hue/saturation/brightness to shift. Is this normal, and can it be fixed?

[Andrey /MAG/]

problem with your 'blksize' value.
Yes. This value causes another behavior of 64-bit plugin than 32-bit. It shows that conversion from 32 into 64 was with some inaccuracies.
Here, if I change it to '16' instead of '32', it works like it should.
I have done frame-by-frame comparing with 'subtract' and found that difference still present.
the 'searchparam' value 6 looks like an overkill. I did a lot of tests here with MFlowFPS (including your sample), and beyond 'searchparam' value 3 there's no visible gain, but the CPU usage increases a lot.
In some cases 'searchparam=3' was too small to find long vectors. I still use MVTools 2.4.7 (32bit) and 'searchparam' produce not big increase of CPU usage.
MVTools 2.5.10 is not so fast as 2.4.7.

I can guarantee you that the script I told you before gives a better final result. Of course you can tweak some parameters in order to get a better performance. What you think? :)
Thank you. I compared it too. The results is prety well. Better than ones of my script.

[Hiritsuki]

I found a filter this can help someone video non-waste bitrate, it's Dedup to using to making VFR video.
and it's no x64 ver.

[Stephen R. Savage]

I have done frame-by-frame comparing with 'subtract' and found that difference still present.

This was addressed on page two or three, and is the result of SoftWire performing rounding differently from JoshyD's native assembly.

I found a filter this can help someone video non-waste bitrate, it's Dedup to using to making VFR video.
and it's no x64 ver.

I've found that Dedup doesn't contribute significantly at all to compression ratio.

[Hiritsuki]

I've found that Dedup doesn't contribute significantly at all to compression ratio.

Oh, using big bitrate to compress video like anime ,dedup is good use.

[Pho3n1x]

i have a problem about the ffms2.dll from kemuri-_9's FFMS2 (The Fabulous FM Source 2)
when i open the avs in virtualdib64 it comes no funtion about ffvideosource.

my script is

loadplugin("C:\Program Files (x86)\AviSynth 2.5\plugins64\ffms2.dll")
ffvideosource("E:\Encode\0_2.mkv")
crop(0,22,-0,-22)

The same script with directshowsource works well but not with the ffms2.dll.

[Zathor]

Replace LoadPlugin with LoadCPlugin and everything should be fine.

[Wilbert]

@Avs developers: I notice that resizing RGB images causes the hue/saturation/brightness to shift. Is this normal, and can it be fixed?
Does that also happen with the regular v2.58? Could you post a script which shows this?

[Stephen R. Savage]

Does that also happen with the regular v2.58? Could you post a script which shows this?

I can confirm this with 2.58 standard.

The script I used was:

a = ImageSource("test.png")
b = Spline36Resize(a, 1280, 720).Spline36Resize(a.width, a.height)
Interleave(a, b)

I tested this with the following image (after decoding to PNG format):

http://img684.imageshack.us/img684/7910/testjc.png

After resizing, the image is noticeably darker:

http://img651.imageshack.us/img651/446/shiftcxifb.png

This also appears to happen with BicubicResize and LanczosResize, though it is a bit difficult to see with bicubic due to all the blur. Converting to YV12 first does not result in the same problem.

[Hiritsuki]

Some anime had few stationary frame only music runing.
So, can anyone make this "dedup" to x64 ver. ? please ...:)

[osgZach]

You've already asked once. If someone has the time and energy they will eventually get to it. But you are currently the only one asking for it. Repeating your request will not make it happen faster, you'll just make people ignore you.

[Hiritsuki]

You've already asked once. If someone has the time and energy they will eventually get to it. But you are currently the only one asking for it. Repeating your request will not make it happen faster, you'll just make people ignore you.

Sorry about my post I just explain what that filter is useable :(