Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 11th April 2010, 11:26   #321  |  Link
noee
Registered User
 
Join Date: Jan 2007
Posts: 530
What is "frfun7" in the script above (automttap3, line 90)? That appears to be a missing component for 64bit.

Edit:

Okay, it appears to be a function in hqdn3d.dll, which I have added to the plugins64 folder. I still get an error when trying to run "Lanczosmtplus" above, saying "no function named frfun7".

Last edited by noee; 11th April 2010 at 11:42.
noee is offline   Reply With Quote
Old 11th April 2010, 14:28   #322  |  Link
kemuri-_9
Compiling Encoder
 
kemuri-_9's Avatar
 
Join Date: Jan 2007
Posts: 1,348
Quote:
Originally Posted by noee View Post
What is "frfun7" in the script above (automttap3, line 90)? That appears to be a missing component for 64bit.

Edit:

Okay, it appears to be a function in hqdn3d.dll, which I have added to the plugins64 folder. I still get an error when trying to run "Lanczosmtplus" above, saying "no function named frfun7".
frfun7 is separate filter: a quick search lead to this thread and this binary
haven't seen the source lying around anywhere so won't be able to be ported...
__________________
custom x264 builds & patches | F@H | My Specs
kemuri-_9 is offline   Reply With Quote
Old 11th April 2010, 22:12   #323  |  Link
Stephen R. Savage
Registered User
 
Stephen R. Savage's Avatar
 
Join Date: Nov 2009
Posts: 336
Build 4/10/2010 displays highly incorrect results for horizontal resizing in YUV 4:2:0. No artifacts are seen in RGB. I tested a resize of 1920x1080 -> 704x1080 and 1920x1080 -> 704x480. Both show strong tearing artifacts in all resizers.

Last edited by Stephen R. Savage; 16th April 2010 at 00:43.
Stephen R. Savage is offline   Reply With Quote
Old 12th April 2010, 01:04   #324  |  Link
ryrynz
Registered User
 
ryrynz's Avatar
 
Join Date: Mar 2009
Posts: 2,812
To be fair Stephen, this isn't an official release.

I'm sure JoshyD tests all he can given he most likely has other projects.

You shouldn't really ask that sort of a question from someone who is doing this in their free time, free of charge for the benefit of the community.
ryrynz is offline   Reply With Quote
Old 12th April 2010, 02:23   #325  |  Link
kemuri-_9
Compiling Encoder
 
kemuri-_9's Avatar
 
Join Date: Jan 2007
Posts: 1,348
It's also impossible for one person to test every scenario that can occur with avisynth.
__________________
custom x264 builds & patches | F@H | My Specs
kemuri-_9 is offline   Reply With Quote
Old 12th April 2010, 08:45   #326  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 1,621
Quote:
Originally Posted by noee View Post
What is "frfun7" in the script above (automttap3, line 90)? That appears to be a missing component for 64bit.
Very strange... For me, when i load a script using automttap3,
VDub crash, nothing is telling me that there is an unknow function.
I'll check for the hqdn3d.dll in my 32 bits version, but i don't remember having this, so, i need checking.
jpsdr is offline   Reply With Quote
Old 12th April 2010, 15:13   #327  |  Link
osgZach
Registered User
 
Join Date: Feb 2009
Location: Waterloo, WI - USA
Posts: 650
You're a knowledgeable fellow Stephen, feel free to volunteer helping him out with stuff...
osgZach is offline   Reply With Quote
Old 12th April 2010, 16:25   #328  |  Link
Zep
Registered User
 
Join Date: Jul 2002
Posts: 587
Quote:
Originally Posted by Stephen R. Savage View Post

Do you actually test your builds before releasing them? I don't mean to be rude, but this seems to be blatantly obvious.
But you are being rude. How about you simply point out problem and say thanks.
Zep is offline   Reply With Quote
Old 12th April 2010, 19:27   #329  |  Link
turbojet
Registered User
 
Join Date: May 2008
Posts: 1,840
I did some benchmarks of the common filters I use and a few slower filters here's the numbers (messy I know but I couldn't think of a better way). Trends and notes below may make more sense

Code:
1080p lanczosresize(1280,720) --ref 4 --psy-rd 1.0:0.2 2pass
ver  32bit 32mt  64bit 32sm2 64sm2 32sm4 64sm4 32mt2 64mt2
2.57 73.71 70.79 72.23 58.60   -   40.43   -   70.75   -
2.58 74.47 70.41 79.35 60.25 61.14 43.34 41.10 71.43 76.41
2.60 81.11 74.73   -   60.53   -   43.84   -   72.16   -

1080p --ref 4 --psy-rd 1.0:0.2 2pass
ver  32bit 32mt  64bit 32sm2 64sm2 32sm4 64sm4 32mt2 64mt2
2.57 45.60 45.68 47.44 40.30   -   31.36   -     -     -
2.58 45.06 45.77 46.80 40.83 40.66 37.54 38.33   -     -
2.60 43.48 45.79   -   41.02   -   37.10   -     -     -

1080p --preset medium
ver  32bit 32mt  64bit 32sm2 64sm2 32sm4 64sm4 32mt2 64mt2
2.57 17.27 17.64 18.39 16.14   -   14.84   -     -     -
2.58 17.76 17.60 18.14 16.35 16.22 14.94 15.32   -     -
2.60 17.89 17.89   -   16.13   -   15.72   -     -     -

1080p lanczosresize(1280,720) --preset medium
ver  32bit 32mt  64bit 32sm2 64sm2 32sm4 64sm4 32mt2 64mt2
2.57 35.49 34.32 35.36 26.75   -   24.23   -   33.86   -
2.58 35.90 34.20 37.27 27.76 29.98 21.66 21.89 34.29 36.29
2.60 37.13 35.33   -   28.71   -   22.14   -   34.65   -

480i TFM().TDecimate() --preset medium
ver  32bit 32mt  64bit 32sm2 64sm2 32sm4 64sm4 32mt2 64mt2
2.57 86.62 87.03 86.97 80.27   -   72.92   -   84.59   -
2.58 86.95 86.62 89.69 81.01 86.97 68.85 84.41 85.14 86.97
2.60 86.82 86.70   -   80.16   -   73.56   -   85.06   -

480i TDeint() --preset medium
ver  32bit 32mt  64bit 32sm2 64sm2 32sm4 64sm4 32mt2 64mt2
2.57 78.07 77.94 87.51 83.57   -   76.54   -   86.92   -
2.58 77.26 78.04 89.70 83.88 89.70 72.44 78.00 44.07 44.30
2.60 77.49 77.91   -   83.21   -   71.94   -   39.70   -

480i LeakKernelDeint(order=1) --preset medium 
ver  32bit 32mt  64bit 32sm2 64sm2 32sm4 64sm4 32mt2 64mt2
2.57 103.0 103.7 105.5 71.94   -   51.66   -   79.23   -
2.58 102.1 102.1 108.7 69.80 78.00 50.42 52.00 79.81 78.00
2.60 102.6 103.2   -   70.57   -   49.58   -   79.31   -

480i TGMCmod_beta1(eedi2) --preset medium
ver  32bit 32mt  64bit 32sm2 64sm2 32sm4 64sm4 32sm6 64sm6 32mt2 64mt2 32mt4 64mt4
2.57 3.62  3.88   *1   4.56    -   7.97    -   11.98   -   7.76    -   11.86   -
2.58 3.87  3.86  4.96  4.19  5.83  8.30  10.66 12.63 16.16 7.46  9.36  11.48 14.11
2.60 3.69  3.87    -   4.18    -   8.02    -   12.03   -   7.11    -   10.92   -

480i LSFMod --preset medium
ver  32bit 32mt  64bit 32sm2 64sm2 32sm4 64sm4 32mt2 64mt2
2.57 42.12 41.81  *1    *2    *2    *2     -   64.23   -  
2.58 41.62 41.76 62.95  *2    *2    *2    *2   64.59 70.35 
2.60 50.93 42.56   -    *2     -    *2     -   64.19   -
Legend
32bit: 32 bit sourceforge version
32mt: 32 bit MT version
64bit: 64 bit (2.57 isn't MT, 2.58 is MT, no 2.60 build)
32sm2: 32 bit SetMTMode(2,2)
64sm2: 64 bit SetMTMode(2,2)
32sm4: 32 bit SetMTMode(2,4)
64sm4: 64 bit SetMTMode(2,4)
32sm6: 32 bit SetMTMode(2,6)
64sm6: 64 bit SetMTMode(2,6)
32mt2: 32 bit MT(2,2)
64mt2: 64 bit MT(2,2)

*1: temporalsoften unrecognized function in 64 2.57 MT
*2: SetMTMode().LSFMod() deadlocks with 25% CPU usage (only 1 thread serving?)

Trends:
- avisynth.dll from mt is slower when resizing
- Without slower filters MT() is slow SetMTMode() is slower
- with slower filters SetMTMode is very fast, MT() is fast
- 64 2.58 MT is faster except when resizing with fast x264 settings (loss from MT? coreavc?)

Notes:
- 2.58 MT and 2.60 MT have issues with MT.dll and TDeint, 2.57 MT doesn't
- 2.57 MT is often faster but sometimes slower then 2.58 MT, 2.60 MT
- 2.60 MT randomly caused 3 BSODs

All sources were AVC decoded by CoreAVC 2.0 on a quad core cpu. All used 95%+ cpu except SetMTMode.TGMC: 1 thread 25% 2=33% 4=66% 6=99%
MT(TGMC): 1 thread 30% 2=55% 4=90% 5=85% 6=85%
MT(LSFMOd): 1 thread=60% 2=90%

Last edited by turbojet; 12th April 2010 at 19:30.
turbojet is offline   Reply With Quote
Old 12th April 2010, 19:35   #330  |  Link
turbojet
Registered User
 
Join Date: May 2008
Posts: 1,840
Concerning 64 bit plugins, some organization with them would be nice. Alphabetically? Category+Alphabetically?

On the other hand, if you want to include the dll's in a separate directory with new versions the install script could easily move them for auto loading. Or just one big pack, downloading them individually is kind of annoying to begin with but easier updating I guess.
turbojet is offline   Reply With Quote
Old 12th April 2010, 23:10   #331  |  Link
JoshyD
Registered User
 
Join Date: Feb 2010
Posts: 84
@Stephen
I noticed the artifacts last night myself, I'm currently looking into this. I did test with a variety of resizes before releasing, but apparently missed some corner cases. I'll revert the code for now, and come back soon with something more robust. It's hard to write the code and then test with all possible input variations. It really is hard to code and then extensively test all cases in the free time I do have. I'm sorry I screwed up this time, but, if I don't put out the code I do have, I'll never find some of the oddities that arise from code changes. I got excited on this one and jumped the gun.

For perspective, taking an already large project and just understanding it well enough to make minor modifications to it is difficult. There's not a million comments through the code, and you have to use a little intuition and some educated guesses as to the intended end result of the function. There's still oddities in the original source for 2.5.8 (Temporal::Soften accumulate_line_mode2 is commented as having funny results regardless of variable choice for the inline asm) and there were definitely times I found myself looking at the source and saying "what??". There's actually a pretty funny comment in squid_80's 2.5.7 64 bit port in this vein.
Code:
void Assign(const AVSValue* src, bool init) {
    if (src->IsClip() && src->clip)
      src->clip->AddRef();
    if (!init && IsClip() && clip)
      clip->Release();
    // make sure this copies the whole struct!
	/* UGH! what the- ?
    ((__int32*)this)[0] = ((__int32*)src)[0];
    ((__int32*)this)[1] = ((__int32*)src)[1];
	*/
	this->clip = src->clip;
	this->type = src->type;
	this->array_size = src->array_size;
  }
I did the same thing when originally looking at the code, then compared the function to squid's conversion, and realized, yes, this was weird.

As per usual, older builds that do function as they should remain available. If I screw it up, roll back for the time being. It's a crappy answer, but it's the best I can do.

I guess kind of take this as the community took 2.6alpha2. That was released kind of as a community test. When you're just a few guys (the avisynth team) or just one guy (me) having the code put through the paces by the community at large expedites the process of bug finding / squashing.

As for personal testing methodology, I have a few core files I work from, and a few scripts that use a variety of filters, both internal and external. They usually turn up my blunders, but this time, they failed to do so. Apparently, I need to vary my sources / tests more.

@Turbojet
Thank you for your time running the comparisons . . . they're a nice insight into the good and bad of the various builds, as well as the pro's and cons of threading. As a point of reference, can you list the machine specs, I may tabularize these in a wiki page, if that's alright.

The problem with the way avisynth's cache was hacked around is that single threaded performance inherently suffers. The added complexity to a core component of the program eats away at overall performance.

As to the thread deadlock, I'll see if I can make it happen. I've had it run out of memory on me before, which was the root of the problem, but I'm guessing something popped up complaining about a deadlock? I have had this happen on occasion when I've done something stupid/illogical in the code.

The filters *DO* need to be organized, eventually I may just redirect to the google code hosted project, and format a wiki page there, for now, a clean-up of the first post is in order . . . after I figure out why the resizer is eating it. I'm fairly positive it's the horizontal part that's going awry, as the vertical code didn't change.

@levi
The monotonic points problem appears to be related to the caching. I get the error when trying to run through code with the regular cache, but it's all rosy when using SetMTMode(). It's definitely the frame rate information getting lost in translation. For now though, it's taking the number 2 spot on the todo list, so I can shore up the resize code.

Only so many hours in the day, I'll come back with something more stable soon.

EDIT: First post in alphabetical order, cleaned, easier to read . . . and the resize functionality is working properly, at least on my end, the clip that was showing artifacts last night no longer does.

Last edited by JoshyD; 13th April 2010 at 01:46.
JoshyD is offline   Reply With Quote
Old 13th April 2010, 02:35   #332  |  Link
Andrey /MAG/
SVP developer
 
Join Date: Jul 2008
Location: Russia
Posts: 23
Hello.
Still reminding about 'blending bug' in MVTools.
Maybe I can do something to solving this problem?
What the source of the problem and how can I help to fix it?
Andrey /MAG/ is offline   Reply With Quote
Old 14th April 2010, 03:21   #333  |  Link
Delerue
Registered User
 
Join Date: Jun 2005
Posts: 365
Quote:
Originally Posted by Andrey /MAG/ View Post
Hello.
Still reminding about 'blending bug' in MVTools.
Maybe I can do something to solving this problem?
What the source of the problem and how can I help to fix it?
The interesting thing is that I can't reproduce the problem you're talking about. Everything works like it should be (i.e. equal to the x86 version). Here's the script I'm using:

Code:
setMTMode(2,12)
LoadPlugin("%path%\mvtools2 x64.dll")
source = ffdshow_source()
super = source.MSuper(pel=1)
backward_vec = MAnalyse(super, blksize=8, dct=10, overlap=2, isb = true, search=3, searchparam=2)
forward_vec = MAnalyse(super, blksize=8, dct=10, overlap=2, isb = false, search=3, searchparam=2)
source.MFlowFps(super, ThSCD1=350, blend=false, backward_vec, forward_vec, \
      num=2*FramerateNumerator(source), den=FramerateDenominator(source))
distributor()
I've tried to set blend to 'true' and also remove the blend parameter. It still works flawlessly. Can you show your script so we can help you?
Delerue is offline   Reply With Quote
Old 14th April 2010, 06:57   #334  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 924
Has anybody compiled a x64 version ov MVTools 1.x?
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 14th April 2010, 11:34   #335  |  Link
Andrey /MAG/
SVP developer
 
Join Date: Jul 2008
Location: Russia
Posts: 23
Quote:
Originally Posted by Delerue View Post
I can't reproduce the problem
The problem is in specific means of function parameters in script. See details (source, script and results) in zip-file: http://www.sendspace.com/file/4u8s0t

This fact is submitted by JoshyD in his post.
Andrey /MAG/ is offline   Reply With Quote
Old 14th April 2010, 16:31   #336  |  Link
Delerue
Registered User
 
Join Date: Jun 2005
Posts: 365
Quote:
Originally Posted by Andrey /MAG/ View Post
The problem is in specific means of function parameters in script. See details (source, script and results) in zip-file: http://www.sendspace.com/file/4u8s0t

This fact is submitted by JoshyD in his post.
There seems to be a problem with your 'blksize' value. Here, if I change it to '16' instead of '32', it works like it should. Also the 'searchparam' value 6 looks like an overkill. I did a lot of tests here with MFlowFPS (including your sample), and beyond 'searchparam' value 3 there's no visible gain, but the CPU usage increases a lot. I can guarantee you that the script I told you before gives a better final result. Of course you can tweak some parameters in order to get a better performance. What you think?
Delerue is offline   Reply With Quote
Old 14th April 2010, 19:49   #337  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,368
I just wanted to say thanks to everyone who is working hard on this project. This is a substantial development, and is making huge strides in AviSynth performance.

THANK YOU!!!

~MiSfit
Blue_MiSfit is offline   Reply With Quote
Old 14th April 2010, 23:52   #338  |  Link
7ekno
Registered User
 
Join Date: Jul 2007
Posts: 389
Quote:
Originally Posted by Blue_MiSfit View Post
I just wanted to say thanks to everyone who is working hard on this project. This is a substantial development, and is making huge strides in AviSynth performance.
+1, Awesome work guys, keep it up! Viva la 64bit!

Am sure there will be hiccups along the way and the more information provided here (as opposed to critisism) will help all x64 users as a whole!

7
7ekno is offline   Reply With Quote
Old 16th April 2010, 00:44   #339  |  Link
Stephen R. Savage
Registered User
 
Stephen R. Savage's Avatar
 
Join Date: Nov 2009
Posts: 336
Artifacts confirmed fixed. Thanks a lot, JoshyD. By the way, I didn't notice the new build when it was originally uploaded, as you did not make a post about it. This is not a problem, but is there any chance of getting a RSS feed or similar to follow progress?

Also, could you add the original version numbers for the plugins listed on the first page?

@tormento: Only mvtools2 is up, but why do you need mvtools 1.x? If you have a custom script that requires it, perhaps I could help you migrate your script to mvtools2.

@Avs developers: I notice that resizing RGB images causes the hue/saturation/brightness to shift. Is this normal, and can it be fixed?

Last edited by Stephen R. Savage; 16th April 2010 at 00:50.
Stephen R. Savage is offline   Reply With Quote
Old 16th April 2010, 07:02   #340  |  Link
Andrey /MAG/
SVP developer
 
Join Date: Jul 2008
Location: Russia
Posts: 23
Quote:
Originally Posted by Delerue View Post
problem with your 'blksize' value.
Yes. This value causes another behavior of 64-bit plugin than 32-bit. It shows that conversion from 32 into 64 was with some inaccuracies.
Quote:
Originally Posted by Delerue View Post
Here, if I change it to '16' instead of '32', it works like it should.
I have done frame-by-frame comparing with 'subtract' and found that difference still present.
Quote:
Originally Posted by Delerue View Post
the 'searchparam' value 6 looks like an overkill. I did a lot of tests here with MFlowFPS (including your sample), and beyond 'searchparam' value 3 there's no visible gain, but the CPU usage increases a lot.
In some cases 'searchparam=3' was too small to find long vectors. I still use MVTools 2.4.7 (32bit) and 'searchparam' produce not big increase of CPU usage.
MVTools 2.5.10 is not so fast as 2.4.7.

Quote:
Originally Posted by Delerue View Post
I can guarantee you that the script I told you before gives a better final result. Of course you can tweak some parameters in order to get a better performance. What you think?
Thank you. I compared it too. The results is prety well. Better than ones of my script.
Andrey /MAG/ is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 00:53.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.