Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 21st February 2014, 04:18   #641  |  Link
TurboPascal7
Registered User
 
TurboPascal7's Avatar
 
Join Date: Jan 2010
Posts: 270
Note to all people having any kind of issues with mt build: first of all, try removing SetFilterMTMode("", MT_MULTI_INSTANCE) from your script. I have absolutely no idea why this was introduced in the first place since it only creates problems. Yes, this will cripple performance but no one promised you it'll be easy.

Second - setting any MT mode on script functions does nothing. For example, SetFilterMTMode("Dither_Crop16", MT_MULTI_INSTANCE) doesn't change anything. You can put any other MT mode there and it will still work they way it works by default. Only plugin functions matter.

Third - situation where one plugin works with mode 1 when used before filter X and only with mode 2 when X is not present is pretty much impossible (assuming there are no bugs in the core of course ). MT mode is a property of a filter and it doesn't depend on where it's used, only on filter implementation and input parameters (unless the filter is completely broken). I understand this is not helping much and thank you for posting scripts with this kind of problems, but most likely it means that MT mode of one of the filters was not "guessed" right. Hopefully we can get authors of some plugins who are still alive (dither for instance) to supply correct MT modes for every function in their plugins, either as a registration routine in the plugin itself, or SetFilterMtMode line. If we can't, well... we'll be guessing.

EDIT: as innocenat pointed out on IRC, it's actually possible to have filters changing their behavior if they're called before a filter with MT mode 3. All filters are getting serialized and any MT mode you're applying to them should not matter. This is an implementation detail though and you should not rely on it in any way since it might get changed without notice.

Now, to Prefetch(1)/Prefetch(0) issue. As ultim explained (no one bothered to read the full MT implementation tbh because it feels a bit (over)complicated), the number that you supply to prefetch is the number of prefetching threads. So in theory Prefetch(1) should give you two threads - one main thread and one prefetching thread. Prefetch(0) should disable any multithreading.

As usual, practice doesn't match theory very well and Prefetch(1) actually gives you one working thread (dunno if it's the main or the created one). Yes, this is a bug and it most likely will be fixed.

But if you think about it, this API design is very bad. Most users don't know and don't need to know the difference between prefetching threads and the main thread. Adding Prefetch(4) to their script, they absolutely reasonably expect it to work in 4 threads, trying to explain to them that it's actually 5 would be silly. So this bug kinda makes the API better.

In short - yes, currently Prefetch(1) and Prefetch(0) are almost equivalent. In the future one of these might error out, or this behavior will be kept (probably more reasonable).
__________________
Me on GitHub | AviSynth+ - the (dead) future of AviSynth

Last edited by TurboPascal7; 21st February 2014 at 06:43.
TurboPascal7 is offline   Reply With Quote
Old 21st February 2014, 13:14   #642  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 1,240
TurboPascal7, thank you the script function vs mt mode note, although I knew it, did not check which Dither function was implemented in script.

Luckily I have cought an error message displayed for a few tenth of seconds (then made a screenshot video)
"Dither_resize16: resizing ratio too low or kernel support too high." In MT_SERIALIZED mode it does not occur. (v1.24)
pinterf is offline   Reply With Quote
Old 21st February 2014, 20:05   #643  |  Link
mark0077
Registered User
 
Join Date: Apr 2008
Posts: 1,055
Guys maybe this is a stupid question, but has someone (or is it even possible) to use this with SVP? I have been trying with no luck to replace my SetMTMode calls with SetFilterMTMode calls towards SVP functions. Maybe its not even possible unless SVP add support in their dlls?
mark0077 is offline   Reply With Quote
Old 22nd February 2014, 00:22   #644  |  Link
hank315
HCenc author
 
Join Date: Nov 2003
Location: Netherlands
Posts: 570
Just installed Avisynth+ to run some tests as frame server for HCenc.
Most tests finished OK but the most basic test failed: Avisource input only.
Frames seem to have a wrong pitch.

Source is RGB in an AVI container.
Probably I'm doing something stupid because if this is a bug it should have been reported earlier I guess.

Used script:
Code:
s = AVISource("part2.avi")
s = ConvertToYV12(s)
return(s)
Source can be found here.


BTW, Avisynth 2.6.a5 and 2.5.8 handled the source OK.
Is it correct frame->GetPitch is mod32 in Avisynth+ ?
__________________
HCenc at: http://hank315.nl
hank315 is offline   Reply With Quote
Old 22nd February 2014, 00:26   #645  |  Link
TurboPascal7
Registered User
 
TurboPascal7's Avatar
 
Join Date: Jan 2010
Posts: 270
Quote:
Originally Posted by hank315 View Post
Is it correct frame->GetPitch is mod32 in Avisynth+ ?
Most of the times - yes. But you still can get unaligned input after crop(align=false) for example.
__________________
Me on GitHub | AviSynth+ - the (dead) future of AviSynth

Last edited by TurboPascal7; 22nd February 2014 at 01:23. Reason: clip->crop
TurboPascal7 is offline   Reply With Quote
Old 22nd February 2014, 09:38   #646  |  Link
Zathor
Registered User
 
Join Date: Nov 2009
Posts: 2,402
Quote:
Originally Posted by http://forum.doom9.org/showpost.php?p=1669533&postcount=3987
Code:
ClearAutoloadDirs()
AddAutoloadDir("C:\Program Files\MeGUI\tools\avs\plugins")
LoadPlugin("C:\Program Files\MeGUI\tools\avisynth_plugin\NicAudio.dll")
RaWavSource("E:\audio.wav")
SSRC((AudioRate()*25)/24).AssumeSampleRate(AudioRate()).TimeStretch(pitch=Float((AudioRate()*25)/24)*100.0/Float(AudioRate()))
return last
"An error occurred: Input audio sample format to SSRC must be float."
Quote:
Originally Posted by http://forum.doom9.org/showpost.php?p=1669533&postcount=3988
Seems portable version is AviSynth+ (ClearAutoloadDirs(), AddAutoloadDir()), and I think have a new version of SSRC but maybe without the automatic conversion of input data type than AviSynth 2.5.8 have.
When source was decoded from AC3, DTS, etc. with NicAudio or BassAudio the audio samples are float, but from losless sources the audio samples are int, and SSRC from AviSynth+ show the Error.
It seems that the SSRC filter in AviSynth+ cannot accept int audio while the AviSynth filter does. Reason seems to be that the automatic conversion mentioned here seems to be not working. Is that a missing feature/intended or a bug?
Zathor is offline   Reply With Quote
Old 22nd February 2014, 12:37   #647  |  Link
Zep
Registered User
 
Join Date: Jul 2002
Posts: 587
Quote:
Originally Posted by TurboPascal7 View Post

But if you think about it, this API design is very bad. Most users don't know and don't need to know the difference between prefetching threads and the main thread. Adding Prefetch(4) to their script, they absolutely reasonably expect it to work in 4 threads, trying to explain to them that it's actually 5 would be silly. So this bug kinda makes the API better.

In short - yes, currently Prefetch(1) and Prefetch(0) are almost equivalent. In the future one of these might error out, or this behavior will be kept (probably more reasonable).
Thanks turboPascal your post was exactly the info i was looking for from a high level users eyes. For me I don't need the super low level details but I do want to know enough so i can tweak performance or at least a good guess as where to start/try.

thanks
Zep is offline   Reply With Quote
Old 22nd February 2014, 15:09   #648  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 1,057
Quote:
Originally Posted by Zathor View Post
It seems that the SSRC filter in AviSynth+ cannot accept int audio while the AviSynth filter does. Reason seems to be that the automatic conversion mentioned here seems to be not working. Is that a missing feature/intended or a bug?
It's by design:
https://github.com/AviSynth/AviSynthPlus/commit/d40f4a2e833998199dec0e0e4737cf69c4607b6c


There was also a forum post that mentioned it or explained the rationale (that it shouldn't do that kind of conversion without the user's knowledge), but I can't find it right now.
qyot27 is offline   Reply With Quote
Old 22nd February 2014, 19:04   #649  |  Link
Zathor
Registered User
 
Join Date: Nov 2009
Posts: 2,402
Thanks, I have added ConvertAudioToFloat() to the script logic.
Zathor is offline   Reply With Quote
Old 23rd February 2014, 12:36   #650  |  Link
zerowalker
Registered User
 
Join Date: Jul 2011
Posts: 1,100
Here is a Bug Report, a Script that fails for no reason while it works on AVS 2.6 according to another user.

http://forum.doom9.org/showthread.php?p=1670117#post1670117

Hope that helps.
zerowalker is offline   Reply With Quote
Old 24th February 2014, 08:42   #651  |  Link
innocenat
Registered User
 
innocenat's Avatar
 
Join Date: Dec 2011
Posts: 77
Just stating in this topic also: the above issues happens in r1561 but not r1576. No fix needed. Refer to linked topic for more information.
__________________
AviSynth+
innocenat is offline   Reply With Quote
Old 24th February 2014, 10:23   #652  |  Link
TurboPascal7
Registered User
 
TurboPascal7's Avatar
 
Join Date: Jan 2010
Posts: 270
Quote:
Originally Posted by tormento View Post
Some testing done this afternoon with 1080P material.
The following hangs in MeGui preview:
PHP Code:
LoadPlugin("D:\eseguibili\media\DGDecNV\DGDecodeNV.dll")
SetFilterMTMode(""2)
SetFilterMTMode("DGSource"3)
DGSource("E:\in\something.dgi")
#CompTest(1)
ChangeFPS(last,last,true)
SMDegrain (tr=6,PreFilter=2,thSAD=600,contrasharp=false,refinemotion=true,lsb_out=true,plane=0,chroma=false,Globals=2)
SMDegrain (tr=6,            thSAD=600,contrasharp=false,refinemotion=true,lsb_in =true,plane=0,chroma=false,Globals=1,mode=6)
Prefetch(3
Both works ok under AviSynth_MT from Set, which is even a bit faster.
I can confirm it being very slow (on 1080p) and probably unstable but it doesn't hang out completely for me in avspmod. Please try to remove SetFilterMTMode("", 2) line and see if it works (performance aside). This is most likely caused either by some globals magic or wrong mt mode specified for some filters.

Quote:
Originally Posted by lansing View Post
mpeg2source doesn't seem to work with avs+ mt on. I have a 1080p mpeg2 video which I indexed with dgindex, and trying to test with mt on, it gave me a green screen onload with avspmod, and when I try seek, the program just hangs. All three mt mode will hang.

Code:
SetFilterMTMode("MPEG2Source", 2)
MPEG2Source("sample.d2v", cpu=0)
Prefetch(4)
Can't reproduce. Mode 1 is indeed broken but that's to be expected. Modes 2 and 3 work without issues here. Does it happen with some specific source or any sources in general?

Quote:
Originally Posted by hank315 View Post
Most tests finished OK but the most basic test failed: Avisource input only.
Frames seem to have a wrong pitch.
Source is RGB in an AVI container.
Used script:
Code:
s = AVISource("part2.avi")
s = ConvertToYV12(s)
return(s)
Source can be found here.
Reproduced, reposted on github. Avisource indeed can't handle mod32 pitch in this case. I will look into later this week.

pinterf
Firesledge added functions from dither here and they all seem to be NICE_FILTERs, so I guess the issue is somewhere in mvtools2. I'll probably play with it some time soon but wouldn't mind anyone testing it before me.
__________________
Me on GitHub | AviSynth+ - the (dead) future of AviSynth
TurboPascal7 is offline   Reply With Quote
Old 24th February 2014, 17:57   #653  |  Link
ultim
AVS+ Dev
 
ultim's Avatar
 
Join Date: Aug 2013
Posts: 359
Quote:
Originally Posted by TurboPascal7 View Post
But if you think about it, this API design is very bad. Most users don't know and don't need to know the difference between prefetching threads and the main thread. Adding Prefetch(4) to their script, they absolutely reasonably expect it to work in 4 threads, trying to explain to them that it's actually 5 would be silly. So this bug kinda makes the API better.
I kindly disagree and will keep it this way even in the future. Hopefuly I will also find why Prefetch(1) is the same as Prefetch(0) and fix it.

Why I think it is better this way? First, the name of "Prefetch" is well chosen, because it describes exactly what the filter does: it prefetches frames. When inserted into a script, the user is basically telling the core a command to "Prefetch me frames in addition to the one you are working on now!". So the name is fine. So when you see the function declared as "Prefetch(nThreads)", what is the most intuitive way to interpret it? Isn't the first thing you think of "prefetch using nThreads threads"? I think it is, and this is exactly what AviSynth+ is doing now. Going TurboPascal's way would mean "Prefetch frames for me using one less threads than what I told you"! That is way more counterintuitive than the current implementation.

I do agree with TurboPascal that users shouldn't need to concern themselves with implementation details. They shouldn't, which is why I find it odd at all that he explained this. A higher number gives more threads: that is IMHO no less and no more than what a user must know in this case to find the optimal parameter for his number of cores, for best performance on his computer.

As for bugs, please remember, there is a reason this was published as a test build.
__________________
AviSynth+

Last edited by ultim; 24th February 2014 at 17:59.
ultim is offline   Reply With Quote
Old 24th February 2014, 19:04   #654  |  Link
ultim
AVS+ Dev
 
ultim's Avatar
 
Join Date: Aug 2013
Posts: 359
Well, bad news folks, we didn't get accepted for GSoC this year. Of course our "staff" will remain available for anybody wishing to pick up one of the projects, for help. There is always a next year, and we continue to hack around AviSynth ourselves too.

The preparation for GSoC was not all in vain though. We discussed a lot of things about the future of AviSynth's ecosystem. Thanks to GSoC, we managed to define what we want to do, defined our priorities and wishes clearly. As "by-products", we even got ourself a nice set of coding guidelines for future contributions, and improved our infrastructure just a bit further. Also, now that GSoC isn't playing for us in 2014, I can extend our ideas list with all those that we did discuss and want to have, but for various reasons deemed unfit for Google.
__________________
AviSynth+

Last edited by ultim; 24th February 2014 at 19:21.
ultim is offline   Reply With Quote
Old 24th February 2014, 19:45   #655  |  Link
TurboPascal7
Registered User
 
TurboPascal7's Avatar
 
Join Date: Jan 2010
Posts: 270
Quote:
Originally Posted by ultim View Post
I kindly disagree and will keep it this way even in the future.
We can go full democracy and ask users to vote. Question is: "how many threads do you expect the following script to have? Assuming no filters are multithreaded internally."
Code:
DgSource("commie.dgi")
tfm().tdecimate()
debilinear(1280, 720)
gradfun3()
Prefetch(4)
My answer is 4. It's perfectly reasonable that you like your implementation (you wouldn't write it this way in the first place otherwise), but I believe it's non-obvious for anyone without programming experience.
__________________
Me on GitHub | AviSynth+ - the (dead) future of AviSynth
TurboPascal7 is offline   Reply With Quote
Old 24th February 2014, 20:13   #656  |  Link
MasterNobody
Registered User
 
Join Date: Jul 2007
Posts: 519
I don't see anything bad with having 5 threads when using Prefetch(4) if all hard work is done by this prefetch threads and one more (main) light thread is only needed to orchestrate other threads. That is mostly semantic of what x264 have for --threads and the only exception is --threads 1 when main thread and working threads become the one and the same thread.
MasterNobody is offline   Reply With Quote
Old 24th February 2014, 23:35   #657  |  Link
TurboPascal7
Registered User
 
TurboPascal7's Avatar
 
Join Date: Jan 2010
Posts: 270
Quote:
Originally Posted by MasterNobody View Post
I don't see anything bad with having 5 threads when using Prefetch(4) if all hard work is done by this prefetch threads and one more (main) light thread is only needed to orchestrate other threads. That is mostly semantic of what x264 have for --threads and the only exception is --threads 1 when main thread and working threads become the one and the same thread.
There's nothing wrong with having five or even more threads if all but four of them are idling most of the time, that's a minor implementation detail. But I thought the plan was to have main thread doing the same kind of stuff prefetch threads are doing (e.g. main thread requests frame 0, prefetch threads request frames 1, 2, 3 and 4).
__________________
Me on GitHub | AviSynth+ - the (dead) future of AviSynth
TurboPascal7 is offline   Reply With Quote
Old 25th February 2014, 00:03   #658  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,146
Quote:
Originally Posted by TurboPascal7 View Post
Can't reproduce. Mode 1 is indeed broken but that's to be expected. Modes 2 and 3 work without issues here. Does it happen with some specific source or any sources in general?
mode 2 and 3 were fine with 480p source, but not for 1080 mpeg2 videos, I tried on a few clips already. This time only prefetch(0) work, all other prefetch would gave a green screen, and then after a few jumps in timeline, the program would hang.
lansing is offline   Reply With Quote
Old 25th February 2014, 00:45   #659  |  Link
TurboPascal7
Registered User
 
TurboPascal7's Avatar
 
Join Date: Jan 2010
Posts: 270
Quote:
Originally Posted by lansing View Post
mode 2 and 3 were fine with 480p source, but not for 1080 mpeg2 videos, I tried on a few clips already. This time only prefetch(0) work, all other prefetch would gave a green screen, and then after a few jumps in timeline, the program would hang.
Reproduced, thanks for the report.
__________________
Me on GitHub | AviSynth+ - the (dead) future of AviSynth
TurboPascal7 is offline   Reply With Quote
Old 25th February 2014, 07:29   #660  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,146
Another problem, I tired running mctd script on a 1080 avc stream. I loaded the script into virtualdub and ran analysis pass, and it returned an error "could not allocate video frame. Out of memory".

Code:
dgsource("test.mkv", engine=1)
mctd()
lansing is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 18:01.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.