Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 21st August 2008, 22:50   #1041  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
Quote:
Originally Posted by mikeytown2 View Post
Is there any way to improve MVAnalyse?
Yes, I am sure. But I do not know how exactly

More seriously: I consider how to use blocks of different sizes.
It is knewn method (macroblocks, etc). For big features it is better to use big blocks (stable and robust motion estimation), but for small features we need in small blocks.
To do no broke and rewrite MVTools (which coded with constant block size only approach), i try to add function MVRefine for two-stage estimation.

First stage: coarse_vectors=MVAnalyse(blksize=big)
Second stage: fine_vectors=MVRefine(vectors, blksize=small, thSAD=200)
Old vectors are used as a predictors.
This way we will preserve good vectors (update their SAD only),
and replace (re-estimate) bad vectors. Old vectors are used as a predictors. Probably refining is at finest level only.

I almost finished the first beta, hope can sent it to beta-testers soon

Of course, I am not sure, that results will be excited.
But it certainly will be slower.
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.
Fizick is offline   Reply With Quote
Old 22nd August 2008, 00:49   #1042  |  Link
Ranguvar
Registered User
 
Ranguvar's Avatar
 
Join Date: Feb 2007
Location: ::1
Posts: 1,236
Of course it'll be slower xD It's something of an AviSynth law...

but seriously, thanks for your great work You, josey_wells, Manao, everyone
Ranguvar is offline   Reply With Quote
Old 22nd August 2008, 02:31   #1043  |  Link
josey_wells
Registered User
 
Join Date: Dec 2007
Location: VA, USA
Posts: 116
Quote:
Originally Posted by Fizick View Post
I tried latest version on old Ahtlon 64, so I use classic MVAnalyse-MVDegrain2 script.
The speed is about 8 fps (blksize=16, overlap4).
But I have 10 fps with v1.9.5.7 for same script.
BTW, I recompile v1.9.6.7 with my development env (VCToolkit2003), but speed is still 8 fps.
Disabling mt=false do nothing.
I had disabled the BitBlt isse routines and doing memcpy only. The isse routines have been reinstated.

I finally got the rest of what I feel was missing to the multithreaded implementation of MVTools. Here is the new version.

http://rapidshare.com/files/13913936...9.6.8.rar.html

This version does away with the MT flag to control multithreading and adds the following two parameters for MVAnalyseMulti, MVDegrainMulti, MVDegrain1, MVDegrain2, MVDegrain3:

1. PreFetch - This will prefetch x frames before starting the threads for processing(default is 1)
2. Threads - This is the maximum number of threads to run concurrently(Default is 64) i.e all threads at once.

For example with the following line
MVMulti=last.MVAnalyseMulti(refframes=2, PreFetch=2, Threads=4)

There can be at most 2*refframes*prefetch=8 threads at once, and at most 4 are running at a time.

For example with the following line
MVDegrainMulti(MVMulti, PreFetch=2, Threads=4)

There can be at most 3*prefetch=6 threads at once, and at most 4 are running at a time.

Some notes:
1. When any thread completes and more data is to be processed a new thread is immediately started
2. Any prefetch greater than 1 will cause results to be cahced and the result return if still within cache. this will be seen as incrementing by prefetch in virtualdub since the cached result is returned immediately

As an example of a script which sets the parameters on a quad core.

Code:
#motion denoise
function Denoise(clip c, int "iLevel", int "iTHSAD", int "iBlksize", int "iOverlap", int "iPel", bool "bChroma", \
		      bool "bTrueMotion", int "iSearch", int "iSharp", int "iDct", int "iPrefetchAnalyse", \
		      int "iPrefetchDegrain", int "iThreads", int "iQueuedThreads")
{
    iLevel          =default(iLevel, 2)                    #default level for degrain 
    iBlksize        =default(iBlksize, 8)                  #default block size
    iOverlap        =default(iOverlap, iBlksize/2)         #default 1/2 block size
    iTHSAD          =default(iTHSAD, 400)                  #default
    iPel            =default(iPel, 4)                      #quarter pixel
    bChroma         =default(bChroma, true)	           #use chroma
    bTrueMotion     =default(bTrueMotion, true)            #use true motion
    iSearch         =default(iSearch, 2)                   #diamond search
    iSharp          =default(iSharp, 2)                    #6 tap sharp filter
    iDct            =default(iDct, 1)                      #use DCT for SAD 
    iThreads        =default(iThreads, 4)                  #max number of threads
    iQueuedThreads  =default(iQueuedThreads, iThreads)     #number of queuedthreads
    iPrefetchAnalyse=default(iPrefetchAnalyse, LCM(2*iLevel, iQueuedThreads)/(2*iLevel)) 
    iPrefetchDegrain=default(iPrefetchDegrain, LCM(3, iQueuedThreads)/3)                

    MVMulti=c.MVAnalyseMulti(refframes=iLevel, pel=iPel, blksize=iBlksize, overlap=iOverlap, chroma=bChroma, \
			     truemotion=bTrueMotion, search=iSearch, sharp=iSharp, dct=iDct, Threads=iThreads, \
			     prefetch=iPrefetchAnalyse, idx=1)
    return c.MVDegrainMulti(MVMulti, thSAD=iTHSAD, Threads=iThreads, prefetch=iPrefetchDegrain, idx=1)  
}

# Computes the Greatest Common Divisor of two integers
# (Euclid's Algorithm)
function GCD(int a, int b)
{
    return (b == 0) ? a : GCD(b, a % b)
}

# Computes the Least Common Multiple of two integers
function LCM(int a, int b)
{
    return (a*b) / GCD(a, b)
}
This will prefetch the minimum amount of frames so that the total threads avaialbe is an integral multiple of the number of threads allowed to queued.

Code:
Hello I have found a bug.
Download this steam http://rapidshare.com/files/13884330...muxed.zip.html
Load it (MPEG2 file) in avisynth and add those lines to the avisynth script:
source=last
MVForBack = source.MVAnalyseMulti(refframes=1, pel = 2, overlap=2, overlapv=2, sharp=1, idx = 1,blksize=16,search=3,chroma=true,global=true)
source.mvshow(MVForBack)

Look at the frame just before the scenechange, motiondetection is totaly broken.
The line for show should be
source.MVShow(MVForBack.MVMultiExtract(0)) #show bac1

With this build I acheived about a 10-20% improvement and up to 80-90% improvement depending on script.

You may have to set SetMemoryMax down to a value such as 256 is memory problems arise since more prefetching cost more memory and under Win32 memory is limited to about 2GB.

Have fun!
josey_wells is offline   Reply With Quote
Old 22nd August 2008, 03:27   #1044  |  Link
mikeytown2
Resize Abuser
 
mikeytown2's Avatar
 
Join Date: Apr 2005
Location: Seattle, WA
Posts: 623
@josey_wells
TempGaussMC_beta1 bombs with this latest build, tried all 4 dll versions.
mikeytown2 is offline   Reply With Quote
Old 22nd August 2008, 03:54   #1045  |  Link
josey_wells
Registered User
 
Join Date: Dec 2007
Location: VA, USA
Posts: 116
Quote:
Originally Posted by mikeytown2 View Post
@josey_wells
TempGaussMC_beta1 bombs with this latest build, tried all 4 dll versions.
I have found the problem and it has to do with multiple MVAnalyseMulti or MVDegrain in the script.

I will post fix tomorrow.
josey_wells is offline   Reply With Quote
Old 22nd August 2008, 07:38   #1046  |  Link
morsa
the dumbest
 
Join Date: Oct 2002
Location: Malvinas
Posts: 494
Could this be of any use?

http://research.microsoft.com/~larryz/ZitnickICCV05.pdf


A question: What would happen if one were analizing a wavelet decomposed image, then interpolating it and recomposing it again?
Could it give any improvements quality wise?
morsa is offline   Reply With Quote
Old 22nd August 2008, 14:57   #1047  |  Link
josey_wells
Registered User
 
Join Date: Dec 2007
Location: VA, USA
Posts: 116
Quote:
Originally Posted by mikeytown2 View Post
@josey_wells
TempGaussMC_beta1 bombs with this latest build, tried all 4 dll versions.
Here is the fix. I skipped V1.9.6.9 since I consdider this a major release.

http://rapidshare.com/files/13925780...9.7.0.rar.html

I now consider all optimizations to MVAnalyseMulti and MVDegrain's completed.

If there is enough interest then I can look at multithreading some of the other functions but need a break first. I will also post complete comparisons of this code vs V1.9.5.7 soon disregarding all white space changes.

Have fun!
josey_wells is offline   Reply With Quote
Old 22nd August 2008, 16:41   #1048  |  Link
Adub
Fighting spam with a fish
 
Adub's Avatar
 
Join Date: Sep 2005
Posts: 2,685
If I may, but why is the "Threads" option default to 64? Why not have it default to the number of cores found or something along those lines?

Edit: And also, thank you very much for your hardwork!! By all means, take a break, you deserve it!

Edit2:

Okay, bug report from kokotier:

Quote:
<kokotier> hello
<kokotier> i saw you have asnwered
<kokotier> to the mvtools thread
<Merlin7777> yes?
<kokotier> canyou tell to josey_wells than the last buid is buggy
<kokotier> http://forum.doom9.org/showthread.ph...74#post1172974
<Merlin7777> Wait, did you read his reply?
<kokotier> yes but is doesnt change anything
<kokotier> motiondetection is broken
<kokotier> you can test yourself
<kokotier> dl my sample
<kokotier> and on the latest buid i have green frames with it :-
<Merlin7777> It's having trouble connecting. But didn't you post it online? Anyways, I will post and tell him that you are still having problems. Same script?
<kokotier> i cant post on doom9 sorry
<kokotier> i hve to wait more days
<Merlin7777> Yeah, I know. But are you using the same script? Or are you also encountering the bug with the suggestion he posted?
<kokotier> source=last
<kokotier> MVForBack = source.MVAnalyseMulti(refframes=1, pel = 2, overlap=2, overlapv=2, sharp=1, idx = 1,blksize=16,search=3,chroma=true,global=true)
<kokotier> source.MVShow(MVForBack.MVMultiExtract(0))
<kokotier> with the suggestion i have the same bug
<kokotier> it seems that is overlapv=2 that is buggy

Edit3:
Quote:
its weird
<kokotier> with refframe=2
<kokotier> 2 frame before the scene chnge is bad
<kokotier> with refframe=3
<kokotier> 3 frame before the scene change is bad
<kokotier> and so on..

Edit 4: (man, I am really going to town on this thing.)

Kokotier, (who is in the waiting period after registering, by the way) walked me through what he is doing, and I have to agree, it is a bit odd. with Refframe=1, the frame before the scenechange is filled with motion vectors, even though there is no motion. But since it is a scenechange, it might be that way on purpose.

However, that's not the issue. As Kokotier pointed out, if you change the refframe to 2, then two frames before the scenechange is filled with motion vectors, then a "good frame", aka, no motion vectors, THEN the scenechanges. It is some odd behavior I must say. But I don't necessarily know if it is wrong. So I will let the experts decide.
__________________
FAQs:Bond's AVC/H.264 FAQ
Site:Adubvideo

Last edited by Adub; 22nd August 2008 at 17:33.
Adub is offline   Reply With Quote
Old 22nd August 2008, 19:33   #1049  |  Link
morsa
the dumbest
 
Join Date: Oct 2002
Location: Malvinas
Posts: 494
I have found that if I use the histogram equalization filter from VCmohan at the MVanalyze stage, results get improved a lot.
Does it sound logical to use histogram equalization internally Fizick?
morsa is offline   Reply With Quote
Old 23rd August 2008, 17:05   #1050  |  Link
thetoof
Sleepy overworked fellow
 
Join Date: Feb 2008
Location: Maple syrup's homeland
Posts: 933
Quote:
Originally Posted by josey_wells View Post
If there is enough interest then I can look at multithreading some of the other functions but need a break first.
Thanks a lot! Could you also consider updating the docs? (well, you or anyone feeling like it )
mvflowfps would be the next function to multithread imo...
__________________
AnimeIVTC() - v2.00
-http://boinc.berkeley.edu/-
Let all geeks use their incredibly powerful comps for the greater good (no, no, it won't slow your filtering/encoding :p)
thetoof is offline   Reply With Quote
Old 23rd August 2008, 20:37   #1051  |  Link
Sagekilla
x264aholic
 
Join Date: Jul 2007
Location: New York
Posts: 1,752
@morsa: How exactly did you use the equalization, and how much did the results improve?
__________________
You can't call your encoding speed slow until you start measuring in seconds per frame.
Sagekilla is offline   Reply With Quote
Old 24th August 2008, 07:30   #1052  |  Link
Delerue
Registered User
 
Join Date: Jun 2005
Posts: 365
Quote:
Originally Posted by thetoof View Post
Thanks a lot! Could you also consider updating the docs? (well, you or anyone feeling like it )
mvflowfps would be the next function to multithread imo...
I think the same.

EDITED: BTW, I downloaded the last version, and use it with this old script, but I got crashes all the time:

Code:
SetMtmode(1,5)
source=ffdshow_source()
SetMTMode(2)
LoadPlugin("C:\arquivos de programas\avisynth\plugins\mvtools.dll")
backward_vec = source.MVAnalyse(blksize=16, overlap=0,  isb = true,  pel=2, search=2, idx=1)
forward_vec = source.MVAnalyse(blksize=16, overlap=0, isb = false, pel=2, search=2, idx=1)
source.MVFlowFps(backward_vec, forward_vec, num=2*FramerateNumerator(source), \
   den=FramerateDenominator(source), mask=1, idx=1)
distributor()
I tried to remove the MT lines, but it doesn't help. Also tried other CPU versions of MVTools (although I have a Core2Duo), and it doesn't help either.

Last edited by Delerue; 24th August 2008 at 07:47.
Delerue is offline   Reply With Quote
Old 24th August 2008, 14:42   #1053  |  Link
egrimisu
Registered User
 
egrimisu's Avatar
 
Join Date: Jan 2008
Location: Romania - neighbor of Dracula
Posts: 327
for using the next code in multithread :
source = last
backward_vec3 = source.MVAnalyse(isb = true, delta = 3, pel = 2, overlap=4, sharp=1, idx = 1)
backward_vec2 = source.MVAnalyse(isb = true, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
backward_vec1 = source.MVAnalyse(isb = true, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec1 = source.MVAnalyse(isb = false, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec2 = source.MVAnalyse(isb = false, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec3 = source.MVAnalyse(isb = false, delta = 3, pel = 2, overlap=4, sharp=1, idx = 1)
source.MVDegrain3(backward_vec1,forward_vec1,backward_vec2,forward_vec2,backward_vec3,forward_vec3,thSAD=400,idx=1)

should be called as :
source = last
backward_vec3 = source.MVAnalysemulti(isb = true, delta = 3, pel = 2, overlap=4, sharp=1, idx = 1)
backward_vec2 = source.MVAnalysemulti(isb = true, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
backward_vec1 = source.MVAnalysemulti(isb = true, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec1 = source.MVAnalysemulti(isb = false, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec2 = source.MVAnalysemulti(isb = false, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec3 = source.MVAnalysemulti(isb = false, delta = 3, pel = 2, overlap=4, sharp=1, idx = 1)
source.MVDegrain3(backward_vec1,forward_vec1,backward_vec2,forward_vec2,backward_vec3,forward_vec3,thSAD=400,idx=1)
egrimisu is offline   Reply With Quote
Old 24th August 2008, 16:56   #1054  |  Link
Dreassica
Registered User
 
Join Date: May 2002
Posts: 384
Using that line I'm gettting a "MVAnalysemulti has no named argument "isb"" error msg.
Dreassica is offline   Reply With Quote
Old 24th August 2008, 17:03   #1055  |  Link
Sagekilla
x264aholic
 
Join Date: Jul 2007
Location: New York
Posts: 1,752
Because that's the incorrect usage of MVAnalyseMulti. It has no "delta" or "isb" syntax anymore, since a single instance of MVAnalyseMulti is supposed to be used as so:

Code:
# Ref frames is your delta, but it looks backwards AND forwards at the same time. No need for isb=true/false
vectors = source.MVAnalyseMulti(refframes = 3)

I don't remember if vector extraction has been implemented, but it was supposed to be done as so if you want old school MVDegrain:

Code:
vectors = source.MVAnalyseMulti(refframes=1)
bvec1 = vectors.MVMultiExtract(0)
fvec1 = vectors.MVMultiExtract(1)
source.MVDegrain1(bvec1,fvec1)
__________________
You can't call your encoding speed slow until you start measuring in seconds per frame.
Sagekilla is offline   Reply With Quote
Old 24th August 2008, 20:06   #1056  |  Link
egrimisu
Registered User
 
egrimisu's Avatar
 
Join Date: Jan 2008
Location: Romania - neighbor of Dracula
Posts: 327
sorry, i'm a hard head so what will be the code in the new version for the code that writen up?
egrimisu is offline   Reply With Quote
Old 24th August 2008, 20:12   #1057  |  Link
mikeytown2
Resize Abuser
 
mikeytown2's Avatar
 
Join Date: Apr 2005
Location: Seattle, WA
Posts: 623
old
Code:
source = last
backward_vec3 = source.MVAnalyse(isb = true, delta = 3, pel = 2, overlap=4, sharp=1, idx = 1)
backward_vec2 = source.MVAnalyse(isb = true, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
backward_vec1 = source.MVAnalyse(isb = true, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec1 = source.MVAnalyse(isb = false, delta = 1, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec2 = source.MVAnalyse(isb = false, delta = 2, pel = 2, overlap=4, sharp=1, idx = 1)
forward_vec3 = source.MVAnalyse(isb = false, delta = 3, pel = 2, overlap=4, sharp=1, idx = 1)
source.MVDegrain3(backward_vec1,forward_vec1,backward_vec2,forward_vec2,backward_vec3,forward_vec3,thSAD=400,idx=1)
NEW
Code:
source = last
vectors = source.MVAnalyseMulti(refframes=3, pel = 2, overlap=4, sharp=1, idx = 1)
source.MVDegrainMulti(vectors,thSAD=400,idx=1)
refframes=1 is MVDegrain1()
refframes=2 is MVDegrain2()
refframes=3 is MVDegrain3()
refframes=4 is MVDegrain4()
ect...
mikeytown2 is offline   Reply With Quote
Old 25th August 2008, 10:42   #1058  |  Link
egrimisu
Registered User
 
egrimisu's Avatar
 
Join Date: Jan 2008
Location: Romania - neighbor of Dracula
Posts: 327
There's no need for adding delta = 3 anymore?
egrimisu is offline   Reply With Quote
Old 25th August 2008, 15:11   #1059  |  Link
Sagekilla
x264aholic
 
Join Date: Jul 2007
Location: New York
Posts: 1,752
Nope. Delta is now refframes, but refframes encompasses both delta and isb. There's no need to set as being backwards or forwards vector since it stores both in the same clip.
__________________
You can't call your encoding speed slow until you start measuring in seconds per frame.
Sagekilla is offline   Reply With Quote
Old 25th August 2008, 15:46   #1060  |  Link
bairradino
Registered User
 
Join Date: Jan 2003
Location: po
Posts: 89
@ Mikeytown2

Your code
Code:
source = last
vectors = source.MVAnalyseMulti(refframes=1, pel = 2, overlap=4, sharp=1, idx = 1)
source.MVDegrainMulti(vectors,thSAD=400,idx=1)
gives the following error:
there is no function named "MVAnaliseMulti".
bairradino is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 20:08.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.