Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 27th September 2008, 21:03   #21  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
here is v2.0.0.3 alpha
http://avisynth.org.ru/mvtools/mvtools-v2.0.0.3.zip

project file for codeblocks: mvtools-super.cbp
New documentaton is missing. Current using (may be changed without notification ):

avisource()
super=MVSuper(hpad=8, vpad=8, pel=2, sharp=2, rfilter=1)
vb=MVAnalyse(super,isb=true) # any old MVAnalyse params besides above
vf=MVAnalyse(super,isb=false)
MVDegrain1(last,mvbw=vb,mvfw=vf,super=super)

I note some speed increasing and memory usage decreasing. Please test.

About syntax and implementation.
superframe contains all source video data. So, it is possible implement clients (like MVDegrain1) without usage of source (i.e. last clip):
MVDegrain1(super,mvbw=vb,mvfw=vf)

But super clip does not contain source audio (i killed it). Of course we can use audiodub.
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.

Last edited by Fizick; 27th September 2008 at 21:15.
Fizick is offline   Reply With Quote
Old 27th September 2008, 23:45   #22  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,431
Sorry to bring up the 'implicit last' issue yet again, but it's such a fundamental part of Avisynth usage that I think it's important to get it as right as possible. Would this be a valid sequence:

avisource()
MVSuper(hpad=8, vpad=8, pel=2, sharp=2, rfilter=1)
vb=MVAnalyse(isb=true)
vf=MVAnalyse(isb=false)
MVDegrain1(mvbw=vb,mvfw=vf)

with the last 3 implicitly using the result of MVSuper?

Why kill the audio? I don't understand that. Or does it hide extra data in what would otherwise be audio properties?
Gavino is offline   Reply With Quote
Old 28th September 2008, 01:04   #23  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
Looking at the proposal of superframe concept, it seems that some possible kind of processing "tricks" are lost.

One "trick" you could do with v1.x is this:

Code:
source = last
vb     = source.MVAnalyse(isb=true, idx=1)
vf     = source.MVAnalyse(isb=false,idx=1)

alt    = source.blur(1)  #  example for an "alternate" spatial or spatio-temporal filter

result = alt.MVDegrain1(vb,vf,idx=1)  #  *same* idx !

This way you get "automatically" weighting-in of the alternate filter in places where compensated blocks get less weighting (due to thSAD). If all compensations "fail" completely, the result is 100% of the alternate filter.

Now with the new superframe concept, it seems no more possible to do this kind of "mixing". You get only vanilla "no processing" for failed compensation areas. To get that "functionality" (though it's a trick) back, it would require to additionally include the alternate filtered clip into the superframe?

Some more thoughts should be thrown about handling of "failed compensation" blocks in MVDegrain. (par ex., the mix achieved by the given code example is not optimal either - in areas where compensation is successful, it gives too much weight to "alternate".)
But for today, it's too late now.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 28th September 2008, 04:19   #24  |  Link
IanB
Avisynth Developer
 
Join Date: Jan 2003
Location: Melbourne, Australia
Posts: 3,167
@Fizick,

I had a thought on a way, for a function like MVSuper, to return a set of clips.

Currently you pack all the levels into a monster single clip, "super". Strikes me that you would really like each level to be a single ordinary clip and be able pass all these level clips around as a single unit.

If you create all these PClips and then assign each to unique var name, you only need return all the var names, say as a string.

Code:
AVSValue __cdecl Create_MVSuper(AVSValue args, void* user_data, IScriptEnvironment* env)
{
  ...
  // 
  AVSValue level0clip = AVSValue((PClip)(new LevelClip(child, 0, ... , env)));
  level0clip = env->Invoke("Cache", level0clip); // Add a cache
  AVSValue level1clip = AVSValue((PClip)(new LevelClip(level0clip.AsClip(), 0, ... , env)));
  level1clip = env->Invoke("Cache", level0clip); // Add a cache
  ...

  // Use hex of object pointer to make a unique name
  char *level0name = env->Sprintf("MVInternal_%08x", (int)level0clip.AsClip());
  char *level1name = env->Sprintf("MVInternal_%08x", (int)level1clip.AsClip());
  ...
  //
  env->SetGlobalVar(level0name, AVSValue(level0clip));
  env->SetGlobalVar(level1name, AVSValue(level1clip));
  ...

  // Pass back all the relevant info
  char *ReturnString = env->Sprintf("hpad=%d, vpad=%d, pel=%d, levels=%d, level0=%s, level1=%s, ...",
      hpad, vpad, pels, levels, level0name, level1name, ...);

  return AVSValue(ReturnString);
}
Thoughts?
IanB is offline   Reply With Quote
Old 28th September 2008, 05:52   #25  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
IanB,
If we would have so many multi-level clips (and GetFrames calls), will it pollute the avisynth cache?
Or it is dependent not on number of clips (frames), but on their sizes only?

BTW, i have got about 10% speed increasing with v.2.0, probably due to better caching
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.

Last edited by Fizick; 28th September 2008 at 05:55.
Fizick is offline   Reply With Quote
Old 28th September 2008, 06:15   #26  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
Gavino,
I agree, that now it is a very good chance to change (improve or brake) the syntax.

Implicit last works with super in MVAnalyse v2.0.
Implicit last will work perectly with MVDegrain1 and other client fuctions if I make all clips parameters as annamed:
Code:
replace 

c[super]c[mvbw]c[mvfw]c[thsad]i....

to:

cccc[thsad]i...
but in this case the syntax
MVDegrain1(super=super, mvbw=vb, mvfw=vf) will be not accepted (There are no named param super...)
so we must use :

MVDegrain1(super, vb, vf)

It is easy change of code , it may be done in any moment. I ask opinion from other users and script writers
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.
Fizick is offline   Reply With Quote
Old 28th September 2008, 06:35   #27  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
Didée,
yes this is a trick.
I do not see that it is used in any script probably beside your secret unpubliched
It is not documented way. If you need in such processing, it may be somehow implemented.

Well, here is suggested way to use different clips for motion estimation and compensation:

avisource()
super=MVSuper() # super original clip
f=blur(1.0) # filtered clip
fsuper=MVSuper(f) # super filtered clip
vb=MVAnalyse(fsuper,isb=true)
vf=MVAnalyse(fsuper,isb=false)
MVDegrain1(last,super=super,vb,vf)

So, for your trick the last line should be changed to :

MVDegrain1(f,super=super,vb,vf)

this should works in v2.0.0.3, please check.

BTW, I considered to remove source clip reading in next MVDegrain1 alpha (and get all info from super source clip), but after your request it will be preserved.
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.

Last edited by Fizick; 10th October 2008 at 22:39. Reason: new unnamed clips syntax
Fizick is offline   Reply With Quote
Old 28th September 2008, 08:09   #28  |  Link
IanB
Avisynth Developer
 
Join Date: Jan 2003
Location: Melbourne, Australia
Posts: 3,167
Quote:
Originally Posted by Fizick
If we would have so many multi-level clips (and GetFrames calls), will it pollute the avisynth cache?
Or it is dependent not on number of clips (frames), but on their sizes only?
Yes it is the total size as in SetMemoryMax(Megabytes).

As far as I understand your new code, (still reading it), all the clips and their GetFrame calls are already present in some abstracted form. My theory is to try and make them all separate regular Avisynth clips as much as possible. If a component contains image data then as a separate clip you should be able to process it with any normal Avisynth filter, e.g. once you have a level clip you may use any resizer available to make the next level if you want.

The example with Didée above with separate level clips would be a lot better. Only the levels actually used in a Super would be rendered and cached. As you currently have it you must build both complete Super clip at all levels, maybe this is still required and there is no gain to be found, I do not currently understand enough to predict.

Also for multithreading having components distinct means that maybe they can be processed in parallel.




Also for your implicit last problem you can overload the function definitions twice and use the user data value to know which one is current.
Code:
  env->AddFunction("MVDegrain1",
    "ccc[thSAD]i[thSADC]i[plane]i[limit]i[super]c[idx]i[thSCD1]i[thSCD2]i[isse]b",
    Create_MVDegrain1, (void*)0);
  env->AddFunction("MVDegrain1",
    "c[thSAD]i[thSADC]i[plane]i[limit]i[super]c[idx]i[thSCD1]i[thSCD2]i[isse]b[mvbw]c[mvfw]c",
    Create_MVDegrain1, (void*)2);
In the second pattern you must make the order different enough so that the parser cannot mismatch missing non-optional arguments. Unfortunately this changes the arg numbers but by clever use of the user data value the code can still be simple. e.g.
Code:
AVSValue __cdecl Create_MVDegrain1(AVSValue args, void* user_data, IScriptEnvironment* env)
{
  int A = (int)user_data;
  int plane = args[5-A].AsInt(4);
...
  int thSAD = args[3-A].AsInt(400);

  return new MVDegrain1(
      args[0].AsClip(),
      args[A?12:1].AsClip(), // mvbw
      args[A?13:2].AsClip(), // mvfw
      thSAD, // thSAD
      args[4-A].AsInt(thSAD), // thSADC
      YUVplanes, // YUV planes
      args[6-A].AsInt(255), // limit
      args[7-A].AsClip(), // prep clip
      args[8-A].AsInt(0),
      args[9-A].AsInt(MV_DEFAULT_SCD1),
      args[10-A].AsInt(MV_DEFAULT_SCD2),
      args[11-A].AsBool(true),
      env);
}
IanB is offline   Reply With Quote
Old 28th September 2008, 12:02   #29  |  Link
AVIL
Registered User
 
Join Date: Nov 2004
Location: Spain
Posts: 408
My two pence:

Quote:
Originally Posted by Fizick View Post
It is easy change of code , it may be done in any moment. I ask opinion from other users and script writers
I usually don't use clip arguments as named. For me will be transparent if you declare it unnamed.
AVIL is offline   Reply With Quote
Old 28th September 2008, 16:12   #30  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
Probably currently i will agree with Gavino and AVIL, and wait when IanB make some internal trick in Avisynth 2.6.X to have option to mark some named agruments mandatory (some modifier like "!")

c[super]c![mvbw]c![mvfw]c![thsad]i....

IanB,
probably trick wih strings is possible, but I am not very like globals and checking of non-coincided clip names (with multitreading).
But i like idea to have an option to produce super clip without MVTools. That is why I remove all non-pixel metadata from super frames.
It is possible to process super clip wit some user filter.
Generally say, my goals of MVTools v2.0 are more speed, clarity and stability. With using of normal avisynth cache and removing idx and other hacks.
IMO, these goals are achieved

Note. I assume that it is not very hard to add some MVTools parameters (mostly MVAnalysisData) to videoinfo structure in avisynth.h like:

int mvtools_version;
int mvtools_hpad; (or may be simply mvtools1)
...
etc, and some reserved space for future using:
mvtools_reserved1; (or may be simply reserved1)
mvtools_reserved2;
...

It is only several bytes fo my metadata parameters.
If we will add them to the end of vi structure, we can preserve compatibility with 2.5 plugins, right?
I do not see many other plugins what need in such parameters.
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.
Fizick is offline   Reply With Quote
Old 29th September 2008, 00:37   #31  |  Link
IanB
Avisynth Developer
 
Join Date: Jan 2003
Location: Melbourne, Australia
Posts: 3,167
Quote:
Originally Posted by Fizick View Post
Probably currently i will agree with Gavino and AVIL, and wait when IanB make some internal trick in Avisynth 2.6.X to have option to mark some named agruments mandatory (some modifier like "!")

c[super]c![mvbw]c![mvfw]c![thsad]i....
Be aware that it will be a very long wait. I have given you several workarounds in the past and have just given you another one.


Quote:
probably trick with strings is possible, but I am not very like globals and checking of non-coincided clip names (with multitreading).
You don't have to use a string, it was just an idea to pass information to allow you to find the needed clip objects without the problems we had using AVSValue arrays in scripts. There is no problem using globals here. By using the hex of the clips object address in memory for the name they will be unique even with multi threading. Two objects cannot occupy the same memory. By assigning the clip to a var it means you can use env->Invoke() or other functions to process it internally. The vars do not need to be global they just need to have just enough scope so you can latter get the PClip value and add it into the graph.

Quote:
But I like idea to have an option to produce super clip without MVTools. That is why I remove all non-pixel metadata from super frames.
It is possible to process super clip with some user filter.
But still the super clip is not a stardard clip, it is packed data from what should really be a matching set of clips. I would urge you most strongly to explore all means to stop packing what should really be individual but associated clips. You have a lot of complicated code to manage this packing and it will just go away with individual PClips.

Quote:
Generally say, my goals of MVTools v2.0 are more speed, clarity and stability. With using of normal avisynth cache and removing idx and other hacks.
IMO, these goals are achieved
And yes you are achieving that goal. I am just trying to provide extra ideas to do even better.

Quote:
Note. I assume that it is not very hard to add some MVTools parameters (mostly MVAnalysisData) to videoinfo structure in avisynth.h like:

int mvtools_version;
int mvtools_hpad; (or may be simply mvtools1)
...
etc, and some reserved space for future using:
mvtools_reserved1; (or may be simply reserved1)
mvtools_reserved2;
...

It is only several bytes for my metadata parameters.
If we will add them to the end of vi structure, we can preserve compatibility with 2.5 plugins, right?
I do not see many other plugins what need in such parameters.
Inside your plugin you can extend any structure in any way you want, but any extra data will not be preserved outside your plugin. VideoInfo is 48 bytes long so once outside your plugin only the first 48 bytes will ever be copied.

vi.num_audio_samples=nHeight+(nHPad<<16)+(nVPad<<24)+((_int64)(nPel)<<32)+((_int64)nModeYUV<<40)+((_int64)nLevels<<48);

If you continue with using a packed super clip then you might as well just pack this data into every super clip frame. If you use my string idea then it comes for free. If you use an AVSValue array as we had discussed a while ago then it is also for free.

Last edited by IanB; 29th September 2008 at 00:40.
IanB is offline   Reply With Quote
Old 30th September 2008, 04:46   #32  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
AVIL,
I switch to planar packed to interleaved YUY2 (same as kassandro) for super clip storage in next alpha. It is faster.
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.
Fizick is offline   Reply With Quote
Old 30th September 2008, 04:48   #33  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
IanB, can 48 bytes be extended in v2.6?
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.

Last edited by Fizick; 8th November 2008 at 09:06.
Fizick is offline   Reply With Quote
Old 30th September 2008, 07:22   #34  |  Link
IanB
Avisynth Developer
 
Join Date: Jan 2003
Location: Melbourne, Australia
Posts: 3,167
Quote:
Originally Posted by Fizick
IanB, can 48 bytes be extended in v2.6?
It would not help because baked in code in any 2.5 filters will still only copy 48 bytes.
IanB is offline   Reply With Quote
Old 30th September 2008, 15:32   #35  |  Link
AVIL
Registered User
 
Join Date: Nov 2004
Location: Spain
Posts: 408
Quote:
Originally Posted by Fizick View Post
AVIL,
I switch to planar packed to interleaved YUY2 (same as kassandro) for super clip storage in next alpha. It is faster.
Due to mt-masktools usage, converting my yuy2 clips to planar form is a frequent task. Surerly i can profite previous conversions when use mvtools. No problem then.

An idea. ¿Could be make compatibility between kassandro's YUY2 planar form and YV16?

Anyway thanks for your efforts.
AVIL is offline   Reply With Quote
Old 30th September 2008, 18:05   #36  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
IanB,
it is not a irresistible reason to stop progress.
Let's add first extra vi parameter as an some keymark (Is26ExtraParamsUsed), and check this key in new 2.6 filters and plugins with appropriate ThrowError message if it is not coincide.

AVIL,
are you use Avisynth v2.6 prealpha? What do you mean as "compatibility" ? Some format conversion function (plugin)?
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.
Fizick is offline   Reply With Quote
Old 30th September 2008, 18:38   #37  |  Link
AVIL
Registered User
 
Join Date: Nov 2004
Location: Spain
Posts: 408
@Fizick:

Actually i use version 2.5.7.

For compatibility means that clips used by interleaved2planar()/planar2interleaved() follows the avisynth planar structure :

http://avisynth.org/mediawiki/Filter..._Planar_Images

so almost native (or external) plugin that accepts planar files (yv12, yv16,...) could accepts this clips too. Number of conversions in complexe scripts (see http://forum.doom9.org/showpost.php?...&postcount=150) could be dramatically reduced.
AVIL is offline   Reply With Quote
Old 30th September 2008, 19:19   #38  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
AVIL,
I missed, that MaskTools recently (in February ) got support of yuy2 planar.
I will consider to add support of this to MVTools
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.
Fizick is offline   Reply With Quote
Old 2nd October 2008, 13:12   #39  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
here is next 2.0.7.0 alpha.
http://avisynth.org.ru/mvtools/mvtools-v2.0.7.0.zip
Docs is absent.
Implemented functions:
MVSuper(source, hpad, vpad, pel, levels, chroma, sharp, rfilter, pelclip, isse, planar)
MVAnalyse(super,blksize,...)
MVFlowFps(source,super,mvbw,mvfw,...,planar=true)
MVFlow(source,super,vec,...,planar=true)
MVCompensate(source,super,vec,...,planar=true)
MVShow(source,vec,...,planar=true)
MVMask(source,vec,...,planar=true)
MVDegrain1(source,super,mvbw,mvfw,...,planar=true)
MVDegrain2(source,super,mvbw,mvfw,mvbw2,mvfw2,...,planar=true)

So, all functions (besides MVAnalyse) got planar parameter for YUY2 planar input and output. Default = false (i.e. normal interleaved YUY2), slower.
All (mandatory) clip parameters lost their names, and mvbw=vb is not correct syntax now.
Use unnamed syntax. I will consider IanB's suggestion later, afrer small "vacations"
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.
Fizick is offline   Reply With Quote
Old 13th October 2008, 21:21   #40  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
here is next 2.0.9.0 alpha.
http://avisynth.org.ru/mvtools/mvtools-v2.0.9.0.zip

Implemented more functions without idx: MVSCDetection, MVDepan, MVFlowInter, MVBlockFps, MVFlowBlur, MVDegrain3, MVRecalculate
Fixed crashes with MVSuper(chroma=false)
Some small changes and fixes for v2.0 of course
Documentation for v2.0, see here: http://avisynth.org.ru/mvtools/mvtools2.html

I also have an intention to remove some obsolete functions, in particular MVDenoise, MVFlowFPS2
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.
Fizick is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 09:26.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.