Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 21st October 2007, 17:11   #1  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
MVTools without idx

MVTools without idx?

I consider how to remove idx from MTools to make it native avisynth plugin without internal buffers and hack,
in particular for multithread.

What is idx?
It is not my design (but Manao). Here is my analisys.
MVtools internally calculate frames hierarhy structure: original size, reduced by 2, by 4, etc, and upscaled to pel accuracy (3 additional planes for pel=2). Also different YUV planes are stored.
Structures with different idx calculated for different clips.
Various MVtools functions share same memory internally for this structure (to prevent duplicate calculations and memory usage).
first MVAnalyse reserve the memories, with some hack, store and give pointer of arrays of idx to other instances or functions (in number of audio channels).

Suggestion:
Introduce new function to remove internal hack with pointer:

MVPrepare (clip, int "pel", int "level", bool "chroma", int "sharp")

input is source clip.
output is clip of complex "superframes", i.e. hierarhical frame planes.

Usage:

source=avisource("progressive.avi")
prepared=MVPrepare(source,pel=2)
vb=MVAnalyse(prepared, isb=true)
vf=MVAnalyse(prepared, isb=false)
MVDegrain1(source,vf,vb,prepared)

Impementation problems:

How to store structures in clip.
What format of clip? Probably for YV12 we could store separately Y,U.V superframes in yuv planes, and original (or padded) width.
But for YUY2 we can't (we do not have YV16 now).
probably it is better to store in RGB format with height=1 (same as vector stream).
So, frames YUV may be mixed (interleaved).

So, it will be like (pel=2):

plane00Y (level 0, pel index 0, Y)
plane01Y
plane02Y
plane03Y

plane00U
plane01U
plane02U
plane03U

plane00V
plane01V
plane02V
plane03V

plane10Y (level 1)
plane10U
plane10V

plane20Y (level 2)
plane20U
plane20V
...


For every subplane we need to store several (class) parameters.
They are currently:
Uint8 **pPlane;
int nWidth;
int nHeight;
int nExtendedWidth;
int nExtendedHeight;
int nPitch;
int nHPadding;
int nVPadding;
int nOffsetPadding;
int nPel;
bool isse;
bool isPadded;
bool isRefined;
bool isFilled;

Seems, they should be stored in avisynth framebuffers too (before every plane data).
Probably offset of every plane from start of framebuffer must be stored in framebuffer.
Currently planes memory are allocared with "new", so they are in different places.
Probably in every planes parametes block we should also store offset to next plane parameters block.
Seems, redesign is large but may be done.

Next problem is properties transfer.
MVanalyse need in width, height, colorspace of original clip.
Wel, some of them may be transtered instead of clip "audio" properties (audio_samples_per_second, sample_type, num_audio_samples, nchannels)
audio_samples_per_second should be 0,
sample_type may be used as pixel_type
num_audio_samples may be used as width
ncannles as height

(may be i missed something)

Sorry my English and mess.
May somebody can do it (may be myself)

...to be continued...

Last edited by Fizick; 10th October 2008 at 20:20.
Fizick is offline   Reply With Quote
Old 21st October 2007, 19:04   #2  |  Link
MfA
Registered User
 
Join Date: Mar 2002
Posts: 1,075
If you are feeling up to it why not add a generic metadata interface to 2.6 instead of storing stuff in a separate clip?

IanB suggested frame->GetUserData(key) and frame->SetUserDate(key, value) as the methods.

You'd have to use a LUT which used the pointer of the VideoFrame object together with the key to find the relevant data (because you can't add actual data/pointers to the object itself without breaking binary compatibility).

Last edited by MfA; 21st October 2007 at 19:29.
MfA is offline   Reply With Quote
Old 21st October 2007, 19:41   #3  |  Link
krieger2005
Registered User
 
krieger2005's Avatar
 
Join Date: Oct 2003
Location: Germany
Posts: 377
I don't know if it is possible to use it in avisynth but is it not easier to replace the idx-functionality with some shared-memory-mechnism? The you should not create new Functions (for the avisynth-user), don't need to expand the avisynth-functionality and can create interfaces you like.

EDIT: I see. This discussion can be read at the MT-Thread

Last edited by krieger2005; 21st October 2007 at 20:09.
krieger2005 is offline   Reply With Quote
Old 21st October 2007, 23:27   #4  |  Link
AVIL
Registered User
 
Join Date: Nov 2004
Location: Spain
Posts: 408
Hi,

To maintaint YUY2 compatibility perhaps you could use YUY2 planar as in kassandro's filters. Then you need not reinvent the wheel.

See section "Color spaces":

http://home.pages.at/kassandro/RemoveGrain/

See also this thread :

http://forum.doom9.org/showthread.php?t=91598

Thanks for your works.
AVIL is offline   Reply With Quote
Old 22nd October 2007, 02:37   #5  |  Link
IanB
Avisynth Developer
 
Join Date: Jan 2003
Location: Melbourne, Australia
Posts: 3,167
@Fizick,

First up another instance of a former problem I stumbled over. Manao's bad habit of using child->GetFrame(n, env)->GetReadPtr(), there is another instance in MVClip.cpp line 90.

For other readers, a quick summary. GetFrame returns a PClip which is a smart pointer. When the PClip goes out of scope the underlying frame data gets released. With the above construct the temporary intermediate PClip goes out of scope immediatly, rendering the returned pointer as a pointer to potentially free'd and/or reuseable memory. In the single threaded case, you probably get away with it as long as you do not do any GetFrames or NewVideoFrame calls before you are done using the data under the pointer. In the multithreaded case any random call to NewVideoFrame could well steal the frame buffer just returned.

Okay down to discussion.

MVAnalyze() currently returns a RGB32 Clip, height=1, Width=<big enough to hold the vectorFields data>, nchannels=&analysisData.

analysisData holds various state data about what is happening and what info is currently stored. This is a very bad thing! Even in the single thread case, doing disjoint frame access to a MV function spoils any stored state data making it only usefull when frame access order is exactly as expected.

A proper fix for the issue is to first simplify and normalize analysisData based on the state for frame "n". And then include it or references to it in the returned MVAnalyze()'s clip GetFrame data.

The idea is to make GetFrame the only method of passing data between a MV function and MVAnalyse(). And all the state data that MVAnalyse stores be packed into the frames returned by GetFrame. i.e. extend the current use in addition to just the vectorFields data include the state for it as well.

I do not think you will need to add an MVPrepare, however it may simply the design if after normalizing analysisData there is sufficient common state data that doing a GetFrame chain between MVAnalyse() and MVPrepare() could radically simply the GetFrame chain between MV{whatever}() and MVAnalyse().

A cunning way to manage any buffers needed, like plane00Y etc, in MVAnalyze() would be to build pseudo clips internally for each type and attach a normal avisynth Cache then use GetFrame() to retrieve the required one. It would be quite legal to return a set of PClip's and "N" values in the vectorFields data if needed.

So a MV{whatever} would do {MVAnalyse}->GetFrame(n, env) calls and retrieve the state and vectorFields for frame "n". Any MVCore interface routines needed would be called with the appropriate PClip and the value "N" from the returned data. The core routine then internally does a {PClip}->GetFrame(N, env) to retrieve whatever buffer it wants. If it is in the cache then it comes for free else it gets built as normal.

The thing to keep in mind when (ab)using the avisynth cache to stash data buffers is they may get reused, so you always need to be able to regenerate the original contents. And everytime you generate contents they must be bit identical.

:Edit: - Okay having read more I see mvCore is process global, so to replace the IDX concept you will need MVPrepare or similar. Above coloured grey I now see is partially wrong.

With MVPrepare you can make mvCore a member of the MVPrepare object and buffer all the MVGroupOfFrames objects thru GetFrame calls.

:Edit 2: For your MVPrepare output clip stick with RGB32, height=1, for the moment. This offers 32bit storage units and no restrictions. In 2.6 I have a "Raw Data" pixel type on my todo list, it will probably internally mirror RGB32 but externally it will be a distinct pixel type that most filters will refuse to process (stuff with).

For passing your metadata around just jam a pointer to it into the front of your GetFrame data, it will also have the benefit of being a signature to check in case users try to feed a wrong clip into your routines.

For holding the plane data you can continue to manage it internally and just pass pointers in your GetFrame data, this will require minimal changes, maybe do this first.

Or you can go the whole hog and build a PClip for each plane and do {PClip}->GetFrame(n, env) to recover each element. Or perhaps somewhere in the middle with grouping an entire level into a PClip. As I read more I may understand better the issues on how to decide.

Last edited by IanB; 22nd October 2007 at 08:46.
IanB is offline   Reply With Quote
Old 22nd October 2007, 17:54   #6  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
Mfa, metadata is not implemented in current 2.6 alpha.
(besides new pixel_type).


krieger2005,
I do not know about shared-memory-mechnism. How user can say to avisynth (without idx or MVprepare), what data could be shared and what not?

AVIL,
Yes, thanks, I remember about it, and similar approach may be used.

IanB,
1. Are you sure that you get latest version 1.8.4 of MVTools?
IMO, the "former problem" was solved (thank to your advice) in v.1.4.13 about one year ago.
There is no any GetFrame call in MVClip now.

2. IMO the new MVPrepare is the only way to remove idx effectively.
Without common MVPrepare (and without idx), every MVAnalyse(for different offsets or direction) will need in own (duplicate) calculation of same super frame interpolation. It will also result in huge memory eating (superframes are big) - without sharing.
It is equivalent to case of unique idx for every MV function now.

to be continued
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.
Fizick is offline   Reply With Quote
Old 22nd October 2007, 18:12   #7  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
3. Other attempt to decribe how MVTools and idx works (it is not to IanB, but to somebody else and myself .

For example consider MVDegrain1() is requested by frame n=10 (and it is start of processing).
MVDegfrain1 requests vector frame 11 from MVnalyse(backward). To produce it, MVnalyse(backward) is need in superframe 10 and 11.
It calculate them and put both them (10,11) to "idx internal clip".
MVDegrain1 also requests vector frame 9 from MVnalyse(forward). To produce it, MVnalyse(forward) is need in superframe 9 and 10.
It can get superframe 10 from "idx internal clip", calculate superframe 9 and put it to "idx internal clip".

Note: supperframes with same N are the SAME for any direction (backward,forward) and any frame estimation delta!
Of course, for same source clip. That is why we use idx.

Next step is n=11 (for usual sequencial access).
Two needed superfames (10, 11) are already calculated at previous step and may be taken from "idx internal clip", besides new superframe 12.

The cache buffer size is limited (MV_BUFFER_FRAMES=10),
when all spaces are filled, the oldest superframe is deleted and replaced by new.

If access is not sequencial, story is not so good.
For example if we use SelecEven after MVDegrain, the frame n=12 is requested instead.
We need in superframes 11, 12, 13.
Superframes 9, 10, 11 are in "idx internal clip" cache, so we have 11 and must calculate new superframes 12 and 13 now.


4. About IanB's suggestion to create set of (pseudo)clips internally in MVAnalyse.
As I wrote above, there are several MVanalyse command in script, so these clips will be duplicated again.
How to communicate and share calculated frames buffers? With some globals?
Do not forget, several source clips (now with different idx) may be in a script, for example with MVFlowfps2 or prefiltered clips.

Probably we could create these pseudoclips in MVPrepare.
But I do not quite understand about "quite legal return a set of PClip's and "N" values".
Can "prepared" clip (see example script above) be not single clip but (AVSValue) ARRAY of pseudoclips (with array size = number of hierarchy scale levels)?
Is it possible in Avisynth syntax? (I never saw any such filter. )
If yes, this way may be considered.

I wrote above about some problem with YUY2 case (as AVIL mentioned, we could try use YUY2 planar as in kassandro's filters).
Storage of pel-interpolated data must be considered too: one big upscaled clip or 4 clips of subframes(as currently implemented).

Of course, it would be fine do not put any extra metadata to these videoframes,
but I am afraid "audio" properties number is not enough.
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.
Fizick is offline   Reply With Quote
Old 22nd October 2007, 20:30   #8  |  Link
tsp
Registered User
 
tsp's Avatar
 
Join Date: Aug 2004
Location: Denmark
Posts: 807
Quote:
Originally Posted by Fizick View Post
Of course, it would be fine do not put any extra metadata to these videoframes,
but I am afraid "audio" properties number is not enough.
What about saving it in the frame that MVPrepare returns with all the other stuff? If you are going to save the extra parameters for the subplane you could also save the original width,height, colorspace, etc for the original clip. It shouldn't take many cycles to copy that.
__________________
Get my avisynth filters @ http://www.avisynth.org/tsp/
tsp is offline   Reply With Quote
Old 22nd October 2007, 23:12   #9  |  Link
IanB
Avisynth Developer
 
Join Date: Jan 2003
Location: Melbourne, Australia
Posts: 3,167
@AVIL, 2.6 Has planar YUY2, it is called YV16.

@Fizick, Yes I got confused with many archived sources. I have 1.8.4 but stupidly was reading an old 1.4.8 source sorry for causing any panic attack.

Yes I now agree, the MVPrepare type approach, is a very favourable approach. I changed my post above 2 times as I read more source, so some is not quite relevant anymore.

I do not think my internal pseudo clip idea is that good any more. For the future it may be desirable to use external filters (i.e NNEDI, etc) to generate/calculate the superframe clips, so I think it would be helpful make the superframe data regular clips where possible.

As tsp says for the metadata just put the data or a pointer to the data in the MVPrepare returned frame.

Last edited by IanB; 23rd October 2007 at 00:09.
IanB is offline   Reply With Quote
Old 23rd September 2008, 22:12   #10  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
I am still preparing for MVPrepare

more correct syntax should be with padding parameters:

MVPrepare (clip, int "pel", int "levels", bool "chroma", int "sharp", int "hpad", int "vpad", clip "pelcip")

currently in MVTools padding=blocksize.

IMO, to have superframe compatible with external resizing tools, it is better do not put any parameters to frame data,
and try transmit them in packed (hacked) audio properties.
Every parameter (pel, ... vpad) value is really small, so we can use BYTE.
Four parameters = one int, for example sample_type.

this way superframe is simply stacked multilevel image data.
Every color plane may be stored indpendently for YV12.
For YUY2 we all waiting for avisynth 2.6 (but packing-repacking way is also possible).
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.

Last edited by Fizick; 25th September 2008 at 19:19.
Fizick is offline   Reply With Quote
Old 24th September 2008, 17:43   #11  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
question (to TSchneide?)
Why align planes in MVPlane ?
I see x256sad cache spilt code, and understand why ALIGN_SOURCEBLOCK in PlaneOfBlocks.
But anyway refence block may be in any place, so why to align it?

I want create planes data in normal Avisynth frames and do not want to add additional aligment.
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.

Last edited by Fizick; 25th September 2008 at 04:34. Reason: I am stupidly post wrong user nick
Fizick is offline   Reply With Quote
Old 26th September 2008, 04:21   #12  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
OK, here is preliminary alpha version 2.0.0.1
http://avisynth.org.ru/mvtools/mvtools20.dll

It contains only MVPrepare function.

Can anybody suggest better name for it and for clips its produced (for future MVAnalyse)?
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.
Fizick is offline   Reply With Quote
Old 26th September 2008, 10:32   #13  |  Link
mikeytown2
Resize Abuser
 
mikeytown2's Avatar
 
Join Date: Apr 2005
Location: Seattle, WA
Posts: 623
Quote:
Originally Posted by Fizick View Post
Can anybody suggest better name for it and for clips its produced (for future MVAnalyse)?
MV Prep
MV Pointer
MV Container
MV Root
MV Threads
MV Superframe
MV Wrapper
mikeytown2 is offline   Reply With Quote
Old 26th September 2008, 15:28   #14  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
I considered MVMulti , but it is a little confused

prep=MVPrepare(source)
vec=MVEstimate(source,prep)
MVDegrain(source,vec,prep)

source may be omitted, but it will mostly provide audio and some clip properties.

I like to have dozen Motion properties in avisinth.h v2.6
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.
Fizick is offline   Reply With Quote
Old 27th September 2008, 13:07   #15  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,431
Quote:
Originally Posted by Fizick View Post
prep=MVPrepare(source)
vec=MVEstimate(source,prep)
MVDegrain(source,vec,prep)

source may be omitted, but it will mostly provide audio and some clip properties.
Will this new scheme fix the problems with 'implicit last' that we discussed recently?
Gavino is offline   Reply With Quote
Old 27th September 2008, 14:50   #16  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
implicit last will be functional if I replace named clip parameters to unnamed. I am not sure that i will do it.

but anyway all existant script must be changed

avisource()
super=MVSuper()
vecb=MVAnalyse(super,isb=true)
vecv=MVAnalyse(super,isb=true)
MVDegrain1(last,super,vb,vf)

we need in users opinions.
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.

Last edited by Fizick; 27th September 2008 at 15:27.
Fizick is offline   Reply With Quote
Old 27th September 2008, 16:36   #17  |  Link
TSchniede
Registered User
 
Join Date: Aug 2006
Posts: 77
Quote:
Originally Posted by Fizick View Post
question (to TSchneide?)
Why align planes in MVPlane ?
I see x256sad cache spilt code, and understand why ALIGN_SOURCEBLOCK in PlaneOfBlocks.
But anyway refence block may be in any place, so why to align it?

I want create planes data in normal Avisynth frames and do not want to add additional aligment.
It's simple... the functions we use from x256 are only faster because they can depend on aligned access. The best case would be to compare only two aligned blocks, but we must shift either the source or the destination block freely, so one of them should be unaligned in most cases. The trouble is, that only MMX tolerates unaligned access most SSE instructions don't. To make matters worse, any newer intel chip hat a noticeable penalty for a memory access across physical cache line borders and an even worse penalty if that also happens to coincide with a memory page border. So there are unfortunately only two possible options right now: use simple MMX and ignore the penalty (which is essentially using the old isse functions) or use a work around like in x256sad and align the memory blocks (this could be done once for each frame if all data is sized in accordingly (padding and the like)).
Compensating on both sides source and destination is only possible for MMX but the overhead should be far greater than the gain (ok it would be possible but it is certainly slower).
__________________
GA-P35-DS3R, Core2Quad Q9300@3GHz, 4.0GB/800 MHz DDR2, 2x250GB SATA HD, Geforce 6800
TSchniede is offline   Reply With Quote
Old 27th September 2008, 16:57   #18  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
I do not understand.

SAD(pSrc[0], nSrcPitch[0], pRef0, nRefPitch[0])

Src is artificantly aligned (ALIGN_SOURCE_BLOCK)

But Ref block is GetRefBlock(vx, vy). Why align Ref plane, if vector may be any, and Ref block will be unaligned (in most cases) anyway?
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.
Fizick is offline   Reply With Quote
Old 27th September 2008, 17:59   #19  |  Link
TSchniede
Registered User
 
Join Date: Aug 2006
Posts: 77
Quote:
Originally Posted by Fizick View Post
I do not understand.

SAD(pSrc[0], nSrcPitch[0], pRef0, nRefPitch[0])

Src is artificantly aligned (ALIGN_SOURCE_BLOCK)

But Ref block is GetRefBlock(vx, vy). Why align Ref plane, if vector may be any, and Ref block will be unaligned (in most cases) anyway?
I have to admit, this is somewhat excessive. It's mostly because I did that first, but it prevented overlapped blocks. There is still a small profit though, as the copy code doesn't compensate for cacheline splits. Ensuring that most the cacheline splits occur on the borders(16xY blocks without overlap) or at least exactly in the middle (16xY with 8 overlap or 8xY without overlap) reduces the penalty. It might be less than the alignment penalty though - I forgot to check
__________________
GA-P35-DS3R, Core2Quad Q9300@3GHz, 4.0GB/800 MHz DDR2, 2x250GB SATA HD, Geforce 6800
TSchniede is offline   Reply With Quote
Old 27th September 2008, 18:19   #20  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
TSchniede,
Please also look to new v2.0 code ( when it will be released ) to check the issue.
I use usual avisynth framebuffer as a storage of all (super) planes.

Preliminary alpha v2.0.0.3 is probably almost ready (not tested) with functions MVSuper, MVAnalyse, MVDegrain1, MVShow only.
(other MVXXX code temporary removed from project).
I can upload MVTools.DLL and source
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.
Fizick is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 18:51.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.