AviSynth 2.5.7 RC-1 [Sept 16th]

[IanB]

Here is release candidate 1 for Avisynth 2.5.7. The intention was for this to be a bug fix release only, but a few minor gems have been slipped in. :)

Notes:- I forgot to turn of the COM environment debugging option when I made this build. The final release version will not have this feature. (It can cause problems with miserable software that employs anti-debugging technology. GRR!)

2.57 Changes from 2.56
Additions
Installer option for avisynth.lib, avisynth.exp and avisynth.map files.
DirectShowSource() now sets _HIRESOUTPUT property of WMAudio Decoder DMO. Allows 24 bit samples and 6 channel decoding.
Avisynth COM object now supports IAVIStreaming interface. Many apps blindly expect it and fail to load AVS scripts.
Conditional Reader now also accepts T, F, Yes and No for bool data.
DirectShowSource() log debug messages to file.
DirectShowSource() force set framecount.
DirectShowSource() accept audio streams of type WAVE_FORMAT_IEEE_FLOAT.
Added KillVideo() verb.
TimeStretch now exposes the tuning parameter of the SoundTouch library.
Adding global OPT_AllowFloatAudio=True to your script enables WAVE_FORMAT_IEEE_FLOAT audio output.
Explicitly request all the channels available in the audio stream [acm AC3]
Explicitly try to request float, 32 bit then 24 bit samples from the audio codec.
Accept raw audio streams of type WAVE_FORMAT_IEEE_FLOAT.
Added installer option to add "Avisynth Script" item to Explorer shell for new .avs file creation.
Added avs_delete_script_environment and avs_subframe_planar to avisynth_c interface.
Hack to allow Fraunhoffer MP3 codec to work when wBitPerSample==16. (Squid_80)
Added portugese translation (by RoLon), and partly french translation (by JasonFly)
Subtitle multi line text, set LSP arg and use \n. (foxyshadis, tateu)
xxxFPS("preset") string preset FPS values. (Tritical)
Better avisynth_c cdecl/stdcall mismatch detection and protection (Tritical).
Bugfixes
Fixed ConvertFPS() blend mode mix_ratio was wrong.
Fixed AviSource() handling of dud AVI files that start with drop frames, just return the first key frame.
Fixed DirectShowSource() handling of pin renegotiation.
Fixed Overlay() "PC_Range" argument handling.
Fixed SubTitle() crash when resources exhausted.
Fixed Resizers cropping argument validation.
Fixed % operator bad argument text message.
Fixed stuffing Audio buffer with video graphics on Exception. Now returns E_FAIL. Video stream subsequently will return error text in graphic form.
Fixed KillAudio() not resetting nchannels and sample type to 0.
Fixed Conditional Reader sscanf buffer overrun.
Fixed Version() and VersionString() timestamp discrepancies.
Fixed Serious inaccuracies in TimeStretch. It is still not sample exact!
Fixed Negative int arg handling for ApplyRange() and Animate().
Fixed Installer handling of readonly/inuse target files.
Fixed ConvertAudio SSE2 to Float alignment test.
Fixed (auto)LoadPlugin altname generation. (Tritical)
Fixed SaveString memory block overrun.
Fixed $Plugin!Functionname!Param$ bug. (Fizick)
Fixed registry handle leak on $PluginDir$ lookup. (Dave Brueck)
Fixed memory leaks avisynth_c.
Fixed returning locked/protected VBF's to LRU. (Tritical)
Fixed runtime mixed SEH/C++ exception handling for XPsp2. (Tritical)
Fixed CAVIStreamSynth::Read audio buffer overrun. (Avery Lee)
Fixed DLL handle leak in LoadPlugin. (Tritical)
Fixed Assert("text") no longer parses % args.
Fixed number parser returning inaccurate float conversions.
Fixed ConvertFPS() blend mode not processing of chroma planes.
Fixed resizer resampling pattern attempted use after deletion.
Fixed resizer subpixel shifting functionality being a noop.
Fixed Info() auto font selection metric.
Fixed Conditional error checking of float RHS.
Corrected colours in YUV ColorBars, Now match BT.801-1.
TCPDeliver updates: Client: Fixed crash if client gets disconnected.
TCPDeliver updates: Server: Remember to disconnect clients when shutdown.
Fixed Turn*() YUY2 mod 2 height test.
Fixed AVISource() corrupted error messages.
Fixed AVISource() direct input drop frame handling.
Optimizations
SubTitle() releases all resources when the last frame of the clip is rendered.
SubTitle() releases GDI resources after text map is created.
ApplyRange() only builds 2 chains instead of 3.
Changes
Map file is now generated for release builds.
ConvertTo*() and GreyScale() now accept "Rec601" as a valid matrix name.
DirectShowSource.dll upx'ed.
Upgrade internal copy of SoundTouch library to 1.3.1.
SoundTouch now available in RelSym build.
Animate Integer arg enumeration no longer rounds toward positive infinity. Both positive and negative enumerations are identical. i.e. For Animate(0, 10, "Foo", 0, 0, 10, -10), Foo's arg1 = -arg2 for all frames.
Selecting associate open with Notepad with .avs files in the installer now also includes .avsi files.
The Installer now pushes a recovery dialog box when unwritable files are encountered during an install. The user may manually correct the problem and retry installing that file.
SetMemoryMax() minimum now 4Mb instead of 16. (Tritical)
Remove 50 plugin auto prescan load limit. (Tritical)
COM QueryInterface calls now return S_OK instead of NULL.
Bracketless call of argless function now get a cache. (Tritical)
Over-range numbers now raise a compile time exception.
xxxFPS(float) now uses continued fraction to generate a minimal rational pair. (Raymod2)
ChangeFPS(linear=true) now raises a compile time exception if the speed change ratio is > 10.
ConvertFPS() blend mode works for all pixel formats. (Tritical)
Info() retrofit of 2.60 updates.
TCPDeliver.dll upx'ed.
RGB ColorBars +Q and -I bars, Hue is now correct, Luma is NOT zero to achive this.
AVISource Audio no longer limited to 2 channels.
SaveString memory blocks are now 32 bit aligned.
Default planar chroma planes mod 16 aligned. See SetPlanarLegacyAlignment().
As usual download from sourceforge (http://prdownloads.sourceforge.net/avisynth2/AviSynth_160906.exe?download)

[Mug Funky]

sounds good. will try ASAP.

[Fizick]

avisynth.lib, avisynth.exp and avisynth.map?
It is for c-plugins SDK?
May be to add avisynth.h and avisynth_c.h to them?

[IanB]

Yes. The map is just so it exists and is bound to the appropriate avisynth.dll

And good idea.

[Zarxrax]

Wow, some nice fixes and changes in this release, good job!
On the documentation, would it be possible to add a search feature? I think this could be pretty helpful.

[Wilbert]

On the documentation, would it be possible to add a search feature? I think this could be pretty helpful.
It's just a bunch of html files. How do you propose to do that?

[Zarxrax]

It's just a bunch of html files. How do you propose to do that?

I honestly don't know, though I have seen it in other documentation that uses html (adobe stuff, for example). I just assumed (incorrectly perhaps) that it might be easy to implement.

[IanB]

@Wilbert,

Aaahhhhg, you missed your chance. :D

You should have responded :D "Nice of you to volunteer to implement this useful feature" :D

[foxyshadis]

A .chm! But it's not the easiest thing to get the stupid projects to work. If I can find the right free software that lets you import and compile with no fuss (the microsoft one imports fine, won't compile right!), I'll post it and a project file.

[Fizick]

Zarxrax,
what do you really want to find? :)

foxyshadis,
IMHO, chm is not optimal solution to maintain,
and we use different language (english-russian) for example.

[Suchy]

Is Avisynth optimized using by SSE2 (or MMX and SSE)?

I know that there is plugin for MultiThreating, but it isn't stabile and bugless (f. ex. after removing interlace, there appears frames with half of previous and half of next frame)

[Zarxrax]

Zarxrax,
what do you really want to find? :)

The current documentation is a great reference if you know what you are looking for, but if you don't know what you are looking for, it can be difficult to find anything.

For example, what if I want to adjust the saturation of my video? How would I determine which filter to use, besides reading the descriptions of every one until I find "Tweak"? With a search function, I could just search for "saturation" and expect it to tell me about the Tweak filter.

@others:
I have come across this open-source javascript which appears that it may provide the kind of search functionality that I am talking about: http://www.kscripts.com/scripts.shtml
If this kind of thing seems satisfactory, I could indeed try to implement it myself.

[Fizick]

may be, we can build and provide some static html file with references to important terms.

[Wilbert]

A .chm! But it's not the easiest thing to get the stupid projects to work. If I can find the right free software that lets you import and compile with no fuss (the microsoft one imports fine, won't compile right!), I'll post it and a project file.
A chm file is easy to make (there is already such a project file in the CVS, but it just needs to be updated a bit).

@others:
I have come across this open-source javascript which appears that it may provide the kind of search functionality that I am talking about: http://www.kscripts.com/scripts.shtml
If this kind of thing seems satisfactory, I could indeed try to implement it myself.
This could be what we are looking for!

I don't get it completely. You need to install Ksearch and thus perl to build the DBM database. Is it possible to access this database without installing Ksearch (and thus perl)?

[Zarxrax]

I don't get it completely. You need to install Ksearch and thus perl to build the DBM database. Is it possible to access this database without installing Ksearch (and thus perl)?
You mean for the end user? The "database" is just some stuff formatted into a javascript file, so it should be able to run on any browser that supports javascript.

[Prettz]

I'm not quite sure what's wrong here. I'm getting some horrible chroma shifting when I use Crop. The same problem was present with 2.5.7a3; I just installed RC1 to make sure and it's there in this version too, and it is. This seems like something so obvious that I'm assuming there's something wrong on my end. It's screwing up the same with "align" set to both true and false.

Here's a script that has it:

MPEG2Source("J:\terminator\terminator_full_01.d2v")
Telecide(order=1,chroma=true,post=0)
Decimate(cycle=5,mode=0)
Crop(2,58,-2,-60)


I've included a screenshot of the original before cropping and the result right after.

Odder still: when I follow the Crop immediately with a LanczosResize(704,352), I don't see any results of chroma shifting anymore.

[squid_80]

Are you sure it's not your YV12 codec?

[GodofaGap]

For example, what if I want to adjust the saturation of my video? How would I determine which filter to use, besides reading the descriptions of every one until I find "Tweak"? With a search function, I could just search for "saturation" and expect it to tell me about the Tweak filter.
Most browsers have a text search function (e.g. CTRL+F). If you click on filters page and search for saturation, Tweak is the first thing you'll see. :)

[Wilbert]

Most browsers have a text search function (e.g. CTRL+F). If you click on filters page and search for saturation, Tweak is the first thing you'll see.
That's within a page. We are talking about offline searching in the AviSynth docs. Thus start in the startpage and search for a keyword in all htmls.

You mean for the end user? The "database" is just some stuff formatted into a javascript file, so it should be able to run on any browser that supports javascript.
You are probably correct, but i can't find it in the faq:

http://www.kscripts.com/ksearch/faqs.html

If i understand it correctly you need to insert the following into one of your htmls (for which i assume you need KSearch):


To add a search box, insert the following HTML code:

<FORM ACTION="http://my_web_site.com/cgi-bin/ksearch/ksearch.cgi" METHOD="GET" NAME="search">
<INPUT TYPE="text" NAME="terms"><INPUT TYPE="submit" VALUE="Search">
</FORM>

or do i misunderstand it completely?

[foxyshadis]

That's the online ksearch. There's an offline version under it: http://www.kscripts.com/ksearch-client-side/faqs.html

[Prettz]

Are you sure it's not your YV12 codec?
I would never have imagined something could be wrong with that, but it seems you're right. A ConvertToRGB32() at the end fixes it. I first noticed it when the Crop came before some filters, so I assumed the chroma shifting was happening at the Crop.

The width was mod 4 (which I'm pretty sure has always been fine for raw YV12 before), but if I replace what I posted with Crop(4,58,0,-60,align=true), the chroma shifting isn't there anymore. What could possibly be wrong with my YV12 codec, and how might it have happened?


edit: Nevermind, the chroma shifting IS still there with Crop(4,58,0,-60,align=true), but the effect is much smaller. That doesn't make the problem not confusing as hell, though. Some kind of pitch error I guess. My memory might be bad, but I'm certain I've looked at scripts with mod 4 width before in VD without problems.

[IanB]

@Prettz,

In VDub make sure Options/Preferences/AVI/"Directly Decode YCbCr (YUV) Sources" is turned on.

The YV12 to RGB converter in some DivX and XviD version is known to require mod 8 or even 16 width to work correctly.

[IanB]

Hey language team, any chance of some activity in each of the translations before I make this RC-1 the real thing. Russian and English seem the only loved ones.

[Kostarum Rex Persia]

I hope so. Translation on Serbian language would be very very nice.

Who is in the translation team?

[Prettz]

@Prettz,

In VDub make sure Options/Preferences/AVI/"Directly Decode YCbCr (YUV) Sources" is turned on.

The YV12 to RGB converter in some DivX and XviD version is known to require mod 8 or even 16 width to work correctly.
Thanks a lot. My VD version is so old it doesn't even have that option, so it's time to get up to date. I wonder what I did to make my VfW YV12 handling go nutty, though.

I know Xvid installs a YV12 codec; doesn't Avisynth install one as well?

[Fizick]

IanB,
please notify me at last two days before final release,
I must syncronize the Russian with your latest changes

[Fizick]

I hope so. Translation on Serbian language would be very very nice.

Who is in the translation team?

you can become this member :)

[Wilbert]

I know Xvid installs a YV12 codec; doesn't Avisynth install one as well?
XviD IS a codec. No, AviSynth doesn't install a codec, you should do that yourself.

[Kostarum Rex Persia]

I contacted IanB via PM, so I hope that I will become a member of translation team.

[tebasuna51]

Additions
...
DirectShowSource() accept audio streams of type WAVE_FORMAT_IEEE_FLOAT..
...

Maybe a typo?

DirectShowSource(), with ffdshow, accept audio streams of type WAVE_FORMAT_IEEE_FLOAT in v2.5.6. And also with header WAVE_FORMAT_EXTENSIBLE.

WavSource() accept audio streams of type WAVE_FORMAT_IEEE_FLOAT in 2.5.7

Btw, what is the problem to accept WAVE_FORMAT_EXTENSIBLE?

[IanB]

@tebasuna51,

Yes it should be "... accept raw audio streams ..."


I am not aware of problems accepting WAVE_FORMAT_EXTENSIBLE please elaborate. This stuff is difficult to test due to the lack of stable sources of data to test against, so simple examples of faults with a logfile are very helpfull.

[tebasuna51]

I am not aware of problems accepting WAVE_FORMAT_EXTENSIBLE please elaborate. This stuff is difficult to test due to the lack of stable sources of data to test against, so simple examples of faults with a logfile are very helpfull.
Maybe the problem is the interface (BeHappy), but the error message seems generated in vd_audio.cpp:
Starting job 16_in_ex.wav->16_in_ex_.wav
Error: BeHappy.AviSynthException: No compatible ACM codec to decode 0xFFFE audio stream to PCM.
at BeHappy.AviSynthClip..ctor(String func, String arg, AviSynthColorspace forceColorspace, AviSynthScriptEnvironment env)
at BeHappy.Encoder.encode()
When wFormatTag is 0xFFFE (WAVE_FORMAT_EXTENSIBLE) the SubType must be checked, WAVE_FORMAT_PCM or WAVE_FORMAT_IEEE_FLOAT are valid and the audio stream must be processed in consequence.

Also Bepipe don't work but the error message is less sophisticated:
bepipe --script "WavSource(^D:\16_in_ex.wav^)" > output.wav
...
ERROR: Can't find audio stream!

The sample wav used is 16_in_ex.wav (http://www.mytempdir.com/945735)

[Ebobtron]

@tebasuna51
you asked, "Btw, what is the problem to accept WAVE_FORMAT_EXTENSIBLE?"


0x0001 WAVE_FORMAT_PCM PCM
0x0003 WAVE_FORMAT_IEEE_FLOAT IEEE float
0x0006 WAVE_FORMAT_ALAW 8-bit ITU-T G.711 A-law
0x0007 WAVE_FORMAT_MULAW 8-bit ITU-T G.711 µ-law
0xFFFE WAVE_FORMAT_EXTENSIBLE Determined by SubFormat

Above is a list of "wFormatTag"s,


WAVE_FORMAT_IEEE_FLOAT, is a valid format outside of WAVE_FORMAT_EXTENSIBLE.

That is why it works with WavSource. When WAVE_FORMAT_EXTENSIBLE is used, the format tag is moved to another location in the header you incorrectly identified as the subtype. The SubFormat member of the WAVEFORMATEXTENSIBLE Structure is not available to AviSource.

WavSource, a subfuction or mode of AviSource, when presented with 0xFFFE as the format tag, goes looking for a decompressor (which it should).

As written and compiled AviSynth doesn't know what WAVE_FORMAT_EXTENSIBLE is unless it is looking though DirectShowSource.

Just took a quick glance at the latest Free PSDK and WAVE_FORMAT_EXTENSIBLE is not in the VFW (Video For Windows) documentation. But it is part of direct show and has been.

If it could be added to the AviSource functions in the future, is beyond my ability to guess. I am still trying to decide what a WAVE_FORMAT_EXTENSIBLE file header is suppose to look like. To me the format is a little vague, but I am pretty dumb.


Hope that helps.

[squid_80]

I can think of one problem: it's always possible to convert a wFormatTag to a Subformat GUID, but not the other way around. So I'm not sure how you'd go about handling a GUID which didn't convert easily to a corresponding wFormatTag, without some sort of look-up table for known types (for instance there's about half a dozen different GUID definitions for AC3, while most acm codecs which support AC3 via waveformatex use wFormatTag 0x2000).

[Ebobtron]

@squid_80,

The conversions would not be an issue.

The rest of your post shifts over to an apples verses oranges kind of the thing.

Just different uses of the same structure.

[squid_80]

My point was it's not just as simple as saying, "Oh it's a WAVE_FORMAT_EXTENSIBLE structure, I'll just convert the SubFormat GUID to a wFormatTag and go looking for an ACM codec to handle it..." since it's not always possible to get a wFormatTag from a GUID. In some cases it's possible, but not always.

[Ebobtron]

Sorry IanB,

Thought I could help, I was wrong. Think I'll stick to asking questions.

bye

[tebasuna51]

When WAVE_FORMAT_EXTENSIBLE is used, the format tag is moved to another location in the header you incorrectly identified as the subtype. The SubFormat member of the WAVEFORMATEXTENSIBLE Structure is not available to AviSource.
For me the subtype is the first two bytes (WORD) of the field SubFormat (GUID) and have a value 0x0001 (WAVE_FORMAT_PCM) for int samples and 0x0003 (WAVE_FORMAT_IEEE_FLOAT) for float samples.
KSDATAFORMAT_SUBTYPE_PCM 00000001-0000-0010-8000-00aa00389b71
KSDATAFORMAT_SUBTYPE_IEEE_FLOAT 00000003-0000-0010-8000-00aa00389b71
KSDATAFORMAT_SUBTYPE_DRM 00000009-0000-0010-8000-00aa00389b71
KSDATAFORMAT_SUBTYPE_ALAW 00000006-0000-0010-8000-00aa00389b71
KSDATAFORMAT_SUBTYPE_MULAW 00000007-0000-0010-8000-00aa00389b71
KSDATAFORMAT_SUBTYPE_ADPCM 00000002-0000-0010-8000-00aa00389b71
KSDATAFORMAT_SUBTYPE_MPEG 00000050-0000-0010-8000-00aa00389b71
We only need to check 0x0001 and 0x0003 not all possible GUID values.

But if SubFormat is unknown inside AviSource-WavSource we can do nothing. Thanks.

Edit: If we have access to wav header is really easy (if we plan reject anything than PCM or FLOAT) change wFormatTag for this first two bytes of SubFormat. Is used in Aften (http://forum.doom9.org/showthread.php?p=853988#post853988).

[Ebobtron]

@ tebasuna51,
I am, again, sorry. You may call a pig a cow, but I may not understand. Others will not either.

I didn't mean to suggest that you didn't know your way around the data types, especially the really big ones.

My suggestion is that you did not understand why AviSynth(WavSource()) would load WAVE_FORMAT_IEEE_FLOAT and not load WAVE_FORMAT_EXTENSIBLE.

Again I am assuming that your asking a question and you would like the answer.

Sorry I has incorrect.
done.

[Fizick]

It seems, there is some bug in Avisynth smart cache impementation,
see MVTools thread
http://forum.doom9.org/showthread.php?p=867682#post867682
I discovered, that backward and forward vectors clips sometimes swapped after 12 frames.
Version 2.5.5 and v2.5.6beta3 (with not smart cache) works fine with MVDenoise.
I try to find workaround with some SetCacheHints,
but some problem still exists.
Also I remember similar problem with Restore24.

[IanB]

Btw, what is the problem to accept WAVE_FORMAT_EXTENSIBLE?Okay, to clarify, I take it more fully this question is :-

Why does WavSource() fail to accept WAVE_FORMAT_EXTENSIBLE type files.

The answer to this question is because the code only has access to a WAVEFORMATEX structure not a WAVEFORMATEXTENSIBLE structure, which is required to access the subtype information. The code has been borrowed from VirtualDub a fair while ago. It is very convoluted and may well be lacking some of the graces needed for these newer formats.

Use DirectShowSource("plugh.wav", video=false) to read WAVE_FORMAT_EXTENSIBLE type files.

[IanB]

@Fizick,

What do you expect the access pattern to be from Denoise_Temporal() in il9ad's post?

[Fizick]

The MVDenoise bug is reproduced even with simplest script with two motion vectors clips,
see Didйe post http://forum.doom9.org/showthread.php?p=868502#post868502.

The bad news, that MVdegrain is also not bugs-free when seeking backward.

[Fizick]

test script

avisource("h:\test.avi")
converttoYv12()
assumetff
separatefields().selecteven
i=last
vf=last.invert.mvanalyse(pel=2, blksize=8,isb=false,delta=2)
vb=mvanalyse(pel=2, blksize=8,isb=true,delta=2)
i.mvdenoise(vf,vb,y=true,tht=1000,thmv=1000,thsad=1000)


Try seek frame by frame in VDub, you will see transition at about 50-100 frames.

IanB,
the most bad, that cache miss results in not to slowdown, but to mess of frames!
For MVDenoise, it seems, the SetCacheHints(CACHE_RANGE,3) addition to all VectorsClips corrects the bug.
Have clip arrays get the cache?
And I still disagree with your opinion about plugins writters as complete idiots. :) Why we can not use CACHE_RANGE=0,1,2 ? (It reseted to default.)

What is "access pattern" ? List of frames requested from filter or by filter? I will try to add some debug commands to get the list.
Today I created (and released) bug-fixed MVTools v1.4.12 with debug output for debugview utility.

[IanB]

the most bad, that cache miss results in not to slowdown, but to mess of frames!Well there is something very wrong here. Rule 1 for GetFrame(n) is that for each n it MUST return exactly the same frame n contents.For MVDenoise, it seems, the SetCacheHints(CACHE_RANGE,3) addition to all VectorsClips corrects the bug.No it just hides the bug. By rule 1 above filters must work correctly (maybe slowly) without the cache.

Why we can not use CACHE_RANGE=0,1,2 ? (It reseted to default.)In the context of the call, 0 and 1 make no sense so I ignore the call. And as a courtesy to avoid boundary conditions 2 is promoted to 3. Probably in the long run we should change the value of the key and fully define it's use and values.What is "access pattern" ? List of frames requested from filter or by filter? I will try to add some debug commands to get the list.Yes access pattern is the list of frames requested by the function for a given input request n.

And the question is what do you expect the list of frames to be, not what you measure. I can debug the filter and see what they are, I need to know what you think it is supposed to be so I can compare reality with theory.

[Fizick]

The bug is not only wrong frame number n.
Bugged frame N is frame N+1 taken from another vector clip!

avisource("h:\test.avi").converttoYv12().separatefields().selecteven
other=avisource("h:\other.avi").converttoYv12().separatefields().selecteven
i=last
vf=other.mvanalyse(pel=2, isb=false,delta=1)
vb=mvanalyse(pel=2, isb=true,delta=1)
i.mvdenoise(vf,vb,y=true,tht=1000,thmv=1000,thsad=1000)

I try build expected list.

[Fizick]

src=Avisource("test.avi")
vf=src.MVAnalyse(isb=false)
vb=src.MVAnalyse(isb=true)
MVdenoise(src,vf,vb)


Assumed frames pattern

VirtualDub requested frame 0 from MVdenoise
MVDenoise:
- requested source frame 0 from clip src;
- created new destination frame 0, and copied src frame 0 to it;
- requested frame 0 from motion vector clip vF (it stores both vectors and compensated data);
- requested frame 0 from motion vector clip vB (it stores both vectors and compensated data);
- wait these vectors frames, make some averaging data of src(0), vF(0), vB(0), write it to dest frame 1;
- return dest frame 0.

When MVAnalyse is asked for vF frame 0, it:
- requested src frame 0,
- created new output vector frame 0, with specific video properties (height=0, width=size);
- check if may request frame 0-1=-1 (no); write default data to dest vector clip vF frame 0 with BAD flag;
- return dest vector frame vF(0) (bad)

When MVAnalyse is asked for vB frame 0, it:
- requested src frame 0,
- created new output vector frame 0, with specific video properties (height=0, width=size);
- check if may request src frame 0+1=1 (yes); request src frame 1, analyze motion (1 to 0) and write vectors data and compensated data to dest vector clip vB frame 0 with Good flag;
- return dest vector frame vB(0).



VirtualDub requested frame 1 from MVdenoise
MVDenoise:
- requested source frame 1 from clip src;
- created new destination frame 1, and copied src frame 1 to it;
- requested frame 1 from motion vector clip vF (it stores both vectors and compensated data);
- requested frame 1 from motion vector clip vB (it stores both vectors and compensated data);
- wait these vectors frames, make some averaging data of src(1), vF(1), vB(1), write it to dest frame 1;
- return dest frame 1.

When MVAnalyse is asked for vF frame 1, it:
- requested src frame 1,
- created new output vector frame 1, with specific video properties (height=0, width=size);
- check if may request frame 1-1=0 (yes); request src frame 0, analyze motion (0 to 1) and write vectors data and compensated data to dest vector clip vB frame 1 with Good flag;
- return dest vector frame vf(1)

When MVAnalyse is asked for vB frame 1, it:
- requested src frame 1,
- created new output vector frame 1, with specific video properties (height=0, width=size);
- check if may request src frame 1+1=2 (yes); request src frame 2, analyze motion (2 to 1) and write vectors data and compensated data to dest vector clip vB frame 1 with Good flag;
- return dest vector frame vB(1).

and so on.
So, supposed pattern:

src(0) from MVDenoise
vf(0) from MVDenoise
src(0) from MVAnalyse Forward
vb(0) from MVDenoise
src(0) from MVAnalyse Backward
src(1) from MVAnalyse Backward

src(1) from MVDenoise
vf(1) from MVDenoise
src(1) from MVAnalyse Forward
src(0) from MVAnalyse Forward
vb(1) from MVDenoise
src(1) from MVAnalyse Backward
src(2) from MVAnalyse Backward

src(2) from MVDenoise
vf(2) from MVDenoise
src(2) from MVAnalyse Forward
src(1) from MVAnalyse Forward
vb(2) from MVDenoise
src(2) from MVAnalyse Backward
src(3) from MVAnalyse Backward


may be I am wrong, MVTools has complex internal structure.
(and some hack with callback)

[IanB]

pRef = child->GetFrame(n + offset, env)->GetReadPtr(PLANAR_Y);
pRefU = child->GetFrame(n + offset, env)->GetReadPtr(PLANAR_U);
pRefV = child->GetFrame(n + offset, env)->GetReadPtr(PLANAR_V);
nRefPitch = child->GetFrame(n + offset, env)->GetPitch(PLANAR_Y);
nRefPitchUV = child->GetFrame(n + offset, env)->GetPitch(PLANAR_U);Code like this is going to cause serious grief every time the cache cannot provide the exact same vfb to each and every call.

You gotta a lot of fix'en to do.

[Fizick]

This will be better?
ref = child->GetFrame(n + offset, env)
pRef = ref->GetReadPtr(PLANAR_Y);
pRefU = ref->GetReadPtr(PLANAR_U);
pRefV = ref->GetReadPtr(PLANAR_V);
nRefPitch = ref->GetPitch(PLANAR_Y);
nRefPitchUV = ref->GetPitch(PLANAR_U);

It is mostly Manao's code, and I added some parts too :)
Well, I replaced all these in mvanalyse,
but your's cache algo is bugged anyway :)

[Kostarum Rex Persia]

Feature request for 2.57 final: optimized multi-threaded AviSynth for dual/quad core machines.

[Fizick]

Kostarum Rex Persia,
No, wait version 2.6 or later for this feature.
v2.5.7 is mostly bugfix release

[IanB]

This will be better?
ref = child->GetFrame(n + offset, env);
...That will stand a chance of working. The other is just plain wrong!

PVideoFrame ref is a smart pointer. As long as ref is in scope you have an ownership stake in the associated VFB. When it goes out of scope or is reassigned your use is automatically released and any pointers you might have cannot be trusted anymore.

As I suggested many months ago it is safest to get all of your PVideoFrame's at the top of the code. This is a defensive coding technique that avoids confusion and complicated bugs like we have here.


This code is highly illegalpRef = child->GetFrame(n, env)->GetReadPtr(PLANAR_Y);The contents of the pointer pRef are invalid immediatly. The intermediate PVideoFrame goes out of scope immediatly. The returned pointer is pointing into a VFB that you do not have an ownership stake. The VFB is as such free and can be reused at any time. In the single tasking case the code may limp along and pretend to work, in the multitasking case ......


but your's cache algo is bugged anywayIt may well have bugs but this is not one of them. The MV code is not using the Avisynth API correctly. The cache is only there to try to prevent re-rendering of frame already rendered. It is not there to fix broken code. You are trying to fix a VFB ownership problem by using a side effect of CACHE_RANGE to trick the cache into keeping the VFB in use for you. In future versions this side effect may not exist.

I suggest you use CACHE_NOTHING while you work out all these MV bugs. Also the bugged version of AddGrain() that returned different grain each time you called it might help as useful diagnostic.

[Fizick]

thanks,
i tried use in MVAnalyse
ref = child->GetFrame(n + offset, env);
...
it does not cure the bug.

It seems, that bugged code is at mvClipArray->Update(n)

[IanB]

And many many more places :(
You gotta lot of fix'en to do.
If I can find time I will try to read thru the code.

[Fizick]

IanB, thanks for advise.
I really found many places,
but the main is:


void MVClip::Update(int n, IScriptEnvironment *env)
{
FakeGroupOfPlanes::Update(reinterpret_cast<const int*>(child->GetFrame(n, env)->GetReadPtr()));
}
...
mvClip.Update(n, env)


It is code for updating vector clip's data. I replace it by new code:


void MVClip::Update(PVideoFrame &fn, IScriptEnvironment *env)
{
FakeGroupOfPlanes::Update(reinterpret_cast<const int*>(fn->GetReadPtr()));
}
...
PVideoFrame mvn = mvClip.GetFrame(n, env); // store PVideoFrame
mvClip.Update(mvn, env);


But IMHO it may be considered not as MVTools bug, but as Avisynth bug (not cache, sorry, but smart pointers),
at least until you documented this info in http://www.avisynth.org/FilterSDK :)

So, if you find time, please write some small article to Avisynth SDK about VFB processing and cache. You may use code of any of my numerous bugged plugins as example "how to not to do" :)
But please provide "how to do" fragments too.
I recall http://forum.doom9.org/showthread.php?t=100715

To be on topic:
May be we must add SDK doc to AviSynth offline doc?

[smok3]

about offline (javascript) search engines..., well i think the easiest solution is to provide the docs as single html file as well, so anybody can use ctrl+F.

[tritical]

temporalsoften is missing a:

delete[] planeDisabled;

in its destructor resulting in a very, very small memory leak.

[Wilbert]

@IanB,

We talked about this before, but people still have the grey frames problem in DSS:

http://forum.doom9.org/showthread.php?t=116924

I remember from our conversation that it is not an AviSynth problem, but it is a problem nonetheless. I hope there will be some "fix" before v2.57 final it out.

[IanB]

Yes, I was going to change the default behaviour to "crash and burn" mode instead of doing grey frames mode. (But I am not sure people will be very happy with runtime exceptions being thrown and killing there encodes)

Perhaps somebody has some better ideas on how to resolve the issue. (and displaying the last good frame does not work at the start of the stream)

[Guest]

displaying the last good frame does not work at the start of the stream Then display the first good frame (that's what DGMPGDec does).

What exactly is the problem? Is there a thread with an analysis?

[IanB]

Nice thought, if only I could get a first good frame.

The problems is DirectShow doesn't deliver a frame, it just hangs. I added code to implement Timeouts instead of hanging forever, but I now have the problem of what to do with the failure. Currently the options are a "Grey Frame" or a runtime Exception, neither of which I am really happy with.

Most of the problems appear to be deadlocks in some non-descript spliter. AVS is probably more a victim of this because it builds 2 separate graphs, one for audio only and one for video only rather than a single graph that renders both audio and video and plays them synchronously. (Remember AVS GetFrame() and GetAudio() are independantly asynchronous)

[Ebobtron]

IanB
Although almost pathetically minor, when AviSynth is presented with a thoroughly useless script(no audio, no video, *no errors*), AviSynth spills the following message to video out.

“” The script’s return value was not a video clip “”
script example: (yes, just a single digit)
1

In light of today’s and future audio only capability, should this maybe read

“The script’s return value was not a clip” or whatever might be more accurate.

*no errors* above refers to the following not throwing an error when such a simple script which contains no syntax errors is used.

try
{
res = env->Invoke("Import", AVSValue(&scpt,1));
}
catch(AvisynthError err)

res.IsClip() does return false, so error detection is fine, just the wording to me seems wrong with the current version.

:thanks:

[IanB]

@Ebobtron,

Yes "video" clip is a tad redundant, like "PIN" Number, I think we can accomodate.

[Ebobtron]

It is me again,

While updating the user docs for FilmCutter (http://forum.doom9.org/showthread.php?t=97438) I noticed that the english versions of

getchannel.htm
and
directshowsource.htm

both contain the word "ordening".
I suppose in the place of "ordering" or is "ordening" suppose to be related to "ordinal".

From what I can tell ordening isn't a word, but what do I know.


Thanks again.

[foxyshadis]

Would a dev mind taking a look at this video (STR2311_cut.avi)? Avisynth consistently crashes (occasionally just a system exception) with LZOCodec-encoded videos. It decodes fine in vdub, and works fine if re-encoded to ffv1 or anything else before loading it into avisynth. You'll need LZOCodec (http://usuarios.lycos.es/lzocodec/) first. [Mirror (http://www.free-codecs.com/download/LZOcodec.htm)]

I just remembered that it internally uses YUY2 for everything, and setting pixel_type="YUY2" fixes it. I'll report the crash to the LZOCodec author, it must be crasing while convering to YV12.

[Fizick]

Ebobtron,
I will change it to "ordering"

[Fizick]

I want to ask about this (rathe old) change of v2.5.7:
* Bracketless call of argless function now get a cache. (Tritical)

what is agrless, and why agrless only get a cache?

[Wilbert]

what is agrless
"without arguments". A function call without arguments and without brackets. Remember the old discussion about filter and filter() where the internal cash of the former was disabled resulting (sometimes) in a speed loss?

link: http://forum.doom9.org/showthread.php?t=100166

[Fizick]

I remember this discussion, but I supposed, that now in v2.5.7 any filter call without brackets get a cache. But it is seems, that it is not a true for v2.5.7? So, only very small amount of filters like Weave (argless) will get a cache? And any other internal or external filter (which has some parameters) will not get a cache, when it is called without brackets (with default parameters assumed)?

[IanB]

No!, FIlter() and Filter(a, b, c) alway had and still do get a cache.

Filter previously did not get a cache. Now in 2.5.7 it gets one the same as the other cases (or so Tritical assures me :D )

[pandy]

Bug in ImageSource() function.

0-255 level conversion to 16-235 (16-240 for chroma) is always enabled...

[dbloom]

Back to the "Search" idea for the documentation...

Searching could be implemented by making a simple Javascript spider that uses XMLHTTPReq. Basically, when you do a search, it starts loading pages (the table of contents, then all pages linked to from the table of contents...) and presents a list of results that are the pages in which the keywords appear.

This should run pretty fast because all of the files are stored locally.

[Fizick]

IanB, thanks,
so more correct is:
* Bracketless call of argless function now get a cache too. (Tritical)
:)

dbloom,
I do not see any reason for special 'Avisynth-specific' search.
You say about common method.

[Wilbert]

Bug in ImageSource() function.

0-255 level conversion to 16-235 (16-240 for chroma) is always enabled...
I'm pretty sure this is not the case. Could you post a sample pic?

@Fizick,
IanB, thanks,
so more correct is:
* Bracketless call of argless function now get a cache too. (Tritical)
Yes.

[tritical]

Another minor bug, line 477 in avisynth_c.cpp should be changed from:

p->env->AtExit(shutdown_func_bridge, d);

to

p->env->AtExit(shutdown_func_bridge, dd);

Now in 2.5.7 it gets one the same as the other cases (or so Tritical assures me )
Not sure about assurances :D, but I do remember testing it at least (hopefully).

[foxyshadis]

Bug in ImageSource() function.

0-255 level conversion to 16-235 (16-240 for chroma) is always enabled...

ImageSource only imports in RGB, so I'm not sure how level conversion is even an issue. Unless you mean the JPEG decoder; JPEG is specified to be 0-255 YUV->0-255 RGB, and cannot use the normal "TV" mode. However, I've never seen avisynth incorrectly decode jpeg.

[IanB]

@Tritical, done, Thx!

@Pandy, ImageSource uses the DevIL image library and as foxyshadis points out we only ever decode to RGB. If the library incorrectly decodes an image to RGB[16..235] then please report it to the IL group.

As a work around do the followingImageSource("blah.jpg", pixel_type="RGB32")
Levels(16, 1.0, 235, 0, 255)

@Fizick, too, also, as well ;)

@dbloom, possible, please post some working samples.

[dbloom]

@dbloom, possible, please post some working samples.

working on it :)

[Fizick]

Here is some test result for avisynth v2.5.6a:

avisource("yuy2.avi")
limiter()
fluxsmootht()
I got fps=29


avisource("yuy2.avi")
limiter
fluxsmootht()
I got fps=24.

avisource("yuy2.avi")
degrainmedian()
degrainmedian()
I got fps =18


avisource("yuy2.avi")
degrainmedian
degrainmedian()
I got fps =10

With v2.5.7 result are fine (without slowdown).
But limiter and degrainmedian are not 'argless'.
So, more correct is:

* Bracketless call of function now get a cache

[pandy]

I'm pretty sure this is not the case. Could you post a sample pic?


This is source which i try to encode (ordinary RGB colorbar 0,255)
And this is simple script which i use for this...

ImageSource("0_255.png", end = 1499, fps = 25.0)
PointResize (720,576)
ConvertToYUY2(interlaced=False)
ColorYUV (analyze=true)

[foxyshadis]

You need ConvertToYUY2(matrix="PC.601") or "PC.709" in order to convert to YUV 0-255 levels. The default is "Rec601", that's where the range reduction is coming from.

[sh0dan]

I backported some changes to the 2.6 TCPDeliver (larger TCP buffers), and my own tests show a great improvement over the last version.

http://klauspost.googlepages.com/TCPDeliver_25_14oct2006.zip

You should be able to hit about 90% usage on the server, and 100% on the client, given optimal load distribution, but most likely, you'll just hit one of these in the real world.

Changes are in CVS, but not included in RC 1.

[pandy]

You need ConvertToYUY2(matrix="PC.601") or "PC.709" in order to convert to YUV 0-255 levels. The default is "Rec601", that's where the range reduction is coming from.

??????

Ok, i see... sorry, i forget about this - im not using ConvertToYUY2 in such way... I prefer rather ColorYUV (levels="PC->TV") method.
Sorry for mess!
Thank You foxyshadis!

[Isochroma]

Will the new TCPDeliver.dll work with 2.5.6a?

[sh0dan]

@Isochroma: Shouldn't be a problem. It should work with all 2.5 versions.

[DDogg]

I backported some changes to the 2.6 TCPDeliver (larger TCP buffers), and my own tests show a great improvement over the last version.

http://klauspost.googlepages.com/TCPDeliver_25_14oct2006.zip

You should be able to hit about 90% usage on the server, and 100% on the client, given optimal load distribution, but most likely, you'll just hit one of these in the real world.

Yes, and Yes. 90-100. Great job! I hope we can start a thread, finally, to concentrate the information on TCPDeliver into one easy to find reference. Maybe even some real docs might come out of it :)

[Wilbert]

Regarding AviSource and multichannel AC3/DTS. It doesn't work for me. I installed the AC3/DTS ACM codec: vac3acm (http://sourceforge.net/project/showfiles.php?group_id=66022). (There is also the AC3 ACM codec from fccHandler (http://fcchandler.home.comcast.net/AC3ACM/index.html), but i didn't try that one yet.)
If i open an Avi with AC3 or DTS in VirtualDub (1.7.0) it shows up as having 5 channels (i guess that's a bug of Avery's part, but i will contact him about it ...). It indeed uses vac3acm to decode the audio of the clip. But if i open them in AviSynth (latest v2.57):

AviSource(...)
Info()

info returns two channels for the audio stream.

[niiyan]

I made a Japanese translation team two month ago.
We are trying to translate AviSynth docs to Japanese.
But it will take a lot of time to complete translation.
So, we decided to release the pages we already finished translating.

You can download the Japanese version of AviSynth docs from here:
http://translation.avisynth.info/index_en.html

Now it includes only 9 pages.
But we will continue to tranalate, add new pages and release it once a month.

I have a few questions.

1. Is it possible to add Japanese translation to AviSynth installer after we complete translation?
2. Should we release this temporal Japanese translations under GPL? I mean, should we include gpl.txt?

[Fizick]

niiyan,
thanks for contribution.
Yes, it will take a lot of time. But team work can speed-up (i know it).
Answers:
1. Yes, it is possible of course. Usually all "official" translations are placed to CVS at sourceforge.net by one of translator (coordinator of translation). He (you ?) must become Avisynth project member. Firstly register at sourceforge. Then ask project admin (Wilbert or Ian or Richard) to become project member (as document writer). Then you will be able to commit (and update) your files to CVS reposirary with special software (TortoiseCVS). You may put files one by one right now (tomorrow :)).
We will update installer to include these files when basic part will be translated.

2. The license terms of Avisynth documentation is interesting question. I am sure that it is GPL compatible at least:).

Do you want distribute the translation separately?
May be you can add some readme file and write that
"This translation is a draft version of a part of official Avisynth documentation. Avisynth is Copyright (C)..... under GPL..."
and include GPL.txt, yes.
You may add some translation history, authors, dictionary to this file (see "aboutrussian.htm").

[jmac698]

I found a bug in 2.56, maybe it can be addressed in new version. When resizing downwards, interlacing is not taken into account. If I reduce an interlaced video, there is ghosting in both fields now.
A practical use is restored upconverted material; where each field was resized.
I would ask for bool interlaced option in resizers.

[Mug Funky]

that's not exactly a bug, but a bool "interlaced" could be useful. preferred method for resizing interlaced is motion-adaptive bob and then resize though - no matter how an interlaced resize is done, it's working on half the information it needs, and you'll get slightly crunchy diagonals even if the per-field misalignment is taken care of.

[jmac698]

I know it will still be interlaced; if my intent is to resize without deinterlacing, then resizing fields is the only option.
If something was pointresize upconverted, then I am only recreating the original version. In real life there is a 4 tap hardware filter; but I can undo this fairly well.
Regardless of the most pleasing results, in no case should a possibly interlaced video be simply weaved and resized.

[Fizick]

generally, resizing is for progressive only,
so we can add "interlaced" flag, but it must produce the exception "Avisynth XXXResise: sorry, only progresive source supported!" :)

[IanB]

I would ask for bool interlaced option in resizers.Adding options like this only encourages people to use it. As M.F. points out it is a truely dreadfull way to resize interlaced material. The primative needed to implement this (mis)feature are available in the scripting language already.

[jmac698]

Then cause an error, document the issue in the manual. IMHO, I don't support a built-in filter doing something knowably horrible.

This was a reallife problem; mrestore doesn't work with some bobbers, it's a known issue. Only a simple bob() will work. For speed I had to resize first; that's when I noticed I can't resize interlaced properly. There is no workaround presently.
http://forum.doom9.org/showthread.php?p=903815#post903815

[foxyshadis]

Have you seen the simpleresize plugin? It comes with the very option you ask for, and already does the work to realign the fields properly.

[niiyan]

@Fizick

niiyan,
1. Yes, it is possible of course. Usually all "official" translations are placed to CVS at sourceforge.net by one of translator (coordinator of translation). He (you ?) must become Avisynth project member. Firstly register at sourceforge. Then ask project admin (Wilbert or Ian or Richard) to become project member (as document writer). Then you will be able to commit (and update) your files to CVS reposirary with special software (TortoiseCVS). You may put files one by one right now (tomorrow :)).
We will update installer to include these files when basic part will be translated.

I understand the process to commit translation files. But we've just started translation, so I will ask project admin again after finishing translation of basic part.

2. The license terms of Avisynth documentation is interesting question. I am sure that it is GPL compatible at least:).

Do you want distribute the translation separately?

Yes. I would like to distribute the Japanese translation separately and temporally until it is added to AviSynth installer.

We will discuss about the license later and decide whether we release the Japanese translation under GPL at least by next month.

[IanB]

filter doing something knowably horrible.No filter knows that an image is interlaced (or anything else for that matter), some very clever filters try to measure interlace'ness, all have problems getting it right. You the human are responsible for makeing valid choices in your scripts. I can't protect you from yourself. G.I.G.O. From the other thread your real problem is trashed video messes up your deinterlacing/bobing procedure. Learn to use SeparateFields, Weave and the subpixel cropping functionality of the resizers.

[Wilbert]

We will discuss about the license later and decide whether we release the Japanese translation under GPL at least by next month.
As i understand it GPL is only possible for code, and not for documentation. IF we want to license the documentation, we can use 'GNU Free Documentation License' or whatever. Of course we need the required permission from everyone involved when we should to do this.

[Fizick]

Wilbert,
GNU GPL is possible for documentation.
IMO the only expicitly declared avisynth license (by Ben and later) is GNU GPL or at least some compatible (like public domain) for some parts.
The switching to GNU FGL is hard for reason you say (and it is complex).
I want discuss with you a little later :)

[Wilbert]

Wilbert,
GNU GPL is possible for documentation.
Please explain.

[Fizick]

http://www.gnu.org/licenses/gpl-faq.html#TOCWhyNotGPLForManuals
It is possible to use the GPL for a manual, but the GNU Free Documentation License (GFDL) is much better for manuals.

other opinion:
http://home.twcny.rr.com/nerode/neroden/fdl.html

[Mug Funky]

this might be a cache thing, but why is this:
doubleweave()
assumetff().separatefields().selectevery(4,0,3)
selectevery(10,0,1,4,3,6,5,8,9)
weave()

about twice as fast as this?
assumetff()
separatefields()
selectevery(10,0,1,4,3,6,5,8,9)
weave()

while giving identical results?

i imagine it's because of some peculiar seeking that's going on (especially in mpeg-2), but it seems kinda strange.

...and (tritical's recompile of) smartdecimate is faster than both.

basically my method of detecting film patterns can be used in IVTC, but i'm not exploiting it yet. the snippets above are part of it :)

[IanB]

Yes the 1st input access pattern is linear, while the 2nd input access pattern is 0 0 2 1 3 2 4 4 which I imagine would give mpeg2source a mildly hard time.

Try putting achangefps(last, last, true)at the top of the script, it will enforce linear pre-access of getframe calls.

The DoubleWeave case should be very marginally slower due to the blit every 2nd access. In both cases the Weave does a blit every frame. I suspect SmartDecimate only blits the frames with crossed fields i.e 2 out of 4. optimally it might only need to blit 1 field if very cunning.

As I keep saying blits are very very slow when compared to pointer shuffling. :D

[Mug Funky]

cool. good to know. i just sort of assumed the field-shuffle type filters were very low cost.