AviSynth 2.5.8 [Dec 30th] - Release! (was RC-5 [Dec 27th])

[Leak]

It is an OS problem.
It's not an OS problem, as memory fragmentation only happens inside a program's address space - an x86 OS only hands out memory in pages (usually 4 kB in size) which don't have any fragmentation issues.

It's up to each application and/or library to support the memory management it wants/needs - but usually it's just malloc()/free() and that's that.

Java or C#/.NET CLR on any platform will garbage collect/compact memory without a hitch, but for C/C++ your program and all libraries it uses need support for doing that.

Usually it's just garbage collection and no memory compaction since that needs knowledge where the pointers in your objects are *AND* a mechanism to make sure nothing accesses the objects you're compacting while the compaction is being done...

But unless your OS is 100% managed (like MS's Singularity (http://en.wikipedia.org/wiki/Singularity_(operating_system))) it's up to the applications/runtime libraries to support all this, not the OS itself.

[josey_wells]

I recently pulled down V2.5.8 RC3 and tried to compile. I successfully got it to compile but there were >600 compiler warnings.

One could argue whether the secure versions of the library functions are really needed or whether some of the typecast are needed - IMO some of them are. But there were also a significant number of variable scoping problems.

Below is source code and diffs to sourceforge version.

http://rapidshare.com/files/140234230/AviSynth_V2.5.8.rar.html

http://rapidshare.com/files/140234231/AviSynth_Compare.rar.html

Also, there does not seem to be a debug version of the devil.lib included.

Heopfully some of these changes will improve this already great product.

[Gavino]

Java or C#/.NET CLR on any platform will garbage collect/compact memory without a hitch...
Garbage collection does have its downsides - mysterious bugs.

Long ago, I worked on an Algol68 compiler whose garbage collector started collecting itself. Talk about disappearing up your own backside... :eek:

[@IanB: Sorry for going off-topic]

[IanB]

The other part about running out of address space at 1.3GB usage is that Windows is a Virtual Memory environment. You really should be checking the amount of VM allocated to the process as well. And do not forget, in that 2Gb of available address space also lives the code, data and stack segments. 1.3GB is an incredible amount of memory. To get some context it is room for 170 high definition rgb32 frames or 2700 standard definition YV12 frames. The most temporally demanding filters may need upto 15 frames from the cache. Unless a given frame is requested more than once in a script the cache can offer no performance increase.

And talking about garbage collection and compaction environments here is off topic, this thread is about problems with version 2.5.8 Release Candidates, currently number 3. Please open new threads to discuss all these other interesting but off topic subjects.

Also release candidate means we are in Mortein/Baygon* the last few bugs just before we push the product out the door mode. We do not make radical changes like switch compilers or start using the "safe" libraries.


* - Bug spray products.

[josey_wells]

Also release candidate means we are in Mortein/Baygon* the last few bugs just before we push the product out the door mode. We do not make radical changes like switch compilers or start using the "safe" libraries.


* - Bug spray products.

No problem, these can be queued for V2.5.9.

[IanB]

Changelist with respect to 2.5.8 RC3: See 1st post for full change list.

Additions:

* Updated Japanese documentation [20080813] (Niiyan).

Bugfixes:

* Fixed DirectShowSource() grey frames after end of stream + 30 frames.
* Fixed Histogram() Classic mode restore graph brightness. Regression.
* Fixed Compare() graph pixel values exceeding YUV limits.
* Fixed AddBorders() args negative value clamping.

Changes:

* Initial Memory Max value clamped to 512MB.
* Default Memory Max value restored to quarter of Free memory. Minimum 16Mb. As per 2.5.7.

As usual download from Sourceforge (http://sourceforge.net/project/showfiles.php?group_id=57023&package_id=105994&release_id=623171).
Enjoy! :cool:


If we do not find any show stopper problems this could become the official 2.5.8 release. ;)

[Fizick]

IanB, I try translate changes.
What is difference: "Initial Memory Max value" (< 512 MB) versus "Default Memory Max value" (quarter of free, >16 MB)?
It is same parameter?

EDIT: So, what is (memory) changes "with respect to v2.57" ?

[IanB]

I try translate changes.
What is difference: "Initial Memory Max value" (< 512 MB) versus "Default Memory Max value" (quarter of free, >16 MB)?
It is same parameter?Yes, they are the same thing.EDIT: So, what is (memory) changes "with respect to v2.57"?The "quarter of free, Minimum 16MB" is the same as 2.5.7 again.

The Maximum of 512MB is new, i.e. on a Vista64 with 4GB free 2.5.7 would start with a 1GB MemoryMax and could possibly crash out the 2GB address space limit, now 2.5.8rc4 limits to 512MB.

Earlier 2.5.8's used 50% of the free and crashed out the 2GB address space limit even easier.

All this just effects the Default or Initial MemoryMax value.

:devil: Users can still use SetMemoryMax() to set higher or lower limits to get themselves into trouble. :devil:

[Fizick]

IanB,
1. Thanks.
2. May be you (we) can somehow summarize some of the 2.5.7 to 2.5.8 changes for releasenotes.htm?
To provide the most important changes here.
Now it simply duplicates the changelist.htm
Similar to "DirectShowsource improvements,..."

Thoughts ? :)

3. Your sourceforge link is https instead of http :)

[LoRd_MuldeR]

2. May be you (we) can somehow summarize some of the 2.5.7 to 2.5.8 changes for releasenotes.htm?

http://forum.doom9.org/showpost.php?p=1142622&postcount=1

[Wilbert]

That's not a summary :)

[IanB]

3. Your sourceforge link is https instead of httpFixed! Damn those sourceforge https junkies.

Yes the release notes are a bit tatty but ...

[Fizick]

Ian, not related to summary, but please do not skip: I make some small changes (mostly typos of russian doc) at CVS

[IanB]

@Fizick,

So I take this as a request to make a 2.5.8 RC4a. I'll find some time latter this week.


Anybody else got some last minute doco/language type updates? Squeak now!

[Fizick]

I do not asked for it :)
But I ask somebody look to uptated description of ConditionalFilter (I do not use it much) at CVS http://avisynth2.cvs.sourceforge.net/avisynth2/docs/english/corefilters/conditionalfilter.htm?r1=1.6&r2=1.7 and wiki
http://avisynth.org/mediawiki/ConditionalFilter

[Wilbert]

What's the difference between a filter and a function in this case?

[Fizick]

IMO, AverageLuma(clip) and similar expressions with numeric result are not filters, but functions (in ConditionalFilter).
Filter must return filtered clip (in ScriptClip).

Previous text used "filter" and "function" in not consistant manner.
There were also some other non-clear fragments like this for ConditionalFilter:


The string filter can be any internal filter, but also some filters which are predefined (the Runtime Functions).


What internal filter may be alone used in ConditionalFilter?
Why internal and predefined only? What about plugin or user script functions?

The right changes are not obvious for me, it is hard topic for me, and proper English too, that is why I ask for discusssion. :)

[Gavino]

I ask somebody look to uptated description of ConditionalFilter (I do not use it much) at CVS http://avisynth2.cvs.sourceforge.net/avisynth2/docs/english/corefilters/conditionalfilter.htm?r1=1.6&r2=1.7 and wiki
http://avisynth.org/mediawiki/ConditionalFilter
I have taken the liberty to update the wiki. I hope this is OK.

This was before I saw Fizick's conversation with Wilbert, but I think that just confirms what the issue was, and I agree with Fizick.

[Fizick]

Gavino,
O.K.
I also added current_frame to list (see wiki).
Waiting for comments from Wilbert (and others), then shall update CVS.

[Gavino]

I have further updated the same wiki page to state explicitly that line breaks may be used in the ScriptClip string.

[Wilbert]

The changes are fine with me.

[Fizick]

Adding show="true" will display the actual values on the screen.)

What is usefulness of "show" parameter in ScriptClip?

[Gavino]

What is usefulness of "show" parameter in ScriptClip?
I imagine it's for debugging. Particularly if the filter string is constructed dynamically (eg by string concatenation which is a common approach inside functions).

It's curious that 'show' in ScriptClip (and FrameEvaluate) disables the actual filtering (it just returns the input clip) whereas in ConditionalFilter it acts in addition to the normal function.

[Fizick]

OK, I uploaded the updated text to CVS.
I hope that new text is fine not only with Wilbert, but with Avisynth code too :)

[IanB]

Changelist with respect to 2.5.8 RC4 [080831] :

Changes:

* Updated Russian documentation (Fizick).

Note : Other contents are the same as AviSynth_080831.exe


Changelist with respect to 2.5.8 RC3: See 1st post for full change list.

Additions:

* Updated Japanese documentation [20080813] (Niiyan).

Bugfixes:

* Fixed DirectShowSource() grey frames after end of stream + 30 frames.
* Fixed Histogram() Classic mode restore graph brightness. Regression.
* Fixed Compare() graph pixel values exceeding YUV limits.
* Fixed AddBorders() args negative value clamping.

Changes:

* Initial Memory Max value clamped to 512MB.
* Default Memory Max value restored to quarter of Free memory. Minimum 16Mb. As per 2.5.7.

As usual download from Sourceforge (http://sourceforge.net/project/showfiles.php?group_id=57023&package_id=105994&release_id=625750).
Enjoy! :cool:


If we do not find any show stopper problems this could become the official 2.5.8 release. ;)

[Quark.Fusion]

TCPsource/TCPserver sometime lost connection and broke my encode, also AviSynth crash a lot when I run script with TCPsource second time. (not tested with today's build)
Why TCPsource not tries to reconnect?

[IanB]

@Quark.Fusion,

TCPsource/TCPserver are unchanged from 2.5.7

[Fizick]

IanB,
sorry, but I made litle changes of doc again :)
(more compact look of overview.htm, english and russian)

[Fizick]

and again :)

[tacman1123]

Not sure if this is the right place to report possible bugs for RC4, if not, please let me know where is a better place.

I installed RC4, and the sound came out slower, a much lower frequency. I replaced DirectShowSource.dll with an older one


12/24/2006 05:12 AM 24,576 DirectShowSource.dll


and it started working again. Script was fairly trivial, but I'd be happy to post it (and my avi) if someone would like it. My AVI files have been somewhat problematic, especially in regard to sound (I have another thread here detailing that issue), so it's certainly possible that it's a bad avi, but since it is working with an older DirectShowSource.dll, I thought I'd mention it.

Tac

[Fizick]

tacman1123, of course, please provide link to your previous thread (post), and may be some Avi fragment.

[IanB]

@tacman1123,

Post the log from DirectShowSource(... , logfile="tac.log", logmask=-1) for both d..s..s...dll version!

[tacman1123]

cool, I'll do that! (The avi file is very big). Of course, now I'm having a hard time replicating the problem. Question: if I rename the dll in my plugins directory, is it definitely using the new version when the script runs? Or might it be caching a version? Is there a way I can force my script to use one over the other? I tried simply renaming the dll and calling it with that name, but that didn't work.

[IanB]

Use the {dllname}_{functionname} form of access the functions. i.e. DllA_DirectShowSource() and DllB_DirectShowSource()

And the logfile has a version stamp as the 1st line.

[Fizick]

small doc update again :)
(SetMemoryMax according to http://forum.doom9.org/showthread.php?p=1177819#post1177819 )

[Wilbert]

Someone reported:
My AviSynth is version 2.58, build date Aug 30, 2008. I have tried "ntsc_video" several times & I always get the same error message.
... being "ntsc_video" not recognized.

[Gavino]

Someone reported ... "ntsc_video" not recognized.
No, the error message he reported was "Video framerate doesn’t match" when importing into DVD slideshow GUI, which makes me suspect the fault lies with that program.

Further evidence is that he also got the same error for
AssumeFPS(30000,1001,true).

[DeathTheSheep]

Ever since 2.5.8 alpha 2, directshowsource with convertfps is terrible. The resultant video is very choppy. All I'm doing is converting an mkv's framerate from 24000/1001 to 23.976. I've tried every release up until RC4a, and all have the same glaring bug. Going back to alpha 1 [070518] works perfectly. I've reported this bug before and was told simply that directshowsource was hopelessly b0rked. Well, over a year's worth of releases, and getting to a late 'release candidate' stage... the bug's still here. ;)

Great work nonetheless!

[IanB]

@DeathTheSheep,

Seems you have made 3 posts alluding to the subject, 12th November 2007, 03:38, 28th March 2008, 07:50 and 28th March 2008, 09:15. None really form a concise bug report with any evidence, logs or supporting material.

CVS history of directshow_source.cpp----------------------------
revision 1.28
date: 2008/06/12 04:17:26; author: ianb1957; state: Exp; lines: +46 -7
Remember Allocator given in NotifyAllocator(), return it for GetAllocator();
GetAllocatorRequirements() E_NOTIMPL.
----------------------------
revision 1.27
date: 2008/05/26 22:22:17; author: ianb1957; state: Exp; lines: +106 -47
Add TID + PrintState to log, nuke PulseEvent + Use IGraphBuilder (Avery Lee),
Fix QueryId() need CoTaskMemAlloc (Dean Pavlekovic),
Fix QueryAccept change VideoInfo (Haali).
----------------------------
revision 1.26
date: 2007/10/08 23:09:39; author: ianb1957; state: Exp; lines: +14 -6
get convertfps=false logic right, fix Flush task interlock.
----------------------------
revision 1.25
date: 2007/09/11 06:43:04; author: ianb1957; state: Exp; lines: +122 -18
Refactor GetFrame, ConvertFPS now only relies on get_sample.GetSampleStartTime,
fix jitter problems with recent M$ asf spliter giving bogus SampleEndTime values.
----------------------------
revision 1.24
date: 2007/05/01 05:49:58; author: ianb1957; state: Exp; lines: +6 -1
support WAVE_FORMAT_EXTENSIBLE
----------------------------I guess you are complaining vaguely about the changes at version 1.25/26 2007 Sep 11 & Oct 08. I completely rewrote the GetFrame code to only use the DirectShow sample start times and not use the stop times at all. The previous code used the DirectShow sample stop time and failed miserably when it was not correctly provided. A few recent splitters seem to be not setting this value correctly.

The algorithm I implemented is to calculate the start time of the requested frame number. Compare this value with the DirectShow sample start time of the next sample, for as long as the value is strictly less than, keep returning copies of the current frame. I have extensively tested the code and it does exactly as intended.

I have to assume the algorithm is not what you require, but you have not offered any suggestion for an alternate algorithm.

One issue I am aware of that may be affecting your results is timecode rounding that a few splitters seem to do. For a CFR 23.976 fps stream, we would expect timecodes to be :-

--- 0, 4170837, 8341675, 12512512, 16683350, 20854187, 25025025, ..., 95929262, 100100100, 104270937, ...

but a splitter might give something like this :-

--- 0, 4200000, 8300000, 12500000, 16700000, 20900000, 25000000, ..., 95900000, 100100000, 104300000, ...

This results in returned samples being 0, 0, 2, 3, 3, 4, 6, ..., 23, 24, 24, ... not exactly what one might find useful.

You can limp around this particular issue by doubling the request framerate and selecting the odd samples, i.e.DirectShowSource(... , fps=23.967*2, convertfps=true).SelectOdd()But the bottom line is you need to understand your material and the Directshow software you are using and script accordingly.

[Fizick]

Directshow si never will be finished complitely :)
In any case, Directshowsource is a plugin and may be updated later (if needed), as an small independent upgrade to Avisynth 2.5.8.
Also for same reason DeathTheSheep can use older (alpha 1) DirectshowSource if he do not want provide a log ;).

[DeathTheSheep]

It's embarrassing to admit this, but I have no idea what/where the avisynth log is. I will happily provide it if I came upon this information. ;)

Cheers!

PS: I'm using Haali's splitter on what appears to be a fixed framerate mkv.
PPS: Though I have no suggestion for an "alternate" algorithm, I'd suggest preliminarily that both behaviors be retained, with the option to choose "convertfps=stoptimes" to use the older behavior.

[Fizick]

DirectShowSource(avifile,.., logfile="mylog.log")
http://avisynth.org/mediawiki/DirectShowSource
;)

[IanB]

PS: I'm using Haali's splitter on what appears to be a fixed framerate mkv.The solution I proffered above is specifically for mkv container conversion.... this particular issue by doubling the request framerate and selecting the odd samples, i.e.DirectShowSource(... , fps=23.967*2, convertfps=true).SelectOdd()If you look carefully the old version DSS, using stop time, does not get the VFR conversion right. With mkv containers the problem is less pronounced as the predicted stop times are the ceiling value, but this causes a problem whenever frame duplication is required, the next frame is repeated instead of the current one.

As I said :-But the bottom line is you need to understand your material and the Directshow software you are using and script accordingly.

[Wilbert]

One issue I am aware of that may be affecting your results is timecode rounding that a few splitters seem to do. For a CFR 23.976 fps stream, we would expect timecodes to be :-

--- 0, 4170837, 8341675, 12512512, 16683350, 20854187, 25025025, ..., 95929262, 100100100, 104270937, ...

but a splitter might give something like this :-

--- 0, 4200000, 8300000, 12500000, 16700000, 20900000, 25000000, ..., 95900000, 100100000, 104300000, ...

This results in returned samples being 0, 0, 2, 3, 3, 4, 6, ..., 23, 24, 24, ... not exactly what one might find useful.
Is there any easy way to find out when this is happening (without creating the log file)?

[Wilbert]

@IanB,

I will receive the Czech translation very soon, and I will add it in the coming days.

[Archimedes]

Standalone installation doesn’t work for me. A simple AviSource("video.avi") in the program directory of AviSynth doesn’t work.

http://img150.imageshack.us/img150/9559/avisyntherrorml4.jpg (http://imageshack.us)

[Fizick]

Archimedes,
1. what about version() script?
2. previous v2.5.7 works fine?

[Archimedes]

Same thing with Version(). Normal Installation works fine.

[IanB]

Install the main files in the Product directory (Nonfunctional non-Admin install) ...The Standalone installation is not intended to be functional out of the box. It just unpacks all the distribution files into your product directory. You are then responsible for manually updating the AviFile registry entries.

It is for cases where you cannot run the normal installer with admin privileges, but you have a backdoor to get the AviFile registry entries updated or you have an application that can use avisynth.dll directly without needing the AviFile registry keys.

[Archimedes]

Thank you. So the registry entries are always needed.