AviSynth 2.5.6 RC 2 Release Candidate [October 7th]

[IanB]

Dear all,

Here is the second Release Candidate of my review and refactor work on Avisynth 2.5.6.

A link down to the lastest Release Post (225) (http://forum.doom9.org/showthread.php?p=720959#post720959)

==========================
See below for progressive updates
===========================

Whats being done.

Evil EBX not saved compiler bug CRASH!
GreyScale() YUY2 fixed - others being hunted!

Convert.cpp
Review and refactor is still in progress.
MMX ConvertToYUY2(Matrix="...") support"
Added PC Levels RGB<->YUV [0..255] conversion options (Matrix="PC.601") and (Matrix="PC.709")
Added Rec.709 RGB->YUV conversion options (Matrix="Rec709").
RGB Greyscale(matrix="Average") support for unweighted [ Y=(R+G+B)/3 ] luma.
------------------------------------
Many 2^n MMX width restrictions gone
Fast MMX RGB24 -> RGB32 convertion
Legacy C++ RGB32 -> RGB24 reinstated
RGB Greyscale(matrix="rec709") support for Rec.709 luma weightings.
MMX YUY2 Greyscale()

Unrecognised Exception - Problem
I've given up trying to classify the code that causes this problem and am just sledge hammering the instances I find with a null try/catch. Here is a list of the affected modules found so far :-
- audio.cpp
- avs_soundtouch.cpp
- color.cpp
- field.cpp
- focus.cpp
- fps.cpp
- levels.cpp
- resample.cpp
- source.cpp

Fps.cpp
Added copy FPS from another clip to ConvertFPS and ChangeFPS as per AssumeFPS.

Avi_source.cpp
Fix WavSource() leaving .WAV files open.

Edit.cpp
Correct multi-channel audio fading

Focus.cpp
Full review and refactor of Blur/Sharpen code
Fixed YV12 width<=16 fatal crash
YV12 H MMX code no longer double blurs every 8th pixel
Edge boundaries now processed
Now MMX, not just SSE restricted
Some MMX versions faster
Most 2^n MMX width restrictions gone
When H or V blur==0 now skips that pass

Layer.cpp
Fixed BGR - GBR typo

Please report bugs in these reviewed modules in this thread.

Please post your script!

Regards
IanB

[Mug Funky]

well, no crashes yet, but i don't think i'm testing the right bits :)

i never got unrecognised exceptions, really. i get plenty of access violations, but i've not been able to track these down. i have a suspicion it's both mvtools and a really whacky interpretation of mpeg-2 spec as a source. but i can't be more specific because there's always other plugs involved, and the crashes only happen after a large number of frames, usually at a slow fps with lots of filtering.

[Wilbert]

Great!

I updated the installer (and added the italian documentation):

http://www.arachnotron.nl/permanent/doom9/avs/AviSynth_241104.zip
http://www.geocities.com/wilbertdijkhof/AviSynth_241104.zip
http://bbsrv.imp.leidenuniv.nl/~arachno/pub/avs/AviSynth_241104.zip

edit: geocities is complaining again. Perhaps someone can download it tomorrow and host it for a while?

Some remarks:

"RGB Greyscale(matrix="Average") support for unweighted (equal value) luma." means luma = (r+g+b)/3 I guess ...

Adding PC Levels RGB<->YUV [0..255] conversion options (Matrix="PCLevels")
This means that the RGB [0,255] <-> YUV [0,255] is only possible with Rec.601 coefficients right?

Perhaps you can add it as a new option instead (called pc_range = true / false(default) to keep it consistent with overlay for example). This way you can also implement the conversion with Rec.709 coefficients.

[IanB]

Originally posted by Wilbert

"RGB Greyscale(matrix="Average") support for unweighted (equal value) luma." means luma = (r+g+b)/3 I guess ...

Correct. r=g=b = (r+g+b)/3

This means that the RGB [0,255] <-> YUV [0,255] is only possible with Rec.601 coefficients right?

At the moment. Patients! Patients! The MMX code might be able to do both 601 and 709 with both TV and PC levels, I am still writing it.

I'll do a pass on the doco when I get the current batch of code finished (unless you beat me to it).

If you have a minute can you please spot check some YUV values for the darker saturated colours I think I might be out by 1 at the black end.

IanB

[sh0dan]

Just PM me when you need me to update the sourceforge alpha.

[Mug Funky]

try this to get wilbert's installer:

http://mugfunky.no-ip.info/AviSynth_241104.zip

you must allow port 8080... some workplace firewalls reject it, so be warned.

i'm an apache n00b :(

oh, btw: this is being hosted off my machine, which means when it gets turned off, the file is gone. i'll leave it on for tonight, and hopefully most of tomorrow, but if the power fails while i'm asleep then there's not much i can do (the grid is pretty unreliable here, too. should be fine for tonight though).

[Wilbert]

Thx! I also added some additional links above.

[IanB]

Also available from http://avisynth2.sourceforge.net/24Nov2004/AviSynth_241104.exe

IanB

[JuanC]

The AudioGraph filter (http://forum.doom9.org/showthread.php?s=&threadid=59412&highlight=audiograph) is crashing (vanishing) virtualdubmod ever since the first AVIsynth 2.56 alpha release.

Could there be anything wrong in the filter? warpenterprises has the filter and source code here (http://www.avisynth.org/warpenterprises/files/audgraph_25_dll_20030813.zip).

[IanB]

Originally posted by JuanC
The AudioGraph filter is crashing (vanishing) virtualdubmod ever since the first AVIsynth 2.56 alpha release.And why didn't you call sooner?

This has exposed a really nasty compiler bug, I begin to appreciate Avery's profanity re Microsloth's compiler bugs. (And he tells me .Net is even worse)

1. In this case the compiler assumed the "ebx" register was preserved across an __stdcall (it's not supposed to) and
2. registers used inside __asm blocks are supposed to be preserved in the entry/exit prologue (well guess what they don't all always get saved). [Starts with F, ends with k, 4 letters]!!! You get the idea ...

I've fixed and tested YUY2 GreyScale() which AudioGraph needs. I'll put an interim checkpoint .dll up tonight for you to test, watch for the EDIT of the 1st post in this thread.

Before I ask Sh0dan to put up a new full package, I'll do a pass through the code over the next few nights to find any more of this evilness.


Could there be anything wrong in the filter? ... http://www.avisynth.org/warpenterprises/files/audgraph_25_dll_20030813.zipYes, the version pointed to above is the "_" to many version. But that is a different problem. Sh0dan has a fixed version.

IanB

[JuanC]

Originally posted by IanB
And why didn't you call sooner? :) It just took me some time to empty my autoload plugins directory and try the filters one by one ... There could be some other problematic filters ...

Originally posted by IanB
I've fixed and tested YUY2 GreyScale() which AudioGraph needs. I'll put an interim checkpoint .dll up tonight for you to test, watch for the EDIT of the 1st post in this thread. Thanks!!! I will be happy to test.

Originally posted by IanB
... ( about AudioGraph ) ... Yes, the version pointed to above is the "_" to many version. But that is a different problem. Sh0dan has a fixed version. I got the one from ShOdan's former website. I didn't notice warpenterprises had the old one. ShOdan is no longer hosting it.

[lark]

I got the one from ShOdan's former website. I didn't notice warpenterprises had the old one. ShOdan is no longer hosting it.

i was looking for it yesterday, but couldn't find (the link in the old post didn't work any more.
could you please direct me to the correct spot?

regards
t :)

[Wilbert]

I'll do a pass on the doco when I get the current batch of code finished (unless you beat me to it).
I updated it last week, but didn't commit it yet.

If you have a minute can you please spot check some YUV values for the darker saturated colours I think I might be out by 1 at the black end.
I will do that if my internet connection works again :)

[sh0dan]

The modified AudioGraph can be downloaded from here (http://beta.zenaria.com/kpo/avisynth/index.html)

[lark]

Originally posted by sh0dan
The modified AudioGraph can be downloaded from here (http://beta.zenaria.com/kpo/avisynth/index.html)

thanks a 1000000
t :)

[Richard Berg]

Originally posted by sh0dan
Just PM me when you need me to update the sourceforge alpha.
It looks like Ian is doing great things for the project, and I don't think I can return anytime soon, so feel free to add him as an admin if you wish. Of course, if anyone needs things done to the website/server, you know where to find me :)

[Wilbert]

I've put up a new release

http://bbsrv.imp.leidenuniv.nl/~wilbert/AviSynth_191204.exe

changelog:

additions:
* Added an "[sse]b" option in Tweak to re-enable the SSE code if required (maybe an AMD might run it faster).
* Added Limiter(show=show_luma/show_luma_grey/show_chroma/show_chroma_grey) shows out of bound luma/chroma; *_grey makes the rest of the pixels grey.
* Added offsets and analyze option (ala ColorYUV) in RGBAdjust.
* Added ColorBars(pixel_type="YUY2").

bug files:
* Fixed Normalize (it was only sampling half of the samples per mouthful; it was ignoring the scaling factor).

optimizations:
* Tweak: added C code, which is now faster than the old iSSE code.
* Reworked some code in some audio filters which uses 64 bit operations (many of the routines are between 2 and 5 times faster).

changes:
* Normalize uses 256K transfer buffer to analyze peaks (should be > 5 times faster now); and displays the frame number of the peak.

I hope I didn't forget anything. Btw, the included docs in the installation are updated.

[sh0dan]

Januar 6th Beta 1 Release:

Great work by Ian:

- ConvertToFloat() fast SSE & SSE2 + Opt output copy.
- ConvertFromFloat() fast SSE2.
- ResampleAudio() support SFLOAT.
- Large amount of stability fixes.
- ShowFiveVersions() YV12 + Grey fill 2 corner rectangles.
- Fast Tweak, RGB Bias+Analyse, Limiter show.
- ChangeFPS/ConvertFPS(clip, ...) as per AssumeFPS
- Correct multi-channel audio fading.
- Optimizations in Merge.


Users with P3 should test the following script:
ColorBars()
ConvertAudioto16Bit()
ConvertAudioto32Bit()
ConvertAudioto16Bit()
histogram("stereo")


As usual download from Sourceforge (https://sourceforge.net/projects/avisynth2/)

[IanB]

@sh0dan,

Ta!

Dear All,

And also in this release...
- Reworked Cache and memory management
Please break out your most evil cache busting scripts and test, test and test more.

Also First Bug! (See I do test my own code) :D

Converting from Float, ConvertAudioTo32Bit() SSE and SSE2, saturate incorrectly in the positive direction. For normalized values >= 1.0f it returns MIN_INT (0x80000000) instead of MAX_INT (0x7FFFFFFF). (Sounds really bad)

Can someone with a 3DNow capable processor please test the following script.ColorBars()
Amplify(1.1) # or more
ConvertAudioTo32Bit()
ConvertAudioTo16Bit() # MaybeThe problem is related to the limited precision available with 32bit floats. The scaled values actually available at the top end are 2147483520 (MAX_INT-127) and 2147483648 (MAX_INT+1). The easy solution is to clamp positive saturation at 2147483520 (0x7FFFFF80).

Anybody Like to suggest some neat SSE(2) code to do the exact positive saturation.

IanB

[Manao]

> - Reworked Cache and memory management

Very interesting, I'll see if the ugly behavior with FuPP's scripts and the old masktools version is still preset or not. Anyway, thanks a lot !

[Arachnotron]

Can someone with a 3DNow capable processor please test the following script
Sounds ok on my Athlon XP 1800+/Asus A7N8X

[Manao]

IanB : Impressive ! the changes you made in the cache seems to work : the script i PMed you some times ago is now functionning as expected :) The speed gain on this particular script is awesome ( from 6 fps to 14 fps )

[tritical]

IanB, I have a question about something in cache.cpp... this might be dumb so bear with me, but I'm wondering about what is happening. This code inside Cache::GetFrame, for the h_policy == CACHE_NOTHING handling:
if (h_policy == CACHE_NOTHING) { // don't want a cache. Typically filters that only ever seek forward.
__asm mov ebx,ebx // Hack! prevent compiler from trusting ebx contents across call
PVideoFrame result = childGetFrame(n, env);
if (result->vfb) env->ManageCache(MC_ReturnVideoFrameBuffer, result->vfb);
return result;
}
Won't the call to env->ManageCache() to return the vfb cause the caches of filters that use CACHE_ALL around a filter that uses the CACHE_NOTHING policy to have frames dropped from their lists that shouldn't necessarily be dropped? I've been testing this script:

mpeg2source()
tfm()
tfmpp()
tdecimate()

while allowing tfm/tfmpp/tdecimate to default to CACHE_ALL, and that one works perfectly. However if I add in a filter that uses CACHE_NOTHING like this:

mpeg2source()
tfm()
tfmpp()
avstimer() <= uses CACHE_NOTHING
tdecimate()

now the caches before tdecimate and tfmpp only store 2 frames in their lists which results in an awful lot of the getframe calls from tdecimate needing frames to be redone from tfm on up.

If I comment out that line from above, or set tfm/tfmpp/tdecimate to use a CACHE_RANGE policy then everything works fine.

[Mug Funky]

Users with P3 should test the following script:

it loads (no vanishing)

i get a 1k tone going between left and centre of the stereo field every second or so. that what you're expecting?

there's no (audible) distortions in the tone, so i'm going to assume the conversions all maintain their integrity.

[edit]

Can someone with a 3DNow capable processor please test the following script

my p3 makes a sound like a buzz. opening the avs in audition (cool! i didn't realise you could do that without an avi wrapper) reveals the positive clipping at negative fullscale, as you said.

[Arachnotron]

my p3 makes a sound like a buzz. opening the avs in audition (cool! i didn't realise you could do that without an avi wrapper) reveals the positive clipping at negative fullscale, as you said.Ok, I misunderstood the purpose :o

On my 3Dnow! capable athlon, the script results in clipping, just as I suppose it should. I loaded the script in vdub and saved a short .wav, which I loaded in audacity. Here (http://www.arachnotron.nl/temp/3dnow_amplify.jpg) is a screendump from a zoomed-in sinus.

[edit]sinus.. er... sine :o

[Mug Funky]

ah! yours is doing the right thing then. mine's flipping the positive clips to the negative side. as you can imagine it sounds pretty bad :)

[Leak]

Originally posted by IanB
- Reworked Cache and memory management
Please break out your most evil cache busting scripts and test, test and test more.

Since you seem to be quite informed when it comes to the cache now, could you (or whoever is well-versed on the topic) perhaps add a bit of documentation for SetCacheHints on AviSynth.org, or how caching works in general?

I guess I'm not the only one here that doesn't really understand it fully, and I guess a reduction of unneeded caching of frames that don't need to be cached can only be beneficial, performance-wise. :)

np: Panasonic - Rutina (Kulma)

[IanB]

@Arachnotron, @Mug Funky,

Thanks guys for the tests (and the screen grab). It's a relief knowing I didn't stuff up the 3DNow code. Not being able to test code paths is a real downer.

@Manao,

Nice to know it make some difference. Your original case was caused by exceeeeedingly long cache chains all pointing at a small number of valid, inuse, VFB's.

@Leak,

Yeah I probably will knock up some info when I get all the bits and pieces integrated. I'm also trying to force myself to write lots of comments in the source.

Originally posted by tritical
IanB, I have a question about something in cache.cpp. I'm wondering about what is happening. This code inside Cache::GetFrame, for the h_policy == CACHE_NOTHING handling: if (h_policy == CACHE_NOTHING) { // don't want ...
_asm mov ebx,ebx // Hack! prevent compiler ...
PVideoFrame result = childGetFrame(n, env);
if (result->vfb)
env->ManageCache(MC_ReturnVideoFrameBuffer, result->vfb);
return result;
}Won't the call to env->ManageCache() to return the vfb cause the caches of filters that use CACHE_ALL around a filter that uses the CACHE_NOTHING policy to have frames dropped from their lists that shouldn't necessarily be dropped?
...
while allowing tfm/tfmpp/tdecimate to default to CACHE_ALL, and that one works perfectly. However if I add in a filter that uses CACHE_NOTHING like this:

I assume you actually have this setup, 2 x avstimer, coz only 1 is useless. I have also marked in the cache instances for clarity.mpeg2source()
-> cache <= CACHE_ALL
tfm()
-> cache <= uses CACHE_NOTHING
avstimer()
-> cache <= CACHE_ALL (maybe absent)
tfmpp()
-> cache <= uses CACHE_NOTHING
avstimer()
-> cache <= CACHE_ALL (maybe absent)
tdecimate()
-> cache <= CACHE_ALL


now the caches before tdecimate and tfmpp only store 2 frames in their lists which results in an awful lot of the getframe calls from tdecimate needing frames to be redone from tfm on up.

If I comment out that line from above, or set tfm/tfmpp/tdecimate to use a CACHE_RANGE policy then everything works fine.Yeap, full points for a bug find. ;)

Avstimer() is a special case, Crop(), etc, are all in the same boat, in that they pass VFB's straight thru unmodified. By telling the cache below it CACHE_NOTHING, that cache dutifully returns all the associated VFB's to the tail of the queue for early reuse. Now when the cache above gets that same PFrame, it's VFB it has already been returned, and will get reused pdq. I think I will be taking that out in the next cut, I need to design a definative way to know a VFB is really unloved before I return it. :devil:

CACHE_RANGE is a lot more aggressive with it's VFB members, it actually bump's their use count while they are in scope, this has 2 effects. a) the VFB can't be stolen, b) all calls to MakeWritable() return a copy (and cost a blit). I tried this logic generally and it can cause severe memory shortage. 640x480 YUY2 VFB's are 600kb each, locking 7 buffers (radius 3) is 4.2Mb per cache instance, 24 cache instances (effectively script lines) and there's 100Mb. And the performance is not as good when strange things are not happening. :D

IanB

[scharfis_brain]

restore24 is not working anymore correctly with AVS 2.5.6 beta1.
(it produces jerky motion and lets blends sneak thru)

but restore24 works very nice using the predecessor, AVS 2.5.6 alpha 19-12-2004

might this be a problem of smartdecimate?
smartdecimate prooved being not very stable in past, unfortunately. but sadly this new AVS-version seems to broke everthing.


I will test this more tomorrow with some minor complex scripts.

[IanB]

Originally posted by scharfis_brain
restore24 is not working anymore ...Got a current link to it, the "Gleitz" link I have say "Not Found" (I think it does, mien Deutch ist nicht zehr gut) ;)

IanB

[Boulder]

I UL'd it here : http://www.saunalahti.fi/sam08/restore24.rar

[scharfis_brain]

updated package here
http://forum.gleitz.info/attachment.php?attachmentid=70159

I still need to test in a less complex environment.
Restore24 is way too complex....

[Wilbert]

I got a report from the german forum that SeparateFields separate the fields of field-based material, although it shoudn't do anything. This is indeed the case with the latest binary. I will look at it.

[stickboy]

neuron2 and I have been complaining about that inconsistency between SeparateFields() and Weave() for a long time. :)

Now, should SeparateFields() on a field-separated clip be a no-op, or should it generate an error? (And likewise, should Weave() on a non-field-separated clip be a no-op, or should it be an error?)

I'm leaning toward error, especially if you're going to be changing behavior that might affect existing scripts.

[Wilbert]

Yes, perhaps it's good to give an error. Saying that you should apply AssumeFrameBased before applying SeparateFields to field-based material.

I didn't check Weave. Does it also do something on frame-based material?

[IanB]

Sh0dan drop the test for already IsFieldBased() between revs 1.4 and 1.5 in the old 2.x branch (prior to the big file shuffle)

My vote is for "errors" when it's inconsistant, i.e. can only Weave() something that IsFieldBased()==True, can only SeparateFields() something that IsFrameBased()==True.

@Wilbert

On IsFrameBased() material :-

DoubleWeave() goes thru some motions and produces 1 same frame and 1 field offset frame.

Weave() goes thru the same motions as above but appears to do nothing. Remember Weave() is implemented as DoubleWeave().SelectEven()

IanB

[Wilbert]

Weave() goes thru the same motions as above but appears to do nothing. Remember Weave() is implemented as DoubleWeave().SelectEven()
Hmm. That complicates it a bit, because DoubleWeave handles field-based and frame-based material. Do you know whether it is possible to throw an exception anyway (using Weave on frame-based material)?

btw, there's still a bug in DoubleWeave and YV12.

[IanB]

Originally posted by Wilbert
Hmm. That complicates it a bit, because DoubleWeave handles field-based and frame-based material. Do you know whether it is possible to throw an exception anyway (using Weave on frame-based material)?Just commited to CVS, Weave() enforces IsFieldBased(), SeparateFields() enforces IsFrameBased(), DoubleWeave() untouched. Now we wait to see what the native think. :p

btw, there's still a bug in DoubleWeave and YV12.Fixed! Was missing offset calc code for Chroma, only had the Luma code. :D

IanB

[Wilbert]

About ShowFiveVersions


AviSource("F:\TestStreams\telecined-clip.avi") # YV12 clip
DoubleWeave()
a = Pulldown(0,2).Subtitle("0,2")
b = Pulldown(1,3).Subtitle("1,3")
c = Pulldown(2,4).Subtitle("2,4")
d = Pulldown(0,3).Subtitle("0,3")
e = Pulldown(1,4).Subtitle("1,4")
ShowFiveVersions(a,b,c,d,e)


resulted in a black clip (YUY2 mode works fine though).

[IanB]

Wilbert,

resulted in a black clip (YUY2 mode works fine though).Most strange :confused: I cannot reproduce it.

Q1. Is the output fully black, including the 2 grey corner blocks, or just the 5 frame elements?

Q2. Is the subtitle text visible or is that black as well?

Q3. Does a SeparateFields() first change the behavior? (Test 1)

Q4. Does a convertToRGB32() (or YUY2) at the end change anything? (Test 2, 3)

Q5. Does a StackHorizontal(a, b, c) suffer the same fate as the Show5? (Test 4)

Q6. Does a YV12 BlankClip() clone behave the same? (Test 5)

Q7. What are you displaying this with? What YV12 display codec are you using? Is it Xvid or DivX? What is the final total width? Is it mod 16? Does a Crop help? (Test 5)

AviSource("F:\TestStreams\telecined-clip.avi") # YV12 clip
# BlankClip(Last, Pixel_type="YV12", Color=$56789A) # Test 5
# SeparateFields() # Test 1
DoubleWeave()
a = Pulldown(0,2).Subtitle("0,2")
b = Pulldown(1,3).Subtitle("1,3")
c = Pulldown(2,4).Subtitle("2,4")
d = Pulldown(0,3).Subtitle("0,3")
e = Pulldown(1,4).Subtitle("1,4")
ShowFiveVersions(a,b,c,d,e)
# ConvertToRGB32() # Test 2
# ConvertToYUY2() # Test 3
# StackHorizontal(a, b, c) # Test 4
# Crop(0, 0, 640, 480) # Test 6

IanB :confused:

[Wilbert]

To summarize all the answers:

ShowFiveVersions(a,b,c,d,e)
BicubicResize(2*720,480)

works. It's DivX 5.1.1 to blame. What YV12 decoder are you using?


ShowFiveVersions(a,b,c,d,e)
BicubicResize(3*720,480)

is black though.

[sh0dan]

Could someone remove "threshold /=100.0f;" in line 419 in conditional_functions.cpp? I'm without CVS access ATM.

It seems like I'm dividing threshold by 100 two times. ;)

[Wilbert]

Could someone remove "threshold /=100.0f;" in line 419 in conditional_functions.cpp? I'm without CVS access ATM.
done

[tritical]

I am having a very strange problem with a build of AviSynth 2.5.6 compiled from cvs source checked out a few days ago. Whenever a script is closed some of the filters in the filter chain are not having their destructors called. For example, if I use this script (with the cache instances inserted):

mpeg2source()
cache()
blur()
cache()
tfm()
cache()
tfmpp()
cache()
blur() *
cache() *

tfm()/tfmpp()'s destructors are never called. In fact, the only filters that have their destructors called are the last two (marked with * above). If I use this script:

mpeg2source()
cache()
blur()
cache()
blur()
cache()
blur() *
cache() *

only the final two have their destructors called. Reverting back to a build from the 6th of january fixed the problem. Was wondering if anyone else has experienced this or if something is royally messed up on my system?

[IanB]

@tritical,

I'm a little short on AviSynth time presently, but I'll have a look the first chance I get.

Wilbert is about to build a new bundle, when it's done test it to see if the problem occurs (if so then it's a bug, if not then it's your build).

IanB

[Wilbert]

February 21st Beta 2 Release:

Great work by Ian:

additions:
- Added options to ShowSMPTE: ShowSMPTE(offset="00:00:00:30", x=360, y=576, font="georgia", size=24, text_color=$ff0000)
- Added Optional FPS=24 parameter to Dissolve and Fade*() for processing audio only clips.
- Added FadeIn0(), FadeOut0(), FadeIO0() variants that don't add an extra frame.
- Added Fractional resampling support in ResampleAudio().
- Added HasVideo() and HasAudio() script functions.
- Added Level option to Tone().
- Added ColorBars(pixel_type="YUY2, YV12").
- Added env->ManageCache() interface in AviSynth.h.
- Added VideoFrameBuffer 16 byte guardband protection/detection in Debug mode.
- Added EBX compiler bug protection/short circuiting to Cache::GetFrame().

fixes:
- Dissolve and Fade*() process audio only clips, assumes 24fps for position calcs.
- Fixed wrong chroma in DoubleWeave() of FrameBased YV12 material.
- Fixed crash from BlankClip() with an audio only clip template.
- Fixed AudioCache corruption on buffer resize.
- Fixed ResampleAudio() clicks/pops due to creeping error.
- Fixed SSE/SSE2 ConvertAudioTo16/32Bit() positive value saturation.
- Fixed ConvertToRGB() src->rowsize==8 crash.
- Fixed Memory leak, deleted VideoFrameBuffers.
- Fixed env->NewVideoFrame() returning short frame buffer.
- Fixed vi.FrameFromAudioSamples() truncation.
- Fixed ShowFiveVersions() unused corner rectangles had random contents. Now grey filled

changes:
- Histrogram: in color_mode - YUY2: Invalid values (below 16 and above 235) will be colored brown/yellow-ish. Made those values more visible.
- Evaluate: Now breaks outs and reports system exceptions instead of reporting the useless "Evaluate: Unrecognized exception!".
- BlankClip() no longer gobbles any implicit last clip as the template clip. You have to explicitly declare you want a template clip i.e. BlankClip(Last). "BlankClip()" now always returns the default blank clip.
- Avisynth.h FrameFromAudioSamples/AudioSamplesFromFrames() now test for divide by zero and return zero when encountered instead of crashing.
- SeparateFields() now throws an exception for an IsFieldBased() source.
- Weave() now throws an exception for an IsFrameBased() source.
- Cache policy CACHE_NOTHING currently no long returns all VFB's. Under review.
- Avisynth.h SetFPS() now test for zero numerator or denominator if found it sets num=0, den=1.

As usual download from Sourceforge (https://sourceforge.net/projects/avisynth2/).

PS:
1) I can't commit anything to CVS atm.
2) The package is a bit larger due to two png pics.

[tritical]

Just wanted to say thanks for the build Wilbert and thanks for the many improvements/fixes IanB :). This build seems to suffer from the same problem I mentioned earlier, the destructors for some filters are not called on exit. I dled and tested the jan 6 beta on sourceforge and it works fine, another build I had from around february 8 or 9 seems to be ok as well so it must a side effect from something changed recently.

[Richard Berg]

With the new alpha, I'm getting a hard quit whenever there's an exception (even a simple syntax error). Same in VDub / VDM / WMP -- the app just closes.

[JuanC]

Originally posted by Richard Berg
With the new alpha, I'm getting a hard quit whenever there's an exception (even a simple syntax error). Same in VDub / VDM / WMP -- the app just closes. I've got the same problem here

[Boulder]

I also got 'memory could not be 'read'' errors when exiting VDubMod. There are none with the earlier beta.

[Wilbert]

I'm sure IanB will look at these issues (cache problems). But Boulder, could you empty plugin folder and try again.

[GrofLuigi]

No problems with beta2 here.

GL

[clickit]

Since now I used all plugins without the loadplugin(....) command
but in this beta doesn't seems to work without it
I tried to play this simple script

MPEG2Source("film.d2v")

in MPC and the application just closed !!!
when i put the loadplugin("....\DGDecode.dll") command it worked fine

edit : all that happened with latest beta2

[IanB]

Tritical,

:o Yes indeed the windout code is not working. A problem with the way I hacked in allowing more arguments. I've CVS'ed up the repairs.

Get a replacement DLL from here. Feb 28th, 2005 Avisynth.dll (http://avisynth2.sourceforge.net/28Feb2005/avisynth.dll) Replace the one in C:\Windows\System.

Richard,

:confused: I can't reproduce it, but I did have a big hack at "Evaluate: Unrecognized exception!" code to make it fess up what the exception actually was. I could have broken it, or it could be a manefestation of the destructor bug above, this bug made things very sick in lots of unrelated places. Try again with the fixed version.

IanB

[JuanC]

Thanks a lot IanB, I have just dnlded the new DLL, but there's still the "vanishing" problem with a simple sintax error.

[IanB]

JuanC, Richard,

:confused: What are your platform details?

AMD or Intel? K6, Piii, Piv, Opteron, etc

OS? W98, Wme, W2K, XP, XPsp2, etc

Config? etc?

IanB

[JuanC]

My PC is a P4/2GHz(Northwood), 1GB DDR266, intel 845 chipset. WinXP-SP2.

[tritical]

@IanB
Yep, works fine on my comp now, thanks.

[Wilbert]

If I make errors in my script (using VDubMod's script editor) the following happens: Instead of showing the proper error message it says

"Cannot open file "": The system can't find the path specified."

If I reopen the faulty script in VDubMod I get the proper error message.

Athlon XP, W2K SP4.

[Richard Berg]

Same problem with new DLL. Plugin dir is empty. Scripts w/o errors work ok. Athlon XP. Windows XP SP2.

What compiler are we using now?

[JuanC]

Well, definitely it’s not a hardware related issue (in my case at least). My PC is multi booted. I have Win ME as an option. It’s rather clean/ naked, with just the critical updates; Directx 7.1, wmp 7.0…; and some old programs. I just tried AVIsynth 2.56 beta2 and also the latest dll running VdubMod. The syntax errors show as they should: VDubMod is not "vanishing" in WinME.

Some ideas come to my mind: Though my WinXP installation is rather old (I haven’t re-installed it since 2-3 years ago), it's fully functional. No problems with the previous AVIsynth 2.56 beta (jan.6). I was wondering what the relevant differences are between my XP and ME installations, and then I think I’d ask: Could these two make any difference: ?

Some time ago I had Visual Studio 6.0 with SP5 (and/or 6???) installed. Could there be some remaining stuff?
Also, to use some application(s?) (I don’t remember which one) I had to install MS .NET runtime, and it has been "windowsupdated" at least once.

Hope this helps…

[IanB]

Dear all,

DAMN! Microsoft SP2

Stay tuned...

IanB


Richard,

I'm still using the standard VC6 sp5 + PP.

[Richard Berg]

I will try a compiler that's not 7 years old and see if it helps anything :)

(free download (http://lab.msdn.microsoft.com/express/visualc/))

[Arachnotron]

I was playing a bit with sysinternals filemon and the 210205 build. Upon an error in an avs script, vdubmod and mplayerRC simply crash (no file not found error)

I tried a trace with filemon while opening a test.avs from the explorer using the "open with" context menu to open it in mplayerRC. I get this stuff in the log. Mind, I get the same when playing back an avs without an error in it. Maybe it is of help to someone. :)

22:20:37 explorer.exe:2564 OPEN C:\cap\AviSynth 2.56\test.avs\:Docf_SummaryInformation:$DATA FILE NOT FOUND Options: Open Access: All
22:20:37 explorer.exe:2564 OPEN C:\cap\AviSynth 2.56\test.avs\:SummaryInformation:$DATA FILE NOT FOUND Options: Open Access: All
22:20:37 explorer.exe:2564 OPEN C:\cap\AviSynth 2.56\test.avs\:Docf_SummaryInformation:$DATA FILE NOT FOUND Options: Open Access: All
22:20:37 explorer.exe:2564 OPEN C:\cap\AviSynth 2.56\test.avs\:SummaryInformation:$DATA FILE NOT FOUND Options: Open Access: All
22:20:37 explorer.exe:2564 OPEN C:\cap\AviSynth 2.56\test.avs\:Docf_SummaryInformation:$DATA FILE NOT FOUND Options: Open Access: All
22:20:37 explorer.exe:2564 OPEN C:\cap\AviSynth 2.56\test.avs\:SummaryInformation:$DATA FILE NOT FOUND Options: Open Access: All
22:20:37 explorer.exe:2564 OPEN C:\cap\AviSynth 2.56\test.avs\:Docf_SummaryInformation:$DATA FILE NOT FOUND Options: Open Access: All
22:20:37 explorer.exe:2564 OPEN C:\cap\AviSynth 2.56\test.avs\:SummaryInformation:$DATA FILE NOT FOUND Options: Open Access: All
22:20:37 explorer.exe:2564 OPEN C:\cap\AviSynth 2.56\test.avs\:Docf_SummaryInformation:$DATA FILE NOT FOUND Options: Open Access: All
22:20:37 explorer.exe:2564 OPEN C:\cap\AviSynth 2.56\test.avs\:SummaryInformation:$DATA FILE NOT FOUND Options: Open Access: All
22:20:37 explorer.exe:2564 OPEN C:\cap\AviSynth 2.56\test.avs\:Docf_SummaryInformation:$DATA FILE NOT FOUND Options: Open Access: All
22:20:37 explorer.exe:2564 OPEN C:\cap\AviSynth 2.56\test.avs\:SummaryInformation:$DATA FILE NOT FOUND Options: Open Access: All
22:20:37 explorer.exe:2564 OPEN C:\cap\AviSynth 2.56\test.avs\:Docf_SummaryInformation:$DATA FILE NOT FOUND Options: Open Access: All

[edit] Never mind, just installed 2.55 and while with that the avs error messages work as they should, these file not found errors are still in the filemon output.

[azsd]

with this feb28 dll replaced with installed beta1 2005-01-06
when I use unassigned variables or some other arg incorrecly,it will terminal the VDubMod directly(in last 01-06 build it will give an tip),and some correct scripts working in provours builds will randomizely get nero recode disappear while 2pass encoding,some times at 60%,some time at 80%.

So I choice get to use that alpha version back soon.

[IanB]

Here is an attempt to resolve the problems with XP sp2 and script exceptions causing fatal aborts.

XP sp2 users please test!

A replacement .dll is available here Mar 10th, 2005 Avisynth.dll (http://avisynth2.sourceforge.net/10Mar2005/avisynth.dll), just replace the one in your windows system directory.

A little help from someone with a VC6sp5 development environment on XP sp2 might be needed. I have no problem with W2K or W98 so I am guessing blind as to the problems cause.

Regards
IanB

[esby]

@IanB:

I am under windows XP & SP2.
I can test if you want.
Just tell me what kind of scripts to test.
I also have vc6.2 installed.

esby

[IanB]

Esby,

Originally posted by esby
I am under windows XP & SP2.
I can test if you want.
Just tell me what kind of scripts to test.
Ta muchly!

The reported problem was that script errors cause various video editors (Vdub, Tmpg, etc) to just cold abort, instead of reporting the syntax error. So run script with a syntax error, i.e. missing comma or bracket, a spelling mistake, etc...

I guess the test is reproduce the problem with both Wilberts 21st Feb release and my 28th Feb replacement avisynth.dll, then try my Mar 10th replacment avisynth.dll to see if the problem is fixed. If so then report it here, jobs done :D

If the problem still occurs, then suck down the latest CVS, build a debug version, check to see the problem still occurs and then run it up under the debugger and start with the bug spray. The most likely place to look is in "avisynth/src/core/parser/expression.cpp", the code near the top the file, the __try/__except addition is where I was last working.

T.I.A.

IanB

[esby]

Well I did some testing.

BigError.avs

BigError()


BigError2.avs

try{
BigError()
}
catch(error)
{
version()
}


with the dll I got from your link:

First, it says jan 23 2005 with version().

BigError.avs crashes abruptely.
BigError2.avs crashes abruptely.

Now if I build a cvs version.

Apparently it seems a bit weird.
with a debug version,
BigError.avs will display the normal error message.
BigError2.avs will display the version message, which is normal.
and with a release version, it will crashes.
I am not sure of what's going on, but I'll check sunday.

esby

[IanB]

Dear XP-sp2 users,

Well here is another attempt, I'm sort of figuring on a compiler glitch, so I've tweaked the code to improve the readability of the generated assembler and have proof read it.

Mar 15th, 2005 Avisynth.dll (http://avisynth2.sourceforge.net/15Mar2005/avisynth.dll), it is intended as a direct replacement of the avisynth.dll from the released 2.5.6alpha2 package. (Just put it in your Windows\System directory).

@Esby,

Thanks, try the new one please. The CVS commit should trickle thru to the public server in a few hours.

IanB

[JuanC]

I still get vdubmod "vanishing" on any avisynth syntax error with this last (march 15th) build.

Thanks for your hard work Ian. Is there any other thing, besides compiling avisynth (I wouldn't be able to do so), I could do to help test?

[IanB]

JuanC,

This is very frustrating :mad: blind debugging is a real <insert favourite profanity here>!!

Here is a version with what I think is the problem change backed out, can someone please test this.

Back out Avisynth.dll (http://avisynth2.sourceforge.net/15Mar2005backout/avisynth.dll), it is intended as a direct replacement of the avisynth.dll from the released 2.5.6alpha2 package. (Just put it in your Windows\System directory).

IanB

[Boulder]

With that dll, I get a correct error message when I try to load a clip that isn't there. If that's what you were after, it works:)

EDIT: and syntax errors appear correctly as well.

[IanB]

Boulder,

Thanks, at least I know I have the right haystack now I have to find the needle:( maybe if I just sit down it will stick me.:sly:

To confirm :-

1. You are running XP-sp2,
2. the .DLL I put up this morning causes silent aborts with script errors,
3. the backout .DLL I put up the evening works A-Okay.

Looks like I am gunna have to get onto an XP-sp2 to solve this.:mad:M$

Regards
IanB

[Boulder]

Originally posted by IanB

To confirm :-

1. You are running XP-sp2,
2. the .DLL I put up this morning causes silent aborts with script errors,
3. the backout .DLL I put up the evening works A-Okay.

Yep, all three match.

[IanB]

Boulder, others,

Just a thought, did M$ update MSVCRT.DLL in SP2 because the runtime routine _except_handler3() is the other variable in the equation.

I'm currently running version 6.10.8924.0, what version does SP2 clock up?

Thinking out loud...

IanB

[Boulder]

I've got 7.0.2600.2180, which kind of adds up since isn't .2600.2180 XP w/ SP2, at least .2600 means WinXP.

The file creation date is in August 2004 so it would be around the time when SP2 was released.

[Zom-B]

in my WinXP SP2 C:\WINDOWS\system32 the msvcrt.dll got this Version Info:

7.0.2600.2180 (xpsp_sp2_rtm.040803-2158)

[IanB]

Yes those needles in haystacks really do sting when you sit on them.

Just did a quick google and it looks like M$ :devil: has done something bad with MSCVRT.DLL version 7.0.2600.2180, no concrete proof yet but it's a place to start to look.

Later dudes
IanB

[JuanC]

Just confirming:

\WINDOWS\SYSTEM32\MSVCRT.DLL 7.0.2600.2180 (xpsp_sp2_rtm.040803-2158)

The newer build doesn't make vdubmod to "vanish" on avisynth script syntax errors. It displays the correct error message. :) Thanks againg for your work!

EDIT: I noticed a change in the color bands to each side of the image showing the invalid colors in YV12 (<16 & >235), when using the Histogram() filter in classic mode. :) I preferred the old darker color in previous builds ;)

[Wilbert]

EDIT: I noticed a change in the color bands to each side of the image showing the invalid colors in YV12 (<16 & >235), when using the Histogram() filter in classic mode. :) I preferred the old darker color in previous builds ;)
I changed that. Those old dark colors are barely visible when making screenshots. Btw, it works for YUY2 too now.

[JuanC]

Originally posted by Wilbert
...Btw, it works for YUY2 too now. Great! Thanks :)

[azsd]

LanczosResize/BicubicResize/BilinearResize have strange frames in special target size:

when target width beside in 732 to 916,it's have an offseted channel s picture:

http://my1993.com/PpsDES/Cap/avs_resize_w852.jpg

Windows 2003,Avisynth 2.5.6 Pre-Releases.

[azsd]

sorry,I found the strange images may caused by xvid as YV12 decoder.

[digitalman]

This build fixed the problems I was having with DVR MPEG2 files. It would crash with certain plugins, or with certain programs like QUENC.

[Wilbert]

AviSynth 2.5.6 beta 3 Release (May 5th):

additions:
- Added method VideoInfo.MulDivFPS(unsigned multiplier, unsigned divisor) does rational scaling with normalizing and overflow protection of FPS property.
- Added offsets, gamma and analyze option (ala ColorYUV) in RGBAdjust.
- Added Spline16Resize, Spline36Resize, GaussResize and LanczosResize(tap=xx)
- Added options to ShowFrameNumber: ShowFrameNumber(offset=10, x=360, y=576, font="georgia", size=24, text_color=$ff0000)
- Added integer offset to ShowSMPTE: ShowSMPTE(offset_f=10)

fixes:
- Fixed FPS overflow with Select...(), Interleave() and variants.
- Fixed subtract mode in Overlay (chroma is correctly subtracted).
- Fixed align parameter in Subtitle.
- Revert "Evaluate: Unrecognized exception!" change for hard quit whenever there's an exception instead of showing error message (XPsp2 possible problem due to XPsp2's new msvcrt.dll).
- Windout code is working again (broking in the previous betas).
- Corrected multi-channel audio fading.

optimizations:
- Text overlay antialiaser tweaked, 8% faster Work in progress, more to come.
- RGB32 FlipHorizontal() code tweaked.

changes:
- Planar alignment (default 16 bytes) now based on chroma planes, luma alignment is (still) 2x chroma.
- Support for [vfr content in DSS]: DirectShowSource("F:\Guides\Hybrid\vfr_startrek.mkv", fps=119.88, convertfps=true) turns vfr into cfr stream by adding frames.

As usual download from Sourceforge (http://sourceforge.net/projects/avisynth2/).

[Boulder]

Something is broken..setting any postprocessing level in MPEG2Source (the latest DGIndex/DGDecode combination) gives garbage output. With parameter CPU=0 no problems. IanB's last version works just fine.

[tritical]

I noticed that as well, but it appears to be a bug in dgdecode not avisynth. When dgdecode calls postprocess it uses width as the pitch instead of the actual y plane pitch. In this avisynth release it looks like UV pitch is calculated first for YV12, and by default is 16 byte aligned, so the y pitch ends up being 32 byte aligned, and thus not equal to the width for non-mod 32 frames (720xXXX for example). Only one line needs to be changed to fix things though :).

[stickboy]

Originally posted by Wilbert
- Added method VideoInfo.MulDivFPS(unsigned multiplier, unsigned divisor) does rational scaling with normalizing and overflow protection of FPS property.

- Fixed FPS overflow with Select...(), Interleave() and variants.Awesome. I kept meaning to add some code to do this (analogous to my AssumeScaledFPS (http://www.avisynth.org/stickboy/) function) but laziness and other work kept overwhelming me. :)

[IanB]

Guys,

This version has the "Evaluate: (Un)Recognise(d) Exception." code back in, very much distilled and refined, and under XPsp2 it still cause host programs (editors/viewers) to hard terminate. Under XPsp1/W2K/Nt4/98/Me everything seems to work A-Okay.

So I am again looking for some very kind soul with an XPsp2 platform with VS6sp5+pp (std AVS build env) development environment, to build a "Release" .dll with symbols and run it up under the debugger, load a script with a deliberate error and tell me where, why, how, when and what Microsoft did in sp2 that so royally screws this simple piece of code.

The problem is in the 3 routines around line 144 of core\parser\expression.cpp and seems to be something awry in msvcrt.dll to do with the routine __except_handler3

T.I.A.
IanB

[iNFO-DVD]

just tried this beta 3 release and now have a green bar down the left side when viewing movie's via avs, wern't there in previous version, doesn't happen with all movies.

[IanB]

Originally posted by iNFO-DVD
just tried this beta 3 release and now have a green bar down the left side when viewing movie's via avs, wern't there in previous version, doesn't happen with all movies.
:script:
At a guess I'm gonna say this is something to do the increased YV12 plane alignment, and odds are it's not gonna be in the core avisynth.dll (but it might be:o ). However we do support a very large number of client pluggins and server applications, Tritical has already found one in DGdecode, so a little bit of extra information like :-
1. The application(s) that exhibit the problem.
2. Some info about the movies like formats and frame sizes.
3. Your script.
4. Width of the green bar, 8 pixels, 16 pixels, 100 pixels.
5. Your OS platform (XPsp2, XP, W2K, Nt4, ME, 98, etc)
6. Your processor P-III, P-IV, K6, Athelon, etc

Any little bit can be extremely helpful.

IanB

[iNFO-DVD]

I've removed it now and haven't got time to test right now as it's 2:40 am, and bed's calling.... :D

But, my OS = WinXP Pro SP2
Intel P4 3GHz

The green bar appeared in an NTSC clip (29.970), but not in my PAL or FILM clip, but that could have been coincidence. I tried to snapshot the green bar, but failed, I'd guess it was somewhere between 8 to 16 pixels wide, ran down the left and appeared to have interlacing..... you know, thick weave bars....

the script was just a simple one, the bare minimum.

Get more info for ya when i wake...

[iNFO-DVD]

It was on my mind, I re-installed it and managed to get a screen-shot of it, it's attached to this message.

It's an NTSC clip, well, a bit hybrid actually, anyway, doesn't happen all the way through, just different places and when displayed, it flashes.

[sh0dan]

:script:

[IanB]

For reference the image is 726x486.
The image seems to have a 3 pixel border all around.
The green bars are 16 pixels wide and 2 pixels high with a 2 pixel gap.
The 1st occurance, lines 01, seems absent.
The pattern appears on lines 45,89,...
The lower half of the green bar is effected by the adjacent luma, this may be a JPG artifact local to the image.

Starting to smell like the DGDecode problem from a few posts up.

IanB

Edit : B I N G O Look, smells, feels, quacks like a duck - must be a duck:sly:

[iNFO-DVD]

my script was simply:

LoadPlugin(blah blah blah \DGDecode.dll")
MPEG2Source( blah blah blah \VTS_03_PGC_04_1.d2v")

BTW, the movie/clip in question is:

UNBREAKABLE
NTSC REGION 1
(DISK 2) SPECIAL FEATURES
VTS03
PGC04

[Boulder]

Originally posted by IanB

Starting to smell like the DGDecode problem from a few posts up.

Is Don aware of the issue? I didn't see any posts in the appropriate forum regarding it.

[neuron2]

Please explain the alignment changes and why they are considered important enough to break existing applications.

[neuron2]

Originally posted by iNFO-DVD
LoadPlugin(blah blah blah \DGDecode.dll")
MPEG2Source( blah blah blah \VTS_03_PGC_04_1.d2v")
I don't see any post-processing here. Are you guys saying that DGDecode is broken for everything now, and not just for PP?

[amango]

Only postprocessing is broken (for example "cpu=4") in DGDECODE with this new build of AVISynth (creates funny pictures). With no PP DGDECODE works fine with this build.

[neuron2]

Look at iNFO-DVD's script. Where is the postprocessing???

[MacAddict]

Only seen it here using iPP as well, nothing else. Perhaps he posted the wrong script?

[vinetu]

Originally posted by MacAddict
Perhaps he posted the wrong script?
Indeed he posted blah blah blah :)
what's wrong with this? ;)

[neuron2]

The blah blah blah is for the path. The cpu parameter comes after the filename. tritical says this also affects normal processing when there are RFFs.

Again, I ask, why was this alignment change made? Is the person who did it willing to fix all the broken applications? Why wasn't this discussed at all so that affected people could have their say?

[iNFO-DVD]

excuse me, i certainly did NOT post the wrong script..... The bla bla bla was instead of the path which included my 'real' name.

it doesn't matter what script i use and whether there's post processing or not, the outcome is the same.

[Cyberia]

For what it's worth, I can reproduce video corruption using this AviSynth beta. Corruption was not there before I applied the beta.

It seems to be PP based, because CPU=0 produces a clean image.

Here's my script:LoadPlugin("F:\DVD\AviSynth\plugins\DGDecode.DLL")
mpeg2source("D:\vidtemp\FTE6\erg.d2v", cpu2="ooooXX")


Every CPU and CPU2 setting I tried seems to be affected, but I haven't tried them all. iPP does not change the problem.

Here's a screenshot.

[IanB]

Originally posted by neuron2
Again, I ask, why was this alignment change made? Is the person who did it willing to fix all the broken applications? Why wasn't this discussed at all so that affected people could have their say?

The alignment change was requested so SSE code would work on the chroma planes. Increasing the luma alignment from 16 to 32 should not effect any correctly written code, it should just occasionally waste a little more memory, obviously this is not quite the case.

The issue has been discussed in http://forum.doom9.org/showthread.php?s=&threadid=90527 It is only in developement, if it doesn't work out, we can always change it.

IanB

[iNFO-DVD]

BTW, I just tried CPU=0, the image was the same.....

[neuron2]

All versions of mpeg2dec and its derivatives have this "incorrectly written" code, which happens to work fine until the alignment rules are changed. Now I'd like to know who is going to fix all of those filters, or are you happy to leave them behind?

Isn't there some way to provide polymorphism, so that existing filters can operate in the old way without requiring changes?

[Wilbert]

I'm also worried about it. It might affect other filters too. I think we can't take that risk.

[IanB]

Don,

The fact is the filters are broken already, you can't feed a frame that has been cropped or is not mod 16 width into any of the mpeg2dec stuff. With mod 32 alignment the restriction becomes a little tighter at mod 32 width.

Version 2.5.6 is the current development branch and this is where we need to push the envelope a little to see what is possible, what is feasible and what breaks.

Fine if mod 32 turns out to be an insurmountable problem then we leave it out in the final released version and try again in 2.5.7, just like we will probably have to do with the "Evaluate: (Un)Recognised exception." SEH problem with XPsp2.

Mod 16 aligned chroma is actually very important if we want to use SSE code. Somewhere down the track I'd like to see an SSE2 resizer core and I'd like to have the infrastructure in place and well tested before I get a round tuit:sly:

If I can find time I'll have a proper look at the code and see what can be done. At first glance it looks like just the conv4xxto4xx series engines just need to be taught to use pitch and/or GetRowSize(planar_aligned)instead of width.

IanB

[Nic]

@IanB: If you could let me know (either here, or by PM) exactly what I'm looking for with the SEH XP SP2 problem, I'll look into it for you.

I'm sure making DGIndex/Decode use your new alignment will be a pain but probably for the best. But I think the ramifications could run to old unsupported filters too...So could there be some switch/pragma/regkey for turning the alignment back to 16 (8 for chroma)?

Because, technically Ian your dead right, the filters are broken to an extent, however they do work perfectly well for a lot of people and to take that away would be quite bad...

-Nic

Edit:
ps. I have to use VC6 SP6, so no Pro Pack 5. So i'll have to make minor changes...Plus, where do _aligned_malloc/free come from, they're not in any platform SDK I use, but maybe I'm out of date.
@Mods: If this is all a bit too off topic, I'll continue it in the development thread.

[neuron2]

Originally posted by IanB
The fact is the filters are broken already, you can't feed a frame that has been cropped or is not mod 16 width into any of the mpeg2dec stuff. mpeg2dec, etc., are source filters; you don't feed frames into them.

[sh0dan]

mpeg2dec3 could simply request NewVideoFrame(-8) - that should force mod 8 aligned chroma, and not necessarily mod 16 (I don't assume this has changed).

But you need to include encapsulate it in the AlignPlanar class. That class was specificly made for filters that do not always return frames with pitches that are what Avisynth requires. The definision is in avisynth.h, and I just moved the implementation to src/core/alignplanar.cpp for easier use in plugins.

The reasons I didn't push for mod 16 chroma:

* Main reason: A pure SSE2 filter is not useful - you need fallback to at least iSSE, so switching to fallback routine on unaligned pitches should be an option.

* Do it in the Win64 port. All filters would need recompile anyway.

My personal opinion: Move it to 2.5.7 and get 2.5.6 out the door without breaking stuff. With ALL mpeg2dec versions/forks having this problem there will be too big problems for people when upgrading.

[Cyberia]

Originally posted by neuron2
mpeg2dec, etc., are source filters; you don't feed frames into them.

But the original vob does feed frames into mpeg2dec...

I believe IanB means that if the source video (vob, etc) has a non-mod-16 width, this problem will happen with MPEG2Dec (and derivatives), regardless of the AviSynth version. Does anyone have a vob like that that we could test this theory against?

[neuron2]

@Cyberia

MPEG2 widths are always a multiple of 4 so they're guaranteed to be 16-aligned.

[IanB]

Originally posted by neuron2 MPEG2 widths are always a multiple of 4 so they're guaranteed to be 16-aligned. Come again mod 4 width implies mod 4 (32 bit) alignment.

I just knocked up a 712x480 Mpeg2 (TMpg would only do mod 8) and just about every piece of Mpeg2 software I have fell out of the sky, only PowerDVD survived and it upsize the image to 720x480 with a 8 pixel black border on the right, so only half marks.mpeg2dec, etc., are source filters; you don't feed frames into them.True for MPEG2Source, but all the auxilary filters; LumaFilter, YV12toRGB24, YV12toYUY2, BlindPP & DeBlock; expose the alignment challenged internals nicely.Originally posted by sh0dan
* Main reason: A pure SSE2 filter is not useful - you need fallback to at least iSSE, so switching to fallback routine on unaligned pitches should be an option.Pure SSE2 can handle any alignment, it just takes a big hit with non-mod 16 alignment, however it is still faster than the equivalent MMX code especially when some of the fancy fangled intructions are used.

Fallback code should be for supporting lesser machines, not poor programming. And yes I admit to doing this just like everybody else, I just should know better. :sly: * Do it in the Win64 port. All filters would need recompile anyway.Ah em rewrite I hear you say :angry: f... Micro$oft, Softwire may yet save the day;)

I think leaving the current alignment in until we are nearer to release to shake any other gremlins loose, then take it out until 2.5.7 to save unconfortable problems in a released 2.5.6 version is the go.

IanB

[neuron2]

@IanB

Why can't you just make the default be 8-byte chroma alignment with the new mechanism? Then old stuff would still work but you could ask for 16-byte chroma alignment if you want it.

[Leak]

Originally posted by neuron2
Why can't you just make the default be 8-byte chroma alignment with the new mechanism? Then old stuff would still work but you could ask for 16-byte chroma alignment if you want it.

Well, I guess the idea is to make sure most stuff uses 16-byte chroma alignment as soon as possible (i.e. from the get-go) as otherwise you'd again have to blit the frame to re-align it, probably several times in a filter chain...

np: Ricardo Villalobos - Theogenese (Alcachofa)

[IanB]

Originally posted by neuron2
Why can't you just make the default be 8-byte chroma alignment with the new mechanism? Then old stuff would still work but you could ask for 16-byte chroma alignment if you want it. Leak has got it in one. ;)

8 byte chroma, 16 byte luma is the current way. The future way want to be 16 byte chroma, 32 byte luma.

Some history
In Kassandro request he wanted most child filters returning mod 16 aligned chroma planes and I agree with him it is a worthwhile enhancement. My first thought was to just realign the chroma planes but all the baked in code enforceing chroma must be half luma in avisynth.h stopped that idea cold, a full recompile of the world is not practical and I don't want to be lynched. Next thought was bump FRAME_ALIGN=32 in avisynth.h again the baked in nature of the constant meant somethings would still slip thru the cracks and non-planar formats would be wasting memory and squandering cache lines. As I researched the internals of AVS for a better answer I found that by shifting the definition of planar alignment from the luma plane to the chroma plane in NewVideoFrame(), Kassandro could have his cake and eat it to. There is no baked in component, no recompiling and it works straight away with a new version. Only problem is there is code out there that thinks it knows about frame buffer layout. :mad:

IanB

P.S. Like the debug build 64 byte guard regions I added recently to shake out bounds error I'm starting to think it might be a good idea in debug builds to randomize frame alignment somehow to shake out any smartalec code lurking. Thorts? :D

P.P.S. :sly: I have an idea for a script verb to set the alignment for the next filter to overcome these legacy problems, just got a small chicken before egg problem with source filters like MPEG2Source(), let me ponder upon it a while, I see some illumination.

[tsp]

Is there a way to get the current alignment at runtime?

so something like this softwire instruction from XtoRGB would work right( it will work anyway because RGB24 is still 16 bytes aligned right?)

cmp( edx,(width*3+15)/16*16/3*3);

[IanB]

(int)(frame->GetReadPtr())
frame->GetPitch(PLANAR_Y)
frame->GetPitch(PLANAR_U)Then check the bottom bits for zero.

:confused: Why check the width. The pitch is what is interesting.:confused:

This is where Mpeg2dec has gone wrong it assumes width == pitch.

To present aligned frames AVS rounds the width up to a multiple of Alignment, this is the pitch. It mallocs a memory buffer pitch*height+(a little bit) it then calculates an offset from the beginning of the buffer such that the address of buff[offset] is appropriatly aligned in memory, this address is what is returned by frame->GetReadPtr(), (a little bit) is the maximum value of offset.

IanB

[tsp]

IanB: Put what if the frame only contains zeros. One way to do it might be to reqeust a frame of size 1x1 and see how big it is. Also the softwire code is only called in the constructor(where the assembler code is generated) so the pitch is rather useless in the test because it can change from frame to frame but the single instruction does use the pitch instead of the width. This part calculates the minimum pitch if it's 16 Byte aligned
(width*3+15)/16*16
and the last part /3*3 rounds it down to the nearest pixel

[edit]
so the optimal code should like this:


cmp( edx,(width*3+ALIGNMENT-1)/ALIGNMENT*ALIGNMENT/3*3);

[tritical]

Question about setcachehints using CACHE_RANGE. If I am requesting frames that are in the range of n-4 to n+4 assuming n is the current frame... then I should call SetCacheHints with (CACHE_RANGE, 4). In the setcachehints method it doubles this to 2*4+1, but then in the GetFrame method under the CACHE_RANGE handling I see this:

// Check if it is out of scope
else if (abs(h_video_frames[i]->frame_number-n)>h_radius) {

That should be:

// Check if it is out of scope
else if (abs(h_video_frames[i]->frame_number-n)>=h_total_frames) {

shouldn't it? Let's say current frame is 16 and so I start by requesting 12. Then with >h_radius it deletes cached frames 17, 18, and 19, which I am about to request. And when I go up to requesting frames 16+ it starts dropping frames off the other end! That doesn't seem to be correct.

I changed it to >=h_total_frames and tested, it made a major difference in speed :).

[IanB]

@tsp,

I am a little confused as to what you are actually asking so I'll answer both ways. If I still have it wrong PM me to set me right.

1a. For an input frame the worst alignment you can get will be a minimum pixel unit; 32bits for RGB32 & YUY2, 24bits for RGB24, 16bits for YV12 luma & 8 bits for YV12 chroma; This can be caused by say a 1 unit left Crop().

1b. Also for input frames the pitch can be many times the width. Again with a little help from our friend say Crop(most_of_the_width).


2a. For Env->NewVideoFrame(vi, align), for "align" >= 0 this will provide a guaranteed pitch aligned >= "align" Note! >= i.e. Ask for 4 get 16, ask for 32 get 32. And default for "align" is FRAME_ALIGN (currently 16)

2b. However if "align" is < 0 the pitch is force aligned to "-align" i.e. "align"=-1 pitch==rowsize, "align"=-3 pitch==rowsize+2/3*3. Rowsize is Width*BytesPerPixel. And it is generally unfriendly to force wierd alignments (but RGB24 is already unfriendly)



@tritical,

Originally posted by tritical
// Check if it is out of scope
else if (abs(h_video_frames[i]->frame_number-n)>h_radius) {

That should be:

// Check if it is out of scope
else if (abs(h_video_frames[i]->frame_number-n)>=h_total_frames) {

shouldn't it?Yeap it's wrong.

I am halfway thru reviewing the cache, I have done a slight pretidy up and built and installed some infrastructure I will need. I just need a large lump of time and to psyche myself up for it. Currently CACHE_RANGE stops tracking frames that go out of scope so even if the frame hasn't been reused yet it can't be found.

IanB

[Backwoods]

In the current beta, this is happening:

http://forum.doom9.org/showthread.php?s=&threadid=90042&perpage=&pagenumber=2

function TimeDenoise(clip clip, float "strength", bool "interlaced"){
strength = default(strength, 1.0)
interlaced = default(interlaced, true)

order = (clip.getparity == true) ? 1 : 0

clip = (interlaced == true) ? clip.ConvertToYV12(interlaced = true) : clip.ConvertToYV12()
clip = (interlaced == true) ? clip.TDeint(mode=1,order=order,type=2) : clip

clip = clip.FFT3DFilter(sigma = strength / 2, bt = 2, bw = 48, bh = 48)

backward_vectors4d = clip.MvTools_MVAnalyse(blksize = 16, isb = true, lambda = 1000, delta = 4, idx = 1, pel = 2)
backward_vectors3d = clip.MvTools_MVAnalyse(blksize = 16, isb = true, lambda = 1000, delta = 3, idx = 1, pel = 2)
backward_vectors2d = clip.MvTools_MVAnalyse(blksize = 16, isb = true, lambda = 1000, delta = 2, idx = 1, pel = 2)
backward_vectors1d = clip.MvTools_MVAnalyse(blksize = 16, isb = true, lambda = 1000, delta = 1, idx = 1, pel = 2)
forward_vectors1d = clip.MvTools_MVAnalyse(blksize = 16, isb = false, lambda = 1000, delta = 1, idx = 1, pel = 2)
forward_vectors2d = clip.MvTools_MVAnalyse(blksize = 16, isb = false, lambda = 1000, delta = 2, idx = 1, pel = 2)
forward_vectors3d = clip.MvTools_MVAnalyse(blksize = 16, isb = false, lambda = 1000, delta = 3, idx = 1, pel = 2)
forward_vectors4d = clip.MvTools_MVAnalyse(blksize = 16, isb = false, lambda = 1000, delta = 4, idx = 1, pel = 2)
clip = clip.MvTools_MVDenoise(backward_vectors4d, backward_vectors3d, backward_vectors2d, backward_vectors1d,
\ forward_vectors1d, forward_vectors2d, forward_vectors3d, forward_vectors4d, tht = int(strength * 6),
\ thsad = int(strength * 800), thSCD1 = int(strength * 1000))

clip = clip.ConvertToYUY2()
clip = (interlaced == true) ? clip.separatefields().selectevery(4,0,3) : clip
clip = clip.VagueDenoiser(threshold = strength, wavelet = 3, method=1, nsteps=16 ,chromaT=0.8, Wiener = false,
\ percent = 80, interlaced = false)
clip = clip.LimitedSharpen(ss_x = 2, ss_y = 2, strength = 255, wide = true)
clip = clip.Unfilter(int(strength * 4), int(strength * 4))
clip = (interlaced == true) ? clip.Weave() : clip

return clip
}

This function is no longer working.

[Wilbert]

You know the drill. Post the most simple script which is not working.

[IanB]

Dear All,

I've just commited changes into CVS to add a SetPlanarLegacyAlignment(clip, bool)] function. It attaches itself to the child->GetFrame() chain so it works backwards up your script, i.e. you put it after the plugin that is making illegal assumptions about memory layout.
Mpeg2Source("test.d2v")
SetPlanarLegacyAlignment(True)
SuperSSE2ChromaFilter()
OtherSSE2AwareFilter()
SetPlanarLegacyAlignment(False)
BlindPP()
SetPlanarLegacyAlignment(True)

@Backwoods,

Try this when a build becomes available.

IanB

[neuron2]

OMG, you've got to be kidding. You know people think Avisynth is hard to use already. Wait until average users have to understand planar alignment.

[Fizick]

IanB,
Probably in this case you must use different greeting, not:
Dear All,

but:
Dear neuron2
:D

[Backwoods]

Originally posted by Wilbert
You know the drill. Post the most simple script which is not working.

I posted the function which was not working, which I figured would be enough since the script is just calling on the function. The script was also in the link. Either way, here is the script and I will test it again once a new build is up.

LoadPlugin("C:\Program Files\AviSynth 2.5\yea\fft3dfilter.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\yea\mvtools.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\yea\unfilter.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\yea\Tdeint.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\yea\MaskTools.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\yea\VagueDenoiser.dll")
Import("C:\Program Files\AviSynth 2.5\yea\LimitedSharpen.avs")
Import("C:\Program Files\AviSynth 2.5\yea\TimeDenoise.avs")

AVISource("otto.avi")
TimeDenoise(1.0,interlaced=false)

[Wilbert]

most simple script ...

So, start with TDeint, FFT3DFilter, MvTools_MVAnalyse, separately ...

[Backwoods]

Originally posted by Wilbert
most simple

So, start with TDeint, FFT3DFilter, MvTools_MVAnalyse, seperately ...

UnFilter and MvTools_MVAnalyse make the test script crash.

AVISource("otto.avi")
MvTools_MVAnalyse(blksize = 16, isb = true, lambda = 1000, delta = 4, idx = 1, pel = 2)
AVISource("otto.avi")
UnFilter(HSharp, VSharp)

[neuron2]

Originally posted by Fizick
IanB,
Probably in this case you must use different greeting, not:
Dear All,

but:
Dear neuron2
:D I don't get it. Please explain.

[IanB]

@Neuron2,

Don, I think Fizick, might be tugging our chains, just a little :p
you might be petulant and I might be condesending or is it the other way round:D :D :D :D :D

OMG, you've got to be kidding...

I am, in reality I expect that most people still in need of legacy alignment to just stick one SetPlanarLegacyAlignment(true) at the end of their script and say dang it. I also hope pluggin's with problems are repaired by the time 2.5.6 is finished.

I was just demonstrating how you can turn it off and on throughout a script and that it is not a simple global switch.

IanB

[neuron2]

@IanB

Got it, thank you. :cool:

[Fizick]

neuron2,
I am sorry for too short post.
I tryed to tell a joke (almost ;) ), but my English is not so good, and it was late of night here.

Now i try write longer.

I really wanted to say that:
1. IanB really makes the things hardly understandable to wide public.
2. Probably you are one of a few who could fully understand it.
3. But you could be more kind to his innovations (in beta version).
May be suggest some solutions. (I can not do it due to lack of serious programming experience).

So, I was (and I am) for world peace. :)

IanB,
thanks for explanation. :)
I really want to say, that you could be more respectual to Neuron2,
and then he will permit you to do anything you want. :)
Sorry for offtopic.
Now I have a real question. Currently we try to translate Avisynth doc to Russian language.
IanB, I see your recent update of Avisynth doc on CVS. Is the current CVS documentation up-to-date with your latest Avisynth changes?
When you suppose v.2.5.6 will be released? We want to finish the translation to that time and put it to installation package.

[neuron2]

Originally posted by Fizick
I really want to say, that you could be more respectual to Neuron2,
and then he will permit you to do anything you want. :)
Please don't speak for me. I feel quite adequately respected by IanB.

Don't impute motives to the forthright expression of opinions. Speaking one's mind about technical issues does not constitute a lack of respect.

[Wilbert]

UnFilter and MvTools_MVAnalyse make the test script crash.

AVISource("otto.avi")
UnFilter(HSharp, VSharp)


AVISource("otto.avi")
MvTools_MVAnalyse(blksize = 16, isb = true, lambda = 1000, delta = 4, idx = 1, pel = 2)



Unfilter: you did substitute numbers for HSharp and VSharp right (at least in the short script above)? You also checked with AviSource("otto.avi").Info returns indeed YV12 (needed in both scripts)? So, try

AVISource("otto.avi")
ConvertToYV12()
UnFilter(4, 4)

MVAnalyse returns a bunch of numbers, so that's why it fails. Try


source = AVISource("otto.avi").ConvertToYV12()
vectors = source.MVAnalyse(isb = false)
source.MVShow(vectors)


If that works try


AVISource("otto.avi")
ConvertToYV12()
FFT3DFilter(sigma=3)

[Backwoods]

Ok, for one thing, I'm an idiot. I thought my AVI was already YV12 with Langarith but it was YUY2.

works:
AVISource("otto.avi")
ConvertToYV12()
FFT3DFilter(sigma=3)

works(but wouldn't the YUY2 AVI work too? I thought the new DLL supported YUY2, either way):
AVISource("otto.avi")
ConvertToYV12()
UnFilter(4, 4)

works:
source = AVISource("otto.avi").ConvertToYV12()
vectors = source.MVAnalyse(isb = false)
source.MVShow(vectors)


TimeDeNoise with ConvertToYV12() still does not work and shouldn't I be getting an error as to why it crashes instead of just a close out from VirtualDub?

[Fizick]

Backwoods,
please re-post your problem with fft3dfilter (if any) to its thread with more detail.
http://forum.doom9.org/showthread.php?s=&threadid=85790&highlight=fft3dfilter

[sh0dan]

Looking at this thread (http://forum.doom9.org/showthread.php?s=&threadid=94184) I seem to get worried.

I think messing with pitches is a generally bad idea. You create a bunch of problems.


* Initial byte alignment cannot be guarranteed to be 16 byte aligned, unless every crop is bitblitted. This is required for fast SSE2.

* SSE2 is not so widespead that you don't need to have fallback anyway.

* Only 20-40% of all images will not have 16 byte aligned chroma. (Rough estimate)

* Chroma data isn't that much data.

* Only planar data is currently guaranteed to have this alignment.

* SSE2 filters could just create their aligned frames and do the bitblit themselves. The blit will just be placed elsewhere - it's inavoidable in most cases (MPEG2, AVI, DSS input).

* Memory footprint will unnecessarily (for 95% of all filters) be slightly larger.

[kassandro]

Originally posted by sh0dan
Looking at this thread (http://forum.doom9.org/showthread.php?s=&threadid=94184) I seem to get worried.

Well, in this code the assumption "width=pitch" was made and you never should do this. Unfortunately, this problems seems to stem from mpeg2dec3, which I still have to use, because I use mostly .d2v files created by ProjectX. Unfortunately in DGDecode a cheap fix was done by replacing the new default alignment with the old one, i.e. NewVideoFrame(vi, -16). That may easily lead to an inhomogenuous pitch (the pitch may change from frame to frame), which slows down some filter like my RemoveDirt quite a bit and may easily crash quite a few filters, which tacitly make the assumption of homogeneous pitch.


* Initial byte alignment cannot be guarranteed to be 16 byte aligned, unless every crop is bitblitted. This is required for fast SSE2.

I don't understand this at all.


* SSE2 is not so widespead that you don't need to have fallback anyway.

For more than 2 years Intel produces only SSE2 capable chips and AMD sole non-SSE2 compatible chips are at the very low end. Probably, in 2006 no CPU will be sold, which is not SSE2 compatible.


* Only 20-40% of all images will not have 16 byte aligned chroma. (Rough estimate)

This is an extremely rough estimate! For me almost 100% of all frame would have no 16 byte aligned chroma. I always start with 720x576 YV12 frames and crop at the very end. Thus if each filter would use the default NewVideoFrame(vi), frames will never have a 16 byte aligned chroma.


* Chroma data isn't that much data.

After all its 50% of all data and later on you complain about wasting a tiny little bit of memory with the improved alignment.


* Only planar data is currently guaranteed to have this alignment.

because we only need it there.


* SSE2 filters could just create their aligned frames and do the bitblit themselves. The blit will just be placed elsewhere - it's inavoidable in most cases (MPEG2, AVI, DSS input).

As I already mentioned above, this would easily lead to an inhomogoenoeus pitch. On the other hand, if all filter would creat their frames with the default NewVideoFrame(vi) then we would always have a homogeneous pitch unless we use Crop or SeperateFields. As I rarely use SeperateFields and nearly always crop at the end of the script, I would always have a homogeneous pitch, if each filter uses the default alignment. Actually, I would never allow another alignment. Then the "dirty programming problem" in DGDecode/Mpeg2dec3 and in other filters would have to be fixed properly.

[neuron2]

Originally posted by kassandro
Unfortunately in DGDecode a cheap fix was done This guy likes to try to hammer me at every opportunity, because he imagines I have wronged his buddy 'mf'. It's getting tiresome.

I did not make any fix in DGDecode. I made available an experimental version to *test* operation under the new beta. I even named it DGDecodeX so that nobody would confuse it with a sanctioned version. Tests such as this help to expose all the issues that we would have to address if this alignment change goes forward.

@sh0dan

The -8 sets up 8-alignment on the chroma and 16-alignment on the luma under the new beta.

[kassandro]

Originally posted by neuron2
This guy likes to try to hammer me at every opportunity, because he imagines I have wronged his buddy 'mf'. It's getting tiresome.

This has really nothing to do with this old story. You are very easy to hammer. The not yet prohibited word "cheap" seems to be enough.

I have already mentioned one drawback of changing the default alignment: inhomogeneous pitch. There is a second one. Make an MPEG2 encoding of a video with width 712 or any other width not divisible by 16. Then, even with the old 2.5.5 alignment, you won't have pitch=width for the luma and probably have the same troubles. The "expensive" fix would be to remove the pitch=width assumption at all places in the source code. You will probably agree, that your fix of enforcing pitch=width is very cheap compared such an expensive fix, but in my opinion the expensive fix is the only one, with which one can really sleep well.

[sh0dan]

Originally posted by kassandro

* Initial byte alignment cannot be guarranteed to be 16 byte aligned, unless every crop is bitblitted. This is required for fast SSE2.
I don't understand this at all.

To be able to use MOVDQA, the load adress must be 16 byte aligned.
Besides, adding a "process the remaining 8 bytes, if there are any" after your loop shouldn't be a problem. You could even do it in C for that matter.


For more than 2 years Intel produces only SSE2 capable chips and AMD sole non-SSE2 compatible chips are at the very low end. Probably, in 2006 no CPU will be sold, which is not SSE2 compatible.

Not really as relevant, as what's out there. ISSE could be acceptable minimum now, but SSE2 - no way.


Then the "dirty programming problem" in DGDecode/Mpeg2dec3 and in other filters would have to be fixed properly.
I'm talking about end-user-perspective, not programmer-perspective.

Making thousands of already installed MPEG2DEC3/DgDecode explode when users install 2.56 is NOT the way to go, IMO. We've had enough of (unintentional) problems, with filters breaking because of an AviSynth update, and I don't think breaking the most popular is going to help anyone.

[sh0dan]

Originally posted by neuron2

The -8 sets up 8-alignment on the chroma and 16-alignment on the luma under the new beta.
@IanB: If this is true, you need to assure that AviSource and DSS behaves corectly. Both of these create VideoFrames that emulate the alignment which both of these are supposed to give. Do a search for "NewVideoFrame(" in the sources.

[kassandro]

Originally posted by sh0dan


To be able to use MOVDQA, the load adress must be 16 byte aligned.
Besides, adding a "process the remaining 8 bytes, if there are any" after your loop shouldn't be a problem. You could even do it in C for that matter.
[/QUOTE]
I now understand, what you meant. If you want to use MOVDQA always, you must bitblt if you crop the left side by an amount, which is not a multiple of 16. I didn't even think of this possibility and therefore I always check alignment and use an unaligned routine if necessary. As croping on the left side should only be done once and usually at the end, where you can also avoid the bitblt with a special option. Vertical cropping may be used earlier, to remove big black bars at the top and the bottom, but there no bitblt is necessary. Thus your idea makes a lot of sense.
The advantage of 16 byte alignment goes well beyond MOVDQA. For instance, if esi, edi, ebx are multitples of 16 you can do

__asm movdqa xmm0, [esi]
__asm pavgb xmm0, [ebx]
__asm movdqa [edi], xmm0

If esi, edi, ebx are not multitples of 16 you must do

__asm movdqu xmm0, [esi]
__asm movdqu xmm1, [ebx]
__asm pavgb xmm0, xmm1
__asm movdqu [edi], xmm0




Not really as relevant, as what's out there. ISSE could be acceptable minimum now, but SSE2 - no way.

CPUs without SSE2 are not effected by the alignment changes at all.


Making thousands of already installed MPEG2DEC3/DgDecode explode when users install 2.56 is NOT the way to go, IMO. We've had enough of (unintentional) problems, with filters breaking because of an AviSynth update, and I don't think breaking the most popular is going to help anyone.
There is nothing intrinsically bad about raising the alignment. In fact, the 2.5.6 beta did exhibit a programming bug in MPEG2DEC3/DgDecode, which of course must be fixed before 2.5.6 is released. As I pointed, this bug is already there with the old alignment, if the widths of the mpeg2 video is 712. If we would now return to the old alignment we simply would surrender to lousy programming. When we move to 64 bit, we certainly want the chroma to be better aligned. It is therefore better to have these troubles now than later, when we have a lot of other troubles as well.

EDIT: It must be a multiple of 32 - not 16 !

[Backwoods]

Originally posted by Fizick
Backwoods,
please re-post your problem with fft3dfilter (if any) to its thread with more detail.
http://forum.doom9.org/showthread.php?s=&threadid=85790&highlight=fft3dfilter

I don't know what the problem is now. Maybe I should revert to another build and see if TimeDeNoise() errors out.

[LigH]

{ edit - Sorry, just found: "FrameNumber" ~ "current_frame" }

[tsp]

Originally posted by LigH
And now for something completely different:

But: Is there any specific reason why there is no available function to have a condition or value based on the frame number? E.g. something like

Subtitle(string(clip.FrameNumber)+"/"+string(clip.FrameCount))

(just a simplified example) would be great, but seems to be impossible at the moment.

ehh like this?

scriptclip("""subtitle(string(current_frame)+"/"+string(FrameCount))""")

try it it works even in 2.55 :p

Originally posted by kassandro

There is nothing intrinsically bad about raising the alignment. In fact, the 2.5.6 beta did exhibit a programming bug in MPEG2DEC3/DgDecode, which of course must be fixed before 2.5.6 is released. As I pointed, this bug is already there with the old alignment, if the widths of the mpeg2 video is 712. If we would now return to the old alignment we simply would surrender to lousy programming. When we move to 64 bit, we certainly want the chroma to be better aligned. It is therefore better to have these troubles now than later, when we have a lot of other troubles as well.

when we move to 64 bit all filters will have to be recompiled anyway(so that could be a good reason start using pitch instead of width ;) ) so why not just implement the new alignment in the 64 bit version and let the current filters work in the 32 bit version? How expensive are a BlitBit anyway??

[ARDA]

@all
I've been doubting for more than a week to write this post. So my
apologizes in advance if I've misunderstood something.

To avoid too long post I'll not quote.

To discuss nowadays if SSE2 should be included or not seems to me
useless. SSE4 are coming soon and easier conditions for SSE2 are still discussed.

Right aligment seems a condition for any future development.

sh0dan brought a revolution in avisynth when he introduced YV12 colorspace to mention
just the beginning. We all were acquainted to see fast improvements, bugs and their fixes
one day and another. Nowadays we are missing his every day contributions; but I guess
everybody understands that people has also private life.

At that time there were also some voices complaining that all filters should be recompiled, etc...
But great changes has always a cost; and mistakes are unavoidable.

neuron2 has made so many contributions to this comunity that the list would be endless.
Mistakes, maybe, but who hasn't made? So any other considerations are completely wrong.

IanB has started a new and important change with right aligment for SSE2 instructions.
Some considerations about the general gain if just one filter with SSE2 is in the whole chain
has been made.
Obviously just one in the general process is useless; but is that the right question?.
I do not think so. The right question is to create the neccessary conditions to
be able to start step by step including SSE2 where possible and important.

To wait till 64 bits version. I sincerly don't understand. When we most of us have 64 bits machines
and a stable operating system, SSE2 will probably be and old thing (exaggerated). SSE2 are in most computers in market nowadays (with low prices) and already usable under 32 bits operating systems.
Are there here some personal preferences or any other reason I don't get to understand?

I must add I think all should be done in a controlled way to avoid conflicts. And maybe this is the origin
of this small polemic.

A bigger amount of memory will be needed with latest changes cause the bigger pitches.
That's a price should be discussed and suggestions could be done. New CPUs with bigger and faster
caches and another technic improvements of bandwith will not suffer too much.
Many others cpus could have a penalty depending on the algo and many other conditions that
would be too long to expose. High Definition will be our worst enemy about memory questions.

Some final words. Communication and collaboration seem to be the most important today.
And not less important please avoid adjectives and sarcasm. Remember that many members of this forum
have not English language as native one and can misunderstood some posts. Anyway I prefer forum discussion than private message or mailing list; many young developers will take advantage of technics posts.
My apologizes for my poor English.

As always I hope this can be usefull.
ARDA

[neuron2]

Originally posted by ARDA
Mistakes, maybe, but who hasn't made? Just for the record, I neither designed nor implemented any code that makes the pitch=width assumption. It was all inherited from MPEG2DEC3.

[ARDA]

Originally posted by neuron2
Just for the record, I neither designed nor implemented any code that makes the pitch=width assumption.
It was all inherited from MPEG2DEC3.

I was speaking in a general way, not pointing that; I know perfectly well where many of those inherited problems come from.

A little off topic but I've made a version of Lumafilter which never finished; fixing static global variables which are still present in DGDecode.dll ; plus SSE2 version; plus row loops which avoid problems with frame loops after separatefileds.
Most things I've done are written at top of cpp. Sources and dll included. I've renamed LumaYV12 to avoid conflict names.
In spite of the fact it's not finished, is a usable plugin. If you prefer to include it in DGDecode.dll delete the link.

[IanB]

Wow! Fortunatly most of the issues have been redressed already.

The -8 sets up 8-alignment on the chroma and 16-alignment on the luma under the new beta. Yes this is not as intended, I will fix it.

How expensive are a BlitBit anyway??Consider a blit the fastest filter that works on all planes and doesn't manipulate it's data in place. Many filters that only manipulate 1 or 2 planes and/or manipulate data in place are faster than a blit.

Just to reiterate and put peoples minds at rest. The final release version of 2.5.6 will not change the default alignment. I will continue to explore chroma priority alignment as the default for a while longer because I believe it is very important to get the infrastructure for 128 bit aligned memory access correct, in place and working (just not as default).

And guys please try and keep the strong emotive stuff out of the posts it makes gleening for the technical (important) stuff harder.

IanB

[neuron2]

DGMPGDec 1.3.1b5 has a proper pitch fix made by tritical. Please see the beta thread in the DVD2AVI/DGIndex forum.

Feedback will be appreciated there.

Thank you, tritical.

[tsp]

IanB: Could you make the code a little more threadsafe by changing the VideoFrame:: operator new(unsigned) to something like this:

void* VideoFrame::operator new(unsigned) {
// CriticalSection
for (LinkedVideoFrame* i = g_VideoFrame_recycle_bin; i; i = i->next)
if (i->vf.refcount == 0)
{
InterlockedIncrement(&i->vf.refcount);
return &i->vf;
}
LinkedVideoFrame* result = (LinkedVideoFrame*)::operator new(sizeof(LinkedVideoFrame));
result->next = g_VideoFrame_recycle_bin;
g_VideoFrame_recycle_bin = result;
return &result->vf;
}

and the VideoFrame constructor to something like this:

VideoFrame::VideoFrame(VideoFrameBuffer* _vfb, int _offset, int _pitch, int _row_size, int _height)
: vfb(_vfb), offset(_offset), pitch(_pitch), row_size(_row_size), height(_height),offsetU(_offset),offsetV(_offset),pitchUV(0) // PitchUV=0 so this doesn't take up additional space
{
InterlockedIncrement(&vfb->refcount);
InterlockedExchange(&refcount,0);
}

VideoFrame::VideoFrame(VideoFrameBuffer* _vfb, int _offset, int _pitch, int _row_size, int _height, int _offsetU, int _offsetV, int _pitchUV)
: vfb(_vfb), offset(_offset), pitch(_pitch), row_size(_row_size), height(_height),offsetU(_offsetU),offsetV(_offsetV),pitchUV(_pitchUV)
{
InterlockedIncrement(&vfb->refcount);
InterlockedExchange(&refcount,0);
}

It would be better if refcount was initialized to 1 and cache, ScriptEnviroment::MakeWritable, ScriptEnviroment::NewVideoFrame, ScriptEnviroment::Subframe VideoFrame::SubFrame decreased refcount by 1(to 0) just before returning the frame to PVideoFrame

[IanB]

tsp,

Perhaps you can elaborate on your scenario for thread unsafeness.

In my few minutes thinking about it I agree the code is thread unsafe (lots of AVS is) and the scenario I foresee is 2 threads scaning the freelist together, both grab the same VideoFrame and set about using it, screwing each other in the process. And the window is quite wide, the VideoFrame is "free" until a PVideoFrame eventually does an AddRef(). Your proposed solution does not resolve this conflict, the "InterlockedExchange(&refcount,0)" effectively make it free again.

I assume therefore you have another conflict in mind.

To resolve my scenario VideoFrame's need more carefull management of the free list. Something along the lines of find a candidate, Interlocked increment refcount, take it off the free list, check to see if we got it to ourself (i.e. refcount==1) if okay use it, else do recovery and restart scan for another candidate. Possibly there needs to be some other locking as well.

This subject kinda warrants it's own thread. It's a can of worms.

IanB

[tsp]

This subject kinda warrants it's own thread. It's a can of worms.


the discussion will continue in this thread: http://forum.doom9.org/showthread.php?t=95596

[neuron2]

@IanB

Are you aware of the corruption issue with 2.56 discussed here:

http://forum.doom9.org/showthread.php?p=676095#post676089

Falling back to 2.55 solved the problem.

[LigH]

Yes, the HC encoder showed the (AFAIR) "usual" pitch shift - diagonal green lines running through the heavily slanted video.

(See attachment.)

[neuron2]

So somebody should inform Hank about this.

[LigH]

hank315 knows it - I did betatests on Windows 2000 for him.

[trevlac]

AviSynth 2.5.6 beta 3 Release (May 5th):

additions:
- Added offsets, gamma and analyze option (ala ColorYUV) in RGBAdjust.


Are the Gamma options in beta 3 ??

I tried all sorts of ways to get RGBAdjust to accept a gg parm, but no luck. I see IanB's code in the CVS ... but I can't get it to work in the build.

BTW: I thought this was a nice addition. I'm not sure of another way to adjust the gamma of each channel seperately. :)

[IanB]

@Neuron2,

Yep, is broken -ve forced pitch option stuffing up DirectShowSource(), fixed.

@trevlac,

From the CVS log Thu May 5 06:20:59 2005 UTC (6 weeks, 6 days ago) by ianb so it probably missed out by a whisker.

IanB

[LigH]

I don't mean to urge - just for curiousity:

Are there still many open issues to solve until the next 2.5.6 beta/RC?

[Wilbert]

1) There's one issue with audio, asf and DirectShowSource. I will report about it here.

2) Another (audio, x264 and DSS) issue which needs to be checked: http://forum.doom9.org/showthread.php?t=96708

Other than that, i'm not sure.

[Wilbert]

AviSynth 2.5.6 beta 4 Release (July 10th):

additions:
- Added DevIL support for RGB32 to ImageSource.
- Added Russian language documentation. Thanks to Fizick and his team. Well done.
- Added Merge() filter. Includes very fast iSSE pavgb code for weight=0.5.
- Added MergeARGB(), MergeRGB() filter.
- Added ShowRed(), ShowGreen(), ShowBlue() filters.
- Added Planar version of env->SubFrame() (thanks TSP).
- Added SetPlanarLegacyAlignment() to select Planar alignment algorithm.
- Added Audio padding control to Trim().
- Added operator MulDiv(multiplicand, multiplier, divisor).
- Added AssumeScaledFPS(multiplier, divisor, sync_audio) maps vi.MulDivFPS.

fixes:
- Fixed 2 bit crosstalk in YUV horizontal resizers, thanks Squid.
- Fixed resource leak in DirectShowSource(), thanks Tritical.
- Fixed minor memory leak in Subtitle() and string(), thanks Tritical.
- Fixed global clip close down problem, thanks Tritical.
- Uninstaller now remove Docs\pictures directory and DirectShowSource and TCPDeliver plugins.
- Fixed ImageReader single file handling.
- Fixed ImageWriter info==false no longer write status text onto frame.
- Trap .WAV clsid handler GPF with filenames gt 47 chars.
- Fixed YUV text access violation, correct bounds alignment tests. (Regression)
- Fixed ExpFunctionCall::Call memory leak, thanks Tritical.
- Fixed PClip leak in MergeChannels(). Destructor chain not called.
- Fixed RGB text alignment inversion. Regression in May 5th ver.
- Fixed vfw resource leaks when opening bad AVI files.
- Fixed rmvb (dss.dll) stuck at 100% during encoding (thx stevencover).
- Fixed Loop audio processing.
- Fixed GeneralConvolution crash with cropped input.
- Restored forced (negative) planar luma alignment functionality.
- Fixed a stack of memory leaks, thanks Tritical.
- Fixed Direct AVISource input of raw YV12 and I420 sources.
- Fixed (fingers crossed) "Evaluate: Recognized exception!" Problems with XPsp2 hard terminate.
- Fixed CACHE_RANGE internal scope test, thanks Tritical.
- Fixed Mask() calc of greyscale, red/blue swapped.

optimizations:
- Run sort -unique|detab on color_rgb.avsi, got rid of all the duplicates.
- YV12 MergeLuma(), MergeChroma() include very fast iSSE pavgb code for weight=0.5.
- Cleanup VideoFrame garbage collection on script close (Thanks TSP).

changes:
- New improved cache!
- Uninstaller now deletes DirectShowSource.dll and TCPDeliver.dll from plugin directory.
- ImageReader doesn't auto fail over to DevIL for DIB/BMP type files. (DevIL crashs on palletted BMPs.)
- ImageWriter now throws an exception for non RGB24 format with DevIL? processing.
- ImageReader/Writer info text colour now pale yellow $f0f080.
- Replaced VideoFrame::SubFrame() with env->SubFrame() (Ready for MultiThreading, Thanks TSP).

As usual download from Sourceforge (http://sourceforge.net/projects/avisynth2/).

PS, this will probably be the last beta :)

[tritical]

A little report on the new cache... it seems to be treating the frame_range that is given for CACHE_RANGE as a diameter instead of a radius. Changing line 731 to if (frame_range*2 + 1 <= h_span) // Use the largest size when we have multiple clients and adding inh_span = h_span*2 + 1; after line 737 seems to make it work as expected. Hopefully IanB can report on the behavior. Also, just wondering where did the MAX_CACHE_RANGE limit of 21 come from? What should one do if a filter could require a larger range than that? just use the default CACHE_ALL?

[IanB]

@Tritical,

Yes I am in rather a quandry about h_span definition, the only internal example of CACHE_RANGE use is TemporalSoften and it uses the parameter as a diameter, probably because the range expunge code previously was wrong. I did a search here and in google generally, and I got nothing definitive. The implication was that it is a radius, but because of the previous bug it has mostly been used used as a diameter (except by you :D ) and as a diameter it has slightly finer control.

As for the limit I thought +/-11 frames was pretty generous, I can easily be convinced to make it higher.

Anyhow I have erred on the side of safety. Where it is to small it will give the new heuristic Frame locker a good workout. In many case CACHE_RANGE should be redundant.

Thorts?
IanB

[tritical]

Having it be a diameter is no problem as long as it is clearly stated as such. Since there has been a lot of ambiguity due to it always being mentioned as a radius but, due to the bug, never actually working that way, I think a clear definition of which it is would be good. Also, having it be a diameter now would insure that filters calling it with that convention wouldn't have unnecessary slowdowns with older avisynth versions (as some of my filters do atm).

On the limit, I was mainly worried because I saw unexpected behavior when using a CACHE_RANGE that was too small, but after looking at it some more it does eventually work itself out after cache_limit increases enough that it starts promoting frames. Out of curiosity, what is the cost of calling env->ManageCache(MC_PromoteVideoFrameBuffer, result->vfb) on all frames generated in the cache regardless of h_span? Or in other words, how much is the gain from not calling it on frames that will be protected?

[LigH]

According to katjarella, HC encoder works well with AviSynth 2.56 b4:

Example picture (http://katjarella.gleitz.info/sub/hc15beta.jpg)

[IanB]

@LigH,

Damn! Thanks for the prompt. The planar alignment issue got missed in the notes.

@All,

** In this release the default alignment for planar images is back as it has always been, 16 Luma, 8 Chroma. **

So all that naughty code out there that assumes a relationship between width and pitch will work again. To test any pluggins with the upcoming SSE2 optimized alignment put a SetPlanarLegacyAlignment(False) at the end of the script.


Having it be a diameter is no problem as long as it is clearly stated as such....Thats where the quandry came from, no clear statement, just vague inuendo. Okay I guess we need a list of all the places people know this issue is alluded to. I still need to do a cleanup pass on the source after any bugs have been shaken out and a nice comment block can be part of that as a start.

On the limit, I was mainly worried because I saw unexpected behavior when using a CACHE_RANGE that was too small,...Previously if the range was to small you effectivly got no cacheing, all the frames got purged as it thrashed thru the list each time and there was no tracking of frames in the pool. Now frames in the range are protected from reuse, other frames take there chances in the pool, so getting it wrong is not that serious anymore. And there is a fairly high price to the LRU pool for giving to many frames protection.

Out of curiosity, what is the cost of calling env->ManageCache(MC_PromoteVideoFrameBuffer, result->vfb) on all frames generated in the cache regardless of h_span? Or in other words, how much is the gain from not calling it on frames that will be protected?The cost per say is negligable. The issue is that frames with protection don't need to take there chance in the LRU pool with other frames and as such I don't promote them so they don't flush other frames competing for normal cacheing. Only when there is a chance that LRU pool caching will help do frames get "promoted". So I guess the gain is improved caching for regular frames.

IanB

[yaz]

thx for the improvements !!!

anyway, would some mod fit the header of the thread to the new release, pls ! (it still refers to b3, may 6)

thx
y

[azsd]

some scripts get 2fps encode speed with 2.5.6 b4....
if replaced the avisynth to b3 or other CVS (16 align) build,the render speed should be about 18fps.
I don't konw is it caused by witch filter,if commented the FluxSmoothST line then speed resumed,but In my option is caused by avisynth.dll

video = DirectShowSource("[FREEWIND][Stand_Alone_Complex_2nd_GIG][DVDRIP][H.264_DD][05].avi", audio=false)
audio = DirectShowSource("vol05.wav")

video = ChangeFPS(video, 19.98)

main = video

main = FluxSmoothST(main, 8, 3)
main = deen(main, "a2d", 1, 8, 12, 3)
main = aWarpSharp(main, 2.5, 2, 0.30, 1)

video = main

video = TextSub(video, "[FREEWIND][Stand_Alone_Complex_2nd_GIG][DVDRIP][H.264_DD][05].tc[M2].ass")

video = LanczosResize(video, 368, 208)

video = Tweak(video, 0, 1.01, -2, 1.025, true)

video = AudioDub(video, audio)

return video

[IanB]

azsd,

Does this simplified script show the problem?main = DirectShowSource("blah.avi", audio=false)

main = FluxSmoothST(main, 8, 3)
main = deen(main, "a2d", 1, 8, 12, 3)

return main

Does the script start fast then go slow after about 100 frame?

Does left arrowing the script back to the start then trying again make the problem go away?

Anyone got a url for the source to FluxSmooth?

Anyone make this script break with a debug version?

IanB

[azsd]

yes,the scrollbar slow down at the frame last time backward from.

the FluxSmooth 1.1a src avaliable here:

hxxp://www.indeus.com/sansgrip/avisynth/

mmm...

It looks like authors webpage moved now,
I had make a mirror here:
hxxp://my1993.com/downloads/source/sansgrip/FluxSmooth-1.1a_src.zip
(same as 0729 build on WarpEnterise collection page)

[LigH]

Why do you mask your links?

[azsd]

lots of anime forums I was daily visted has the rule "don't post direct link" (as manners because direct links will caught by google may cause some ISP block that homepage account.)
so I developed a habit for their recommend _http/hxxp url.

[IanB]

axsd,

Oops looks like I lost a line of code. Try this replacment avisynth.dll (http://avisynth2.sourceforge.net/14Jul2005/avisynth.dll), just replace the one in your windows system directory.

IanB

[azsd]

tested.
parse speed get allright now.

thx IanB!

[Zarxrax]

I believe I have found a bug in the Normalize() filter.
If you do something like DirectShowSource("file.avi").Normalize() then the audio output will be silent. If you do Avisource("file.avi"), or DirectShowSource("file.mpg") then it works fine. Seems to be an issue with loading avi through directshowsource.

[azsd]

I have same slient result with load some mp2 files with DirectShowSource() .
the problem is reproduce,but occured with special audio file,not all audio clip failed in normalize,it may cause by you audio content in your avi between mpg,or matrix mixer pluged in dshow graphic but not in vfw render path.

[IanB]

Do a Normalize(show=true).Info()

1. What is the calculated normalizing factor.
2. What is the peaks frame number.
3. What is the audio sample type.
4. What is the total frame count.

IanB

P.S. Is the audio codec being used capable of seeking? If it cannot then this is the expected behaviour. Normalize reads all the audio samples looking for the maximum value then tries to seek back to the begining, if seeking is not supported DSS just returns silence.

[Zarxrax]

Amplify Factor: 1.1262
Amplify DB: 1.0323
At frame: 383

Sample Type: Integer 16 bit
Total Frame Count: 650
Audio Codec: PCM

[IanB]

It's DirectShowSource() bug:- Reading the very end of the audio stream sets the internal "end_of_stream" flag. The read code checks this on entry and bails when it's set. Nothing ever resets this flag. Fix coming slowly :(

IanB

Edit: After hitting the end of the stream, and seeking back to the begining the stream does not render correctly, it sort of has jitter between the returned block of samples. :mad: Much research required....

[Wilbert]

bug-report: http://forum.doom9.org/showthread.php?p=687962#post687962

The script crashes (ie VDub just quits when loading the script) with v2.56b3 and beyond.

[IanB]

Wilbert,

Was stack overflow from Invoke, fix in CVS, see .../core/avisynth.cpp v1.25

IanB

[Zarxrax]

It's DirectShowSource() bug:- Reading the very end of the audio stream sets the internal "end_of_stream" flag. The read code checks this on entry and bails when it's set. Nothing ever resets this flag. Fix coming slowly :(


IanB

Edit: After hitting the end of the stream, and seeking back to the begining the stream does not render correctly, it sort of has jitter between the returned block of samples. :mad: Much research required....

Sorry to trouble you more, but I have come across a few files that also don't return any audio after using normalize while using Avisource(), so perhaps this is a problem in avisource too. One of the files used ADPCM codec, the other is MP3.

[tritical]

I noticed that with avisource as well. It's because fZeroRead gets set and is never reset. Also, with the directshowsource fix, wouldn't it be better to be setting end_of_stream=false before calling StartGraph() at the end of the seek method? Otherwise, it just sits there until startgraphs WaitForSingleObject call times out because receive returns immediately while end_of_stream is still set to true and it never sets the event... at least that is what happens on my comp.

Also, there seems to be a typo in avisynth_c.h. Two lines at the beginning use "ACSC_USE_STDCALL" while throughout the rest of avisynth_c.h and avisynth_c.cpp it is checking "#ifndef AVSC_USE_STDCALL" notice the C and V switch. I think this was pointed out once in an old thread but it is still there. Or is it really suppose to be that way for some odd reason :D?

[IanB]

Not sure what to do about these stuffed ACM (and DirectShow) codecs. I tried a simple cheat in Normalize() of not getting the last sample, but it doesn't help, you have to avoid having the codec access and decode the last segment. This can mean stopping a few 10,000 samples before the end for a codec with good compression. :eek:

May be a float option specifying seconds from the the end to stop the pass 1 scan, but this is horribly ugly and cludgey. :scared:

Thorts anybody? :confused:

@tritical,

fZeroRead good observation.

Before StartGraph() make sense.

"A[CV]SC_USE_STDCALL" hmmm it's certainly confusing but changing it will change the entrypoint name it searches for. :(

ACSC_USE_STDCALL is defined, so AVSC_CC = "__stdcall"

AVSC_USE_STDCALL is not, so entrypoint is "avisynth_c_plugin_init"

Guess we need to survey the C pluggins and see whats being done.

Never just rains but it sure does pour. :mad:

IanB

[cweb]

Is using avisource safe in this version, if you don't do a Normalize?

[IanB]

It should be no different to previous version regarding the audio source statements, (I haven't worked on them, yet). As far as I can tell they have always been broken after accessing the end of the stream in some mode or other.

Normalize has been given a full rework, so now surely trips up the other latent bug. But now it will correctly normalize all formats and channels. Previously it only normalized 1/(number of channels) of the data, hence only tripped the bug for mono audio.

As a work around, run Normalize(show=true) and then edit your script to use Amplify(number displayed), this also has the benefit of not having to reparse the audio track every time you reload the script, it can take several minutes.

IanB

[Zarxrax]

Normalize has been given a full rework, so now surely trips up the other latent bug. But now it will correctly normalize all formats and channels. Previously it only normalized 1/(number of channels) of the data, hence only tripped the bug for mono audio.

IanB

Does this mean that for stereo tracks, normalize is silencing one channel and leaving only mono audio? Or does it leave both channels intact?
I ask because I'm currently working on a project intended for public viewing which involves nearly 400 video clips, each with their own avisynth script using the normalize function, so I can't really afford to be screwing up all the audio.

[IanB]

@Zarxrax,

No, on the sampling pass it only looked at 1/n samples, hence possibly/probably missed the highest peak. On the amplify pass it processed all the data in all the channels but possibly using the wrong factor. If you are interested have a look at the CVS diffs for audio.cpp

IanB

[Fizick]

return to cache_range question.
What i must set in plugin for pure spatial processing:
child->SetCacheHints(CACHE_RANGE, 0);
or
child->SetCacheHints(CACHE_RANGE, 1);
or
child->SetCacheHints(CACHE_NOTHING, 0);
or
nothing ?

and what for temporal procesing (use prev, cur, next frames):
child->SetCacheHints(CACHE_RANGE, 3); ?

[IanB]

@Fizick,

For pure spatial processing do nothing, the default is optimized for this case. Only diagnostic filters like the time measuring ones should ever use CACHE_NOTHING. Remember the case in scripts with 2+ filters attaching to 1 other filter.

The CACHE_RANGE value is currently a diameter to match the behaviour of previous AVS versions, so 3 is the right value. Around post #170 (http://forum.doom9.org/showthread.php?p=685179#post685179) tritical and myself briefly discussed the issue, it is still unresolved, please express your (and everybody else) thoughts on the matter.

IanB

[Fizick]

Of course, i read your discussion with Tritical.
O.K., here is my thoughts.
1. For pure spatial we must use child->SetCacheHints(CACHE_RANGE, 0);
Almost all old spatial filters use this settings.
2. Yes, i suspect of some old bug with cache.
Firstly I set CACHE_RANGE to radius (in old avisynth versions). But later i discover, that the speed of my temporal filters was decresead. So, I usually experimentally increase the range to increase the speed. So, now i understand, that i empirically use it as diameter. But some my filters probably use not optimal setting. Degrainmedian and DeSpot use range=2, because of range=1 result in slowdown (probably 3 will better for prev, currennt and next frame?) But in DePan I increased it some more (reserve). :)
IMHO, old avisynth (2.5.5) do not strictly consider cache range as diameter,
it was some mix (bugs).
So, It is really not easy issue.
Of course, i can recompile my plugins.
IMHO, the most important problem is not using new plugins in old avisynth versions,
but using old plugins in new avisynth.
However, if new cache strategy is more tolerant to wrong cache range setting,
it is not issue.
So, we must test new super-cache strategy.
But when I look to cache.cpp in CVS, i see that all cache_range code is disabled (commented), and some debug messages is there.
Is it the same code as in 2.5.6beta4 ?
4. sorry for my long and stange post.
Can we

[IanB]

1. For pure spatial we must use child->SetCacheHints(CACHE_RANGE, 0);
Almost all old spatial filters use this settings.You should not do this, this is very bad, particularly for the older AVS versions, the new cache is somewhat defensive about this but not completely immune.V1=AviSource("blah.avi")
V2=V1.SomeFilter() # This is a bad filter, it does (CACHE_RANGE, 0)
V3=V1.OtherFilter() # Because of the above all frames must be built
ThirdFilter(V2, V3) # at least 2 timesI have thought about catching crazy Hint settings, my initial cut is to just nail excessive values but I can see I should also defend against uselessly low values as well. For all AVS versions in general non-temporal filters should not play with the cache hinting. Doing nothing says I am a normal spacial domain only filter.2. Yes, i suspect of some old bug with cache.
Firstly I ... experimentally increase the range to increase the speed. So, now i understand, that i empirically use it as diameter. But some my filters probably use not optimal setting. Degrainmedian and DeSpot use range=2, because of range=1 result in slowdown (probably 3 will better for prev, current and next frame?) But in DePan I increased it some more (reserve). :) Practically it turns out to be Diameter. For the classic "Prev, Current and Next" case a range of 2 is appropriate if the filter askes for the frames in the right order, i.e. Prev (last current), Current (last next) and the Next (built a new, trashes prev). If you ask for Next 1st then you need a range of 3 and of course the user can always use the Reverse() filter which inverts the above argument, so you should probably use 3.So, It is really not easy issue.Amen! Keeping consistancy of interface make things like this very difficult ;) Of course, i can recompile my plugins.This generally won't help much, the problems are in the various versions of AviSynth.Dll, but fixes like range from 2 to 3 can't hurt.IMHO, the most important problem is not using new plugins in old avisynth versions,
but using old plugins in new avisynth.
However, if new cache strategy is more tolerant to wrong cache range setting,
it is not issue.Hence the ultimate decission to go with Diameter. This make all filters old and new as consistant as possible across all AVS versions.So, we must test new super-cache strategy. But when I look to cache.cpp in CVS, i see that all cache_range code is disabled (commented), and some debug messages is there. Is it the same code as in 2.5.6beta4 ?Testing, now there is a really good idea ;) and of course reporting. :D

The cache_range functionality (protected VFB's) is folded into the main code now, the old code, commented out, is for reference so as problems or short commings get reported I can compare easily old and new code. I will probably delete it in the final released version of 2.56.

IanB

[Fizick]

1. So, CACHE_RANGE=0 is really bad? But some time ago i saw recomendation use it.
2. I look to MipSmooth code. It use CACHE_RANGE=1 (it has temporal mode too).
3. I think, we must ask about cache other plugins writers too (Kassandro ? - He uses CACHE_NOTHING in Removegrain )

[IanB]

1. So, CACHE_RANGE=0 is really bad? But some time ago i saw recomendation use it.It was a complicated way to say CACHE_NOTHING and in debug builds it would cause a sanity assertion. I do not know of the reference, maybe they were making the recomendation for a very specific case. The new cache more or less treats this as default caching, it probably trips over a boundary condition and causes the current frame to be protected for 1 cycle.2. I look to MipSmooth code. It use CACHE_RANGE=1 (it has temporal mode too).So this would dodge the debug build sanity assertion. It did provide a 1 frame cache, almost but not quite CACHE_NOTHING. Not a really well thought out choice.3. I think, we must ask about cache other plugins writers too (Kassandro ? - He uses CACHE_NOTHING in Removegrain )It seems I need to protect more against silly values.

A point to consider, for a purely linear script, one with no temporal filters or tricky cross dependancies, the cache is absolutely useless. If no frame is ever fetched a second time there is nothing to cache. So for this case dicky cache setting won't matter. And as the cache does take a small number of cpu cycles to manage disabling it must yield a very small overall speed improvement.

But for temporal filters and scripts with cross dependancies i.e. uses stack*(), merge*(), weave(), layer(), overlay(), etc the gain from having an effective cache is immense. The key point for a temporal filter author is they know with certainty the access pattern of there filter and they can give the cache important information in the form of a hint i.e. "I am going to fetch the same frames 7 times, as a priority please keep my last 7 GetFrame results."

As for cross dependancies this is purely in the domain of the script jockey. A filter author cannot begin to guess the many and wonderous ways a script will build very complicated cross dependancies using their filter, so they should do nothing to inhibit the successful action of the cache.

This is the major advance with the new cache, the old cache just cached everything and hoped that any subsequent fetches happened before a VFB got reused. The new cache tracks repeated requests for frames and promotes only candidate frames, so only frames likely to be requested a second time get cached. Single use VFB's do not polute the cache. Also protected VFB's from a CACHE_RANGE request do not join the cache thus also do not polute the main cache but they do reduce the overall size of the main cache, so the range specified should be as accurate as possible. To small and you probably cycle thru all the protected frames before getting to use them, to big and you rob the main cache.

IanB

[Fizick]

I make some new test with custom versions of DegRainmedian (with different CACHE_RANGE). I use two or more DegrainMedian, with conbination with DeFreq, fft3dfilter, fluxsmooth, Reverse.
Here is results:
1. For Avisynth 2.5.5 I have big slowdown with CACHE_RANGE=0 or =1.
2. For Avisynth 2.5.5 CACHE_Range=2 works perfectly. There no any difference with CACHE_RANGE=3 or more.
3. For latest 2.5.6 (14 july) I have no any slowdown for ANY CACHE_RANGE (0,1,2,3)

What is the most indicative script for cache settings?
(V1, V2, V3 does not show nothing)

[IanB]

I make some new test with custom versions of DegRainmedian (with different CACHE_RANGE). I use two or more DegrainMedian, with conbination with DeFreq, fft3dfilter, fluxsmooth, Reverse.
Here is results:
1. For Avisynth 2.5.5 I have big slowdown with CACHE_RANGE=0 or =1.Classic example of cache cycling, each frame is destroyed just before it can be reused.2. For Avisynth 2.5.5 CACHE_Range=2 works perfectly. There no any difference with CACHE_RANGE=3 or more.Enough is enough, more is NOT better, just wastefull.3. For latest 2.5.6 (14 july) I have no any slowdown for ANY CACHE_RANGE (0,1,2,3)CACHE_RANGE no longer excludes the standard cache, if the hint is wrong it is no longer catastrophic. However you take your chances, if a given script places extreme demands on the cache then a misleading or wrong hint will bite you, to small and you get cache buffer cycling and purging, to large and you deplete the main cache for other filters.What is the most indicative script for cache settings?
(V1, V2, V3 does not show nothing)AviSource("blah.avi")
A=Trim(20, 0) # Adjust "20" to tune degree of cache abuse
B=Trim(10, 0) # optional extra cache abuse
FilterThatGivesCacheHint()
StackHorizontal(Last, A, B)This style of script exposes how weak the old cache was, even without a filter giving bad hints it is easy to tune for total devistation, particularly if "blah.avi" is a keyframe challenged file. The true test is to use SetMemoryMax() to see how low can you go before it collapses.

IanB

[Fizick]

IanB,
thanks for some clarification.

But I decided to add some info.
I check ALL plugins source availiable (all from warpenterprises site and my own collection).
Here is the FULL list of plugins that have SetCacheHints explicit call. (please add more if you know).

Colormatrix by Wilbert: (CACHE_RANGE,1) (if use hints from DGDecode)
Convolution3D by Vlad: (CACHE_RANGE, 2)
Defreq by Fizick: (CACHE_RANGE,0)
Defliker by Fizick: (CACHE_RANGE,cachecapacity), cachecapacity = lag*2+2
DeGrainMedian by Fizick: (CACHE_RANGE,2)
DePanEstimate: (CACHE_RANGE,range+1) range is radius
DePanInterleave: (CACHE_RANGE,num) num = prev + next + 1, Invoked Interleave: (CACHE_RANGE,num)
DePan: (CACHE_RANGE,cacherange) cacherange = 1+2*abs(int_offset)
Despot since v3.3.2 by Fizick: (CACHE_RANGE, 2)
GetDups by Fizick: (CACHE_RANGE,0) (it is wrong for temporal but works somehow ?!)
Fluxsmooth by SansGrip: (CACHE_RANGE, 1) for YV12
fft3dfilter by Fizick: spatial (CACHE_RANGE,0), temporal (CACHE_RANGE,2)
LeakKernelDeint by Leak: (CACHE_RANGE,2)
MipSmooth by shodan: (CACHE_RANGE,1)
Removedirt06 by Kassandro: some internal (cache ? CACHE_RANGE : CACHE_NOTHING, cache) cache=true or false
Removedirt09 by Kassandro: more complex internal using, some clips with (CACHE_RANGE, 0), some with user (?) defined (CACHE_RANGE, cache)
Removegrain06 by Kassandro: spatial (CACHE_NOTHING, 0), temporal (CACHE_RANGE, 2);
today Kassandro tell me, that recent Removegrain uses spatial (CACHE_RANGE, 0)
TFM by tritical: (CACHE_RANGE, 1)
TDecimate by tritical: (CACHE_RANGE, 1)
Vaguedenoiser since v.0.30 by Fizick: (CACHE_RANGE,0)

(There are also some plugins without sources.)
I see, that I am in good team :).
So, ALL plugins with explicit CACHE_RANGE used wrong settings?
May be this settings must be considered not as wrong but as quite correct?
I say about spatial (CACHE_RANGE,0) and temporal (CACHE_RANGE,2).

IMHO, Avisynth must detect plugin's CACHE_RANGE=0 special case and consider it as the mark of spatial only plugin.
Temporal plugins above used mostly CACHE_RANGE=1 or =2.
I do not see, that is more compatible with diameter (=3) than radius (=1).

Here is my crazy suggestion:
May be Avisynth must consider plugin's CACHE_RANGE value as a DISTANCE between boundary frames?
Consider 3 frames (100, 101, 102) to be cached. Distance between frame 100 and frame 102 is 102-100=2.
If diameter (number of frames) = 3, then distance= (diameter-1) , and diameter=distance+1.
So, Avisynth will use internal CACHE_diameter=CACHE_RANGE+1
For CACHE_RANGE=1 we will have diameter=2 (it is qute good)
For CACHE_RANGE=2 we will have diameter=3 (it is perfect).
For CACHE_RANGE=0 we will have diameter=1 or it may be considered as special case (converted to CACHE_ALL - not-protected VFB?).
So, we must not recomplile all plugins. Compatibility will be quite good.
Is it enough crazy idea? :)

[IanB]

Fizick,

That is pretty close to what I was already thinking, although I hadn't thought about the diameter=distance+1, this is good idea, particularly for small values. As you have found out the new cache doesn't slow down with bad values like the old one so it is no longer much of an issue.

Unfortunatly we can't fix it for previous versions, which is where it hurts the most. However for previous versions doing something like Crop(0,0,0,0).FilterWithBadCacheHints() should side step the problem if there ever is one.

IanB

[Wilbert]

Some throw error bugs:

1) Tweak. Feeding a RGB32 clip to Tweak, opening it in VDubMod, doesn't throw the correct error:

http://www.geocities.com/wilbertdijkhof/NoScriptEditor-tweak.jpg

Although i don't see anything strange in the code.

2) Tweak, SwapUV (and others). Feeding a RGB32 clip to filters which don't accept RGB, using the scripteditor in VDubMod, doesn't throw the correct error:

http://www.geocities.com/wilbertdijkhof/ScriptEditor.jpg

This happens with all exceptions, not only feeding it with the wrong color format.

I have a W2K SP4 if that matters anything (and latest CVS).

[IanB]

Damn missing exception table code bug. I'll have to do another pass thru all the asm listings, fortunatly I'm getting good at finding these. I'll try to find time this week, then we can start doing 2.5.6-RC1, the DirectShowSource rework looks ready to go.

IanB

Edit: I've searched all the .asm's and fix another half dozen SEH bugs. I just commit'd the source fixes to CVS. 2.5.6-RC1 here we go.

[Wilbert]

@IanB,

Bug (1) above is corrected, but (2) still doesn't show the correct error. Just open a correct script in VDubMod, open the script editor, mess up, F5 (refresh), and you will get the

Cannot open file "":
The system cannot find the path specified.

error, instead of the correct exception.

Of course it might be a problem in VDubMod, but it used to work.

[sh0dan]

I created patch for Temporalsoften to enable RGB32 mode, and loosen restriction on YUY2 to MOD4. All modes where already supported by the code, but I seem to have forgotten to enable them.

Not committing since this (maybe) isn't in time for 2.5.6.

http://zenaria.com/kpo/temporalsoften.patch

[IanB]

@Sh0dan,

Might as well commit it, it's just constructor parameter validation.

It will be a few evenings before I can find/fix bug (2) above.

IanB

[Wilbert]

Sh0dan, is it true that this scenechange setting is not implemented for RGB32?

[sh0dan]

@Wilbert: Not really. I did one for the conditional filters. It's basicly the same as a regular scenechange - it just masks out the last byte, so that garbage in the 'A' channel doesn't confuse the math.

It should be fairly straigtforward to implement it into temporalsoften.

[Wilbert]

Despite the small issues above, i had some time and made a new release:

AviSynth 2.5.6 RC 1 Release (September 4th):

Additions:
* RGB32 mode in TemporalSoften
* *Resize(), src_height and src_width when negative work as in crop.
* Added options to DirectShowSource (seekzero, timeout and pixel_type).
* Added AudioDubEx(), blindly accepts video and audio streams.
* Added Load_Stdcall_Plugin(), alias for LoadCPlugin() (won't disappear when avisynth_c.dll is loaded).

Bugfixes:
* Masked "Evaluate: System Exception - Access Violation" in :- FadeIO*(), RGBAdjust(), Tweak(), Lanczos*Resize() and GaussResize().
* Fixed rounding in YUY2 turnleft/right chroma.
* Fixed AVSC_USE_STDCALL declaration in avisynth_c.h (was ACSC_USE_STDCALL).
* Fixed BlankClip(clip) now competely duplicates the donor clip's VI including parity.
* Fixed AssumeTFF/BFF() to correctly update internal parity state.
* Fixed Animate audio switching.
* Fixed aligned UVpitch from width rounding.

Changes:
* The avisynth_c plugin entry point is now officially "avisynth_c_plugin_init@4" (don't include @4 anywhere the compiler does it for you), this is not actually a change due to a bug in avisynth_c.h, which incidently caused it to be this already.
* Info() now autoselects a smaller font to fit info in small frames.
* Info() now distinguishes between "assumed" field parity and field parity.
* Animate now selects parity through the filter chain.

As usual download from Sourceforge (http://sourceforge.net/projects/avisynth2/).

[Wilbert]

@Wilbert: Not really. I did one for the conditional filters. It's basicly the same as a regular scenechange - it just masks out the last byte, so that garbage in the 'A' channel doesn't confuse the math.

It should be fairly straigtforward to implement it into temporalsoften.
Ok, i wasn't sure you already implemented it, because you added RGB32 mode :)

[tritical]

Was the "fZeroRead" not being reset in VC_audio.cpp problem ever fixed? I couldn't find any mention of it in cvs. All it seems to need is for fzeroread to be set to false in AudioStreamSource::Seek if it is currently set to true and samples < end_samp.

[IanB]

Opps, No, cracks, slipage thru thereof.

IanB

Edit: 12th Sep, Done! Into CVS soon(ish).

[Mug Funky]

i seem to be getting a 1 gig barrier with "2vuy" files and avisynth. to clarify, i load the avi into virtualdub and it shows 2975 frames. i load the same avi through avisynth and i get 1535 frames (that's right on the 1 gig threshold for uncompressed yuy2). this happens even if i change the fourcc of the file, or use the fourCC option in avisource (i force to uyvy, as that's the format the files are in).

also, by default loading these files as uyvy seems to return YV12 with progressively downsampled chroma. forcing yuy2 b0rks the luma. right now i'm loading like this:
y=AVISource("F:\GZ - son of godzilla cap test\GZ.avi",fourcc="uyvy",pixel_type="yv12").converttoyuy2()
uv=AVISource("F:\GZ - son of godzilla cap test\GZ.avi",fourcc="uyvy",pixel_type="yuy2")

mergechroma(y,uv).crop(0,0,720,480)

the crop is in there because the original file is 720x486, which is not mod4 and therefore doesn't behave nicely.

providing samples might be a problem as this only seems to affect 2vuy files that are over 1 gig... i've done heaps of DV stuff with avisource and there's been no problem. sodding blackmagic-design haven't got back to me about their crappy tarted-up uncompressed codecs that only return RGB24 while storing 4:2:2, so it looks like reverse-engineering is the only solution until i get a PC with a decklink card in it.

[IanB]

Mug Funky,

Ah, I've always wanted to use Avery's sparse avi file tool and you present the perfect opportunity.

In VirtualDub under tool->Create sparseAVI, Select source AVI File -> {one of your files}, Select sparse AVI file -> {some good name}. You now have the structure of your AVI container without any content, it should be quite small, stick it somewhere I can get it.

Now for some content, use VirtualDub to chop a few frames off the front (say <1Mb) of the same file you sparsed above, also stick it somewhere I can get it.

And just for good measure decode 1 frame (with useful colour picture content and from the segment chopped above) and present it as a lossless compressed image (bmp/gif/tiff NOT jpeg) somewhere I can get it.

Also what, where, how, why, which, .... about codec and application producing the AVI files.

IanB

[Mug Funky]

hmm. sparse avi file, huh? sounds interesting. if you're around melbourne i could just hand a DVD to you though... i'll give that sparse trick a shot when i'm less busy :)

on the provenance of these avi files. i'm pretty sure they're not completely compatible avi files. here's how they were made:

- a blackmagic quicktime file was captured on a mac through a decklink card. 8-bit uyvy uncompressed, with the fourCC "2vuy". quicktime doesn't offer much in the way of capturing formats, and i don't want to risk dropped frames just yet.

- the mov was then transmuxed into an avi with ffmpeg like so:

ffmpeg -i xxxx.mov -vcodec copy -an xxxx.avi

this copies the video stream as is, and ditches the audio (it's 4 mono channels... i'll figure this one out later :)).

- the avi is then loaded into avisynth as uyvy rather than 2vuy, because the stock codec for this only returns RGB24 (this is the stupidest thing i've ever seen). see my last post for the exact loading script.

thanks for your help :)

[Wilbert]

Did you try RawSource?

[Mug Funky]

hehe... 'twas the first thing i tried. had a problem with frame headers, so the image would shift by 4 pixels every frame. i'm sure it'd be trivial to correct. i was really impressed by the speed though, compared to ye olde QTreader.

what would be really good would be a set of splitter filters for avisynth - so there'd be no need for directshow or avisource so long as we're dealing with uncompressed frames. however, i think something like that would be so close to "libavsource" that it'd probably be easier to implement that...

for now i'm having to stick to RGB24 input and convertbacktoyuy2().

[JasonFly]

Not very technical question here but I've noticed that avisynth french doc is far from being complete. Waht is the procedure if I want to help translating it?

[Wilbert]

@Mug Funky,

Please give WarpEnterprises a short sample. I'm sure he will look at it.

@JasonFly,

The last person who added stuff to the french docs is jernst, but that's already a year ago or so. I will send you a pm at the end of this week.

[Wilbert]

I would like to ask you to put all your regular encoding stuff aside, and start testing this build :) If no bugs are found this will version be released as v2.56 stable (say next weekend).

AviSynth 2.5.6 RC 2 Release (October 7th):

Additions:
* SSE3 capable CPU detection in env->GetCPUFlags and Info().

Bugfixes:
* Fixed memory leaks in Overlay and AVSChar/AVSTime.
* Fixed End_of_Stream reset on seek in AudioStreamSource::
* Fixed SegmentedDirectShowSource() argument parsing.
* Fixed *Resize(), src_height and src_width when negative work correctly.
* Fixed minor memory leak in env.VSprintf(), [ul]case() and *str() also remove 4k limits, thanks Tritical.
* Fixed Normalize scribling into memory for float samples.

Optimizations:
* Performance improvents in transfer functions in TCPDeliver.
* Normalize() for 16 bit stop when a max-int value sample is seen.

Changes:
* Stop extra search of LIBC, add relsym build - Release with Symbols.

As usual download from Sourceforge (http://sourceforge.net/projects/avisynth2/).

[IanB]

@Wilbert,

Which French documentation set did you build with?

IanB

[fccHandler]

I have always had some problems encoding .avs video with Windows Media Encoder 9. For a long time I've blamed it on WME9, but recently I've come to suspect Avisynth. The problem is that the "wmeenc.exe" process doesn't shut down properly. The Encoder program closes, but an instance of "wmeenc.exe" remains in Task Manager hogging a LOT of memory. If I then run another encoding session of WME9 with an Avisynth file, and finish the session, yet another instance of "wmeenc.exe" lingers in Task Manager. These will continue to accumulate, and strangely (perhaps after 3 or 4 such sessions) WME9 starts to misbehave, and its output .wmv files become increasingly corrupt.

I started to suspect Avisynth last week, because I tried changing my strategy to use only AVI files as input to WME9, and the problem went away. It's a big inconvenience to have to do that though. Tonight I updated my Avisynth to version 2.5.6 RC2 and tested it again in WME9, but still "wmeenc.exe" is not shutting down cleanly for some reason.

Another weird thing is that WME9 won't show the progress indicator, or estimate the encoding duration when the input is an Avisynth script.

I'll be grateful if you guys can look into these issues before you go final. Let me know if there's anything I can do to help.

[tritical]

Not sure if this is currently being worked on or not, and this is only MHO, but I think the seh/c++ exception handling mix in readhelper that will cause a crash in xp sp2 should be addressed before a 2.5.6 final is released. A large number of avisynth users will be using avisynth with vdub in xp sp2, in which case the hard termination is worse then simply disabling the seh code... which would at least allow c++ exceptions that are thrown (for example a filter that calls env->ThrowError() within getframe()) to deliever the correct error msg. Of course, this would sacrifice breaking out system exceptions on other systems where the seh/c++ mix doesn't cause problems. From what I have read it seems like there are four options:

1.) Disable seh code
2.) Replace seh with manual exception handling
3.) Switch Avisynth to static linking with the runtime library
4.) Get microsoft to fix the problem in xp sp2's msvcrt.dll

Going with option 2 would keep the ability to determine the system errors and wouldn't be difficult to do... I actually have it working here on my comp. Anyways, I think it would be a good idea to have it working with xp sp2 before a final 2.5.6 release.

[Wilbert]

Which French documentation set did you build with?
The latest (some pages were translated recently). Why do you ask?

[Fizick]

tritical,
option 3) is linking with libcmt.lib ?
Why not? Almost all my plugins linked so way.

[squid_80]

http://www.virtualdub.org/oldnews

Scroll nearly all the way to the bottom to the entry dated 1/14/2003 and Avery discusses dynamic vs static linking.

[IanB]

@fccHandler,

Got any hints where to start looking?

@tritical,

> 4.) Get microsoft to fix the problem in xp sp2's msvcrt.dll
Good luck, they won't even admit there is a problem. What really urks is the bug effect every version back to the year dot.

@Wilbert,

Coz "some pages were translated recently" :D

@Fizick, squid_80,

Never just simply rains, it has to really pour.

[fccHandler]

@fccHandler,

Got any hints where to start looking?
I'm afraid not, as I'm not familiar with the code. If I get time I'll download it and look at it though. I would run it in a debugger and watch what methods WME9 is using to connect and disconnect through Avisynth. It seems to me that some object (thread, handle, dll, etc.) is not being released for whatever reason, and prevents a clean shutdown.

Again, I don't know whether to blame WME9 or Avisynth, but the fact that WME9 works OK with real AVI files suggests to me that something is faulty in Avisynth's emulation.

[IanB]

@fccHandler,

Okay, having butchered a machine to make it run, how do I get it to do *.AVS files. I get that more than useless com error 0x80004002 interface unsupported. Sh0dan has the same problem.

IanB

ASF = Advanced Streaming Format, NOT!

[fccHandler]

In "Properties / Sources," choose to source from "Both device and file," then you can browse for and open an .avs file as your video source.

BTW, thank you very much for looking into this.

[IanB]

Okay, now how do I get to see the problem? It seems to work okay, it starts, it stops, it shutsdown, it makes horrid wmv files. Can I get audio somehow?

[fccHandler]

Certainly. Select an audio file as your Audio source.

You get "horrid" wmv files? I don't know how to respond to that... Perhaps you are simply attacking the quality of WME9, but IMHO that's not relevant (or welcome) here.

I thought I presented my issues clearly in my earlier post, but here they are again:

The problem is that the "wmeenc.exe" process doesn't shut down properly. The Encoder program closes, but an instance of "wmeenc.exe" remains in Task Manager hogging a LOT of memory. If I then run another encoding session of WME9 with an Avisynth file, and finish the session, yet another instance of "wmeenc.exe" lingers in Task Manager. These will continue to accumulate, and strangely (perhaps after 3 or 4 such sessions) WME9 starts to misbehave, and its output .wmv files become increasingly corrupt.

Another weird thing is that WME9 won't show the progress indicator, or estimate the encoding duration when the input is an Avisynth script.
Please tell me if you are able to reproduce these issues. If not, I'll just have to assume that it's something peculiar to my system.

[IanB]

Certainly. Select an audio file as your Audio source.Okay, no AVS audio :(

You get "horrid" wmv files? I don't know how to respond to that... Perhaps you are simply attacking the quality of WME9, but IMHO that's not relevant (or welcome) here.Don't mind me, I'm still smarting after my recent runnin with DirectShowSource.cpp and ASF/WMV. So yes I can generate WMV files, and they have expected content.

I thought I presented my issues clearly in my earlier post, but here they are again: ...

Please tell me if you are able to reproduce these issues. If not, I'll just have to assume that it's something peculiar to my system.No I can't reproduce the issues. But I only tried 2 scripts "ColorBars()" and "AviSource("Fred.avi")" fred is 30 secs, huffyuv compression. So I am assuming some level of complexity is involved, either in the AVS script and WME9 configuration. :script:

IanB

[sh0dan]

Well - in my test, I didn't get it to open AVS files, but it still didn't shut down properly, and I had to kill it in the Task manager.

[fccHandler]

Okay, no AVS audio :(
Nope. But in my case I'm always working with separate video and audio sources anyway.

I am assuming some level of complexity is involved, either in the AVS script and WME9 configuration.
Not really. I often use .avs simply to trim commercials from TV captures. My procedure is to open my Huffyuv TV capture in VirtualDub, locate the trim positions I want and write a very basic script like:

AVISource("CAPTURE.avi")
Trim(a,b) + Trim(c,d) + Trim(e,f)

Now I open the .avs in VirtualDub and save its trimmed audio as a .wav, so I can balance and normalize the audio in Audition. This gives me the .avs video and .wav audio sources I'll input into WME9.

As for WME9 setup, I usually do this:

Start WME9...
Click the "properties" button
In the "Sources" tab:
- Source from "Both device and file"
- Video = browse for .avs file
- Audio = browse for .wav file
In the "Output" tab:
- Clear the "Pull from encoder" checkbox
- Check the "Archive to file" checkbox
- Select output .wmv filename
In the "Compression" tab:
- Destination = File archive
- Video = High quality video (VBR 95)
- Audio = High quality audio (VBR 75)
- If necessary, push "Edit" to tweak the presets
Finally, click the "Start encoding" button.

That's about it.

To be fair, I'll admit that WME9 can be cantankerous and ill-behaved. Mine doesn't crash, but it sure doesn't like Avisynth. It works fine with real AVI files, but that requires an extra step (and extra HDD space) to perform what should be a very simple task. Were you able to determine (in a debugger) what methods it uses to open the .avs?

[sh0dan]

"gRefCnt" does reach zero, and "CAVIFileSynth::~CAVIFileSynth()" is called, and WMVEncoder is allowed to unload the DLL, but it never calls "DllCanUnloadNow" to check.

[fccHandler]

That's a little deep for me. I was thinking at a higher level, like, in what way are .avs files handled differently than real AVI files?

It's been years since I last tried to compile Avisynth, and I didn't have much success back then. I'm willing to try again though (maybe tonight).

[Boulder]

To be fair, I'll admit that WME9 can be cantankerous and ill-behaved. Mine doesn't crash, but it sure doesn't like Avisynth. It works fine with real AVI files, but that requires an extra step (and extra HDD space) to perform what should be a very simple task.
Slightly OT, but does a dummy AVI file created by AVS2AVI work?

[fccHandler]

Well I don't know what AVS2AVI is, and I'm not willing to install additional software to make this work. I installed WME9 and Avisynth; that should be enough. I believe it should "just work" out of the box.

I jumped a few hoops to compile Avisynth, but my compiled version doesn't work, and I don't understand the code well enough to make it work. I'm at the mercy of the Avisynth developers...

If you can reproduce and fix my issues with WME9 + Avisynth, then that will be awesome. Otherwise, just forget about it. I haven't found any references anywhere of anyone else reporting these issues. I can work around it by rebooting between every WME9 encoding session. (Aaargh...)

Anyway, thank you very much for your efforts.

[Leak]

Well I don't know what AVS2AVI is,
A stripped-down version of IIRC VirtualDubMod with only a command-line interface that takes an AVS file as input and produces a video-only AVI file as the output

and I'm not willing to install additional software to make this work.
I guess he meant that you should try if it works when you use a plain AVI file instead of an AVS file...

[Wilbert]

I take it Bould means makeAVIS:
http://cvs.sourceforge.net/viewcvs.py/avisynth2/avisynth/distrib/ffvfwAVIS.exe

[Boulder]

I take it Bould means makeAVIS:
http://cvs.sourceforge.net/viewcvs.py/avisynth2/avisynth/distrib/ffvfwAVIS.exe
Exactly, I remembered that it was AVS2AVI, but the wrapper I meant was makeAVIS.

[Pookie]

How about just downloading 1 file as a workaround until the issue is truly resolved ?

http://www.sysinternals.com/Files/PsKill.zip - command line process killer

Then, in a batch file:


;BEGIN
; Encode Audio from Command Line
"C:\Program Files\Windows Media Components\Encoder\wmenc.exe" /start "%userprofile%\desktop\audio.wme"
; Ping localhost in order to delay script for a few seconds
Ping 127.0.0.1
; Kill wmenc if it is still loaded
pskill wmenc
; Encode Video from Command Line
"C:\Program Files\Windows Media Components\Encoder\wmenc.exe" /start "%userprofile%\desktop\video.wme"
Ping 127.0.0.1
pskill wmenc
;END

[fccHandler]

I guess he meant that you should try if it works when you use a plain AVI file instead of an AVS file...
Yes, AVIs work perfectly. Only .avs files cause problems.

@Pookie:
I did try manually terminating the leftover processes between sessions, but the problem remains that the .wmv output files become increasingly corrupt after several WME9 + Avisynth sessions. It's a very strange thing.

[Wilbert]

Yes, AVIs work perfectly. Only .avs files cause problems.
Boulder mentioned makeAVIS which create fake avis, just like VFAPI. Please try that.