AviSynth 2.5.5 CVS Releases. [August 26th]

[sh0dan]

@You Know: It's the same bug as described above. RC2 will probably be out soon.

[You Know]

problem descript from Dreassica??
ah OK

NB other function as AvarangeLuma ecc ecc works

[IanB]

Originally posted by sh0dan
@IanB: Seems like there is still some optimizations bugs left. :(

I have do disable: "Global Optimizations" and "Full Optimization", otherwise we get a crash, in code very similar to what you experienced. I can however turn on "Frame-Pointer Omission" without experiencing problems. Your reaction?

I was concerned that "Frame-Pointer Omission" was just shuffling the deck and sending the bug to ground, looks like it's true. :( I really have only classified the bug so far, not found the antidote yet.

Disabling "Global Optimizations" and "Full Optimization" is just about the whole lot, but this may be what we have to do. Perhaps we #pragma the few places (non-mmx getframe/getaudio) that optimizing is really important and hand check that code.

One further issue that may have been overlooked is that any routine on the call stack, with stack based objects created (most typically AVSValue's or {I,P}Clip's), at the time the exception is thrown is a candidate for this problem. This includes third party filters.

Until we find the recipe that 100% triggers the compiler to emit dud code. The only way I can think of to be sure is run with no optimatization (like you suggest) or hand check every routines entry setup code against the SEH destructor calls at the end, not a pleasant sounding task.

IanB :(

[sh0dan]

August 20th Release Candidate 2:


- Fixed "Unknown exception" in conditional filter expressions. (August 17th regression)
- Updated docs.

[LigH]

Usually, one assumes: Why using Crop and *Resize, if resizer functions may crop while resizing as well? In fact, this shall even be much more efficient than running one after each other (like DVD2SVCD vs. GordianKnot).

Well, unfortunately, there is a difference: Cropping wihle resizing doubles the bottom image line, while cropping and resizing is clean - using AviSynth from 29th July 2004 as well as the mentioned new RC2.

Please see examples in this thread in the German doom9/Gleitz forum (http://forum.gleitz.info/showthread.php?t=15459) (unfortunately, downloading binaries from my server became forbidden, so I only can attach files to threads now).

[You Know]

bug or limitation?

i have notice that (with beta 2) if i open larger avi 9/12 Gb after 5100th it freeze and repeat frame 5100 eternally

Have AVIFileSource a limit of 4GB?

EDIT: installed RC2 same problem with larger avi
recode 50% on anime movie :(((((((((

[esby]

Could you put more details ?
and the filterchain used?

esby

[You Know]

a1 = AVIFileSource("E:\part1.avi")
return a1


part1 file was 12.404.xxx KB (uncompress YV12 source)


a1 = AVIFileSource("F:\part3.avi")
return a1


part3 file was 9.404.000 KB (uncompress YV12 source)


a1 = AVIFileSource("C:\part2.avi")
return a1


part2 file was 12.3??.xxx KB (uncompress YV12 source)

with file of only 2GB it works :(

[esby]

mmmmm

I don't want to vex you, but if you are using segments of more than 2 or 4 gb,
you are bound to get problems (depending the os you are using).
(that s why vdub and other programs got the multi segmented avisource function)

(and in that case check multisegmented source loading function)

esby

[You Know]

Originally posted by esby
you are bound to get problems (depending the os you are using).

if you have FAT32 limit are 2^32 = 4gb file size with NTFS no.
C++ function can get and seek using a long but handicap can be bypass using int origin parameter ... i think no made example for now


Originally posted by esby
that s why vdub and other programs got the multi segmented avisource function[/B]

yes but i did not known there was this limitation. I open Part1.avi and append segment part2 and part3 with virtualdubmod and i can seek where i want in file and no size limitation !;)

Now i rencoding all (2fps script on 127343 frame lenght) using segment to 2048 but avisynth could support very large file as vdm can.

[You Know]

mfToon 0.52 script does not work with RC2. I think to rest on beta 2 for ever.
bye

[esby]

es but i did not known there was this limitation. I open Part1.avi and append segment part2 and part3 with virtualdubmod and i can seek where i want in file and no size limitation !

Now i rencoding all (2fps script on 127343 frame lenght) using segment to 2048 but avisynth could support very large file as vdm can.


just open the 3 files with vdubmod, set direct stream for video & audio
and use the save by segments feature, no?

esby

Are there other known issues of xp SP2 & avisynth btw?

[You Know]

Originally posted by esby
just open the 3 files with vdubmod, set direct stream for video & audio
and use the save by segments feature, no?

no comment ... maybe i have not enought space to segmet avi, if you can see part file are all on different partition.

Originally posted by esby
Are there other known issues of xp SP2 & avisynth btw?

my PC never seen xp, windows 2000 rulez

[LigH]

If you want to visualise the differences between "Resize incl. crop" and "Crop + resize", you may want to check this kind of "template":
MPEG2Source("DVDSource.d2v")
Clip1=Lanczos4Resize(512,208,0,72,720,440)
Clip2=Crop(0,72,720,440).Lanczos4Resize(512,208)
Subtract(Clip1,Clip2).Levels(96,1,160,0,255)

[sh0dan]

@LigH: Thanks - I'll look at it.

Edit: Fixed.

[sh0dan]

General update:

It seems like the new optimization settings break plugins. I've encountered problems with the "AviSynth Subtitler" plugin. Re-enabling optimizations FIXES the problem.

So for 2.5.5 I think we should keep the "old" optimization settings. Any thoughts?

[esby]

... optimization ... breaking plugins
Could it be the same types of problems that mug-funky is having with my last dsynth build?

esby

[LigH]

Fixed? Fine!
__

Another bug reported by the mentioned german user:
V1=AVISource("8kHz_Mono_CBRMP3.avi",audio=true,pixel_type="YV12")
\.Crop(14,2,424,332)
\.BilinearResize(336,240)
\.Blockbuster(method="noise",detail_min=1,detail_max=100,mean=0,variance=10,cache=2048)
\.AddBorders(8,0,8,0)
\.AssumeFPS(29.970,sync_audio=true)
\.SSRC(48000,fast=false)
A=AudioDub(V1,MergeChannels(GetChannel(V1,1),GetChannel(V1,1))
He expects the two channels to be identical; instead he reports changing sync/async and differences between both channels of a supposed dual-mono => stereo audio stream. Shall he try EnsureVBRMP3Sync (http://www.avisynth.org/EnsureVBRMP3Sync) although he has a CBR stream? Or is the sampling rate too low for SSRC? Or is SSRC really optimized for not-too-far downsampling? ResampleAudio(48000) instead shall work well.

[sh0dan]

He should definately try "EnsureVBRMP3Sync" first. This is in general recommended for ACM-compressed audio. I guess the audio cache helps resampleaudio here.

SSRC shouldn't have any problems with ratechanges this size.

[Wilbert]

After AssumeFPS(sync_audio=true) your audio has some weird samplerate.
I might be totally wrong, but I thought SSRC doesn't do "arbitrary" samplerate conversions.

from ssrc.c:

frqgcd = gcd(sfrq,dfrq);

fs1 = sfrq / frqgcd * dfrq;

if (fs1/dfrq == 1) osf = 1;
else if (fs1/dfrq % 2 == 0) osf = 2;
else if (fs1/dfrq % 3 == 0) osf = 3;
else {
fprintf(stderr,"Resampling from %dHz to %dHz is not supported.\n",sfrq,dfrq);
...
}


What you should do is

AVISource("8kHz_Mono_CBRMP3.avi",audio=true,pixel_type="YV12")
EnsureVBRMP3Sync # for audio cache
AssumeFPS(29.970,sync_audio=true)
AssumeSampleRate(8000)
SSRC(48000,fast=false)
...

[IanB]

Originally posted by sh0dan
General update:

It seems like the new optimization settings break plugins. I've encountered problems with the "AviSynth Subtitler" plugin. Re-enabling optimizations FIXES the problem.

So for 2.5.5 I think we should keep the "old" optimization settings. Any thoughts? While the compiler is definitly forgetting to emit some code at times, I am starting to doubt it is the optimiser causing it. Changing the /O... settings just seems to shuffles the deck, making the bug appear and hide apparently randomly.

Below is a CVS history extract of the "Release" cpp control strings.1.1 ! # ADD CPP /n.../GX /Zi /O2 /Ob2 /D..."AVISYNTH_EXPORTS" ...
1.6 ! # ADD CPP /n.../GX /Zi /O2 /Ob2 /D..."AVISYNTH_C_EXPORTS"...
1.13! # ADD CPP /n.../GX /O2 /Ob2 /D..."AVISYNTH_C_EXPORTS"...
1.17! # ADD CPP /n.../GX /Zi /O2 /Ob2 /D..."AVISYNTH_C_EXPORTS"...
1.18! # ADD CPP /n.../GX /Zd /O2 /Ob2 /D..."AVISYNTH_C_EXPORTS"...
1.19! # ADD CPP /n.../GX /Zd /O2 /Oy- /Ob2 /D..."AVISYNTH_C_EXPORTS"...

1.13! # SUBTRACT CPP /Z<none>
1.14-

/Zi Debug Program Database
/Zd Line numbers only
/Oy- Optimize Frame Pointer omission (don't!)

1.1 27 Nov 2003 restructured
1.6 08 Jan 2004 change /D "AVISYNTH_EXPORTS" --> /D "AVISYNTH_C_EXPORTS"
1.13 28 Jan 2004 delete /Zi
add subtract /Z
1.14 22 Feb 2004 delete subtract /Z
1.17 17 Jun 2004 add /Zi
1.18 05 Jul 2004 change /Zi --> /Zd
1.19 12 Aug 2004 add /Oy-The only effective optimiser change was my /Oy- suggestion. The other changes, "AVISYNTH_C_EXPORTS" in January and the various /Zi /Zd debug info changes I can't see having an effect. I am starting to look for other systematic changes made.

IanB :(

[Fizick]

... break plugins.
Yes, it is true for DeSpot.

[sh0dan]

Originally posted by IanB
[...]The only effective optimiser change was my /Oy- suggestion.
.. But that one broke the conditional filters (August 17th binary).

I think we'll have to live with the Resizer/ColorYUV crashing instead of reporting the correct error. At least until 2.55 is out the door.

[esby]

btw, since we are about compiler & performance setting,
what would be the performance impact to compile with rtti enabled actually?

esby

[DDogg]

Not complaining, but just wanted to mention that the speed of encoding the simple test script below is slower using AviSynth_200804 than with 2.54. Using 2.55 gives 1.763 RT and 2.54 shows 1.994. I'll try to do a comparison using a real source when time allows.

#start of CCESpeedTest01.avs
ColorBars (720,480)
ShowSMPTE(24)
converttoyuy2()
#end

[sh0dan]

@DDog: As you might have noticed we are struggling with optimizer errors. RC2 had most optimizations disabled, but broke plugins. So we are choosing the "lesser of two evils" in RC3 optimizations are reenabled.

RC3 released - nothing major:
- Reenabled optimizations. RC2 broke plugins.
- Fixed duplicated line in resize+crop operations (beta regression)


Test, test and test! :)

[LigH]

Another bug report for RC2 by user 'doxville' (he just registered here and is not yet able to post, so he is afraid that this bug is not seen before "release"):
AVISource("D:\capture.avi")
converttoYV12()
FadeIO(25)
deen()
Using deen_25_dll_20030119:

Without "FadeIO(25)", VirtualDub and MPC immediately crash without any message.

Using deen_25_dll_20030813:

Without "FadeIO(25)", an error message (http://forum.gleitz.info/attachment.php?attachmentid=68300) appears.

We know that Deen is ... well, a bit sensible. But at least, without parameters, it shall have been useable. More details, if you understand german, in this thread (http://forum.gleitz.info/showthread.php?t=15585).
__

Oh, sh0dan posted RC3 just while I was writing this - be sure he will check this issue with RC3 soon!

[sh0dan]

@LigH: Simultanious post ;)

Try RC3 - it might have a better impact in Deen. :)

[IanB]

@sh0dan,

I hesitate to suggest such a ham fisted solution but it may get us out of the bind. :sly:

As part of tracking down what was happening I encased everything intry {
...<stuff>...
}
catch (...) {
throw;
}so I had markers to breakpoint while debugging the optimised (scrambled) release code. It is effectivly a no-op but it did cause the compiler to emit the missing code (which of course defeated my debugging attempt :mad: )

There are of course no guarantees, I have a tame zoo of only 2 reproducable cases and it hides the bug in both. What do you think?

IanB :(

[DDogg]

As you might have noticed we are struggling with optimizer errors. Yep, knew that, but I would rather err toward redundant/useless info rather than not providing something that might have helped provide a clue.

Anyway, the RC3 build brought the speed back equivalent to 2.54 as I gather you expected. Thanks.

[FuPP]

Not sure this is the right place to post this but I experience a crash (in virtualdub for instance) with these lines. Is this a normal behaviour, is this a syntax problem, or does it show something going wrong ?

LoadPlugin("D:\video\avsfilters\yv12\DGdecode.dll")
mpeg2source("E:\tests\test.d2v",idct=6)
converttorgb24()
bilinearresize(576,432)
trim(30000,30100)

global debut=0

test()

function test(clip c)
{
debut=debut+1
return debut==10 ? c : test(c)
}

[You Know]

RC3
- Fixed duplicated line in resize+crop operations (beta regression)


beta 2 and RC2 function Resize

YV12 space

XXXResize(700, 398)

very dangerous color layer shift from Luma Layer see HERE (wwwstud.dsi.unive.it/~nfalco/doom9/YV12cs.jpg)

XXXResize(702, 398)
XXXResize(706, 398)

ecc ecc (not all value) -> Envaluare: Unrecognized error

[Wilbert]

@Shodan,

I removed the MS pics again (i can't put it in a table and put text next to it). They might be usefull though if we add it to sourceforge.

I got a few questions:

1) How does YV12 progressive/interlaced upsampling to YUY2 work?

2) How does YUY2 upsampling to 4:4:4 YUV work? Are the missing chroma samples (of the even pixels) averaged from the odd ones next it?

Or is it using this cubic convolution interpolation for (1) and (2), as in

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnwmt/html/YUVFormats.asp

3) What does FixBrokenChromaUpsampling exactly do (which samples is it replacing)?

Totally unrelated:

4) Did you already finished this filter:

http://forum.doom9.org/showthread.php?s=&threadid=81191

It would be nice if it is finished before the v2.55 release :)

[Manao]

You know : the chroma shifting is not created by avisynth, but by XviD's YV12 decoder. The bug was fixed in XviD beta 2 or 3, but came back with the XviD RCs.

[You Know]

Originally posted by Manao
You know : the chroma shifting is not created by avisynth, but by XviD's YV12 decoder. The bug was fixed in XviD beta 2 or 3, but came back with the XviD RCs.

i do not understand why you speak about xvid (i have 1.0.1) but img is taken after load avs script from virtual dub mod (are not take from compressed video)

[Manao]

Virtual dub is using XviD decoder to understand how to process YV12 raw data. Without it, you can't use an avisynth script that is outputting YV12 with virtual dub.

Do the following test to convince you that this decoder is really the culprit : Do a resize to 700x398, load it in virtual dub : it doesn't work ( chroma shift ). Now, do the same resize, and add a "Addborders(4,0,0,0)" at the end of the script : no chroma shifting. So inside avisynth, the result of the resize is correct.

[sh0dan]

Originally posted by Wilbert
1) How does YV12 progressive/interlaced upsampling to YUY2 work?

YUY2 to RGB is sampled by interpolating the missing chroma horizontally. (50/50 on each side of the missing pixel)

RGB to YUY2 has two modes. Default mode is using a 1-2-1 kernel to interpolate chroma. There is also ConvertBackToYUY2 which only uses chroma in the leftmost pixel - this can be used to avoid a half-pixel chroma drift to the left on multiple conversions.

2) How does YUY2 upsampling to 4:4:4 YUV work? Are the missing chroma samples (of the even pixels) averaged from the odd ones next it?
Where? The only place we have 4:4:4 is inside Overlay().


3) What does FixBrokenChromaUpsampling exactly do (which samples is it replacing)?

It is simply swapping the chroma samples to their correct lines.


4) Did you already finished this filter:

No - I have had no time to look at it at all.

[Wilbert]

YUY2 to RGB is sampled by interpolating the missing chroma horizontally. (50/50 on each side of the missing pixel)

RGB to YUY2 has two modes. Default mode is using a 1-2-1 kernel to interpolate chroma. There is also ConvertBackToYUY2 which only uses chroma in the leftmost pixel - this can be used to avoid a half-pixel chroma drift to the left on multiple conversions.
Ok, I understand that.

I'm still confused about YV12->YUY2 and FixBrokenChromaUpsampling though.

If I understand it correctly:

YV12 (progressive) -> YUY2:
* odd chroma lines are just copied.
* even chroma lines (2n) are the average of the lines 2n-1 and 2n+1.

YV12 (interlaced) -> YUY2:
* chroma lines (n) are the average of the lines n-2 and n+2 (alternating a 75-25 average).

How does FixBrokenChromaUpsampling attempt to correct this (ie interlaced YV12 upsampled progressively) by just swapping chroma samples? I must be missing something obvious :confused:

No - I have had no time to look at it at all.
I will make an unoptimized version myself :)

[tritical]

YV12 (progressive) -> YUY2:
* Uses alternating 75/25 25/75 averages of nearest chroma samples

YV12 (interlaced) -> YUY2:
* Is also using alternating 75/25 25/75 averages, but of the nearest chroma samples from the correct field

The above weights are from the mpeg2 specs (based off mpeg2 chroma placement)... I'm guessing yv12(interlaced)->yuy2 is using 75/25 for speed in isse instead of using 7/8,1/8 3/8,5/8 averages, which are what the samplings would suggest?

[IanB]

Originally posted by Wilbert
How does FixBrokenChromaUpsampling attempt to correct this (ie interlaced YV12 upsampled progressively) by just swapping chroma samples? I must be missing something obvious :confused:

"The MS DV codec upsamples the chroma channels incorrectly", this filter repairs that fault. Nothing specificaly to do with other upsampling errors.

For each group of 4 lines it swaps the chroma of the middle 2.1 --> 1
2 \/> 3
3 /\> 2
4 --> 4
5 --> 5
6 \/> 7
7 /\> 6
8 --> 8IanB

[Mug Funky]

hmm... ImageSequence.dll is crashing VDM. i hear of this happening quite a lot, but this is the first time i've been hit with the problem. is there a painfully simple solution (apart from moving this file out of the plugins directory?)

[esby]

Am I wrong, or was imagesequence (or an equivalent plugin) put in the core already?
-> imageReader() and imageWriter()?

esby

[Mug Funky]

that's what i'd thought... maybe that dll has been in my plugins directory for a very long time now...

[Fizick]

BTW, there are 3 different Avisynth.h files in released 2.55 source.
What is the best? :)
The last?

[IanB]

Originally posted by Fizick
BTW, there are 3 different Avisynth.h files in released 2.55 source.I assume you mean the following 3 :-

./avisynth/src/core/avisynth.h
./avisynth/src/plugins/DirectShowSource/avisynth.h
./avisynth/src/plugins/TCPDeliver/avisynth.h

The 1st is the one used to build the current avisynth.dll, so should be used to build new plugins (i.e. ones that want 2.55 exclusive features).

However if you desire backwards compatibility (say for 2.53 or 2.54) you should use the avisynth.h published at that time.

The 2 included with the plugins above are the previous version for that reason.

See http://cvs.sourceforge.net/viewcvs.py/avisynth2/avisynth/src/core/avisynth.h for the CVS history.

IanB

[Fizick]

I.e. if I will use avisynth.h from 2.55 source for my plugin build,
it will not work in Avisynth 2.54 ? :confused:

[IanB]

Originally posted by Fizick
I.e. if I will use avisynth.h from 2.55 source for my plugin build,
it will not work in Avisynth 2.54 ? :confused: Not quite. If you use features only available from 2.55 then expect unpredictable behaviour (crashes) in earlier versions.

By using the older avisynth.h the compiler protects you from yourself, if the features not defined then you cannot use it.

This is why changing avisynth.h requires so much thought and care. It is to maintain compatibility.

IanB

[Kintaro]

blur() is causing instant crashes on yv12 clips with width >=16

[IanB]

Your script please.

Edit: Don't bother, I take you mean width <= 16.

Problem is MMX code needs mod 8 pitch for loops.

Chroma width = 0.5 x Luma width so causes pitch <= 8 and hence crashes.

Workaround :- AddBorders and Crop

I'll fix the code and CVS it.

IanB

[esby]

To answer a bit more about "features only available from 2.55 then expect unpredictable behaviour (crashes) in earlier versions."

This is not totally unpredictable, but very predictable on some situation from what I tested while playing with dsynth.

The main problem about using new method, is that if you call them on an instance of an object that does not have the method, you'll got a crash, via a pure virtual function call (call to a method that does not exist).

For example dsynth redefines IClip and add some method.
for example CanDirectStreamVideo().
While there is no problem when calling internal filters with it, since they'll be defined with it, external filter, not based on dsynth will generate crash (pvfc).

Now the reverse approach, if you build a filter using the dsynth avisynth.h,
and if you try to load it with the normal avisynth, it will not load, since the version returned is different.
Supposing that you hack back the version reply, and have it loaded,
if a normal method of this filter is called, the execution will be carried out, as long these methods do not try to call any new methods on a child object. And if such child are normal object, you'll get pvfc again.

Now, if you call method that have been changed...
depending the way they were declared, the old version might be called (let's say a bugged one), and the new version may be called in some other case too. Which may result in unpredictable cases. Almost I am not myself sure of it, I will probably retest it tomorrow.

So, as IanB said, if your filter does not uses new function or method, it will probably load and work correctly, in the reverse case, expect non correct result as best, and crashes in most case.

esby

PS: Of course, you can say 'i will detect differences, and not call non existing method if the object are not the same type, but here the problem is that you don't have rtti enabled, so you can't really check the type back. But I believe you may detect it in a way or an other, if you know what to check for, and thus, avoid pvfc or erratic comportment on some specific part.