New filter: Autolevels

[T. Anschütz]

Hi,
some time ago I needed an autogain/autolevels filter and was unsatisfied with what ColorYUV had to offer, so I wrote my own. The main improvement over ColorYUV(autogain=true) is that the amount of gain is averaged over a number of frames which helps with scenes that have flashing lights etc.
This filter was useful for a VHS transfer I had to do recently. Maybe somebody else will find it useful, too.
The parameters are explained on the web page and in the readme file.

URL: http://ta41.zendurl.com/autolevels_en.php

Comments are always welcome, of course.

[IanB]

:( Any source code ?

Yes snapping the gain on a frame by frame basis, as ColorYUV does, sucks!

[TheRyuu]

How does it compare to HDRAGC?
(not at home otherwise I would try it)

I'll have to test this one out since I got a few bad sources that could use some gain.

[T. Anschütz]

:( Any source code ?

I re-uploaded the zip file with the source code. This is my first c++ program, so apologies if the code is awkward to read. Also, I haven't translated the comments, so they are all in German.

As for HDRAGC, I tried it out but it didn't seem suited well for my purposes. I think it uses a more complex algorithm, where my plugin basically just stretches the luma histogram and applies some averaging, but it was exactly what I needed.

[IanB]

Thanks for the source code.

As your filter make very heavy use of the cache you may wish to use the NewVideoFrame model for this filter instead of the MakeWriteable model for your output frame. This will avoid possibly needing to rerender the middle frame on subsequent calls.

Leak has brought to my attention some weaknesses in my algorithm for MakeWriteable VFB protection. It may be better not to depend on avisynth internals to defend against extreme cases of cache usage.

[T. Anschütz]

Thanks for the source code.

As your filter make very heavy use of the cache you may wish to use the NewVideoFrame model for this filter instead of the MakeWriteable model for your output frame. This will avoid possibly needing to rerender the middle frame on subsequent calls.

Leak has brought to my attention some weaknesses in my algorithm for MakeWriteable VFB protection. It may be better not to depend on avisynth internals to defend against extreme cases of cache usage.
Sorry for the late reply... I finally got around to looking at my code again. I noticed I was calling child->GetFrame() even for frames whose min / max / isSceneChange values were already known, which is of course silly, so I fixed that. It should call child->GetFrame() only once per frame now, at least for consecutive frames.

As for creating a new frame rather than modifying the original frame object, I see your point, but I'm assuming this isn't an issue any more, now that I'm doing my own caching (of the frame parameters), so I'm not calling GetFrame n times for each frame any more.
But maybe I'm wrong and using NewVideoFrame would still make things more efficient. If so, let me know.

Version 0.3 of the filter is up on my website.

[IanB]

boolean Autolevels::isSceneStart(int frameno, IScriptEnvironment* env) {
...
if (frameno > 1) {
PVideoFrame frame = child->GetFrame(frameno, env);
PVideoFrame prevframe = child->GetFrame(frameno-1, env);frame and prevframe are still fetched but are no longer used? :confused:
IntPair Autolevels::getMinMax(int frameno, IScriptEnvironment* env) {
PVideoFrame frame = child->GetFrame(frameno, env);
const unsigned char* pY = frame->GetReadPtr(PLANAR_Y);
...
hash_map<int, IntPair>::const_iterator mmIter = MinMaxCache.find(frameno);You still fetch the frames before checking you MinMaxCache, you probably only need to fetch it if there is no cache entry.

If you adjust your code to only fetch the "new" frame for the MinMaxCache and the "current" frame for outputing then the MakeWriteable model will be appropriate.

Also check on the libraries implementation of hash_map you may need to manually clear them in your destructor code.

[T. Anschütz]

You are absolutely right, I don't know what I was thinking. Thanks for spotting this!
I uploaded the fixed version, it is still v0.3, same filename.

With the unnecessary GetFrame calls are gone, the filter is a lot faster :)

I tried calling the hash_map destructor in the Autolevels destructor, but that crashed VDub, so I left it out for now. I need to actually read the documentation when I have some time.

[IanB]

Yes, it should be very much faster ;)

The short coming with some hash_map implementations is that it forgets the memory allocated to elements when it destructs.

As a workaround you may need to do a MinMaxCache.clear(), etc, somewhere in your code, Autolevels::~Autolevels might be a good place.

[rfmmars]

This is by far the best "autolevels" filter. It does get fooled on rare occasions in changing low light conditions, snapping quickly up and down a couples of levels. What you might do is to have it fade up or down slowly in low light so not to be noticed.

Excellent filter,

Richard
photorecall.net

[RickA]

Very nice. Will have to give this a try later on. Also, always good to see another Anschutz about. ;-)

Cheers,
Rick

[T. Anschütz]

Very nice. Will have to give this a try later on. Also, always good to see another Anschutz about. ;-)

Cheers,
Rick

http://forum.stuttgarter-zeitung.de/board/images/smiles/icon_beer.gif

[T. Anschütz]

This is by far the best "autolevels" filter. It does get fooled on rare occasions in changing low light conditions, snapping quickly up and down a couples of levels. What you might do is to have it fade up or down slowly in low light so not to be noticed.

Excellent filter,

Richard
photorecall.net
Thank you, I'm glad you like it.
What you are seeing is the filter "detecting" a scene change when there is none. To get rid of this, you can set sceneChgThresh to a higher value. At sceneChgThresh >= 256, scene change detection is effectively disabled and the "snapping" effect should be completely gone.

Of course, with scene change detection turned off, you may get undesirable effects at actual scene changes where the luma values change abruptly from one frame to the next. In these cases you don't want any fading to occur. So just try different values for sceneChgThresh to find out what works best in your situation.

Possibly, maybe, you will find that fades around scene changes aren't a problem and that sceneChgThresh = 256 is the way to go for you.

If you are a perfectionist, you can also deal with the false positives / false negatives by using the frameOverrides parameter, manually overriding the frames it falsely detects as scene changes or when it fails to detect one.

Even better would be a more intelligent scene change algorithm. Right now it looks at the maximum and minimum luma values of consecutive frames.
If someone knows a better algorithm that is not too complicated to implement, let me know.

[plane]

The link seems dead?

[T. Anschütz]

You are right, ZendURL is having problems. This link works for me:

http://www.zendurl.com/t/ta41/autolevels_en.php

[thymej]

Sorry thanks

[superuser]

thanks for the filter, will try it out.

[superuser]

thanks T. Anschütz for the filter. works gr8. If possible can you spin out a version say simple levels and provide more control to the user by input paramaters for the filters such as:

- gain to be achieved
- control for color control, if any, in luma and chroma plains (sorry this too much but thought may be will ask :) )

even in its native form filter is cool. thnxs again.

[AlanHK]

You are right, ZendURL is having problems. This link works for me:

http://www.zendurl.com/t/ta41/autolevels_en.php

I can't get this filter to work.

If I try to load the dll
LoadPlugin("p:\autolevels.dll")
I get an error message with some gibberish, as below.

I'm using Avisynth 2.57 on Win2k.

Also I note that the web page mentioned above has some problems, as the angle brackets used to indicate frame ranges are interpreted as HTML and thus invisible.

[unskinnyboy]

Does anyone have a working version of the latest version of Autolevels? If so, can they upload it somewhere, please? The previous URL is dead, and this don't seem to be hosted anywhere else.

[Chainmax]

I second that request, I'd like to try this filter out on some digicam footage and compare it to HDRAGC.

[NormanBates]

I found Autolevels v0.3 on my hdd and uploaded it for you.
Here is the link:
http://www.sendspace.com/file/7i6yx0

[unskinnyboy]

I found Autolevels v0.3 on my hdd and uploaded it for you.
Here is the link:
http://www.sendspace.com/file/7i6yx0
:thanks: This does seem the latest version!

[AlanHK]

Does anyone have a working version of the latest version of Autolevels? If so, can they upload it somewhere, please? The previous URL is dead, and this don't seem to be hosted anywhere else.

Attached.

But as you see above, I didn't have much luck with it.

[unskinnyboy]

AlanHK, Try the version NormanBates linked above. It works fine for me. Maybe yours is a corrupted version?

[AlanHK]

AlanHK, Try the version NormanBates linked above. It works fine for me. Maybe yours is a corrupted version?
They're identical.

[unskinnyboy]

They're identical.OK. Your attachment was unapproved (still is) when I posted my earlier reply, so I wasn't sure if it was. Not sure what your issue is then, sorry. Sure is specific to just you though. Works for me on my current Vista machine, and it used to work on my previous XP machine too (before it crashed a few months ago, that is).

[chriszxl]

links did not work............

[NormanBates]

Again:-)
I found Autolevels v0.3 on my hdd and uploaded it for you.
Here two links:
http://www.sendspace.com/file/z16kmr
http://rapidshare.com/files/135978459/autolevels0.3.zip.html

[Sharktooth]

direct download: http://www.webalice.it/f.corriga/misc/autolevels0.3.zip

[dreamy83]

URL is dead can some1 upload autolevels again
thanks

[Nightshiver]

No it's not. Use the direct download link from Sharky, it still works.

[AlanHK]

The file attached to my post above (http://forum.doom9.org/showthread.php?p=1136676#post1136676) is okay.

[IYG]

What is this filter's advantage?

[carlmart]

I would like to know what are the parameters to load on autolevels. The link is dead.

Any suggestions?

[sidewinder711]

Here you go:

http://www.mediafire.com/?ywjkgzmclgy

Cheers!

[EugenSU]

Something wrong occurs to colours after the filter. The picture is constantly painted in any colour. Has submitted on an input a black-and-white picture. On an exit has received floating displacement of colours. U and V components constantly float. I work with normal YUY2.

[frustum]

The wiki

http://avisynth.org/mediawiki/External_filters

says that autolevels works on YV12, YUY2, RGB32, RGB24 format pixels.

Certainly it works on YV12. If I convertToRGB24() before autolevels, it still works, but the effect is small. If I convertToRGB32() first I don't see any difference from the unprocessed video.

But my biggest problem is that ConvertToYUY2().autolevels() produces a very saturated purple video (it might be different on different clips, but this is what I'm seeing).

Is the wiki simply wrong?

I've worked around it with this:

(orig is in YUY2 format)
tmp = orig.convertToYV12().autolevels().convertToYUY2()
orig = MergeLuma(orig,tmp)

but if there is a more direct way to operate on YUY2() without the two conversions, I'd like to know.

[IanB]

Ouch! Yes this filter really only has a code path for YV12, although it should give correct results for RGB24 and near correct results for RGB32. It will be using the min(a, r, g, b) and the max(a, r, g, b) for its determinations, so the contents of the A channel can upset things. Unfortunately it will just make a mess of YUY2 chroma.

Given the filter just ploughs through RowSize bytes of data per line, you might get away with this :-(orig is in YUY2 format)
tmp = orig.GreyScale().autolevels() # make all chroma bytes 128
orig = MergeLuma(orig, tmp) # Overlay new luma bytes

[frustum]

Ian, your suggestion works well enough for the one scene I've tried it on. Thanks for the tip. Also thanks for the work you are doing on avisynth!

I've just had a look at the source for the autolevels plugin. I know nothing about avisynth data structures and internals, but it assumes the pixel data is planar. Won't it just treat y,u,v as three luma values? So if the image is very dark, it will think there are a bunch of y=128 pixels after performing the GrayScale() conversion, won't it?

Overall, it seems wrong to say autolevels() supports anything but YV12 (except in special cases) right now.

[Wilbert]

Did someone drop a mail/pm to the author about this issue?

[frustum]

Wilbert, the source code gives his name (Theodor Anschütz), but no contact info. When I tried to go to his website listed on the wiki for autolevels, it was a dead link.

[sinz718]

Awesome filter worked great for my current project, thanks!

[frustum]

I sent a pm to the author, but his profile says he joined doom9 in order to post his filter, then last posted a few months later, in the fall of 2007, so I suspect he won't notice it.

I've had a look at the code and it is something I could pick up and generalize, and perhaps improve it in other ways. It would be a good way for me to get my feet wet. However, the whole reason I started using avisynth was in order to transfer and restore about 35 8mm films to DVD in time for Christmas, as a present to my siblings. I may not have time to work on the plugin until that task is done.

However, I will mention a few weaknesses (other than blindly assuming YV12 format pixels) I see in the code, in order of difficulty.

(1) The average luma is computed by looking +/- N frames from the current frame (settable as a parameter). It starts its average from either frame -N or the most recent scene change before the current frame. It ends its average either at frame +N or at the frame before the last scene change before frame +N. This is all perfectly good unless there is more than one scene change detected after the current frame. This is cured by adding one "break;" after noticing a scene change after the current position, and is a very minor point.

(2) Once the rolling time averaged min/max luma for the current frame is computed, the luma distribution is shifted to map to 16 .. 235. This is desirable for some purposes, other times mapping to 0..255 is desirable. Perhaps this should be a parameter.

(3) The scene detection logic is very simple minded, although it probably works most of the time. The "min" and "max" luma for the frame are computed by ignoring roughly the outer 1% of the brightest/darkest pixels, so outliers don't have undue influence. If either the min or max for frame N has changed by more than a threshold value as compared to the previous frame, it is considered to be a new frame. This is probably pretty easy to produce false positives as well as false negatives.

Just from thought experiments, no actual data gathering, I think this can be improved. Using changes in the min/max is OK, but the threshold needs to be higher to avoid false positives, at the expense of false negatives. We can add other tests to help fix this.

Each frame in the averaging window has already exhaustively (and expensively) measured the luma histogram, but only the min/max are cached for the frame. It would be better save the whole histogram as well (only 256 ints). In addition to detecting whether the min/max changed significantly, with the histogram we could add tests for whether the mean (or median) has changed, and whether the autocorrelation between the histograms of successive frames has changed much. Further, rather than just considering whether frames n-1 and n correlate, look at the correlation for each successive pair. If there is a sudden change in the correlation it would be less prone to false positives. That is, even if the correlation changes a lot between frames n-1 and n, it is meaningful only if the correlation has been somewhat stable either before or after frame n. This will avoid a situation I've seen where in the middle of an ordinary scene the luma flashes rapidly up and down a few times.

[frustum]

Well, I couldn't resist, and I downloaded msvc++ 2010 express and set up virtualdub and avisynth on a virtual xp machine.

I now have a 0.4alpha version of autolevels. It fixes a couple small bugs, is probably measurably faster, supports yuy2, planar yuv, and interleaved rgb24/32 surfaces.

Great. The problem is that I take this dll which works fine in the virtual 32b xp machine and bring it to my real windows 7 64b box. It is running a 32b version of avisynth and 32b version of virtualdub too. When I try to load my newly minted autolevels, I get this bland error:
Avisynth open failure:
LoadPlugin: unable to load
"<path to my dll>\autolevels_0.4a.dll"
(<path to my invoking script>, line 5)

Am I overlooking something obvious, or is this where things start to get difficult?

[kemuri-_9]

you probably linked the dll in a fashion that requires the visual studio 2010 runtime to be installed on any computer you want to use the dll on.

[frustum]

kemuri-_9 -- you seem to be right. In the project properties for C/C+ code generation, I used to have "Runtime Library: Multi-threaded DLL (/MD)". Changing that to "Multi-threaded (/MT)" now allows my dll to run on my native machine.

However, the dll size went from 32 KB (similar to the size of the existing autolevels.dll) to a whopping 138 KB. Debugging info is disabled and optimization is enabled, as this is a release build.

I tried comparing my .vcxproj file to a 2006-era .vcproj file (from AddGrain) to find the difference, but despite being related, I can't correlate many of the options.

Are there any visual studio wizards that can spot the option which isn't set right? Meanwhile, I'll keep fiddling...

code generation command line:

/Zi /nologo /W3 /WX- /O2 /Oy- /D "WIN32" /D "NDEBUG" /D "_WINDOWS" /D "_USRDLL" /D "AUTOLEVELS_EXPORTS" /D "_WINDLL" /Gm- /EHsc /MT /GS /fp:precise /Zc:wchar_t /Zc:forScope /Fp"Release\autolevels.pch" /Fa"Release\" /Fo"Release\" /Fd"Release\vc100.pdb" /Gd /analyze- /errorReport:queue

linker command line:

/OUT:"C:\jim\video\autolevels\Release\autolevels.dll" /INCREMENTAL:NO /NOLOGO /DLL "kernel32.lib" /MANIFEST /ManifestFile:"Release\autolevels.dll.intermediate.manifest" /ALLOWISOLATION /MANIFESTUAC:"level='asInvoker' uiAccess='false'" /PDB:"C:\jim\video\autolevels\Release\autolevels.pdb" /SUBSYSTEM:WINDOWS /OPT:REF /OPT:ICF /PGD:"C:\jim\video\autolevels\Release\autolevels.pgd" /TLBID:1 /DYNAMICBASE /NXCOMPAT /MACHINE:X86 /ERRORREPORT:QUEUE

[kemuri-_9]

what are you asking a question about?

the size increase with statically linking in the MSVC runtime into the DLL is normal.
the original size is small because it requires the use of an environment located MSVC runtime.

personally though, a ~100KB increase from statically linking in the MSVC runtime is extremely small.

[frustum]

kemuri-_9 -- that is reassuring, but I have looked at a bunch of the other plugins on warpenterprises. some come with source code, fewer still contain their msvc project files. It seems that people do usually specify /MT (multithreaded) instead of /MD (multithreaded DLL), yet all of the dlls weigh in under 40KB. In the grand scheme of things an extra 100KB isn't a whole lot these days, but it still seems to indicate I have some setting wrong.

[cretindesalpes]

It depends on which parts of the runtime library you are using. As kemuri said, no need to worry about these few extra kilobytes. Older MSVC versions may also produce lighter binaries given the same settings.