Image Sequence I/O for the core

[Richard Berg]

Just returned home so if you have suggestions on syntax, I'm listening :) Anyway, this feature is far from done, but it's about time I started some documentation. Right now the arguments & defaults are:

ImageWriter(clip c, file = "c:\", type = "png", compression = [varies based on image type])

It then turns the frame number into a six-digit representation and appends it to the given filename. Thus, the output with default parameters would be c:\000000.png, c:\000001.png, etc.

Accepted values for "type" are png, jpeg, and bmp. There's still a crashbug in the first two so use BMPs if you want to play around right now. The BMP writer accepts any packed format (YUY2, RGB24, RGB32); JPEG requires RGB24 with YV12 possibly coming later; PNG requires RGB32.

[Richard Berg]

I should mention: I find the writer syntax decent enough -- let me know if you disagree! -- but am still looking for ideas on the reader. Notably:

I want it to interface with the image libraries similarly (image->decompress() in the GetFrame body), but need to get the image info (dimensions, colorspace, etc.) somehow before creating the destination buffer
I want to support both single images (for creative use with layer/mask/etc.) as well as sequences, even though the filter will always be queried for a given frame #


Perhaps WarpEnterprises can share his experience with the imagesource plugin...

[sh0dan]

Regarding BMP's - YV12 can also be considered a "packed format". Just like the AVI-frames, it assumes 4byte pitch alignment for Y, and half the alignment for UV.

You can see how BMP-like YV12 frames are handled in main.cpp ("ReadFrame(void* lpBuffer, int n)" - around line 721).

[N_F]

Would I be correct if I assume this writes out every frame as a single picture?

[WarpEnterprises]

with the mods I made we were able to use ImageSequence for our real-world application (reading steady shot images made from a quality control camera once a second, named with 'seconds afer midnight', some images missing.)

The best solution I could imagine for finding the image size and format is to scan for the first image found (and maybe stopping after 1000 unsuccessful attempts - scanning is not infinitly fast). This can be done in the construction function.

The only thing which I could not make:
return NO FRAME when there is no image instead of returning a BLACK FRAME.

Do we have some set of really different BMP, JPG or do you know a programm which can create such rarities?
I'm thinking of 32bit, CMYK, 8bit palette.. to perform some torture tests.

[Richard Berg]

Ok, I am back from travel once again and want to finish work on these filters. Sorry to have left PNG/JPEG in a non-working state while I was gone...hopefully I can find the cause of the crashbug now, though I'm always open to having another pair of eyes help out.

For the source filter, any objection to using "sprintf syntax" from ImageSequence? I think there would also be a parameter to indicate a single still frame, which I'd treat like StaticImage in source.cpp.

[Wilbert]

about ImageWriter:

It would be nice if you could add a parameter to ImageWriter to indicate which frames you want to save as bmp files (and that you save all the frames by default).

[Richard Berg]

You mean, something like this?


# writes the image if 100 <= frame <= 200, otherwise does nothing
ImageWriter(start=100, end=200)


That's certainly possible.

[Richard Berg]

Ok, the above syntax is implemented. Default for both is 0 (where end=0 means no upper limit).

PS the documentation is a bit wrong. BMPs will be written in whatever the source colorspace is, not just RGB24 (though planar formats aren't implemented yet). RGB24 & RGB32 files should be readable by any app; YUY2 and YV12 will probably only be readable by Avisynth.

[Richard Berg]

Note: while starting to add planar support I found a bug in the BMP engine that occurred with non-mod-4 images, so check the latest CVS before complaining of this :)

[sh0dan]

Just saw:
infoHeader.biPlanes = vi.IsPlanar() ? 3 : 1;

I don't know if it means anything, but IIRC YV12 is still marked as 1 plane in AVI files. Could however just be lazyness from the codec writers. ;)

[Richard Berg]

I think marking YV12 as such is necessary for the ImageReader to know it's YV12. (Although, it may be possible to infer from the image dimensions + bfSize...) In any case, I doubt any "standard" BMP readers will be able to parse any of the YUV-space formats anyway.

[Richard Berg]

I think I've found the crashbug in PNG/JPEG: the libraries aren't thread-safe! Since we compile Avisynth as a multithreaded DLL, the multithread versions of <iostream> are used -- usually a good thing, but it's causing those libraries to (in the disassembly) CALL non-code addresses.

I'll investigate using the DevIL library linked here (http://forum.doom9.org/showthread.php?s=&threadid=26855&pagenumber=2).

[Wilbert]

# writes the image if 100 <= frame <= 200, otherwise does nothing
ImageWriter(start=100, end=200)
Ok, thanks!

PS the documentation is a bit wrong. (...)
I will change that as soon as Sh0dan puts a new binary up.

[sh0dan]

There will be a new binary up soon - however the image part still isn't very useful as long as there are these bugs.

[Richard Berg]

I've begun work with the DevIL library and deleted the libjpeg/png stuff. Can someone please verify that the latest CVS code compiles correctly?

[sh0dan]

You need to include the libraries for the linker, to avoid having to reference them globally in options.

(They need to be put into Project Settings/Linker/General)

[Richard Berg]

Aha - I did it for Debug but not Release. Should work now.

BTW, BMP works for YV12 now, though no apps I know of will read it. I'm considering changing the option to use my writer to "ebmp" (extended) or similar so that DevIL BMPs can be supported too (if only for RGB).

[Richard Berg]

Minus any objections, I've changed my BMP writer to only be activated when the requested extension is "ebmp". All Avisynth formats are supported; RGB formats should be readable by any image apps. This is the default.

The supported values for 'format' are now: bmp, dds, ebmp (see above), jpg/jpe/jpeg, pal, pcx, png, pbm/pgm/ppm, raw, sgi/bw/rgb/rgba, tga, tif/tiff. RGB input is required.

There is currently a bug in the library (I think!) that corrupts the output of compressed formats (jpeg, png, etc.; bmp, tga, etc. work fine) when fed RGB32 -- stick with RGB24 for best results until I hear back from the DevIL team. Unfortunately the internal Avisynth conversions to/from RGB24 still appear to be buggy; use ConvertToRGB32.ConvertToRGB24.

Edit: forgot to mention, the 'compression' parameter will be dropped. It was useful when talking directly to libjpeg/libpng, but this option is not exposed by the new library.

Edit2: One more thing -- when using the DevIL routines, the default behavior is to throw an error rather than overwrite existing files. I'm going to change this if possible, but FYI.

[sh0dan]

I made some changes in the YV12 <-> RGB, that I recently committed - is this where you are experiencing problems, or???

[Richard Berg]

@shodan --

Using the latest CVS code, these conversions fail:

YV12 -> RGB24 (image corruption)
YUY2 -> RGB24 (image corruption)
RGB24 -> YV12 (access violation)
RGB24 -> YUY2 (access violation)

Based on the apparent bugginess of the library's handling of RGB32 I'd prefer to simply restrict input to RGB24, however I don't think it would serve the community very well to insist they use conversions that don't work ;)

@all -- the DevIL failure to overwrite files like the EBMP writer does has been "fixed" by simply deleting the existing files if any. Let me know if this is undesireable behavior.

[Richard Berg]

My first shot at an ImageReader is up. Syntax borrows heavily from WarpEnterprises' plugin.

[sh0dan]

I'll look at it as soon as I get a chance - unfortunately my home computer has crashed (badly), so I have to wait until I get to work monday.

Sorry - I though I had tested it properly. I couldn't test it that much, since I couldn't compile after the update.

[Richard Berg]

Sorry to hear about your computer sh0dan. I know RGB24 is a pain, but it seems to be what the rest of the world prefers :rolleyes: Let me know if there are further troubles compiling.

[Richard Berg]

Ok, I've had a chance to test out ImageReader more thoroughly, so time to write some documentation :)

Syntax:

ImageReader(file = "c:\", start = 0, end = 1000, fps = 24, use_devil = false)

file: name of the image file(s), where frame #'s can be specified using "sprintf syntax (http://www.cplusplus.com/ref/cstdio/sprintf.html)". For example, the files written by ImageWriter's default parameters can be referenced with "c:\%06d.ebmp".

start, end: if a frame not in this range is requested, a black frame is returned and no file I/O is done. 'end=0' does NOT mean 'no upper bound' as with ImageWriter. The first file (i.e., corresponding to 'file' after 'start' is substituted in) MUST exist in order for image parameters to be read in correctly.

fps: frames per second of returned clip. Since it only takes integers, I may add a fps_denominator term if that's deemed necessary to match clips.

use_devil: when false, the first file is attempted to be parsed as an EBMP, then defaulting to DevIL upon failure. When true, execution skips directly to DevIL processing. You should only need to use this if you have BMP files you don't want read by ImageReader's internal (E)BMP parser.

The resulting video clip is RGB24 if DevIL is used, otherwise is whatever the EBMP was written as (all Avisynth formats supported).

Known bug: compressed image formats are returned upside-down by DevIL. Dunno how to fix without scanning the filename for known-buggy extensions and inverting appropriately; I've filed a bug report with DevIL.

[sh0dan]

I reproduced (and hopefully fixed) the YUY2 -> RGB24 error. YV12 uses YV12 -> YUY2 -> RGB24 to achieve the best possible conversion, so it was also affected. I tried with a bunch of resolutions, but please test on.

I have not been able to reproduce the access violation - could you tell me exactly where it crashes (in Debug mode), and your script?

New binary is up so you can test the version. I hope I put in all the .dll's needed!

[Richard Berg]

Whatever you did, all conversions work now. Thanks -- I can force RGB24 input now :devil:

I can't really tell which DLLs you're packaging, since the installer doesn't say :) In any case, the only DLL really needed is devil.dll. Libjpeg & libpng are definitely not used anymore. ilu & ilut are only in /distrib because that's where I installed DevIL to -- I don't see myself linking against them anytime soon.

[sh0dan]

ok - I was including oth ilu and ilut, just in case. You can see which files are included in avisynth.nsi, which is the installer script.

You are sure, that devil.dll isn't referencing either of these two?

In general I tend to do RGB24 support, if it isn't a major recode, so actually most functions still support it. RGB24 -> YUY2 is still unoptimized C-code - even though a modification of the current assembler shouldn't be that big a deal. I'll see what I can do!

[sh0dan]

- Just updated conversion with an RGB24 -> YUY2 MMX routine. However looking through the old RGB32 -> YUY2 code I realize that it could probably be made faster if it was rewritten again. Maybe some day, when I'm in the mood. ;)

Anyway - I tested on a fair amount of resolutions again, but it is possible there is some sneaky bug hiding somewhere.

[Richard Berg]

You are sure, that devil.dll isn't referencing either of these two?
I'm sure -- before running your installer I didn't have them in \winnt\system32 on my system. Dependencies do work the other way however: if we ever wanted to use the image manipulations in ilu, devil.dll would be required; the functions in ilut require all three.

[Richard Berg]

About time I ran some benchmarks. A few surprises, but all in all I'm very pleased.

Straight copy: 40 fps
Devil writer, BMP: 21 fps
EBMP writer, YV12: 19 fps
EBMP writer, YUY2: 16 fps
Devil writer, TIFF: 16 fps
Devil writer, JPEG: 15 fps
EBMP writer, RGB24: 12 fps
Devil writer, PNG: 3 fps

Devil reader, BMP: 33 fps
Devil reader, JPEG: 32 fps
EBMP reader, YV12: 28 fps
Devil reader, PNG: 24 fps
EBMP reader, YUY2: 23 fps
Devil reader, TIFF: 17 fps
EBMP reader, RGB24: 13 fps

[Blankman]

ImageWriter definitely appeals as useful. But for speed, I currently assemble a sequence of BMPs into an uncompressed AVI using VirtualdubMod.

But creating an AVI sequence is not my final goal. It's just an intermediate step to get the data stream into AviSynth.

I normally preview the video stream, spot bad frames, copy bad frames to BMP files, correct BMP files, assembles BMPs into an Patch AVI, source both video and patch AVI files into AviSynth, replace bad video frames with patch frames.

Currently I emulating this process with scripting, but eventually I would like to create a filter to load images with a file mask such as "patch\([0-9]+\).bmp" into the appropriate clip frame "\1".

[Richard Berg]

Can you give me a better idea of what syntax additions would be helpful? I'm not sure I understand that 'patch' string correctly.

If you need to import assorted ranges of files, perhaps the Conditional filter would be helpful?

[sh0dan]

Not easy, but doable:

function get_filename(int n) {
ret_val=""
ret_val = (n<10000) ? ret_val+"0" : ret_val
ret_val = (n<1000) ? ret_val+"0" : ret_val
ret_val = (n<100) ? ret_val+"0" : ret_val
ret_val = (n<10) ? ret_val+"0" : ret_val
ret_val = ret_val + String(n)
file = "c:\temp\2\patch"+ ret_val + ".bmp"
return file
}

ImageReader(file = "c:\temp\2\test%05d.bmp", start = 0, end = 235, fps = 24, use_devil = false)
#scriptclip("subtitle(filename)")
scriptclip("return Exist(filename) ? patched : last")
FrameEvaluate("patched = ImageReader(file = filename , start = 0, end = 0, fps = 24, use_devil = false).loop()")
FrameEvaluate("filename = get_filename(current_frame)")

Original files are here called:
c:\temp\2\test00000.bmp
c:\temp\2\test00001.bmp

Patch files are named:

c:\temp\2\patch00045.bmp
c:\temp\2\patch00320.bmp

There are two tricky things in this: Constructing the filename manually (the upper function).

And the fact that variables are assigned "backwards" - that means that variables used early in the script is defined later in the script.