Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 5th August 2012, 08:42   #1  |  Link
turbine
Registered User
 
Join Date: Jul 2012
Posts: 7
ImageSource(), ImageWriter(), x264-compressed still photos, avisynth improvements

YES... FINALLY I can post! The five-day waiting period is NOT a very kind policy, and definitely one that turns away contributors. I'd suggest a better solution would be requiring moderation during that period. Having said that, I appreciate this forum and its contributors. With a bit of trial and error the posts have been helpful and making sense of avisynth's impressive capabilities but cumbersome scripting format and design flaws.

I've been experimenting with using x264 to encode libraries of still images with impressive results. I'd like to share my results so far so that they may help others and assist further avisynth development.

FIRST... we need a script to construct the clip from images:

---begin "image seq reader.avs"------------------------------------------------------------
LoadPlugin("C:\Program Files\megui\tools\ffms\ffms2.dll")

fc = 23808
base_filename = "D:\img\album1\img"
img_type = "png"
digits = 6
h = 2048
w = 2048
fpsrate = 1.00
w_filename = "width.txt"
h_filename = "height.txt"
newline = Chr(13) + Chr(10)
w_header = "Type int" + newline + "Default -1" + newline
h_header = "Type int" + newline + "Default -1" + newline
matte = BlankClip(length=fc,w,h,"RGB32",fpsrate,audio_rate=0,channels=0) #init globally
frame = BlankClip(length=0,w,h,"RGB32",fpsrate,audio_rate=0,channels=0) #init globally

#Build returnclip
rclip = ScriptClip(matte,"""
cf = current_frame
img_filename=base_filename+RightStr("000000000"+String(cf),digits)+"."+img_type
frame=ImageSource(file=img_filename, start=0, end=0, fps=fpsrate, use_DevIL=false, info=false, pixel_type="RGB32")
return Layer(matte, frame, "add", level=257)
""")

rclip = WriteFileStart(rclip, w_filename, "w_header", append=true)
rclip = WriteFileStart(rclip, h_filename, "h_header", append=true)
rclip = WriteFile(rclip, w_filename, "String(current_frame) + Chr(32) + String(frame.Width())")
rclip = WriteFile(rclip, h_filename, "String(current_frame) + Chr(32) + String(frame.Height())")

return rclip.ConvertToYV12()

---end "image seq reader.avs"------------------------------------------------------------


...And to extract the images:


---begin "image seq writer.avs"------------------------------------------------------------

#LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\GScript.dll")
LoadPlugin("C:\Program Files\megui\tools\ffms\ffms2.dll")

srce_filename = "D:\img\album1.mp4"
dest_filename = "D:\img\extract\img" #includes image filename prefix
dest_delimiter = "_"
image_type = "png"
w_filename = "width.txt"
h_filename = "height.txt"

FFVideoSource(srce_filename)

w = FFVideoSource(srce_filename).Width()
h = FFVideoSource(srce_filename).Height()
fc = FrameCount()
i = 0 #iteration

ScriptClip("""
cf = current_frame
ImageWriter(Trim(cf, -1).ConvertToRGB().Crop(0,0,img_w,img_h), dest_filename+RightStr("000000"+String(cf),6)+dest_delimiter, 0, 0, image_type, info=false)
AddBorders(0,0, w-img_w, h-img_h, $111111)
ConvertToYV12()
""")

ConditionalReader(h_filename, "img_h", show=false)
ConditionalReader(w_filename, "img_w", show=false)

---end "image seq writer.avs"------------------------------------------------------------

MANY avisynth shortcomings quickly became apparent when I got into this project. Although it's not common for video to have varying frame dimensions, here it is a necessity. I found I couldn't even TEMPORARILY have a frame a different size, it had to be another clip entirely. There also appears to be no way to attach metadata to a clip or a specific frame, hence the parallel data files "width.txt" and "height.txt"

Constructing those files was no picnic either, due to the basic flaw of having separate per-script commands and per-frame commands, and variable scope issues. To make matters worse, basic control structure and conditional operators (for-next and if-then-else) loops are not possible in the normal fashion without an add-on such as Gscript. Gscript actually worked great, but didn't return a single frame until the entire clip was constructed. Basic file handing is also missing.

If that wasn't enough, I learned that avisynth has a bad habit of ignoring commands which do not contribute to the final output, requiring hacks.

DEVELOPMENT SUGGESTIONS:
Avisynth could *greatly* benefit from an additional mode that allows more straightforward scripting, the use of current_frame referencing globally, if-then statements, for-next loops, and an option to elminate the ignoring of commands which don't affect the final output, some sort of "force" option.

New object types: More than anything else, what's missing are frame and metadata classes. ( http://avisynth.org/mediawiki/Script_variables ). A clip object would consist of a set of frames, and both frame and clip classes would contain metadata classes, which users could extend, depending on the desired destination container.

A "force" option which guarantees all commands would execute (unless restricted by conditional control structure, of course, i.e. "IF(i>2)AND(i<1)THEN" would never happen)

Output: Some sort of console and video output would be great. MessageClip() and similar functions are useful, but a poor substitute.

The existing WriteFile() and ConditionalReader() functions are dependent on clip objects. Basic "normal" file I/O functionality would be highly useful, and scriptwriters could customize and extend their configuration files to their heart's content. I'd suggest FileOpen(), FileClose(), FileWrite(), FileRead(), each returning a bool for error condition, which could be read by GetFileError(). The important thing is these commands would be completely independent of clip or frame objects.

Finally, extending ImageSource() and ImageWriter() to be methods of a frame class would be a lifesaver. I see this has been somewhat solved by others' excellent work with the Immaavs and Corona plugins, although it appears Immaavs does not handle subfolders with wildcards, and I don't believe the Corona plugin can't return the filename into a string, it can only write it to the image.

This takes me to my next battle and where I could use some help: What's a good way to pick up where the Corona reader leaves off? Specifically, I'd like these scripts to handle an entire folder, including subfolders, and get the filenames into a string, so that they can be written to a file along with the dimensions, keep track of how many images are written, etc. The writing script would then recreate the path.

Lastly, I greatly appreciate the contributions of those that have taken avisynth as far as it has and who have posted about their problems and solutions, they've been a big help when found via Google searches. I understand avisynth's limitations are the result of the thinking at the time it was initially designed a decade or more ago, along with limitations of the AVI container itself (which led to problematic hacks such as the "packed" bitstream, a hack which caused far more problems than it solved).

Best regards,
turbine

Last edited by Guest; 5th August 2012 at 12:46. Reason: 12
turbine is offline   Reply With Quote
Old 5th August 2012, 12:18   #2  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,867
I agree with a lot of what you're saying, but a reply you will get may be, that Avisynth wasn't made for this kind of thing so you shouldn't use it. Someone might also say it's easier to do this in a plugin.

I like developing in script because of the immediacy of the write-test cycle. There's some workarounds for some of what you're saying, but I've been thinking of making some kind of array package to deal with the meta-frame issue.

I don't like having to add a bunch of plugins either, but with avslib, ffms2, grunt, gscript, you can get around a lot of limitations.

As for alternatives, imagemagick has a script, it's difficult too but fairly powerful.

If you want change, the best way is to write it yourself and try to get a patch accepted.
jmac698 is offline   Reply With Quote
Old 5th August 2012, 15:06   #3  |  Link
SEt
Registered User
 
Join Date: Aug 2007
Posts: 374
Avisynth scripts are not imperative programs "apply this to that", but rather more like functional programming – it describes filter connection path, but how actually frames flow, which frames are processed/skipped and in what order greatly depends on filters. Avisynth has no knowledge how to "process everything".
SEt is offline   Reply With Quote
Old 5th August 2012, 15:34   #4  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 1,069
Avisynth script is pretty much babby's first graph description language. People seem to think that because it has conditionals and functions it's an actual general-purpose scripting language, but it isn't and it looks like you found that out the hard way.

In general, if you want to do something within the Avisynth framework that either doesn't seem doable or seems prohibitively complicated, the answer is almost always "do it with a different tool" or "do it in a plugin". You can do almost anything in plugins, especially if you only care about compatibility with yourself, because then you can basically treat video frames like arbitrary memory buffers and do whatever you want.

Asking for new features in Avisynth is not going to get you anywhere, really. The codebase is really showing its age and the framework has so many silly limitations that it's really not worth building on. Furthermore, there is only one active developer of note (IanB) who is even working on it at all, and his priorities run more to fixing bugs and cleaning up code than adding new features (the upcoming version, Avisynth 2.6, has been in development for several years and still doesn't really offer anything new or interesting at all, except a few new colorspaces).

Last edited by TheFluff; 5th August 2012 at 20:20.
TheFluff is offline   Reply With Quote
Old 5th August 2012, 18:33   #5  |  Link
turbine
Registered User
 
Join Date: Jul 2012
Posts: 7
Thanks for the replies. I would have probably learned a lot sooner had I been able to POST in the first five days.

SEt, that's exactly the point, It needs to be able to "process everything." Nobody writes a "just execute this if you feel like it" command.
TheFluff, sounds like AviSynth needs a rewrite from the ground up, or to be abandoned for a better video scripting language.

What's a better video scripting language to use instead? Ideally one that Avisynth could optionally call for compatibility purposes, with two lines like such as:

LoadPlugin("C:\Program Files\megui\tools\ffms\videoscript2012.dll")
VideoScript2012("Video Generated with a versatile object-oriented video scripting language.vs2")
turbine is offline   Reply With Quote
Old 5th August 2012, 20:08   #6  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 6,832
Quote:
It needs to be able to "process everything."
Dont be silly.

Goodbye.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 5th August 2012, 20:49   #7  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,867
No, he has a point. Why would a language not execute certain commands? The answer lies in the type of optimizing Avisynth does. It tries hard to not run filters when it doesn't have to. It also has a cache system of processed frames. The most hasic example is SelectEvery. This will process only the frames requested.

The way around this is to ensure that all frames are requested.
Check out the call plugin which can call any other program from script.

Again, ImageMagick has it's own script language.
jmac698 is offline   Reply With Quote
Old 5th August 2012, 20:54   #8  |  Link
gyth
Registered User
 
Join Date: Sep 2011
Posts: 86
Quote:
Nobody writes a "just execute this if you feel like it" command.
http://en.wikipedia.org/wiki/Lazy_evaluation

The opposite of lazy actions is eager evaluation, sometimes known as strict evaluation. Eager evaluation is commonly believed as the default behavior used in programming languages.[citation needed]

Can I use this thread as a citation?

Quote:
What's a better video scripting language to use instead?
Add avisynth support to Python???
gyth is offline   Reply With Quote
Old 5th August 2012, 21:10   #9  |  Link
turbine
Registered User
 
Join Date: Jul 2012
Posts: 7
Thank you jmac, the point is obvious to most, I'd even call it a no-brainer. At a minimum it should have the ability to DISABLE the feature, or better documented workarounds. It's more than frames, of course -- not all statements are clip/frame related.

Stainless's dismissive attitude is not unique, it's one which has plagued software engineering for decades. The result is we often see software designed with limited regard for how others will use it because it's often designed by people with limited regard for others.

I'll check out ImageMagick. My first attempt with it didn't run, but I didn't troubleshoot it when I learned it wouldn't solve the subdirectory problem.

I appreciate the responses.
turbine is offline   Reply With Quote
Old 5th August 2012, 21:20   #10  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,867
I sort of agree with you; Avisynth may be "not meant" to do the things I want to do, but there's simply no other choice I've found. I did actually go as far as to try to do something in ImageMagick, and I wrote a script in it. But there's still a lot of useful filters in Avisynth/Virtualdub that I can't get elsewhere. There's another option though; calling Avisynth from say, a .net program and combing the two, but if you could program at that level, you probably wouldn't need much help.

Something else that comes close is Matlab. It's an interactive script, you can write a GUI in it, it imports video, and there's "toolboxes" for it. There's free clones like freemat, octave, etc.
jmac698 is offline   Reply With Quote
Old 5th August 2012, 21:38   #11  |  Link
turbine
Registered User
 
Join Date: Jul 2012
Posts: 7
I was thinking C++, not C# or other .NET platform, more cross-platform that way. Java is another option.

"Lazy Evaluation" -- appropriately named. Should be OFF by default, I'd argue, but most would settle for the ability to turn it off. It sounds like many of the common problems people have posted about for years could have been avoided by having this ability combined with a better solution for runtime commands, but that starts to get back into the topic of redesigning it from the ground up.

I too didn't see much choice, megui depends on avisynth. I may just see about wrapping the whole thing in a C++ or C# app. Thanks again.
turbine is offline   Reply With Quote
Old 5th August 2012, 21:52   #12  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 6,832
Quote:
I appreciate the responses.
You're Welcome.

Sorry for the being dismissive. It's really horses for courses and Avisynth is unlikely to change as desired.
As Jmac suggests, ImageMagick might be the better choice.

I also empathize with the 5 day post gripe, they could at least tell you before you register and save you the bother.

Welcome to the site, and have a nice day whether you stay or not.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 6th August 2012 at 02:10.
StainlessS is offline   Reply With Quote
Old 5th August 2012, 22:08   #13  |  Link
SEt
Registered User
 
Join Date: Aug 2007
Posts: 374
You people just don't understand what Avisynth script is. Again: it is not a program. It's a path, graph. The script is not executed, so there is nowhere to apply strict or lazy evaluation.
SEt is offline   Reply With Quote
Old 5th August 2012, 22:49   #14  |  Link
gyth
Registered User
 
Join Date: Sep 2011
Posts: 86
Quote:
Originally Posted by turbine View Post
At a minimum it should have the ability to DISABLE the feature, or better documented workarounds.
I might not be understanding the problem you are having.
I think I've run into similar issues with imagewriter.

http://avisynth.org/mediawiki/ImageWriter
Note that frames are not written to the output file until they are actually rendered by ImageWriter. Therefore, output is produced only for rendered frames, and there will be no output at all if the result of the filter does not contribute to the final result of the script.
Code:
a=ImageSource("sm444.png")

a.subtitle("not rendered").ImageWriter("not_rendered", type="png")

a.subtitle("rendered").ImageWriter("rendered", type="png")
Is your issue something like that?
gyth is offline   Reply With Quote
Old 5th August 2012, 22:56   #15  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 1,069
Quote:
Originally Posted by jmac698 View Post
No, he has a point. Why would a language not execute certain commands? The answer lies in the type of optimizing Avisynth does. It tries hard to not run filters when it doesn't have to. It also has a cache system of processed frames. The most hasic example is SelectEvery. This will process only the frames requested.
I think you've misunderstood how it works. As several people have pointed out, Avisynth script isn't an imperative programming language, it's a dataflow-oriented graph description language. As SEt says, the script isn't really executed; it just describes a graph through which the environment pulls video frames, where some nodes happen to have side-effects.

As for "everything should be possible", there is a reason why we don't have One Programming Language To Rule Them All. One size does not fit all, and while Avisynth script isn't very sophisticated and lacks some potentially useful features, it does make it very easy to create video filter chains, which is what it was intended for. If you really do want everything to be possible, program in assembly language (or better yet, machine code), and if that sounds too complicated for you, maybe you should sacrifice some possibilities for the convenience of simplicity.

Last edited by TheFluff; 6th August 2012 at 08:12.
TheFluff is offline   Reply With Quote
Old 5th August 2012, 23:49   #16  |  Link
turbine
Registered User
 
Join Date: Jul 2012
Posts: 7
I understand Avisynth is not a typical scripting language, and doesn't behave like a programming language. I'm just sayin' it SHOULD, or at least would be highly beneficial if it WOULD. I don't expect it to, it sounds like it's almost as dead as Latin, and probably the next place for the community to go would be an object-oriented successor, which will probably happen at some point by a whole new set of individuals to solve a different problem. Open-source and other similar projects are funny like that.

Interesting comparison, I used to code 8-bit assembly in my teens and early 20's back in the 80's and 90's. I can't see that sort of approach being too popular in 2012-2015.
turbine is offline   Reply With Quote
Old 6th August 2012, 02:26   #17  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 6,832
Quote:
Output: Some sort of console and video output would be great. MessageClip() and similar functions are useful, but a poor substitute.
In appeasement for my being dismissive, thought I might submit something that may partially fulfill your above requirement,
I was going to implement this anyway, just not planning to do it today, here an extract from the RT_Stats thread in
Avisynth usage forum (about to place an update there with the below update):

Code:
RT_debug(string s1, ... , string sn)
Non-clip function. Takes one or more strings, joins them together and then outputs the combined
string to OutputDebugString(), if strings are NOT space separated, it inserts a single space between them.
Useful during development Compile-time/Run-time scripts to monitor status in real time.
Use DebugView utility to view and capture output.
http://technet.microsoft.com/en-gb/sysinternals/bb545027

Usage eg:- RT_debug("TestFunc:", "["+String(current_frame)+"]","Going pretty good so far @","Line "+String(19))

In DebugView, filter for eg "TestFunc:" to skip all the extraneous stuff generated by other plugins/programs.
(In DebugView "Menu:Edit-Filter/Highlight", and enter "TestFunc:".
EDIT: Here:
http://forum.doom9.org/showthread.php?t=165479
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 6th August 2012 at 02:33.
StainlessS is offline   Reply With Quote
Old 6th August 2012, 08:16   #18  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 1,069
I don't really understand what you want to accomplish with your script, but it doesn't look very complicated and I don't understand why you've chosen Avisynth as your tool even after determining beyond a shadow of doubt that it isn't very suitable for the task. I think you'd have a lot better luck with hacking something together using Perl and ImageMagick.

Quote:
Originally Posted by turbine View Post
I understand Avisynth is not a typical scripting language, and doesn't behave like a programming language. I'm just sayin' it SHOULD, or at least would be highly beneficial if it WOULD. I don't expect it to, it sounds like it's almost as dead as Latin, and probably the next place for the community to go would be an object-oriented successor, which will probably happen at some point by a whole new set of individuals to solve a different problem. Open-source and other similar projects are funny like that.
If you want a real programming language, use a real programming language. Language design is very hard and it's much easier to create bindings for an existing language than it is to invent a new one. Unfortunately, Avisynth's collection of filters are pretty tightly tied to the scripting language, so it's really hard to create useful bindings for a different language.

I don't really understand your hardon for object-orientedness either. The class/object/inheritance model is a poor way to represent video frames and video clips, and most filter cores are written in very low-level C (or even assembler) for performance reasons. The choice of C++ for the Avisynth plugin interface has also proven to be a very bad choice.

Last edited by TheFluff; 6th August 2012 at 08:27.
TheFluff is offline   Reply With Quote
Old 6th August 2012, 10:07   #19  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,867
>I think you've misunderstood how it works.
I was trying to translate points of view here. I understand his point of view and yours.

When you're new to Avisynth, all these questions come up. I didn't find the manual very clear (in fact it still frustrates me, there's no real index to all keywords).

And saying it's a filter graph is just semantics; I can write a loop that does nothing but calculate a factorial, if I want to. You could even argue that it's Turiing complete and therefore like any other language. There's even GUI plugins to Avisynth; you can get keyboard input and interact with it just like a real program. Just let the video play forever and forget that it's a video, cause it's really a program

Last edited by jmac698; 6th August 2012 at 10:17.
jmac698 is offline   Reply With Quote
Old 6th August 2012, 10:15   #20  |  Link
turbine
Registered User
 
Join Date: Jul 2012
Posts: 7
Reasonable questions...

I chose Avisynth because I thought it was just a matter of getting the syntax set up right to get desired behavior, I didn't really think there was an alternative, and also to learn avisynth.

The goal is to be able to compress any given image library using x264, and extract it.

Object-oriented has been the standard for a long time due to its advantages. On the contrary it's the intuitive way to way to represent video and audio, the standard "is a" and "has a" questions we ask ourselves when creating classes. A video "has a" set of frames. A frame "has" one or more images (1080p vs. 480p, 1.77 AR vs. 1.33 for example), an image "has a" set of pixels.

Performance would take a hit, but isn't really a big concern when the weak link in the chain is typically the codec (x264 usually) and it's not realtime. Realtime frame-serving would arguably be a different story.

I realized it could be useful to create a parallel clip to store metadata, a RGB32 pixel can store a 32bit float or four ASCII characters, for example.

Problem is, I don't see a function to read/write individual pixels, only ShowPixelValues() which doesn't look like it returns the values. It doesn't appear PixelInfo() PsetRGB() is available anymore. http://forum.doom9.org/archive/index.php/t-108602.html.

Uncompressed audio could work if it's possible to read/write individual samples...I've not looked into it yet, though. I downloaded AVSLib, so I'll see what's possible between that and Gscript.

If Avisynth development is pretty much dead, then it is, and not much can be done about these concerns, which I gather are pretty common. On the other hand, if it is still commonly used, there's probably still value in adding a new run mode so that a user could choose to have it execute all commands as written, instead of it's current "lazy" execution model. Doesn't it seem like that would solve so many common problems?
turbine is offline   Reply With Quote
Reply

Tags
file i/o, frame, imagesource, imagewriter, metadata

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 22:37.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.