Panning across a still image not smooth, plus slow rendering

[pshute]

I'm a first time AviSynth user, and new to any kind of video editing, so I'm not sure what sort of results to expect.

Problem 1:
I've got a long image, say 300 pixels by 10,000 pixels, and I've written a script to pan very slowly across it. It works ok, but the panning isn't perfectly smooth. When played from a rendered avi, at least a couple of times a second it seems to jerk very slightly.

Is this the best I can expect? Is it just a limitation of the graphics ability of my own computer?

Problem 2:
I've also found that it's slow to render. I can't play it straight from the script in real time in VirtualDub or Media Player, and I think it takes about 10 minutes to render a 3 minute video. Is this also the best I can expect, or is it caused by the method I'm using?

I'm importing the image as the required number of frames, then using the external Zoom plugin to shift each frame to the required X offset (no zooming), and then clipping them all to a 16:9 frame size.

I would have just clipped each frame, but I haven't worked out how to iterate through them yet, and the Zoom plugin does that for me. Could it be that the Zoom plugin is just slow? Or is it silly to expect to make a movie 10,000 pixels wide before clipping?

I'd post my script here, but I haven't got it with me at the moment. I've waited my 5 days after registering before I can post, and I can't wait one more till I get back to the PC.

[IanB]

Welcome to the forum.

First yes we need to see your script.

But as a guess the jerking may be caused by rounding errors in your scroll calculations. It is very important to move exactly the same number of pixels each frame, the eye can tell even if it is off by just 1 pixel, ie a pattern of 2, 2, 2, 2, 2, 2, 3, .... will be very noticeable.

[Didée]

Simple horizontal scrolling? Not too difficult. And yes, it could be zoom.dll is not the fastest of all things to use.

A little picture of 23600*3000 pixels: <click> (http://upload.wikimedia.org/wikipedia/commons/b/b6/Earth%27s_Location_in_the_Universe_%28JPEG%29.jpg)

A little script.
setmemorymax(512)

imagesource("Earth's_Location_in_the_Universe_(JPEG).jpg")

pwidth = 1920 # set to desired
pheight = 1080 # output resolution

pwidthF=float(pwidth)
pheightF=float(pheight)

bicubicresize(width()*pheight/height(),pheight)
trim(1,1).loop(1440*3)

animate( 0,1440*3, "bicubicresize", pwidth,pheight, .33,.33, 0.0, 0.0, pwidth,pheight,
\ pwidth,pheight, .33,.33, width()-pwidthF,0.0, pwidth,pheight)

return(last)

On my i7-860, this renders roughly at ...
~ 50fps for 1920x1080 output (i.e. 2x realtime)
~100fps for 1280x720 output (i.e. 4x realtime)

[pshute]

Welcome to the forum.

First yes we need to see your script.

But as a guess the jerking may be caused by rounding errors in your scroll calculations. It is very important to move exactly the same number of pixels each frame, the eye can tell even if it is off by just 1 pixel, ie a pattern of 2, 2, 2, 2, 2, 2, 3, .... will be very noticeable.
That could be it, but I'll have to check.

I deliberately made sure to let it round to a calculated value, as I have to keep the panning exactly in synch with an audio track because the image is a spectrogram.

If I have to always pan each frame by the same amount then I suppose I need to adjust the image size a little.

[pshute]

Welcome to the forum.

First yes we need to see your script.

But as a guess the jerking may be caused by rounding errors in your scroll calculations. It is very important to move exactly the same number of pixels each frame, the eye can tell even if it is off by just 1 pixel, ie a pattern of 2, 2, 2, 2, 2, 2, 3, .... will be very noticeable.
Yes, that was the answer. I was panning by some odd figure like 0.741 pixels per frame, so after rounding it was repeating this pattern: 0 pixels, 1 pixel, 1, 1, 0, 1, 1, 0, 1, 1, 1.

No wonder it was jerky, thanks for that.

I tried it with 1 pixel/frame, and that was nice and smooth, but out of synch with the audio of course. So I have to adjust the image length to get that right.

I've often wondered why audio programs like Audacity and Sonic Visualiser don't have a perfectly smooth cursor movement, and this must be why. I started out trying to do screen captures from them, and that makes it even worse - I assume because the frame rates aren't synched with the movement.

[pshute]

Simple horizontal scrolling? Not too difficult. And yes, it could be zoom.dll is not the fastest of all things to use.

A little picture of 23600*3000 pixels: <click> (http://upload.wikimedia.org/wikipedia/commons/b/b6/Earth%27s_Location_in_the_Universe_%28JPEG%29.jpg)

A little script.
setmemorymax(512)

imagesource("Earth's_Location_in_the_Universe_(JPEG).jpg")

pwidth = 1920 # set to desired
pheight = 1080 # output resolution

pwidthF=float(pwidth)
pheightF=float(pheight)

bicubicresize(width()*pheight/height(),pheight)
trim(1,1).loop(1440*3)

animate( 0,1440*3, "bicubicresize", pwidth,pheight, .33,.33, 0.0, 0.0, pwidth,pheight,
\ pwidth,pheight, .33,.33, width()-pwidthF,0.0, pwidth,pheight)

return(last)

On my i7-860, this renders roughly at ...
~ 50fps for 1920x1080 output (i.e. 2x realtime)
~100fps for 1280x720 output (i.e. 4x realtime)
Thanks, I'll give that a try when I get time. Can you please explain why you've used no clip parameters in any of those filters? Is there a concept of a current clip in AviSynth?

[Gavino]

Yes, that was the answer. I was panning by some odd figure like 0.741 pixels per frame
That's not an issue if you use Didée's method from post #3, since the resizers support subpixel resampling. (Notice that he uses floating point values for the additional resizer arguments.)

[IanB]

Thanks, I'll give that a try when I get time. Can you please explain why you've used no clip parameters in any of those filters? Is there a concept of a current clip in AviSynth?
Yes, it is called "Implicit Last" :search:

Whenever you do not specify a source clip or do not assign the result to a clip the reserved variable "Last" is assumed.

So :-....
imagesource("Earth's ....
....
bicubicresize(width()*pheight/height(),pheight)
trim(1,1).loop(1440*3)

animate( 0,1440*3, ...is interpreted as ....
last=imagesource("Earth's ....
....
last=last.bicubicresize(width()*pheight/height(),pheight)
last=last.trim(1,1).loop(1440*3)

last=last.animate( 0,1440*3,....
Also note trick, "trim(1,1).loop(1440*3)" which forces the known static image load and initial resize to be performed on frame 1 only, then copied from the cache for all the remaining frames.

[pshute]

That's not an issue if you use Didée's method from post #3, since the resizers support subpixel resampling. (Notice that he uses floating point values for the additional resizer arguments.)
Sorry, I missed this posting. Yes, you're right, that does smooth it out. I think making it 1 pixel per frame might be slightly smoother, but given that I'm not even sure I can tell the difference, for the amount of trouble it will save adjusting image widths, this is a very good solution.

On the image I'm using (much wider, but not as tall), I'm now getting around 170fps, so a vast improvement over using the Zoom plugin. It's slightly jerky if I watch it in real time in VirtualDub, but very good from a saved avi file.

Thank you very much, Didée. (And Gavino for pointing out this side benefit.)

[pshute]

Also note trick, "trim(1,1).loop(1440*3)" which forces the known static image load and initial resize to be performed on frame 1 only, then copied from the cache for all the remaining frames.
Is this more efficient than
imagesource("Earth's_Location_in_the_Universe_(JPEG).jpg", end=1440*3, fps=30)

[IanB]

A Happy New Year to all.


Imagesource internally uses a similar technique to force a single frame.

The trim/loop thing above is done after the resize operation so more is included in the single frame protection trick, so it is more "efficient" in this case.

You can also use FreezeFrame() to elicit the same behaviour.

[pshute]

Simple horizontal scrolling? Not too difficult. And yes, it could be zoom.dll is not the fastest of all things to use.

A little picture of 23600*3000 pixels: <click> (http://upload.wikimedia.org/wikipedia/commons/b/b6/Earth%27s_Location_in_the_Universe_%28JPEG%29.jpg)

A little script.
setmemorymax(512)

imagesource("Earth's_Location_in_the_Universe_(JPEG).jpg")

pwidth = 1920 # set to desired
pheight = 1080 # output resolution

pwidthF=float(pwidth)
pheightF=float(pheight)

bicubicresize(width()*pheight/height(),pheight)
trim(1,1).loop(1440*3)

animate( 0,1440*3, "bicubicresize", pwidth,pheight, .33,.33, 0.0, 0.0, pwidth,pheight,
\ pwidth,pheight, .33,.33, width()-pwidthF,0.0, pwidth,pheight)

return(last)

On my i7-860, this renders roughly at ...
~ 50fps for 1920x1080 output (i.e. 2x realtime)
~100fps for 1280x720 output (i.e. 4x realtime)
I've just tried your exact script with the same sample image to get a speed comparison. On my i5-2400, 1920x1080 renders at barely 1fps, and 1280x720 is just 30fps. Does that sound right for this processor? 640x360 is around 150fps.

[pshute]

I'm now trying this panning routine on a very wide image - 285,000 pixels wide x 513 high - and the frame rate of encoding (with VirtualDub) is very low, I assumed because the bicubicresize function is continually clipping out a short section from the large file.

If my frame size is 240x136 or above, I get 1fps or less.

But at 192x108 or below, suddenly I'm getting 200fps. Does this mean I'm running out of memory or something? Is there anything I can do to fix this? I'm not seeing memory usage going above about 2.6GB of 4GB on this Win7/64 bit machine.

I thought of splitting the image into parts, but it sounds complicated because there'll have to be overlaps so there are no gaps where it pans off the end of a segment.

[IanB]

This is what happens when the cache stops working.

285,000 x 513 x 4 bytes per pixel = 584,820,000 bytes -> 557MB, try at least SetMemoryMax(1115).

However SetMemoryMax() will not exceed the lesser of MEMORYSTATUS.dwAvailVirtual and MEMORYSTATUS.dwAvailPhys as returned by the GlobalMemoryStatus() call. i.e. effectively the ram not currently in use for anything else on the whole operating system.

Output the value returned from SetMemoryMax() to see what you are actually getting.


This is the exact code that does the SetMemoryMax call :-int ScriptEnvironment::SetMemoryMax(int mem) {
if (mem > 0) {
MEMORYSTATUS memstatus;
__int64 mem_limit;

GlobalMemoryStatus(&memstatus); // Correct call for a 32Bit process. -Ex gives numbers we cannot use!

memory_max = mem * 1048576i64; // mem as megabytes
if (memory_max < memory_used) memory_max = memory_used; // can't be less than we already have

if (memstatus.dwAvailVirtual < memstatus.dwAvailPhys) // Check for big memory in Vista64
mem_limit = (__int64)memstatus.dwAvailVirtual;
else
mem_limit = (__int64)memstatus.dwAvailPhys;

mem_limit += memory_used - 5242880i64;
if (memory_max > mem_limit) memory_max = mem_limit; // can't be more than 5Mb less than total
if (memory_max < 4194304i64) memory_max = 4194304i64; // can't be less than 4Mb -- Tritical Jan 2006
}
return (int)(memory_max/1048576i64);
}

[cretindesalpes]

I'm now trying this panning routine on a very wide image - 285,000 pixels wide x 513 high
You could also transpose it (or turn it) outside Avisynth, load it and pan vertically, then transpose back the resulting frames. At least, this will improve data locality.

[pshute]

This is what happens when the cache stops working.

285,000 x 513 x 4 bytes per pixel = 584,820,000 bytes -> 557MB, try at least SetMemoryMax(1115).
I had it set to 512, as per Didee's example. I had looked at the documentation for that, and misread it as meaning that was the maximum possible. I've set it higher, and now I'm getting 22fps. Thanks for that! Now to try for higher.
However SetMemoryMax() will not exceed the lesser of MEMORYSTATUS.dwAvailVirtual and MEMORYSTATUS.dwAvailPhys as returned by the GlobalMemoryStatus() call. i.e. effectively the ram not currently in use for anything else on the whole operating system.

Output the value returned from SetMemoryMax() to see what you are actually getting.
I'm getting variable results. If I set it to 1200, one time I might get about 700 actual, the next 1200.

I'm pressing F2 to reload the script to check the value. After loading the script into VirtualDub, does the value ever change, despite it reporting one value?

And could someone please assist with making this diagnostic easier to obtain? At the moment, I'm placing
messageclip(string(setmemorymax()))
at the end of my code, running it to get the value and then commenting it out and running it again to see how fast I can render.

I tried making the message into a short clip to splice to the start of my video so I can see both, but after trying to deal with its complaints about missing audio, different frame sizes and now incompatible video types, I've given up on it.

I also tried overlaying it, but the message filled the frame and hid the video.

[pshute]

You could also transpose it (or turn it) outside Avisynth, load it and pan vertically, then transpose back the resulting frames. At least, this will improve data locality.
Data locality? Surely it still all has to be in memory, and it still has to do the work of cropping each frame out?

[IanB]

To log the available MemoryMax you could use WriteFileStart(). It will write the message as the script is compiled.

Mem=setmemorymax(1115)

imagesource("Earth's_Location_in_the_Universe_(JPEG).jpg")

WriteFileStart("Mem.log", "Mem") # Write as script is compiled

pwidth = 240 # set to desired
pheight = 136 # output resolution
pFrames = 1440 # Output duration

pwidthF=float(pwidth)
pheightF=float(pheight)

bicubicresize(width()*pheight/height(),pheight)
trim(1,1).loop(pFrames*3)

animate( 0,pFrames*3, "bicubicresize", pwidth,pheight, .33,.33, 0.0, 0.0, pwidth,pheight,
\ pwidth,pheight, .33,.33, width()-pwidthF,0.0, pwidth,pheight)

return(last)

[Asmodian]

Data locality? Surely it still all has to be in memory, and it still has to do the work of cropping each frame out?

cretindesalpes is talking about the order it is accessed in memory, give it a try. :)

http://en.wikipedia.org/wiki/Locality_of_reference

[IanB]

As for data locality, yes it will all be in memory, but for CPU cache performance the L1 and L2 cache locality for this style of script will be better if the scroll was vertical instead of horizontal. i.e. all the bytes for a crop session are together.

Of course you need to balance speeding 1 line of the script with adding a TurnLeft() at the end. So the gain may actually be a loss.

[pshute]

I'm wondering if anyone can explain the difficulties I'm having getting these panning videos to work smoothly on Youtube on my PC.

This one works ok for me with Firefox:
http://www.youtube.com/watch?v=mFP4WnMmqvU
It's just slightly jittery, but it's a very slow pan, less than one (interpolated) pixel per frame.

But this much faster one is almost unwatchable:
http://www.youtube.com/watch?v=PZGB97CTzzo
That's closer to 8 pixels per frame. It's much smoother when I play the avi file directly, and I assume the jitter I see there is because 8 pixels per frame is just jittery.

And why do they all look great when I play them from Youtube on my iPad? Does this mean that all my troubles are related to a crappy viewer in Firefox?

[HMJ]

setmemorymax(512)

imagesource("Earth's_Location_in_the_Universe_(JPEG).jpg")

pwidth = 1920 # set to desired
pheight = 1080 # output resolution

pwidthF=float(pwidth)
pheightF=float(pheight)

bicubicresize(width()*pheight/height(),pheight)
trim(1,1).loop(1440*3)

animate( 0,1440*3, "bicubicresize", pwidth,pheight, .33,.33, 0.0, 0.0, pwidth,pheight,
\ pwidth,pheight, .33,.33, width()-pwidthF,0.0, pwidth,pheight)

return(last)

On my i7-860, this renders roughly at ...
~ 50fps for 1920x1080 output (i.e. 2x realtime)
~100fps for 1280x720 output (i.e. 4x realtime)

Thanks for this script. It is just what I was looking for and this is the only forum thread that has this information. One question though. Why are you resizing twice? That seems very inefficient to me and you are making a lossy copy of an already lossy copy of the original without any apparent reason to do so--or is there? I just use the Crop filter in Animate and it works just fine. Keep in mind that Crop takes int arguments that are multiples of 2. Also, how are you getting such high render rates? I'm running VirtualDub via WINE on a Mac Pro with 8 x 2.4Ghz cores and I'm only getting 24 fps. Of course WINE really IS an emulator ;) so that tends to slow things a bit, but not THAT much. I may also have installed the 32-bit version of VD for better compatibility but even so…

[HMJ]

I'm wondering if anyone can explain the difficulties I'm having getting these panning videos to work smoothly on Youtube on my PC.

This one works ok for me with Firefox:
http://www.youtube.com/watch?v=mFP4WnMmqvU
It's just slightly jittery, but it's a very slow pan, less than one (interpolated) pixel per frame.

But this much faster one is almost unwatchable:
http://www.youtube.com/watch?v=PZGB97CTzzo
That's closer to 8 pixels per frame. It's much smoother when I play the avi file directly, and I assume the jitter I see there is because 8 pixels per frame is just jittery.

And why do they all look great when I play them from Youtube on my iPad? Does this mean that all my troubles are related to a crappy viewer in Firefox?

Same problem here. YT player is crap on FF. It is better on Safari but still not as good as standalone. That's why I use the excellent Download YouTube Videos as MP4 (http://userscripts.org/scripts/show/25105) Greasemonkey script and play them on a standalone player. You can play the video directly or download it first.

And they play just as well on iPad as on a standalone player because iPad doesn't do FLASH, the crappiest media UI ever conceived. Right up there with JAVA and javascript, the two best vectors for infecting computers with malware. iDevices receive a Quicktime compatible version of the video automatically.

[HMJ]

imagesource("Earth's_Location_in_the_Universe_(JPEG).jpg", end=1440*3, fps=30)

Imagesource internally uses a similar technique to force a single frame.

The trim/loop thing above is done after the resize operation so more is included in the single frame protection trick, so it is more "efficient" in this case.

You can also use FreezeFrame() to elicit the same behaviour.

Thanks for this neat trick. It speeds up render rates by ~6x! I combine this with the above usage of ImageSource so as to control the frame rate directly.

I simply do this:

fps = 60
duration = 20
frames = fps * duration


Then add

end=frames, fps=fps

to the ImageSource call. This way I easily control frame rate and duration.

I also prefer FreezeFrame because it is more intuitive and a little cleaner looking in the code.

FreezeFrame(1, frames, 1)

[Gavino]

FreezeFrame(1, frames, 1)
FreezeFrame(1, frames-1, 0)
is better. (Frames are numbered from zero, not 1)

[Gavino]

Why are you resizing twice? That seems very inefficient to me and you are making a lossy copy of an already lossy copy of the original without any apparent reason to do so--or is there? I just use the Crop filter in Animate and it works just fine. Keep in mind that Crop takes int arguments that are multiples of 2.
That's exactly the reason to use resize instead of Crop().
Crop() is fine if you want to pan by exactly some multiple of 2 pixels per frame, but usually you want some completely arbitrary rate (eg 0.741 as in original query) to fit in with timing requirements.

[HMJ]

FreezeFrame(1, frames-1, 0)
is better. (Frames are numbered from zero, not 1)

Then the example you give should be

trim(0,1).loop(1440*3)

Actually the first way works fine since all frames are the same in this case and the Animate end frame is the correct frame.

[HMJ]

That's exactly the reason to use resize instead of Crop().
Crop() is fine if you want to pan by exactly some multiple of 2 pixels per frame, but usually you want some completely arbitrary rate (eg 0.741 as in original query) to fit in with timing requirements.

But that doesn't get around the double resizing issue and the ensuing loss of quality. That restriction is only on the width and height parameters, not the origin I believe. The timing issue is dependent on the origin. I don't know if Animate is rounding this parameter or not.

[Gavino]

Then the example you give should be
trim(0,1).loop(1440*3)

Actually the first way works fine since all frames are the same in this case and the Animate end frame is the correct frame.
I didn't give the example, but anyway Trim(0,1).Loop(...) is wrong as it would double the length (repeating both frames 0 and 1). I normally use Trim(0,-1).Loop(...), but Trim(1,1).Loop(...) works just as well - the important thing is that the Trim() should select just a single frame.

However, looking at the example again, I see the Animate end frame is actually wrong - it should be 3*1440-1, since that is the frame number of the last frame. As it stands, the animation doesn't quite reach the right hand edge of the image.

EDIT: Using your approach with ImageSource(end=frames) and FreezeFrame(), the animation is correct, since there are now frames+1 frames. However, the duration is now 'wrong' (or at least different from before).

But that doesn't get around the double resizing issue and the ensuing loss of quality. That restriction is only on the width and height parameters, not the origin I believe. The timing issue is dependent on the origin. I don't know if Animate is rounding this parameter or not.
Crop() restricts both the origin and the dimensions.
Animate will round a parameter to the nearest integer if both the initial and final values are integers. That's why it's important to ensure you pass floats when animating a resizer, otherwise you will get a jerky pan.

If you're bothered about the quality loss from double resampling, you could make the first resize affect only the height and keep the original width into the animation. However, the script would be a lot slower, since on every frame in the animation, it would be doing a large horizontal downsize, rather than a simple resampling without downsizing. So basically, it's trading a slight quality loss for a big speedup.