Panning across a still image not smooth, plus slow rendering

[pshute]

I'm a first time AviSynth user, and new to any kind of video editing, so I'm not sure what sort of results to expect.

Problem 1:
I've got a long image, say 300 pixels by 10,000 pixels, and I've written a script to pan very slowly across it. It works ok, but the panning isn't perfectly smooth. When played from a rendered avi, at least a couple of times a second it seems to jerk very slightly.

Is this the best I can expect? Is it just a limitation of the graphics ability of my own computer?

Problem 2:
I've also found that it's slow to render. I can't play it straight from the script in real time in VirtualDub or Media Player, and I think it takes about 10 minutes to render a 3 minute video. Is this also the best I can expect, or is it caused by the method I'm using?

I'm importing the image as the required number of frames, then using the external Zoom plugin to shift each frame to the required X offset (no zooming), and then clipping them all to a 16:9 frame size.

I would have just clipped each frame, but I haven't worked out how to iterate through them yet, and the Zoom plugin does that for me. Could it be that the Zoom plugin is just slow? Or is it silly to expect to make a movie 10,000 pixels wide before clipping?

I'd post my script here, but I haven't got it with me at the moment. I've waited my 5 days after registering before I can post, and I can't wait one more till I get back to the PC.

[IanB]

Welcome to the forum.

First yes we need to see your script.

But as a guess the jerking may be caused by rounding errors in your scroll calculations. It is very important to move exactly the same number of pixels each frame, the eye can tell even if it is off by just 1 pixel, ie a pattern of 2, 2, 2, 2, 2, 2, 3, .... will be very noticeable.

[Didée]

Simple horizontal scrolling? Not too difficult. And yes, it could be zoom.dll is not the fastest of all things to use.

A little picture of 23600*3000 pixels: <click>

A little script.

setmemorymax(512)

imagesource("Earth's_Location_in_the_Universe_(JPEG).jpg")

pwidth  = 1920  #  set to desired
pheight = 1080  #  output resolution

pwidthF=float(pwidth)
pheightF=float(pheight)
 
bicubicresize(width()*pheight/height(),pheight)
trim(1,1).loop(1440*3)

animate( 0,1440*3, "bicubicresize", pwidth,pheight, .33,.33, 0.0,            0.0, pwidth,pheight, 
 \                                  pwidth,pheight, .33,.33, width()-pwidthF,0.0, pwidth,pheight)

return(last)

On my i7-860, this renders roughly at ...
~ 50fps for 1920x1080 output (i.e. 2x realtime)
~100fps for 1280x720 output (i.e. 4x realtime)

[pshute]

Originally Posted by IanB

Welcome to the forum.

First yes we need to see your script.

But as a guess the jerking may be caused by rounding errors in your scroll calculations. It is very important to move exactly the same number of pixels each frame, the eye can tell even if it is off by just 1 pixel, ie a pattern of 2, 2, 2, 2, 2, 2, 3, .... will be very noticeable.

That could be it, but I'll have to check.

I deliberately made sure to let it round to a calculated value, as I have to keep the panning exactly in synch with an audio track because the image is a spectrogram.

If I have to always pan each frame by the same amount then I suppose I need to adjust the image size a little.

[pshute]

Originally Posted by IanB

Welcome to the forum.

First yes we need to see your script.

But as a guess the jerking may be caused by rounding errors in your scroll calculations. It is very important to move exactly the same number of pixels each frame, the eye can tell even if it is off by just 1 pixel, ie a pattern of 2, 2, 2, 2, 2, 2, 3, .... will be very noticeable.

Yes, that was the answer. I was panning by some odd figure like 0.741 pixels per frame, so after rounding it was repeating this pattern: 0 pixels, 1 pixel, 1, 1, 0, 1, 1, 0, 1, 1, 1.

No wonder it was jerky, thanks for that.

I tried it with 1 pixel/frame, and that was nice and smooth, but out of synch with the audio of course. So I have to adjust the image length to get that right.

I've often wondered why audio programs like Audacity and Sonic Visualiser don't have a perfectly smooth cursor movement, and this must be why. I started out trying to do screen captures from them, and that makes it even worse - I assume because the frame rates aren't synched with the movement.

[pshute]

Originally Posted by Didée

Simple horizontal scrolling? Not too difficult. And yes, it could be zoom.dll is not the fastest of all things to use.

A little picture of 23600*3000 pixels: <click>

A little script.

setmemorymax(512)

imagesource("Earth's_Location_in_the_Universe_(JPEG).jpg")

pwidth  = 1920  #  set to desired
pheight = 1080  #  output resolution

pwidthF=float(pwidth)
pheightF=float(pheight)
 
bicubicresize(width()*pheight/height(),pheight)
trim(1,1).loop(1440*3)

animate( 0,1440*3, "bicubicresize", pwidth,pheight, .33,.33, 0.0,            0.0, pwidth,pheight, 
 \                                  pwidth,pheight, .33,.33, width()-pwidthF,0.0, pwidth,pheight)

return(last)

On my i7-860, this renders roughly at ...
~ 50fps for 1920x1080 output (i.e. 2x realtime)
~100fps for 1280x720 output (i.e. 4x realtime)

Thanks, I'll give that a try when I get time. Can you please explain why you've used no clip parameters in any of those filters? Is there a concept of a current clip in AviSynth?

[Gavino]

Originally Posted by pshute

Yes, that was the answer. I was panning by some odd figure like 0.741 pixels per frame

That's not an issue if you use Didée's method from post #3, since the resizers support subpixel resampling. (Notice that he uses floating point values for the additional resizer arguments.)

[IanB]

Originally Posted by pshute

Thanks, I'll give that a try when I get time. Can you please explain why you've used no clip parameters in any of those filters? Is there a concept of a current clip in AviSynth?

Yes, it is called "Implicit Last"

Whenever you do not specify a source clip or do not assign the result to a clip the reserved variable "Last" is assumed.

So :-

....
imagesource("Earth's ....
....
bicubicresize(width()*pheight/height(),pheight)
trim(1,1).loop(1440*3)

animate( 0,1440*3, ...

is interpreted as

....
last=imagesource("Earth's ....
....
last=last.bicubicresize(width()*pheight/height(),pheight)
last=last.trim(1,1).loop(1440*3)

last=last.animate( 0,1440*3,....

Also note trick, "trim(1,1).loop(1440*3)" which forces the known static image load and initial resize to be performed on frame 1 only, then copied from the cache for all the remaining frames.

[pshute]

Originally Posted by Gavino

That's not an issue if you use Didée's method from post #3, since the resizers support subpixel resampling. (Notice that he uses floating point values for the additional resizer arguments.)

Sorry, I missed this posting. Yes, you're right, that does smooth it out. I think making it 1 pixel per frame might be slightly smoother, but given that I'm not even sure I can tell the difference, for the amount of trouble it will save adjusting image widths, this is a very good solution.

On the image I'm using (much wider, but not as tall), I'm now getting around 170fps, so a vast improvement over using the Zoom plugin. It's slightly jerky if I watch it in real time in VirtualDub, but very good from a saved avi file.

Thank you very much, Didée. (And Gavino for pointing out this side benefit.)

[pshute]

Originally Posted by IanB

Also note trick, "trim(1,1).loop(1440*3)" which forces the known static image load and initial resize to be performed on frame 1 only, then copied from the cache for all the remaining frames.

Is this more efficient than

imagesource("Earth's_Location_in_the_Universe_(JPEG).jpg", end=1440*3, fps=30)

[IanB]

A Happy New Year to all.


Imagesource internally uses a similar technique to force a single frame.

The trim/loop thing above is done after the resize operation so more is included in the single frame protection trick, so it is more "efficient" in this case.

You can also use FreezeFrame() to elicit the same behaviour.

[pshute]

Originally Posted by Didée

Simple horizontal scrolling? Not too difficult. And yes, it could be zoom.dll is not the fastest of all things to use.

A little picture of 23600*3000 pixels: <click>

A little script.

setmemorymax(512)

imagesource("Earth's_Location_in_the_Universe_(JPEG).jpg")

pwidth  = 1920  #  set to desired
pheight = 1080  #  output resolution

pwidthF=float(pwidth)
pheightF=float(pheight)
 
bicubicresize(width()*pheight/height(),pheight)
trim(1,1).loop(1440*3)

animate( 0,1440*3, "bicubicresize", pwidth,pheight, .33,.33, 0.0,            0.0, pwidth,pheight, 
 \                                  pwidth,pheight, .33,.33, width()-pwidthF,0.0, pwidth,pheight)

return(last)

On my i7-860, this renders roughly at ...
~ 50fps for 1920x1080 output (i.e. 2x realtime)
~100fps for 1280x720 output (i.e. 4x realtime)

I've just tried your exact script with the same sample image to get a speed comparison. On my i5-2400, 1920x1080 renders at barely 1fps, and 1280x720 is just 30fps. Does that sound right for this processor? 640x360 is around 150fps.

[pshute]

I'm now trying this panning routine on a very wide image - 285,000 pixels wide x 513 high - and the frame rate of encoding (with VirtualDub) is very low, I assumed because the bicubicresize function is continually clipping out a short section from the large file.

If my frame size is 240x136 or above, I get 1fps or less.

But at 192x108 or below, suddenly I'm getting 200fps. Does this mean I'm running out of memory or something? Is there anything I can do to fix this? I'm not seeing memory usage going above about 2.6GB of 4GB on this Win7/64 bit machine.

I thought of splitting the image into parts, but it sounds complicated because there'll have to be overlaps so there are no gaps where it pans off the end of a segment.

[IanB]

This is what happens when the cache stops working.

285,000 x 513 x 4 bytes per pixel = 584,820,000 bytes -> 557MB, try at least SetMemoryMax(1115).

However SetMemoryMax() will not exceed the lesser of MEMORYSTATUS.dwAvailVirtual and MEMORYSTATUS.dwAvailPhys as returned by the GlobalMemoryStatus() call. i.e. effectively the ram not currently in use for anything else on the whole operating system.

Output the value returned from SetMemoryMax() to see what you are actually getting.


This is the exact code that does the SetMemoryMax call :-

int ScriptEnvironment::SetMemoryMax(int mem) {
  if (mem > 0) {
    MEMORYSTATUS memstatus;
    __int64 mem_limit;

    GlobalMemoryStatus(&memstatus); // Correct call for a 32Bit process. -Ex gives numbers we cannot use!

    memory_max = mem * 1048576i64;                          // mem as megabytes
    if (memory_max < memory_used) memory_max = memory_used; // can't be less than we already have

    if (memstatus.dwAvailVirtual < memstatus.dwAvailPhys) // Check for big memory in Vista64
      mem_limit = (__int64)memstatus.dwAvailVirtual;
    else
      mem_limit = (__int64)memstatus.dwAvailPhys;

    mem_limit += memory_used - 5242880i64;
    if (memory_max > mem_limit) memory_max = mem_limit;     // can't be more than 5Mb less than total
    if (memory_max < 4194304i64) memory_max = 4194304i64;   // can't be less than 4Mb -- Tritical Jan 2006
  }
  return (int)(memory_max/1048576i64);
}

[cretindesalpes]

Originally Posted by pshute

I'm now trying this panning routine on a very wide image - 285,000 pixels wide x 513 high

You could also transpose it (or turn it) outside Avisynth, load it and pan vertically, then transpose back the resulting frames. At least, this will improve data locality.

[pshute]

Originally Posted by IanB

This is what happens when the cache stops working.

285,000 x 513 x 4 bytes per pixel = 584,820,000 bytes -> 557MB, try at least SetMemoryMax(1115).

I had it set to 512, as per Didee's example. I had looked at the documentation for that, and misread it as meaning that was the maximum possible. I've set it higher, and now I'm getting 22fps. Thanks for that! Now to try for higher.

However SetMemoryMax() will not exceed the lesser of MEMORYSTATUS.dwAvailVirtual and MEMORYSTATUS.dwAvailPhys as returned by the GlobalMemoryStatus() call. i.e. effectively the ram not currently in use for anything else on the whole operating system.

Output the value returned from SetMemoryMax() to see what you are actually getting.

I'm getting variable results. If I set it to 1200, one time I might get about 700 actual, the next 1200.

I'm pressing F2 to reload the script to check the value. After loading the script into VirtualDub, does the value ever change, despite it reporting one value?

And could someone please assist with making this diagnostic easier to obtain? At the moment, I'm placing

messageclip(string(setmemorymax()))

at the end of my code, running it to get the value and then commenting it out and running it again to see how fast I can render.

I tried making the message into a short clip to splice to the start of my video so I can see both, but after trying to deal with its complaints about missing audio, different frame sizes and now incompatible video types, I've given up on it.

I also tried overlaying it, but the message filled the frame and hid the video.

[pshute]

Originally Posted by cretindesalpes

You could also transpose it (or turn it) outside Avisynth, load it and pan vertically, then transpose back the resulting frames. At least, this will improve data locality.

Data locality? Surely it still all has to be in memory, and it still has to do the work of cropping each frame out?

[IanB]

To log the available MemoryMax you could use WriteFileStart(). It will write the message as the script is compiled.

Mem=setmemorymax(1115)

imagesource("Earth's_Location_in_the_Universe_(JPEG).jpg")

WriteFileStart("Mem.log", "Mem") # Write as script is compiled

pwidth  = 240  #  set to desired
pheight = 136  #  output resolution
pFrames = 1440  #  Output duration

pwidthF=float(pwidth)
pheightF=float(pheight)
 
bicubicresize(width()*pheight/height(),pheight)
trim(1,1).loop(pFrames*3)

animate( 0,pFrames*3, "bicubicresize", pwidth,pheight, .33,.33, 0.0,            0.0, pwidth,pheight, 
 \                                     pwidth,pheight, .33,.33, width()-pwidthF,0.0, pwidth,pheight)

return(last)

[Asmodian]

Originally Posted by pshute

Data locality? Surely it still all has to be in memory, and it still has to do the work of cropping each frame out?

cretindesalpes is talking about the order it is accessed in memory, give it a try.

http://en.wikipedia.org/wiki/Locality_of_reference

[IanB]

As for data locality, yes it will all be in memory, but for CPU cache performance the L1 and L2 cache locality for this style of script will be better if the scroll was vertical instead of horizontal. i.e. all the bytes for a crop session are together.

Of course you need to balance speeding 1 line of the script with adding a TurnLeft() at the end. So the gain may actually be a loss.