Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 7th September 2015, 06:28   #1  |  Link
tsk1979
Registered User
 
Join Date: Nov 2013
Posts: 20
Avisynth and virtualdub 32bit, going OOM

I am making a timelapse.

I have various sequences ranging from 450-700 frames.

So I just go with
a1=imagesource(...)
a2=imagesource(...)
.
.
Then I dissolve a1,a2.... and so on

I am seeing that when processing longer sequence (700 images of 1080P) size, it goes OOM.

I'm using 32bit because I need MSU deflicker plugin.
I am not sure if I am hitting the 2GB limit too. It seems odd that ImageSource will hit the memory limit.

Machine is Xeon i7 1231v3 and RAM is 16GB. I also have a Nvidia medium end graphics card(not that it matters).

Any idea I can figure out how much memory did my process request? Did it request more than 2GB?

Alternatively, I can assemble a video in 64bit and then run deflicker by reading that video in and then reading it out, but that is going to degrade quality(unless I go for uncompressed video(which will be massive running into many GBs)

Reading that video in 32bit is going to give me more problems I guess.

Those who do timelapse with 4000-5000 frames, how do you do it?
tsk1979 is offline   Reply With Quote
Old 7th September 2015, 06:57   #2  |  Link
vivan
/人 ◕ ‿‿ ◕ 人\
 
Join Date: May 2011
Location: Russia
Posts: 649
Dealing with a lot of clips is rather painful in avisynth.
You can rename all files and load them all a once, check examples here http://avisynth.nl/index.php/ImageSource
vivan is offline   Reply With Quote
Old 7th September 2015, 06:58   #3  |  Link
raffriff42
Retried Guesser
 
raffriff42's Avatar
 
Join Date: Jun 2012
Posts: 1,377
Right, if ImageSource is like AVISource, there's a limit of around 20-30 instances in 32bit no matter how much RAM you have.
http://forum.doom9.org/showthread.php?t=131687

Can you read all the images into one clip, then trim it into subclips and dissolve those...
m=imagesource(...)
a1=m.Trim(0, 500)
a2=m.Trim(501-999)
...
raffriff42 is offline   Reply With Quote
Old 7th September 2015, 07:41   #4  |  Link
tsk1979
Registered User
 
Join Date: Nov 2013
Posts: 20
Thanks, so do you recommend I have one Imagesource command reading about 3700 frames, then trim them and then dissolve?

I was using around 9 imagesource commands. I guess that is the culprit.

Now I have named my files from 1..480 etc., and kept in different folders. I guess I need to rename them and then put them in one folder.

Lets see how that goes!

Alternatively I was thinking I will create 9 different uncompressed avi's and then load them 9 different avisource/directshowsource, but I guess its going to go boom on me.

I will try the single command route.

I was thinking having too many images in a single imagesource was the problem. But I guess having 9 imagesource commands was the problem

Thanks!
tsk1979 is offline   Reply With Quote
Old 7th September 2015, 10:30   #5  |  Link
davidhorman
I'm the Doctor
 
Join Date: Jan 2004
Posts: 1,414
You could try:

Code:
imagesource(...) # sequence of images
changefps(3*framerate) # pad it out a bit
convertfps(10*framerate) # 10 frame dissolve (fixed)
assumefps(30) # or whatever the target framerate is
Restrictions: "image on" time is can only be an integer multiple of image dissolve time.
__________________
My AviSynth filters / I'm the Doctor

Last edited by davidhorman; 7th September 2015 at 17:02. Reason: edited to fix - changed changefps to convertfps
davidhorman is offline   Reply With Quote
Old 7th September 2015, 11:18   #6  |  Link
creaothceann
Registered User
 
Join Date: Jul 2010
Location: Germany
Posts: 360
Quote:
Originally Posted by tsk1979 View Post
Now I have named my files from 1..480 etc., and kept in different folders. I guess I need to rename them and then put them in one folder.

Lets see how that goes!

Renamer has an option for that: add a rule, insert meta tag :File_FolderName:, add all files, rename, move all files into one directory.
creaothceann is offline   Reply With Quote
Old 7th September 2015, 13:00   #7  |  Link
tsk1979
Registered User
 
Join Date: Nov 2013
Posts: 20
Quote:
Originally Posted by davidhorman View Post
You could try:

Code:
imagesource(...) # sequence of images
changefps(3*framerate) # pad it out a bit
changefps(10*framerate) # 10 frame dissolve
assumefps(30) # or whatever the target framerate is
Restrictions: "image on" time is can only be an integer multiple of image dissolve time.
I could not understand this. You are saying I should create 9 different videos using this and then just add them(lossless adding)
tsk1979 is offline   Reply With Quote
Old 7th September 2015, 16:04   #8  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,380
Quote:
Originally Posted by davidhorman View Post
Code:
changefps(10*framerate) # 10 frame dissolve
???

All this does is add 9 duplicates to every frame.
Perhaps you meant ConvertFPS(0.1*framerate) ?
But ConvertFPS() only blends two adjacent frames, and in any case the output frame rate must be at least two-thirds of the source rate.
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 7th September 2015, 16:37   #9  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,412
Quote:
Originally Posted by tsk1979 View Post
Alternatively I was thinking I will create 9 different uncompressed avi's and then load them 9 different avisource/directshowsource, but I guess its going to go boom on me.
You don't have to use "uncompressed" ; you can use lossless compression such as lagarith, ut video codec, FFV1 , even x264 in lossless mode. Expect filesizes to be ~ 1/3 to 1/2 of uncompressed

Do you use the deflicker plugin after applying the dissolves? That can sometimes cause problems (not in terms of the memory issue, but in terms of the deflicker algorithm)

Quote:
Those who do timelapse with 4000-5000 frames, how do you do it?
Timelapses work ok in NLE's. Maybe because of x64, maybe because of better memory handling. I think even the x86 versions handle them ok - maybe they "release" memory in a better way, or maybe they do a better job of caching to disk, not sure
poisondeathray is offline   Reply With Quote
Old 7th September 2015, 17:03   #10  |  Link
davidhorman
I'm the Doctor
 
Join Date: Jan 2004
Posts: 1,414
I probably misunderstood your task.
__________________
My AviSynth filters / I'm the Doctor
davidhorman is offline   Reply With Quote
Old 7th September 2015, 18:29   #11  |  Link
tsk1979
Registered User
 
Join Date: Nov 2013
Posts: 20
Quote:
Originally Posted by poisondeathray View Post
You don't have to use "uncompressed" ; you can use lossless compression such as lagarith, ut video codec, FFV1 , even x264 in lossless mode. Expect filesizes to be ~ 1/3 to 1/2 of uncompressed

Do you use the deflicker plugin after applying the dissolves? That can sometimes cause problems (not in terms of the memory issue, but in terms of the deflicker algorithm)



Timelapses work ok in NLE's. Maybe because of x64, maybe because of better memory handling. I think even the x86 versions handle them ok - maybe they "release" memory in a better way, or maybe they do a better job of caching to disk, not sure
Yes, I apply deflicker in virtualdub, so I presume it will be applied after dissolve since I load script through virtualdub.


I can do this.
Use 9 different imagesource, no filters, create a lossless compressed video x264

Then I just pass it to directshosource with all the filters I want?
tsk1979 is offline   Reply With Quote
Old 10th September 2015, 12:32   #12  |  Link
vivan
/人 ◕ ‿‿ ◕ 人\
 
Join Date: May 2011
Location: Russia
Posts: 649
To be honest I was thinking about hundreds of ImageSource calls, not just 9... It shouldn't be a problem. Do you do any other processing other then Dissolve() (and Deflicker in VDub)? MT?

As for memory consumption - you could try AVSMeter.

For lossless I prefer lossless codecs like new MagicYUV or UT Video - they are very fast (VirtualDub will be able to open resulting .avi files, in avisynth you should use AVISource). With x264 don't use directshowsource (it depends on system codecs and might be unreliable - like skipping/inserting frames) - use ffvideosource or LWLibavVideoSource instead.
vivan is offline   Reply With Quote
Reply

Tags
timelapse, virtualdub

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 20:31.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.