Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Closed Thread
 
Thread Tools Search this Thread Display Modes
Old 25th October 2014, 05:58   #841  |  Link
ryrynz
Registered User
 
ryrynz's Avatar
 
Join Date: Mar 2009
Posts: 3,645
Quote:
Originally Posted by chainik_svp View Post
Is it dead or alive? o_O
In other words, if I think I found a bug - should I try to fix it myself or post here and wait for support?
Pretty much dead. I wouldn't rely on anyone getting to this in a timely fashion, if you're able to work on it go for it IMO.
ryrynz is offline  
Old 25th October 2014, 11:44   #842  |  Link
innocenat
Registered User
 
innocenat's Avatar
 
Join Date: Dec 2011
Posts: 77
Quote:
Originally Posted by chainik_svp View Post
Is it dead or alive? o_O
In other words, if I think I found a bug - should I try to fix it myself or post here and wait for support?

===
Short version of issue is: x64 MT version of AVS+ doesn't free memory so if I want to run some memory consuming script inside MPC-HC x64 + ffdshow x64 it will eat some amount of memory on each seek.


enter simple script into ffdshow:
Code:
SetFilterMTMode("",2)
SetFilterMTMode("ffdShow_source",3)
ffdShow_source()
BilinearResize(3000,1500)
BilinearResize(1920,1080)
Prefetch(10)
and then just check and uncheck "Avisynth" check box while looking at Task Manager memory graph

===
and it's not an issue with 32-bit version
Do you have revision number/where did you get the build? If it's resizer issue the I can look into it, but no promise since ultim isn't here to release it.
__________________
AviSynth+
innocenat is offline  
Old 25th October 2014, 12:06   #843  |  Link
chainik_svp
Registered User
 
Join Date: Mar 2012
Location: Saint-Petersburg
Posts: 239
It's not a resizer issue, BilinearResize() is used just as an example to illustrate it's not a some filter issue but AVS+ in general.
Still it can be not AVS+ x64 issue but ffdshow x64 but I prefer to think that memory allocated by AVS+ should be freed by AVS+.

where did you get the build?

latest sources from MT branch built by me
__________________
SVPflow motion interpolation
chainik_svp is offline  
Old 25th October 2014, 13:34   #844  |  Link
chainik_svp
Registered User
 
Join Date: Mar 2012
Location: Saint-Petersburg
Posts: 239
Code:
 [for every frame]
  {
    VideoFrame *frame = it->second;
    if (frame->vfb->refcount == 0)
      delete frame->vfb;

	delete frame;
  }
why "if (frame->vfb->refcount == 0)" ??
vfb->refcount decremented in VideoFrame destructor so it can be (and it IS) >0
and ~VideoFrame doesn't delete vfb but just releases it

=====
Fixed it, see https://github.com/AviSynth/AviSynthPlus/issues/49

Is there any chance to include this fix to the main branch and release a new official build?
__________________
SVPflow motion interpolation

Last edited by chainik_svp; 25th October 2014 at 15:20.
chainik_svp is offline  
Old 25th October 2014, 20:44   #845  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,556
Ouch. That makes sense, now I know why I was getting out-of-memory when testing some script variations last month.
foxyshadis is offline  
Old 25th October 2014, 21:49   #846  |  Link
chainik_svp
Registered User
 
Join Date: Mar 2012
Location: Saint-Petersburg
Posts: 239
this isn't all the truth

if the filter was instantiated - it'll never be deleted...

in other words if you'll write some filter doing something like this:

SomeFilter::SomeFilter()
{ buffer = new BYTE[1000000000]; }

SomeFilter::~SomeFilter()
{ delete[] buffer; }

than you'll end with 1 GB of memory stolen cause it'll never be freed
__________________
SVPflow motion interpolation

Last edited by chainik_svp; 25th October 2014 at 21:54.
chainik_svp is offline  
Old 25th October 2014, 22:37   #847  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 1,416
My question is: why is this specific to 64-bit, though? The snippet doesn't look specialized to me (noting, of course, I have no real knowledge of what you have to do for 64-bit vs. 32-bit in the code itself).

What I'm getting at is, is there any chance that it's caused by being miscompiled? Does the behavior change with different compilers (VC2012 vs. VC2013 vs. ICL), or different optimization levels in the same compiler?

(although, yes, I recognize the fact that if it's being miscompiled there's a chance of it being from a legitimate bug that simply exposes it, and the bug needs fixing)
qyot27 is offline  
Old 25th October 2014, 22:57   #848  |  Link
chainik_svp
Registered User
 
Join Date: Mar 2012
Location: Saint-Petersburg
Posts: 239
My question is: why is this specific to 64-bit, though?

asked myself the very same question, still no answer
may be there's a difference in procedure of loading/unloading DLLs between x32 and x64 modes oO

What I'm getting at is, is there any chance that it's caused by being miscompiled?

not a chance, I can see from the code that some objects are NOT deleted by destructor

===
in fact these bugs are very uncommon
I mean the only chance to face them is to use AVS+ inside ffdshow - then you can load/unload AVS several times in the same process
__________________
SVPflow motion interpolation

Last edited by chainik_svp; 25th October 2014 at 23:08.
chainik_svp is offline  
Old 26th October 2014, 16:04   #849  |  Link
chainik_svp
Registered User
 
Join Date: Mar 2012
Location: Saint-Petersburg
Posts: 239
Just wasted all the Sunday to find out that ScriptEnvironment:: prefetcher is not deleted in destructor.
Which leads to all MT-enabled filter objects are not freed.
__________________
SVPflow motion interpolation
chainik_svp is offline  
Old 22nd December 2014, 22:05   #850  |  Link
mapg
Registered User
 
Join Date: Dec 2014
Posts: 8
AVISource: Image frozen when riff index > 256 is reached

Hi,

Let me explain the issue.

Maximum riff index added by ffmpeg in the avi video container is 256, after that point a warning is shown as: "Invalid riff index 257 > 256", later ... "Invalid riff index 258 > 256" and so on ...

Well, when I encode using your Avisynth+ 64-bit and AVISource, when encoder (i.e: x264) reaches riff index 257, the image stays frozen from there to the end of video, namely, scenes stop until the end of video.

Therefore if you are encoding an uncompressed AVI video which is bigger than 256~260 GB (approximately), and the input format is a AVS script (using Avisynth+ and JUST the command: AVISource), at some point of time (when riff index > 256 is reached), the video is frozen, damaged after all.

This problem doesn't happen if I use Avisynth+ and tag DirectShowSource instead of AVISource.

I think that there are two options:

1) Avisynth+ team adds some fix to AVISource to avoid this issue.
2) ffmpeg team allows adding riff indexes bigger than 256 to AVIs.

What do you think?

Let me suggest that you should fix it at your end too.

Kindest regards,
Mapg

BTW: I didn't test if this is also happening in Avisynth+ 32-bit
mapg is offline  
Old 23rd December 2014, 08:14   #851  |  Link
colours
Registered User
 
colours's Avatar
 
Join Date: Mar 2014
Posts: 308
Creating a 257 GB AVI file just to verify this report would be somewhat difficult for the zero active Avisynth+ developers so don't expect any updates on this end.

Besides, this looks more like an FFmpeg issue. FFmpeg would be able to handle over a hundred million RIFF chunks if only one line was changed.

The OpenDML spec doesn't seem to indicate any upper bound on the number of chunks, but I didn't bother reading through it very carefully. It would be pretty weird if there was an artificial 256-chunk limit, though.
__________________
Say no to AviSynth 2.5.8 and DirectShowSource!

Last edited by colours; 23rd December 2014 at 08:18.
colours is offline  
Old 23rd December 2014, 10:11   #852  |  Link
mapg
Registered User
 
Join Date: Dec 2014
Posts: 8
As you can see here, there is such limit in ffmpeg (libavformat/avienc.c) ...

Code:
if (avi->riff_id > AVI_MASTER_INDEX_SIZE) {
      av_log(s, AV_LOG_ERROR, "Invalid riff index %d > %d\n", avi->riff_id, AVI_MASTER_INDEX_SIZE);
      return AVERROR(EINVAL);
}
Yes, it's certainly weird as none of the most professional video editing software packages add this unfortunate feature.

I will try to forward this issue to ffmpeg team, but I think that AviSynth+ should have a look at this too.

Last edited by mapg; 23rd December 2014 at 10:15.
mapg is offline  
Old 23rd December 2014, 17:05   #853  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 1,416
AVISource relies on the VfW interface to do its parsing (and does so by using VirtualDub's parser, IIRC). If you can open it in VDub but not AviSynth+, then it's an AVISource issue that would need to be resolved by updating AVISource. While FFmpeg might have that limit, it would likely depend on the parser. The DirectShow AVI parser obviously wouldn't have a limit like that if DirectShowSource works. If VDub doesn't like it either, it's probably in VfW and there's little that can be done about it short of finding a third party AVI demuxer for VfW (but honestly, that's probably more trouble than it's worth).
qyot27 is offline  
Old 3rd January 2015, 11:04   #854  |  Link
mapg
Registered User
 
Join Date: Dec 2014
Posts: 8
I have tested the issue using FFMS2 and works well, namely there isn't any frozen image once AVI_MASTER_INDEX_SIZE (256GB) is reached.

Definitively the problem is in AVISource from Avisynth+

Mapg

Last edited by mapg; 4th January 2015 at 12:44.
mapg is offline  
Old 25th January 2015, 18:55   #855  |  Link
sl1pkn07
Pajas Mentales...
 
Join Date: Dec 2004
Location: Spanishtán
Posts: 496
great news (?)

since wine 1.7.25, can run Avsynth +r1689!



only need in winecfg set msvcr110.dll and msvcp110.dll to 'native,built'

https://bugs.winehq.org/show_bug.cgi?id=17273#c61

greetings
sl1pkn07 is offline  
Old 28th January 2015, 07:25   #856  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 1,416
Also confirmed working with VC2013 builds (msvcp120.dll and msvcr120.dll).

But getting it working in Wine 1.7 requires the workaround I posted earlier. Trying to install the 2012 and 2013 runtimes from vcredist_*.exe fails, requiring the user to make sure they get the DLLs themselves and install them (read: copy into system32). The built-in version errors out on the GetConcurrency function, but at least it can actually use the Microsoft runtimes now and operate.
qyot27 is offline  
Old 29th January 2015, 20:08   #857  |  Link
sl1pkn07
Pajas Mentales...
 
Join Date: Dec 2004
Location: Spanishtán
Posts: 496
I tried the 64bits version (with 'win64' winearch) and don't work. tested with 'Windows XP' and 'windows 7' mode

the 32bits only work in 'Windows XP' mode (see the capture avobe). but fail in 'Windows 7' mode

for installation (include vcredis 2012), first need set the wine with 'windows 7' mode, install avs+ r1689 from line0, then back to 'Windows XP' mode and set msvc{r,p}110.dll to native with winecfg

tested with 3 clean prefixes, wine only 32bits (winearch win32), wine only 64bits (winearch win64) and wine mixed (clear winearch variable)

the message is: failed to load avisynth.dll

greetings
sl1pkn07 is offline  
Old 11th February 2015, 22:12   #858  |  Link
martin53
Registered User
 
Join Date: Mar 2007
Posts: 407
I know similar questions have been asked in this forum more than once, but that was before AviSynth+, which released the tight doctrine a bit, and maybe less specific.

Is it possible in general (and worthwile...) that the gurus add a Dictionary element to the VideoInfo interface, together with methods to add and delete a key/item pair, and to get the item for a specific key, and also make the latter methods available to scripts?

Specifically, a pair of functions could then exchange information through this interface.

Say
function 1 prepares a 3d- or 16-bit, or certain colorspace frame so that a standard filter chain, which is older and not 100% compatible, can handle this clip. E.g. 3d side-by-side video could be cropped and interleaved, then processed as double frame rate 2d clip.
Function 2 stands at the end of the filter chain, and will automatically reverse the "tunneling", if it knows that there is need to. Some plugins used "hints" for that, i.e. manipulated certain pixel LSBs, etc; but this is not robust against filtering.

In my imagination, function 1 could call Add(clip, key, string) to add the dictionary entry (key,string) to the clip's VidoInfo, and this would be forwarded through the filter chain like the other clip properties.
Function 2 could call ExistsKey(clip, key), and/or GetKey(clip, key), which would signal, resp. return the string.
This could look like
Code:
#Assume Function1 calls MediaInfo to get information about the clip display ratio, which is not width:height ratio, i.e. nonsquare pixels
#Function1 gives the clip this freename property
AddKey("DisplayRatio", string(retrieved_display_ratio)) 

... #standard filter chain with unchanged width/height
... #deblocking and other filters need unstretched, uncropped input

#function2
GetKey(clip, "DisplayRatio")
   \ ? BiCubicResize(...)
   \ : last
#as the last filter in the chain, function2 sets the output width/height ratio in order to meet display ratio with square pixels
...
One more example to make it as clear as possible: There is a function 'ConvertBackToYUY2()' which should be used when the clip originally was converted from YUY2 to a different color space. It would be more elegant to attach the property that it was originally YUY2 to a clip - compared to today's script language limits.

EDIT: here is another example for the usefulness of arbitrary clip properties ('tunneled' in this case)

Last edited by martin53; 15th February 2015 at 10:55.
martin53 is offline  
Old 2nd March 2015, 12:33   #859  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 1,416
Time for a test build, I guess:

AviSynth+ r1718

Basically, I merged in most of the 22 commits that make up 2.6 RC1. Some are mostly applied 1:1 (including the 7 HTML docs commits), others are adapted (a couple of them heavily cut down from their original form). Four of the commits were outright rejected because they're irrelevant for AviSynth+.

This is also probably a better option than continuing to rely on r1689, since there were some MT fixes applied after r1689.


Built with Visual Studio 2013, and it includes both 32-bit and 64-bit .dlls (no 64-bit DirectShowSource because I didn't feel like trying to get the SDKs lined up first). Note that I do not have access to a 64-bit Windows install, so those of you that do, please confirm that the 64-bit .dlls work.
qyot27 is offline  
Old 3rd March 2015, 00:56   #860  |  Link
burfadel
Registered User
 
Join Date: Aug 2006
Posts: 2,229
Thanks, been looking forward to an update!

However, the download doesn't work. It tries to download then fails (tried different browsers). Any chance of another mirror?

EDIT:
Kept trying, then Mediafire showed the 'repair download' screen. After that it worked.

Last edited by burfadel; 3rd March 2015 at 00:59.
burfadel is offline  
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 06:31.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.