Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 8th November 2018, 23:46   #4301  |  Link
GMJCZP
Registered User
 
GMJCZP's Avatar
 
Join Date: Apr 2010
Location: I have a statue in Hakodate, Japan
Posts: 516
I have a problem with ImageWriter, when using the following script, for example:

Code:
ConverttoRGB24().ImageWriter("", 5,5, "bmp").ConvertToYV12()
I get an error with the DevIL library. I am using AVS + 2728. I never had problems with the old AVS.
__________________
By law and justice!

Flea Market
GMJCZP is offline   Reply With Quote
Old 9th November 2018, 08:48   #4302  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,729
Which is "an error"? Maybe it tells about the reason.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 9th November 2018, 17:37   #4303  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 1,068
Quote:
Originally Posted by StainlessS View Post
Yep.
With my test clip (854x480@30, 3775 Frames), memory usage max's out at about 1.73GB
I checked the 32 bit vdub2 process with SysInternals vmmap. It seems that we are near 2GB with this script and parameters. As the internal caches are getting filled (until the set limit) Windows suddenly cannot allocate memory. Even when I set the max cache size to the minimum of 64MB, it could not request new memory buffer from the system after a while. Either avisynth or virtualdub - whichever is unlucky - will feel deep unhappyness and crash. So I think the problem is not avisynth related.
pinterf is offline   Reply With Quote
Old 9th November 2018, 22:06   #4304  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 6,192
Quote:
So I think the problem is not avisynth related.
Thank you for looking in to this P, much appreciated.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 9th November 2018, 23:23   #4305  |  Link
GMJCZP
Registered User
 
GMJCZP's Avatar
 
Join Date: Apr 2010
Location: I have a statue in Hakodate, Japan
Posts: 516
Quote:
Originally Posted by LigH View Post
Which is "an error"? Maybe it tells about the reason.
I am using a borrowed PC because I have not had internet for several months, so I do not remember exactly how the message says, the issue is that I can not save the frame and it is something related to DevIL. I do not know if anyone can try to recreate the error using the script.
__________________
By law and justice!

Flea Market
GMJCZP is offline   Reply With Quote
Old 9th November 2018, 23:27   #4306  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 1,068
Perhaps it wants to write to c: root which is not allowed?
pinterf is offline   Reply With Quote
Old 9th November 2018, 23:32   #4307  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 6,192
Quote:
Originally Posted by pinterf View Post
Perhaps it wants to write to c: root which is not allowed?
Yeh[EDIT: good catch] , possibly. Maybe try,

Code:
Colorbars
ConverttoRGB24().ImageWriter(".\", 5,5, "bmp").ConvertToYV12()
".\" is current directory, and at least it works as expected here.

EDIT: With just "" here on W7x64Avs32, I got nothing, no output and no error message, perhaps reason that you could not
remember the error message

EDIT: "." alone did NOT work [EDIT: Same result as ""].
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 9th November 2018 at 23:49.
StainlessS is offline   Reply With Quote
Old 10th November 2018, 00:34   #4308  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 6,192
Quote:
Originally Posted by StainlessS View Post
EDIT: "." alone did NOT work [EDIT: Same result as ""].
Actually, I was in sub directory of D:\ drive ("D:\NewFolder"), and "." actually wrote to the root of D:\.

EDIT:
So far as I remember,
"D:" should be current directory on D drive,
"D:\" should be root directory on D drive,
"." should be current directory, current drive (also think ".\" should be same).

EDIT: Damn, forum keeps swallowing '\' characters.

EDIT: Also, ".." should be parent directory to current directory, think "..\" should be same.

EDIT:
Where below script in "D:\NewFolder\NewFolder"
This writes to root directory of D: (I think should be parent directory to current directory ie D:\NewFolder\)
Code:
Colorbars ConverttoRGB24().ImageWriter("..", 5,5, "bmp").ConvertToYV12()
This writes to D:\NewFolder (I think should be parent directory to current directory ie D:\NewFolder\)
Code:
Colorbars ConverttoRGB24().ImageWriter("..\", 5,5, "bmp").ConvertToYV12()
Something seems to be messed up somewhere, dont know if ImageWriter or Windows 7.

EDIT: Also "" should be interpreted same as "." and ".\", ie current directory.
Excepting for case where eg "D:\" specifies root directory of drive D, a trailing '\' just explicitly specifies that it is a directory
rather than a file, but "." and ".." already imply a directory where trailing "\" should be superfluous unless a node is appended after
the slash (ie another directory or file name node).
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 12th November 2018 at 01:30.
StainlessS is offline   Reply With Quote
Old 15th November 2018, 17:25   #4309  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Posts: 2,354
Weird issues with VDub plugin "LogoAway 4.01"

In one of my video conversion scripts I need to remove the station logo, for this I have been using LogoAway 4.01 by Krzysztof Wojdon for a long time. After switching to AVS+ I got random crashes using this filter, and luckily I believe I solved the issue, but it took me a long time and a lot of detective work...

My hardware and software setup first:

Core i5 3rd generation with 8GB RAM
Win7 64-bit
Avisynth+ r2728-MT 32-bit
SetMTMode.avsi from this link: http://publishwith.me/ep/pad/view/ro.rDkwcdWn4k9/latest
StaxRip v. 1.1.9.0 (last stable 32-bit version)
X264 r2935 32-bit

I use "Prefetch(4)" with this CPU (2 physical cores plus Hyperthreading). This normally runs stable. But after getting these random crashes I first made tests to find out which filter was to blame, and it was LogoAway.

It does not happen with all sources, and sometimes the crash occurs shortly after the conversion start, sometimes it happens in the middle and sometimes at the end. I get this Windows popup that X264.exe has stopped working and must be shut down.

I am quite sure that X2364 is not to blame, I tried 3 different builds of the same release without any difference. Reducing the AVS+ threads from 4 to 2 still gave me the crashes, but this time without the Windows popup. X264 just stopped working, CPU load went down to 0, no error message.

Removing the "SetMTMode.avsi" also made no difference (except slowing down encoding speed by 0.3 fps). What did fix it of course was to disable MT altogether, and forcing MT_SERIALIZED for LogoAway also worked, but speed was not any faster than without MT.

The final solution came just by trial and error, and I really do not understand why it works. If I specify MT_MULTI_INSTANCE explicitly for LogoAway either in the conversion script or in the "SetMTMode.avsi" then the crashes miraculously disappear. This is reproduceable each and every time.

How is this possible? The docs say that MT_MULTI_INSTANCE is the default MT mode which always gets used as long as no different MT mode is explicitly specified. So it should not make a difference if I specify MT_MULTI_INSTANCE in the script or not, right? But obviously it does...

Or does AVS+ treat VDub plugins differently from other filters?


Any thoughts?

Cheers
manolito

Last edited by manolito; 15th November 2018 at 17:35.
manolito is offline   Reply With Quote
Old 16th November 2018, 10:38   #4310  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 1,068
Quote:
Originally Posted by manolito View Post
How is this possible? The docs say that MT_MULTI_INSTANCE is the default MT mode which always gets used as long as no different MT mode is explicitly specified. So it should not make a difference if I specify MT_MULTI_INSTANCE in the script or not, right? But obviously it does...

Or does AVS+ treat VDub plugins differently from other filters?
Just looking at the code I see that no special MT mode is set for vdub filters, so I expect them to behave as MT_MULTI_INSTANCE. I'll check it.
pinterf is offline   Reply With Quote
Old 16th November 2018, 17:17   #4311  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 1,068
I could not reproduce, for me it works like MT_MULTI_INSTANCE. I'd need the source clip or a short fragment with that it still crashes. And a (minimized) script itself.
Maybe it's not the filter itself which fails, just when it appears in a specific sequence.
Once I had an issue when I was debugging TIVTC for weeks because it crashed x264, then it turned out that an mvtools2 function did not clear the processors's mmx state and conflicted with x264 and it was just a coincidence that tivtc seemed to be a culprit.
pinterf is offline   Reply With Quote
Old 16th November 2018, 19:02   #4312  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Posts: 2,354
Thanks pinterf for looking into it...

The source file I used for my tests was a huge 2 hour HD captured TV file. Please give me some time to find a smaller source which reliably crashes...

Otherwise I did not use any elaborate filters in my script. I should probably mention that for FineSharp I use old and proven versions of RemoveGrain (original by Kassandro from 2005) and MaskTools v2.0a48. Upgrading to later versions is not an option because I need to use the same scripts under WinXP with a non-SSE2 CPU.

I will report back soon...

Cheers
manolito
manolito is offline   Reply With Quote
Old 18th November 2018, 00:26   #4313  |  Link
wonkey_monkey
Formerly davidh*****
 
Join Date: Jan 2004
Posts: 1,570
A question about expr(), if someone can answer - when the AVX/SSE optimisations are enabled, it processes four (or more) pixels at once - is that correct? Like if you were adding the values of two clips, it would load several pixels into two SSE/AVX registers and add them together in one operation?
__________________
My AviSynth filters / I'm the Doctor
wonkey_monkey is offline   Reply With Quote
Old 18th November 2018, 01:54   #4314  |  Link
GMJCZP
Registered User
 
GMJCZP's Avatar
 
Join Date: Apr 2010
Location: I have a statue in Hakodate, Japan
Posts: 516
Thanks for the replies pinterf and TinMan, and sorry for responding late, but I still do not have internet.

On the AVS wiki page, when you talk about ImageWriter, give this example:

Code:
# Write frame 5 to "C: \ 000005.PNG"
ImageWriter ("", 5, 5, "png")
When putting the quotes "" I always worked with the old AVS, capturing the frame and saving it in the same place in the script, no matter where in the Hard Disk it is. When placing
Code:
".\"
It works for AVS +, so my question is why the difference between AVS and AVS+?

Note: the error message varies, which does not always appear, says:

ImageWriter: error 'Could not open file' in DevIL library writing file "000005.bmp"
DevIL version 0.

Until I got an error message with strange characters, the problem is the incompatibility of "" in ImageWriter with AVS+ , but not with AVS, why?
__________________
By law and justice!

Flea Market

Last edited by GMJCZP; 18th November 2018 at 02:00.
GMJCZP is offline   Reply With Quote
Old 18th November 2018, 09:18   #4315  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 1,068
Quote:
Originally Posted by davidhorman View Post
A question about expr(), if someone can answer - when the AVX/SSE optimisations are enabled, it processes four (or more) pixels at once - is that correct? Like if you were adding the values of two clips, it would load several pixels into two SSE/AVX registers and add them together in one operation?
Expr handles 2x4 pixels (XMM registers) or 2x8 pixels (AVX2) at a time. Code is generated for two sets of XMM/YMM registers unless optSingleMode=true - I introduced this parameter for curiosity, if it is any faster to use only a single register set - one lane. (Looking at the generated code of a slightly more complex expression it turned out that the registers were heavily swapped to and from temporal memory variables because the code required too many physical XMM/YMM registers but we had only a limited number of them).

Note that I added AVX2 - and not AVX - code generation to Expr, though the calculations are using 32bit floats inside. Many helper asm instructions I'm using are available only from AVX2. On the other hand, old processors with AVX-only support suffer from not having real 256 bit internally, e.g. memory access would still happen in 2x128bit mode which is sometimes slower than using 2x128bit memory load manually which can use different ports in parallel. At least for my old AVX-only i7 that was the case.

See optSSE2, optAVX2 and optSingleMode in http://avisynth.nl/index.php/Expr

EDIT:
In Avisynth+ frame alignment is 64 bytes since I added AVX2 codegen to Expr to allow the two-lane (2x256 bit YMM register) operation for 32 bit float-type inputs - and as a side effect allows painless AVX512 usage for present and future filters. Why 64 bytes? For float-type pixel input in order to use 2xYMM registers we have to read and write 512bits = 64 bytes at a time, which requires 64 byte line padding.

Later when one-lane mode (optSingleMode=true option) was put in the codegen core, it became possible to have all calculations in two-lane (2x256bit = 2x8 float pixels) mode, except the last chunk of a line which could be done either for 8 pixels (one 256 bit YMM register, like in optSingleMode=true) or 2x8 pixels (2x256bit YMM registers), depending on the source clip width. Using such an adaptive treating of the last pixels it would no longer require 64 byte alignment and line padding, because the old 32 bytes padding would have been enough, anyway I left the 64 bytes frame aligment in general thinking of the plenty future filters using AVX512.

Last edited by pinterf; 18th November 2018 at 10:07. Reason: add info
pinterf is offline   Reply With Quote
Old 18th November 2018, 12:18   #4316  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 1,068
Quote:
Originally Posted by GMJCZP View Post
On the AVS wiki page, when you talk about ImageWriter, give this example:

Code:
# Write frame 5 to "C: \ 000005.PNG"
ImageWriter ("", 5, 5, "png")
... so my question is why the difference between AVS and AVS+?

Until I got an error message with strange characters, the problem is the incompatibility of "" in ImageWriter with AVS+ , but not with AVS, why?
Yep, avs+ worked like wiki says, but that part of wiki is wrong.
Code:
"c:\"
is the default value of "file" parameter if it is not provided. When file parameter is given and it's empty string it should get the current directory.

Probably when it was rewritten for avs+ from the classic avs source, they missed that special case.

EDIT: fixed the example on wiki

And yes, it should work (and now the git dev version works) like classic avisynth does.

Last edited by pinterf; 18th November 2018 at 13:21. Reason: wiki fix
pinterf is offline   Reply With Quote
Old 18th November 2018, 20:34   #4317  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Posts: 2,354
Quote:
Originally Posted by manolito View Post
I will report back soon...
Finding a shorter source which still revealed the issue was unsuccessful. On two long source files I did get the crashes during the first 10 minutes into the clip, but cutting out the first 15 minutes did not bring any results. With the cut-out sources the conversions went without problems. So the source files themselves are not to blame.

Specifying MT_MULTI_INSTANCE explicitly for the LogoAway filter again seemed to make the crashes disappear first, but on a very long source file with a length of more than 2 and a half hours the crashes did return towards the end of the clip.

My conclusion is that the LogoAway filter needs MT_SERIALZED to work reliably. Sorry for the false alarm...


Cheers
manolito
manolito is offline   Reply With Quote
Old 19th November 2018, 09:04   #4318  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 1,068
O.K. then. By seeing the source code it could be established whether it behaves well in MT environment or not but I suppose we don't have it.
pinterf is offline   Reply With Quote
Old 19th November 2018, 12:36   #4319  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 6,192
Quote:
but I suppose we don't have it
I found this, dont know if he has the original source (from the VirtualDub2 thread).

by b2kguga,
Quote:
Im giving a try building a new version of LogoAway. So far, the rebuild is doing well. Im doing it in assembly and debugging it also with my own assembler. One of the things im having difficulties to understand is that inside my ConfigDialog it have a call to the toggle function of Vdub.
https://forum.doom9.org/showthread.p...01#post1777801

EDIT: b2kguga last activity April 2017, so maybe a bum steer.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 19th November 2018 at 12:40.
StainlessS is offline   Reply With Quote
Old 22nd November 2018, 01:47   #4320  |  Link
GMJCZP
Registered User
 
GMJCZP's Avatar
 
Join Date: Apr 2010
Location: I have a statue in Hakodate, Japan
Posts: 516
Quote:
Originally Posted by pinterf View Post
Yep, avs+ worked like wiki says, but that part of wiki is wrong.
Code:
"c:\"
is the default value of "file" parameter if it is not provided. When file parameter is given and it's empty string it should get the current directory.

Probably when it was rewritten for avs+ from the classic avs source, they missed that special case.

EDIT: fixed the example on wiki

And yes, it should work (and now the git dev version works) like classic avisynth does.
Thanks pinterf.
What you say means that the next version of avs+ will admit "" as it does the classic avs?
__________________
By law and justice!

Flea Market
GMJCZP is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 04:21.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.