Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 6th February 2019, 08:16   #4481  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 1,175
Quote:
Originally Posted by StainlessS View Post
EDIT: Or in full as here: (3 out of the 4 original lines look messed up to me)
Code:
//RGB:
      if constexpr(mode == LIGHTEN) {
        alpha = luma_ovr > (luma_src + thresh) ? alpha : 0;    // As original
      } else { // EDIT: DARKEN
        alpha = luma_ovr < (luma_src - thresh) ? alpha : 0
      }


//YUV:
      if constexpr (mode == LIGHTEN) {
        alpha_mask = ovr > (src + thresh) ? level : 0;
      }
      else { // EDIT: DARKEN
        alpha_mask = ovr < (src - thresh) ? level : 0;
      }
I spent too much time yesterday with not understanding what really happens and why my code produces different results than existing rgb version. I was thinking it over by a bottle of dark Staropramen and finally voted for this very same logic.

"Where overlay is brigher by threshold" =>
Where overlay is brigther by 10 =>
Where overlay > src + 10

and "Where overlay is darker by threshold" =>
Where overlay is darker by 10 =>
Where overlay < src - 10
pinterf is offline   Reply With Quote
Old 6th February 2019, 08:32   #4482  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,833
Powered by beer™
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 7th February 2019, 08:01   #4483  |  Link
ajp_anton
Registered User
 
ajp_anton's Avatar
 
Join Date: Aug 2006
Location: Stockholm/Helsinki
Posts: 775
Quote:
Originally Posted by StainlessS View Post
As far as comments concerned, below is demo of how handy '[* ... *]' is,
Code:
code...
Its use totally evaded me until RaffRiff42 pointed it out some time ago [I had even forgot that it existed as a comment option], the nesting comments can be very useful.

EDIT: Above is not prototype for current posted DDG, will be as above in next version.
But I would love to be able to do
Code:
Function DropDeadGorgeous(clip c,String DB,Int "ScanAhead", Int "X",Int "Y",Int"W",Int "H",Bool "Show", Bool "Verb",
    \
    \ Int   "Prefilter",                                                                               # Prefilter
    \ Int   "SPad",     Int  "SPel",         Bool "SChroma",    Int "SSharp",       Int "SRFilter",    # MSuper
    \ Int   "ABlkSize", Int  "ABlkSizeV",                                                              # MAnalyse
    \ Int   "ASearch",  Int  "ASearchParam", Int  "APelSearch",                                        # MAnalyse
    \ Bool  "AChroma",  Bool "ATrueMotion",                                                            # MAnalyse
    \ Int   "AOverlap", Int  "AOverlapV",                                                              # MAnalyse
    \ Int   "ADct",     Bool "ATryMany",                                                               # MAnalyse
    \ Int   "RthSAD",   Int  "RBlkSize",     Int  "RBlkSizeV",                                         # MRecalculate
    \ Int   "RSearch",  Int  "RSearchParam",                                                           # MRecalculate
    \ Bool  "RChroma",  Bool "RTrueMotion",                                                            # MRecalculate
    \ Int   "ROverlap", Int  "ROverlapV",    Int  "RDct",                                              # MRecalculate
    \ Float "Iml",      Bool "IBlend",       Int  "IthSCD1",    Int "IthSCD2",                         # MFlowInter
    \
    \ Int   "SOSthSCD2"
    \ )
# is just a lot more handy when doing some quick coding, especially when you just want to quickly comment and un-comment a single line in the middle of a chain of \'s. And # is easier to write, with a european keyboard you need a lot of awkward keypresses with altgr for [ and shift for *.

Last edited by ajp_anton; 7th February 2019 at 08:03.
ajp_anton is offline   Reply With Quote
Old 7th February 2019, 09:10   #4484  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 6,746
@ajp_anton,

I imagine that your requirement would involve quite a lot of tricky work and probably at multiple places in the parser source code (with potential for lots of new and exciting bugs for the user, even in long existing scripts).
EDIT: Not sure if it would be worth the risk.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 7th February 2019 at 09:35.
StainlessS is offline   Reply With Quote
Old 10th February 2019, 20:07   #4485  |  Link
almosely
Registered User
 
Join Date: Dec 2006
Location: Germany
Posts: 43
Hi,

after migrating from AviSynth 2.6.0 MT (SEt) (x86) to AviSynth+ 0.1.0 r2772 MT (x64) - and changing some filters - I discovered some things, that should be helpful to know for everybody else.

1) I switched from DGSource (from DGDecNV 2052) to DGSourceIM (beta 50) and discovered, that DGSourceIM is very unstable in general. Whether I used AVS 2.6.0 (x86) in ST/MT-Mode or AVS+ (x64) ST/MT, my encodings crashed (started with Simple x264 Launcher) at random times (error message something like "encodin process is not responding anymore" within Simple Launcher), but only full encodes, never compression or AVSMeter tests. So I switched back to DGSource and let my GeForce GTX 660 Ti do the decoding job. That is completely stable again.

2) I tried to save energy. That has been the reason in first place to switch to AVS+ and exchange DGSource against DGSourceIM too (I have a Core i5-3470 CPU with integrated Intel HD Graphics 2500 iGPU). To get that running, I have to enable Multi-Monitor-Support for my iGPU within my ASUS-BIOS (as the second GPU) and than enable a second imaginary screen output within Windows 7 to the HD Graphics, so that QuickSync (and the iGPU in general) became available at all. I did that becaus the GTX consumed 24 Watts extra only for decoding with DGSource and another 9 Watts extra for using FFT3DGPU. DGSourceIM is using just 1 Watt extra for the decoding job and FFT3DFilter (CPU based) is slowing down the encoding just 9%. So I was saving aprox. 30-50% of energy for the encoding task in the end. But, because of the strong unreliability of DGSourcIM I had to switch back the DGSource (and my GTX). Then I discoverd, that the GTX is switching from it's P8 Performance State (lowest) to P0 (highest) at the moment of using DGSource or FFT3DGPU, which results in that tremendously insane energy consuming. It is not switching away from P8 when decoding video-clips within Firefox, but it is also switching when using LAV-Filters within MPC-HC btw. So, I discovered a neat function of the Nvida Inspector tool, called Multi Display Power Saver (available through right-clicking the button "Show Overclocking"). Within there I restricted the GTX to the permanently use of the P8 Performance Level and let the Multi Monitor Power Saver autostart with Windows 7. That reduced the power consuming to the level of using the HD Graphics for decoding with DGSourceIM, but using DGSource of course. And there are no setbacks at all by letting the GTX running with P8 state permanently. There's also an option, to let the GTX run with either P5 or P0, when a specific .exe file is running, so just put the .exe of a game etc. as an exception there, and everythin runs fast and sound where it's needed. FFT3DGPU is running too slow with P8-State, but maybe fast enough with P5 - but, I do not use FFT3DGPU anymore, because FFT3DFilter is saving much more energy and is way better quality wise.

3) While getting used to AVS+ I discovered the mtmodes.avsi file, where a lot of MT-Modes are tested and predefined already. There's the need to change something (for everybody):

SetFilterMTMode("CompTest", MT_SERIALIZED)
SetFilterMTMode("ColorMatrix", MT_SERIALIZED)
SetFilterMTMode("RequestLinear", MT_SERIALIZED)
SetFilterMTMode("GradFun3", MT_MULTI_INSTANCE)

and remove the the following at the very end:

if (FunctionExists("avstp_set_threads")) {
# this isn't actually optimal, because it will also disable avstp threads if running
# on a single-threaded filter chain
avstp_set_threads(0, 1)

and put instead the following line within your personal .avs-script at the end, right before Prefetch(), but only when activating MT-Mode (using prefetch).

avstp_set_threads(1)

AVS+ is running faster (and still stable) when letting avstp do it's internal mt-job when using GradFun3-Filter in ST-Mode. There's no need to deactivate it in general, only for MT. And this line has to be set at the "very" end of the avs-script, as pointed out by the developer of avstp.dll. It is working - I tested it with AVSMeter by watching the amount of threads.

ColorMatrix is running absolutely fine in MT-Mode within AVS+ (x64), but only it it's placed within a bubble of sequential frame order. So, to ensure that, RequestLinear has to be used and to be defined as MT_SERIALIZED. It makes no sense to use RequestLinear as MT_MULTI_INSTANCE. Therefore the following sript is working totally fine for me (I tested it multiple times with ST and MT encodings of the same clip) - no differences within the x264 logfiles. I did encounter differences when not using RequestLinear as MT_SERIALIZED of course.

DGSource()
CompTest(5, 60)
ColorMatrix(hints=true)
RequestLinear(rlim=50, clim=50)
...
avstp_set_threads(1)
Prefetch(3)
return last

That's working just fine as an example.

Last edited by almosely; 10th February 2019 at 20:25.
almosely is offline   Reply With Quote
Old 10th February 2019, 20:45   #4486  |  Link
videoh
Registered User
 
Join Date: Jul 2014
Posts: 711
Quote:
Originally Posted by almosely View Post
1) I switched from DGSource (from DGDecNV 2052) to DGSourceIM (beta 50) and discovered, that DGSourceIM is very unstable in general.
True. DGDecIM is deprecated due to crappy Intel support/drivers. I should withdraw it.

Thank you for your interest in DG tools!
videoh is offline   Reply With Quote
Old 11th February 2019, 12:16   #4487  |  Link
DJATOM
Registered User
 
DJATOM's Avatar
 
Join Date: Sep 2010
Location: Ukraine, Bohuslav
Posts: 152
Quote:
Originally Posted by videoh View Post
True. DGDecIM is deprecated due to crappy Intel support/drivers. I should withdraw it.

Thank you for your interest in DG tools!
Well, I hope you'll add SW decoding in DGDecodeNV someday or implement another tool for that. We only have LWLibavSource and DGDecodeIM(..., engine=2) options for encoding on servers w/o NVidia GPU.
__________________
Me on GitHub | My Telegram
DJATOM is offline   Reply With Quote
Old 12th February 2019, 10:43   #4488  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Location: Berlin, Germany
Posts: 2,498
@ almosely

Thanks very much for your post about AVS+ settings and mtmodes.avsi. The settings you suggested seem to completely avoid the crashes I had with AVS+ so far...

I often convert my DVB-T2 TV captures from HD HEVC to SD AVC (old school like yourself, I still watch my movies on a CRT TV set). I use StaxRip (older 32-bit version), my AVS scripts are quite basic, and I was interested in AVS+ solely for some speed gains through the MT capability. High bit depth and fancy color spaces are not my thing. I also use 32-bit tools exclusively, my laptop is a ThinkPad with a Core i5 3rd generation CPU and 8GB RAM.

But up to now I could not get it stable with AVS+, I randomly got crashes just like the ones you described (Win7-64). This happened with and without the mtmodes.avsi, taking out filters one by one did not help, only way to get it stable was to disable MT. So I repeatedly reverted to good old AVS 2.61.

Last night I applied your suggested changes and did 2 long conversions, and to my surprise everything was absolutely stable. Of course I need to do more conversions to be sure, but it does look very promising. I have no idea which one of your suggestions did the trick, though. My script does contain ColorMatrix, maybe this was the culprit. The other thing which really surprised me was that avstp_set_threads made a difference. For all I know my script does not use any avstp-aware filter, still the encode got a little bit faster when I added the command before the prefetch call. Do you have an explanation?

Whatever, thanks again for your tips. You should update the mtmodes.avsi here:
http://avisynth.nl/index.php/AviSynt...lling_MT_modes

so others can profit from the changes, too.


Cheers
manolito


//EDIT//
Oops, I forgot one thing...
All of the above is for using DSS2Mod as my source filter. My overall speed increase by using AVS+ with these settings is about 1fps for 4 threads. With 3 threads the speed is almost the same.

But when using ffms2 as my source filter (latest stable GitHub version 2.23.1) the encoding speed drops considerably after a few minutes into the encode. About 5fps slower than with AVS 2.61. Not good, what could cause this?

Last edited by manolito; 12th February 2019 at 11:37.
manolito is offline   Reply With Quote
Old 12th February 2019, 18:36   #4489  |  Link
Groucho2004
 
Groucho2004's Avatar
 
Join Date: Mar 2006
Location: A wretched hive of scum and villainy
Posts: 4,218
Quote:
Originally Posted by manolito View Post
But when using ffms2 as my source filter (latest stable GitHub version 2.23.1) the encoding speed drops considerably after a few minutes into the encode. About 5fps slower than with AVS 2.61. Not good, what could cause this?
Have you tried the "threads = 1" parameter with ffms2?
__________________
Groucho's Avisynth Stuff
Groucho2004 is offline   Reply With Quote
Old 13th February 2019, 01:50   #4490  |  Link
almosely
Registered User
 
Join Date: Dec 2006
Location: Germany
Posts: 43
@manolito

You're welcome! Nice, I could help someone with my discoveries :-)
almosely is offline   Reply With Quote
Old 13th February 2019, 03:04   #4491  |  Link
GMJCZP
Registered User
 
GMJCZP's Avatar
 
Join Date: Apr 2010
Location: I have a statue in Hakodate, Japan
Posts: 506
I took the liberty to prepare mtmodes based on almosely's observations. I post it here and not on its corresponding page to continue the tests and subsequent discussions. Thanks almosely.

Here
__________________
By law and justice!

Flea Market
GMJCZP is offline   Reply With Quote
Old 13th February 2019, 15:19   #4492  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Location: Berlin, Germany
Posts: 2,498
Quote:
Originally Posted by Groucho2004 View Post
Have you tried the "threads = 1" parameter with ffms2?
Thanks for the tip, I tried it, but this parameter really backfired...

With the default "threads=-1" which uses the number of logical cores reported by Windows (4 in my case) I got a speed of 21.47 fps. Using "threads=1" the speed dropped to 15.66 fps. With DSS2Mod the speed was 24.49 fps.

After many more speed benchmark tests I can say that the AVS+ MT feature is not all that useful when using very basic AVS scripts which I usually do. The real speed difference gets obvious when using more complex scripts like the MysteryX FrameRateConverter script. This script makes extensive use of MVTools, and here I get a speed difference of almost 80%.


Cheers
manolito
manolito is offline   Reply With Quote
Old 14th February 2019, 10:11   #4493  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,833
Hint: FFMS2 and L-SMASH Works may use different splitters, which also affects the following decoder. So compare FFVideoSource with LwLibavVideoSource (and possibly even LSMASHVideoSource for ISO Media containers, like MP4 / MOV / 3GPP) too.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 14th February 2019, 14:53   #4494  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Location: Berlin, Germany
Posts: 2,498
Yes, I will check this out on my next conversion later tonite...
Can you recommend a version which causes the least possible problems? (including download link) And would LSMASHVideoSource work for HEVC in an MKV and TS/MTS container?
manolito is offline   Reply With Quote
Old 15th February 2019, 08:02   #4495  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,833
1. MeGUI provides a well tested version. I won't guarantee for "the best that there is", though. Unfortunately, it is not regularly rebuilt, and the most recent version may have some flaws (e.g. using AviSynth 2.5 headers).

2. No, LSMASHVideoSource does not support MKV or TS containers, only containers compliant to the ISO base media file format. For all other containers, you will need LwLibavVideoSource using the libavformats demultiplexers and its indexer. Decoding HEVC should be supported.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid

Last edited by LigH; 15th February 2019 at 08:12.
LigH is offline   Reply With Quote
Old 15th February 2019, 09:14   #4496  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 6,746
Quote:
ISO base media file format
Code:
    Function IsISOFileName(String s) { s=Lcase(RT_GetFileExtension(s)) Return(s==".mov"||s==".mp4"||s==".3gp"||s==".3g2"||s==".mj2"||s==".dvb"||s==".dcf"||s==".m21")}
If you dont use RT_, maybe make your own "GetFileExtension, reverse string, look for '.', chop off extension, reverse extension.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 15th February 2019 at 11:30.
StainlessS is offline   Reply With Quote
Old 15th February 2019, 19:52   #4497  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Location: Berlin, Germany
Posts: 2,498
Thanks LigH for your LSMASH links.

I did a couple of speed benchmark tests using the usual source filters (sorry, no DG filters), and these are the results:

Source was again a captured German DVB-T2 file. 1080p HEVC at 50 fps, converting it to SD AVC using StaxRip. Very basic AVS script, it did include ColorMatrix and the VDub filter Logoaway. Everything 32-bit. Using the current AVS+ MT 32-bit build, added the almosely tweaks to the MTModes.avsi. Using 4 threads on my Core i5 3rd generation everything runs stable, no crashes.

FPS tests were done with the latest X264 build by LigH, I want to test the overall speed, just testing the script with AVSMeter is misleading for me.


DSS2Mod (preroll=15) : 25.83
ffms2 2.23.1 : 22.91
ffms2000 test 8 : 23.04
LWLibav r784 XP : 24.40
LWLibav r929 : 24.44
LWLibav r941 : 24.41

All FPS measurements were taken at 10% into the encode. The clear winner is DSS2Mod followed by LWLibav. The old XP version which does not even need SSE2 is just as fast as the later versions.


Cheers
manolito

Last edited by manolito; 17th February 2019 at 21:10.
manolito is offline   Reply With Quote
Old 17th February 2019, 19:20   #4498  |  Link
wonkey_monkey
Formerly davidh*****
 
wonkey_monkey's Avatar
 
Join Date: Jan 2004
Posts: 1,739
Some more errors on the Wiki?

http://avisynth.nl/index.php/Filter_...24_.2F_IsRGB32

Code:
bool IsRGB() const;
bool IsRGB24() const;
bool IsRGB32() const;

All of them will return true if the colorspace is RGB (in any way). The last two return true if the clip has the specific RGB colorspace (RGB24 and RGB32).
That can't be right, can it? Same just below:

Code:
bool IsYUV() const;
bool IsYUY2() const;
bool IsYV24() const;   // v5
bool IsYV16() const;   // v5
bool IsYV12() const;
bool IsYV411() const;  // v5
bool IsY8() const;     // v5

All of them will return true if the colorspace is YUV (in any way). The last six return true if the clip has the specific YUV colorspace (YUY2, YV24, YV16, YV12, YV411 and Y8).
Maybe I'm missing something but I don't think that makes sense.
__________________
My AviSynth filters / I'm the Doctor
wonkey_monkey is offline   Reply With Quote
Old 17th February 2019, 19:33   #4499  |  Link
Groucho2004
 
Groucho2004's Avatar
 
Join Date: Mar 2006
Location: A wretched hive of scum and villainy
Posts: 4,218
Quote:
Originally Posted by wonkey_monkey View Post
Code:
bool IsRGB() const;
bool IsRGB24() const;
bool IsRGB32() const;

All of them will return true if the colorspace is RGB (in any way). The last two return true if the clip has the specific RGB colorspace (RGB24 and RGB32).
That can't be right, can it?
Makes sense to me (as does the YUV logic). Can you elaborate on your objection?
__________________
Groucho's Avisynth Stuff
Groucho2004 is offline   Reply With Quote
Old 17th February 2019, 20:57   #4500  |  Link
wonkey_monkey
Formerly davidh*****
 
wonkey_monkey's Avatar
 
Join Date: Jan 2004
Posts: 1,739
"All of them will return true if the colourspace is RGB (in an way)"

If that's true, then they would all be true whether it's RGB24, RGB32, or Planar RGB. Which means you'd have no way to distinguish RGB24 and RGB32.
__________________
My AviSynth filters / I'm the Doctor
wonkey_monkey is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 05:32.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.