Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 15th April 2021, 23:55   #221  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 307
Zopti v1.1.0 released

A new major release is here! Zopti now supports executing multiple scripts simultaneously which can lead to very nice performance improvement. This can be very useful when the script itself is not able to utilize all of your cores.

Here's the full release info:
  • added support for multithreaded execution of scripts with argument -threads
    -default value of -threads is 1 meaning one script is executed at a time just like before
    -all optimization algorithms (nsga-ii, spea2, mutation, exhaustive) support the the -threads option
    -due to the nature of the heuristic algorithms (nsga-ii, spea2, mutation) there can be at most P threads running the scripts at the same time, where P is the population size
    -exhaustive algorithm has no thread limits
    -validate mode also supports -threads. there the maximum number of threads is the number of validated results (size of the pareto front)
    -note: using more than one thread will make time measurements less accurate but will not change the quality measurements
  • avsr version update (v0.2.2)
  • added jMetal.log.ini to disable multithreading related JMetal logging
  • updated Zoptilib to version 1.0.9m
  • bugfix: evaluate could not parse log files when Zopti was used

Download link is at the first post. I will shortly post some data on the multithreading performance.
zorr is offline   Reply With Quote
Old 16th April 2021, 00:32   #222  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 307
I tested the new -threads argument using the denoising tutorial script, this time using GMSD as the similarity metric and using 50 frames instead of 5. Here's the script:

Code:
SetCacheMode(0)

TEST_FRAMES = 50		# how many frames are tested
MIDDLE_FRAME = 100		# middle frame number

RawSourcePlus("D:/optimizer/test/flower/flower_cif.yuv", width=352, height=288, pixel_type="I420")
source=ColorYUV(levels="PC->TV")
noisy=source.AddGrain(25, 0, 0, seed=1)

#denoised=noisy.FFT3DFilter(sigma=4, bt=4, bw=16, bh=16, ow=8, oh=8)		# best settings by Fizick

sigma = 400/100.0 	# optimize sigma = _n_/100.0 | 400..600 ; filter:x 5 % 0 == | sigma
bt = 5			# optimize bt = _n_ | 3,5 | blockTemporal
blockSize = 14		# optimize blockSize = _n_ | 6,12 ; min:overlap 2 * | blockSize
overlap = 16		# optimize overlap = _n_ | 4..6 ; max:blockSize 2 / | overlap
denoised=noisy.FFT3DFilter(sigma=sigma, bt=bt, bw=blockSize, bh=blockSize, ow=overlap, oh=overlap)

# cut out the part used in quality / speed evaluation
source = source.Trim(MIDDLE_FRAME - TEST_FRAMES/2 + (TEST_FRAMES%2==0?1:0), MIDDLE_FRAME + TEST_FRAMES/2)
denoised = denoised.Trim(MIDDLE_FRAME - TEST_FRAMES/2 + (TEST_FRAMES%2==0?1:0), MIDDLE_FRAME + TEST_FRAMES/2)
last = denoised

last = GMSD(source, denoised, show=false)

# measure runtime, plugin writes the value to global avstimer variable
global avstimer = 0.0
AvsTimer(frames=1, type=0, total=false, name="Optimizer")

# per frame logging (gmsd, time)
global delimiter = "; "
global resultFile = "perFrameResults.txt"	# output out1="gmsd: MIN(float)" out2="time: MIN(time) ms" file="perFrameResults.txt"

# write "stop" at the last frame to tell the optimizer that the script has finished
global frame_count = FrameCount()

WriteFileIf(resultFile, function() {
    current_frame == frame_count-1
}, function() {
	gmsd = 0.0
	str = ""
	for (i = 0, frame_count-1) {
		value = propGetFloat("_PlaneGMSD", offset = -i)
		gmsd = gmsd + value
		if (i>0) { str = str + e"\n" }
		str = str + string(current_frame - i) + delimiter + string(value) + delimiter + string(avstimer)
	}
	return str + e"\nstop " + string(gmsd)
}, append=false)

return last
There's also a VapourSynth version of the script which I posted to the VapourSynth Zopti thread.

The script only has 246 valid parameter combinations to test so we can try all of them using
Code:
zopti denoise_ex.avs -alg exhaustive -threads 1
It takes 179 seconds to test all of the combinations using one thread. Let's see the performance when using more threads:



The best result is about 20 seconds using 16 threads (on a Ryzen 3900X which has 12 cores / 24 threads).

Another way to look at the scaling is to calculate how much faster we get the results when using -threads:



Using 16 threads is 8,6 times as fast as using only one thread. Of course this is just one data point and I don't mean to imply that you can always get such a performance improvement. I would have liked to include more tests using more real-world usage scenarios (HD source etc) but my stock cooled processor cannot handle those for more than a few seconds, it becomes so hot that my PC shuts down.

Comparing AviSynth and VapourSynth it looks like VS has the edge when using one or just a few threads, but the differences almost vanish when using 12 or more threads. The VS is also able to utilize more than 16 cores while with AVS the performance starts to degrade. The point where more threads are just slowing down is probably dependent on the specific script, I will have to run more tests on that.

Last edited by zorr; 16th April 2021 at 00:58.
zorr is offline   Reply With Quote
Old 16th April 2021, 05:14   #223  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 5,080
Thanks zorr, looks like a very handy approach. I'll try testing it over the weekend.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 4th May 2021, 10:32   #224  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 1,734
Let's say I want to do compression/quality tests on 1-3% of a whole movie, changing the parameters on SMDegrain in a fixed range, mostly tr and thSAD.

Is there a way to batch that process? Usually I have to manually set a frame range and have multiple avs scripts.

Can you help me?

My usual script, with the parts I want to have permutations in red:

SetFilterMTMode("DEFAULT_MT_MODE", 2)
LoadPlugin("D:\Eseguibili\Media\DGDecNV\DGDecodeNV.dll")
DGSource(something)

SMDegrain (tr=4, thSAD=400, refinemotion=false, n16_out=true, mode=0, contrasharp=false, PreFilter=4, truemotion=true, plane=4, chroma=true)

Prefetch(6)
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 4th May 2021, 21:42   #225  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 307
Quote:
Originally Posted by tormento View Post
Let's say I want to do compression/quality tests on 1-3% of a whole movie, changing the parameters on SMDegrain in a fixed range, mostly tr and thSAD.

Is there a way to batch that process? Usually I have to manually set a frame range and have multiple avs scripts.
How do you evaluate the quality? Do you have a "reference" video you could compare the different settings with? Zopti needs some way to tell the quality and usually that means comparing two videos.

If you use your own eyes to measure the quality and just want to generate the different versions... well, Zopti doesn't output any video so it can't help you there. But it could generate you the different script variations if you provide a file with the parameter combinations you want, like this
Code:
1 tr=4 thSAD=40
1 tr=5 thSAD=40
1 tr=4 thSAD=50
etc.
zorr is offline   Reply With Quote
Old 5th May 2021, 11:02   #226  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 1,734
Quote:
Originally Posted by zorr View Post
How do you evaluate the quality?
I don't need automatic quality comparison, just a way to automate the changes in the script.
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 28th May 2021, 21:35   #227  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 307
Zopti v1.2.0 released

Time for a new update! This version has some major additions so I bumped the version to 1.2.0.
  • new option: -timeout (default is disabled). if the execution of Avisynth script takes longer than timeout seconds
    -the script execution process (and subprocesses) will be terminated
    -the result is set to worst possible as a penalty so it will not be considered as a valid candidate
    -useful with scripts where certain parameter combinations can result in execution times several hundred times slower than usual
    -example: zopti script.avs -pop 24 -iters 1000 -timeout 60
  • new option: -continue
    -aborted optimization can be continued from the last complete generation (one generation is N iterations, where N is the population size)
    -give the log file name of the aborted optimization as the value of continue
    -old results up to last complete generation are copied to a new log file and optimization will continue from there
    -useful if optimization was unexpectedly aborted or the CPU resources are needed for something else for a while
    -example: zopti script.avs -pop 24 -iters 10000 -continue "mvtools spea2 run-01.log"
    -NOTE: it is possible to change the optimization parameters (population, mutation count etc.) from those used in the continued log
    -NOTE: random seed is not restored so the results will not be identical to what they would have been if execution had not been aborted
    -NOTE: only works with algorithm spea2 (for now)
  • new visualization mode: history
    -displays the best found value and all tried values of certain optimized parameter (given with -param)
    -invalid results (script execution has failed OR timed out) are highlighted in orange
    -also shows the phase of dynamic iteration (if applicable)
    -accepts parameter -range to limit the displayed history to certain iteration range
    -example: zopti -mode evaluate -vismode history -param lambda (displays all history of parameter "lambda")
    -example: zopti -mode evaluate -vismode history -param lambda -range 1000 5000 (displays history from iteration 1000 to iteration 5000)
    -example: zopti -mode evaluate -vismode history -param lambda -range -10000 (displays history of the last 10000 iterations)
  • argument -dynphases can now be zero (previously minimum was 1)
  • pareto front and total runtime is displayed also after mutation or exhaustive algorithm is finished
  • if population is based on a log file and the pareto front is smaller than the population size, the population will be filled with mutations of the pareto front instead of completely random combinations
  • if a log file for the optimization already exists it will be cleared in the beginning of the optimization
  • XChart updated to version 3.8.0 with some customization

And here's what -vismode history looks like:



The black thick line shows the best found value of the chosen parameter. The red diamonds indicate a point where a new best result has been found and this parameter's value was changed. The white diamonds indicate a new best result but this parameter's value remained the same.

In the example optimization was finished and then restarted with different settings at around 190 000 iterations.

The -timeout was needed when I started running MVTools2 tests with all the parameters and it turned out that some combinations took an enormous amount of time to finish (typical time was 5 seconds but some combinations took over 30 minutes...). Using a timeout of 60 seconds solved that problem.

-continue is useful if you want to keep optimizing a result that is finished, you can try different settings. Also very handy when the optimization has been aborted by hardware failure or something like Windows update... you can just restart it using the same settings. This also makes it possible to run huge iteration counts little by little even if you need/want to use the computer for other things once in a while.

Download link updated at first post.
zorr is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 01:53.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.