Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
2nd February 2019, 15:32 | #201 | Link |
Registered User
Join Date: Sep 2010
Location: Russia
Posts: 85
|
I don't think I understand how to use the new method. I tried incorporating it into my script and thing went wrong I think, the pareto graphs look weird and the "best" results now are terribly slow and very different from anything I've gotten before.
Where should I get a x64 version of GaussianBlur? How do you determine the radius that should be used there? Here's the script https://pastebin.com/c65dhju2, I launch it like F:\avsoptim\optimizer.bat F:\avsoptim\genBC.avs -iters 7h -runs 2 -pop 24 |
2nd February 2019, 18:21 | #202 | Link | |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Quote:
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? |
|
2nd February 2019, 18:44 | #203 | Link | |
Registered User
Join Date: Mar 2012
Location: Texas
Posts: 1,666
|
Quote:
Also I don't recall a 64-bit version of the latest VariableBlur (v0.7), only v.05 is available: |
|
2nd February 2019, 20:04 | #204 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Hi RD, glad that you are still lurking about
Just checked, I got v0.5 only on x64. OK, had a bit more of a look, here is v0.7 (x86 only) with source, but its got lots of asm. http://www.mediafire.com/file/1ugcb8...r_0.7.zip/file
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 2nd February 2019 at 21:27. |
2nd February 2019, 22:26 | #205 | Link | |
Registered User
Join Date: Mar 2018
Posts: 447
|
Quote:
Replace the lines 135-146 with this and it should work. Code:
scharr_orig = scharr(orig_yv12) scharr_inter = scharr(inter_yv12) FrameEvaluate(last, """ #global ssim = SSIM_FRAME(scaled_orig, scaled_inter) global ssim = SSIM_FRAME(orig_yv12, inter_yv12) global bssim = BSSIM_MOD(scharr_orig, scharr_inter, ssim) global ssim_total = ssim_total + (bssim == 1.0 ? 0.0 : bssim) """) |
|
3rd February 2019, 00:22 | #206 | Link |
Registered User
Join Date: Mar 2018
Posts: 447
|
New release, new name! AvisynthOptimizer is called "Zopti" from now on. Why? 'Cause it supports VapourSynth as well, that's why. Download the version 1.0-beta here.
There's some other improvements as well, here's a complete list:
A few notes about upgrading: I renamed the optimizer.ini to zopti.ini and the optimizer.bat to zopti.bat. If you already have the .ini you can just rename it to zopti.ini and it will work. I have used the -priority option to set the thread priority lower because when VapourSynth/MVTools is running a heavy task the other programs can become sluggish with the default priority. Priority option is Windows only for now, Linux probably doesn't even need it. The error handling options are useful when running the VapourSynth MVTools which is currently not as robust as Pinterf's version. With -errors ignore or -errors log you can keep running the optimization even when the plugin crashes. I still recommend running with the default error handling option (which stops the execution) unless you are expecting errors. I will make another thread to the VapourSynth section to tell more about the VapourSynth-specific stuff. Last edited by zorr; 3rd February 2019 at 00:36. Reason: Added version number |
10th April 2019, 12:59 | #207 | Link |
Registered User
Join Date: Sep 2010
Location: Russia
Posts: 85
|
Is this version abandoned and will any development only happen for the VS fork? I don't feel like learning a whole new language just to continue processing video as I did before just fine with AVS.
For some reason I was unable to find this thread in the thread list for Avisynth Development section, it said "Displaying 12 of 12 threads". Now that I bumped it, it's displayed there and says "13 of 13". How do I see all threads at once? |
10th April 2019, 20:44 | #209 | Link | |
Registered User
Join Date: Mar 2018
Posts: 447
|
Quote:
I haven't made much progress lately, so it's quiet on both fronts. |
|
10th April 2019, 22:04 | #210 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Oh do stop it.
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? |
4th April 2021, 23:57 | #211 | Link | ||
Registered User
Join Date: Mar 2018
Posts: 447
|
Quote:
Quote:
I tried some other ways. You can create a new variable for each frame (using Eval) and store the metric there. It doesn't solve the synchronization problem though, there are sometimes missing variables when it is time to combine the results. So that method only works without Prefetch. It would be useful for Avisynth to have a callback function which would be triggered when all the frames have been processed. I also thought that maybe there's no need for any extra variables at all, we can simply read the metric from all the frames at the last frame using the offset parameter of propGetFloat(). The good news is that it actually works and is faster than the default method which writes to a file at each frame. The bad news is that the speedup is simply due to writing to the file only once (saves about 1,6ms per frame) and there is no additional speedup when using Prefetch. Here's the code: Code:
TEST_FRAMES = 50 # how many frames are tested MIDDLE_FRAME = 100 # middle frame number RawSourcePlus("D:/optimizer/test/flower/flower_cif.yuv", width=352, height=288, pixel_type="I420") source=ColorYUV(levels="PC->TV") noisy=source.AddGrain(25, 0, 0, seed=1) #return noisy #denoised=noisy.FFT3DFilter(sigma=4, bt=4, bw=16, bh=16, ow=8, oh=8) # best settings by Fizick sigma = 400/100.0 # optimize sigma = _n_/100.0 | 100..800 | sigma bt = 4 # optimize bt = _n_ | -1..5 | blockTemporal blockSize = 32 # optimize blockSize = _n_ | 2..64 ; min:overlap 2 * | blockSize overlap = 16 # optimize overlap = _n_ | 0..32 ; max:blockSize 2 / | overlap denoised=noisy.FFT3DFilter(sigma=sigma, bt=bt, bw=blockSize, bh=blockSize, ow=overlap, oh=overlap) # cut out the part used in quality / speed evaluation source = source.Trim(MIDDLE_FRAME - TEST_FRAMES/2 + (TEST_FRAMES%2==0?1:0), MIDDLE_FRAME + TEST_FRAMES/2) denoised = denoised.Trim(MIDDLE_FRAME - TEST_FRAMES/2 + (TEST_FRAMES%2==0?1:0), MIDDLE_FRAME + TEST_FRAMES/2) last = denoised last = GMSD(source, denoised, show=false) # measure runtime, plugin writes the value to global avstimer variable global avstimer = 0.0 AvsTimer(frames=1, type=0, total=false, name="Optimizer") # per frame logging (ssim, time) global delimiter = "; " global resultFile = "perFrameResults.txt" # output out1="gmsd: MIN(float)" out2="time: MIN(time) ms" file="perFrameResults.txt" # write "stop" at the last frame to tell the optimizer that the script has finished global frame_count = FrameCount() WriteFileIf(resultFile, function() { current_frame == frame_count-1 }, function() { gmsd = 0.0 str = "" for (i = 0, frame_count-1) { value = propGetFloat("_PlaneGMSD", offset = -i) gmsd = gmsd + value if (i>0) { str = str + e"\n" } str = str + string(current_frame - i) + delimiter + string(value) + delimiter + string(avstimer) } return str + e"\nstop " + string(gmsd) }, append=false) Prefetch(4) return last By the way this is a bit off-topic on this Vapoursynth forum. If any mods happen to read this it might be a good idea to move the posts starting from Worby's to the Avisynth Zopti thread. |
||
5th April 2021, 12:34 | #212 | Link |
Pig on the wing
Join Date: Mar 2002
Location: Finland
Posts: 5,733
|
I'm unable to get the new version to work with this test script. It processes the same step for minutes and if you take a look at Task Manager, the I/O read bytes shows that the source file has been read multiple times. Tested with and without Prefetch at the end.
Code:
SetCacheMode(0) orig = FFVideoSource("c:\zopti\lotr_fotr.avi") b = -75/100.0 # optimize b = _n_/100.0 | -150..50 | b c = 15/100.0 # optimize c = _n_/100.0 | -100..100 | c downscaled_width = 1920 downscaled_height = 808 alternate = BicubicResize(orig, downscaled_width, downscaled_height, b=b, c=c).Lanczos4Resize(orig.width(),orig.height()) GMSD(alternate, orig, show=false) # measure runtime, plugin writes the value to global avstimer variable global avstimer = 0.0 AvsTimer(frames=1, type=0, total=false, name="Optimizer") # per frame logging (ssim, time) global delimiter = "; " global resultFile = "perFrameResults.txt" # output out1="gmsd: MIN(float)" out2="time: MIN(time) ms" file="perFrameResults.txt" # write "stop" at the last frame to tell the optimizer that the script has finished global frame_count = FrameCount() WriteFileIf(resultFile, function() { current_frame == frame_count-1 }, function() { gmsd = 0.0 str = "" for (i = 0, frame_count-1) { value = propGetFloat("_PlaneGMSD", offset = -i) gmsd = gmsd + value if (i>0) { str = str + e"\n" } str = str + string(current_frame - i) + delimiter + string(value) + delimiter + string(avstimer) } return str + e"\nstop " + string(gmsd) }, append=false) #Prefetch(threads=24, frames=1) return last
__________________
And if the band you're in starts playing different tunes I'll see you on the dark side of the Moon... |
6th April 2021, 10:58 | #214 | Link | |
Pig on the wing
Join Date: Mar 2002
Location: Finland
Posts: 5,733
|
Quote:
I'm currently using the latest Avs+. I was wondering if AVSTimer is one reason for seemingly serialized processing despite the prefetch value set. The GRunT stuff like WriteFileIf is MT-friendly if I remember correctly.
__________________
And if the band you're in starts playing different tunes I'll see you on the dark side of the Moon... Last edited by Boulder; 6th April 2021 at 11:47. |
|
6th April 2021, 15:32 | #215 | Link |
Pig on the wing
Join Date: Mar 2002
Location: Finland
Posts: 5,733
|
Hmm..not that either. It must then be the way Avisynth parses things as all that is left is the wait for all frames to finish to get the final result. Strange though that using 24 threads instead of disabling Prefetch doesn't seem to have any effect even when the metric calculation per frame is the slow part.
__________________
And if the band you're in starts playing different tunes I'll see you on the dark side of the Moon... |
6th April 2021, 22:06 | #216 | Link | |
Registered User
Join Date: Mar 2018
Posts: 447
|
Quote:
Then I took out the scaling and only left the GMSD calculation. The time is still the same and the Prefetch behaviour remains. So it's the GMSD (and probably MDSI) which is slow and doesn't get any benefit from multiple threads. I guess it's because the multithreading only accelerates plugins and GMSD is implemented as a script. I tried to include a Prefetch call into the GMSD.avsi but it doesn't allow it there. |
|
6th April 2021, 22:25 | #217 | Link | |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Quote:
Something like AvsMeter may be able to look at it from 'outside' and on total throughput of frames delivered, and not necessarily in any kind of order. [although there would also have to be some kind of serialzation when played at output - which may/will affect AvsMeter too]
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 6th April 2021 at 22:30. |
|
7th April 2021, 01:30 | #218 | Link |
Registered User
Join Date: Jan 2018
Posts: 2,156
|
VariableBlur 0.7 (x86 & x64)
https://drive.google.com/file/d/1tiC...ew?usp=sharing |
7th April 2021, 05:31 | #219 | Link | |
Pig on the wing
Join Date: Mar 2002
Location: Finland
Posts: 5,733
|
Quote:
Both GMSD and MDSI use ScriptClip to put things in the frame properties, I don't see any other special thing in those functions. They're mostly Masktools stuff which runs multithreaded just fine.
__________________
And if the band you're in starts playing different tunes I'll see you on the dark side of the Moon... |
|
7th April 2021, 15:23 | #220 | Link |
Pig on the wing
Join Date: Mar 2002
Location: Finland
Posts: 5,733
|
It's definitely something in those metrics functions, just posted to the Avisynth thread. I can only think the use of ScriptClip doing something which trips the multithreading.
__________________
And if the band you're in starts playing different tunes I'll see you on the dark side of the Moon... |
|
|