Zopti

[Iron_Mike]

https://github.com/theChaosCoder/zoptilib
Mdsi and butteraugli no longer contaminate the clip. That means metrics=['ssim', 'gmsd', 'mdsi', 'butteraugli'] is the same as metrics=['mdsi', 'butteraugli', 'ssim', 'gmsd'] now.
This also means that you won't see the heatmap...

could you elaborate on that ?

is this in regards to the RGB conversion for MDSI/Butteraugly ?

Also, running Butteraugly via Zopti takes quite a while on my system... way longer than VMAF, GMSD, SSIm etc... is this normal ?

Any settings to speed this up ?

Thanks.

[ChaosKing]

Yes, it doesn't pass the rgb converted clip but instead it copies the clip props to the old clip now.
Butteraugli is sloooow, can't do much about it.

[Iron_Mike]

Yes, it doesn't pass the rgb converted clip but instead it copies the clip props to the old clip now.
Butteraugli is sloooow, can't do much about it.

okay, so if I run metrics=['ssim', 'gmsd', 'mdsi', 'butteraugli'] in the same Zopti call, it will convert the clips to RGB for MDSI/Butter but will do GMSD/SSIM calc in YUV ?

[ChaosKing]

okay, so if I run metrics=['ssim', 'gmsd', 'mdsi', 'butteraugli'] in the same Zopti call, it will convert the clips to RGB for MDSI/Butter but will do GMSD/SSIM calc in YUV ?

Yes and yes and the "auto-rgb-convert" is still present for mdsi and butter.

[Iron_Mike]

does SSIM/GMSD via Zopti/muvsfunc upscale the chroma planes to the src resolution of the clip if we specifiy planes=1|2 (e..g is src is 444 but dist is 420) ?

[WorBry]

No it does not. That was the whole point of my last post in the 'SSIM/GMSD metrics thread':

https://forum.doom9.org/showthread.php?p=1870863#post1870863


You posted the same question there and before I could answer, deleted it and are now asking it here. What gives ?

No resize or weights are applied to the 'processed-as-luma' chroma planes.

e..g is src is 444 but dist is 420) ?

That combination would throw an error irrespective of the plane being processed.

"clip1" and "clip2" must be of the same format!

[Iron_Mike]

No it does not. That was the whole point of my last post in the 'SSIM/GMSD metrics thread':

https://forum.doom9.org/showthread.php?p=1870863#post1870863

You posted the same question there and before I could answer, deleted it and are now asking it here. What gives ?

what gives ?

well, Brian... take a really hard guess why I opted to move this to the ZOPTI thread instead of leaving it in the GMSD thread.... ? considering we're using Zopti as a wrapper... :cool:


in any case - back to the point - so in case of GMSD/SSIM scoring planes 1/2 of a 444 src vs 420 dist, is Zopti/muvsfunc downscaling the src to the dist resolution or does the GMSD/SSIM metric not require both clips to be in the same resolution ?

[WorBry]

I repeat...


That combination would throw an error irrespective of the plane being processed.

"clip1" and "clip2" must be of the same format!

[Iron_Mike]

I repeat...

yeah, that's why I ask here in the Zopti thread...

I just ran a 444 ref against a 420 dist, scored GMSD/SSIM in planes 1|2 and it did not throw an error (meaning: chroma resolution of src/dist was NOT the same)... also (as far as I understand it), vs.format does not include src/Y resolution...

so, question remains...

[ChaosKing]

yeah, that's why I ask here in the Zopti thread...

I just ran a 444 ref against a 420 dist, scored GMSD/SSIM in planes 1|2 and it did not throw an error (meaning: chroma resolution of src/dist was NOT the same)... also (as far as I understand it), vs.format does not include src/Y resolution...

so, question remains...

Could you provide your script? I always get "clip1" and "clip2" must be of the same format!

My test

#c is 420
a = mvf.GetPlane(c, 1)
c = core.fmtc.resample(c, css="444")
b = mvf.GetPlane(c, 1)#.fmtc.resample(css="444")

c = muf.SSIM(a,b)

[WorBry]

Could you provide your script? I always get "clip1" and "clip2" must be of the same format!


Quite !

[Iron_Mike]

Could you provide your script? I always get "clip1" and "clip2" must be of the same format!

my bad, u're right - I have a block of code there for SSIM/GMSD (via Zopti) that auto-converts dist to src format in case format differs - to avoid that exact error... ;-)

so what are opinions about how to weigh the GMSD/SSIM results if the css of the dist differs ?

[WorBry]

so what are opinions about how to weigh the GMSD/SSIM results if the css of the dist differs ?

You don't. You either convert the css of the 'distorted' clip to that of the reference or vice versa. They must be the same format - period. It's the fundamental requirement of Full Reference Image Quality Assessment. You seem to be laboring under this notion that it should somehow be different. 'Full Reference' - think about it.

[Iron_Mike]

You don't. You either convert the css of the 'distorted' clip to that of the reference or vice versa. They must have the same format - period. It's the fundamental requirement of Full Reference Image Quality Assessment. You seem to be laboring under this notion that it should somehow be different.

well Brian, a "fundamental requirement" is to read posts properly... or to put 1+1 together :D :eek:

I was referring to weighing the chroma plane results (if dist had different css than ref) when combining them with the Y plane result for a total GMSD/SSIM score based on all three planes...............

in addition... chroma plane results can also be (optionally) weighed (in a combined score) even if both ref/dist have the exact same css considering the fact that color drift has less perceived impact in human vision...


whatever it is that u're mumbling in regards to "format" has nothing to do with any of it... just because u're matching a format so u can run the metric does not mean the data is (or should be considered) equal........

[WorBry]

Someone else deal with this please. I'm done with this nonsense.

Also Iron_Mike, please desist in referring to me by anything other than my forum user name. I don't know what you are trying to convey, but it's rather creepy and smacks of trollism.

[ChaosKing]

well Brian, a "fundamental requirement" is to read posts properly... or to put 1+1 together :D :eek:

I was referring to weighing the chroma plane results (if dist had different css than ref) when combining them with the Y plane result for a total GMSD/SSIM score based on all three planes...............



But to compare different css you have to resize the chroma plane to match the same clip format for ssim/gmsd. It's similar to a rgb and yuv clip, one has to be converted wich alters your final score a little bit.

hmm a score based on all three planes... For that I tried to see how the video looks like if you extremely blur the chroma plane. Surprisingly only the colors are a bit off.
I guess the final score could be as simple as Plane1 80% + P2 10% + P3 10% ... I don't know what a good weighted total score should look like.

c2_1 = mvf.GetPlane(c2, 1).std.BoxBlur(hradius=5, vradius=5)
c2_2 = mvf.GetPlane(c2, 2).std.BoxBlur(hradius=5, vradius=5)
c2 = core.std.ShufflePlanes(clips=[mvf.GetPlane(c2, 0), c2_1, c2_2], planes=[0, 0, 0], colorfamily=vs.YUV)

a = mvf.GetPlane(c, 1)
b = mvf.GetPlane(c2, 1)
#a=c
#b=c2
c = muf.SSIM(b,a).text.FrameProps()
c.set_output()

p.s the ssim score for the blurred chroma plane is 0.8

https://i.imgur.com/TEgxbwD.jpg
https://i.imgur.com/MCDuF5f.png

[WorBry]

I don't know what a good weighted total score should look like.

I'm not sure. As mentioned in the other 'SSIM: GMSD metric thread', Zorr looked at the code for the AVISynth and FFMPEG SSIM metrics to see how they derive an overall score for the Y, U and V channels:

Zorr checked out the code for ffmpeg SSIM (AVISynth SSIM also) earlier in the thread. In essence, it's akin to the 'fast' SSIM metric in the MSU Quality Tool:

https://forum.doom9.org/showthread.php?p=1866030#post1866030

https://forum.doom9.org/showthread.php?p=1866162#post1866162



The Avisynth version is not doing the gaussian kernel at all - it's implemented using summed area tables. That's a faster but lower quality way to calculate it, the MSU Quality measurement tool page (http://www.compression.ru/video/quality_measure/info.html#ssim) has an example of the difference.

.....Avisynth SSIM has a plane argument which defaults to 0 and then it returns a weighted sum of the luma and chroma channels:
(0.8 * Y) + (0.1*(U+V))


But the AVISynth version is YV12 only.

I looked at the ffmpeg source. It's doing the fast version too - no gaussian kernels there. The total SSIM takes into account all the planes but the weighting is different, each plane is scaled by the resolution is has. So for example with YUV420 the color planes have 4 times smaller weight. The constants 0.01 and 0.03 appear in the code and I assume that means the weights k1 and k2 are the same as in other implementations.


And:

To improve speed, this implementation uses the standard approximation of overlapped 8x8 block sums, rather than the original gaussian weights

https://github.com/FFmpeg/FFmpeg/blob/master/libavfilter/vf_ssim.c

So the ffmpeg variant applies different weights according to pixel format of the clips being tested, but they must have the same pixel format and resolution

https://ffmpeg.org/ffmpeg-filters.html#ssim

Of course the FFMPEG metric can also be run on RGB clips, generating scores for the individual R,G,B channels and an overall aggregate score. But it's still using the 'fast' and less precise 'standard approximation of overlapped 8x8 block sums'.

Is there provision in the original Matlab SSIM code for measuring and weighting the chroma or some other reference where this is defined - I don't know ?

I checked out the demo 'Pro' version of the MSU-QMT, btw - the demo version intentionally generates incorrect results as a kind of 'metric watermark' but it shows the options that are available for the different measures. Looks like the 'precise' (gauss blur) and 'fast' SSIM variants are measuring Y luma only.

And then there are these other 'SSIM algorithm-based' variants - DSSIM and SSIMULACRA - intended (like Butteraugli) primarily for measuring differences in still images, which process in CIE Lab* color space and apply weights to the L, a and b channels to derive an overall score:

https://github.com/kornelski/dssim

https://github.com/cloudinary/ssimulacra

https://cloudinary.com/blog/detecting_the_psychovisual_impact_of_compression_related_artifacts_using_ssimulacra

Personally, I'd be more interested in deriving a composite SSIM score for the combined chroma channels, rather than (or in addition to) an 'overall' score, but that would also require consideration of the weightings.

[VS_Fan]

... Personally, I'd be more interested in deriving a composite SSIM score for the combined chroma channels, rather than (or in addition to) an 'overall' scoreHow about stacking (horizontally or vertically) the u and v planes in vapoursynth, then generate YUV/Y4M files so you can obtain SSIM metrics by using any of the tools mentioned?

I would expect the result to be very close to averaging the results of the individual u and v planes.

[WorBry]

Now that's a novel idea. I was thinking of maybe merging the luma-converted U and V planes.

[VS_Fan]

With the same general idea, one could obtain an “an 'overall' score” by stacking all the planes:


ret = some_video_source
#This should work to preserve vertical resolution, and stacking all Y,U and V, no matter the css (chroma subsampling: 411, 420, 422, 444)
retc = core.std.StackHorizontal(clips=[retU,retV]) if ret.format.subsampling_h == 0 else core.std.StackVertical(clips=[retU,retV])
ret = core.std.StackHorizontal(clips=[retY,retc])


I would also expect this ‘overall’ result to be close to that of applying the weights as you exposed they do in ffmpeg: “each plane is scaled by the resolution it has”, and I guess they are something like:


sub- Y U V
sampling ratio ratio ratio

4:1:1 2/3 1/6 1/6
4:2:0 2/3 1/6 1/6
4:2:2 1/2 1/4 1/4
4:4:4 1/3 1/3 1/3

[Iron_Mike]

@WorBry: I'm glad u finally woke up and entertain the idea of "weights" - a concept used in scientific formulas and equations since the dawn of time, and that includes the field of color science.

This is good, this could indicate progress !

Maybe next time before u have ur temper tantrum, u can put the bottle aside, and read posts properly. Zorr did directly mention weights to u in regards to ffmpeg's implementation but u seem to have chosen to ignore that... until u remembered.

well, to quote u: what gives ? :cool:


I don't know what a good weighted total score should look like.


yeah, hence me throwing it out there and asking for opinions... ;-)

[Iron_Mike]

sub- Y U V
sampling ratio ratio ratio

4:1:1 2/3 1/6 1/6
4:2:0 2/3 1/6 1/6
4:2:2 1/2 1/4 1/4
4:4:4 1/3 1/3 1/3

good stuff getting the ball rolling...

this was my pretty much my first approach.

so what about the idea - in case of src/ref and dist/enc having different css - of flipping that table and the more css the distorted clip has (compared to src/ref), to give chroma score more weight than luma ?

that would bring the total score down, but with a 420 dist clip, the chroma drift/delta will have a larger perceived impact than with a 444 dist clip...

for example:
source is 444 and the dist is 420... we could consider giving the chroma score more weight than luma, b/c that color drift will have a larger impact


for WorBry: no, just b/c u put the 420 dist in a 444 clip to run the metric does not make the 420 data inside that new clip a "444" encode... it's resampled 420 data

[poisondeathray]

I think the dist. should always be converted to the ref. pixel format

The metric should take into account the results of resizing the data - that's the whole idea of metrics measuring the effect of chroma subsampling . It should be a measureable phenomenon (and it is with standard metrics)

eg
a) 444 source reference
b) a is converted to "b" 4:2:0 by algorithm "z" (this means the U,V planes are resized by 1/2 width, 1/2 height)

You need to convert b back to 444 to compare against "a" . ie. resize back the U,V planes *2 width/height .

- you expect 422 to do better than 420 - and you do with ssim, psnr
- you expect to get slightly different results with different resize algorithms (e.g. sharper isn't always "better" ; sharper actually tends to get "penalized" by some metrics like ssim/psnr)
- serial conversions get blurrier colors, and worse results, as expected (because really you're up/down scaling the chroma planes)

Or if you're starting with a 420 reference ; and you're testing against something at got converted to 444 . That up/down conversion should be reflected in the scores as a larger deviation from the original reference.



Don't mix in "weights" ; it's a separate concept dealing with human perception of color vs. black/white . Your perception doesn't change , based on different pixel format of videos - it's the source that is changing . Weighting should be the same whatever you decide the formula is . You don't become less sensitive to color with 444 vs 420 . The effect of chroma subsampling should be measured and reflected by the metric

[Iron_Mike]

I think the dist. should always be converted to the ref. pixel format

The metric should take into account the results of resizing the data - that's the whole idea of metrics measuring the effect of chroma subsampling . It should be a measureable phenomenon (and it is with standard metrics)

eg
a) 444 source reference
b) a is converted to "b" 4:2:0 by algorithm "z" (this means the U,V planes are resized by 1/2 width, 1/2 height)

You need to convert b back to 444 to compare against "a" . ie. resize back the U,V planes *2 width/height .

- you expect 422 to do better than 420 - and you do with ssim, psnr
- you expect to get slightly different results with different resize algorithms (e.g. sharper isn't always "better" ; sharper actually tends to get "penalized" by some metrics like ssim/psnr)
- serial conversions get blurrier colors, and worse results, as expected (because really you're up/down scaling the chroma planes)

Or if you're starting with a 420 reference ; and you're testing against something at got converted to 444 . That up/down conversion should be reflected in the scores as a larger deviation from the original reference.

Don't mix in "weights" ; it's a separate concept dealing with human perception of color vs. black/white . Your perception doesn't change , based on different pixel format of videos - it's the source that is changing . Weighting should be the same whatever you decide the formula is . You don't become less sensitive to color with 444 vs 420 . The effect of chroma subsampling should be measured and reflected by the metric

@pd: all agreed, but we were elaborating combining scores of various planes into a combined metric score for all planes... and in a 420 dist (from a 444 ref), the chroma gets a different treatment than the luma in the encoding process...

so when u combine metric score for luma and chroma planes, one could consider that... or are u saying the chroma metric scores already state that and u would just pull the avg ?

Btw, weights weren't mixed in (in my posts) - weights are constants - luma vs. chroma perception is one, but u can have additional weights for other effects... for example, the luma vs. chroma weight also only kicks in if you're actually dealing w/ color footage (so it is a conditional factor)...

[WorBry]

@WorBry: I'm glad u finally woke up and entertain the idea of "weights" - a concept used in scientific formulas and equations since the dawn of time, and that includes the field of color science.

This is good, this could indicate progress !

Maybe next time before u have ur temper tantrum, u can put the bottle aside, and read posts properly. Zorr did directly mention weights to u in regards to ffmpeg's implementation but u seem to have chosen to ignore that... until u remembered.

well, to quote u: what gives ? :cool:

yeah, hence me throwing it out there and asking for opinions... ;-)

I have never not had an 'idea of weights'. That's what my last series of tests in the 'SSIM and GMSD metrics' thread was about - to see if any weighting or resizing is being applied to the luma-converted U and V planes in the muvsfunc SSIM and GMSD implementations:

https://forum.doom9.org/showthread.php?p=1870863#post1870863

And your original question about 'weights' here was:


...so what are opinions about how to weigh the GMSD/SSIM results if the css of the dist differs ?

Which both I and ChaosKing answered:

You don't. You either convert the css of the 'distorted' clip to that of the reference or vice versa. They must be the same format - period. It's the fundamental requirement of Full Reference Image Quality Assessment. You seem to be laboring under this notion that it should somehow be different. 'Full Reference' - think about it.

(You still seem to have that notion)

But to compare different css you have to resize the chroma plane to match the same clip format for ssim/gmsd. It's similar to a rgb and yuv clip, one has to be converted wich alters your final score a little bit.


Now enough of your insults and overt gas-lighting.

@VS_Fan and Poisondeathray. Thanks. Alot to think about there.

[poisondeathray]

@pd: all agreed, but we were elaborating combining scores of various planes into a combined metric score for all planes... and in a 420 dist (from a 444 ref), the chroma gets a different treatment than the luma in the encoding process...

so when u combine metric score for luma and chroma planes, one could consider that... or are u saying the chroma metric scores already state that and u would just pull the avg ?

Btw, weights weren't mixed in (in my posts) - weights are constants - luma vs. chroma perception is one, but u can have additional weights for other effects... for example, the luma vs. chroma weight also only kicks in if you're actually dealing w/ color footage (so it is a conditional factor)...




The idea of "weights" for a combined Y/U/V aggregate metric score, should reflect that the Y plane should recieve a proportionally higher weighting due to human perceptions - I think everyone will agree on the general idea, but might disagree on the actual formula for the weighting

All I'm saying is the relative weighting shouldn't change because of subsampling . Your perception of the proportion of black/white vs. color importance doesn't suddenly change if you watch a 4:4:4 video or a 4:2:0 video.

That lower quality of 4:2:0 should already be reflected in the lower U, V scores for 4:2:0 . That up/down converison is the "penalty" already incurred . The "different treatment" of chroma is exactly what you're measuring in the first place when you measure U-SSIM, V-SSIM, or U-PSNR, V-PSNR or whatever metric

Yes you can have other weights, other categories, combine /mix/match in any way you want , analyze it in whatever way you want, call it whatever you want ; but eitherway you're measureing Y,U,V separately - so you should have the "raw" scores

[Iron_Mike]

The idea of "weights" for a combined Y/U/V aggregate metric score, should reflect that the Y plane should recieve a proportionally higher weighting due to human perceptions - I think everyone will agree on the general idea, but might disagree on the actual formula for the weighting

All I'm saying is the relative weighting shouldn't change because of subsampling . Your perception of the proportion of black/white vs. color importance doesn't suddenly change if you watch a 4:4:4 video or a 4:2:0 video.

That lower quality of 4:2:0 should already be reflected in the lower U, V scores for 4:2:0 . That up/down converison is the "penalty" already incurred . The "different treatment" of chroma is exactly what you're measuring in the first place when you measure U-SSIM, V-SSIM, or U-PSNR, V-PSNR or whatever metric

Yes you can have other weights, other categories, combine /mix/match in any way you want , analyze it in whatever way you want, call it whatever you want ; but eitherway you're measureing Y,U,V separately - so you should have the "raw" scores

@pd: yup, all agreed.

now stop dancing around the subject, what would be your weights/formula for a combined metric score of all planes ? :D

[poisondeathray]

I don't think there is a commonly accepted formula/value for all planes. There are no scientific studies that conclusively measure this value to be "x/y/z"

Just pick one and as long as you state your assumptions, criteria, and reasoning- it's ok :) . e.g if you decide 0.8/0.1/0.1 , say so

What's not ok - is if you post results, but no details - ie. lack of transparency .

e.g. if you measure a black/white movie - you might still decide to look at U/V . There are old movies that have discoloration and chroma noise / rainbows due to the transfer process for example .

Everyone knows (or should know) , that all metrics have various pros/cons and are context dependent. But the trends can still be useful to look at

[Wilbert]

@Iron_Mike, refrain your language. There is no need to be disrespectful to others.

[Boulder]

I have a small feature request regarding cases where colorspace conversion to RGB is required. It would be nice if Zopti as an option could do the conversion just once and save the resulting file as an intermediate file to use in all the subsequent iterations. I don't know how much time consuming it actually is, but I'd expect that the number of iterations is usually so big that over a long test run, it would same a nice amount of time.

[zorr]

I have a small feature request regarding cases where colorspace conversion to RGB is required. It would be nice if Zopti as an option could do the conversion just once and save the resulting file as an intermediate file to use in all the subsequent iterations.

That's a bit tricky, currently Zopti doesn't know what the input file is so it's unable to change it. I think it could save the converted file using clip.output() (someone can correct me here if I'm wrong) and then read that one instead, but I'm not sure if it's any faster if the file is in uncompressed format which the output() supports. Does someone have experience on using output() and reading the resulting file?

[ChaosKing]

Btw Asd-g ported some metrics like MDSI, GMSD or vsSSIM to avs https://github.com/Asd-g/AviSynthPlus-Scripts

[Boulder]

I was trying to run Zopti with these parameters (rest of the script is not there):

zopti = Zopti(r'results.txt', metrics=['mdsi'], matrix='709')

b = -75/100.0 # optimize b = _n_/100.0 | -150..50 | b
c = 15/100.0 # optimize c = _n_/100.0 | -100..100 | c

And I get this error when trying to evaluate the results. The process itself goes fine. If I add 'time' to the metrics list, it works.
java.lang.NumberFormatException: For input string: "b=-75"
at sun.misc.FloatingDecimal.readJavaFormatString(Unknown Source)
at sun.misc.FloatingDecimal.parseFloat(Unknown Source)
at java.lang.Float.parseFloat(Unknown Source)
at java.lang.Float.<init>(Unknown Source)
at avisynthoptimizer.Result.parseValues(Result.java:93)
at avisynthoptimizer.Result.<init>(Result.java:240)
at avisynthoptimizer.LogFile.read(LogFile.java:276)
at avisynthoptimizer.AviSynthOptimizer.evaluateResults(AviSynthOptimizer.java:3564)
at avisynthoptimizer.AviSynthOptimizer.main(AviSynthOptimizer.java:1872)
ERROR: For input string: "b=-75"

The reason why I wanted to skip measuring the time is that I don't want the time to reset "the counter" after which it's clear that the result doesn't get any better. I'm only looking for the best value for the actual measurement.

This is the command line I've used:
for %f in (*.vpy) do zopti "%f" -alg mutation -iters dyn -dyniters 15 -dynphases 2 -pop 1 -runs 1 -mutcount 1 -mutamount 0.1 0.01 -initial script

[zorr]

I was trying to run Zopti with these parameters (rest of the script is not there):

zopti = Zopti(r'results.txt', metrics=['mdsi'], matrix='709')

b = -75/100.0 # optimize b = _n_/100.0 | -150..50 | b
c = 15/100.0 # optimize c = _n_/100.0 | -100..100 | c

And I get this error when trying to evaluate the results.

I tried to reproduce this but it's working on my end. I think there was a version which didn't handle the missing time measurement but I believe it was fixed at some point. However, the version I tested with was the latest which I haven't released so there's a small chance that it's only fixed in my version. Which version of Zopti are you running?

[Boulder]

I tried to reproduce this but it's working on my end. I think there was a version which didn't handle the missing time measurement but I believe it was fixed at some point. However, the version I tested with was the latest which I haven't released so there's a small chance that it's only fixed in my version. Which version of Zopti are you running?

Mine is version 1.0-beta, should be the latest released one.

[zorr]

Mine is version 1.0-beta, should be the latest released one.

The latest is 1.0.1-beta, link is available in the first post. :)

[Boulder]

The latest is 1.0.1-beta, link is available in the first post. :)

That's what I have, it looks like the application just reports 1.0-beta. I verified that the files in the latest archive are the same as what I have in my Zopti folder.

[zorr]

That's what I have, it looks like the application just reports 1.0-beta. I verified that the files in the latest archive are the same as what I have in my Zopti folder.

All right, can you provide a few lines from your log file? Does it look something like this (I used SSIM here)?

# output out1="SSIM: MAX(float)" file="results.txt"
7.86225 b=0 c=0
7.862938 b=1 c=1
7.854728 b=1 c=0 # phase=0,00
7.889888 b=-4 c=1
7.8532143 b=-18 c=1
7.8917427 b=-4 c=6
7.8901386 b=-4 c=7

[Boulder]

Yes, looks similar:
# output out1="MDSI: MIN(float)" file="results.txt"
30.7859 b=-60 c=30
31.428144 b=-47 c=12
30.756887 b=-63 c=30 # phase=0,00
30.797626 b=-59 c=30

[WorBry]

Btw Asd-g ported some metrics like MDSI, GMSD or vsSSIM to avs https://github.com/Asd-g/AviSynthPlus-Scripts

Didn't see this before. Excellent. Love the color-enhanced maps. How would one print a log file that compiles the per-frame test results in this case?

[zorr]

How would one print a log file that compiles the per-frame test results in this case?

Looks like there is no way to read the similarity metric back into a script variable which is how the current SSIM metric is handled. Perhaps this could be implemented by passing a (global?) variable name and GMSD / MDSI functions would assign that variable?

[WolframRhodium]

Looks like there is no way to read the similarity metric back into a script variable which is how the current SSIM metric is handled. Perhaps this could be implemented by passing a (global?) variable name and GMSD / MDSI functions would assign that variable?

It is possible to write the data to a file via FrameEval or get_frame.

For example,

def output_frameprop(
clip: vs.VideoNode, filename: str, prop_name: str
) -> vs.VideoNode:

records = []
frames = clip.num_frames

def output(n, f, clip=clip):
records.append((n, f.props[prop_name]))

nonlocal frames
frames -= 1

if frames == 0:
with open(filename, "w") as f:
f.write(f"frameIdx {prop_name}\n")
f.writelines(
f"{idx:8} {value:{len(prop_name)}}\n"
for idx, value in sorted(records))

return clip

return core.std.FrameEval(clip, output, clip)


clip = calculate_some_metric(clip)
clip = output_frameprop(clip, "stats.txt", metric_name)


And evaluate the script from beginning to end via vspipe / benchmark utility from vsedit. The result is then in file "stats.txt".

[ChaosKing]

@WolframRhodium I think he means the avisynth version. For VS we already have this scrpt here https://github.com/theChaosCoder/zoptilib/blob/master/zoptilib.py

[WorBry]

Yes, I mean the ported AVISynth version. It appears that Asd-g is not a forum member.

[WolframRhodium]

Sorry about that, I did not read the thread carefully. I thint it is possible through WriteFile, but I am not very familiar with it.

[ChaosKing]

Looks like there is no way to read the similarity metric back into a script variable which is how the current SSIM metric is handled. Perhaps this could be implemented by passing a (global?) variable name and GMSD / MDSI functions would assign that variable?

I think it should work like in Vapoursynth now.

https://github.com/Asd-g/AviSynthPlus-Scripts/blob/6604ddaadb84697c0c718a9fefd085e7e60c46a2/MDSI.avsi#L103

"""propSet("_FrameMDSI",

Avisynth+ 3.X supports clip props just like vapoursynth.

They are read for show=true at the script bottom ScriptClip("""Subtitle("FrameMDSI: " + String(propGetFloat("_FrameMDSI")))"""

So maybe zopti needs a "zoptilib.py" version for avisynth now?

[zorr]

Thanks Chaos, I didn't notice that the metric can indeed be read from the frame properties. So here's an example script using GMSD:

TEST_FRAMES = 5 # how many frames are tested
MIDDLE_FRAME = 50 # middle frame number

RawSourcePlus("D:/optimizer/test/flower/flower_cif.yuv", width=352, height=288, pixel_type="I420")
source=ColorYUV(levels="PC->TV")
noisy=source.AddGrain(25, 0, 0, seed=1)
#return noisy

#denoised=noisy.FFT3DFilter(sigma=4, bt=4, bw=16, bh=16, ow=8, oh=8) # best settings by Fizick

sigma = 400/100.0 # optimize sigma = _n_/100.0 | 100..800 | sigma
bt = 4 # optimize bt = _n_ | -1..5 | blockTemporal
blockSize = 32 # optimize blockSize = _n_ | 2..64 ; min:overlap 2 * | blockSize
overlap = 16 # optimize overlap = _n_ | 0..32 ; max:blockSize 2 / | overlap
denoised=noisy.FFT3DFilter(sigma=sigma, bt=bt, bw=blockSize, bh=blockSize, ow=overlap, oh=overlap)

# cut out the part used in quality / speed evaluation
source = source.Trim(MIDDLE_FRAME - TEST_FRAMES/2 + (TEST_FRAMES%2==0?1:0), MIDDLE_FRAME + TEST_FRAMES/2)
denoised = denoised.Trim(MIDDLE_FRAME - TEST_FRAMES/2 + (TEST_FRAMES%2==0?1:0), MIDDLE_FRAME + TEST_FRAMES/2)
last = denoised

last = GMSD(source, denoised)

global total = 0.0
global gmsd_total = 0.0
FrameEvaluate(last, """
global gmsd = propGetFloat("_PlaneGMSD")
global gmsd = (gmsd == 0.0 ? 1.0 : gmsd)
global gmsd_total = gmsd_total + gmsd
""")

# measure runtime, plugin writes the value to global avstimer variable
global avstimer = 0.0
AvsTimer(frames=1, type=0, total=false, name="Optimizer")

# per frame logging (gmsd, time)
delimiter = "; "
resultFile = "perFrameResults.txt" # output out1="gmsd: MIN(float)" out2="time: MIN(time) ms" file="perFrameResults.txt"
WriteFile(resultFile, "current_frame", "delimiter", "gmsd", "delimiter", "avstimer")

# write "stop" at the last frame to tell the optimizer that the script has finished
frame_count = FrameCount()
WriteFileIf(resultFile, "current_frame == frame_count-1", """ "stop " """, "gmsd_total", append=true)

return last

This is just a slightly modified version of the script I used in the hands-on tutorial (https://forum.doom9.org/showthread.php?p=1851670#post1851670). Other than renaming the variables the only relevant changes are highlighted in red.

In this script GMSD gave indentical optimal values compared to SSIM except for the sigma which is 415 instead of 479. I didn't try MDSI yet.

[Boulder]

Is there a way to utilize Avisynth+'s Prefetch to speed up calculation? I tested it briefly but there were problems probably related to writing to the file asynchronously.

[zorr]

Is there a way to utilize Avisynth+'s Prefetch to speed up calculation? I tested it briefly but there were problems probably related to writing to the file asynchronously.

I haven't tried it myself but it's worth an investigation. I think there's going to be problems if multiple threads are writing to the same file.

Is it possible to force WriteFile to execute on one thread while the filters before it are multithreaded? The documentation states that you cannot set MT mode on script function calls, but would placing a MT_SERIALIZED plugin call just before WriteFile have that side effect? Or can you just say SetFilterMTMode("WriteFile", MT_SERIALIZED)?

On the other hand, perhaps the WriteFile is already MT_SERIALIZED but the threads are writing into it in arbitrary order and the last line is not always last as it should be. In that case the per frame results should be written into a temporary thread-safe variable which would be written to file in correct order when all the frames are finished. It works like that in Vapoursynth using a map, but Avisynth doesn't have such a data structure so I'm not sure how to implement that. Perhaps a simple array would suffice as long as it's thread safe.

[Boulder]

I'm not sure if WriteFile is internally marked as MT_SERIALIZED, but in general, it should already have a "correct" mode set. It is very much likely that the order of frames is very random since they finish at different times.

Setting both Prefetch(threads=1, frames=24) and Prefetch(threads=24, frames=1) give this error: java.lang.NumberFormatException: For input string: "1I don't know what 'delimiter' means.0.179144I don't know what 'delimiter' means.3150.284912" and the log file is full of these errors.