Zopti

[Are_]

My OS is Gentoo Linux, I used self compiled with GCC builds of everything (via package manager).

[EDIT]
clip = core.resize.Bicubic(clip, format=vs.YUV444PS)
clip = core.resize.Bicubic(clip, format=vs.RGBS, matrix_in_s=matrix)
clip = core.resize.Bicubic(clip, format=vs.RGBS, transfer_in_s=transfer, transfer_s='linear')
You will need to work around the fact that bitdepth is in the format constant.

[Are_]

I know you don't need, but the shortcuts sometimes produce a different result, that 50% is not free, last time I checked out it was not that different though.

[ChaosKing]

Yep, this shows definitely some differences.
clip = core.resize.Bicubic(orig, format=vs.RGB24, matrix_in_s=matrix)
long = core.resize.Bicubic(clip, format=vs.RGB24, transfer_in_s=transfer, transfer_s='linear')

short = core.resize.Bicubic(orig, format=vs.RGB24, matrix_in_s=matrix, transfer_in_s=transfer, transfer_s='linear')

clip = core.Butteraugli.butteraugli(short, long)

[poisondeathray]

Yep, this shows definitely some differences.
clip = core.resize.Bicubic(orig, format=vs.RGB24, matrix_in_s=matrix)
long = core.resize.Bicubic(clip, format=vs.RGB24, transfer_in_s=transfer, transfer_s='linear')

short = core.resize.Bicubic(orig, format=vs.RGB24, matrix_in_s=matrix, transfer_in_s=transfer, transfer_s='linear')

clip = core.Butteraugli.butteraugli(short, long)


It's different because the "long" method uses 2 discrete operations (more rounding errors at 8bit precision)

If you use RGBS for the 1st clip step, you get same results

So the short method is better, faster, more accurate

[ChaosKing]

ok thx.
A quick test with RGBS first and then convert to RGB24:

clip = core.resize.Bicubic(orig, format=vs.RGBS, matrix_in_s=matrix)
long = core.resize.Bicubic(clip, format=vs.RGBS, transfer_in_s=transfer, transfer_s='linear')

short = core.resize.Bicubic(orig, format=vs.RGBS, matrix_in_s=matrix, transfer_in_s=transfer, transfer_s='linear')

short = core.resize.Bicubic(short, format=vs.RGB24)
long = core.resize.Bicubic(long, format=vs.RGB24)

clip = core.Butteraugli.butteraugli(short, long)
Now there are 0 differences.
Then I tested the speed (RGBS-long vs RGBS-short) ... and wow ... a 100fps difference!!! 163 fps vs 260 fps

[zorr]

clip = core.resize.Bicubic(clip, format=vs.YUV444PS)
clip = core.resize.Bicubic(clip, format=vs.RGBS, matrix_in_s=matrix)
clip = core.resize.Bicubic(clip, format=vs.RGBS, transfer_in_s=transfer, transfer_s='linear')
You will need to work around the fact that bitdepth is in the format constant.

Thanks, got it working and there are no crashes anymore.

If you use RGBS for the 1st clip step, you get same results

So the short method is better, faster, more accurate

I verified this myself, the short method is equivalent to the long method in quality and is faster so it's preferred.

I also wanted to see if there are differences between fmtconv and core.resize. Looks like there are some, core.resize gives slightly brighter result as can be seen in the levels histogram.

https://i.postimg.cc/L508gSjb/rgbtest-vpy-2.gif

It's much more difficult to see any differences in the actual video frame though (and especially in the example as the colors have been munged by gif).

The differences are smaller (although spread to wider area) when using dithering with core.resize so I will enable that, performance didn't change much with the different dithering options. [EDIT] The third frame is the difference heat map from Butteraugli.

https://i.postimg.cc/Rq5VLRb7/rgbtest-vpy-none.png

https://i.postimg.cc/yxS8R5n6/rgbtest-vpy-error-diffusion.png

[EDIT] I have updated Zoptilib (https://github.com/theChaosCoder/zoptilib) with the new RGB conversion implementation. You can still use matrix '601' even though core.resize doesn't have it, it's the same as '470bg' and '170m' (those two give identical results).

[zorr]

I made some comparisons of the five similarity metrics. I used the same video as in the above post. Perhaps not the best choice since it looks like it was recorded with a potato but the small size makes the optimization task much faster.

I added some noise with grain.Add(25, seed=3) and then denoised it with FFT3DFilter using only bt, blockSize (arguments bw and bh), overlap (arguments ow and oh) and sigma. FFT3DFilter has many more arguments but the goal here is to compare the similarity metrics and not to find the best possible quality.

I ran an optimization with the default settings except set the iterations to 10000 to make sure I find the best settings, five runs per metric, five frames used in the metric calculation. No downscaling in any of the metrics (which is the default in the latest Zoptilib).

The first image here shows the best found result for each metric and heatmaps of the argument pairs bt / sigma and blockSize / overlap. The reddish dot in the maps is located at where the best result was found.

https://i.postimg.cc/HdHLQjrQ/comparison-1.png

Let's first focus on what the metrics agree on: the best result has bt=5 which is the maximum supported (two previous, current and two next frames). Also the best overlap is always half the blockSize.

BlockSize varies from 10 to 22. VMAF result is an outlier in this group, the other metrics have best blockSize at 10, 12 or 14.

Sigma varies from 308 to 482. Butteraugli has the smallest sigma, SSIM the largest. It's interesting that Butteraugli really dislikes the larger sigmas, VMAF does that too but it's a bit hard to see in this image. I suspect that there are some artifacts with the larger sigmas but most of the metrics are not sensitive to it. I will do some manual image quality comparisons later to very this.

SSIM, GMSD and MDSI are very "smooth", there are no large jumps in results when the parameter's value changes a little (with the exception of bt where large jumps happen at the lowest values). Butteraugli and VMAF look rougher. The smoothness is a good feature when trying to find the optimal settings, makes the search easier.

The VMAF results are pretty weird. It really likes bt=-1 ("sharpen only") which the other metrics considered inferior. There was a really good set of parameters at sigma=170 (actually any sigma, it doesn't change the result) blockTemporal=-1 blockSize=50 overlap=0 with score 98.62038 whereas the best result at sigma=381 blockTemporal=5 blockSize=22 overlap=11 had score 98.62366. The difference was small enough that three out of five runs didn't find the optimal value but this secondary set instead.

To be continued...

[zorr]

Let's take a closer look at how the metrics behave near the optimal result. I run exhaustive searches where two of the four arguments were locked to the best result found with each metric. The other two arguments are tested with every combination.

The first pair is sigma and bt (blockTemporal):

https://i.postimg.cc/3K6mLyRw/comparison-2-ex-si-bt.png

I have limited the x resolution to 100 in the maps but the results were calculated for every sigma from 100 to 800 (divided by 100 so the actual range is 1.0 - 8.0) so each box represents the best found result of 8 sigmas.

SSIM and GMSD are very similar, GMSD doesn't like the high sigmas as much though (and it's also reflected in the sigma of the best result).

MDSI is a bit more focused, area of good results is smaller. While SSIM and GMSD didn't have a large improvement from bt=4 to bt=5 MDSI thinks there's a bigger difference.

Butteraugli is more uneven compared to the above three. The best sigma seems to get smaller with smaller bt. For some reason Butteraugli thinks bt=0 is much better than the above three metrics.

And finally VMAF... still thinking bt=-1 means high quality. It also strongly dislikes bt=1 and bt=2 with high sigmas, so much that it's hard to see any differences elsewhere.

Let's see the other pair blockSize / overlap:

https://i.postimg.cc/9W65cp0H/comparison-2-ex-bs-ol.png

Again SSIM and GMSD are very similar, but GMSD is more focused.

MDSI continues the trend and is even more focused. All three have better scores when overlap increases. Looks like the optimal value is always at overlap = blockSize/2 for every blockSize.

Butteraugli is kinda close to MDSI near the optimum result but there are some weird stripes elsewhere.

The VMAF on the other hand looks like it had a meltdown. There's no clear optimal range, values jump up and down even with small changes. The above mentioned rule overlap=blockSize/2 is also broken. Very hard for the optimizer to make sense of.

To be continued...

[ChaosKing]

Maybe it means that visually there is not that much difference? Have you checked some good and some bad candidates yourself? Or this source needs a different model?

[zorr]

Maybe it means that visually there is not that much difference? Have you checked some good and some bad candidates yourself? Or this source needs a different model?

No I haven't done that yet, but I intend to. It's possible that there isn't that much visual difference but it would be strange if doing mere sharpening (bt=-1) would look as good as 5D temporal denoising.

[zorr]

Before I start comparing the results visually I wanted to make one more exhaustive search using the knowledge gained earlier. Since we now know the best result always has bt=5 that can be left out of the search. Also the best result has overlap=blockSize/2 (except with VMAF, but it lost the vote on this one) so that can be used as well. The only things remaining are sigma and blockSize, so let's see what that looks like. I calculated results for even blockSizes only.

https://i.postimg.cc/q43GYPss/comparison-3-ex-si-bs.png

It's the same story again: GMSD is a bit more focused than SSIM, MDSI is more focused than GMSD. SSIM doesn't give almost any penalty for the higher sigmas but Butteraugli clearly dislikes them. GMSD and MDSI give smallish penalty for the higher sigmas. And again Butteraugli is a bit uneven and has a couple of stripes but the amount is quite moderate and I don't think it will be much of a problem for the optimizer.

The first VMAF map is almost white because the scores for the small blockSizes are so bad. In the second map I started from blockSize 10 up and we can see the details better. VMAF has more stripes and "noise" but it's easy to see that it also dislikes the higher sigmas.

I also wanted to see how VMAF sees the other highly rated parameter combination sigma=170 blockTemporal=-1 blockSize=50 overlap=0. So in this run the bt was set to -1 and overlap to 0. Looks like I was wrong when I said the sigma doesn't effect the score. It does, but only with specific blockSizes. The best blockSize 50 has red dots everywhere because the same top result was found in 699 out of 801 sigmas. For some reason the blockSizes divisible by 10 are usually good.

[ChaosKing]

Now it needs to be tested with some other sources to confirm the results xD I wonder how the results will change with a "HQ/clean" source or a clip with fine details.

Btw would it be possible to implement parallel execution to speed up things? Where the 'time' dimension is not important like in your test :)

[zorr]

Now it needs to be tested with some other sources to confirm the results xD

:scared:

I wonder how the results will change with a "HQ/clean" source or a clip with fine details.

Yes you're right, we can't make any deep conclusions from this one source. All right, I will do another test run. Perhaps Boulder's blacksails.avi would be a good one.

Btw would it be possible to implement parallel execution to speed up things? Where the 'time' dimension is not important like in your test :)

Yes. JMetal (the metaheuristics library used in Zopti) supports parallel evaluators, I just need to make sure that my own code is thread-safe before I enable that. For some of the algorithms (mutation, exhaustive) I have to make my own parallel implementation but that's not a problem.

There might not be that much of a speed improvement though. At least on my 4-core machine most of the cores are fully utilized even with one evaluation thread when using Vapoursynth.

[ChaosKing]

There might not be that much of a speed improvement though. At least on my 4-core machine most of the cores are fully utilized even with one evaluation thread when using Vapoursynth.

My 8 core 16 thread cpu is still hungry though :D
yeah the speed increase will be mainly in shorter clips or 1 frame clips.
But I think we can also scale better if we limit the thread count in VS and use multiple instances.

p.s. Does someone know there to find very clean animated (cartoon/anime) hd material. Rick and Morty for example looks very clean and doesn't have halos, noise, etc

[Mystery Keeper]

p.s. Does someone know there to find very clean animated (cartoon/anime) hd material. Rick and Morty for example looks very clean and doesn't have halos, noise, etcDisney's "Frozen" seems noiseless.

[WolframRhodium]

Here (https://gist.github.com/WolframRhodium/d4b117ccc98081e40a70946b884dbe36) is my port of WaDIQaM (https://github.com/lidq92/WaDIQaM), a CNN-based full-reference image quality evaluator.
(The paper also addresses the problem of no-reference evaluation, but the PyTorch re-implementation only contains a pre-trained FR model. Lets check it first)

One should install PyTorch and download the model at here (https://github.com/lidq92/WaDIQaM/blob/master/models/WaDIQaM-FR-TID2008) before testing.

I post the .py file on GitHub's gist since this is experimental.

[ChaosKing]

Another toy \o/

Mini tut how to install in portable fatpack
From here https://pytorch.org/get-started/locally/ I installed the packages.

Start powershell inside VapourSynth64 folder and run this command:
.\python.exe -m pip install https://download.pytorch.org/whl/cu100/torch-1.0.1-cp37-cp37m-win_amd64.whl <- (I used cuda 10)
.\python.exe -m pip install torchvision

Maybe torchvision is not necessary?

Quick benchmark 1080p
- cpu: 1.14 fps (Ryzen1700 @ 3.7ghz)
- cuda: 19.5fps (GTX 1070)


My script
clip = core.lsmas.LWLibavSource(r"E:\a.mkv").grain.Add(60)
clip2 = core.lsmas.LWLibavSource(r"E:\a.mkv")

clip = mvf.ToRGB(mvf.Depth(clip, 32))
clip2 = mvf.ToRGB(mvf.Depth(clip2, 32))

clip = mm.vs_wadiqam(clip, clip2, model_path=r"D:\Fansubs\WaDIQaM-FR-TID2008", seed=2019, use_cuda=False)

clip = core.text.FrameProps(clip, props=['Frame_WaDIQaM'])
clip.set_output()

Without addgrain I get a score of 6.x and with grain 4.x. vmaf would show 99-100 (with same clip for both inputs), but 6 seems so random + it stays between 6.0 - 6.4 depending on the current frame number. Is this correct?

[zorr]

Thanks Wolfram, it's an interesting metric.

I have updated Zoptilib (https://github.com/theChaosCoder/zoptilib) but it's not related to WaDIQaM, just some minor improvements I have done. I figured Chaos is about to add the WaDIQaM there soon so I better make my edits first. :)

[WolframRhodium]

Maybe torchvision is not necessary?

Yes, it has been removed. Thanks.


Without addgrain I get a score of 6.x and with grain 4.x. vmaf would show 99-100 (with same clip for both inputs), but 6 seems so random + it stays between 6.0 - 6.4 depending on the current frame number. Is this correct?


I'm not sure. I got similar results here. Maybe I should switch to official Chainer-based implementation later. :confused:`

Another consideration is that, while the official implementation uses every 32x32 sized patches in an image in quality evaluation, current PyTorch-based implementation only randomly samples 32 patches (adjustable) for acceleration.
It seems that it is acceptable to use every patches? The algorithm runs faster than I expected.

[WolframRhodium]

Official deepIQA (https://github.com/dmaniry/deepIQA) (WaDIQaM-FR and WaDIQaM-NR) has been ported in the same address (https://gist.github.com/WolframRhodium/d4b117ccc98081e40a70946b884dbe36).

[zorr]

This is what the best results look like for each metric. On the first row is the original frame and the frame after added noise. The rows after that show the best denoising result from each metric and an enhanced comparison view (I used muvsfunc Compare with power 1.4 and mode 2).

[EDIT] Seems like there are some weird patterns in that image when shown on the forum page. Those are not present if you open the image alone (at least in Firefox).
[EDIT2] Apparently that was because I had the page at 110% magnification which also scales images.

https://i.postimg.cc/FrcDTgnV/denoise2-analyze-vpy-0.png

It's really hard to see any major differences between the contenders. The task seems to have been too easy and using more noise would perhaps made the differences more clear. Oh well, I can always run the tests again...

The lower sigma preferred by Butteraugli has left some noise to the sky but on the other hand looking at the enhanced view the branches are no longer sticking out as clearly as in the other versions.

The last row shows VMAF with sigma=170 blockTemporal=-1 blockSize=50 overlap=0 which scored almost as good as the best settings. Clearly this mode is inferior and has hardly removed any noise. So at least we can conclude that sometimes VMAF gives bad results.

[VS_Fan]

... The last row shows VMAF with sigma=170 blockTemporal=-1 blockSize=50 overlap=0 which scored almost as good as the best settings. Clearly this mode is inferior and has hardly removed any noisebt= -1 is "Sharpen only". It won't reduce any noise but only sharpen the image.

May be bt=1 (2D) Spatial only is what you could use to get some interesting results. Also, you could try different noise values (sigma, sigma2, sigma3 and sigma4) for high, mid-high, mid-low and lower frequencies

[WorBry]

@ WolframRhodium,

Regarding MDSI. Is there provision in the original code for producing the Gradient Similarity (GS), Chromacity Similarity (CS) and pooled Gradient-Chromacity Similarity (GCS) maps shown in the original paper ?

http://i.imgur.com/tNSGgiHl.png (https://imgur.com/tNSGgiH)

https://arxiv.org/pdf/1608.07433.pdf

I've been using quality metrics to compare various 'visually lossless' intermediate codecs and I'm seeing some interesting results with MDSI that appear to be chroma-related - they don't occur when testing with greyscale images. The similarity maps would helpful in identifying the implicated image components.

[WolframRhodium]

Regarding MDSI. Is there provision in the original code for producing the Gradient Similarity (GS), Chromacity Similarity (CS) and pooled Gradient-Chromacity Similarity (GCS) maps shown in the original paper ?


I have updated the code of MDSI. Usage:

src, gs, cs, gcs = MDSI(src, ref, show_maps=True)


You may have to normalize these maps yourself for better visualization.

[ChaosKing]

Official deepIQA (https://github.com/dmaniry/deepIQA) (WaDIQaM-FR and WaDIQaM-NR) has been ported in the same address (https://gist.github.com/WolframRhodium/d4b117ccc98081e40a70946b884dbe36).

So, your "old" WaDIQaM is now vs_wadiqam_pytorch.py?
And vs_wadiqam_chainer is new with 2 "new" implementations , wadiqam_fr & wadiqam_nr (Full-reference WaDIQaM & No-reference WaDIQaM)?
Correct? Is it ready to test?

[WolframRhodium]

So, your "old" WaDIQaM is now vs_wadiqam_pytorch.py?
And vs_wadiqam_chainer is new with 2 "new" implementations , wadiqam_fr & wadiqam_nr (Full-reference WaDIQaM & No-reference WaDIQaM)?
Correct? Is it ready to test?

Yes, and I think it is ready to test. I have verified scores on some images.

Note I have make a modification to the code to make the result deterministic. Issue (https://github.com/dmaniry/deepIQA/issues/9#issuecomment-469211408)

[WorBry]

I have updated the code of MDSI. Usage:

src, gs, cs, gcs = MDSI(src, ref, show_maps=True)


You may have to normalize these maps yourself for better visualization.

Thanks a lot. ChaosKing/zorr, any chance you could update zoptilib accordingly ? Cheers.

[zorr]

Thanks a lot. ChaosKing/zorr, any chance you could update zoptilib accordingly ? Cheers.

You can set extra arguments using addParams.

For example in this case:

zopti.addParams('mdsi', dict(show_maps=True))

[WorBry]

Thanks, but I'm getting this error:

final = alt_clip.std.FrameEval(functools.partial(calc, clip=alt_clip, data=data), prop_src=prop_src)
AttributeError: 'tuple' object has no attribute 'std'

[ChaosKing]

Yeah show_map returns not a clip but a clip tuple https://github.com/WolframRhodium/muvsfunc/blob/116aa4a8db0403d44450e9115504e585453f7e3d/muvsfunc.py#L4584
I think FrameEval awlays expects a clip as return!?

[ChaosKing]

When I try to show the clip in vsedit I get this error msg:
GREY color family cannot have RGB matrix coefficients
orig = mvf.ToRGB(orig)
muv.MDSI(clip, orig, show_maps=True)[1]

and with mvf.ToYUV( muv.MDSI(clip, orig, show_maps=True)[1] )
Resize error 1026: YUV color family cannot have RGB matrix coefficients.

How can I remove RGB matrix coefficients? Or what should I do?

[WorBry]

Yeah show_map returns not a clip but a clip tuple https://github.com/WolframRhodium/muvsfunc/blob/116aa4a8db0403d44450e9115504e585453f7e3d/muvsfunc.py#L4584
I think FrameEval awlays expects a clip as return!?

So what to do ?

[WorBry]

When I try to show the clip in vsedit I get this error msg:
GREY color family cannot have RGB matrix coefficients
orig = mvf.ToRGB(orig)
muv.MDSI(clip, orig, show_maps=True)[1]

and with mvf.ToYUV( muv.MDSI(clip, orig, show_maps=True)[1] )
Resize error 1026: YUV color family cannot have RGB matrix coefficients.

How can I remove RGB matrix coefficients? Or what should I do?


core.std.SetFrameProp(clip, prop='_Matrix', delete=True)


I'm confused now. Is this a way to output the maps ? What does the [1] at the end of the MDSI call line do ?

[ChaosKing]

I'm confused now. Is this a way to output the maps ? What does the [1] at the end of the MDSI call line do ?

show_maps returns multiple clips
if show_maps:
return clip1, gs_hvs, cs, gcs

muv.MDSI(clipA, clipB, show_maps=True)[0] <-- 0 means clip1
muv.MDSI(clipA, clipB, show_maps=True)[1] <-- 1 = gs_hvs, 2 = cs, 3 = gcs

OR

clips = muv.MDSI(clipA, clipB, show_maps=True)
clip = clips[0]
gs_hvs = clips[1]
...

This works now, thx HolyWu
orig = muv.MDSI(orig, orig.grain.Add(42), show_maps=True)[2]
orig = core.std.SetFrameProp(orig, prop='_Matrix', delete=True)

[WorBry]

Yep, that works. Thanks.

[Iron_Mike]

Hi guys,

trying to run some metrics via Zopti... got a couple of questions...

(1) what is the easiest way to pass an image sequence (specifically EXR/DPX) to Zopti ?

(2) @WolframRhodium: why does muvsfunc only allow clips of the exact same format for the SSIM / GMSD metric ?

Thanks.

[poisondeathray]

(1) what is the easiest way to pass an image sequence (specifically EXR/DPX) to Zopti ?


ImageMagick .

The vapoursynth version supports many pixel formats, 10bit, 16bit, float .... all in their native format (e.g. DPX will be 10bit RGB, instead of 16bit)
http://www.vapoursynth.com/doc/plugins/imwri.html

(The avs version is very outdated, 8bit, limited format support)

[WolframRhodium]

(2) @WolframRhodium: why does muvsfunc only allow clips of the exact same format for the SSIM / GMSD metric ?

I expect people to convert format of clips themselves. I believe that allows the program to be correct, stable and concise.

[Iron_Mike]

ImageMagick .

The vapoursynth version supports many pixel formats, 10bit, 16bit, float .... all in their native format (e.g. DPX will be 10bit RGB, instead of 16bit)
http://www.vapoursynth.com/doc/plugins/imwri.html

(The avs version is very outdated, 8bit, limited format support)

thank you, that worked.

[Iron_Mike]

I expect people to convert format of clips themselves. I believe that allows the program to be correct, stable and concise.

trying to understand the approach here... your implementation can only compare the exact same video format ?

so if we want to compare a 16bit full RGB master with a 12/10/8 bit encode in various chroma sub sampling formats, we would convert the master to the format of the encode... ?

or how is this done correctly ?

Thanks.

[WorBry]

I have updated the code of MDSI. Usage:

src, gs, cs, gcs = MDSI(src, ref, show_maps=True)


You may have to normalize these maps yourself for better visualization.

The traces are very faint, but an Unsharp Mask (two passes if needs be) brings them up nicely.

[WolframRhodium]

your implementation can only compare the exact same video format ?

Yes.


so if we want to compare a 16bit full RGB master with a 12/10/8 bit encode in various chroma sub sampling formats, we would convert the master to the format of the encode... ?

Normally it would be:

yuv420p8 = core.resize.Bicubic(rgb48, format=vs.YUV420P8, matrix_s='709')

for Blu-ray material.

But it won't work for every case, like chroma locations other than left, or different matrix specification.

[Iron_Mike]

yuv420p8 = core.resize.Bicubic(rgb48, format=vs.YUV420P8, matrix_s='709')

thanks for that, ran some test with the resize cmd you posted and with core.fmtc... results seems to be almost identical...

now, why do you downscale the master and not upscale the encode to the masters format before comparison ?

Thanks.

[WolframRhodium]

now, why do you downscale the master and not upscale the encode to the masters format before comparison ?

It's just a trade-off. Downscaling is fast but inaccurate, while upscaling is accurate but slow. I'm just make a demonstration here instead of providing a high-quality conversion. (The quality of upscaling by nnedi3 is even higher, for example)

[WorBry]

@ WolframRhodium,

Regarding MDSI. Is there provision in the original code for producing the Gradient Similarity (GS), Chromacity Similarity (CS) and pooled Gradient-Chromacity Similarity (GCS) maps shown in the original paper ?

http://i.imgur.com/tNSGgiHl.png (https://imgur.com/tNSGgiH)

https://arxiv.org/pdf/1608.07433.pdf

I've been using quality metrics to compare various 'visually lossless' intermediate codecs and I'm seeing some interesting results with MDSI that appear to be chroma-related - they don't occur when testing with greyscale images. The similarity maps would helpful in identifying the implicated image components.

Thought the results might be of interest:

https://forum.doom9.org/showthread.php?p=1868159#post1868159

[ChaosKing]

https://github.com/theChaosCoder/zoptilib
New parameter: showstats - show metric stats per frame (vmaf and time are not supported) - Zopti(r'results.txt', metrics=['ssim', 'gmsd', 'mdsi', 'butteraugli', 'time'], showstats=True, matrix='709')
Updated WaDIQaM (pytorch) and added WaDIQaM nr/fr (chainer), untested bcs it needs cupy and other stuff. Lots of gigabytes und modules needs to be installed first.
https://i.imgur.com/RuoSMxc.png

Now I don't like the forced heatmap in butteraugli :/

[WorBry]

https://github.com/theChaosCoder/zoptilib
New parameter: showstats - show metric stats per frame (vmaf and time are not supported) - Zopti(r'results.txt', metrics=['ssim', 'gmsd', 'mdsi', 'butteraugli', 'time'], showstats=True, matrix='709')

Very useful. Thanks for that.

Now I don't like the forced heatmap in butteraugli :/

Me neither.

[zorr]

https://github.com/theChaosCoder/zoptilib
New parameter: showstats - show metric stats per frame

Great addition. It would look cleaner if Butteraugli stats were lined with the others. So can we shorten it to... butt?

[ChaosKing]

done :)

[ChaosKing]

https://github.com/theChaosCoder/zoptilib
Mdsi and butteraugli no longer contaminate the clip. That means metrics=['ssim', 'gmsd', 'mdsi', 'butteraugli'] is the same as metrics=['mdsi', 'butteraugli', 'ssim', 'gmsd'] now.
This also means that you won't see the heatmap...