NHW Project - a free state-of-the-art image/video compression codec

[nhw_pulsar]

Originally Posted by benwaggoner

AOM seems to care about VMAF as well, which is the least-bad broadly available metric. But not even VMAF solves the problem of how to score temporal variations in quality, like I or B frame strobing. All our metrics are fundamentally per-frame metrics, and it isn't uncommon for two frames with the same metrics to look quite a bit different, even if the overall subjective quality is in the same ballpark. We commonly see this encoding grain, where the grain texture "jumps" between frames in an unnatural and distracting way that metrics are poor at detecting.

Concerning VMAF, is it true that if you improve/increase contrast of the image then you really "bump" the VMAF score? I'll maybe have to inspire from VMAF, but I'll rather make my metrics neatness-sensitive rather than contrast-sensitive.

[nhw_pulsar]

Hello,

For those interested, I have released the 0.2.6 version of NHW.This new version has again a better neatness and a better precision thanks to an improved pre_processing.

More at: http://nhwcodec.blogspot.com/

Personally I find that this new version really starts to be interesting, and I would like then to reconnect with the industry, if you think you can help me in this process, do not hesitate to let me know.

Many thanks.
Cheers,
Raphael

[nhw_pulsar]

Hello,

For those interested, I have released the 0.2.7 version of NHW.This new version has a better precision and less oversharpening, but still a very good neatness (for me), thanks to an improved pre-processing.

More at: http://nhwcodec.blogspot.com/

I am sorry for changing versions so quickly, but it seems that NHW is ultra-sensitive to the pre_processing, and so any modifying of it has quite an impact on the visual quality...
-I think I'll continue to focus on the pre_processing for the 0.2.x version series.From 0.3.0 version, I'll try to improve other stages: like better quantizing, better wavelet coding, add a post-processing filter that will remove aliasing,...-

Any feedback on this new version welcome!

Cheers,
Raphael

[nhw_pulsar]

Hello,

Just a quick update, for those it might interest, thanks to a new contributor, NHW has now a new and better cli interface (and cli separation from the library).

I have also more deeply visually evaluated the last version 0.2.7, and I start to be really satisfied now (I'll continue still to explore/refine the pre_processing as much as I can for now).Some people reminded me that PSNR and SSIM are catastrophic with NHW, but for my eyes, NHW performs really visually better than these metrics could suggest...

Also some people don't agree that NHW is faster than JPEG, actually in 2014 I could download IJG JPEG binaires on the net compiled from IJG JPEG C code, I found that it could be a fair comparison because IJG provides an optimized C code but with no assembler, SIMD, multithreading,... and NHW is totally unoptimized C code, and so based on these binaries, NHW was faster than IJG JPEG, but apparently there is a problem with these IJG JPEG provided binaires, at least some people think there is necessarily a problem... If you could shed some light on NHW speed vs JPEG speed, it would also be quite useful.

Cheers,
Raphael

[nhw_pulsar]

Hello,

Tansy has made a speed comparison between NHW and NConvert JPEG, and in his study, NHW is faster to encode and faster to decode than JPEG.

More details on tansy's study here: https://encode.su/threads/3002-Is-en...ll=1#post77107

Cheers,
Raphael

[nhw_pulsar]

Hello,

I have released the 0.2.8 version of NHW.This new version has a better precision, and still a very good neatness, thanks again to an improved pre_processing.

Also, this new version has a new and better cli interface.Be careful to the new cli encoding/decoding syntaxes.-See readme or -h option for more details.-

As a reminder, NHW is developed/optimized for -q8 to -q23 compression settings for now.

More at: http://nhwcodec.blogspot.com/

Cheers,
Raphael

[nhw_pulsar]

Hi,

A developer contacted me because he has modified and improved NHW_neatness_metrics.As a ready-made testing tool, he recommends using jpeg-recompress ( https://github.com/ImageProcessing-E...peg-recompress ): jpeg-recompress and jpeg-compare. It contains the necessary minimum metrics for comparison: psnr, mpe, ssim, ms-ssim, smallfry, shbad, nhw, cor, corsh.More instruction to use the tool, here: https://github.com/rcanut/NHW_Neatness_Metrics/issues/1 .Also after first test, NHW new metrics seems to work well, for example it could be better than SSIM (and PSNR) with jpeg-recompress.So this is quite interesting.

For those interested, here is also the Github for the new metric: https://github.com/ImageProcessing-E...ns/libsmallfry .

Do not hesitate to give your feedback to the author if you have time to test.

Cheers,
Raphael

[nhw_pulsar]

Hello,

For those interested, I have released the 0.2.8.1 version of NHW.This new version has a better neatness and a better precision thanks to an improved pre_processing based on psychovisual tunning.

More at: http://nhwcodec.blogspot.com/

Cheers,
Raphael

[nhw_pulsar]

Hi,

I have released the 0.2.8.2 version of NHW.This new version has a little more neatness and a little more precision thanks to the further improvement of the pre_processing.

I did not make extensive visual tests, but for example at -q15 and -q14 settings, I find that this new version starts to have a good balance between neatness and precision.

More at: http://nhwcodec.blogspot.com/

Cheers,
Raphael

[nhw_pulsar]

Hello,

With the recent improvements, the NHW codec starts to be interesting and some people share that opinion and urge me then to adapt NHW to any image size because it's very important.

The problem is that I won't be able to do that big effort, and so my only solution is to try to find a company/organization that could be interested in NHW and that would like to adapt it to any image size.

Because it's rather difficult, as AOM, MPEG, JPEG can not evaluate NHW because their evaluation test images are bigger than 512x512, and so it is also a very difficult position for me that the compression bodies answer me:"We don't know if your work is good or bad, we simply just can't evaluate it..."

If you think a company/organization could invest some time in NHW, do not hesitate to give some advice.

Cheers,
Raphael

[nhw_pulsar]

Hello,

For those interested, I have released the 0.2.9 version of NHW.I continued my improving of the pre_processing and so this new version is better.

As I said, from 0.3.0 version when I finish the pre_processing, I'll make a demo between version 0.3.0 and version 0.2.0 showing the visual impact of the pre_processing.I think I am not far from it, as I start to be really satisfied of the balance between neatness and precision with this version, and with even more pushing the pre_processing, maybe to its limit (?) ... -I'll also have to look at other stages...-

More at: http://nhwcodec.blogspot.com/

Cheers,
Raphael

[nhw_pulsar]

Hello,

For those interested, I have released the 0.2.9.1 version of NHW.I have psycho-visually fine-tuned the pre_processing in the encoder, and so this new version is better.

It is not so evident however as I validate my versions by eyes, and my eyes were maybe a little tired this time, hope there is no regression/drop in image quality with this version.

More at: http://nhwcodec.blogspot.com/

Any feedback welcome!

Cheers,
Raphael

[nhw_pulsar]

Damn, there was a bug in the previous version! Very sorry.

I have updated the new version.

More at: http://nhwcodec.blogspot.com/

Cheers,
Raphael

[Selur]

Any plans on expanding to higher resolution support for next year?
My guess is that most companies&co would want to be sure that a new format is high resolution capable and scales properly with multithreading (especially since newer cpus mainly get more and more cores).

[nhw_pulsar]

Originally Posted by Selur

Any plans on expanding to higher resolution support for next year?
My guess is that most companies&co would want to be sure that a new format is high resolution capable and scales properly with multithreading (especially since newer cpus mainly get more and more cores).

Hello,

Many thanks for your advice and that's right many people urge me now to adapt NHW to any image size because it is very very important.But this is a very big task and unfortunately I won't be able to do that massive effort next year.

Nevertheless, I think it is totally feasible to adapt NHW to any image size/expand to higher resolution with no quality drop.For multithreading, I don't know how it scales as NHW wavelet codec is not block-based but processes on the whole image size, so I don't know what a good multithreading strategy could be for NHW wavelet codec...

Cheers,
Raphael

[benwaggoner]

Originally Posted by nhw_pulsar

Concerning VMAF, is it true that if you improve/increase contrast of the image then you really "bump" the VMAF score? I'll maybe have to inspire from VMAF, but I'll rather make my metrics neatness-sensitive rather than contrast-sensitive.

That was originally true about VMAF, but more recent versions have a mode to turn off the extra contrast bias. Since VMAF is a machine learning model trained on subjective ratings, updated subjective ratings or base metrics or better underlying ML technology will update the model and thus yield different results. So it's much more a "best effort" with continuous improvement than an actual algorithm with reproducible results year-on-year like PSNR and SSIM. And there are different models for different use and that give different scores.

For example, with the same 4K source, the VMAF score will be lowest using the UHD model, in the middle using the HD model, and at the highest using the mobile model. This is a by-design feature, not a bug, as per-pixel distortion is a lot more visible on a big 4K TV than on a phone.

Note that VMAF is a video metric only tested and trained on moving images. Anecdotally it's not bad for still images, but still won't be that accurate. Better than PSNR and SSIM at least.

VMAF isn't that good at comparing adaptive quantization techniques either; --tune ssim can increase VMAF scores while worsening subjective quality.

Big picture, you can certainly play around with VMAF, but you need to verify that its scores match subjective quality for your scenarios. And that you're picking the right model (latest non-contrast sensitive UHD, probably) will be quite important.

[nhw_pulsar]

Originally Posted by benwaggoner

That was originally true about VMAF, but more recent versions have a mode to turn off the extra contrast bias. Since VMAF is a machine learning model trained on subjective ratings, updated subjective ratings or base metrics or better underlying ML technology will update the model and thus yield different results. So it's much more a "best effort" with continuous improvement than an actual algorithm with reproducible results year-on-year like PSNR and SSIM. And there are different models for different use and that give different scores.

For example, with the same 4K source, the VMAF score will be lowest using the UHD model, in the middle using the HD model, and at the highest using the mobile model. This is a by-design feature, not a bug, as per-pixel distortion is a lot more visible on a big 4K TV than on a phone.

Note that VMAF is a video metric only tested and trained on moving images. Anecdotally it's not bad for still images, but still won't be that accurate. Better than PSNR and SSIM at least.

VMAF isn't that good at comparing adaptive quantization techniques either; --tune ssim can increase VMAF scores while worsening subjective quality.

Big picture, you can certainly play around with VMAF, but you need to verify that its scores match subjective quality for your scenarios. And that you're picking the right model (latest non-contrast sensitive UHD, probably) will be quite important.

Thank you for the explanation!

Just a little question, was VMAF trained specifically on DCT block-based distortion/artifacts? Or can it also efficiently evaluate the different distortion of wavelet codecs like NHW?

Cheers,
Raphael

[benwaggoner]

Originally Posted by nhw_pulsar

Just a little question, was VMAF trained specifically on DCT block-based distortion/artifacts? Or can it also efficiently evaluate the different distortion of wavelet codecs like NHW?

My understanding is that the original VMAF was trained only on x264 encodes, mainly with variation in CRF and frame size. I would hope Netflix has incorporated other codecs and a broader set of parameters in more recent models, but I'm not aware of that one way or another.

I'd be surprised if it had been trained on any wavelet codecs. Perhaps a J2K image sequence, but those are always perceptually lossless.

[nhw_pulsar]

Hello,

Yes that's a good way to keep the topic warm, but for those interested, I have released the 0.2.9.2 version of NHW.I continue to fine-tune the pre_processing in the encoder, and so this new version is better.

More at: http://nhwcodec.blogspot.com/

Cheers,
Raphael

[nhw_pulsar]

Originally Posted by benwaggoner

Big picture, you can certainly play around with VMAF, but you need to verify that its scores match subjective quality for your scenarios.

Yes it makes me happy that experts more and more admit that visual review is very important, and that PSNR and SSIM scores alone tend not to be sufficient anylonger... Just a little example, as it is the NHW thread, the recent versions of NHW have a little worse PSNR and NHW_Neatness_metrics scores, but visually I find they are more pleasant...

Concerning contrast with VMAF, yes there are certainly pros and cons, but visually I really prefer an image that has a little more neatness/contrast than an image that has a little less contrast/neatness, and so the idea to give an extra advantage/bias to contrast in VMAF didn't seem to be a bad one for me at first glance... but that's right there maybe were drawbacks on the other hand...

Cheers,
Raphael