NHW Project - a free state-of-the-art image/video compression codec

[nhw_pulsar]

Hi,

Just a quick message as I finally could test VVC intra image compression thanks to Jamaika's bpgenc_jvetvvc and DecoderApp then I use IrfanView to convert the yuv420 file to bmp image.

For now there is a slight discoloration (colorspace slight mismatch between bpg and IrfanView), so hope that it does not distort too much the evaluation... but however it can give a first idea/review of VVC intra.

For now I have just tested on 10 high quality images I have at NHW -l9 high compression setting, and for me (and again only for me) NHW is visually competitive with VVC intra at high compression, as for me NHW performs very well on these images but VVC intra is also very good.

I just tested on 2 images at -l13 compression setting, and I think that VVC intra starts to be very impressive at very high/extreme compression.

For the complexity, JVET VVC (VTM) takes in average 60sec to encode and 55ms-85ms to decode to yuv420, and totally unoptimized NHW takes 30ms to encode and 15ms to decode (VVC VTM has SIMD, and I also think multithreading).

But again globally, VVC intra seems very good notably visually, which is also in a way normal given the time it takes to compute!...

Cheers,
Raphael

[benwaggoner]

Wow, a minute for how big an image?

I don't sweat reference encoder speed much, because there are hugely unoptimized. I'm sure intra-only will be 100x faster in practical implementations. And decode only being 4x more at this point bodes very well.

[nhw_pulsar]

Hello,

It is actually a one minute encode for a 512x512 image.

The VVC decoder is 4x slower than totally unoptimized NHW decoder, but with the same level of optimization, as my computer has 4 threads and AVX2 SIMD, then NHW decoder could be nearly 4x5=20x faster and so 80x faster than VVC decoder...

[Gravitator]

Здравствуй!
Make a demo like this for an example > https://eclipseo.github.io/image-comparison-web/

[nhw_pulsar]

Hello!

Yes I should make an image comparison like this one.I can not use exactly the same example you pointed out, but I've seen few online websites that offer similar image comparison viewers (that will be easier for me...).

Also I will not be able to use these example images as NHW doesn't encode for now such image sizes, but also the problem is that my high quality images on which NHW works well, have copyright and so I don't think I am allowed to publish them on the Internet...

But I will try to find a solution with an online image comparison provider website, as you're right it is very needed, but please be patient as I am very lazy currently...

Cheers,
Raphael

[nhw_pulsar]

Originally Posted by nhw_pulsar

Hello,

It is actually a one minute encode for a 512x512 image.

The VVC decoder is 4x slower than totally unoptimized NHW decoder, but with the same level of optimization, as my computer has 4 threads and AVX2 SIMD, then NHW decoder could be nearly 4x5=20x faster and so 80x faster than VVC decoder...

Hi benwaggoner,

Sorry for my aggressive unrealistic decoder speed number.To be exact, when I tested in 2008-2009, x264 image compression (which was called UCI codec), I measured quickly that NHW was 1.2x faster to decode than x264, but my computer at that time had 2 threads and SSE2 if I remember correctly, so I estimated that NHW was 1.2x2x2.5= 6x faster than x264/H.264 to decode.From that number, I estimated that NHW was around 15x faster to decode than HEVC, and now all the benchmarks converge to say that VVC is 1.8x slower to decode than HEVC, so normally NHW is 15x1.8= 27x faster to decode than VVC.

So again I apologize for my little aggressiveness, I know that VVC is a well-respected professional codec, and that NHW beside is a total amateur project that won't lead to an industry project.But it's still great to have places like here, where we can talk all kinds of codecs.

Cheers,
Raphael

[benwaggoner]

HEVC and VVC are also better-suited for parallel processing and SIMD in decoding single frames than older codecs. With Wavefront Parallel Processing, you can get one thread per 64 lines of the frame.

AVIF also benefits from SIMD quite a lot, and can get parallelism from tiles. It's much improved from VP8/9 which were essentially serialized for single-frame decode.

[nhw_pulsar]

Ok, thanks for the info.

That's right that I don't know how much NHW is parallelizable and optimisable with SIMD, but really the NHW decoder algorithm has been thought to be extremely fast.And so it could be added to it an efficient post-processing stage that would remove aliasing for example...

[nhw_pulsar]

Hello,

No new version for now, I however generated a lot of different versions, some with more neatness, few others with more precision, but I only evaluate visually and my eyes are maybe tired currently, so no change for now, I stick with this version for now.

Else some people told me and I also realize that I made at least very serious errors of communication, notably when I said that AOM and MPEG codecs lacked of neatness and that NHW has more neatness which is better, these unreferenced accusations and hostile communication have just make these organizations not inclined to help me, like notably to let me a niche.

Today, I apologize for my bad and unfounded communication.Just hope it's still possible today for NHW to find a niche market where it can develop.So far the niche that was suggested to me was a MotionJPEG replacement because NHW is faster and better than MJPEG, but also the major obstacle here is that MJPEG is certainly used for its very low complexity but mostly for the widespread and everywhere support of JPEG.

Still hope that some people here could point me some ways for NHW to develop as a niche (would be great!), on the sidelines of AOM and MPEG if I could find a little place.

Any help very welcome!

Cheers,
Raphael

[nhw_pulsar]

Hello,

Just a quick message because when I stated in my previous post that I have made some versions with more neatness, somebody asked me if it wasn't the aim of my codec to have more neatness (and again it is my only personal criterium, this is absolutely not a reference one)? But actually these versions start also to be over-sharpened, you can maybe like it, but we clearly see the sharpening of the image, which I would prefer avoid for now and stay a little more neutral...

But this is not easy, because I tweaked the pre_processing and the quantization scheme and for now I don't really see significative visual improvements... Maybe I should start to look at a post-processing enhancement stage that will remove aliasing for example and improve image quality... Because that's right that the latest MPEG codec: VVC has at least 3 post-processing stages: deblocking filter, SAO filter, and ALF filter.If you would have some background in aliasing analysis/correction, do not hesitate to get in contact with me!

Cheers,
Raphael

[benwaggoner]

Originally Posted by nhw_pulsar

Just a quick message because when I stated in my previous post that I have made some versions with more neatness, somebody asked me if it wasn't the aim of my codec to have more neatness (and again it is my only personal criterium, this is absolutely not a reference one)? But actually these versions start also to be over-sharpened, you can maybe like it, but we clearly see the sharpening of the image, which I would prefer avoid for now and stay a little more neutral...

But this is not easy, because I tweaked the pre-processing and the quantization scheme and for now I don't really see significative visual improvements... Maybe I should start to look at a post-processing enhancement stage that will remove aliasing for example and improve image quality... Because that's right that the latest MPEG codec: VVC has at least 3 post-processing stages: deblocking filter, SAO filter, and ALF filter.If you would have some background in aliasing analysis/correction, do not hesitate to get in contact with me!

Preprocessing in encoding is a philosophical question as much as any. I tend to stay away from it unless the preprocessing actually improves net accuracy of the final encode.

There was a kerfuffle with VMAF when some encoder developers realize that contrast enhancements prior to encoding would increase VMAF scores even though they reduced accuracy. Some people thought that was a fair optimization, others that it was cheating. They had to add a "no preprocessing" VMAF mode so people could generate scores both ways.

There are always things one can do to improve the subjective quality of content when the viewer doesn't have anything to compare it to. But those will also make it look less like what the creators intended it to look like.

If you do have a sharpening prefilter, I recommend that you make it optional and with a configurable strength. And check edge cases like line art and sharp red text in a serif font, where edge enhancements that would work on continuous tone images might come out much weirder.

Whenever anyone says they have a fancy new encoder, first thing I test is cel animation content and motion graphics, and more often than not the encoder's cool optimizations fall apart with that kind of content.

[nhw_pulsar]

Yes there is a sharpening prefilter in NHW (from -l4 quality setting).But actually this sharpening prefilter is very important not to say essential, because without it, my codec/algorithm really lacks of neatness and is very blurred (blurred edges), and results are bad.Where it complicates is that this sharpening prefilter sharpens a little the original image which can be avoided but in the same time it gives a good neatness to NHW which is essential.For now I try to have a sharpening prefilter setting that does not sharpen too much (as low as possbile) but that gives the maximum neatness, because it is really very important for NHW.-I also agree that the concepts of neatness, sharpness in NHW are not very clear for now...-

Yes there are special cases that you quoted where NHW doesn't perform well, but this should be expected from an extremely faster codec?

[benwaggoner]

Originally Posted by nhw_pulsar

Yes there is a sharpening prefilter in NHW (from -l4 quality setting).But actually this sharpening prefilter is very important not to say essential, because without it, my codec/algorithm really lacks of neatness and is very blurred (blurred edges), and results are bad.Where it complicates is that this sharpening prefilter sharpens a little the original image which can be avoided but in the same time it gives a good neatness to NHW which is essential.For now I try to have a sharpening prefilter setting that does not sharpen too much (as low as possbile) but that gives the maximum neatness, because it is really very important for NHW.-I also agree that the concepts of neatness, sharpness in NHW are not very clear for now...-

Yes there are special cases that you quoted where NHW doesn't perform well, but this should be expected from an extremely faster codec?

Any codec, video, audio, or still, needs to gracefully degrade when out of its sweet spot. The sheer variety of content out there is mind-boggling, and all the common image test libraries lack sufficient representation of important content categories.

The original sin of JPEG is that it was only intended for continuous tone images like photographs, and a pure DCT implementation runs into severe issues with discreet tone images like text, line art, traditional GIF animations, and anything else where sharp edges are the most psychovisually important element.

A great thing about HEIC as an image format is that a HEVC IDR can incorporate the best of JPEG and PNG style compression, using iDCT-like transforms for natural image areas, but switch to transform skip or even lossless CUs for discreet tone areas. That makes for much more graceful degradation for weird kinds of content, and finally made it possible to compress mixed continuous/discreet tone content efficiently and in high quality.

The quality of a codec is reflected in both in where it operates best and where it operates worse.

[nhw_pulsar]

Ok, but handling all the image categories is a huge work for me, and now with my current "structure" I absolutely don't have that time... so for now I will continue to improve NHW for photographs images, and with tiny chance if a company wants to back NHW then more image types handling could happen...

By the way, I wanted to ask you do you think Alliance for Open Media could change their mind about NHW (yes I can dream)? Because I offered them my technology and maybe they would decide to explore it and see what it's worth, but they were never interested, but last time I was in contact with AOM was 1 year and a half ago.As NHW has seriously improved during this period, do you think they could reconsider it, or if they don't answer me anymore it's because they are definitely not interested and I should stop disturbing them now?

Cheers,
Raphael

[benwaggoner]

Originally Posted by nhw_pulsar

Ok, but handling all the image categories is a huge work for me, and now with my current "structure" I absolutely don't have that time... so for now I will continue to improve NHW for photographs images, and with tiny chance if a company wants to back NHW then more image types handling could happen...

By the way, I wanted to ask you do you think Alliance for Open Media could change their mind about NHW (yes I can dream)? Because I offered them my technology and maybe they would decide to explore it and see what it's worth, but they were never interested, but last time I was in contact with AOM was 1 year and a half ago.As NHW has seriously improved during this period, do you think they could reconsider it, or if they don't answer me anymore it's because they are definitely not interested and I should stop disturbing them now?

AV2 is in early development, and this would be a good time to make contributions as part of that standards process.

[nhw_pulsar]

Originally Posted by benwaggoner

AV2 is in early development, and this would be a good time to make contributions as part of that standards process.

Thank you for your answer.Actually the last answer I had from an AOM responsible was about AV2 because I wanted to submit, but I was told that the goal of AV2 will be to have notably better PSNR and SSIM over AV1 (with an assumed complexity increase) which was completely incompatible with NHW's poor PSNR and SSIM results.

So NHW is not of interest for AV2.I was just wondering (and again I can dream...) if AOM wants to do something with the free NHW technology like a side lightweight project/standard with extremely fast computing and good visual quality (better than HEVC, at least for still image)?

[benwaggoner]

Originally Posted by nhw_pulsar

Thank you for your answer.Actually the last answer I had from an AOM responsible was about AV2 because I wanted to submit, but I was told that the goal of AV2 will be to have notably better PSNR and SSIM over AV1 (with an assumed complexity increase) which was completely incompatible with NHW's poor PSNR and SSIM results.

So NHW is not of interest for AV2.I was just wondering (and again I can dream...) if AOM wants to do something with the free NHW technology like a side lightweight project/standard with extremely fast computing and good visual quality (better than HEVC, at least for still image)?

AOM definitely does not prioritize "extremely fast" . And the whole VPx and libaom/libvpx architecture was very PSNR focused back to the On2 days. I think they would struggle to integrate any tools that had poor psnr.

[nhw_pulsar]

Yes, PSNR and SSIM completely govern compression bodies.It's hard but I must accept it, but NHW has however proved that you can have a better visual quality with poor PSNR and SSIM, at least according to my eyes...

[benwaggoner]

Originally Posted by nhw_pulsar

Yes, PSNR and SSIM completely govern compression bodies.It's hard but I must accept it, but NHW has however proved that you can have a better visual quality with poor PSNR and SSIM, at least according to my eyes...

Alas, subjective evaluation comes after PSNR evaluation. VMAF seems to be used more and more, but has its own limitations (like not being able to accurately score adaptive quantization).

The classic MPEG codec development process is highly biased for codecs that deliver high PSNR and MOS-rated subjective quality over 10 second clips with a fixed QP and no adaptive quantization.

[nhw_pulsar]

Originally Posted by benwaggoner

Alas, subjective evaluation comes after PSNR evaluation. VMAF seems to be used more and more, but has its own limitations (like not being able to accurately score adaptive quantization).

The classic MPEG codec development process is highly biased for codecs that deliver high PSNR and MOS-rated subjective quality over 10 second clips with a fixed QP and no adaptive quantization.

I can only speak for still images, but I do not listen to PSNR and SSIM because with my experience they didn't suggest the best choices (regarding visual quality).Ok it will sound unprofessional but I only evaluate with my eyes for NHW, because this is what gives me currently the best (visual) results...

But I also agree that NHW is a fundamentally different technology from the over-dominant DCT block-based intra prediction+residual coding, for me NHW relies on neatness and AOM/MPEG relies on precision, and so then maybe their evaluation process are different...