Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > New and alternative video codecs

Reply
 
Thread Tools Search this Thread Display Modes
Old 2nd July 2020, 18:29   #1  |  Link
IgorC
Registered User
 
Join Date: Apr 2004
Posts: 1,319
HIGIC: High-Fidelity Generative Image Compression (Google Research & ETH Zürich)

https://hific.github.io/

We extensively study how to combine Generative Adversarial Networks and learned compression to obtain a state-of-the-art generative lossy compression system. In particular, we investigate normalization layers, generator and discriminator architectures, training strategies, as well as perceptual losses. In contrast to previous work, i) we obtain visually pleasing reconstructions that are perceptually similar to the input, ii) we operate in a broad range of bitrates, and iii) our approach can be applied to high-resolution images. We bridge the gap between rate-distortion-perception theory and practice by evaluating our approach both quantitatively with various perceptual metrics and a user study. The study shows that our method is preferred to previous approaches even if they use more than 2x the bitrate.
IgorC is offline   Reply With Quote
Old 2nd July 2020, 19:01   #2  |  Link
AntiqueRefrigerator
Registered User
 
Join Date: Apr 2018
Posts: 4
Okay, now that's cool.
AntiqueRefrigerator is offline   Reply With Quote
Old 3rd July 2020, 01:07   #3  |  Link
Asmodian
Registered User
 
Join Date: Feb 2002
Location: San Jose, California
Posts: 4,202
This looks very interesting! I will keep an eye out for the trained model and TensorFlow code.

Make sure to use the full resolution version of the comparison tool, in the scaled down version the two images are mismatched for me.
__________________
madVR options explained
Asmodian is offline   Reply With Quote
Old 3rd July 2020, 08:04   #4  |  Link
Greenhorn
Registered User
 
Join Date: Apr 2018
Posts: 51
This is impressive! If you gave me a blind comparison between the high-BPP HIGIC images and their originals, I think I'd have a hard time telling which came first.

I feel like it'd be far too easy for me to tell that one of them must have been altered in some way, though-- look at the knitting pattern on the girl's red sweater, or the tiles on top of the clock tower. Both have highly irregular patterns in the originals, but much more regular, fine-grained patterns in the HIGIC-generated images. The guy riding the subway also looks like he has goosebumps in the HIGIG image but not in his original.

Admittedly, the paper calls this out as a weakness of their model on a couple of occasions and basically makes "don't use this for sensitive information" its conclusion, so it's not like the people who worked on this are trying to sell it as a miracle cure.

I like how they firmly established that high perceptual quality and low distortion can be opposed to each other, too, and even basically made a metric for it.

Definitely gives me hope for future generations of AI-powered codecs that don't need to redraw as much.
Greenhorn is offline   Reply With Quote
Old 3rd July 2020, 08:33   #5  |  Link
soresu
Registered User
 
Join Date: May 2005
Location: Swansea, Wales, UK
Posts: 188
If not AV2, then certainly AV3 will be heavily influenced by AI/ML compression techniques.

As it is the commits I have seen to the experimental branch of libaom seem to favor NN technqiues, considering the focus of certain CPU core designers like ARM and Intel to improve ML code execution efficiency, one can hope that the codec of the future will be less of a complexity beast with modern CPU's upon its initial release, this I think will be the main benefit of ML being introduced to codecs, even beyond any possible BD rate improvements.
soresu is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 17:40.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.