View Single Post
Old 25th February 2021, 00:38   #4  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
The key thing in all modern codecs is that they convert from the spatial domain (pixels) to the frequency domain (how pixel values change over a region). For an IDR frame, slight and regular changes between adjoining pixels take fewer bits to encode well, and large and irregular changes require more bits to encode well, because it takes a lot more frequencies to add up to the final result. It's the same thing temporally; the more a frame is like the frames it references, the fewer bits it takes. With motion estimation, if there's a very good match between an area of a frame and a frame it references, that can be coded in very few bits. If there's not a good match, a lot more bits are needed to code the difference between the best match and where in the new frame it is referenced from.

Psychovisual factors come in play as well. Noisy areas, like foliage, can withstand more aggressive compression because the artifacts aren't that different than the original texture. But a little blocking in a smooth gradient can be painfully obvious.

Sharp grain/noise is the hardest, because it is high frequencies randomly distributed, so hard to encode for an IDR frame, and overlays everything with that, so prediction is a lot less accurate.

The Film Grain Synthesis approach is compelling, as it just does a grain removal pass during encoding, parameterizes the grain that was removed, and then reconstructs it on playback. All going well, you get the same grain texture, but at a lower bitrate as the encoding itself is grain-free. AV1 is the first reasonably mainstream codec to have mandatory support of FGS in all decoders.

Of course, grain removal, grain parameterization, and grain synthesis have been hard problems for a few decades, and aren't solved by any means. FGS implementations are getting incrementally better, but aren't at the "just check that box and it'll come out fine" stage.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote