AV1 picture too smooth compared with VP9

[gonzooka]

Hi there,

i am trying to encode an "old movie" with 1080P from MP4/AVC-18GB to save some space - and just for trying.

With VP9 it takes in second pass 24 hours with 4 threads, good result!

I would like to use AV1, it uses more threads
(but not all 16, Intel 12600K, hmm)
and does it in a half about.

What me annoys: AV1 is smoothing way too much.
Example: on a face I cant see the wrinkles in the eyes
anymore and beauty spots vanish a bit but obvious.

I tried AV1 with lower preset but no visible effect at all, CRF also.

How can I tune it, not to smooth? Any experiences?


Here the command line for both, AV1 and VP9:
(ffmpeg und AV1encoder are latest stable build.)

ffmpeg -hide_banner -loglevel warning -i input.mp4 -nostdin -strict -1 -pix_fmt yuv420p10le -f yuv4mpegpipe - | SvtAv1EncApp --progress 1 -i stdin --lp 0 --tile-rows 3 --tile-columns 3 --preset 7 -q 20 --mbr 2000k --irefresh-type 2 --enable-restoration 0 --enable-dlf 0 --enable-cdef 0 --keyint 240 -b av1_libsvtav1_output.mkv

ffmpeg -i input.mkv -c:v libvpx-vp9 -b:v 2M -crf 30 -pass 2 -threads 16 -row-mt 1 -c:a copy output.webm

[gonzooka]

footnote:

Encoding the second movie:

with tile rows switch VP9 now it uses all 16 threads with 50%, thats great! But still a lot of time: 30 hours...

ffmpeg -i input.mkv -c:v libvpx-vp9 -b:v 2M -crf 30 -pass 2 
-threads 16 -row-mt 1 -tile-columns 4 -tile-rows 2 -an output.webm

[benwaggoner]

It's a feature of modern codecs that they increasingly degrade into loss of detail instead of creating visible artifacts. I wouldn't expect AV1 to look as good as VP9 at double the bitrate. If you want to do a comparison, start with the same bitrate and see if AV1 has more detail. Then you can reduce the bitrate until you start getting undesired detail loss.

[birdie]

AV1 looks like crap, sorry.

Last time I used it with these libaom 3.5 final release options and it smears out the details like there's no tomorrow:

./aomenc --end-usage=q --cq-level=12 --cpu-used=4 --threads=16 --bit-depth=10 --lag-in-frames=48
    --enable-fwd-kf=1 --enable-qm=1 --enable-chroma-deltaq=1 --quant-b-adapt=1 --mv-cost-upd-freq=2 
    --sharpness=3 --enable-keyframe-filtering=2 --arnr-strength=1 --disable-trellis-quant=0 -o out.webm out.y4m

cq-level=12 !!! That's almost lossless.

I'm disappointed as hell. Now trying vvenc-1.6.1 which is slow as hell but on a first attempt it looks much better:

./vvencapp --preset slower -qp 20 -i out.y4m --output=bit.266

Edit: at qp=20 the bitrate matches the bitrate of the original H.264 video. LOL.

Modern codecs are something. Weird.

Truth to be told I don't know how to use x264 properly. My source is encoded at 40Mbit/sec. I can only get comparable quality if I encode it at ... 35MBit/sec despite the source being produced by a terrible HW encoder and the result produced by the latest revision of x264 with the following flags:

ffmpeg -i source.mp4 -c:a copy -c:v libx264 -preset veryslow -x264opts keyint=180:min-keyint=30:bframes=16 -crf 17 -tune grain result.mkv

[Beelzebubu]

The smoothing you're seeing is from temporal filtering. Try disabling it using --arnr-strength=0 --arnr-maxframes=0 --enable-keyframe-filtering=0.

I'm not sure why you're using custom settings like --enable-qm=1 --enable-chroma-deltaq=1 --quant-b-adapt=1 --mv-cost-upd-freq=2 --sharpness=3, you should use codec defaults for these unless you know what you're doing.

[damian101]

Use tune=ssim.
Arnr-strength 0 is bad, try arnr-strength=2.
Disabling cdef with enable-cdef=0 can also reduce blur quite a bit.

[benwaggoner]

And all indications that these are encoder issues, not bitstream issues. libaom is fundamentally derived from On2's "PSNR above all" approach to the VPx codec series, which absolutely yields the smearing you describe. But other AV1 encoders don't have nearly the same issues.

[writersblock29]

@Gonzooka

Echoing Ben's comment, have you tried SVT AV1? There's a 10-bit version available in Handbrake if you're okay with using a GUI. I've played with it a bit myself, and it seems to saturate a multicore processor pretty well. Plus you can encode a "preview" with your chosen settings so that you can see the results and possibly save yourself a slew of hours waiting for disappointment. Handbrake's Super HQ profile choses an RF of 20, which I feel is tremendous overkill--but you're not me, so that might give you a good starting point.