Benwaggoner HEVC encoding challenge

[Joekiwi]

How well does NVENC do .?


Deleted as used wrong source - was quite surprised how high my y ssim score was - I might try again some other time

these were my settings anyway

NVEncC64.exe --vbrhq 1000 --codec h265 --preset quality --profile main10 --tier high --level 4.1 --output-depth 10 --max-bitrate 4000 --vbv-bufsize 12000 --aq-strength 1 --aq --aq-temporal --bref-mode each --bframes 4 --ref 5 --gop-len 120 --lookahead 32 --slices 1 --multiref-l0 4 --multiref-l1 2 --nonrefp --weightp --ssim --mv-precision q-pel --cabac -i

[Joekiwi]

Ok maybe disregard my last post - I'm using the wrong source - so probably it's probably a much easier to encode .
I just watched Bens' placebo encode and there was much more detail .
I just watched my source for the encode and it also lacked the detail.

I did download the 14 gb file - But when I unzipped it I got nothing - wasn't sure why - so downloaded another file mentioned later on that I think was only 579mbs ( from memory-I'm on a different PC ) .

So NVENC will be worst than my scores - still I can untick copy audio and get another 90 odd kbps to use . Ben did I need all those 4 files in your link to get the Source?

[Sagittaire]

I did download the 14 gb file - But when I unzipped it I got nothing - wasn't sure why - so downloaded another file mentioned later on that I think was only 579mbs ( from memory-I'm on a different PC ) .


Work for me. Perhpabs downloading problem?

Be carefull, Placebo Benwagonner encoding is really old encoding. And I suspect Rate Control problem (really bad quality in start scene). Ma's encoding produce really better overall quality. Bearm 4.5 encoding is really good too (be carefull it's zone encoding like real compressionnist encoding).


I'm using the wrong source - so probably it's probably a much easier to encode

Yes, Benwagonner use Dithering for make YV12 lossless source

[Sagittaire]

Well after many and many tests to find best x265 setting for this ToS source, here my result for only 1000 kbps at this moment. I will use he-aac 5.1 audio at 128 kbps for 1000 kbps, he-aac 5.1 audio at 160 kbps for 1500 kbps and lc-aac 5.1 audio at 192 kbps for 2000 kbps encoding:

WARNING: google drive make AVC transcoding for streaming. if you want to see real HEVC encoding, download the file.


1) crf 1 passe mode:
highest possible metric score in all mode
Preset "veryslow" for 1000 kbps and "optimized slower" for 1500 and 2000 kbps

ToS-1000-crf.mkv (https://drive.google.com/open?id=116WhVYBKrm8bI_qL64HxvKLk6pzrE9bm)
ToS-1500-crf.mkv (https://drive.google.com/open?id=126cXZy5sqtZnoTDJEAbYC-74gr7eJbjl)
ToS-2000-crf.mkv (https://drive.google.com/open?id=1ul_wTfnWdNwESExWCFIWvi4RR2kRzhmn)

D:\Mes Logiciels\Codec\x265>x265.exe --input C:\ToS_1920x800_xdither.y4m --output ToS-1000-cfr.265 --input-res 1920x800 --output-depth 10 --fps 24000/1000 --crf 23.1 --pass 1 --slow-firstpass --stats ToS-1000-crf.log --qcomp 0.50 --preset veryslow --tune ssim --no-limit-modes --rd-refine --hevc-aq --qp-adaptation-range 1.0 --bframes 5 --b-adapt 2 --ref 5 --min-keyint 1 --keyint 120 --rc-lookahead 60 --vbv-maxrate 4000 --vbv-bufsize 12000 --psnr --ssim --tskip --qg-size 64 --merange 64 --analysis-save ToS_1000_R10-crf1_analysis.dat --analysis-save-reuse-level 10 --deblock -1,-1 --limit-sao
x265 [info]: frame I: 221, Avg QP:27.27 kb/s: 11216.30 PSNR Mean: Y:43.857 U:46.103 V:46.112 SSIM Mean: 0.973372 (15.747dB)
x265 [info]: frame P: 3910, Avg QP:29.09 kb/s: 2844.64 PSNR Mean: Y:43.033 U:46.037 V:45.997 SSIM Mean: 0.971443 (15.443dB)
x265 [info]: frame B: 13489, Avg QP:36.49 kb/s: 296.98 PSNR Mean: Y:41.376 U:45.256 V:45.240 SSIM Mean: 0.967579 (14.892dB)
x265 [info]: Weighted P-Frames: Y:6.0% UV:4.8%
x265 [info]: Weighted B-Frames: Y:4.1% UV:2.8%
x265 [info]: consecutive B-frames: 10.8% 7.6% 5.9% 24.9% 21.6% 29.2%

encoded 17620 frames in 30340.70s (0.58 fps), 999.28 kb/s, Avg QP:34.73, Global PSNR: 42.688, SSIM Mean Y: 0.9685090 (15.018 dB)

D:\Mes Logiciels\Codec\x265>x265.exe --input C:\ToS_1920x800_xdither.y4m --output ToS-1500-crf.265 --input-res 1920x800 --output-depth 10 --fps 24000/1000 --crf 20.0 --preset slower --qcomp 0.50 --bframes 5 --b-adapt 2 --min-keyint 1 --keyint 120 --rc-lookahead 60 --vbv-maxrate 4000 --vbv-bufsize 12000 --psnr --ssim --tune ssim --ref 5 --limit-refs 3 --rd-refine --hevc-aq --qp-adaptation-range 1.0 --qg-size 64 --hme --hme-search 0,1,3 --hme-range 24,48,64 --deblock -1,-1 --limit-sao --csv ToS-1500-crf.csv --csv-log-level 1

encoded 17620 frames in 16079.49s (1.10 fps), 1504.59 kb/s, Avg QP:31.57, Global PSNR: 43.905, SSIM Mean Y: 0.9743358 (15.907 dB)

encoded 17620 frames in 18895.70s (0.93 fps), 2002.67 kb/s, Avg QP:29.41, Global PSNR: 44.772, SSIM Mean Y: 0.9775448 (16.487 dB)


2) ABR with 3 passes:
highest possible metric in NPass mode
Preset "veryslow" for 1000 kbps and "optimized slower" for 1500 and 2000 kbps

ToS-1000-abr.mkv (https://drive.google.com/open?id=1R1ulx9icVkKZqhT4CimknFdHLQo3WCZp)
ToS-1500-abr.mkv (https://drive.google.com/open?id=1JEGWR6F8nrnH-QBdQg7DluvSt6OQ9sZj)
ToS-2000-abr.mkv (https://drive.google.com/open?id=14eWSJpgFfbmTj2kkGV1ayFkdMmI-Gf-M)

D:\Mes Logiciels\Codec\x265>x265.exe --input C:\ToS_1920x800_xdither.y4m --output ToS-1000-abr.265 --input-res 1920x800 --output-depth 10 --fps 24000/1000 --bitrate 1000 --pass 2 --stats ToS-1000.log --qcomp 0.50 --preset veryslow --tune ssim --no-limit-modes --rd-refine --hevc-aq --qp-adaptation-range 1.0 --bframes 5 --b-adapt 2 --ref 5 --min-keyint 1 --keyint 120 --rc-lookahead 60 --vbv-maxrate 4000 --vbv-bufsize 12000 --psnr --ssim --tskip --qg-size 64 --merange 64 --analysis-save ToS_1000_R10-abr3_analysis.dat --analysis-save-reuse-level 10 --deblock -1,-1 --limit-sao
x265 [info]: frame I: 221, Avg QP:26.58 kb/s: 10729.30 PSNR Mean: Y:43.864 U:46.118 V:46.127 SSIM Mean: 0.972412 (15.593dB)
x265 [info]: frame P: 3910, Avg QP:29.26 kb/s: 2266.71 PSNR Mean: Y:42.187 U:45.456 V:45.336 SSIM Mean: 0.966785 (14.787dB)
x265 [info]: frame B: 13489, Avg QP:31.23 kb/s: 469.21 PSNR Mean: Y:41.386 U:44.917 V:44.851 SSIM Mean: 0.965572 (14.631dB)
x265 [info]: Weighted P-Frames: Y:6.0% UV:4.7%
x265 [info]: Weighted B-Frames: Y:4.1% UV:2.8%
x265 [info]: consecutive B-frames: 10.8% 7.6% 5.9% 24.9% 21.6% 29.2%

encoded 17620 frames in 34506.66s (0.51 fps), 996.78 kb/s, Avg QP:30.73, Global PSNR: 42.449, SSIM Mean Y: 0.9659273 (14.676 dB)

D:\Mes Logiciels\Codec\x265>x265.exe --input C:\ToS_1920x800_xdither.y4m --output ToS-1500-abr.265 --input-res 1920x800 --output-depth 10 --fps 24000/1000 --bitrate 1500 --pass 3 --stats ToS-1500-abr.log --preset slower --qcomp 0.50 --bframes 5 --b-adapt 2 --min-keyint 1 --keyint 120 --rc-lookahead 60 --vbv-maxrate 4000 --vbv-bufsize 12000 --psnr --ssim --tune ssim --ref 5 --limit-refs 3 --rd-refine --hevc-aq --qp-adaptation-range 1.0 --qg-size 64 --hme --hme-search 0,1,3 --hme-range 24,48,64 --deblock -1,-1 --limit-sao --csv ToS-1500-abr.csv --csv-log-level 1

encoded 17620 frames in 16556.89s (1.06 fps), 1495.10 kb/s, Avg QP:27.88, Global PSNR: 43.626, SSIM Mean Y: 0.9724194 (15.594 dB)

encoded 17620 frames in 19555.37s (0.90 fps), 2000.35 kb/s, Avg QP:26.00, Global PSNR: 44.483, SSIM Mean Y: 0.9761716 (16.229 dB)


3) ABR with 3 passes with default setting:
default setting for rate control, adaptative quantisation and psy-rdo setting at same level research for me, cu, tu ... than crf (1) and abr (2) encoding.
Preset "veryslow" for 1000 kbps and "optimized slower" for 1500 and 2000 kbps.

ToS-1000-psy.mkv (https://drive.google.com/open?id=1ZMvYaqyevoRHuSwqCjMwGqvkhBKKVJ0S)
ToS-1500-psy.mkv (https://drive.google.com/open?id=1tLr6RiSfKgov-tHxmEBEIW-Sg11B3_fr)
ToS-2000-psy.mkv (https://drive.google.com/open?id=15A591qQ0JgS4-7mjH3wItbmBrcHkbAgL)

D:\Mes Logiciels\Codec\x265>x265.exe --input C:\ToS_1920x800_xdither.y4m --output ToS-1000-psy.265 --input-res 1920x800 --output-depth 10 --fps 24000/1000 --bitrate 1000 --pass 2 --stats ToS-1000-psy.log --preset veryslow --bframes 5 --b-adapt 2 --min-keyint 1 --keyint 120 --rc-lookahead 60 --vbv-maxrate 4000 --vbv-bufsize 12000 --psnr --ssim --no-limit-modes --ref 5 --rd-refine --tskip --qg-size 64 --merange 64 --analysis-save ToS_1000_R10-psy1_analysis.dat --analysis-save-reuse-level 10 --csv ToS-1000-psy.csv --csv-log-level 1
x265 [info]: frame I: 221, Avg QP:28.02 kb/s: 10100.18 PSNR Mean: Y:43.571 U:45.588 V:45.560 SSIM Mean: 0.974445 (15.925dB)
x265 [info]: frame P: 3910, Avg QP:31.83 kb/s: 2542.81 PSNR Mean: Y:41.707 U:44.944 V:44.790 SSIM Mean: 0.968909 (15.074dB)
x265 [info]: frame B: 13489, Avg QP:37.81 kb/s: 403.79 PSNR Mean: Y:40.508 U:44.297 V:44.197 SSIM Mean: 0.965387 (14.608dB)
x265 [info]: Weighted P-Frames: Y:5.8% UV:4.7%
x265 [info]: Weighted B-Frames: Y:3.8% UV:2.6%
x265 [info]: consecutive B-frames: 10.8% 7.6% 5.9% 24.9% 21.6% 29.2%

encoded 17620 frames in 32709.81s (0.54 fps), 1000.07 kb/s, Avg QP:36.36, Global PSNR: 41.710, SSIM Mean Y: 0.9662820 (14.721 dB)

D:\Mes Logiciels\Codec\x265>x265.exe --input C:\ToS_1920x800_xdither.y4m --output ToS-1500-psy.265 --input-res 1920x800 --output-depth 10 --fps 24000/1000 --bitrate 1500 --pass 3 --stats ToS-1500-psy.log --preset slower --bframes 5 --b-adapt 2 --min-keyint 1 --keyint 120 --rc-lookahead 60 --vbv-maxrate 4000 --vbv-bufsize 12000 --psnr --ssim --ref 5 --limit-refs 3 --rd-refine --qg-size 64 --hme --hme-search 0,1,3 --hme-range 24,48,64 --csv ToS-1500-psy.csv --csv-log-level 1

encoded 17620 frames in 17323.16s (1.02 fps), 1497.51 kb/s, Avg QP:33.00, Global PSNR: 43.081, SSIM Mean Y: 0.9728444 (15.661 dB)

encoded 17620 frames in 20365.97s (0.87 fps), 1997.96 kb/s, Avg QP:30.48, Global PSNR: 44.045, SSIM Mean Y: 0.9763138 (16.255 dB)


4) Reuse ABR with 3 passes :
transcoding from ABR 3 passes encoding veryslow at 1000 kbps (2) with analysis reuse statistique at really high speed (by factor ~100x)

ToS-1000-hs.mkv (https://drive.google.com/open?id=1Ei4etiAtB1crywdmppKgWpsRKi-bXeQ-)
ToS-1500-hs.mkv (https://drive.google.com/open?id=15zesNqvK079X_46SGzogd9leVAW9mLfh)
ToS-2000-hs.mkv (https://drive.google.com/open?id=1JZ73pzqVZXXQ6gOI_0zeMuJ4iOBTiHmm)

D:\Mes Logiciels\Codec\x265>x265.exe --input C:\ToS_1920x800_xdither.y4m --output ToS-1000-hs.265 --input-res 1920x800 --output-depth 10 --fps 24000/1000 --bitrate 1000 --pass 3 --stats ToS-1000.log --qcomp 0.50 --preset veryslow --tune ssim --no-limit-modes --rd-refine --hevc-aq --qp-adaptation-range 1.0 --bframes 5 --b-adapt 2 --ref 5 --min-keyint 1 --keyint 120 --rc-lookahead 60 --vbv-maxrate 4000 --vbv-bufsize 12000 --psnr --ssim --tskip --qg-size 64 --merange 64 --analysis-load ToS_1000_R10-abr3_analysis.dat --analysis-load-reuse-level 10 --deblock -1,-1 --limit-sao

x265 [info]: frame I: 221, Avg QP:26.75 kb/s: 10671.81 PSNR Mean: Y:43.556 U:46.032 V:46.032 SSIM Mean: 0.971743 (15.489dB)
x265 [info]: frame P: 3910, Avg QP:29.48 kb/s: 2284.96 PSNR Mean: Y:41.984 U:45.347 V:45.232 SSIM Mean: 0.965561 (14.629dB)
x265 [info]: frame B: 13489, Avg QP:31.44 kb/s: 469.18 PSNR Mean: Y:41.190 U:44.813 V:44.752 SSIM Mean: 0.964374 (14.482dB)
x265 [info]: Weighted P-Frames: Y:6.0% UV:4.7%
x265 [info]: Weighted B-Frames: Y:4.1% UV:2.8%
x265 [info]: consecutive B-frames: 10.8% 7.6% 5.9% 24.9% 21.6% 29.2%

encoded 17620 frames in 431.14s (40.87 fps), 1000.08 kb/s, Avg QP:30.95, Global PSNR: 42.275, SSIM Mean Y: 0.9647300 (14.526 dB)

D:\Mes Logiciels\Codec\x265>x265.exe --input C:\ToS_1920x800_xdither.y4m --output ToS-1500-hs.265 --input-res 1920x800 --output-depth 10 --fps 24000/1000 --bitrate 1500 --pass 3 --stats ToS-1000.log --qcomp 0.50 --preset veryslow --tune ssim --no-limit-modes --rd-refine --hevc-aq --qp-adaptation-range 1.0 --bframes 5 --b-adapt 2 --ref 5 --min-keyint 1 --keyint 120 --rc-lookahead 60 --vbv-maxrate 4000 --vbv-bufsize 12000 --psnr --ssim --tskip --qg-size 64 --merange 64 --analysis-load ToS_1000_R10-abr3_analysis.dat --analysis-load-reuse-level 10 --deblock -1,-1 --limit-sao

encoded 17620 frames in 477.85s (36.87 fps), 1499.96 kb/s, Avg QP:27.91, Global PSNR: 43.405, SSIM Mean Y: 0.9709562 (15.369 dB)

encoded 17620 frames in 485.03s (36.33 fps), 2001.12 kb/s, Avg QP:26.12, Global PSNR: 44.120, SSIM Mean Y: 0.9742588 (15.894 dB)


5) Reuse ABR with 3 passes, refine and psy optimisation :
transcoding from ABR 3 passes encoding veryslow at 1000 kbps (2) with analysis reuse statistique, highest possible refine level (higher speed by factor ~3x) and my best psy optimisation. Best result for my eyes.

ToS-1000-jfl.mkv (https://drive.google.com/open?id=1Tm2h5q6tbs0f5oB1Z1iRrZ93viXlCX1C)
ToS-1500-jfl.mkv (https://drive.google.com/open?id=16tf9qyh9-fNKdrAQ0-pp5X4yGctkCL5j)
ToS-2000-jfl.mkv (https://drive.google.com/open?id=1fhWCRjqBHdgSLzXfUVAUxTaSvrW1kcWR)

D:\Mes Logiciels\Codec\x265>x265.exe --input C:\ToS_1920x800_xdither.y4m --output ToS-1000-jfl.265 --input-res 1920x800 --output-depth 10 --fps 24000/1000 --bitrate 1000 --pass 3 --stats ToS-1000.log --qcomp 0.60 --preset veryslow --no-limit-modes --hevc-aq --qp-adaptation-range 1.0 --bframes 5 --b-adapt 2 --ref 5 --min-keyint 1 --keyint 120 --rc-lookahead 60 --vbv-maxrate 4000 --vbv-bufsize 12000 --psnr --ssim --tskip --qg-size 64 --merange 64 --analysis-load ToS_1000_R10-abr3_analysis.dat --analysis-load-reuse-level 10 --deblock -1,-1 --limit-sao --rdoq-level 2 --psy-rdoq 1.0 --psy-rd 0.5 --refine-intra 4 --refine-inter 3

x265 [info]: frame I: 221, Avg QP:27.55 kb/s: 10061.19 PSNR Mean: Y:43.427 U:45.710 V:45.694 SSIM Mean: 0.972341 (15.582dB)
x265 [info]: frame P: 3910, Avg QP:30.28 kb/s: 2294.97 PSNR Mean: Y:41.977 U:45.093 V:44.928 SSIM Mean: 0.966663 (14.771dB)
x265 [info]: frame B: 13489, Avg QP:32.31 kb/s: 476.09 PSNR Mean: Y:41.138 U:44.526 V:44.419 SSIM Mean: 0.965165 (14.580dB)
x265 [info]: Weighted P-Frames: Y:6.0% UV:4.7%
x265 [info]: Weighted B-Frames: Y:4.1% UV:2.8%
x265 [info]: consecutive B-frames: 10.8% 7.6% 5.9% 24.9% 21.6% 29.2%

encoded 17620 frames in 12127.64s (1.45 fps), 999.93 kb/s, Avg QP:31.80, Global PSNR: 42.167, SSIM Mean Y: 0.9655874 (14.633 dB)

D:\Mes Logiciels\Codec\x265>x265.exe --input C:\ToS_1920x800_xdither.y4m --output ToS-1500-jfl.265 --input-res 1920x800 --output-depth 10 --fps 24000/1000 --bitrate 1500 --pass 3 --stats ToS-1000.log --qcomp 0.60 --preset veryslow --no-limit-modes --hevc-aq --qp-adaptation-range 1.0 --bframes 5 --b-adapt 2 --ref 5 --min-keyint 1 --keyint 120 --rc-lookahead 60 --vbv-maxrate 4000 --vbv-bufsize 12000 --psnr --ssim --tskip --qg-size 64 --merange 64 --analysis-load ToS_1000_R10-abr3_analysis.dat --analysis-load-reuse-level 10 --deblock -1,-1 --limit-sao --rdoq-level 2 --psy-rdoq 1.0 --psy-rd 0.5 --refine-intra 4 --refine-inter 3

encoded 17620 frames in 16147.07s (1.09 fps), 1500.61 kb/s, Avg QP:28.97, Global PSNR: 43.469, SSIM Mean Y: 0.9724217 (15.594 dB)

encoded 17620 frames in 14034.95s (1.26 fps), 1998.10 kb/s, Avg QP:27.03, Global PSNR: 44.305, SSIM Mean Y: 0.9759682 (16.192 dB)


Conclusion at 1000 kbps:

- For my eyes, my best encoding (5) produce better quality than default psy x265 setting (4) or previous ultraplacebo Benwagonner encoding, and by far. It's really good quality for only 1000 kbps encoding at 1080p. For me it's a really reasonable quality for streaming on a small screen ... or for reduced internet bandwidth due to COVID19 crisis. For comparison it's by far better quality level than Netflix "premium" quality for same bitrate.
- crf encoding (1) produce best metric and by far but not best visual quality for my eyes. crf encoding has really agressive quantizer for bframe and that mean lower relative quality for low complexity part. Anyway it's higher relative quality for high complexity part too. But this compromise is not good for my eyes at 1000 kbps. At this really low bitrate level (1000 kbps for 1080p source), I prefer high compression for rate control curve (more "cbr like" and less "vbr like").
- Encoding with default x265 setting (3) don't produce good quality for my eyes. There are many problems, particulary on the edge. Perhaps that these defaults psy setting are not good for really low bitrate.
- The most interessing encoding is certainely reuse ABR transcoding (4). I make ABR at "veryslow quality preset" but at stratospheric speed (40.87 fps for 1080p source with little i5 3550 CPU), in 3 passes to have best possible constant quality rate control curve. And the result is impressive. Really better result for my eyes than crf (1) or default setting (4).


Conclusion at 2000 kbps:

- All encoding produce really good quality. Quality difference between profil encoding are not really high like for 1000 kbps encoding. I make these encoding with optimized "slower" x265 profil for (1), (2) and (3), and I obtain equivalent quality than old previous benwagonner encoding with "ultra placebo" x265 profil.
- Default setting for x265 at 2000 kbps (3) produce really better relative subjective quality than for 1000 kbps encoding. At this quality level, default setting are really good.
- You can produce really good quality at high speed with reuse encoding without refine (4).
- My prefered setting for my eyes are always reuse encoding with refine and my psy optimisation (5). For my eyes this profil have better temporal stability, noise retention, and overall quality but this time, by a really small margin.

[Opmox]

deleted

[Boulder]

Regarding comparisons of CRF vs. multipass: what's the verdict on doing the first pass with CRF and then running a second pass using the average bitrate that resulted from the CRF run? It won't work with this fixed bitrate level challenge, I was thinking more on the lines of squeezing some extra quality out of x265 if the 2-pass rate control seems to be better. Maybe there are also some options that could be left out of the CRF pass and activated only in the second pass to speed things up a bit (or use those reuse options)?

[Joekiwi]

Work for me. Perhpabs downloading problem?

Be carefull, Placebo Benwagonner encoding is really old encoding. And I suspect Rate Control problem (really bad quality in start scene). Ma's encoding produce really better overall quality. Bearm 4.5 encoding is really good too (be carefull it's zone encoding like real compressionnist encoding).




Yes, Benwagonner use Dithering for make YV12 lossless source


Thanks Sagittaire,
Will try again maybe later in week . I know NVENC will get absolutely slaughtered by yours and other encodes . - However I am curious .
I use to use paid software that promised superfast encoding speed ( ie via GPU ) . There are probably a lot of people out there like me - that used or still uses such software . - Since installing staxrip as a front end I feel I'm get much better results and that I'm in control.
I should really get to learn the x265 on CPU as well - easy on handbrake as only a few options - easy just to set CRF & preset - but will try to learn it on staxrip with it's myriad of options - or maybe just easier to grab someones command line and use the encoder directly-my Ryzen 3700x can handle it & still do NVENC coding at same time for no cost .

Without Bens' constraints GOP , max bit rate, vbv -I imagine your metrics would be quite better.

Maybe someone else can see good they can achieve by setting minimum 3fps encoding which is a good metric for encoding a movie while you sleep

[Sagittaire]

Here's another encode, same as before, no-sao but I changed rd 4 to 6 and me from umh to star, psnr went from 41.612 to 41.658 :confused:
ToS_x265_1M_41.658.mkv (http://www.mediafire.com/file/1ou8lgme5gxworx/ToS_x265_1M_41.658.mkv/file)


0.05 dB is ridiculous delta for "metric quality". and you can have small Rate Control difference to explain this delta. Choose the best way for your eyes ... ;-)



So you like everything blurry :D

mosquito noise are not detail and ringing are not texture retention ... :D

You have major problem in your encoding:
- Mosquito noise (it's not detail or noise retention but only encoding artefact)
- Ringing arround all the edge
- temporal blocking instability on flat part like the sky (certainely sao off)
- detail retention in low motion scene

I can make captures that will be terrible for your encoding.
There is a simple test to do: see if the flies(?) flying above the actress' head between frame 877 and 907 (37s to 38s) are visible. In your encoding it's not visible (or only on Pframe) simply because you have terrible ringing around the actress' head. Certainely because you have really high difference quality between inter and intra block (or between Pframe and bframe).
You have major temporal blocking and bluring on the rocket (18s to 25s) ... in fact it's the same problem for all the edge everywhere.

http://jfl1974.free.fr/Videos/ToS/Frame-892.jpg


I think that if you use really higher bitrate (with your setting), all your "detail" or "texture" will no longer be there ... :D

[Sagittaire]

Regarding comparisons of CRF vs. multipass: what's the verdict on doing the first pass with CRF and then running a second pass using the average bitrate that resulted from the CRF run? It won't work with this fixed bitrate level challenge, I was thinking more on the lines of squeezing some extra quality out of x265 if the 2-pass rate control seems to be better. Maybe there are also some options that could be left out of the CRF pass and activated only in the second pass to speed things up a bit (or use those reuse options)?

1) I just looked for the right crf value to reach the right bitrate (step 0.1 by step 0.1 for crf value). I try to use stat files from crf mode for make classic multipass mode but the result in not good (multipass encoding seem conserve the crf mode decision for I-P-B quantiser with the same ratio).

2) the right way would be to control the quantizer ratio between differents frames types but these command (--ip-ratio and -ib-ratio) seem doesen't work in the crf mode. (work in multipass mode)

[Boulder]

1) I just looked for the right crf value to reach the right bitrate (step 0.1 by step 0.1 for crf value).

2) the right way would be to control the quantizer ratio between differents frames types but these command (--ip-ratio and -ib-ratio) seem doesen't work in the crf mode. (work in multipass mode)

My thought was to run a CRF 18 encode with my usual script and settings and then run a second pass to utilize the rate control if it indeed is better. That way I would get the filesizes I'm used to without having to do any compressibility tests and get the best quality out of it. I was just thinking out loud if I can loosen any settings or reuse the analysis without affecting the whole idea too much.

The ratios should definitely work, they are for example a part of --tune grain so in case they don't, it should be reported as an issue in the tracker.

[Sagittaire]

My thought was to run a CRF 18 encode with my usual script and settings and then run a second pass to utilize the rate control if it indeed is better. That way I would get the filesizes I'm used to without having to do any compressibility tests and get the best quality out of it. I was just thinking out loud if I can loosen any settings or reuse the analysis without affecting the whole idea too much.

I make tests like that and with reuse encoding too. But like Rate Control work good with NPass, I don't investigate more. But it work. Work good, I don't know, but it's possible.


The ratios should definitely work, they are for example a part of --tune grain so in case they don't, it should be reported as an issue in the tracker.

In fact --ip-ratio and --ib-ratio doesn't work with --crf and --cu-tree on (RDO for AQ decision if my memory is good, but perhaps confusion with x264 here?). If you desactive cu-tree, ip-ratio and ib-ratio work with crf. And --tune grain desactive --cu-tree. In my memory --cu-tree off desactive all Adaptative Quantisation (psy way too).

Like I say, I make many and many test ... ;-)

[benwaggoner]

Here's another encode, same as before, no-sao but I changed rd 4 to 6 and me from umh to star, psnr went from 41.612 to 41.658 :confused:

The higher RD modes allow for more psychovisual optimizations, and those often reduce PSNR while they improve psychovisual quality. <0.5 PSNR shift isn't something that can be assumed to be better/worse without visual inspection.

[Opmox]

The higher RD modes allow for more psychovisual optimizations, and those often reduce PSNR while they improve psychovisual quality.
That make sense thanks!


mosquito noise are not detail and ringing are not texture retention ... :D
...
I think that if you use really higher bitrate (with your setting), all your "magical" detail will no longer be there ... :D

Wait, you can't see the difference here (https://slow.pics/c/JpCTppkx), here (https://slow.pics/c/F5N9jFI3) or here? (https://slow.pics/c/Q7TpM3Bv) :scared: I'm out

[Sagittaire]

That make sense thanks!



Wait, you can't see the difference here (https://slow.pics/c/JpCTppkx), here (https://slow.pics/c/F5N9jFI3) or here? (https://slow.pics/c/Q7TpM3Bv) :scared: I'm out

Will be better if you make capture for your encoding too ... ;-).

Anyway I confirm that I use highest possible curve rate compression (qcomp at 0.50) to have better quality in low motion and less in high motion. For this reason I prefer my ABR encoding to equivalent crf encoding.

Eyes are generaly more sensitive at detail and texture preservation in low motion part and less in high motion. It's for this reason that I prefer the Bearm encoding (HEVC encoder). But your eyes are perhaps more sensible at high motion scene (really short scene in this source). In this case, you can try encoding with higher qcomp default value (0.6 by default, 1.0 will be real VBR).

However I maintain that the quality for your encoding in the low motion scene is terrible. Temporal stability is catastrophic too in low motion. The rendering of moving objects in static scenes (like head movements for exemple) has really visible and have annoying temporal artifacts too. I prefer, and by far, my compromise for overall quality.

Anyway HVS is like taste and colors. Some prefer chocolate and others vanilla. Some users prefer x265 with sao and other not. But for my eyes your encoding is really bad, and particulary in scene introduction.

If Boulder or Benwagoner have an opinion ... ;-)

[Boulder]

In fact --ip-ratio and --ib-ratio doesn't work with --crf and --cu-tree on (RDO for AQ decision if my memory is good, but perhaps confusion with x264 here?). If you desactive cu-tree, ip-ratio and ib-ratio work with crf. And --tune grain desactive --cu-tree. In my memory --cu-tree off desactive all Adaptative Quantisation (psy way too).
They do work in CRF mode with cu-tree and AQ enabled.

P/B ratio 1.3
in:0 out:0 type:I q:21.94 q-aq:19.57 q-noVbv:21.94 q-Rceq:0.99 tex:582913 mv:9220 misc:2973 icu:3600.00 pcu:0.00 scu:0.00 ;
in:4 out:1 type:P q:21.94 q-aq:19.24 q-noVbv:21.94 q-Rceq:0.99 tex:614212 mv:13405 misc:3414 icu:3297.50 pcu:301.25 scu:1.25 ;
in:2 out:2 type:B q:23.08 q-aq:21.04 q-noVbv:23.08 q-Rceq:0.99 tex:386297 mv:23848 misc:6130 icu:959.50 pcu:2405.25 scu:235.25 ;
in:1 out:3 type:b q:24.21 q-aq:22.84 q-noVbv:24.21 q-Rceq:0.99 tex:221768 mv:22069 misc:7085 icu:288.50 pcu:2764.00 scu:547.50 ;
in:3 out:4 type:b q:24.21 q-aq:22.85 q-noVbv:24.21 q-Rceq:0.99 tex:234317 mv:23280 misc:6718 icu:393.75 pcu:2778.00 scu:428.25 ;
in:9 out:5 type:P q:21.94 q-aq:19.38 q-noVbv:21.94 q-Rceq:0.99 tex:589089 mv:11898 misc:3306 icu:3406.25 pcu:191.75 scu:2.00 ;
in:7 out:6 type:B q:23.08 q-aq:21.66 q-noVbv:23.08 q-Rceq:0.99 tex:346593 mv:21229 misc:5560 icu:1007.25 pcu:2402.25 scu:190.50 ;
in:5 out:7 type:b q:24.21 q-aq:22.80 q-noVbv:24.21 q-Rceq:0.99 tex:227102 mv:26712 misc:7243 icu:451.00 pcu:2715.25 scu:433.75 ;
in:6 out:8 type:b q:24.21 q-aq:22.80 q-noVbv:24.21 q-Rceq:0.99 tex:258433 mv:22466 misc:6381 icu:337.75 pcu:2954.25 scu:308.00 ;
in:8 out:9 type:b q:24.21 q-aq:22.78 q-noVbv:24.21 q-Rceq:0.99 tex:212971 mv:24199 misc:6965 icu:322.75 pcu:2758.50 scu:518.75 ;

P/B ratio 1.2
in:0 out:0 type:I q:21.94 q-aq:19.57 q-noVbv:21.94 q-Rceq:0.99 tex:582913 mv:9220 misc:2973 icu:3600.00 pcu:0.00 scu:0.00 ;
in:4 out:1 type:P q:21.94 q-aq:19.24 q-noVbv:21.94 q-Rceq:0.99 tex:614212 mv:13405 misc:3414 icu:3297.50 pcu:301.25 scu:1.25 ;
in:2 out:2 type:B q:22.73 q-aq:20.71 q-noVbv:22.73 q-Rceq:0.99 tex:414031 mv:24937 misc:6025 icu:972.75 pcu:2444.75 scu:182.50 ;
in:1 out:3 type:b q:23.52 q-aq:22.14 q-noVbv:23.52 q-Rceq:0.99 tex:271359 mv:22093 misc:6977 icu:381.75 pcu:2752.25 scu:466.00 ;
in:3 out:4 type:b q:23.52 q-aq:22.18 q-noVbv:23.52 q-Rceq:0.99 tex:284787 mv:22011 misc:6353 icu:424.25 pcu:2814.50 scu:361.25 ;
in:9 out:5 type:P q:21.94 q-aq:19.37 q-noVbv:21.94 q-Rceq:0.99 tex:589521 mv:12838 misc:3399 icu:3394.25 pcu:204.00 scu:1.75 ;
in:7 out:6 type:B q:22.73 q-aq:21.31 q-noVbv:22.73 q-Rceq:0.99 tex:376395 mv:20908 misc:5391 icu:1138.75 pcu:2305.75 scu:155.50 ;
in:5 out:7 type:b q:23.52 q-aq:22.15 q-noVbv:23.52 q-Rceq:0.99 tex:267303 mv:26748 misc:7184 icu:527.00 pcu:2682.50 scu:390.50 ;
in:6 out:8 type:b q:23.52 q-aq:22.11 q-noVbv:23.52 q-Rceq:0.99 tex:305806 mv:22149 misc:6090 icu:502.00 pcu:2792.75 scu:305.25 ;
in:8 out:9 type:b q:23.52 q-aq:22.21 q-noVbv:23.52 q-Rceq:0.99 tex:244590 mv:26586 misc:7529 icu:324.00 pcu:2809.75 scu:466.25 ;

[benwaggoner]

Well I try that too with ToS ... ;-)

--qg-size 64 save 2% size (from my memory)

But it also impairs the flexibility of AQ modes. Adaptive QP can only be done at a 64x64 block instead of 32x32. So if content changes within a 64x64 block, the encoder still has to use a single QP. Smaller qg-size allows for finer tuning for smaller details, but increases signalling overhead. Since metrics (even VMAF) kinda suck at measuring AQ gains, this is something that really need to be subjectively evaluated to see if the 2% gain is a worthwhile tradeoff.

This slower profil seem really good comprise between speed and quality. I wil test that for 1500 and 2000 encoding.
Yeah, slower is the preset where most of the advanced features of x265 and HEVC get enabled, with a lot of early exits and such so the perf hit isn't too bad. I use that as my starting point for pretty much anything.

[Sagittaire]

They do work in CRF mode with cu-tree and AQ enabled.

P/B ratio 1.3
in:0 out:0 type:I q:21.94 q-aq:19.57 q-noVbv:21.94 q-Rceq:0.99 tex:582913 mv:9220 misc:2973 icu:3600.00 pcu:0.00 scu:0.00 ;
in:4 out:1 type:P q:21.94 q-aq:19.24 q-noVbv:21.94 q-Rceq:0.99 tex:614212 mv:13405 misc:3414 icu:3297.50 pcu:301.25 scu:1.25 ;
in:2 out:2 type:B q:23.08 q-aq:21.04 q-noVbv:23.08 q-Rceq:0.99 tex:386297 mv:23848 misc:6130 icu:959.50 pcu:2405.25 scu:235.25 ;
in:1 out:3 type:b q:24.21 q-aq:22.84 q-noVbv:24.21 q-Rceq:0.99 tex:221768 mv:22069 misc:7085 icu:288.50 pcu:2764.00 scu:547.50 ;
in:3 out:4 type:b q:24.21 q-aq:22.85 q-noVbv:24.21 q-Rceq:0.99 tex:234317 mv:23280 misc:6718 icu:393.75 pcu:2778.00 scu:428.25 ;
in:9 out:5 type:P q:21.94 q-aq:19.38 q-noVbv:21.94 q-Rceq:0.99 tex:589089 mv:11898 misc:3306 icu:3406.25 pcu:191.75 scu:2.00 ;
in:7 out:6 type:B q:23.08 q-aq:21.66 q-noVbv:23.08 q-Rceq:0.99 tex:346593 mv:21229 misc:5560 icu:1007.25 pcu:2402.25 scu:190.50 ;
in:5 out:7 type:b q:24.21 q-aq:22.80 q-noVbv:24.21 q-Rceq:0.99 tex:227102 mv:26712 misc:7243 icu:451.00 pcu:2715.25 scu:433.75 ;
in:6 out:8 type:b q:24.21 q-aq:22.80 q-noVbv:24.21 q-Rceq:0.99 tex:258433 mv:22466 misc:6381 icu:337.75 pcu:2954.25 scu:308.00 ;
in:8 out:9 type:b q:24.21 q-aq:22.78 q-noVbv:24.21 q-Rceq:0.99 tex:212971 mv:24199 misc:6965 icu:322.75 pcu:2758.50 scu:518.75 ;

P/B ratio 1.2
in:0 out:0 type:I q:21.94 q-aq:19.57 q-noVbv:21.94 q-Rceq:0.99 tex:582913 mv:9220 misc:2973 icu:3600.00 pcu:0.00 scu:0.00 ;
in:4 out:1 type:P q:21.94 q-aq:19.24 q-noVbv:21.94 q-Rceq:0.99 tex:614212 mv:13405 misc:3414 icu:3297.50 pcu:301.25 scu:1.25 ;
in:2 out:2 type:B q:22.73 q-aq:20.71 q-noVbv:22.73 q-Rceq:0.99 tex:414031 mv:24937 misc:6025 icu:972.75 pcu:2444.75 scu:182.50 ;
in:1 out:3 type:b q:23.52 q-aq:22.14 q-noVbv:23.52 q-Rceq:0.99 tex:271359 mv:22093 misc:6977 icu:381.75 pcu:2752.25 scu:466.00 ;
in:3 out:4 type:b q:23.52 q-aq:22.18 q-noVbv:23.52 q-Rceq:0.99 tex:284787 mv:22011 misc:6353 icu:424.25 pcu:2814.50 scu:361.25 ;
in:9 out:5 type:P q:21.94 q-aq:19.37 q-noVbv:21.94 q-Rceq:0.99 tex:589521 mv:12838 misc:3399 icu:3394.25 pcu:204.00 scu:1.75 ;
in:7 out:6 type:B q:22.73 q-aq:21.31 q-noVbv:22.73 q-Rceq:0.99 tex:376395 mv:20908 misc:5391 icu:1138.75 pcu:2305.75 scu:155.50 ;
in:5 out:7 type:b q:23.52 q-aq:22.15 q-noVbv:23.52 q-Rceq:0.99 tex:267303 mv:26748 misc:7184 icu:527.00 pcu:2682.50 scu:390.50 ;
in:6 out:8 type:b q:23.52 q-aq:22.11 q-noVbv:23.52 q-Rceq:0.99 tex:305806 mv:22149 misc:6090 icu:502.00 pcu:2792.75 scu:305.25 ;
in:8 out:9 type:b q:23.52 q-aq:22.21 q-noVbv:23.52 q-Rceq:0.99 tex:244590 mv:26586 misc:7529 icu:324.00 pcu:2809.75 scu:466.25 ;


yes, my bad. I recheck my test and you are right. However I understood my error. For me, --pbratio 1.00 mean the same average quantizer for pframes and bframes. And it's the case for Npass mode but not for crf mode:


- In crf mode

D:\Mes Logiciels\Codec\x265>x265.exe --input C:\ToS_1920x800_xdither.y4m --output ToS-1.265 --input-res 1920x800 --output-depth 10 --fps 24000/1000 --crf 25.0 --preset fast --qcomp 0.60 --bframes 5 --b-adapt 2 --min-keyint 1 --keyint 120 --rc-lookahead 60 --vbv-maxrate 4000 --vbv-bufsize 12000 --psnr --ssim --tune ssim --ref 5 --limit-refs 3 --rd-refine --hevc-aq --qp-adaptation-range 1.0 --qg-size 64 --merange 64 --deblock -1,-1 --limit-sao --frames 1332 --seek 214 --ipratio 1.40 --pbratio 1.00
x265 [info]: frame I: 16, Avg QP:24.65 kb/s: 17300.27 PSNR Mean: Y:43.462 U:45.390 V:45.166 SSIM Mean: 0.978414 (16.658dB)
x265 [info]: frame P: 275, Avg QP:28.94 kb/s: 2716.00 PSNR Mean: Y:40.881 U:43.748 V:43.285 SSIM Mean: 0.967960 (14.943dB)
x265 [info]: frame B: 1041, Avg QP:33.59 kb/s: 542.67 PSNR Mean: Y:40.540 U:44.024 V:43.546 SSIM Mean: 0.966582 (14.760dB)
x265 [info]: Weighted P-Frames: Y:2.2% UV:1.8%
x265 [info]: consecutive B-frames: 9.3% 1.4% 2.4% 25.4% 32.3% 29.2%

encoded 1332 frames in 128.92s (10.33 fps), 1192.66 kb/s, Avg QP:32.52, Global PSNR: 41.421, SSIM Mean Y: 0.9670086 (14.816 dB)


- In NPass mode

D:\Mes Logiciels\Codec\x265>x265.exe --input C:\ToS_1920x800_xdither.y4m --output ToS-1.265 --input-res 1920x800 --output-depth 10 --fps 24000/1000 --bitrate 1192 --pass 3 --preset fast --qcomp 0.60 --bframes 5 --b-adapt 2 --min-keyint 1 --keyint 120 --rc-lookahead 60 --vbv-maxrate 4000 --vbv-bufsize 12000 --psnr --ssim --tune ssim --ref 5 --limit-refs 3 --rd-refine --hevc-aq --qp-adaptation-range 1.0 --qg-size 64 --merange 64 --deblock -1,-1 --limit-sao --frames 1332 --seek 214 --ipratio 1.40 --pbratio 1.00
x265 [info]: frame I: 16, Avg QP:28.06 kb/s: 10449.46 PSNR Mean: Y:41.454 U:43.768 V:43.453 SSIM Mean: 0.966755 (14.783dB)
x265 [info]: frame P: 275, Avg QP:30.44 kb/s: 2208.40 PSNR Mean: Y:39.873 U:42.736 V:42.292 SSIM Mean: 0.960179 (13.999dB)
x265 [info]: frame B: 1041, Avg QP:30.23 kb/s: 776.34 PSNR Mean: Y:40.138 U:43.218 V:42.801 SSIM Mean: 0.961323 (14.125dB)
x265 [info]: Weighted P-Frames: Y:2.5% UV:2.2%
x265 [info]: consecutive B-frames: 9.3% 1.4% 2.4% 25.4% 32.3% 29.2%

encoded 1332 frames in 145.19s (9.17 fps), 1188.19 kb/s, Avg QP:30.24, Global PSNR: 40.803, SSIM Mean Y: 0.9611523 (14.106 dB)

[mandarinka]

WOW.

That is truly impressive. This reinforces my belief that film grain modeling is enormously important and will really make 1 Mbps 1080p totally viable. This is a game changing feature for AV1!

Do any of the encoders have / plan to integrate this in-loop during encoding? Working with YUV intermediates is pretty painful.

If you leave the denoising part to the encoder, it will probably use something bad that will remove legitimate non-noise detail from the base picture under it. Most standalone denoisers actually do, so using some of the rare exceptions tuned for better quality should be done.

I guess the huge problem here is now that even the good denoisers can't be made to not remove any details if you use the strengths that are required to remove all the noise content.

[Sagittaire]

I guess the huge problem here is now that even the good denoisers can't be made to not remove any details if you use the strengths that are required to remove all the noise content.

Well retain grain/noise is bitrate problem. All the codec can retain noise if you use suffisant bitrate for that. Retain grain is problem at low bitrate and at low bitrate, the codec itself remove detail/grain/noise. And soft denoising is not problem in this case simply because codec will make strong denoising by itself. For me FGM is really usefull in "low bitrate" situation exactly like for this "1000 kbps / 1080p" challenge encoding.

[benwaggoner]

Well retain grain/noise is bitrate problem. All the codec can retain noise if you use suffisant bitrate for that. Retain grain is problem at low bitrate and at low bitrate, the codec itself remove detail/grain/noise. And soft denoising is not problem in this case simply because codec will make strong denoising by itself. For me FGM is really usefull in "low bitrate" situation exactly like for this "1000 kbps / 1080p" challenge encoding.
Yeah. FGM alone can probably save 20% off this particular challenge.

[Sagittaire]

First denoise the video
ffmpeg -i ToS.y4m -vf nlmeans=s=1.5 denoised.yuv
Then use the noise_model application located under ./examples/noise_model: https://aomedia.googlesource.com/aom/+/master/examples/noise_model.c
noise_model --fps=24/1 --width=1920 --height=800 --i420 --input-denoised=denoised.yuv --input=ToS.yuv --output-grain-table=film_grain.tbl
I only kept sY sCb and sCr because the generated .tbl didn't look good.

well I read this paper:
https://norkin.org/pdf/DCC_2018_AV1_film_grain.pdf

In fact I think that you must encode denoised.yuv and FGM will add grain after stream decoding.

It's logical because if you use FGM at medium/high bitrate with noisy source, codec will able to retain noise and FGM will add more noise at this same encoding natively noised.

The good way seem to be:

ffmpeg -i ToS.y4m -vf nlmeans=s=1.5 denoised.yuv

noise_model --fps=24/1 --width=1920 --height=800 --i420 --input-denoised=denoised.yuv --input=ToS.yuv --output-grain-table=film_grain.tbl

aomenc --passes=2 --pass=2 --fpf=firstpass.log --target-bitrate=1220 --kf-max-dist=120 --cpu-used=0 -t 4 --deltaq-mode=2 --film-grain-table=film_grain.tbl -o AV1-0.ivf denoised.yuv


I will try that. Where I can find compiled noise_model.exe ?

[Tadanobu]

Here is one https://drive.google.com/open?id=1T3POhbwvmRsRgNJhv_Lwkx9tmnqaTCyj

Built for Windows 2 months ago.

[benwaggoner]

well I read this paper:
https://norkin.org/pdf/DCC_2018_AV1_film_grain.pdf

In fact I think that you must encode denoised.yuv and FGM will add grain after stream decoding.

It's logical because if you use FGM at medium/high bitrate with noisy source, codec will able to retain noise and FGM will add more noise at this same encoding natively noised.
The task of denoising with parameterization so the noise can be reconstructed is a tricky one. And doesn't need to be normative in an encoder, so they punted that part?

[Blue_MiSfit]

Is it reasonable to generate the grain table with a heavily denoised reference, yet do the actual encoding with a more gently denoised clip?

I've found that getting full grain elimination using tools like SMDegrain or KNLMeansCL requires some pretty heavy handed settings, which can result in the plastic face effect. The grain synthesis is really nice, but I'd like to dial back the denoising strength when actually encoding a bit.

[benwaggoner]

Is it reasonable to generate the grain table with a heavily denoised reference, yet do the actual encoding with a more gently denoised clip?

I've found that getting full grain elimination using tools like SMDegrain or KNLMeansCL requires some pretty heavy handed settings, which can result in the plastic face effect. The grain synthesis is really nice, but I'd like to dial back the denoising strength when actually encoding a bit.
Sounds like a worthy experiment. I can imagine that the grain at playback could wind up being more intense than in the source if the grain table is based assuming more grain was removed than actually was.

[quietvoid]

Is it reasonable to generate the grain table with a heavily denoised reference, yet do the actual encoding with a more gently denoised clip?Yes, that is a good way of doing it. That way it's helping the encoder, but it's probably doing a filter pass either way so more detail might get wiped in the actual encode (maybe).
It's at least better than leaving aomenc do the denoise and the grain modeling at the same time, because it's using very basic denoise methods.

[H2sixty]

how was the source file created?

[excellentswordfight]

how was the source file created?
https://mango.blender.org/about/

https://media.xiph.org/tearsofsteel/

[H2sixty]

https://mango.blender.org/about/

https://media.xiph.org/tearsofsteel/

benwaggoner's file is not listed there, and it says dithered in his download file name. i assume its a dithered copy made from the website file...

[benwaggoner]

benwaggoner's file is not listed there, and it says dithered in his download file name. i assume its a dithered copy made from the website file...
Yes, I rendered it out from the 4K 16-bit PNG files.

IIRC, I went from those to uncompressed v210 in After Effects, and then used FFMPEG for the final conversion to yuv420p .y4m, using xdither.

[benwaggoner]

Thanks for the 2 Mbps AV1. Could you also try a 1 Mbps and a 500 Kbps? We've got samples at those rates in other codecs.

[easyfab]

First try with VVC codec and vvencapp codec (Fraunhofer Versatile Video Encoder )

tos.266 -> https://www.sendspace.com/file/qk2ko0

vvencapp -i ToS_1920x800_xdither.yuv -s 1920x800 -r 24 --preset fast -q 29 -o tos.266

Total Frames | Bitrate Y-PSNR U-PSNR V-PSNR YUV-PSNR
17620 a 912.8453 52.1495 59.0134 58.8494 40.7851

Total Time: 5167.89 sec. Fps(avg): 3.40952 encoded Frames 17620

I don't know a player that decode vvc, I'm using vvdecapp + mpv to play it for the moment :

vvdecapp -b tos.266 -o - | mpv.com --demuxer=rawvideo --demuxer-rawvideo-w=1920 --demuxer-rawvideo-h=800 --demuxer-rawvideo-mp-format=yuv420p10le --demuxer-rawvideo-fps=24 -

or to use with ffmpeg to do what you want :

vvdecapp -b tos.266 -o - | ffmpeg -f rawvideo -s 1920x800 -r 24 -pix_fmt yuv420p10le -i - ....

Here a build of vvcdecapp for those who don't want to build the source : https://www.sendspace.com/file/uhc8j6

[dipje]

Here is one https://drive.google.com/open?id=1T3POhbwvmRsRgNJhv_Lwkx9tmnqaTCyj

Built for Windows 2 months ago.

Any new link? MSVC 2019 build-tools are installing, but finding a compiled version to play around with will make life easier :s.

[benwaggoner]

I've finally added a new source for the challenge, an interesting hybrid anime/CGI title. Links and details in the updated first post (https://forum.doom9.org/showthread.php?p=1853595#post1853595).

It's got some very different properties from Tears of Steel that should stress different encoder features. I look forward to seeing what we all can do with it!

[benwaggoner]

Here's my first test encode, at 1 Mbps ABR.

https://1drv.ms/v/s!AlvIQZWsyeO-k9ly-nzoNI3Uqa-TPQ?e=baJVUi

Even though a ton of its runtime is just white text/graphics on black, it's still a MUCH harder clip to encode than Tears of Steel. Perhaps something could be done with --zones to further save bits from the credits sections to help the hard bits.

I've not done any actual tuning with the parameters. I just did a 2-pass placebo with --tune animation and throwing in a bunch of other features that would hopefully help with this kind of content. So --cu-lossless and --tskip for all the very sharp lines in the text

I did a simple --tune animation --preset slower without the fancy stuff and quality and bit distribution came out pretty much the same; just a hair lower . Which confirms the extra stuff didn't cause some qualitative regression. As expected, 20x faster with -1.5 dB PSNR and -0.22 dB SSIM.

Looking at the .csv log file, it seems that --frame-dup didn't actually set any frames to dup. so raising the threshold there might be helpful. Maybe the dithering added just that extra bit of noise?

I'm not sure if --tune animation is appropriately tuned either. I fear that preset is exactly copied from an email I sent a x265 dev extrapolating a starting point for tuning a --tune animation, and I know I didn't do adequate testing of it :sly:.

I doubt this is a clip that particularly benefits from --hme either.

I should also try --hevc-aq for comparison.

x265.exe --input SolLevante_SDRv2_1080p24_8bit.y4m --level-idc 4.0 --preset placebo --pass 1 --ref 5 --bframes 16 -F 1 --hme --hme-search 2,3,4 --fades --frame-dup --tune animation --tskip --cu-lossless --rd-refine --multi-pass-opt-analysis --multi-pass-opt-distortion --keyint 120 --rc-lookahead 120 --bitrate 1000 --vbv-maxrate 12000 --hrd --aud --vbv-bufsize 12000 --colorprim bt709 --transfer bt709 --colormatrix bt709 -o SolLevante_SDR-1080p_1000_placebo_p1.hevc --psnr --ssim --csv-log-level 1 --analysis-reuse-file SolLevante_SDR-1080p_1000_placebo.dat --stats SolLevante_SDR-1080p_1000_placebo.stats --csv SolLevante_SDR-1080p_1000_placebo_p1.csv

x265.exe --input SolLevante_SDRv2_1080p24_8bit.y4m --level-idc 4.0 --preset placebo --pass 2 --ref 5 --bframes 16 -F 1 --hme --hme-search 2,3,4 --fades --frame-dup --tune animation --tskip --cu-lossless --rd-refine --multi-pass-opt-analysis --multi-pass-opt-distortion --keyint 120 --rc-lookahead 120 --bitrate 1000 --vbv-maxrate 12000 --vbv-bufsize 12000 --hrd --aud --colorprim bt709 --transfer bt709 --colormatrix bt709 -o SolLevante_SDR-1080p_1000_placebo_p2.hevc --psnr --ssim --csv-log-level 1 --analysis-reuse-file SolLevante_SDR-1080p_1000_placebo.dat --stats SolLevante_SDR-1080p_1000_placebo.stats --csv SolLevante_SDR-1080p_1000_placebo_p2.csv

[rwill]

Your rules state:
4 Mbps peak bitrate

You specified:
--vbv-maxrate 12000

[benwaggoner]

Your rules state:
4 Mbps peak bitrate

You specified:
--vbv-maxrate 12000
Crap! Good catch. Will fix ;).

[videoh]

Why is this thread a sticky? Is it to spam your obsolete book? Just wondering.

[rwill]

Why is this thread a sticky? Is it to spam your obsolete book? Just wondering.

What is this forum for anyway ?

To advertise x265 binary builds no one runs ?
A place for foreign students to get their university multimedia assignments solved without the need to do any work ?
Another place for trolls to unleash their time wasting and irritating nonsense posts ?

Now regarding this encoding challenge I could post some Sol Levante encode like this which is quite different from Bens:
https://drive.google.com/file/d/16yag48bOLoAUjmiOEeB_9WPa7DZEkYsL/view?usp=sharing

And then people would take a look and compare and discuss and whatnot.

But no, people nowadays are more interested to post pictures on Insta and toxic one liners on Twitter. Makes no sense to do an encoding challenge with most people on this BBS anyway as everyone only has access to x265 which is a broken encoder. Encoding the sequence in high quality AV1 will take a year and other OSS encoders are still in their infancy. And I get an aneurysm every time I see someone doing encodes at a medium like preset and then still complains about encoding speed because in their opinion it runs too slow on their 15 year old 4 core SSE2 CPU.

[benwaggoner]

If y'all don't like Doom9, you're welcome to not participate, or suggest a better place for these kinds of discussions.

We've had a variety of tests in this challenge provided in a variety of codecs and encoders.

[Gravitator]

Here's my first test encode, at 1 Mbps ABR.

https://1drv.ms/v/s!AlvIQZWsyeO-k9ly-nzoNI3Uqa-TPQ?e=baJVUi




3m:35s-3m:40s dirt hangs from under the picture on the left.

[Gravitator]

Here a build of vvcdecapp for those who don't want to build the source : https://www.sendspace.com/file/uhc8j6
Can you update the link to the decoder?

[Forteen88]

x265 which is a broken encoderYou mean that not just SAO is broken in x265?

[benwaggoner]

You mean that not just SAO is broken in x265?
--hist-scenecut has been broadly found to be defective
--frame-dup doesn't appear to do anything in my tests

[Forteen88]

--hist-scenecut has been broadly found to be defective
--frame-dup doesn't appear to do anything in my testsThanks. Good that I haven't used those options in my encodes, and I've set --no-sao.

[rwill]

The rate control is also producing questionable results, especially with VBV constraints. The problem is that currently it is not guaranteed that x265 produces a consistently better picture when increasing the bitrate. There are also problems with ABR and hitting a target rate when the sequence is complex. For 99.9% of scenes in a movie the results are good but the 0.1% where it breaks is a real problem. People will remember the 0.1% where the immersion is ruined badly due to artifacts. A movie consists of around 170000 pictures, good luck not hitting that 0.1%. See the x265 encoder thread page 406 in this forum about Stacey Spears’ trial of encoding a HEVC BluRay at 90Mbit. The Deer picture has a VMAF of above 90.

So when you run bitrate ladders you are forced to QC each rung, not only the lowest bitrate of each resolution to check if the quality is ok. You cannot trust objective metrics like VMAF or PSNR to detect the errors x265 produces. When you have a 2 hour movie and generate 12 resolution/bitrate combinations thats 24 hours someone has to watch. Archiving that BluRay collection ? You better watch the stream thoroughly before you put it away as a backup.

And when there are encode problems, what are you gonna do ? Frame threads to 1 ? Wavefronts off ? Google it ? Asking for Help on Doom9 ? Couple days of trial and error and you might be able to work around the problem. Hand crafted streams….

Waste of time and resources.

Sure x265 is free but there are recurring problems no one is fixing. Heck, I’d rather try to use Linux on the Desktop again before fixing an x265 encode problem again by twiddling with parameters. You know, a Linux with printing, scanning and doing regular component updates which break half of what I have already set up. I know there are people which value their time at or below minimum wage but I am not one of them.

[benwaggoner]

Sure x265 is free but there are recurring problems no one is fixing. Heck, I’d rather try to use Linux on the Desktop again before fixing an x265 encode problem again by twiddling with parameters. You know, a Linux with printing, scanning and doing regular component updates which break half of what I have already set up. I know there are people which value their time at or below minimum wage but I am not one of them.
What do you use instead of x265?

[rwill]

What do you use instead of x265?

Oh, has my PM not reached you ?

[Fishman0919]

I did a test with Tears of Steel (ToS_1920x800_xdither.y4m) downloaded from the first post.

Used x265 3.5+1 with Vidcoder to make it easy for the test.

I started with Medium preset and went up to Placebo.
I did a few encodings with preset medium and HME.

CRF-23 keyint=250:min-keyint=23:aq-mode=3 was my base settings.

17860

Seems x265's presets need a little tuning.

the Medium-test encoding was with these settings

keyint=250:min-keyint=23:aq-mode=3:subme=7:rd=5:hme=1:hme-range=24,48,72:hme-search=star,star,star:rc-lookahead=90:bframes=8:early-skip=0:rskip=2:selective-sao=4:b-intra=1:max-merge=5:ref=5:lookahead-slices=1:rect=1:amp=1:limit-refs=0:b-intra=1:weightb=1:rd-refine=1

[takla]

https://www.mediafire.com/file/gj66iaq0uqg1x58/Sol_Levante.webm/file


ffmpeg -benchmark -i INPUT.mkv -c:v libvpx-vp9 -b:v 2M -pass 1 -cpu-used 1 -row-mt 1 -g 120 PASS-1.webm
time=51.304s


ffmpeg -benchmark -i INPUT.mkv -c:v libvpx-vp9 -b:v 2M -pass 2 -cpu-used 1 -row-mt 1 -g 120 PASS-2.webm
time=1164.221s

CPU used was AMD Ryzen 9 3900X.
Personally, I wouldn't bother watching that clip with anything under 10Mbps (Except maybe AV1 with -cpu-used 1 at ~8Mbps). 2Mbps is far from "realistic" in this scenario.

I also wouldn't bother with x265 since it would perform worse, as it always does in my comparisons.