Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
2nd July 2019, 18:19 | #1762 | Link | |
Moderator
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,770
|
Quote:
How the encoders got tuned is what matters. And if quality is being compared at fixed encoding time, performance improvements become quality improvements. |
|
2nd July 2019, 19:31 | #1763 | Link | |
Registered User
Join Date: Aug 2009
Posts: 201
|
Quote:
Imagine, for example, people who believe that tube amps or vinyl is better than digital audio. In a double-blind test they would probably still vote for the tube amp or the vinyl because it has distinctive audio characteristics that can't be removed without invalidating the test. They can hear things that they prefer and associate (conciously or not) with quality. On the other hand, they would potentially be fooled by audio that had been processed to sound like tube amps or vinyl or passed through a digital chain before output. I considered this possibility because two recent tests that were presented as being negative for AV1 specifically mentioned that some of their test participants were video engineers. They mentioned this as evidence that it was all done properly, but it seemed like an obvious test methodology failure to me. I think it was Monty from Xiph that said his party trick used to be identifying the encoder used just by listening to mp3s, and I bet certain bitrates and content would let people here do the same with video codecs and there's a possibility their opinion scores would differ from Joe Public as a result. |
|
2nd July 2019, 19:54 | #1764 | Link | |
Registered User
Join Date: Aug 2009
Posts: 201
|
Quote:
http://www.compression.ru/video/code...jective_report On the other hand, similar to how complaints about electric cars are now "I don't like the minimalism of their touchscreen interfaces" when not too long ago you'd hear how they were physical impossibilities, I think the fact that we're now at this level of complaint for the previous generation of royalty-free codecs is a testament to how far we've come. |
|
3rd July 2019, 01:30 | #1767 | Link | ||||
Moderator
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,770
|
Quote:
Quote:
Quote:
Generally I'll have video experts to an initial pass on something to see "is there something that can be seen here?" and then using double-blind testing with a more general population to confirm details. The second is a LOT slower and more expensive than the first, of course. Quote:
|
||||
3rd July 2019, 22:55 | #1769 | Link | |
Moderator
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,770
|
Quote:
Anyway, I requested an optimal Eve encoding for My encoding challenge, but they declined to participate. It is common for encoder vendors who think they are doing some magic things in the bitstream to want to have the bitstream output under NDA and such. I get the impulse, but it just isn't practical for doing actual comparisons or due diligence evaluation. Last edited by benwaggoner; 3rd July 2019 at 22:59. Reason: More details about Eve offer |
|
4th July 2019, 08:12 | #1770 | Link |
Registered User
Join Date: Aug 2009
Posts: 201
|
When you say they "declined to participate" did they respond and say they didn't want to take part or did you just not hear from them after making a broad request in a forum post?
I believe the comment above yours saying ("Have you asked?") Is written by a developer of EVE, which suggests they didn't know they'd been asked, so possibly an email has got lost in a spam trap. |
4th July 2019, 18:07 | #1771 | Link |
Registered User
Join Date: Feb 2019
Posts: 3
|
Hi guys, I've desided to make a comparison of x264, rav1e and x265 encoders with 500, 600, 700, 800, 900, 1000 kbit/s with the following settings:
rav1e -b $g --tiles 6 -s 5 --matrix BT470BG /D/sintel/sintel720.y4m --output sintel720_rav1e_s5_$g.ivf rav1e -b $g --tiles 6 -s 3 --matrix BT470BG /D/sintel/sintel720.y4m --output sintel720_rav1e_s5_$g.ivf x264 -t 2 -m 11 --me umh --weightp 2 --direct spatial --aq-mode 2 --b-adapt 2 -B $g -b 4 -r 6 -I 240 --b-pyramid normal --no-dct-decimate --no-fast-pskip -A all -o sintel720_x264_$g.264 x265 /D/sintel/sintel720.y4m --y4m -o "Sintel720_x265_$g.h265" --rd 3 -b 4 --b-adapt 2 --b-pyramid --ref 6 -I 240 --bitrate $g --aq-mode 2 --weightp --weightb -m 2 --no-early-skip --psy-rd 1 --me star where $g stands for bitrate value My OS is Arch Linux x86_64 and CPU Core i5 8600K, rav1e was build recently and for testing 1191 frames of sintel 1k 16bit (from 12987 to 14177) were taken and converted to 720x306 yuv420p. x265 and x264 are from the distro's repository. To measure MS-SSIM and PSNR-HVS-M daala's tools were used. To measure VMAF score I used ffmpeg's VMAF filter. Results: x265 is a clear winner with 50.59-61.01 fps (lowest to highest bitrate settings) x264 80.08-102.73 fps rav1e s3 1.603-2.187 fps rav1e s5 3.736-4.469 fps Average CPU utilization of rav1e was 66%-70% (and I couldn't increase it). Last edited by Ilya87; 4th July 2019 at 23:42. |
4th July 2019, 22:44 | #1772 | Link |
Registered User
Join Date: Jul 2018
Posts: 80
|
Intel SVT-AV1 0.6 Released With AV1 Decoding, SIMD Optimizations
https://www.phoronix.com/scan.php?pa...1-0.6-Released |
4th July 2019, 23:35 | #1773 | Link | |
Registered User
Join Date: Feb 2019
Posts: 3
|
Quote:
|
|
8th July 2019, 05:39 | #1775 | Link |
Registered User
Join Date: Nov 2009
Location: Northeast Ohio
Posts: 447
|
While now a version old, Phoronix tested (on Linux) the encoding performance of SVT-AV1 v0.5 on the new 3rd gen AMD Ryzen 8core (3700X) and 12core (3900X) chips compared to existing Intel CPUs (primarily the 8core 9900K and 16core 7960X):
https://www.phoronix.com/scan.php?pa...0x-linux&num=4
__________________
____HTPC____ | __Desktop PC__
2.93GHz Xeon x3470 (4c/8t Nehalem) | 4.5GHz 1.24v dual-core Haswell G3258 Radeon HD5870 | Intel iGPU 2x2GB+2x1GB DDR3-1333 | 4x4GB DDR3-1600 |
10th July 2019, 17:38 | #1777 | Link | |
Moderator
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,770
|
Quote:
I note that the top Intel processor used has only half the cores as the top AMD, so this difference could easily be due to multithreading more than per-core performance improvements. But that in no way invalidates the price/performance delta. Also, and AV1 encoder that's running only ~2.5x slower than a HEVC encoder! Of course, I have no idea if the output quality is similar. As always, the key metric is quality @ bitrate @ performance. Last edited by benwaggoner; 10th July 2019 at 17:39. Reason: Used wrong word |
|
10th July 2019, 17:45 | #1778 | Link |
Moderator
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,770
|
Also, I note that the Intel processor used in comparison is from 2017. The current equivalent would probably be the i9-9980XE, which as two more cores and 7% faster clock. That would probably have similar SVT performance to the Threadripper. At more than 2x the price, though (although for an encoding workstation/instance, the CPU is typically less than half the cost).
|
10th July 2019, 20:18 | #1779 | Link |
I am maddo saientisto!
Join Date: Aug 2018
Posts: 95
|
Status report!
"Yes I keep tweaking the params" edition 1st edition: https://forum.doom9.org/showthread.p...49#post1852449 2nd edition: https://forum.doom9.org/showthread.p...87#post1857587 3rd edition: https://forum.doom9.org/showthread.p...75#post1860475 4th edition: https://forum.doom9.org/showthread.p...39#post1871939 Whatever paragraph I don't repeat here can be assumed to be the same as in the aforementioned post First of all: graphs! Click to enlarge Y axis: chosen metric X axis: bits per pixel 720p: 1080p: BD rates for 720p: Code:
Codecs ladder: | x264 relative: x264 -> svtav1 | x264 -> svtav1 RATE (%) DSNR (dB) | RATE (%) DSNR (dB) MSSSIM -10.5381 0.426713 | MSSSIM -10.5381 0.426713 PSNRHVS -11.296 0.557542 | PSNRHVS -11.296 0.557542 HVMAF -19.6867 0.689824 | HVMAF -19.6867 0.689824 ----------------------------|----------------------------- svtav1 -> vp9 | x264 -> vp9 RATE (%) DSNR (dB) | RATE (%) DSNR (dB) MSSSIM -12.4136 0.464516 | MSSSIM -24.2802 1.23124 PSNRHVS -13.288 0.615572 | PSNRHVS -25.1991 1.68477 HVMAF -14.5152 0.598246 | HVMAF -26.3686 2.81799 ----------------------------|----------------------------- vp9 -> x265 | x264 -> x265 RATE (%) DSNR (dB) | RATE (%) DSNR (dB) MSSSIM -1.73618 0.0667664 | MSSSIM -26.2541 1.24552 PSNRHVS -6.07444 0.298073 | PSNRHVS -30.4815 1.87719 HVMAF -9.04578 0.359953 | HVMAF -31.4265 3.28152 ----------------------------|----------------------------- x265 -> av1 | x264 -> av1 RATE (%) DSNR (dB) | RATE (%) DSNR (dB) MSSSIM -20.8531 0.881529 | MSSSIM -39.9238 2.1343 PSNRHVS -16.9627 0.860883 | PSNRHVS -40.3335 2.76154 HVMAF -23.5865 1.00102 | HVMAF -48.1341 3.64521 Code:
Codecs ladder: | x264 relative: x264 -> svtav1 | x264 -> svtav1 RATE (%) DSNR (dB) | RATE (%) DSNR (dB) MSSSIM -14.3136 0.452642 | MSSSIM -14.3136 0.452642 PSNRHVS -10.1078 0.374405 | PSNRHVS -10.1078 0.374405 HVMAF -20.4048 0.58988 | HVMAF -20.4048 0.58988 ----------------------------|----------------------------- svtav1 -> vp9 | x264 -> vp9 RATE (%) DSNR (dB) | RATE (%) DSNR (dB) MSSSIM -19.1279 0.563386 | MSSSIM -34.6951 1.70828 PSNRHVS -21.5428 0.778635 | PSNRHVS -33.6391 2.16168 HVMAF -21.4399 0.750138 | HVMAF -34.3162 3.93015 ----------------------------|----------------------------- vp9 -> x265 | x264 -> x265 RATE (%) DSNR (dB) | RATE (%) DSNR (dB) MSSSIM 8.56339 -0.282927 | MSSSIM -30.5146 1.24699 PSNRHVS 3.02814 -0.139956 | PSNRHVS -32.9536 1.71646 HVMAF -3.70741 0.0299945 | HVMAF -35.6727 3.2304 ----------------------------|----------------------------- x265 -> av1 | x264 -> av1 RATE (%) DSNR (dB) | RATE (%) DSNR (dB) MSSSIM -28.044 1.00637 | MSSSIM -47.6676 2.30149 PSNRHVS -23.4583 0.991831 | PSNRHVS -45.8303 2.79923 HVMAF -26.6387 0.978822 | HVMAF -51.9814 3.88658 x264 157-2970-5493be8 x265 3.1-4-4f6dde51a5db libvpx-vp9 1.8.0-591-g19bda215d SVT-AV1 0.6.0-1424-8977f443 libaom 1.0.0-2036-ge2c1d5ef8 Cmdlines: x264 --preset veryslow --tune ssim --crf 16 -o test.x264.crf16.264 orig.i420.y4m x265 --preset veryslow --tune ssim --crf 16 -o test.x265.crf16.hevc orig.i420.y4m vpxenc --codec=vp9 --frame-parallel=0 --tile-columns=0 --auto-alt-ref=6 --good --cpu-used=0 --tune=psnr --passes=2 --threads=1 --end-usage=q --cq-level=20 --test-decode=fatal --ivf -o test.vp9.cq20.ivf orig.i420.y4m SvtAv1EncApp.exe -i orig.i420.yuv -b test.svtav1.cq20.ivf -w 1280 -h 720 -q 20 -enc-mode 3 -fps-num 24000 -fps-denom 1001 -intra-period 23 aomenc --frame-parallel=0 --tile-columns=0 --auto-alt-ref=1 --cpu-used=4 --tune=psnr --passes=2 --threads=2 --row-mt=1 --end-usage=q --cq-level=20 --test-decode=fatal -o test.av1.cq20.webm orig.i420.y4m VMAF: model used: vmaf_b_v0.6.3, pooling: harmonic_mean, bagging score (arithmetic mean of 21 models' scores) Notes: TearsOfSteel720 and TheFifthElement, two clips in the 720p category, had a vertical resolution incompatible with SvtAv1EncApp (not divisible by 8). They have been padded to 1280x536, so they have been included in this round of measurements again. Meanwhile, rav1e still has got a nasty bug that makes it bloat encodes, which brings up to 25% BD rate regression, so it has been excluded from this edition. Again, no time infos because I use the PC while it encodes etc. etc. If somebody REALLY wants some encoding time infos I can run a battery of encodes under ideal conditions on my favourite 1080p clip (PresageFlowerFight) and report the stats in a followup post (ping @benwaggoner) This concludes this report. As always, I'm open to any kind of feedback to improve my comparisons and my encodes. Last edited by SmilingWolf; 10th July 2019 at 20:25. |
10th July 2019, 20:41 | #1780 | Link | |||
Registered User
Join Date: Nov 2009
Location: Northeast Ohio
Posts: 447
|
Quote:
You can also see from the results that even normal Threadripper like the 12core 2920X (which still uses a somewhat nonstandard die configuration) is getting beaten by the 9900K and 3700X which both use a very traditional CPU core configuration by comparison (one could even argue that the separate I/O die on the 3700X is actually more traditional and is akin to the days of northbridges and external memory controllers ala Athlon XP and Core 2 Duo). Nevertheless, there could very well be a point of diminishing returns in terms of multicore scalabilty for SVT-AV1 that 32c/64t just isn't seeing the utilization that it could otherwise, and even more-so with such the nontraditional core arrangement of the 2990WX. Quote:
Quote:
This tells me that Phoronix wasn't actually trying to use the highest-end Intel CPU parts that are available, even within a given CPU generation.
__________________
____HTPC____ | __Desktop PC__
2.93GHz Xeon x3470 (4c/8t Nehalem) | 4.5GHz 1.24v dual-core Haswell G3258 Radeon HD5870 | Intel iGPU 2x2GB+2x1GB DDR3-1333 | 4x4GB DDR3-1600 |
|||
Thread Tools | Search this Thread |
Display Modes | |
|
|