So... x265 can't match x264 for lossless encoding?

[gdgsdg123]

No matter how hard I try...



"test_x265.mkv": 254,612 KB

ffmpeg -i "%X%" -c:v libx265 -preset placebo -x265-params "lossless=1:keyint=15:no-deblock=1:no-sao=1" -flags +bitexact -fflags +bitexact -map_metadata -1 "test_x265.mkv"

"test_x265_no-strong-intra-smoothing.mkv": 254,614 KB

ffmpeg -hide_banner -i "%X%" -c:v libx265 -preset placebo -x265-params "lossless=1:keyint=15:no-deblock=1:no-sao=1:no-strong-intra-smoothing=1" -flags +bitexact -fflags +bitexact -map_metadata -1 "test_x265_no-strong-intra-smoothing.mkv"

"test_x265_X.mkv": 254,535 KB

ffmpeg -hide_banner -i "%X%" -c:v libx265 -preset placebo -x265-params "lossless=1:keyint=15:me=4:subme=7:no-deblock=1:no-sao=1:no-strong-intra-smoothing=1" -flags +bitexact -fflags +bitexact -map_metadata -1 "test_x265_X.mkv"

"test_x264.mkv": 249,922 KB

ffmpeg -hide_banner -i "%X%" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1" -flags +bitexact -fflags +bitexact -map_metadata -1 "test_x264.mkv"

"test_x264_weightp=0.mkv": 250,526 KB

ffmpeg -hide_banner -i "%X%" -c:v libx264 -preset placebo -qp 0 -x264-params "keyint=15:no-deblock=1:weightp=0" -flags +bitexact -fflags +bitexact -map_metadata -1 "test_x264_weightp=0.mkv"

"test_x265_XX.mkv": 277,259 KB

ffmpeg -hide_banner -i "%X%" -c:v libx265 -preset placebo -x265-params "lossless=1:keyint=15:bframes=0:ref=8:me=4:subme=7:no-deblock=1:no-sao=1:no-strong-intra-smoothing=1" -flags +bitexact -fflags +bitexact -map_metadata -1 "test_x265_XX.mkv"

"test_x265_XXrev.mkv": 277,276 KB

ffmpeg -hide_banner -i "%X%" -c:v libx265 -preset placebo -x265-params "lossless=1:keyint=15:bframes=0:b-adapt=0:no-weightb=1:no-b-intra=1:ref=8:me=4:subme=7:no-deblock=1:no-sao=1:no-strong-intra-smoothing=1" -flags +bitexact -fflags +bitexact -map_metadata -1 "test_x265_XXrev.mkv"

"test_x265_XXFinal.mkv": 277,266 KB

ffmpeg -hide_banner -i "%X%" -c:v libx265 -preset placebo -x265-params "lossless=1:keyint=15:no-cutree=1:psy-rd=0:rdoq-level=0:bframes=0:ref=8:me=4:subme=7:no-deblock=1:no-sao=1:no-strong-intra-smoothing=1" -flags +bitexact -fflags +bitexact -map_metadata -1 "test_x265_XXFinal.mkv"

"test_x265_Finale.mkv": 277,259 KB

ffmpeg -hide_banner -i "%X%" -c:v libx265 -preset placebo -x265-params "qp=0:lossless=1:keyint=15:bframes=0:ref=8:me=4:subme=7:no-deblock=1:no-sao=1:no-strong-intra-smoothing=1" -flags +bitexact -fflags +bitexact -map_metadata -1 "test_x265_Finale.mkv"

"test_x265_FinaleX.mkv": 254,211 KB

ffmpeg -hide_banner -i "%X%" -c:v libx265 -preset placebo -x265-params "lossless=1:keyint=15:bframes=16:ref=16:me=4:subme=7:no-deblock=1:no-sao=1" -flags +bitexact -fflags +bitexact -map_metadata -1 "test_x265_FinaleX.mkv"

"test_x265_FinaleFinal.mkv": 254,335 KB

ffmpeg -hide_banner -i "%X%" -c:v libx265 -preset placebo -x265-params "allow-non-conformance=1:lossless=1:keyint=15:bframes=16:ref=16:me=4:no-deblock=1:no-sao=1" -flags +bitexact -fflags +bitexact -map_metadata -1 "test_x265_FinaleFinal.mkv"

Full command log attached.



Out of ideas... Any suggestion?

[gdgsdg123]

And mediainfo comparison on some of the outputs.



refine-analysis-type?.. what does this thing do?
(BTW "diff x265_Finale x265_XX.png" may shed some light on this)

[nevcairiel]

Lossless is a rather special case, since thats not the primary usage target of codecs like H.264 or H.265, so many of their optimizations just don't apply.
You can probably get a much better result using a dedicated lossless codec.

[gdgsdg123]

Originally Posted by nevcairiel

Lossless is a rather special case, since thats not the primary usage target of codecs like H.264 or H.265, so many of their optimizations just don't apply.
You can probably get a much better result using a dedicated lossless codec.

While there's not that much can be done for optimizations in the lossless case... after all.

Dedicated lossless codec?.. Anything better than x264? Haven't noticed one though...

[Sharc]

Originally Posted by gdgsdg123

Dedicated lossless codec?.. Anything better than x264? Haven't noticed one though...

Huffyuv, Lagarith, UTvideo to name a few.

[benwaggoner]

Originally Posted by Sharc

Huffyuv, Lagarith, UTvideo to name a few.

x265 will absolutely outperform Huffyuv and Lagarith! Interframe compression alone helps a bunch.

And it should be at least a little better than x264. Can you share the x264 command line and results?

[Sharc]

Originally Posted by benwaggoner

x265 will absolutely outperform Huffyuv and Lagarith! Interframe compression alone helps a bunch.

I mentioned these as examples for "dedicated" lossless codecs.
Isn't lossless = lossless? So x265, x264 lossless outperform the others in terms of filesize if this is taken as the performance criterion, as I understand.

[excellentswordfight]

I have always used ffv1 for lossless encoding. Never done an Apple to apples comparison to x264/x265 though. But seen some tests were it event beats x264, even though its far faster (but i would take that with some salt). But might be worth checking out.

[Stereodude]

Originally Posted by Sharc

Huffyuv, Lagarith, UTvideo to name a few.

Are you sure about that? Lossless x264 (--CRF 0 veryslow) is smaller than MagicYUV 2.0 (on 4:2:0 8-bit) which I found to be better than HuffYUV and Lagarith in prior comparisons.

Edit: Lagarith was smaller than MagicYUV on my 720x476 SD test clip, but x264 is smaller still. x265 (--lossless -p veryslow) was bigger than x264

Lagarith: 4.45GB (in AVI)
MagicYUV: 4.82GB (in AVI)
x264: 3.68GB (elementary stream)
x265: 3.81GB (elementary stream)

[benwaggoner]

Originally Posted by Sharc

I mentioned these as examples for "dedicated" lossless codecs.
Isn't lossless = lossless? So x265, x264 lossless outperform the others in terms of filesize if this is taken as the performance criterion, as I understand.

Yeah, it is hard to see how a codec that can do interframe compression wouldn’t beat a intraframe-only codec. HEVC in particular has intraframe prediction and other advanced tools that should squeeze even more bits out.

They aren’t all on even in —preset placebo, though! Using —cu-lossless and —tskip can help improve efficiency and quality of near-lossless encoding. Cu-lossless allows any given CTU to use lossless mode if it is more efficient; real lossless is just using that for the whole frame.

[Stereodude]

Updated post 9 with the x265 results. It is bigger than the x264 encode.

[benwaggoner]

Originally Posted by Stereodude

Updated post 9 with the x265 results. It is bigger than the x264 encode.

Intriguing. Can you share the full command line and the console output? Something interesting is happening here.

[poisondeathray]

I did about 30 tests over the last few years (different sources, different genres), 100% of the time , x264 lossless compresses better than x265 lossless (at least for 8bit 4:2:0) with default settings. Typically a few % better .

FFV1 (in interframe mode) and x264 typically have the higher compression ratio . If source has identical content, duplicate frames, then lagarith can be better (null frame option)

[Stereodude]

Originally Posted by benwaggoner

Intriguing. Can you share the full command line and the console output? Something interesting is happening here.

x265:

x265.exe -f 34164 -p veryslow --lossless --keyint 150 --open-gop -D 8 --colorprim "smpte170m" --transfer "smpte170m" --colormatrix "smpte170m" --sar 32:27...

avs2pipemod[info]: writing 34164 frames of 30000/1001 fps, 720x476,
                   sar 0:0, YUV-420-planar-8bit progressive video.
y4m  [info]: 720x476 fps 30000/1001 i420p8 sar 32:27 unknown frame count
raw  [info]: output file: x265_ll_test_2.0.265
x265 [info]: HEVC encoder version 2.9+15-81373aab81df
x265 [info]: build info [Windows][MSVC 1916][64 bit] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x265 [info]: Main profile, Level-8.5 (Main tier)
x265 [info]: Thread pool created using 16 threads
x265 [info]: Slices                              : 1
x265 [info]: frame threads / pool features       : 4 / wpp(8 rows)
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 3 inter / 3 intra
x265 [info]: ME / range / subpel / merge         : star / 57 / 4 / 4
x265 [info]: Keyframe min / max / scenecut / bias: 15 / 150 / 40 / 5.00
x265 [info]: Lookahead / bframes / badapt        : 40 / 8 / 2
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 1
x265 [info]: References / ref-limit  cu / depth  : 5 / off / on
x265 [info]: Rate Control                        : Lossless
x265 [info]: tools: rect amp limit-modes rd=6 psy-rd=2.00 rdoq=2 psy-rdoq=1.00
x265 [info]: tools: rskip limit-tu=4 signhide tmvp b-intra
x265 [info]: tools: strong-intra-smoothing deblock sao
avs2pipemod[info]: finished, wrote 34164 frames [100%].a 0:00:22
avs2pipemod[info]: total elapsed time is 12722.433 sec.
x265 [info]: frame I:    534, Avg QP:4.00  kb/s: 34259.39
x265 [info]: frame P:   9718, Avg QP:4.00  kb/s: 32024.29
x265 [info]: frame B:  23912, Avg QP:4.00  kb/s: 27268.91
x265 [info]: Weighted P-Frames: Y:11.5% UV:9.0%
x265 [info]: Weighted B-Frames: Y:22.2% UV:17.3%
x265 [info]: consecutive B-frames: 15.8% 12.6% 25.7% 26.8% 8.0% 10.4% 0.5% 0.1% 0.1%
x265 [info]: lossless compression ratio 2.86::1

encoded 34164 frames in 12752.24s (2.68 fps), 28730.85 kb/s, Avg QP:4.00

x264:

x264.2935.x64.exe" --crf 0 --preset veryslow --tune film --threads 12 --lookahead-threads 2 --aq-strength 1.2 --keyint 150 --open-gop --colorprim "smpte170m" --transfer "smpte170m" --colormatrix "smpte170m" --sar 32:27...

avs [info]: 720x476p 32:27 @ 30000/1001 fps (cfr)
x264 [info]: using SAR=32/27
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x264 [info]: profile High 4:4:4 Predictive, level 4.0, 4:2:0, 8-bit
x264 [info]: frame I:1421  Avg QP: 0.00  size:138568
x264 [info]: frame P:32743 Avg QP: 0.00  size:114868
x264 [info]: mb I  I16..4..PCM: 52.7%  0.0% 47.3%  0.0%
x264 [info]: mb P  I16..4..PCM: 29.6%  0.0% 21.6%  0.0%  P16..4: 34.1%  5.0%  7.4%  1.3%  0.8%    skip: 0.3%
x264 [info]: 8x8 transform intra:0.0% inter:61.4%
x264 [info]: coded y,uvDC,uvAC intra: 99.6% 94.6% 94.5% inter: 96.9% 78.3% 78.3%
x264 [info]: i16 v,h,dc,p: 40% 25% 32%  4%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 42% 28% 10%  4%  4%  4%  3%  3%  2%
x264 [info]: i8c dc,h,v,p: 18% 27% 50%  5%
x264 [info]: Weighted P-Frames: Y:19.7% UV:15.4%
x264 [info]: ref P L0: 83.0%  4.8%  7.2%  2.1%  0.7%  0.5%  0.5%  0.2%  0.2%  0.1%  0.1%  0.1%  0.1%  0.1%  0.1%  0.1%
x264 [info]: kb/s:27777.17

encoded 34164 frames, 42.93 fps, 27777.17 kb/s

[excellentswordfight]

Originally Posted by excellentswordfight

I have always used ffv1 for lossless encoding. Never done an Apple to apples comparison to x264/x265 though. But seen some tests were it event beats x264, even though its far faster (but i would take that with some salt). But might be worth checking out.

Did a quick test on a 1min 1080p Bluray sample:

Original: 179 MiB
x264 (preset veryslow, crf 0): 733 MiB
x264 (preset fast, crf 0): 765 MiB
x265 (preset veryslow, lossless): 808 MiB
FFV1: 831 MiB
JPEG2000: 932 MiB
x265 (preset fast, lossless): 934 MiB
Ut Video: 1.30 GiB
HUFFYUV: 1.87 GiB
Uncompressed: 4.17GiB

[Asilurr]

The same seems to happen with libvpx, it just got outmatched on a small "DVD rip" sample (4:3 576p50, 8-bit 4:2:0, ~700 frames):
x264 --preset veryslow --qp 0 || relative filesize 100.0%
x264 --preset fast --qp 0 || 100.6%
vpxenc --cpu-used 0 --qmin 0 --qmax 0 || 104.1%
vpxenc --cpu-used 3 --qmin 0 --qmax 0 || 116.6%

Frankly, the gaps are reminiscent of ESF's results above. Both x265 and libvpx use at least ~5% more bitrate than x264 in lossless scenarios, apparently.

[kolak]

Originally Posted by excellentswordfight

Did a quick test on a 1min 1080p Bluray sample:

Original: 179 MiB
x264 (preset veryslow, crf 0): 733 MiB
x264 (preset fast, crf 0): 765 MiB
x265 (preset veryslow, lossless): 808 MiB
FFV1: 831 MiB
JPEG2000: 932 MiB
x265 (preset fast, lossless): 934 MiB
Ut Video: 1.30 GiB
HUFFYUV: 1.87 GiB
Uncompressed: 4.17GiB

Try 5 different source files with different nature.

[Stereodude]

Originally Posted by kolak

Try 5 different source files with different nature.

Because everyone who has tested it so far with the same results just have happened to find the right footage where it fails to even match x264?

[excellentswordfight]

Originally Posted by kolak

Try 5 different source files with different nature.

Did you see any claims in my post? No, just a quick test, with no opinion in regards to whats better then the other cause I know that there isnt enough data to do so. If you wanna do an more conclusive test go ahead and do it yourself.

But atm, it's not looking too promising for x265 for lossless, and thats even if you take speed out of the equation.

[kolak]

I'm not saying h265 has to be better. If it was confirmed by different people using random sources then then it's what it's. Lossless compression is special case for codecs like h264/5, so I'm somehow not specially surprised about end results.