x265 HEVC Encoder

[benwaggoner]

x265 1.9+167-bebae72f9db7 (http://www.mediafire.com/download/6i1cizeppw4191t/x265_1.9+167-bebae72f9db7.7z) provides a few changes regarding the grain handling, and a new tweak:

--[no-]recursion-skip Enable early exit from recursion. Default enabled
I think these are some actually pretty huge changes in regards to grain. It would be a great build for everyone struggling with detail and grain issues to retest with, particularly but not exclusively, with --tune grain.

MultiCoreWare could presumably use a lot of feedback on the results here.

[x265_Project]

MultiCoreWare could presumably use a lot of feedback on the results here.

Yes, please.

[fauxreaper]

I've tested --no-recursion-skip on a short sample that was posted on devel list (https://mailman.videolan.org/pipermail/x265-devel/2016-May/010350.html) and the results are good.


Reduced ghosting (but ghosting still exists)
Increased visual quality
Reduced bitrate (1/3 less bitrate on same crf)


Encode gets 50% slower.

[littlepox]

Encode gets 50% slower.

ROFT

But at least they made the breakthrough, good job.

[ndkamal]

Can you post a before & after image? Thanks

I have posted some pictures in "Comparisons of x264 vs x265 " section forum

[pingfr]

@littlepox: Would you say it is safe and a quality benefit/visual benefit to use your custom parameters you cooked us since 1.9+1 with the newest --no-recursion-skip feature/tweak they added in 1.9+167?

Something along the lines of:

--preset veryslow --ctu 32 --max-tu-size 16 --crf 18 --no-recursion-skip --tu-intra-depth 2 --tu-inter-depth 2 --rdpenalty 2 --me 3 --subme 5 --merange 44 --b-intra --no-rect --no-amp --ref 5 --weightb --keyint 360 --min-keyint 1 --bframes 8 --aq-mode 1 --aq-strength 1.0 --rd 5 --psy-rd 1.6 --psy-rdoq 8.0 --rdoq-level 1 --no-sao --no-open-gop --rc-lookahead 80 --max-merge 4 --qcomp 0.80 --no-strong-intra-smoothing --deblock -2:-2 --qg-size 16 --pbratio 1.2

:)

Edit: Typo, I meant 1.9+167 not 1.9+197. :p

[littlepox]

@littlepox: Would you say it is safe and a quality benefit/visual benefit to use your custom parameters you cooked us since 1.9+1 with the newest --no-recursion-skip feature/tweak they added in 1.9+197?

Something along the lines of:

--preset veryslow --ctu 32 --max-tu-size 16 --crf 18 --no-recursion-skip --tu-intra-depth 2 --tu-inter-depth 2 --rdpenalty 2 --me 3 --subme 5 --merange 44 --b-intra --no-rect --no-amp --ref 5 --weightb --keyint 360 --min-keyint 1 --bframes 8 --aq-mode 1 --aq-strength 1.0 --rd 5 --psy-rd 1.6 --psy-rdoq 8.0 --rdoq-level 1 --no-sao --no-open-gop --rc-lookahead 80 --max-merge 4 --qcomp 0.80 --no-strong-intra-smoothing --deblock -2:-2 --qg-size 16 --pbratio 1.2

:)

I would say it's unsafe for any version newer than 1.9+10

[pingfr]

@littlepox: Alrighty, that makes sense. Let's wait on newer visually optimzed 1080p and 720p custom profiles from you for 1.9+167++ then. :p

Edit: Typo, I meant 1.9+167 not 1.9+197. :p

[pistacho]

Bug report:

x265 builds 1.9.150 and newer crashes if filenames have accentuated characters on Spanish Windows (CMD with codepage 850 Multilingual Latin I). Probably the problem occurs with other languages as well.

Example:

With build 1.9.169 crashes:

http://115.imagebam.com/download/TWUxm1XWC0LbQkJlBmNTDw/48371/483702433/169.png

Same command line with build 1.9.149 works fine:

http://116.imagebam.com/download/V08y8a6Z5JNu0vli0-upvA/48371/483702432/149.png

Problem exist from this commit: https://bitbucket.org/multicoreware/x265/commits/00ea3784bd36c164c5f799c998d7a09f2cb244bf

Thanks!

[Ma]

x265 builds 1.9.150 and newer crashes if filenames have accentuated characters on Spanish Windows

'--qpfile' is not converted to Unicode yet. Fix should be soon. Thanks for report.
-----------------
Patch file for '--qpfile' and '--csv' attached.
-----------------
There are 2 problems with '--qpfile' -- now it doesn't support Unicode filenames and if you specify nonexistent file GCC build of x265 hangs, VS 2015 build of x265 exits with wired error messages.

So in patch 13370 (https://patches.videolan.org/patch/13370/) Unicode filenames are working with '--qpfile' and '--csv' options and if you specify nonexistent file in '--qpfile' option, x265 displays error message and encode without hangs.

For tests I've compiled multilib version of x265 1.9+169 + 13370 (https://patches.videolan.org/patch/13370/) + 13312 (https://patches.videolan.org/patch/13312/) patches -- www.msystem.waw.pl/x265/x265-1.9+169+input+qpfile+csv.7z

[pistacho]

I confirm that now works again with accented characters. Also if --qpfile file name not exists error message is displayed but continues encoding (not hangs anymore).

Thanks!

[Grojm]

I've tested --no-recursion-skip on a short sample that was posted on devel list (https://mailman.videolan.org/pipermail/x265-devel/2016-May/010350.html) and the results are good.


Reduced ghosting (but ghosting still exists)
Increased visual quality
Reduced bitrate (1/3 less bitrate on same crf)


Encode gets 50% slower.

I have compared the results from the mailing list and the no-recursion-skip version is only slightly better, the issue is far from solved. Also, the no-recursion-skip version was 10% larger, and encoding time goes up. Instead of finding workaround with only minor improvement but at the cost of major slowdown, they should focus on finding the root of this issue. x264 is completely free from this artifact, so ghosting is by no means a tradeoff you have to make for motion based video compression.

[littlepox]

As clear as I remember, x264 used to have all the same blames: slow, blurry, banding/blocking, motion artifacts... It took years to mature, not over a single night.

It shall be same for x265, which is mostly a brand new encoder whose internal logic has changed almost completely. Do not expect miracles out of it to be an immediate, all-round upgrade for x264.

I'm always emphasizing that currently we prefer x264 over x265, NOT because x265 is poor, but x264 is just incredibly good. Do you blame Korean/Japanese/Singaporean table tennis players for their losing to Chinese in competitions? If you ever compare x265 to all other encoders like Adobe H264, Quicktime, libvpx... You'll realize that it's far outstanding.

Thanks again for all the great work of the developers; keep it up.

[benwaggoner]

I have compared the results from the mailing list and the no-recursion-skip version is only slightly better, the issue is far from solved. Also, the no-recursion-skip version was 10% larger, and encoding time goes up. Instead of finding workaround with only minor improvement but the cost of mayor slowdown, they should focus on finding the root of this issue. x264 is completely free from this artifact, so ghosting is by no means a tradeoff you have to make for motion based video compression.
Don't assume that the very first check-in of a new feature includes full optimization or full quality tuning. x265 has gotten a lot better and a lot faster over the last 18 months!

For something as far along as x265, having any single new feature that offers an obvious visual quality improvement is a huge accomplishment. x264 hasn't had one of those since, sheesh, mbtree? Most codec development is about squeezing out a bunch of discreet <<1% compression efficiency gains.

[littlepox]

Don't assume that the very first check-in of a new feature includes full optimization or full quality tuning. x265 has gotten a lot better and a lot faster over the last 18 months!

For something as far along as x265, having any single new feature that offers an obvious visual quality improvement is a huge accomplishment. x264 hasn't had one of those since, sheesh, mbtree? Most codec development is about squeezing out a bunch of discreet <<1% compression efficiency gains.

Hmm...recently x264 has added aq-mode=3 into the vanilla build which gives some significant improvements. However that had been in unofficial builds (like tMod or kmod) for years.

[mandarinka]

Hmm...recently x264 has added aq-mode=3 into the vanilla build which gives some significant improvements. However that had been in unofficial builds (like tMod or kmod) for years.

Whenever I tried that, I wasn't able to find the proper strength setting (that would look better than mode 1 strength 0.8 which I use for cel anime). What strengths are you using?
(Sorry for off-topic.)

[littlepox]

Whenever I tried that, I wasn't able to find the proper strength setting (that would look better than mode 1 strength 0.8 which I use for cel anime). What strengths are you using?
(Sorry for off-topic.)

I'd often pick aq=3:0.8 for high quality jap anime BDRip for crf≈16, 10bit x264.

Not so sure for your case; it's possible aq=1 is still relatively better. try strength=0.7?

[jlpsvk]

I've tested --no-recursion-skip on a short sample that was posted on devel list (https://mailman.videolan.org/pipermail/x265-devel/2016-May/010350.html) and the results are good.


Reduced ghosting (but ghosting still exists)
Increased visual quality
Reduced bitrate (1/3 less bitrate on same crf)


Encode gets 50% slower.


points 1,2 and 4 are correct. but don't know where you get 1/3 bitrate saving. encoded 2 movies, and have only about 130kbps save on the same settings and crf.

[RainyDog]

Hmm...recently x264 has added aq-mode=3 into the vanilla build which gives some significant improvements. However that had been in unofficial builds (like tMod or kmod) for years.

I thought you'd said that aq-mode 3 was a waste of bits in your x265 tune-film thread (http://forum.doom9.org/showthread.php?t=172458) littlepox? Or is it better suited to x264?

[littlepox]

I thought you'd said that aq-mode 3 was a waste of bits in your x265 tune-film thread (http://forum.doom9.org/showthread.php?t=172458) littlepox? Or is it better suited to x264?

Why would you even link them together?
The truth is aq3 is well suited in x264 but ill suited in x265.

[fauxreaper]

points 1,2 and 4 are correct. but don't know where you get 1/3 bitrate saving. encoded 2 movies, and have only about 130kbps save on the same settings and crf.

I used --bframes 8 on that sample. I encoded some bigger samples of anime videos (10-15 minute length) and bitrate reduction with --no-recursion-skip is smaller.

[EncodedMango]

Hello, say I want to encode a video without having to create a raw YUV/Y4M file first, how do I pipe it using only ffmpeg without the x265.exe?
ffmpeg -i "INPUT.avi" -c:v libx265 -preset medium -x265-params crf=20 -pix_fmt yuv420p -c:a libmp3lame -q:a 3 "OUTPUT.mp4"

Assuming YUV4MPEGPIPE is used, could someone tell me how using the above command? Thanks.
(Parameters might be incorrect, it's just to give an example)

[sneaker_ger]

I don't understand. You want to use a pipe but not use x265.exe? Then what do you need the pipe for?
You can either:
a.) use a pipe to send ffmpeg.exe y4m output to x265.exe or
b.) use integrated libx265 of ffmpeg.exe. (Like in your example)

[mandarinka]

x265 1.9+167-bebae72f9db7 (http://www.mediafire.com/download/6i1cizeppw4191t/x265_1.9+167-bebae72f9db7.7z) provides a few changes regarding the grain handling, and a new tweak:

--[no-]recursion-skip Enable early exit from recursion. Default enabled

Is --no-recursion-skip changing how analysis is done at slow settings (veryslow, placebo)?

At first I thought that --recursion-skip is the new functionality and --no-recursion-skip just disables it (and since it is auto set to disabled on RD 6, there would be no change in behaviour from prior state on slowest setting). But I am not so sure now. Does --no-recursion-skip change output compared to builds before the whole feature was introduced?

[fauxreaper]

Is --no-recursion-skip changing how analysis is done at slow settings (veryslow, placebo)?

At first I thought that --recursion-skip is the new functionality and --no-recursion-skip just disables it (and since it is auto set to disabled on RD 6, there would be no change in behaviour from prior state on slowest setting). But I am not so sure now. Does --no-recursion-skip change output compared to builds before the whole feature was introduced?

--no-recursion-skip does more analysis, its a new functionality. --recursion-skip is how analysis worked before.

[mandarinka]

Thanks, I guess I'll have to go back to that painful testing, then.

Damn, I just finished two weeks of encoding.

[Jamaika]

Colleagues clarify to me
How is it the function of the limit-ref?
In my "veryslow" is disabled. In the latest table is one.
https://github.com/videolan/x265/commit/3545d4c0a6e1a9ee3af5a714e99317f7b39bf6cf

[foxyshadis]

Colleagues clarify to me
How is it the function of the limit-ref?
In my "veryslow" is disabled. In the latest table is one.
https://github.com/videolan/x265/commit/3545d4c0a6e1a9ee3af5a714e99317f7b39bf6cf

Then you should update your x265, it's been 1 for veryslow since December.

[EncodedMango]

I don't understand. You want to use a pipe but not use x265.exe? Then what do you need the pipe for?
You can either:
a.) use a pipe to send ffmpeg.exe y4m output to x265.exe or
b.) use integrated libx265 of ffmpeg.exe. (Like in your example)
Okay I might have mixed something up, earlier(many many months ago) when I tried using x265 from ffmpeg, it never worked because it wasn't accepting anything other than Y4M/YUV(?).

It seems to be working directly now. I might have confused x265.exe and ffmpeg here. My mistake sorry.

[Magik Mark]

Build 170 --> 2 pass 10bit x265 "tune none" preset medium

First pass --> 5fps slower
Second pass --> same


Sent from my iPhone using Tapatalk

[x265_Project]

I hope all of you have been doing some encodes with the latest development build of x265. The changes made one week ago were fairly important. We're interested in your feedback on the visual quality of x265 encodes at all presets, compared with earlier builds. --preset veryslow and placebo (with --no-skip-recursion) will show the biggest change, but all presets were affected.

Personally, I'm seeing the best encoding quality I've ever seen from x265, particularly on very challenging content (water, grainy content). I'm also noticing that --preset placebo will now show a meaningful difference from --preset veryslow.

[Magik Mark]

May we ask where to find the default switches of all presets so we could update our GUI. Thanks a lot


Sent from my iPhone using Tapatalk

[x265_Project]

May we ask where to find the default switches of all presets so we could update our GUI. Thanks a lot


http://x265.readthedocs.io/en/default/presets.html

or in the x265 source...
x265 / doc / reST / presets.rst

[Jamaika]

For veryslow is a big improvement. (: However, the figures are even shades of purple.
Codec 1.9+170 (GCC 6.1.1) worked faster because he has now turned 'limit-ref = 1'. However, this message doesn't show.
x265 [info]: References / ref-limit cu / depth : 5 / off / on
As for the 'tune grain'. I understand that this is an option only for the animation, pass = 2,3 and crf. I will not be spoken.

[Ma]

Build 170 --> 2 pass 10bit x265 "tune none" preset medium

First pass --> 5fps slower
Second pass --> same

Could you specify 5fps slower than what version? Could you specify command lines to reproduce?

[tObber166]

1.9+144 vs 1.9+169, I've noticed a slowdown about 0.9 fps on a 1080p clip

preset medium --crf 17.0 --ref 5 --vbv-bufsize 20000 --vbv-maxrate 20000 --rc-lookahead=25 --me 3 --psy-rd 1.0 --no-sao

[x265_Project]

1.9+144 vs 1.9+169, I've noticed a slowdown about 0.9 fps on a 1080p clip

preset medium --crf 17.0 --ref 5 --vbv-bufsize 20000 --vbv-maxrate 20000 --rc-lookahead=25 --me 3 --psy-rd 1.0 --no-sao

Absolute numbers are not very helpful. Can you describe the effect on performance in relative terms?

Did you notice a meaningful improvement in visual quality?

[tObber166]

Absolute numbers are not very helpful. Can you describe the effect on performance in relative terms?

Did you notice a meaningful improvement in visual quality?


-Quality wise, I have not noticed any difference.

-Average bitrate is pretty much the same on both encodes.

-Final size pretty much the same. 10kb smaller on +169


pic: 144 vs 169

http://s32.postimg.org/ss435edoh/144_vs_169.jpg (http://postimg.org/image/ss435edoh/)

[LigH]

Still, is it 0.9 fps slower than 20 fps before, or 0.9 fps slower than 2 fps before?

OK, 12.57 : 13.50 ~ 93.1%

[Motenai Yoda]

it's 12.97 vs 12.06

[Ma]

1.9+144 vs 1.9+169, I've noticed a slowdown about 0.9 fps on a 1080p clip

preset medium --crf 17.0 --ref 5 --vbv-bufsize 20000 --vbv-maxrate 20000 --rc-lookahead=25 --me 3 --psy-rd 1.0 --no-sao

I can confirm the slow-down. I compared ver. 1.9+144 vs 1.9+170 vs 1.9+183 with your options. Each version produces different output file.

In VS 2015, AVX-CPU, clean builds line (x265vsNNNac) I have relative encoding times:
1.9+144 | 100.0%
1.9+170 | 105.7%
1.9+183 | 107.2%

In GCC 6 line (x265gcNNN) I have relative encoding times:
1.9+144 | 100.0%
1.9+170 | 105.7%
1.9+183 | 107.0%

In 1.9+183 version I have relative (to different builds) encoding times:
100.0% | x265vs183ac- VS 2015, AVX, clean
099.9% | x265vs183a - VS 2015, AVX
100.8% | x265vs183 -- VS 2015
101.6% | x265gc183a - GCC 6.1.1, AVX
102.3% | x265gc183 -- GCC 6.1.1

Full data in attached screen.txt

[pingfr]

I must ask what is the intent of the changes pushed in commit 1b2b912?

What does this actually improve/implies for the *end-user in terms of decoding ressources/visual quality?

*end-user = not the person doing the encode, rather the person watching said encoded results.

[jlpsvk]

I used --bframes 8 on that sample. I encoded some bigger samples of anime videos (10-15 minute length) and bitrate reduction with --no-recursion-skip is smaller.

I also encoded with --bframe 8 and also with --qpstep 8. And I encoded the whole movies.

[nandaku2]

I must ask what is the intent of the changes pushed in commit 1b2b912?

What does this actually improve/implies for the *end-user in terms of decoding ressources/visual quality?

*end-user = not the person doing the encode, rather the person watching said encoded results.

Hi,
For multi-pass, the best encode/visual quality is observed when the settings are as similar as possible across passes. Earlier, pass 1 had the turbo settings enabled by default, which means the encodes in each pass were substantially different (even though the actual commandlines were similar). This was causing lower quality multi-pass encodes, and needless confusion for users.

[burfadel]

-Quality wise, I have not noticed any difference.

-Average bitrate is pretty much the same on both encodes.

-Final size pretty much the same. 10kb smaller on +169


pic: 144 vs 169

http://s32.postimg.org/ss435edoh/144_vs_169.jpg (http://postimg.org/image/ss435edoh/)

Try the settings below and see if it is any better quality, without losing too much on the encoding time side of things. I've put a comment in italics next to them separated by ///, don't copy those if you copy the settings :).


--output-depth 10 /// your output depth is currently 8, ideally 8 would only be used if the target device only supports 8 and not 10 bit, which is unlikely at that profile level?...
--rd 4 /// as I came across a couple of weeks ago, 4 isn't the same as 3, it's a little more accurate and potentially higher quality output without noticeably affecting encode speed. --rd 5 provides a nicer picture but is much slower. Documentation says 4 is the same as 3 still, it is NOT. If you don't believe it, do identical encodes, one with 3 and the other 4 and compare the states, like the kb/s for p, b and I frames etc. If they are truly identical all stats should be exactly the same, as with the file size output. The only difference between the two files is the stat info where one will be 3 and the other 4, which wouldn't affect file size
--tu-intra-depth 4 /// slower than default 1 obviously, but --tu-intra-depth doesn't affect speed too much unlike --tu-inter-depth
--rdoq-level 1 /// in my opinion this provides a noticeable PQ improvement, almost like using --rd 5 without as much of the speed penalty!
--early-skip /// performance recovery without noticeable PQ detriment
--fast-intra /// performance recovery without noticeable PQ detriment
--b-intra /// improved PQ, normally on profile slower and above
--tskip /// performance recovery without noticeable PQ detriment
--tskip-fast /// performance recovery without noticeable PQ detriment
--limit-modes /// only set for profiles 6, 7, 8 by default, speed improvment...
--qg-size 16 /// quality setting
--me star /// you already use this, default for profile slow and above. Considering its speed I'm surprised it's not used in place of hex as default
--max-merge 3 /// default for slow mode
--weightb /// default for profile slower and above. Interestingly enough the default for x264 is having this on at medium!
--aq-mode 2 /// [I]this is probably a bit of a controversial setting, I feel there is an improvement. Try it and compare. Don't just compare the foreground, compare the background and more flat areas etc


File size will be different. --rdoq-level 1 increases output file size a little, however the quality difference is perceptively greater than what you would fine with using a lower CRF to achieve the same file size output. If you don't like the file size, you can raise your CRF to get closer to the same file size. I feel by doing so, the PQ is still improved. If you don't believe there is any difference, try a higher CRF like 22 and then compare the two different settings.

You may want to try a slightly higher CRF as the file size will be bigger with the use of Variance AQ. Try 17.5 instead of 17, I believe it will sitll produce worthwhile benefit.

[Motenai Yoda]

--rd 4 /// as I came across a couple of weeks ago, 4 isn't the same as 3, it's a little more accurate and potentially higher quality output without noticeably affecting encode speed. --rd 5 provides a nicer picture but is much slower. Documentation says 4 is the same as 3 still, it is NOT. If you don't believe it, do identical encodes, one with 3 and the other 4 and compare the states, like the kb/s for p, b and I frames etc. If they are truly identical all stats should be exactly the same, as with the file size output. The only difference between the two files is the stat info where one will be 3 and the other 4, which wouldn't affect file size
as even the devs wrote 4 and 3 are the same, in the code there isn't any if of conditional about 3 or 4, also the same encode give little differences coz non deterministic things (you can do many times and get differents results with the same settings), all metrics like average quantizer, ssim and psnr are exactly the same.

rd-level 1 give you better quality for bigger size, generally, for the same size, rd-level 2 is better than 1.

[burfadel]

It's a tough call between --rdoq-level 1 and --rdoq-level 2. Level 2 does mean the output is a little smaller, however I do think you lose a little bit of detail compare to 1 when you look at all aspects of the picture.

At level 1 rate-distortion cost is used to find optimal rounding values for each level (and allows psy-rdoq to be effective). It trades-off the signaling cost of the coefficient vs its post-inverse quant distortion from the pre-quant coefficient. When --psy-rdoq is enabled, this formula is biased in favor of more energy in the residual (larger coefficient absolute levels)

At level 2 rate-distortion cost is used to make decimate decisions on each 4x4 coding group, including the cost of signaling the group within the group bitmap. If the total distortion of not signaling the entire coding group is less than the rate cost, the block is decimated. Next, it applies rate-distortion cost analysis to the last non-zero coefficient, which can result in many (or all) of the coding groups being decimated. Psy-rdoq is less effective at preserving energy when RDOQ is at level 2, since it only has influence over the level distortion costs.
http://x265.readthedocs.io/en/latest/cli.html#cmdoption--rdoq-level

Going by that, wouldn't it mean level 2 isn't as good as level 1 in some aspects? I even tried manipulating the CRF (by lowering it slightly, since you can use decimals) to get bascally the exact same output file size. I still believe even by doing so that level 1 retains some detail more effectively. It may be that for flat images like typical animation that level 2 is generally 'better' as you say, but in a high detail, texture clip with some fine noise that I used, level 1 was more effective. I tried several different encodes, and subjectively I felt that level 1 did a nicer job.

[LigH]

x265 1.9+183-4723933fdec9 (https://www.mediafire.com/download/8o1l8spi8i5fcop/x265_1.9+183-4723933fdec9.7z): --slow-firstpass is now enabled as default; quantization is clipped in 2-pass with grain tuning; --qpfile and --csv support Unicode too.

[Jamaika]

I have a question. I use three codecs X265 1.9+183 different authors (WAW/Ligh /Selur) and received three different results in the binary HEX. (Veryslow/tune grain /hightier/pass2)
Is it possible?

[LigH]

A binary comparison of encoded HEVC video streams doesn't make much sense. I believe there is a chance that the output is not deterministic, but the differences may not be relevant for the image reconstruction. And if you have containers around the video stream, it is almost certain that there will be differences in some header areas, which are not at all related to the image reconstruction (e.g. "junk" chunks).