x265 HEVC Encoder

[Vesdaris]

After several tests I managed to approach the x264.
To do that I use medium profile I change ctu = 16 psy-rd = 1.00 rdoq-level = 2 psy-rdoq = 1.10.



What about CRF?

[LigH]

CRF limits the quality loss. The smaller rate factor, the less loss, the bigger the result. Find your own personal threshold of "visual transparency" (how much loss you find hardly noticable).

[luigizaninoni]

I also found adding --early-skip gave noticeably better performance without a quality or file size hit (or so fractionally slight the speed gain far outweighed any 'loss').

Basically --qg-size 16 is a must, and I believe --aq-mode 3 is as well :).

I tried your suggestions of early-skip and aq-mode 3.

I really like early-skip a lot, speed increase is incredible (about + 60%) and quality loss is minimal.

As far as aq-mode 3 is concerned, I don't know ... it increased bitrate by 25%, admittedly on a darkish clip. You probably could just bump down crf a notch or two and obtain similar results.

[Blowis]

what about crf?

I used CRF 21-22 or 23. For more visual detail is not the largest on the X265.

[Vesdaris]

OK..I've been testing for some time different settings encodign a small sample file. ..
x264 (4380 Kbps) crf18,10bit,very slow
cabac=1 / ref=4 / deblock=1:0:0 / analyse=0x3:0x133 / me=umh / subme=10 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=24 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=8 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=23 / scenecut=40 / intra_refresh=0 / rc_lookahead=60 / rc=crf / mbtree=1 / crf=18.0 / qcomp=0.60 / qpmin=0 / qpmax=81 / qpstep=4 / vbv_maxrate=150000 / vbv_bufsize=187500 / crf_max=0.0 / nal_hrd=none / filler=0 / ip_ratio=1.40 / aq=2:1.00

x265 (2933 Kbps) crf 20, 10bit, medium, tune=grain, rdoq-level=2, psy-rdoq=1.00
wpp / ctu=64 / min-cu-size=8 / max-tu-size=32 / tu-intra-depth=1 / tu-inter-depth=1 / me=1 / subme=2 / merange=57 / no-rect / no-amp / max-merge=2 / temporal-mvp / no-early-skip / rdpenalty=0 / no-tskip / no-tskip-fast / strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / no-fast-intra / no-open-gop / no-temporal-layers / interlace=0 / keyint=250 / min-keyint=23 / scenecut=40 / rc-lookahead=20 / lookahead-slices=6 / bframes=4 / bframe-bias=0 / b-adapt=2 / ref=3 / limit-refs=0 / limit-modes / weightp / no-weightb / aq-mode=1 / qg-size=32 / aq-strength=0.30 / cbqpoffs=0 / crqpoffs=0 / rd=3 / psy-rd=0.50 / rdoq-level=2 / psy-rdoq=1.00 / signhide / deblock=-2:-2 / sao / no-sao-non-deblock / b-pyramid / cutree / no-intra-refresh / rc=crf / crf=20.0 / qcomp=0.80 / qpmin=0 / qpmax=51 / qpstep=4 / ipratio=1.10 / pbratio=1.10


http://screenshotcomparison.com/comparison/153828

It's almost identical. x265 lacks a tiny bit of details here and there but it seems it's really close and encoding speed was nice.

I could prolly increase psy-rd a bit.or maybe change something else...What do you think?

[Blowis]

OK..I've been testing for some time different settings encodign a small sample file. ..
x264 (4380 Kbps) crf18,10bit,very slow


x265 (2933 Kbps) crf 20, 10bit, medium, tune=grain, rdoq-level=2, psy-rdoq=1.00



http://screenshotcomparison.com/comparison/153828

It's almost identical. x265 lacks a tiny bit of details here and there but it seems it's really close and encoding speed was nice.

I could prolly increase psy-rd a bit.or maybe change something else...What do you think?

Use ctu = 16 after increasing the psy-rd = 1.00 and = 1.10 psy-rdoq and also try aq-mode = 2. Especially increases psy-rd and psy-rdoq to sharpen the image
Here are but setting after several tries optimal and fast Profile Medium:

wpp / ctu=16 / min-cu-size=8 / max-tu-size=16 / tu-intra-depth=3 / tu-inter-depth=3 / me=3 / subme=2 / merange=32 / no-rect / no-amp / max-merge=2 / temporal-mvp / early-skip / rdpenalty=0 / no-tskip / no-tskip-fast / strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / no-fast-intra / open-gop / no-temporal-layers / interlace=0 / keyint=240 / min-keyint=23 / scenecut=40 / rc-lookahead=20 / lookahead-slices=5 / bframes=4 / bframe-bias=0 / b-adapt=2 / ref=3 / limit-refs=1 / no-limit-modes / weightp / weightb / aq-mode=2 / qg-size=16 / aq-strength=1.00 / cbqpoffs=0 / crqpoffs=0 / rd=3 / psy-rd=1.00 / rdoq-level=2 / psy-rdoq=1.10 / signhide / deblock / sao / no-sao-non-deblock / b-pyramid / cutree / no-intra-refresh / rc=crf / crf=22.0 / qcomp=0.60 / qpmin=0 / qpmax=51 / qpstep=4 / ipratio=1.40 / pbratio=1.30

On my i7 4774k@4.3Ghz i am between 12.25 to 14.45 fps according to the video.
For a video of 1 hour 40 1080p encoding lasts 3 hours 30.

[Vesdaris]

I'm getting insta crash if I use anything past this (ur settings)

--wpp --ctu=16 --min-cu-size=8 --max-tu-size=16 --tu-intra-depth=3 --tu-inter-depth=3 --me=3 --subme=2 --merange=32 --no-rect --no-amp --max-merge=2 --temporal-mvp --early-skip --rdpenalty=0 --no-tskip --no-tskip-fast --strong-intra-smoothing --no-lossless --no-cu-lossless --no-constrained-intra --no-fast-intra --open-gop --no-temporal-layers --interlace=0 --keyint=240 --min-keyint=23 --scenecut=40 --rc-lookahead=20 --lookahead-slices=5 --bframes=4 --bframe-bias=0 --b-adapt=2 --ref=3 --limit-refs=1

I guess I'm doing something wrong lol

[burfadel]

I tried your suggestions of early-skip and aq-mode 3.

I really like early-skip a lot, speed increase is incredible (about + 60%) and quality loss is minimal.

As far as aq-mode 3 is concerned, I don't know ... it increased bitrate by 25%, admittedly on a darkish clip. You probably could just bump down crf a notch or two and obtain similar results.

Yeah. The quality difference was imperceptible for me, it really only shows if doing a SSIM and PSNR comparison. You can make up and surpass the quality drop by using better settings elsewhere without losing too much of that gain. By using early skip, I also believe the speed penalty of some other settings is minimised, making them viable to use.

Why isn't. --b--intra on by default? What disadvantages are there using --me star seeing as it is so fast?

In any case, when all the settings I use are taken into account, for me at least it is the most suitable outcome speed and performance wise regardless of content. I do use higher b frames, lower aq with animation.

[Ma]

I'm getting insta crash if I use anything past this (ur settings)

--wpp --ctu=16 --min-cu-size=8 --max-tu-size=16 --tu-intra-depth=3 --tu-inter-depth=3 --me=3 --subme=2 --merange=32 --no-rect --no-amp --max-merge=2 --temporal-mvp --early-skip --rdpenalty=0 --no-tskip --no-tskip-fast --strong-intra-smoothing --no-lossless --no-cu-lossless --no-constrained-intra --no-fast-intra --open-gop --no-temporal-layers --interlace=0 --keyint=240 --min-keyint=23 --scenecut=40 --rc-lookahead=20 --lookahead-slices=5 --bframes=4 --bframe-bias=0 --b-adapt=2 --ref=3 --limit-refs=1

I guess I'm doing something wrong lol

I tried to reproduce this crash but I wasn't able (console output attached). Can you specify more details?

[Vesdaris]

I'm not using command line per se.. I'm using Hybrid or Megui where you can put custom commands. I guess they conflict with something.

[foxyshadis]

I'm not using command line per se.. I'm using Hybrid or Megui where you can put custom commands. I guess they conflict with something.

You can find the actual command-line in Megui's log.

[Ma]

I'm not using command line per se.. I'm using Hybrid or Megui where you can put custom commands. I guess they conflict with something.

Thanks for some details. It could be related to outdated x265 version.

Problem with command line that hangs x265 was reported in P.S. part of this message http://forum.doom9.org/showthread.php?p=1746309#post1746309

Fix for this bug is in https://bitbucket.org/multicoreware/x265/commits/fff19fb6cbf9f278725c90dde994dde4afadad6b

You can update x265. (I'm only guessing.)

[foxyshadis]

Use ctu = 16

It's very disappointing that x265 is still so bad at mode decision that restricting ctu to 16 is useful, since that eliminates one of the key advantages over AVC. But I tested it, and it really does make the picture sharper even if it slightly increases noise, and retains grain better. Maybe psy-rd also needs to directly correspond to a much stronger version of --rd-penalty, where the higher the psy-rd the higher the threshold for 64x64 or 32x32 ctu becomes, so that larger can still be used when warranted, and the value of psy-rd directly indicates how much grain/fine detail the user is interested in keeping.

[Blowis]

It's very disappointing that x265 is still so bad at mode decision that restricting ctu to 16 is useful, since that eliminates one of the key advantages over AVC. But I tested it, and it really does make the picture sharper even if it slightly increases noise, and retains grain better. Maybe psy-rd also needs to directly correspond to a much stronger version of --rd-penalty, where the higher the psy-rd the higher the threshold for 64x64 or 32x32 ctu becomes, so that larger can still be used when warranted, and the value of psy-rd directly indicates how much grain/fine detail the user is interested in keeping.


For me the ctu16 it is best to have the detail with the ctu64 so smooth it was a clear image but less detail.
Here are photos and encodings (for the ctu64 i put RD Penalty: 2 Psy-RD: 1.20 Psy-RDOQ: 1.30, if I put as the encoding of ctu16 smooth it over):

wpp / ctu=64 / min-cu-size=8 / max-tu-size=32 / tu-intra-depth=3 / tu-inter-depth=3 / me=3 / subme=2 / merange=32 / no-rect / no-amp / max-merge=2 / temporal-mvp / early-skip / rdpenalty=2 / no-tskip / no-tskip-fast / strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / no-fast-intra / open-gop / no-temporal-layers / interlace=0 / keyint=240 / min-keyint=23 / scenecut=40 / rc-lookahead=20 / lookahead-slices=5 / bframes=4 / bframe-bias=0 / b-adapt=2 / ref=3 / limit-refs=1 / no-limit-modes / weightp / weightb / aq-mode=2 / qg-size=32 / aq-strength=1.00 / cbqpoffs=0 / crqpoffs=0 / rd=3 / psy-rd=1.20 / rdoq-level=2 / psy-rdoq=1.30 / signhide / deblock / sao / no-sao-non-deblock / b-pyramid / cutree / no-intra-refresh / rc=crf / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=51 / qpstep=4 / ipratio=1.40 / pbratio=1.30

Link Image ctu64: http://hpics.li/686abe6

wpp / ctu=16 / min-cu-size=8 / max-tu-size=16 / tu-intra-depth=3 / tu-inter-depth=3 / me=3 / subme=2 / merange=32 / no-rect / no-amp / max-merge=2 / temporal-mvp / early-skip / rdpenalty=0 / no-tskip / no-tskip-fast / strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / no-fast-intra / open-gop / no-temporal-layers / interlace=0 / keyint=240 / min-keyint=23 / scenecut=40 / rc-lookahead=20 / lookahead-slices=5 / bframes=4 / bframe-bias=0 / b-adapt=2 / ref=3 / limit-refs=1 / no-limit-modes / weightp / weightb / aq-mode=2 / qg-size=16 / aq-strength=1.00 / cbqpoffs=0 / crqpoffs=0 / rd=3 / psy-rd=1.00 / rdoq-level=2 / psy-rdoq=1.10 / signhide / deblock / sao / no-sao-non-deblock / b-pyramid / cutree / no-intra-refresh / rc=crf / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=51 / qpstep=4 / ipratio=1.40 / pbratio=1.30

Link Image ctu16: http://hpics.li/93b5428

You see the differences on the teeth, lips, eyes on the ctu64 was a smoother face so clear but loses detail.
Here is the origin of the image x264: http://hpics.li/28b34fb

After a 80 inch TV i see no difference. For me the best is ctu16 all cases.

[undfeatable]

I can't speak to Handbreak specifically (it's more of a heartbreak for me whenever I've tried to us it for anything mildly interesting :)).

But there are a bunch of somewhat esoteric parameters that need to be set for a proper HDR-10 bitstream. Looking at a test file I did a few months ago:
--colorprim bt2020
--transfer 16
--colormatrix bt2020nc
--master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,0)"
--max-cll "1000,274"
--chromaloc 2
And, of course, you'll need to know what your master-display, MaxCLL, and MaxFALL values are for your content.

And you'll need to have a >=10-bit source that's already in the SMPTE 2084 PQ space, not gamma.

Also, I note you're targeting Level 5.1. AFAIK, there aren't any existing HEVC Main10 decoders that support beyond Level 5.1. Which can do 2160p60 :)!
I have those parameters set already, but really could use some help. Im going to post about it on the actual HDR post though to keep this thread on topic.
If the same binary works with SSE4.2 CPU and hangs with AVX2 CPU, you can add '--asm AVX' option or '--asm SSE4.2' option and retest.
Thanks! That solved it. I used --asm SSE4.2 and it now runs!

Question for the x265 crew though. Am I getting the full performance of my computer now or is it being hindered. I have a fully loaded 2015 MacBook pro w/ retina that has a 4.0GHz Intel i7 and 16gb of DDR3 ram. My practice clip is a 8 seconds long and 6GB uncompressed. It seems to take around 5-6 minutes to run through x265 on preset slow. Does that seem about right?

Side question, does everyone have to answer these questions before every post? Its frustrating to post and have to search around to answer a quest before a post!

[LigH]

You probably mean the anti-spam captchas for users with less than 5 posts?

[undfeatable]

You probably mean the anti-spam captchas for users with less than 5 posts?
Thats probably it lol. Its the most annoying "captcha" Ive ever encountered. Its just a random question and I never get it right the first 2 or 3 times. Well, this is post 5 so hopefully its gone for good.

[foxyshadis]

For me the ctu16 it is best to have the detail with the ctu64 so smooth it was a clear image but less detail.

After a 80 inch TV i see no difference. For me the best is ctu16 all cases.

There is definitely more annoying gibbs noise in the ctu16, but it's outweighed by the very annoying huge solid blocks with sharp sides that ctu64 has. With psy-rd 1.2 and rd-penalty 2 that should have been avoided as much as possible already.

My post was about how you shouldn't need to disable one of HEVC's greatest strengths just to get an acceptable encode, though. There has to be a way to fix the use of large blocks so that it works in harmony better.

[x265_Project]

... you shouldn't need to disable one of HEVC's greatest strengths just to get an acceptable encode, though. There has to be a way to fix the use of large blocks so that it works in harmony better.
Agree completely. The ability to code 32x32 and 64x64 blocks is perhaps HEVC's greatest asset. When can do this with low visual distortion, you get a big improvement in encoding efficiency. If you have visible distortion with large CUs, it could naturally be much more visible than with smaller CUs. We've got some ideas about how to improve our analysis to select the best PU and TUs, understanding that the distortion measurements used in x265 (and every HEVC encoder) are imperfect. It's just a matter of having the time to experiment and implement these ideas. Of course, as always, suggestions and contributions are welcomed.

[littlepox]

We've got some ideas about how to improve our analysis to select the best PU and TUs, understanding that the distortion measurements used in x265 (and every HEVC encoder) are imperfect. It's just a matter of having the time to experiment and implement these ideas.

Your efforts are always appreciated; we await the improvements.

[LigH]

Current state:

x265 1.8+167-e951ab673b1c (GCC 4.9.2) (https://www.mediafire.com/download/kkkz0d9mdvzjc3d/x265_1.8+167-e951ab673b1c.GCC492.7z)
x265 1.8+167-e951ab673b1c (GCC 5.2.0) (https://www.mediafire.com/download/4mps11mzk9x72ks/x265_1.8+167-e951ab673b1c.GCC520.7z)

New documented 'K' frametype ("keyframe", exact type depending on circumstances), several speedups and fixes (mostly in sao).

[microchip8]

Any news? (on the ctu & presets improvements & QoL)?

Cheers.

Impatient much?

[burfadel]

Any news? (on the ctu & presets improvements & QoL)?

Cheers.

A watched pot never boils :rolleyes:

[microchip8]

@froggy1: Retarded much?

This is going to get you nowhere ;)

Please don't ask when it's done. It'll be done when it's done

[Blowis]

I wanted to ask is it normal that me=star is faster than umh.
I tested two videos:

1a) Profil medium me= umh : 18.61 fps PSNR : 45.114 SSIM :0.971590
1b) Profil medium me= star : 19.14 fps PSNR : 45.113 SSIM : 0.971609

2a) Profil medium me= umh : 13.23 fps PSNR : 46.188 SSIM : 0.967780
2b) Profil medium me = star : 13.65 fps PSNR : 46.191 SSIM: 0.967778

On a video 2mn subme:star is 7s faster.

Visually I see no difference.

This is different from the x264 ?

[x265_Project]

I wanted to ask is it normal that me=star is faster than umh.
I tested two videos:...
Interesting, but the basic tradeoff with speed is not quality, it's efficiency (quality @ bit rate). What were the bit rates?

[Blowis]

Interesting, but the basic tradeoff with speed is not quality, it's efficiency (quality @ bit rate). What were the bit rates?


I use CRF:23 which gives a bitrate 1 724 Kbps and 2 597 Kbps.

After I tested rdpenalty:2 I lost 19% and 29% encoding time.
Visually minimal difference.

[x265_Project]

I use CRF:23 which gives a bitrate 1 724 Kbps and 2 597 Kbps.

I tested rdpenalty:2 I lost between 19% to 29% encoding time.
Visually minimal difference.

I'm confused. Can you put all the relevant information in a simple table?

command line options changed from default preset
bit rate
Global PSNR
SSIM
FPS

It would also help to know the details of your input video sequence (is it available, what is the pixel resolution, frame rate, and does it have low, medium or high motion)?

Lastly, it would help to know your computer system details (processor, memory, OS).

[Blowis]

I'm confused. Can you put all the relevant information in a simple table?

command line options changed from default preset
bit rate
Global PSNR
SSIM
FPS

It would also help to know the details of your input video sequence (is it available, what is the pixel resolution, frame rate, and does it have low, medium or high motion)?

Lastly, it would help to know your computer system details (processor, memory, OS).

I use StaxRip.
Video 1a Profil Medium and:
wpp / ctu=64 / min-cu-size=8 / max-tu-size=32 / tu-intra-depth=3 / tu-inter-depth=3 / me=2 / subme=2 / merange=32 / no-rect / no-amp / max-merge=2 / temporal-mvp / early-skip / rdpenalty=0 / no-tskip / no-tskip-fast / strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / no-fast-intra / open-gop / no-temporal-layers / interlace=0 / keyint=240 / min-keyint=23 / scenecut=40 / rc-lookahead=20 / lookahead-slices=5 / bframes=4 / bframe-bias=0 / b-adapt=2 / ref=3 / limit-refs=0 / no-limit-modes / weightp / weightb / aq-mode=2 / qg-size=32 / aq-strength=1.00 / cbqpoffs=0 / crqpoffs=0 / rd=3 / psy-rd=1.00 / rdoq-level=2 / psy-rdoq=1.10 / signhide / deblock / sao / no-sao-non-deblock / b-pyramid / cutree / no-intra-refresh / rc=crf / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=51 / qpstep=4 / ipratio=1.40 / pbratio=1.30
Video 1b unless such Me=3

The same for video 2a and 2b.

My Cpu I7 4770K@4.3Ghz DDR 8Go@2.4Ghz SSD 512go Samsung 840Pro Windows 10

[Motenai Yoda]

what x265_Project is trying to say is that you'll get the same quality but different bitrate, for the same bitrate you'll get different quality.

[LigH]

@ Blowis

command line options changed from default preset

That means the command line in your encoder call, not the whole set of internal parameters stored in the auxiliary data in the video stream (that's simply too much to recognize at one glance).

[x265_Project]

A patch with new performance presets was pushed today. We look forward to your feedback.

[Motenai Yoda]

The table shows 2 row for limit-refs, but I'm pretty sure the first was for limit-modes, and limit-modes there aren't at all in the code too, only defaulted at 0.

Anyway I didn't get how to enable the 2 pass mode for cfr+vbv, if I have to.

[x265_Project]

The table shows 2 row for limit-refs, but I'm pretty sure the first was for limit-modes, and limit-modes there aren't at all in the code too, only defaulted at 0.

Anyway I didn't get how to enable the 2 pass mode for cfr+vbv, if I have to.
Thanks for the feedback. We'll get this cleaned up before the patch is committed (tomorrow).

[LigH]

A patch with new performance presets was pushed today.

Just when I am away from my usual building PC ... well, I will try to publish one as soon as I rebuilt my compiling environments elsewhere.

P.S.:

... before the patch is committed (tomorrow).

Oh, then I have a little time, ok.

[LigH]

Merry Christmas (or similar holidays) with a current build with updated preset defaults (e.g. limit-modes/refs; please RTFM).

x265 1.8+187-da48f2690076 (GCC 4.9.2) (https://www.mediafire.com/download/tvvkyqf5md6m0ss/x265_1.8+187-da48f2690076.GCC492.7z)
x265 1.8+187-da48f2690076 (GCC 5.2.0) (https://www.mediafire.com/download/o5qyzouion38e69/x265_1.8+187-da48f2690076.GCC520.7z)

[Ajvar]

Hey guys, I've been away from my horse-powere laptop and so from encoding videos. Can somebody write a short-list of what drastically changed since 1.7.1? In efficiency/speed. Thanks.

[Selur]

"what drastically changed", the presets changed, pme and pmode were added but those only seem to be only really useful on systems 4+ core systems, all in all I wouldn't call any changed 'drastic'

[Blowis]

Hi,
I just tested the last days made. I chose the Medium profile I modified all its: --crf 22 --early-skip --aq-mode 2 --me star --merange 44 --rc-lookahead 60 --keyint 240 --psy-rd 1.3.

wpp / ctu=64 / min-cu-size=8 / max-tu-size=32 / tu-intra-depth=1 / tu-inter-depth=1 / me=3 / subme=2 / merange=44 / no-rect / no-amp / max-merge=2 / temporal-mvp / early-skip / rdpenalty=0 / no-tskip / no-tskip-fast / strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / no-fast-intra / open-gop / no-temporal-layers / interlace=0 / keyint=240 / min-keyint=23 / scenecut=40 / rc-lookahead=60 / lookahead-slices=5 / bframes=4 / bframe-bias=0 / b-adapt=2 / ref=3 / limit-refs=3 / no-limit-modes / weightp / weightb / aq-mode=2 / qg-size=32 / aq-strength=1.00 / cbqpoffs=0 / crqpoffs=0 / rd=3 / psy-rd=1.30 / rdoq-level=0 / psy-rdoq=0.00 / signhide / deblock / sao / no-sao-non-deblock / b-pyramid / cutree / no-intra-refresh / rc=crf / crf=21.0 / qcomp=0.60 / qpmin=0 / qpmax=51 / qpstep=4 / ipratio=1.40 / pbratio=1.30


But I see --Limit-modes : 3 that normally there should be more than 0. You can set it to 0 or must be done manually.

[foxyshadis]

--limit-modes has no argument, it's either on or off. --limit-refs is 0-3.

Changes since 1.8: Modified presets, most are faster, higher quality, or both. Limit-modes. Intra refresh. 2-pass CRF. Monochrome (i400). Big psy-rd changes, possibly improvements. Lots of 12-bit speedups & bug fixes, and some other misc speedups and fixes for 8 & 10. Analysis save/load fixes. Level 8.5 (unlimited) allowed. i64x64 disabled until it's fixed.

See the 1.8 announcement (https://forum.doom9.org/showpost.php?p=1742032&postcount=2754) for changes since 1.7.

[x265_Project]

To see the changes, see this commit... https://bitbucket.org/multicoreware/x265/commits/da48f2690076bc1bc72b1cbf62347e40e30debce


increased the number of reference frames for slower, slow, fast, faster and veryfast
turned on --limit-refs for veryslow, slower, slow, medium, fast, faster, and veryfast
turned on --limit-modes for veryslow, slower and slow
turned on --cutree for faster, veryfast, superfast and ultrafast
increased max CTU size to --ctu 64 for veryfast

[Jamaika]

Updating presets for x265 v1.8.0.188 to improve coding efficiency and speed. | |ultrafast |superfast |veryfast |faster |fast |medium |slow |slower |veryslow |placebo |
| ctu | 32 | 32 | 64 | 64 | 64 | 64 | 64 | 64 | 64 | 64 |
| ref | 1 | 1 | 2 | 2 | 3 | 3 | 4 | 4 | 5 | 5 |
| cuTree | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| limit-refs | 0 | 0 | 3 | 3 | 3 | 3 | 3 | 2 | 1 | 0 |
| limit-modes | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 1 | 0 |

Who knows why the placebo preset is disabled features "limit-refs/modes"? Could they not have been perfect?
When you expect the implementation of the "psy-rdoq = 50.0" for veryslow?
Who can explain to me what are the features "sao, amp, temporal-MVP, signhide, strong-intra-smoothin"? If you don't apply them for quick shots even playing the movie better presented.

[nevcairiel]

Who knows why the placebo preset is disabled features "limit-refs/modes"? Couldn't they have been perfect?

Placebo is meant to be the ultimate preset with 100% of all quality options turned on, irregardless of speed. The two limit options can slightly cost efficiency to gain substantial speed improvements, which is not something you want in placebo.

[Boulder]

To see the changes, see this commit... https://bitbucket.org/multicoreware/x265/commits/da48f2690076bc1bc72b1cbf62347e40e30debce


increased the number of reference frames for slower, slow, fast, faster and veryfast
turned on --limit-refs for veryslow, slower, slow, medium, fast, faster, and veryfast
turned on --limit-modes for veryslow, slower and slow
turned on --cutree for faster, veryfast, superfast and ultrafast
increased max CTU size to --ctu 64 for veryfast
After these changes, what it the recommended range for psy-rd and psy-rdoq if detail/noise/etc. retention is the goal? I've got one particularly difficult movie to encode (The Big Red One), with x264 and CRF18 it's ending up at ~9MBps at 1280x720 even with some denoising. I'd like to test to see what x265 can come up with.

[Jamaika]

Placebo is meant to be the ultimate preset with 100% of all quality options turned on, irregardless of speed. The two limit options can slightly cost efficiency to gain substantial speed improvements, which is not something you want in placebo.
Maybe it's the truth. For which CPUs was executed test presets. I use for veryslow "limit-ref = 3" because faster converts video.

PS Why does X265 v1.8.0.188 not retain the bitrate?
x265 x1.8.0.86 --bitrate 4500 --vbv-bufsize 10000 --vbv-maxrate 10000
Bit rate : 4 595 Kbps
Maximum bit rate : 6 030 Kbps
x265 x1.8.0.188 --bitrate 4500 --vbv-bufsize 10000 --vbv-maxrate 10000
Bit rate : 3 680 Kbps
Maximum bit rate : 4 923 Kbps

[Atak_Snajpera]

What do you think about increasing default --psy-rd value from very low 0.3 to 2?
According to my tests if you want to achieve similar level of details with x264 this must be increased in first place. Default 0.3 value can produce nasty banding effects in dark areas. (see jaw)

Source
http://i.cubeupload.com/AhAG58.png

x264 --veryslow --2pass --bitrate 2048
http://i.cubeupload.com/rKB3Sq.png

x265 --medium --2pass --bitrate 2048
http://i.cubeupload.com/FzFWFZ.png

x265 --medium --2pass --bitrate 2048 --psy-rd 2
http://i.cubeupload.com/DKDrIR.png

x265 --medium --2pass --bitrate 2048 --psy-rd 2 --aq-strength 2
http://i.cubeupload.com/nLhSxJ.png


I've also uploaded whole 10 min samples if you want to check how other scenes look like. (Use staxrip's video comparison tool)
https://mega.nz/#!pJFiHRaS!zxYtdhj4Obvp7CMQTMmwnXsmrdpqD6Ia8n6HIHYGDwo

[x265_Project]

Placebo is meant to be the ultimate preset with 100% of all quality options turned on, irregardless of speed.
That's generally the idea, although there are more things we could do to sacrifice performance for that last little bit of possible quality. We wouldn't want to slow x265 to the point where it is absolutely unusable under any reasonable circumstances. Some extreme quality-centric x265 users might be ok waiting a couple of days for their encode to finish, but not for weeks or months.

--me 4 slows performance by 13X, and I couldn't see a noticeable improvement in quality

--subme 7 slows performance by about 9%, and visually you may see a difference in quality, so we may still consider adding this to placebo

--merange 121 slowed performance by 13%, but I didn't really see a difference in the output bitstream

Of course, you're free to tweak any parameters you would like in an attempt to improve speed or quality. For example, you could also try --preset placebo with --limit-refs and/or --limit-modes, and it would run much faster.

[x265_Project]

What do you think about increasing default --psy-rd value from very low 0.3 to 2?
According to my tests if you want to achieve similar level of details with x264 this must be increased in first place. Default 0.3 value can produce nasty banding effects in dark areas. (see jaw)

Source
http://i.cubeupload.com/AhAG58.png

x264 --veryslow --2pass --bitrate 2048
http://i.cubeupload.com/rKB3Sq.png

x265 --medium --2pass --bitrate 2048
http://i.cubeupload.com/FzFWFZ.png

x265 --medium --2pass --bitrate 2048 --psy-rd 2
http://i.cubeupload.com/DKDrIR.png

x265 --medium --2pass --bitrate 2048 --psy-rd 2 --aq-strength 2
http://i.cubeupload.com/nLhSxJ.png


I've also uploaded whole 10 min samples if you want to check how other scenes look like. (Use staxrip's video comparison tool)
https://mega.nz/#!pJFiHRaS!zxYtdhj4Obvp7CMQTMmwnXsmrdpqD6Ia8n6HIHYGDwo
I think that psy-rd has been improved to the point where it can probably stand a higher default value, and possibly a higher upper limit. We plan to take another look at this.

[Jamaika]

I don't know why there are such differences in image quality in my results. The file size of CRF=37:qcomp=1.00 x264 is three times higher than for CRF=37:qcomp=1.00 X265. Is this correct result? I will present your findings to the X265 CRF=37. The two supposedly identical commands for X265 and two different results from the original.

Test 1
ffmpeg.exe -loglevel info -i "swimming.avi" -s 1920x1080 -r 50000/1000 -an -sn -f image2 -c:v png -pix_fmt rgba "swimming%%003d.png"
bpgenc.exe -v -a -e x265 -c rgb -q 37 -alphaq 37 -m 9 -fps 50 "swimming%%003d.png" -o "swimming_rgba.bpg"
Edit: bpgenc.exe -v -a -e x265 -b 8 -f 444 -c rgb -q 37 -m 9 -fps 50 (only 25) "swimming%%003d.png" -o "swimming_rgb.bpg"
Edit: bpgenc.exe -v -a -e x265 -b 8 -f 444 -c ycbcr -q 37 -m 9 -fps 50 (only 25) "swimming%%003d.png" -o "swimming_yuv444.bpg"
Using x265 preset: placebo
x265 [info]: HEVC encoder version unknown
x265 [info]: build info [Windows][GCC 4.9.2][64 bit] 8bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x265 [info]: Main 4:4:4 profile, Level-4 (Main tier)
x265 [info]: Thread pool created using 4 threads
x265 [info]: frame threads / pool features : 2 / wpp(17 rows)
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 4 inter / 4 intra
x265 [info]: ME / range / subpel / merge : star / 92 / 5 / 5
x265 [info]: Keyframe min / max / scenecut : 25 / 250 / 40
x265 [info]: Lookahead / bframes / badapt : 60 / 0 / 2
x265 [info]: b-pyramid / weightp / weightb : 0 / 1 / 1
x265 [info]: References / ref-limit cu / depth : 1 / 0 / 0
x265 [info]: Rate Control : CQP-37
x265 [info]: tools: rect amp rd=6 rdoq=2 tskip signhide tmvp b-intra
x265 [info]: tools: strong-intra-smoothing deblock sao
Test 2
ffmpeg.exe -loglevel info -y -i "swimming.avi" -s 1920x1080 -r 50000/1000 -an -sn -f yuv4mpegpipe -strict experimental -pix_fmt yuv444p - |
x265.exe --y4m --input-csp i444 --preset placebo --high-tier --crf 37 --ref 1 --bframes 0 --qcomp 1.00 --no-open-gop --no-psy-rd --no-psy-rdoq --no-info --limit-modes --limit-refs 3 --range full - -o "swimming.h265"

y4m [info]: 1920x1080 fps 50/1 i444p8 unknown frame count
raw [info]: output file: swimming.h265
x265 [info]: HEVC encoder version 1.8+188-
x265 [info]: build info [Windows][GCC 5.2.0][64 bit] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x265 [info]: Main 4:4:4 profile, Level-4.1 (Main tier)
x265 [info]: Thread pool created using 4 threads
x265 [info]: frame threads / pool features : 2 / wpp(17 rows)
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 4 inter / 4 intra
x265 [info]: ME / range / subpel / merge : star / 92 / 5 / 5
x265 [info]: Keyframe min / max / scenecut : 25 / 250 / 40
x265 [info]: Lookahead / bframes / badapt : 60 / 0 / 2
x265 [info]: b-pyramid / weightp / weightb : 0 / 1 / 1
x265 [info]: References / ref-limit cu / depth : 1 / 1 / 1
x265 [info]: AQ: mode / str / qg-size / cu-tree : 1 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress : CRF-37.0 / 1.00
x265 [info]: tools: rect amp limit-modes rd=6 rdoq=2 tskip signhide tmvp
x265 [info]: tools: b-intra strong-intra-smoothing deblock sao
View single frames I/P:
http://i66.tinypic.com/v5zxc1.png
BPG x265 v0.9.6
http://i65.tinypic.com/2i436u.png
x265 v1.8.0.188
http://i68.tinypic.com/2gy63jn.png
Summary: Codec X265 with the same parameters fell much worse. The colors and sharpness are not the same. The cause is unknown.

[vivan]

The file size of CRF=37:qcomp=1.00 x264 is three times higher than for CRF=37:qcomp=1.00 X265. Is this correct result?Why it shouldn't? Those are completely different encoders.

Summary: Codec X265 with the same parameters fell much worse. The colors and sharpness are not the same. The cause is unknown.Unknown? How about 3x lower bitrate?

[sirt]

Hi,

Well I really don't know almost anything about HEVC ! I did some random search but I can't find anything clear...is it possible to encode in command line mode ? Can I recode a .264, .mkv, .avs script ?

For example, with a .264, I tried that :


x265.exe --preset placebo --sar 1:1 --profile main --level 5.1 --me 4 --subme 7 --merange 32 --bitrate 10000 --pass 1 -width 1920 --height 1080 -fps 24 rob.h264 -o rob_pass1.mkv
x265.exe --preset placebo --sar 1:1 --profile main --level 5.1 --me 4 --subme 7 --merange 32 --bitrate 10000 --pass 2 -width 1920 --height 1080 -fps 24 rob.h264 -o rob_pass2.mkv
pause


I don't know what happens, probably nothing (I get something similar to --help for example).

I tried something like that one year ago and it seems it is still the same : it is requiered to set widt, height, fps and it doesn't work (apparently). Still can't something else than .yuv be encoded ?

[Boulder]

x265 works just like x264 what comes to command line encoding.

In your example, the fps parameter needs two dashes so it should be --fps 24. But you don't need the parameter if the source and destination fps is the same. Also there's no width or height parameter, at least according to https://x265.readthedocs.org/en/default/index.html.

[sirt]

x265 works just like x264 what comes to command line encoding.

In your example, the fps parameter needs two dashes so it should be --fps 24. But you don't need the parameter if the source and destination fps is the same. Also there's no width or height parameter, at least according to https://x265.readthedocs.org/en/default/index.html.

Thanks Boulder : if I set width, height, fps with "--", same it doesn't work : x265 help comes.

If I don't set anyone :


x265 [error] : yuv : width, height, and FPS must be specified
x265 [error] : unable to open input file <rob.64>


Well I never managed to encode anything with that and it is NOT a yuv video. Just a .264 demuxed from an mkv.

[Boulder]

The parameter for width and height is "input-res"

https://x265.readthedocs.org/en/default/cli.html?highlight=input#cmdoption--input-res

[sirt]

The parameter for width and height is "input-res"

https://x265.readthedocs.org/en/default/cli.html?highlight=input#cmdoption--input-res

Indeed, thanks, something has started with that :


x265.exe --preset placebo --sar 1:1 --profile main --level 5.1 --me 4 --subme 7 --merange 32 --bitrate 10000 --pass 1 --input-res 1920x1080 --fps 24 rob.h264 -o rob_pass1.mkv
x265.exe --preset placebo --sar 1:1 --profile main --level 5.1 --me 4 --subme 7 --merange 32 --bitrate 10000 --pass 2 --input-res 1920x1080 --fps 24 rob.h264 -o rob_pass2.mkv

pause

[LigH]

In general, x265 won't be able to read raw AVC (*.h264), and won't create MKV either. It won't contain libavcodec decoders and multiplexers (except you have a special patched build). Pure x265 builds will only read raw YUV (planar 4:2:0) or Y4M (YUV4MPEG, that's basically raw YUV with a very basic text line as header describing resolution, frame rate, and pixel format), not even AviSynth scripts are supported yet. Using a raw AVC bitstream as input may result in encoded noise because x265 will read it as if it were raw YUV.

To feed an AviSynth script decoding an encoded source format into x265, you can use e.g. avs4x26x. But it will still create only an elementary HEVC stream (except it's compiled with additional multiplexers like L-SMASH MP4). You will usually multiplex it afterwards into an MP4 using MP4Box or L-SMASH, or into an MKV using mkvmerge.

Furthermore, do yourself (and your electricity bill) a favour and don't assume "--preset placebo" to be a sensible setup for x256. It's even more a waste of time than you may know from x264.

[benwaggoner]

--merange 121 slowed performance by 13%, but I didn't really see a difference in the output bitstream.
Did you test that with a variety of frame sizes? I would think you'd be more likely to see an improvement with higher --merange at 4K?

Still, it's not surprising; anything moving fast enough to need that much motion search would be so distorted by motion blur that a good match would be hard to find, and the remaining frequencies would be so low as to be easy to encode. It might take something without motion blur like 4K cel animation to really find good matches at the largest search ranges. I don't know if anyone has ever actually made any 4K cel animation to test that with, though.

[sirt]

In general, x265 won't be able to read raw AVC (*.h264), and won't create MKV either. It won't contain libavcodec decoders and multiplexers (except you have a special patched build). Pure x265 builds will only read raw YUV (planar 4:2:0) or Y4M (YUV4MPEG, that's basically raw YUV with a very basic text line as header describing resolution, frame rate, and pixel format), not even AviSynth scripts are supported yet. Using a raw AVC bitstream as input may result in encoded noise because x265 will read it as if it were raw YUV.

To feed an AviSynth script decoding an encoded source format into x265, you can use e.g. avs4x26x. But it will still create only an elementary HEVC stream (except it's compiled with additional multiplexers like L-SMASH MP4). You will usually multiplex it afterwards into an MP4 using MP4Box or L-SMASH, or into an MKV using mkvmerge.

Furthermore, do yourself (and your electricity bill) a favour and don't assume "--preset placebo" to be a sensible setup for x256. It's even more a waste of time than you may know from x264.

Hi,

Thanks for your answer. I was not planning to encode that anyway, I just gave a try to an old command line. From what you say, I understand it is not possible to encode anything properly with x265.

What am I supposed to do then ? I extracted some of my BluRay of which video streams are AVC or VC-1. Imagine I want to reenode corresponding extracted .264/.h264 stream in HEVC. How can I proceed ? What strikes me is that I read a couple of things about Handbrake and similar *1-click* solutions. It seems you can really encode BluRay video stream with that (using x265). In my case, I don't want to use such things but rather command lines. Unfortunately, from what you say, it seems doing what I want is not possible (you talk about "encoded noise" (?)). I don't even talk about avisynth scripts.

[LigH]

No, at the moment, you won't be able to simply convert anything else than uncompressed YUV video with the single x265 encoder. But you may use ffmpeg including libx265 to convert a compressed source format, as it is able to decode it before feeding the decompressed result to the x265 encoder library inside ffmpeg, and even multiplex the result into a container. The command line with parameters will then be a little more complex. And it scratches the borders of this thread's topic...

The alternative may be to create an AviSynth script and pipe its output into the x265 encoder. Tools like MeGUI or StaxRip may help you doing so. Handbrake or Hybrid may be useful to support encoding with ffmpeg, I believe.

[Ma]

x265.exe --preset placebo --sar 1:1 --profile main --level 5.1 --me 4 --subme 7 --merange 32 --bitrate 10000 --pass 1 --input-res 1920x1080 --fps 24 rob.h264 -o rob_pass1.mkv
x265.exe --preset placebo --sar 1:1 --profile main --level 5.1 --me 4 --subme 7 --merange 32 --bitrate 10000 --pass 2 --input-res 1920x1080 --fps 24 rob.h264 -o rob_pass2.mkv

pause


It will not work.
You can start with this:

ffmpeg -i rob.h264 -loglevel warning -f yuv4mpegpipe - | x265 --y4m - --preset fast --sar 1:1 --profile main --level 5.1 --subme 7 --merange 32 --bitrate 10000 --pass 1 -o NUL
ffmpeg -i rob.h264 -loglevel warning -f yuv4mpegpipe - | x265 --y4m - --preset fast --sar 1:1 --profile main --level 5.1 --subme 7 --merange 32 --bitrate 10000 --pass 2 -o rob_pass2.hevc
mkvmerge rob_pass2.hevc -o rob_pass2.mkv


If you succeed you can change '--preset fast' to '--preset placebo' (and you should forget about '--me 4').

[Jamaika]

Unknown? How about 3x lower bitrate?
You can change HEVC data losslessly and freely from the container BPG to MP4.
https://www.sendspace.com/filegroup/sWTMhCdLGFMoNz6blH0K%2FqPYg50Alend

Elecard StreamEye4 info files BPG & X265:
http://i64.tinypic.com/2djp0f9.pnghttp://i63.tinypic.com/28030gi.png
Size and bitrate files:
swimming_x265_yuv444.mp4 10420204 | 7550 Kbps
swimming_yuv444.bpg 13212111 | 9572 Kbps
swimming_rgb.bpg 17994215 | 13037 Kbps
swimming_rgba.bpg 18040885 | 13037 Kbps

Bitrate bpg for yuv444 is 1.25x higher. However, I am not convinced that this significantly affect the appearance of the film.

Edit:
Inactive functions or don't know the application:
x265 --crf-max 37 --crf-min 34 --level-idc 4.0

[LigH]

@ Ma + sirt:

I would still not recommend any preset slower than "slower" if you want to live to see the result.

Again, and again, and again: "placebo" is not a practically useful preset, it is "the bad example" of all the slowest possible options, some won't even provide better quality despite wasting more time.

[Ma]

I would still not recommend any preset slower than "slower" if you want to live to see the result.

I partially agree. For hi-res encoding '--preset placebo' is slow, but '--preset veryslow' is quite interesting.

You can see how big is the difference in encoding quality between '--preset slower --crf 15' and '--preset veryslow --crf 15' with small sample (only 177 frames)
http://www.msystem.waw.pl/x265/uhd177.y4m

Look for smooth movement of objects.

[ndkamal]

I do some tests with two version of x265 on my laptop with a Core I5-4200, the V18R186 and the V18R188, and the speed increase is very good, with an increase of 71 % on a video with resolution of 1920x1038.

Very good progress. Moreover the quality of picture is a bit better.

x265 V18R186 preset slow :
Neou (672x272) (1.4 Mbps) : 32.21/8.31
24 (672x368) (1.4Mbps) : 31.11/7.44
Alliance (1280x544) (3 Mbps) : 13.08/3.75
Prete (1920x1038) (6 Mbps) : 4.16/1.06

x265 V18R188 preset slow :
Neou (672x272) (1.4 Mbps) : 33.36/13.79
24 (672x368) (1.4 Mbps) : 32.02/12.44
Alliance (1280x544) (3 Mbps) : 13.56/5.43
Prete (1920x1038) (6 Mbps) : 4.45/1.82

Speed increase :
Neou: 65 %
24 : 67 %
Alliance : 45 %
Prete : 71 %

[Boulder]

I partially agree. For hi-res encoding '--preset placebo' is slow, but '--preset veryslow' is quite interesting.It would be interesting to see a comprehensive comparison between medium - slow - slower - veryslow. Veryslow tends to take a very long time even with 720p sources and --limit-modes + --limit-refs 3.

[Nullack]

Hi, Im learning from the "read the docs" and experimenting. SOme Q's pls:

1. Do I assume right that -CRF 0 is not lossless, only -lossless is? -CRF 0 encodes are smaller than -lossless encodes with the same input.
2. Is two pass CRF worthwhile?
3. ffmpeg muxing into mp4 is saying the content is variable framerate in mediainfo. I see there is no constant frame rate option like in x264? So is it all constant frame rate but mediainfo isnt reporting it correctly?

Thanks

[sirt]

It will not work.
You can start with this:

ffmpeg -i rob.h264 -loglevel warning -f yuv4mpegpipe - | x265 --y4m - --preset fast --sar 1:1 --profile main --level 5.1 --subme 7 --merange 32 --bitrate 10000 --pass 1 -o NUL
ffmpeg -i rob.h264 -loglevel warning -f yuv4mpegpipe - | x265 --y4m - --preset fast --sar 1:1 --profile main --level 5.1 --subme 7 --merange 32 --bitrate 10000 --pass 2 -o rob_pass2.hevc
mkvmerge rob_pass2.hevc -o rob_pass2.mkv


If you succeed you can change '--preset fast' to '--preset placebo' (and you should forget about '--me 4').

Ma, thanks. Then what would be the command line for an avisynth script (.avs) ?

[shinchiro]

Veryslow tends to take a very long time even with 720p sources and --limit-modes + --limit-refs 3.

Disabling these setting when using preset veryslow will cut down encoding time by half probably.
--no-rect --no-amp --no-b-intra --no-weightb --no-strong-intra-smoothing --no-sao
The quality probably negligible to notice compare to default veryslow settings.

[rack04]

Ma, thanks. Then what would be the command line for an avisynth script (.avs) ?

Change "rob.h264" to "rob.avs"

[LigH]

1. Do I assume right that -CRF 0 is not lossless, only -lossless is? -CRF 0 encodes are smaller than -lossless encodes with the same input.

CRF will in general not have a constant quantization, it tries to find the quantizer where the loss of quality is below a threshold related to the "rate factor". Only a forced quantization mode can guarantee lossless encoding; CRF will not guarantee that.

The parameter --lossless will enforce lossless encoding by a special mode bypassing several encoding steps. It seems to do even more than you would expect from a parameter -q 0 for constant quantization; both parameters can only be found in the "full help" of the CLI encoder (--log-level full --help) or in the online documentation.

The parameter --cu-lossless does not mean enforcing lossless encoding. It only allows to consider lossless encoding per coding unit.

2. Is two pass CRF worthwhile?

"2-pass CRF" is not really 2-pass in the meaning of running two separate passes; it is rather a per-GOP refinement to ensure that CRF encoding will respect VBV constraints. It is indeed useful for a fast yet rather optimal encoding where you have no specific target size, VBV constraints are most important, and the bitrate may only vary according to the rate factor where you have enough overhead due to a not completely filled decoding buffer.

3. ffmpeg muxing into mp4 is saying the content is variable framerate in mediainfo. I see there is no constant frame rate option like in x264? So is it all constant frame rate but mediainfo isnt reporting it correctly?

MediaInfo already reports VFR at the slightest hint that a container supports a duration per frame. It won't check if all frames have the same duration. Just the fact "the container may support VFR" seems to be relevant enough for MediaInfo.

[sirt]

With an .avi (lossless Lagarith), example :


ffmpeg -i rob.avi -loglevel warning -f yuv4mpegpipe - | x265 --y4m - --preset slower --sar 1:1 --profile high --level 4.1 --subme 7 --merange 32 --bitrate 5000 --pass 1 -o NUL
ffmpeg -i rob.avi -loglevel warning -f yuv4mpegpipe - | x265 --y4m - --preset slower --sar 1:1 --profile high --level 4.1 --subme 7 --merange 32 --bitrate 5000 --pass 2 -o rob_pass2.hevc

pause


I get an error or perhaps a warning


[yuv4mepgpip@054a3d00] Encoder did not produce proper pts, making some up.


Then, it seems it starts (I will let it go and see if it works until the end). Is that important ?

Moreover, am I really forced to use ffmpeg to encode in command line ? I don't understand why we use ffpmeg and a pipe (perhaps, because we don't use yuv and we want to force x265 encode a non yuv stream).

[LigH]

"No proper PTS" is not a serious warning. Not all containers support very detailed per-frame values and assume a contiguous progression of a default value.

[Nullack]

Thanks allot Ligh

[benwaggoner]

You can change HEVC data losslessly and freely from the container BPG to MP4.
Bitrate bpg for yuv444 is 1.25x higher. However, I am not convinced that this significantly affect the appearance of the film.
You could probably increase the cbqpoffs/crqpoffs values some when encoding to 4:4:4 to get back some of the bitrate v. 4:2:0 without visual loss in quality. With 4x the samples, you can almost certainly afford some higher quantization.

[Jamaika]

Thanks for the answer. It's good that there are such functions. Where to begin? Is HEVC analyzer which can the values display of these functions cbqpoffs/crqpoffs?
I see that the values can be from -12 to 12.

[sirt]

Just for your information, I tried above command lines and my (old ?) computer simply shut down after pass 1 started, even with preset "fast". I'm encoding a sample of 4 minutes. It has never happened with x264 before though (I encoded same sample with slow presets, in command line and without ffmpeg). I have no idea what happens as there is no log or error message. Actually I have a problem with my ventilator unit : each time a lot of CPU is requested, this ventilator heats (I hear a lot of noise). It is just disturbing but nothing more. I guess it is due to x265 (I am on a 32 bit Windows vista engine).

[foxyshadis]

Just for your information, I tried above command lines and my (old ?) computer simply shut down after pass 1 started, even with preset "fast". I'm encoding a sample of 4 minutes. It has never happened with x264 before though (I encoded same sample with slow presets, in command line and without ffmpeg). I have no idea what happens as there is no log or error message. Actually I have a problem with my ventilator unit : each time a lot of CPU is requested, this ventilator heats (I hear a lot of noise). It is just disturbing but nothing more. I guess it is due to x265 (I am on a 32 bit Windows vista engine).

Hardware stability problems aren't x265's problem. While there's a hardware subforum here, you'd probably be better off asking at an overclocking enthusiast forum to track down where your hardware problem is and how to fix it. x264 was the best stress test of hardware five years ago for the same reason.

[x265_Project]

Just for your information, I tried above command lines and my (old ?) computer simply shut down after pass 1 started, even with preset "fast". I'm encoding a sample of 4 minutes. It has never happened with x264 before though (I encoded same sample with slow presets, in command line and without ffmpeg). I have no idea what happens as there is no log or error message. Actually I have a problem with my ventilator unit : each time a lot of CPU is requested, this ventilator heats (I hear a lot of noise). It is just disturbing but nothing more. I guess it is due to x265 (I am on a 32 bit Windows vista engine).

It doesn't surprise me to hear reports like this. x265 is more CPU intensive than any application you can find. Overclockers are finding that x265 is the ultimate test of whether an overclocked system is stable. Even when a system proves to be stable with every other overclocking benchmark, it may not be stable when it tries to run x265. This is because of the very high levels of parallelism in our code, and because the code is highly optimized, including vectorization with all of the latest SIMD instruction sets (SSE, SSE2, SSE3, AVX, AVX2, etc.).

You can monitor your CPU temperatures with a utility like http://www.cpuid.com/softwares/hwmonitor.html.

How to fix this? First, if you know what you're doing, you should clean your PC by carefully vacuuming out all of the dust you can get to. A can of compressed air can blow out any remaining dust from your CPU heatsink... but be careful not to damage anything! If your CPU temperature is still soaring (above 60 degrees C or worse) on a desktop PC you can try to remove and reset your heat sink with fresh thermal paste (heat sink compound). Don't spread the paste out, as this will trap air bubbles when you remount the heat sink. Apply the paste in two lines, in an X pattern, as shown at the end of this video (except your lines should be nice and smooth and even, like little wires lying on your CPU) https://www.youtube.com/watch?v=EyXLu1Ms-q4. If you do it right you'll see your CPU temperatures come down quite a bit from where they were before, and hopefully your PC won't crash when you do x265 encodes.

If you have a laptop, and you can't take the steps above, you can also try to go into your system BIOS and manually reduce the clock speed, or you can tell x265 to use fewer CPU threads using the --pools command, which controls the number of software threads in the thread pool. You would have to tell us your CPU type before we could tell you the values to try with the --pools command.

[x265_Project]

I just posted a new blog post with more details about our new performance presets.

http://x265.org/performance-presets/

[me7]

Is the grain tuning recommended for regular film? Particularly for CRF encodes that are not bitrate starved.
If I don't care for hardware decoding, should I just use a 12-bit build?

[nevcairiel]

Even software decoders will be slower on 12-bit (or even incompatible), unless you have a proper reason to use it, I personally wouldn't.

[me7]

No proper reason, just remembering how going from 8-bit to 10-bit helped x264 alleviate banding.

[Atak_Snajpera]

Just add --psy-rd 2 to your command line. You do not need 10/12 bit at all.

Default
http://i.cubeupload.com/FzFWFZ.png

--psy-rd 2
http://i.cubeupload.com/DKDrIR.png

[Ma]

It would be interesting to see a comprehensive comparison between medium - slow - slower - veryslow. Veryslow tends to take a very long time even with 720p sources and --limit-modes + --limit-refs 3.

I encoded 5 samples on i5 3450S, Win7 64-bit, 10-bit output, with options:
--crf 17 --rdoq-level 1 --psy-rd 1 --deblock -1 --keyint 288 --colormatrix bt709
at presets: slow, slower, veryslow, placebo (only first sample).

Relative encoding time of different samples related to '--preset slower' (=100%):

----------------------| slow slower verysl placebo
1920x800-hob.y4m 28,3% 100,0% 158,1% 423,7%
1920x800-ret.y4m 33,6% 100,0% 147,1% --
720p50_parkrun_ter.y4m 24,9% 100,0% 153,3% --
ducks_take_off_1080p50 28,3% 100,0% 150,9% --
park_joy_1080p50.y4m 25,8% 100,0% 154,4% --


Relative encoding time of different samples related to '--preset veryslow':

----------------------| slow slower verysl placebo
1920x800-hob.y4m 17,9% 63,2% 100,0% 268,0%
1920x800-ret.y4m 22,9% 68,0% 100,0% --
720p50_parkrun_ter.y4m 16,2% 65,2% 100,0% --
ducks_take_off_1080p50 18,8% 66,3% 100,0% --
park_joy_1080p50.y4m 16,7% 64,8% 100,0% --


Encoding speed in frames per second (fps) of different samples:

----------------------| slow slower verysl placebo
1920x800-hob.y4m 3,09 0,87 0,55 0,21
1920x800-ret.y4m 2,58 0,87 0,59 --
720p50_parkrun_ter.y4m 5,84 1,45 0,95 --
ducks_take_off_1080p50 1,38 0,39 0,26 --
park_joy_1080p50.y4m 1,89 0,49 0,31 --


Preset 'slow' is much faster than 'slower', preset 'placebo' is much slower than 'veryslow'. Full data in attached file screen.txt

[sneaker_ger]

Just add --psy-rd 2 to your command line. You do not need 10/12 bit at all.

Default
http://i.cubeupload.com/FzFWFZ.png

--psy-rd 2
http://i.cubeupload.com/DKDrIR.png
>=10 bit can still be useful.

x265 1.8+191

source
http://abload.de/img/source_ewsvl.png

8 bit (default)
http://abload.de/img/8bitbyzbv.png

8 bit (--psy-rd 2)
http://abload.de/img/8bit-psy-rd2eolk2.png

10 bit (default)
http://abload.de/img/10bito3zd2.png

12 bit (default)
http://abload.de/img/12bit46zh3.png

In fades it's way more pronounced:
output videos (https://mega.nz/#F!QhEhxb5T!Q8Zkgb1x3CK4RO3rKdoIWQ)
(source (https://mega.co.nz/#!osdxQbCR!vim8f5gAD5nf0w0jf-vEAA3mGySmEOoZQOH_GE3Z2uw), mirror (http://217.160.126.132/lighthouse_lossless.mp4))

[Blowis]

Hi, Happy new year,
I wanted to know if I use vbv-maxrate, vbv-bufsize and I chose Crf: 21. Does the crf-max: 51 is active by default?
If I put crf-max: 30 is the active or not?
If it is active and I put a smaller value than the crf it changes anything?
Thank you

[LoRd_MuldeR]

CRF and VBV operate "side by side".

Simply put, first CRF RC will determine the "desired" bitrate for the current frame (based on the selected CRF and CRF-Max value), and then VBV will check whether that bitrate is currently "acceptable" or whether it has to be "capped".

So, if you only set the VBV limits high enough, you get perfectly "unaffected" CRF. If, on the other hand, you set the VBV limits very low, you will be running into the VBV limits all the time, effectively "overriding" CRF altogether.

Usually, if you make a "reasonable" choice of the CRF value and the VBV limits, you should get more or less "normal" CRF, with VBV kicking in here and there to "attenuate" the biggest bitrate peaks. But it really depends on the source ;)

[Nullack]

Hey guys. Im trying to get my head around an experiment I did. I took a sample cut from a DVD, thats around 125 MB. Then I used ffmpeg to convert it into Y4M for input into x265. Even when I do a recent build of x265 on the placebo and lossless settings Im still around 500 MB encoded in HEVC. Whats happening here? I suspect its something that should be apparent to me but after the excesses of the XMAS break the brain cells need time to work :)

[Atak_Snajpera]

you are converting lossy mpeg2 to lossless x265. It is like converting jpeg to bmp and then to lossless png.

[LigH]

On top, you don't understand that "placebo" is no sensible preset; its purpose is to waste as much time as possible. Even more on lossless results which skip most of the encoding steps a preset would affect at all. ;)

[Nullack]

Im experimenting with format shift from mpeg / dvd transport stream into hevc without quality loss. I only did the y4m step due to x265s needs. Ill try to do some research but any hints on command lines is welcome.

[foxyshadis]

Hey guys. Im trying to get my head around an experiment I did. I took a sample cut from a DVD, thats around 125 MB. Then I used ffmpeg to convert it into Y4M for input into x265. Even when I do a recent build of x265 on the placebo and lossless settings Im still around 500 MB encoded in HEVC. Whats happening here? I suspect its something that should be apparent to me but after the excesses of the XMAS break the brain cells need time to work :)

x265 doesn't know what you started with and just encodes as fancy as you tell it to, even if it's much more than the original, because DVD artifacts create artificial difficulty. Set quality (--crf) higher to reduce the size; default is 28. You'll never get no quality loss; you're better off removing DVD artifacts and then re-encoding at as much bitrate as you need.

btw, you can use ffmpeg and other tools to send y4m directly to x265, without writing a huge file.

[Nullack]

Aha! Now its clear thats a brilliant explanation thanks very much Foxyshadis. Artificial difficulty I like that for what the issue is

[Blowis]

CRF and VBV operate "side by side".

Simply put, first CRF RC will determine the "desired" bitrate for the current frame (based on the selected CRF and CRF-Max value), and then VBV will check whether that bitrate is currently "acceptable" or whether it has to be "capped".

So, if you only set the VBV limits high enough, you get perfectly "unaffected" CRF. If, on the other hand, you set the VBV limits very low, you will be running into the VBV limits all the time, effectively "overriding" CRF altogether.

Usually, if you make a "reasonable" choice of the CRF value and the VBV limits, you should get more or less "normal" CRF, with VBV kicking in here and there to "attenuate" the biggest bitrate peaks. But it really depends on the source ;)

Thank you for your reply.
I just see if I let Crf-Max: 51 (default) after encoding with MediaInfo I read crf-max = 0.0 (that means it's off). If I change as crf-max = 30 I read the crf-max = 30.0 (as it means it is active).
I say its because the X265 site for crf-max it is said:
Specify an upper limit to the rate factor qui May be Assigned to Any Given frame (max Ensuring a QP). This is dangerous When CRF is used in combination with LSV as it May result in buffer underruns. Default disabled.

Cfr mode it is better to leave then disable crf-max: 51.
For I the modified crf-max: 30 so I activated.

[LoRd_MuldeR]

I say its because the X265 site for crf-max it is said:
Specify an upper limit to the rate factor qui May be Assigned to Any Given frame (max Ensuring a QP). This is dangerous When CRF is used in combination with LSV as it May result in buffer underruns. Default disabled.

Well, I think if you set an upper limit for the rate-factor, it may happen that VBV needs to lower the bitrate in order to prevent an imminent buffer underflow, but it simply can't do so, because the rate-factor already reached its limit.

(Actually I don't know why you would want to limit the max. rate-factor)

[Nullack]

On top, you don't understand that "placebo" is no sensible preset; its purpose is to waste as much time as possible. Even more on lossless results which skip most of the encoding steps a preset would affect at all. ;)

According to the read the docs doco on x265 cli for the lossless option:

"Slower presets will generally achieve better compression efficiency (and generate smaller bitstreams)"

So I figured since its a short sample clip I might as well do placebo to see how small it could make the test file wjile also using lossless

Thoughts?

[LoRd_MuldeR]

According to the read the docs doco on x265 cli for the lossless option:

"Slower presets will generally achieve better compression efficiency (and generate smaller bitstreams)"

So I figured since its a short sample clip I might as well do placebo to see how small it could make the test file wjile also using lossless

Thoughts?

Yes, generally, using a "slower" presets gives better compression efficiency, i.e. it results in better "quality per bit" ratio. How the total size will be effected (at a fixed CRF), that's undefined. Might become smaller or bigger.

However, the slower the preset is that you are already using, the less additional improvement can be gained by using an even slower one! Or, in other words: The additional improvements becomes increasingly expensive (in terms of CPU cycles).

Thus going from "veryslow" to "placebo" will slow things down quite a lot... for a very minimal improvement.

(So, if you don't care about encoding time or about your electricity bill, then cranking everything up to "placebo" doesn't hurt. But, under normal circumstances, it's just not worth it)

[Blowis]

Well, I think if you set an upper limit for the rate-factor, it may happen that VBV needs to lower the bitrate in order to prevent an imminent buffer underflow, but it simply can't do so, because the rate-factor already reached its limit.

(Actually I don't know why you would want to limit the max. rate-factor)


I thought if I put crf-max lower than 51 I would have a better quality.
For example I use crf: 21 and I put crf-max: 30 I would have a better quality than crf-max: 51

After I saw that if I put the same parameter of the Medium profile to Profile Faster I faster encoding except that I have exactly the same info with MediaInfo.
I thought if we put the same parameter regardless profile had the same encoding.
It means that there are hidden parameter that belongs to each profile.

[LigH]

@ Nullack:

In addition to LoRd_MuldeR, a little more specific: The "placebo" preset uses the most elaborate set of options, it does not even use "shortcuts" which would usually cause no visually noticable difference but a quite noticable speedup of the encoding. It calculates every case, even the most improbable to affect encoding efficiency.

[foxyshadis]

I thought if I put crf-max lower than 51 I would have a better quality.
For example I use crf: 21 and I put crf-max: 30 I would have a better quality than crf-max: 51
Better quality and a larger file size. You could just lower the crf slightly and get about the same effect. With a crf of 21, QPs over 30 will rarely be used anyway, but when they are, you shouldn't see a difference.

After I saw that if I put the same parameter of the Medium profile to Profile Faster I faster encoding except that I have exactly the same info with MediaInfo.
I thought if we put the same parameter regardless profile had the same encoding.
It means that there are hidden parameter that belongs to each profile.

At the very least, rd, ref, and b-adapt should change. Are you sure they're the same?

[Blowis]

Better quality and a larger file size. You could just lower the crf slightly and get about the same effect. With a crf of 21, QPs over 30 will rarely be used anyway, but when they are, you shouldn't see a difference.



At the very least, rd, ref, and b-adapt should change. Are you sure they're the same?

Hi,
ok I'll leave default crf-max and use crf:21 it suits me.

I use StaxRip I selected the desired profile and I modify the parameters.
As I selected the same parameter but faster is faster than medium. I do not understand because I'm more test with 2 videos.

[LoRd_MuldeR]

I thought if I put crf-max lower than 51 I would have a better quality.
For example I use crf: 21 and I put crf-max: 30 I would have a better quality than crf-max: 51

You can also look at this as follows:

Normally, the encoder can choose the "optimal" (as determined by the encoder's RC algorithms) CRF value freely from the whole range. But, with CRF-Max, you enforce an upper limit for that range.

So, if the encoder normally would have chosen a CRF value above your selected CRF-Max, it is now forced to pick CRF-Max.

Whether this will actually improve the visual quality is uncertain - after all there should be a reason why the encoder originally wanted an even higher CRF for that scene. But one thing is for sure: It will cost more bits ;)

In particular, if you are willing to spend more bits for quality, it may be better to just use a somewhat higher target CRF (or a somewhat higher target bitrate), instead of messing with CRF-Max.

[Sagittaire]

Hi,
ok I'll leave default crf-max and use crf:21 it suits me.

I use StaxRip I selected the desired profile and I modify the parameters.
As I selected the same parameter but faster is faster than medium. I do not understand because I'm more test with 2 videos.

cfr mode is here to reproduce same quality than mulipass mode in only one pass mode without possible bitrate target.

to simplify crf will use low quantizer in easy part (low motion, flat texture ... etc) and higher quantizer in hard part (high motion, textured area, noisy part).

For me, if you want really constrained crf decision and have higher "constant quantizer mode" like, the best way is not limit the min an max quantizer (VBV compliance limitation for exemple). Better to use that:

- lower curve compression with higher qcomp (0.6 by default), 1.0 for qcomp mean no curve compression.
- lower ratio for Iframe and Bframe with ipratio and pbratio
- lower aq-strength for AQ

In addition constant quantizer mean constant quality only for the mathematical codec algorithme and not for eyes. in x264 and x265 code, psy setting, RD curve decison, matrix quantisation ... etc ... will have major implication on frame size even if you have the same quantizer and crf. CRF mode is not absolute quality level: Different source can have really low HVS quality with crf at 20 and really high HVS quality with crf at 40.

[Asmodian]

Better quality and a larger file size. You could just lower the crf slightly and get about the same effect.

I would argue that lowering crf slightly usually results in a better effect. The extra bits from using a high qp might be spent on block which will be used again while the high qp that was capped was almost certainly being used on a block that will not be referenced later. :)

ok I'll leave default crf-max and use crf:21 it suits me.


Try 20.9 too ;)

As I selected the same parameter but faster is faster than medium. I do not understand because I'm more test with 2 videos.

Faster is faster but it is also larger and/or lower quality. You do not get the same quality at the same crf if you change any other setting. Faster presets result in lower quality / size.

[LigH]

New year, new build... new feature:

--[no-]rd-refine Enable QP based RD refinement for rd levels 5 and 6. Default disabled

For each analysed CU, calculate R-D cost on the best partition mode for a range of QP values, to find the optimal rounding effect. Default disabled.

Only effective at RD levels 5 and 6

x26 1.8+201-769081eb5f4c (GCC 4.9.2) (https://www.mediafire.com/download/sdgynjrrbqzfj5y/x265_1.8+201-769081eb5f4c.GCC492.7z)
x26 1.8+201-769081eb5f4c (GCC 5.3.0) (https://www.mediafire.com/download/cccrbrfwa2h0g62/x265_1.8+201-769081eb5f4c.GCC530.7z)

BTW, x265 v1.9 is coming soon. Do we still need GCC 4.9.x builds? Or is GCC 5.3.0 the currently "best" GNU C/C++ compiler version?

[LigH]

Faster is faster but it is also larger and/or lower quality. You do not get the same quality at the same crf if you change any other setting. Faster presets result in lower quality / size.

Often. But not in general. There are always "academical exceptions". And preset placebo is no practical preset, only an "academical waste of time".

[LoRd_MuldeR]

BTW, x265 v1.9 is coming soon. Do we still need GCC 4.9.x builds? Or is GCC 5.3.0 the currently "best" GNU C/C++ compiler version?

GCC 5.x builds are consistently ~10% faster for me (compared to GCC 4.9.x). Didn't notice any apparent problems so far.

[Jamaika]

--[no-]rd-refine Enable QP based RD refinement for rd levels 5 and 6. Default disabled

How is it with these novelties? Can I use rd-refine at the function bitrate for levels 5?
I used
x265 --y4m --input-csp i444 --output-depth 8 --limit-refs 3 --limit-modes --high-tier --rd 6 --psy-rd 0.3 --rdoq-level 2 --psy-rdoq 50.0 --rd-refine --bitrate 6000 --vbv-bufsize 30000 --vbv-maxrate 30000 --preset veryslow ...
and movie places stutters.;) Is there a need for this function faster processors?

[LigH]

To ensure that the preset is overridden by additional single parameters, I would prefer to put the preset option as one of the first in the whole command line...

I doubt that RD refinement has any impact on the decoding speed; it should only optimize the quality retention during a GOP (reduce rounding error accumulation), if I understood it correctly... So if it seems to cause playback issues, then there may be bugs to discover.

[Jamaika]

My mistake. I put the wrong decoder HEVC for yuv444(AYUV) and the film is stuttered. Sorry.

[LigH]

Due to a few important bug fixes, and a "merge with stable":

x265 1.8+205-d94f6c2b45f8 (https://www.mediafire.com/download/bvwrku72t80mkwz/x265_1.8+205-d94f6c2b45f8.7z)

GCC 5.3.0 only from now on.

[K.i.N.G]

>=10 bit can still be useful.

x265 1.8+191

source
http://abload.de/img/source_ewsvl.png

8 bit (default)
http://abload.de/img/8bitbyzbv.png

8 bit (--psy-rd 2)
http://abload.de/img/8bit-psy-rd2eolk2.png

10 bit (default)
http://abload.de/img/10bito3zd2.png

12 bit (default)
http://abload.de/img/12bit46zh3.png

In fades it's way more pronounced:
output videos (https://mega.nz/#F!QhEhxb5T!Q8Zkgb1x3CK4RO3rKdoIWQ)
(source (https://mega.co.nz/#!osdxQbCR!vim8f5gAD5nf0w0jf-vEAA3mGySmEOoZQOH_GE3Z2uw), mirror (http://217.160.126.132/lighthouse_lossless.mp4))

Actually, for my eyes there's allot more banding in the 10bit and 8bit encodes...

[movmasty]

Due to a few important bug fixes, and a "merge with stable":

x265 1.8+205-d94f6c2b45f8 (https://www.mediafire.com/download/bvwrku72t80mkwz/x265_1.8+205-d94f6c2b45f8.7z)

GCC 5.3.0 only from now on.
Works on XP sp3, to whom may interest
and make Internet Friendly Media Encoder works on XP too(1st gui i found, hope to find another asap)

Does not worl with x264_launcher, wich uses other newer(?) builds

Got half the speed of x264 with faster preset.
Size 49% of the heavy compressed x264 with CRF 28, quality a bit lower, smoothing a bit too heavy.

Could someone please put updates on First post?

[LigH]

Well, yes, I still try to build 32-bit binaries XP-compatible; but they are less interesting, can't handle high resolutions due to memory restrictions for 32-bit processes. And high bitdepth builds don't use assembly optimization; they are even more restricted by the memory limit.

[Jamaika]

Due to a few important bug fixes, and a "merge with stable":
Finally any messages.
x265 [warning]: CRF max must be greater than CRF <-- function crf-min/max can use but analizer HEVC Elecard StreamEye4 doesn't show changes
x265 [warning]: NAL HRD parameters require VBV parameters, ignored <-- (use only by high-tier)
x265 : Main 4:4:4 profile, Level-4 (Main tier) <-- can change level, increased level to 5.1
x265 [info]: lowering VBV max bitrate to 12000Kbps
x265 [info]: lowering VBV buffer size to 12000Kb
Other observations:
[I]--limit-refs 3 --limit-mod --rd-refine --high-tier --preset placebo <-- visible quality loss
--high-tier --vbv-maxrate 30000 --vbv-bufsize 30000 --preset placebo <-- no effect on improving the quality
--rd 6 --psy-rd 0.3 --rdoq-level 2 --psy-rdoq 50.0 --preset placebo <-- effect banding, sharper image
--rd 6 --psy-rd 0.3 --no-psy-rd --no-psy-rdoq --preset placebo <-- no effect banding, blur image (ie. animation BPG)
--limit-mod <-- lossless quality

My best settings:
x265.exe --y4m --limit-mod --rd-refine --no-info --no-open-gop --no-hrd
--rd 6 --psy-rd 0.3 --rdoq-level 2 --psy-rdoq 50.0 --bframes 0 --deblock -3:-3 --qcomp 1.00 --me umh
--preset veryslow --no-b-intra --no-sao --rect --no-amp --no-temporal-mvp --no-signhide --no-strong-intra-smoothing
--input-depth 8 --input-res 1920x1080 --input-csp i444 --output-depth 8 --fps 30000/1001 --keyint 30 --crf 28 --vbv-bufsize 30000 --vbv-maxrate 30000 --range limited --output "x265.cc_1.8.0.205.h265" -
--rd-refine <-- function rd-refine for profile Main reduces some banding. In my opinion, it performs better without using the high-tier.
--bframes 0 <-- function reduces the saw at the edges of objects. It is good to use the function qcomp=1.00

The problem of color. The use of color matrix=bt709 in excessive add a particular color will be clipped color. Thus, we have a different color scheme as standard TV.
How do we want to keep the selected colors don't use ColorMatrix.

[Sagittaire]

My best settings:
x265.exe --y4m --limit-mod --rd-refine --no-info --no-open-gop --no-hrd
--rd 6 --psy-rd 0.3 --rdoq-level 2 --psy-rdoq 50.0 --bframes 0 --deblock -3:-3 --qcomp 1.00 --me umh
--preset veryslow --no-b-intra --no-sao --rect --no-amp --no-temporal-mvp --no-signhide --no-strong-intra-smoothing
--input-depth 8 --input-res 1920x1080 --input-csp i444 --output-depth 8 --fps 30000/1001 --keyint 30 --crf 28 --vbv-bufsize 30000 --vbv-maxrate 30000 --range limited --output "x265.cc_1.8.0.205.h265" -

You seem to abuse the local specialty of jamaica ... :devil:

1) Why use vbv without HRD? It's completely useless. VBV are here for hardware profil compliance and must be use with HRD tag I think. You want play your file with hardware decoding? Use VBV can hurt quality in really complexe part: you use it only it's necessary ...

2) bFrame are major implemantation in modern codec. Use bframe is always good for efficience. Modern codec have algorithme for use itself no-bframe if it's useless localy. If you want higher quality for bframe, lower the bframe ratio.

3) --no-b-intra --no-sao --rect --no-amp --no-temporal-mvp --no-signhide --no-strong-intra-smoothing ... ???

[burfadel]

Also why 8-bit? CRF seems pretty high (lower quality) too. That would result in a fairly poor looking 1920x1080 video with those other settings. If space is an issue, it would actually be higher quality to reduce the resolution and choose more suitable options (even close to defaults), 10-bit, and lower CRF.

[Jamaika]

And who says I'm a specialist. I am a poor man who is interested in new codecs. They say that wasted time because I have not bought HEVC editor. 1000 conditions codec should not interest me.:D

1) Why use vbv without HRD? It's completely useless. VBV are here for hardware profil compliance and must be use with HRD tag I think. You want play your file with hardware decoding? Use VBV can hurt quality in really complexe part: you use it only it's necessary ...
Maybe you and right. It is a pity that there is no message. There is only written in the manual: "Default disabled". I am not aware that in the VBV automatically turns on.
Is this parameter as a separate function VBV has some special application?
x265 [info]: VBV/HRD buffer / max-rate / init : 30000 / 30000 / 0.900
When off is still the same message.

2) bFrame are major implemantation in modern codec. Use bframe is always good for efficience. Modern codec have algorithme for use itself no-bframe if it's useless localy. If you want higher quality for bframe, lower the bframe ratio.

--ipratio
Sets the target average increase in bitrate for I-frames as compared to P-frames. Seen as 'keyframe boost' in xvid. Higher values increase the quality of I frames. This makes them better references, which can improve the overall image quality. The problem is that the extra bits taken by the I-frames are taken from the P and B-frames, which makes this variable a balancing act.
http://forum.doom9.org/showthread.php?p=1733944#post1733944
http://x265.readthedocs.org/en/default/presets.html#tuning

3) --no-b-intra --no-sao --rect --no-amp --no-temporal-mvp --no-signhide --no-strong-intra-smoothing ... ???
Good question. What is it used for a preset veryslow? When the films is dynamic these parameters don't improve quality.
The quality probably negligible to notice compare to default veryslow settings.


@burfadel
Everything can be done. Just as I'm wondering how I have AVC 8bit why it convert to 10bit?

[foxyshadis]

1) Why use vbv without HRD? It's completely useless. VBV are here for hardware profil compliance and must be use with HRD tag I think. You want play your file with hardware decoding? Use VBV can hurt quality in really complexe part: you use it only it's necessary ...

Streaming?

--ipratio
Sets the target average increase in bitrate for I-frames as compared to P-frames. Seen as 'keyframe boost' in xvid. Higher values increase the quality of I frames. This makes them better references, which can improve the overall image quality. The problem is that the extra bits taken by the I-frames are taken from the P and B-frames, which makes this variable a balancing act.
http://forum.doom9.org/showthread.php?p=1733944#post1733944
http://x265.readthedocs.org/en/default/presets.html#tuning
It's a balancing act, which means choosing one extreme or the other hurts. Turning them off always hurts. The post you linked to uses 10 b-frames, so it's an odd choice to support your use of none?

Good question. What is it used for a preset veryslow? When the films is dynamic these parameters don't improve quality.
Veryslow is veryslow, it's right there on the tin, its options are only to extract a percent or two more out of the video. A faster profile would offer a better mix of speed for quality than this, maybe the slow preset, based on your option mix. Why --no-temporal-mvp, --no-sao, and --no-signhide? They're extremely fast and save bitrate. It seems like you're just turning off options at random, mixing veryslow with ultrafast, and that's the best way to break a codec.

I think you should go back to the drawing board, start at --profile slow and only modify from there as you find fixable problems. (Note that the new default --psy-rd is much higher than yours, as well.)

[Boulder]

I didn't get any answers regarding --rdoq-level and --psy-rdoq - the default for rdoq-level is 2 for veryslow but if the goal is to retain more grain, is it better to switch to --rdoq-level 1 and have --psy-rdoq around 5.0-10.0 or so? None of the presets seem to use --rdoq-level 1.

[shinchiro]

Why --no-temporal-mvp, --no-sao, and --no-signhide?.

I still disabled sao for good since it still blur the details..

[divxmaster]

After weeks of testing 1080p, I am using (10bit)
--crf 20 --psy-rd 1.5 --psy-rdoq 1.1 --aq-mode 2 --tu-intra-depth 3 --merange 41 --preset slow --bframes 5 --max-merge 5 --min-keyint 23 --keyint 288 --deblock -3:-3 --no-open-gop --rdoq-level 2
This gives great results, with good detail retention and reasonable size of 2-4gb per bluray, depending on source grain.
For heavily grained movies (quantum of solace for example), I use vapoursynth 'vid = haf.SMDegrain(vid, tr=3,thSAD=400,RefineMotion=True,contrasharp=True,pel=2)'

@atak_snajpera Excellent tip on high psyrd. I have compromised on 1.5, as 2 increase bitrate too much for me
@stax76, I wish I had found the excellent video comparison tool in staxrip much sooner!
@Ligh, x265 seems to parse out preset first, and then process the rest of the arguments (I think)

Cheers,
Divxmaster

[Jamaika]

Thank you for answers

It's a balancing act, which means choosing one extreme or the other hurts. Turning them off always hurts. The post you linked to uses 10 b-frames, so it's an odd choice to support your use of none?
For preset veryslow/slower are default value ipratio/bpratio/bframes = 1.40/1.30/8. For BPG animation is bframes = 0.
I thought it was the best option possible. I did not think that the value bframe is so important.
User Sagittaire is right. Best environmental option is bframes = 3 / ipratio = 1.00 / bpratio = 1.00.

I wonder why you can not set default it to presets medium <> veryslow under CRF.
Why --no-temporal-mvp, --no-sao, and --no-signhide? They're extremely fast and save bitrate. It seems like you're just turning off options at random, mixing veryslow with ultrafast, and that's the best way to break a codec.

Tests are needed. Also important is the user's taste.
For ip/bpratio = 1.40/1.30 didn't like quality.

I didn't like switching these functions. I didn't check how it looks for 1.00. Currently I don't have the time and the corresponding film on longer tests with an analysis of each function. Besides, who is going to read and looked thirty photos. Waiting for the new X265 codec version 1.9.
Note that the new default --psy-rd is much higher than yours, as well.
Thanks. User requests have been met. Apparently there is no bandings.

[2themax]

I'm having an issue making x265 adhere to level I am specifying. I realize that it will automatically change the level to match the options, but I need level 5.1 to adhere to the UHD BD spec. Any suggestions on what I can do to make that happen?

x265 --preset veryslow --output "out.hevc" --input "in.yuv" --input-res 3840x2160 --input-csp i420 --fps 24 --profile main10 --level-idc 51 --high-tier --open-gop --keyint 24 --min-keyint 1 --bitrate 65000 --vbv-maxrate 95000 --vbv-bufsize 100000 --pass 1 --colorprim bt709 --transfer bt709 --colormatrix bt709 --hrd --aud --sar 1:1 --output-depth 10

[sneaker_ger]

Unrelated to your problem, but:
--vbv-maxrate 95000 --vbv-bufsize 100000
Buffer must not be greater than maxrate.

[2themax]

Unrelated to your problem, but:

Buffer must not be greater than maxrate.
It's not an issue in the optical disc world. For instance, the BD vbv is 30,000 but the max bit rate is 40,000.

[sneaker_ger]

It's not an issue in the optical disc world.
Spec is spec, nobody cares about anything else.

For instance, the BD vbv is 30,000 but the max bit rate is 40,000.
Read carefully. My statement does not contradict that. 30,000 is not greater than 40,000.

[nevcairiel]

Unrelated to your problem, but:

Buffer must not be greater than maxrate.

Actually buffer being greater than maxbitrate is perfectly fine.
A H.264 L4.0 BD compliant stream has max bitrate 24000 and buffer 30000, while a L4.1 stream has bitrate 40000 and buffer 30000 as well.

The buffer size just means how much will be buffered (duh!), you can equally buffer 2 seconds worth, or 2*maxbitrate. It influences a few things, including memory requirements in the decoder and startup time (to fill the buffer), but there is no problem with it.

[2themax]

Spec is spec, nobody cares about anything else.


Read carefully. My statement does not contradict that. 30,000 is not greater than 40,000.
Sorry I read the second part wrong but you are still incorrect. nevcairiel gave a good example in the above post. BD is its own spec and VBV (CPB in AVC) can be higher than max bit rate.

[sneaker_ger]

3.5.3 STD delay
Maximum of STD delay is 1 second for video stream, 60 seconds for still picture.
BD-ROM_Part3_V3.0_WhitePaper_150724.pdf (http://www.blu-raydisc.com/assets/Downloadablefile/BD-ROM_Part3_V3.0_WhitePaper_150724.pdf)

Am I interpreting that wrong?
Same for AVC according to "Encoding Video for Blu-Ray using H264/AVC" thread (http://forum.doom9.org/showthread.php?t=154533).

[nandaku2]

I'm having an issue making x265 adhere to level I am specifying. I realize that it will automatically change the level to match the options, but I need level 5.1 to adhere to the UHD BD spec. Any suggestions on what I can do to make that happen?

x265 writes into the VPS the minimum decoder level required to decode a video successfully. In this case, Level 5 High-tier decoder support is sufficient to decode the video.

[Jamaika]

--colorprim bt709 --transfer bt709 --colormatrix bt709
I see that everyone uses BT709 for 10bit files. If you record something in Slog camera can be used BT2020? I know they have a problem with the new ColorMatrix some TV.

[2themax]

x265 writes into the VPS the minimum decoder level required to decode a video successfully. In this case, Level 5 High-tier decoder support is sufficient to decode the video.

Correct but I need to somehow override that function so the stream adheres to the UHD BD spec.

[LigH]

UHD BD supports level 5.1, you say (I wonder where you can read that, any sources known?)... as the maximum level, I believe. Why do you believe that lower level results are not supported?

[sneaker_ger]

3.1 General Constraints
Profile
Main 10 profile
general_profile_idc in SPS shall be set to 2.
Tier
High Tier
general_tier_flag in SPS shall be set to 1.
Level
Level 5.1
general_level_idc in SPS shall be set to 153.
Source:
http://www.blu-raydisc.com/assets/Downloadablefile/BD-ROM_Part3_V3.0_WhitePaper_150724.pdf

"shall" = absolute requirement

[LigH]

should < shall < must

[sneaker_ger]

The paper does not offer a definition for the words but usually:
MUST This word, or the terms "REQUIRED" or "SHALL", mean that the definition is an absolute requirement of the specification.
https://tools.ietf.org/html/rfc2119

If you look to the BluRay paper you will see that "shall" is used all the time, very seldomly "must". There is no doubt how it's meant.

[x265_Project]

Correct but I need to somehow override that function so the stream adheres to the UHD BD spec.
We don't have our hands on any UHD-BD players yet, but I would be surprised if the Main 10, High Tier, Level 5.1 HEVC decoder in a UHD-BD player would refuse to decode Main Tier or Level 5.0 stream. If, for UHD-BD specification compatibility reasons, we need to provide the ability to force --level-idc (http://x265.readthedocs.org/en/default/cli.html#cmdoption--level-idc) to use the specified value regardless of the ability for lower level decoders to handle the bitstream, we'll be happy to add this support.

[2themax]

The paper does not offer a definition for the words but usually:

https://tools.ietf.org/html/rfc2119

If you look to the BluRay paper you will see that "shall" is used all the time, very seldomly "must". There is no doubt how it's meant.

From the BD spec.

Shall: indicates an action or feature that is mandatory and must be implemented to claim compliance to this specification.

We don't have our hands on any UHD-BD players yet, but I would be surprised if the Main 10, High Tier, Level 5.1 HEVC decoder in a UHD-BD player would refuse to decode Main Tier or Level 5.0 stream. If, for UHD-BD specification compatibility reasons, we need to provide the ability to force --level-idc (http://x265.readthedocs.org/en/default/cli.html#cmdoption--level-idc) to use the specified value regardless of the ability for lower level decoders to handle the bitstream, we'll be happy to add this support.
Right now because of this I can not import a HEVC stream to even test the theory.

[x265_Project]

From the BD spec.

Shall: indicates an action or feature that is mandatory and must be implemented to claim compliance to this specification.


Right now because of this I can not import a HEVC stream to even test the theory.
Alright. We'll take a look at this.

[2themax]

Alright. We'll take a look at this.
Thanks for looking into it.

[benwaggoner]

Sorry I read the second part wrong but you are still incorrect. nevcairiel gave a good example in the above post. BD is its own spec and VBV (CPB in AVC) can be higher than max bit rate.
Correct.

Note that vbv-maxrate is in kbits/sec and vbv-bufsize is in kbits. Different units. HEVC is a little atypical in that Main Tier the bufsize=maxrate because default max buffer at peak bitrate is generally 1 second. But older codecs like H.264 often had max vbv-bufsize larger than max vbv-maxrate.

[sneaker_ger]

Correct.
But not on BluRay. See post #3131.

[benwaggoner]

But not on BluRay. See post #3131.
Yes, that is my understanding as well.

I haven't tried making a BD HEVC stream yet, though.

[LigH]

A few more bugs fixed, and merged with stable:

x265 1.8+212-792f6ead9c50 (https://www.mediafire.com/download/4adcqgsku2yvy6s/x265_1.8+212-792f6ead9c50.7z)

[LazyNcoder]

Hi guys,
I have a grainy video and I wanna encode it to x265 but it's killing me.
I tried --nr-inter --nr-intra at first. but it's not quite what I have in mind. it removes some noises not all, and in same scene with still camera and everything, remaining noises come and go. I mean it's not practical at all.
I also tried --tune grain. It made the video so blur. Almost removed all the noises but the problem is keyframes. like when the camera changes, all the noises appear for little amount of time and then they disappear. it's so annoying and again, not practical at all.

I'm out.

I also wanted to try StaxRip with RgTools.dll but it says it can't find the dll file but it's right there in the right path.

any suggestion?

[LigH]

Too few details, we are already missing your whole command line and source details (e.g. a MediaInfo analysis to know dimensions and framerate to have a relation to your bitrate control options).

If x265 noticably reduces noise right after a keyframe, you will certainly have a way too low bitrate target. Encoding grainy material requires a lot more bitrate than encoding smooth material, that's a fact, there is no magic to avoid that.

[LazyNcoder]

Too few details, we are already missing your whole command line and source details (e.g. a MediaInfo analysis to know dimensions and framerate to have a relation to your bitrate control options).

If x265 noticably reduces noise right after a keyframe, you will certainly have a way too low bitrate target. Encoding grainy material requires a lot more bitrate than encoding smooth material, that's a fact, there is no magic to avoid that.

It's a regular promotional - but much noisy - 1080p 24mbps 24fps video. nothing too special about the command line, --crf 20 --preset slower --output-depth 10 --rdoq-level 1 --aq-mode 3. with x265 1.8+212-792f6ead9c50 that you've kindly posted above.

I also tried NLMeans noise reduction built-in handbrake. It was awesome. but since handbrake can only produce 8bit videos, the result had huge banding problem even with aq-mode 3.

I used crf 20 with both --tune garin and without it when I used --nr-inter 400 nr-intra 400. --tune grain was blurred no matter what crf I used. Also, lower bitrate can help reducing grain but it also reduce video quality at the same time (duh).

[dipje]

KNLMeansOpenCL is available as a vapoursynth and plugin and runs in basically any kind of bitdepth. I think the AviSynth plugin also has support for 'the ugly 16bit avisynth hack'.


But do you have a noisy source and you want x265 to remove the noise and give good quality video... or do you want to preserve the noise as good as possible? This is not yet clear to me.

You talk about noise-reduction plugins, but at the same time you talk about '--tune grain' (which is to preserve grain, not remove it).

[LazyNcoder]

KNLMeansOpenCL is available as a vapoursynth and plugin and runs in basically any kind of bitdepth. I think the AviSynth plugin also has support for 'the ugly 16bit avisynth hack'.


But do you have a noisy source and you want x265 to remove the noise and give good quality video... or do you want to preserve the noise as good as possible? This is not yet clear to me.

You talk about noise-reduction plugins, but at the same time you talk about '--tune grain' (which is to preserve grain, not remove it).

No, I want to remove the noise. I thought the same about --tune grain but when I used it, I see it's not what I think it is.

I thought there's a simple answer for this, so I just asked. Now, let me walk you guys through it.

Original picture (http://s2.postimg.org/fjaxtoyeh/Original.png)

--tune grain keyframe (http://s27.postimg.org/pasno6yir/Grain_key.png)

--tune grain 7 frames after the keyframe (http://s16.postimg.org/7fdss6xzp/Grain.png)

--nr-intra 200 nr-inter 200 (http://s2.postimg.org/zayv4dovd/Inter.png)

[divxmaster]

As per my post #3123, one page back, I am getting great results with vapoursynth 'vid = haf.SMDegrain(vid, tr=3,thSAD=400,RefineMotion=True,contrasharp=True,pel=2)'
for degraining.
thSAD 400 for heavy grain, 250 for medium and 150 for light grain.
I have tested -nr-intra and -nr-inter a lot, and found it blurs way too much, even on nr 100. I only use it for encodes for mobile devices, at 400, since the screen is so small.
I will have to check out KNLMeansOpenCL also.

Cheers,
Divxmaster

[foxyshadis]

KNLMeansCL, no Open in the name. It's nearly as good as SMDegrain at 10-100x the speed, since it runs on GPU and thus runs faster the better your graphics card. It's far better to use AVS or VS to remove your grain, they're actually dedicated to fixing video, whereas x265 only has one knob that incidentally sometimes works OK. Alternately, you can use ffmpeg to do both filtering and encoding on one command-line, although its filtering isn't quite as high quality.

[LigH]

As far as I understood the --nr-* options, they don't reduce the noise; they rather try to store it in a simplified way, to maintain its amount while reducing its complexity which makes it hard to encode. It's rather a "noise remodeling" than a "noise reduction". Correct me if I'm wrong...

If you want to reduce noise, pre-filtering the source is indeed recommendable. An encoder has to assume that the video it receives is just as you want to see it when its encoding result is decoded again.

[burfadel]

As far as I understood the --nr-* options, they don't reduce the noise; they rather try to store it in a simplified way, to maintain its amount while reducing its complexity which makes it hard to encode. It's rather a "noise remodeling" than a "noise reduction".

I don't believe that is the case, it's just a very mild noise reducer. Typically I've found around 400-450 for both options allows you to use a slightly lower quality factor (meaning higher quality) without affecting picture detail. Basically, I believe the bitrate saved that can go towards a lower quality factor (meaning higher quality) maybe from say, 22 to 21 for example.

[divxmaster]

I don't believe that is the case, it's just a very mild noise reducer. Typically I've found around 400-450 for both options allows you to use a slightly lower quality factor (meaning higher quality) without affecting picture detail. Basically, I believe the bitrate saved that can go towards a lower quality factor (meaning higher quality) maybe from say, 22 to 21 for example.

I initially thought that, but then I used video comparison in staxrip and it showed there was a lot more blur than I thought, when I was just comparing videos side by side. Just checking you have used video comparison or similar on it? I find with crf20, smdegrain makes the file 30% smaller on heavy grain video, so the size is great. This is for Full HD only.
I cannot get video comparison to open the .m2ts at the correct size, so I run a --lossless sample to compare to.

Cheers,
Divxmaster

[nandaku2]

Hi guys,
I have a grainy video and I wanna encode it to x265 but it's killing me.
I tried --nr-inter --nr-intra at first. but it's not quite what I have in mind. it removes some noises not all, and in same scene with still camera and everything, remaining noises come and go. I mean it's not practical at all.
I also tried --tune grain. It made the video so blur. Almost removed all the noises but the problem is keyframes. like when the camera changes, all the noises appear for little amount of time and then they disappear. it's so annoying and again, not practical at all.

any suggestion?

Hello,

We completely agree with you that tune grain in its present form is not acceptable. Can you please try this patch out? (https://patches.videolan.org/patch/12030/). This changes the behaviour of tune grain, and though the whole feature isnt ready as a user option just yet, I think you'll find the results much better and the grain more consistent. Would be great to hear about some initial test results.

[sneaker_ger]

param->rc.cuTree = 0;
:eek:

[Sagittaire]

param->rc.cuTree = 0;
:eek:

No suprise. cuTree use RD curve (if my memory is good) and will certainely use higher quantizer in high textured part of the image. (Noise is by definition high textured area)

Moreover, Noise is by definition in all screen part, and use cuTree is not really usefull here.

To preserve noise, you must generaly reduce temporal and spacial quantizer variation: this patch make that with low ipfactor, ibfactor, high qCompress and desactive all AQ option. It's really not big surprise.

[sneaker_ger]

That's all nice and well but x264 seems to do fine without the sledgehammer that is --no-mbtree.

Anyways, hoping to see (and do) some tests within the next days. Also hoping someone is sparing me the work of compiling a build myself.

[Ma]

Also hoping someone is sparing me the work of compiling a build myself.

GCC 6.0 build (outdated, grain patch now is applied)

[mandarinka]

That's all nice and well but x264 seems to do fine without the sledgehammer that is --no-mbtree.

x264 uses its mbtree by default everywhere but it isn't really great for everything. With things I encode, I disable it virtually every time. For --tune grain in x264, I suspect it is not very helpful, if it doesn't actually harm.

[Ma]

Can you please try this patch out? (https://patches.videolan.org/patch/12030/). This changes the behaviour of tune grain

First impression -- this patch changes output file without '--tune grain' option too.

With '--tune grain' the file size is much smaller (than without this patch) but quality is wrong. I watched results of encoding:
x265 --tune grain -D10 -p veryslow 720p50_parkrun_ter.y4m w.hevc
Sample file is not grainy but result is simply wrong.

[nandaku2]

First impression -- this patch changes output file without '--tune grain' option too.

With '--tune grain' the file size is much smaller (than without this patch) but quality is wrong. I watched results of encoding:
x265 --tune grain -D10 -p veryslow 720p50_parkrun_ter.y4m w.hevc
Sample file is not grainy but result is simply wrong.

With or without this patch, tune grain used on a non-grainy source doesnt make sense.

There are a number of user options to be corrected on this temp patch as of now - which is why it's not yet ready for 1.9. Please try this patch only on grainy videos with tune grain on.

The logic behind turning off cu-tree and AQ is that, on a grainy source every block of the frame has approximately similar texture (the actual content doesnt matter, since they're fully overridden by the grain). Our tests show this is vastly better at preventing grain patchiness.

[Sagittaire]

First impression -- this patch changes output file without '--tune grain' option too.

With '--tune grain' the file size is much smaller (than without this patch) but quality is wrong. I watched results of encoding:
x265 --tune grain -D10 -p veryslow 720p50_parkrun_ter.y4m w.hevc
Sample file is not grainy but result is simply wrong.

...... :readrule::readguid:

If you want make comparison, the first rule is to have same size.

Why make comparison with "much smaller" size. I did not even need to see the encoding that if the size difference is significant then of course that the quality will be lower. And moreover, you use grain tuning with source without grain and with high textural (spacial and temporal) variation. For these source AQ is really important for quality.

[sneaker_ger]

I'm also experiences broken results:

http://abload.de/img/original_8cjp3.png
http://abload.de/img/x265_vanilla_ydjoy.png
http://abload.de/img/x265_patched_hvkhu.png

Settings is 2pass --preset slower --tune grain --bitrate 7000. Patched build undershot bitrate but not enough to explain the extreme difference.

output files (https://mega.nz/#F!l00SDCZY!E39ux22Up96XuEJ-ZrZSEA)
source file (https://mega.co.nz/#!hh0nRLia!7BghO78Nto3t9jVz_AObXbHuFd5HlDn7k4XOb_acysc)
Source AviSynth script with cropping and trimming:
lwlibavvideosource("original.mkv")
assumefps(24000, 1001)
crop(0, 24, 0, -24)
Trim(634,1156)

[Ma]

With or without this patch, tune grain used on a non-grainy source doesnt make sense.

There are a number of user options to be corrected on this temp patch as of now - which is why it's not yet ready for 1.9. Please try this patch only on grainy videos with tune grain on.

I found 20 second sample www.msystem.waw.pl/x265/grain20s.mkv
Is it enough grainy/ugly for tests?

[Motenai Yoda]

@sneaker_ger have you tried with unpatched build and adding --rdoq-level 1 or 0?
I think it's a must for grainy sources.

[LigH]

Again, a "weekly" build with bugfixes; merge with stable.

x265 1.8+221-f548abe8eae8 (https://www.mediafire.com/download/fdt3f3w77khddbu/x265_1.8+221-f548abe8eae8.7z)
__

By the way ... there is really someone trying to create x265vfw (http://sourceforge.net/projects/mpxplay/files/x265vfw/) (currently incomplete, x265vfw_v100_x265b79_20160116 misses some MinGW/GCC DLLs, possibly no static build).

As if stuffing AVC into AVI was not yet pulling at the seams. And now HEVC in AVI; when will the seams burst? AVI is a tough leather.

[Jamaika]

By the way ... there is really someone trying to create x265vfw (http://sourceforge.net/projects/mpxplay/files/x265vfw/) (currently incomplete, x265vfw_v100_x265b79_20160116 misses some MinGW/GCC DLLs, possibly no static build).

As if stuffing AVC into AVI was not yet pulling at the seams. And now HEVC in AVI; when will the seams burst? AVI is a tough leather.
This is some crap. I don't even want to start.
rundll32.exe x265vfw.dll,Configurehttp://i65.tinypic.com/10cq8eq.png

Edit: Adding file 'libgcc_s_dw2-1.dll' doesn't correct the problem.
http://www.dll-files.com/dllindex/dll-files.shtml?libgcc_s_dw2-1

[LigH]

I told you, missing MinGW/GCC DLLs.

[x265.cc]

GCC 6.0 build: www.msystem.waw.pl/x265/x265-grain.7z

you should mention that gcc 6.0 is still experimental an not stable.
Afaik there are no performance optimizations for x86 in gcc 6.0.

[Ma]

you should mention that gcc 6.0 is still experimental an not stable.
Afaik there are no performance optimizations for x86 in gcc 6.0.

Yes, GCC 6.0 is in "regression and documentation fixes stage"
https://gcc.gnu.org/ml/gcc/2016-01/msg00168.html
and first stable version will be 6.1.

The output files in x265 encoding are the same for GCC 6.0, GCC 5.3 and VS 2015 builds, so there is no difference in stability (for x265).

Speed of encoding for CPU with SSE4 or better -- VS 2015 builds always win, for SSSE3 CPU -- GCC 6.0 wins. You can emulate SSSE3 CPU by '--asm SSSE3' option.

I attached simple speed results of emulated SSSE3 CPU for 10-bit encoding by your GCC 5.3 build and my GCC 5.3, GCC 5.3 SSSE3-CPU-msvcr120, GCC 6.0 and GCC 6.0 SSSE3-CPU-msvcr120.

[divxmaster]

I've been recently testing hardware decode of 10bit HEVC on skylake, with interesting results.
From what I understand skylake has 10bit hardware ASSISTED decode, using the inbuilt GPU.
Kabylake will have a full dedicated 10bit h/w decode module in the CPU.
Anyway, I've managed to get MPCHC to say H/W decoding, using dxva2 copyback.
BUT, it only saves around 2% cpu. Peaks at 17% in one scene sw and 15% hw.
So, not doing much. Curious if anyone else has tried it and got different results.

Cheers,
Divxmaster

[kotuwa]

Psy-RD
When the default value for Psy-RD is changed from 0.3 to 2.0,
The internal algorithms and internal stuff changed or not?

I mean, is 0.3 of new 1.8 builds is same as 0.3 in earlier 1.8 builds? Or close?
Or newer builds' 2.0 is closer to 0.3 in older builds?
or somewhere in the middle?
!?

[Jamaika]

I don't know how codec change the internal algorithms. I know that the frames has more details for placebo at psy-rd=2.00 and psy-rdoq=50.0.

[LigH]

I remember that these features have been improved to have less artifacts at higher strengths, therefore it was now safe enough to increase default values and allow a wider range. But I couldn't tell you if there is a kind of scaling.

[Ma]

If you want make comparison, the first rule is to have same size.

OK, so with options '--bitrate 10000 --tune grain' I encoded 20 seconds sample to file_1.hevc, then file_1.hevc to file_2.hevc and so on...

8-bit encoding with options '--bitrate 10000 --tune grain' iterated 100 times:
original sample: www.msystem.waw.pl/x265/grain20s.mkv
patched x265: www.msystem.waw.pl/x265/x265-8b-patch_100.hevc
clean x265: www.msystem.waw.pl/x265/x265-8b_100.hevc

clean x264: www.msystem.waw.pl/x265/x264-8b_100.264

I attached batch files used to encode.

x264 retains brightness and colors, x265 behaves weird.
-----------
10-bit encoding is better (after 100 iterations) and '--preset slow' is much better then '--preset medium'. I've tried also '--rdoq-level 1' option, but it is hard to tell if it is better or not.

10-bit encoding with options '-D10 --preset slow --bitrate 10000 --tune grain' iterated 100 times:
patched x265: www.msystem.waw.pl/x265/x265-10b-slow-patch_100.hevc

[sneaker_ger]

I've been recently testing hardware decode of 10bit HEVC on skylake, with interesting results.
From what I understand skylake has 10bit hardware ASSISTED decode, using the inbuilt GPU.
Kabylake will have a full dedicated 10bit h/w decode module in the CPU.
Anyway, I've managed to get MPCHC to say H/W decoding, using dxva2 copyback.
BUT, it only saves around 2% cpu. Peaks at 17% in one scene sw and 15% hw.
So, not doing much. Curious if anyone else has tried it and got different results.
We have a thread dedicated to decoding performance. It also has results on Skylake if you are interested:
http://forum.doom9.org/showthread.php?p=1735604#post1735604

[divxmaster]

We have a thread dedicated to decoding performance. It also has results on Skylake if you are interested:
http://forum.doom9.org/showthread.php?p=1735604#post1735604

Great, thanks. I had a quick look around but couldn't see one under hevc, I see it is under 'software players' category.
Cheers.

[greenfountain]

OK, so with options '--bitrate 10000 --tune grain' I encoded 20 seconds sample to file_1.hevc, then file_1.hevc to file_2.hevc and so on...

8-bit encoding with options '--bitrate 10000 --tune grain' iterated 100 times:
original sample: www.msystem.waw.pl/x265/grain20s.mkv
patched x265: www.msystem.waw.pl/x265/x265-8b-patch_100.hevc
clean x265: www.msystem.waw.pl/x265/x265-8b_100.hevc

clean x264: www.msystem.waw.pl/x265/x264-8b_100.264

I attached batch files used to encode.

x264 retains brightness and colors, x265 behaves weird.
-----------
10-bit encoding is better (after 100 iterations) and '--preset slow' is much better then '--preset medium'. I've tried also '--rdoq-level 1' option, but it is hard to tell if it is better or not.

10-bit encoding with options '-D10 --preset slow --bitrate 10000 --tune grain' iterated 100 times:
patched x265: www.msystem.waw.pl/x265/x265-10b-slow-patch_100.hevc

From your script, I see you are encoding the content from the previously encoded bitstream for N iterations - why are you encoding it this way? any particular reason?

I downloaded your source and encoded the .mkv file with the same setting (--bitstream 10000 and tune grain). the results are much cleaner with no artifacts at all. applying the patch also shows the same results, with the grains improved in many of the B frames.

vanilla encode : https://www.dropbox.com/s/7o00sceo12uyd9j/grain20s_10000.hevc?dl=0

with latest grain patch : https://www.dropbox.com/s/2llmy9wvf5ogbgw/grain20s_10000_patch.hevc?dl=0

[Ma]

From your script, I see you are encoding the content from the previously encoded bitstream for N iterations - why are you encoding it this way? any particular reason?

Yes, I made 100 iterations of encoding with the same options. Reason -- there are quality imperfections in x265 encoding and after 100th iterations they are amplified and easier to see.

I downloaded your source and encoded the .mkv file with the same setting (--bitstream 10000 and tune grain). the results are much cleaner with no artifacts at all. applying the patch also shows the same results, with the grains improved in many of the B frames.

It looks like you made only 1st iteration (normal encoding) and it is much better than 100th iteration. I can't reproduce exactly your results because I can't find commit 'a53509b6fc0c' (from your encoded samples) and I'm surprise how you decode 'grain20s.mkv' sample to achieve fps=24000/1000?

[LigH]

Version 1.9 has been released as new milestone. From the developer mailing list:

x265 version 1.9 has now been released. This release supports many new features as well as additional assembly optimizations for Main12, intra prediction and SAO. Recently added features lookahead-slices, limit-refs and limit-modes have been enabled by default in the supported presets.

Full documentation is available at http://x265.readthedocs.org/en/stable/

========================================== New Features ==============================================
Quant offsets: This feature allows block level quantization offsets to be specified for every frame. An API-only feature.
--intra-refresh: Keyframes can be replaced by a moving column of intra blocks in non-keyframes.
--limit-modes: Intelligently restricts mode analysis.
--max-luma and --min-luma for luma clipping, optional for HDR use-cases
Emergency denoising is now enabled by default in very low bitrate, VBV encodes
=========================================== API Changes ==============================================
x265_frame_stats returns many additional fields: maxCLL, maxFALL, residual energy, scenecut and latency logging
--qpfile now supports frametype 'K"
x265 now allows CRF ratecontrol in pass N (N greater than or equal to 2)
Chroma subsampling format YUV 4:0:0 is now fully supported and tested
====================================== Presets and Performance ==========================================
Recently added features lookahead-slices, limit-modes, limit-refs have been enabled by default for applicable presets.
The default psy-rd strength has been increased to 2.0
Multi-socket machines now use a single pool of threads that can work cross-socket.

Thanks,
Deepthi Nandakumar
Engineering Manager, x265
Multicoreware, Inc

Only few more fixes for the new milestone:

x265 1.9+3-548a45bbf223 (https://www.mediafire.com/download/8sdpq21gboc7o3b/x265_1.9+3-548a45bbf223.7z)

[LigH]

Some Germans might find it funny — Fefe reports about x265 (http://blog.fefe.de/?ts=a859085f). He is usually notorious for ... uhm, let's say ... "alternative news" (calls himself "conspiracy theorizer" with a tongue-in-cheek), but his main profession is IT security, and he is in general interested in superior technologies.

His conclusion here: x265 became fast enough to be considered interesting for practical use.

[x265_Project]

Some Germans might find it funny — Fefe reports about x265 (http://blog.fefe.de/?ts=a859085f). He is usually notorious for ... uhm, let's say ... "alternative news" (calls himself "conspiracy theorizer" with a tongue-in-cheek), but his main profession is IT security, and he is in general interested in superior technologies.

His conclusion here: x265 became fast enough to be considered interesting for practical use.
Thanks for the link. Translated by Google...
Tue January 26 2016

[l] I have just a little playing around with X265, and wow the progress made! The Encoding Performance has always been massively unacceptable, but I can encode here just full HD material with 18 fps. This is the barrier, in the x264 was acceptable to me. Previously this was more like 3 fps in my experiments. VP9 is still in such regions.
This is a real breakthrough. I'm really impressed that've managed that. It will be interesting what will go there in the future that way.

HEVC is indeed practically usable by codec support her. Current graphics cards support real-time decoding them, sometimes even -Encoding (but since the quality of bandwidth per my experience can not keep up after).

My test runs here just with CRF 28, and he who comes in the test material out below 1000 kbit / sec.

1000 kbit / sec is the sound barrier, at the time as 640x360 dvdrip xvid with the first versions just went well. With the bandwidth makes X265 now Full HD. And that's on hardware today as soon as the time was on hardware of that time :-)

[Boulder]

Has anyone made any tests with --rd-refine?

[agressiv]

The default psy-rd strength has been increased to 2.0

Curious to the reasoning behind this? This increases file size substantially.

[sneaker_ger]

Raise crf or use --bitrate if you want smaller files.

Reasoning:
psyrd: change default to 2.0, increase range to 5.0

Earlier, high psy-rd caused artifacts, recent changes to mode decisions show an
improved response.
https://bitbucket.org/multicoreware/x265/commits/6f44cd5d00ff423b0e2f4817bc25b96d4191e82b

[Motenai Yoda]

Version 1.9 has been released as new milestone. From the developer mailing list:
...
x265 now allows CRF ratecontrol in pass N (N greater than or equal to 2)
...





but how?
x265 [error]: Constant rate-factor is incompatible with 2pass

ps I think I've found the bottom of the causes, in the check
+ CHECK(param->rc.rateControlMode == X265_RC_CRF && param->rc.bStatRead && param->rc.vbvMaxBitrate == 0,
"Constant rate-factor is incompatible with 2pass");
"param->rc.vbvMaxBitrate" is checked, but vbv is enabled even with only the Level and/or Tier so not specifing the --vbv-maxrate it give error even if level with vbv restrictions is specified.
(but is forced this way only for param.rc.rateControlMode = X265_RC_CRF, when X265_RC_ABR (1 or n pass) only the bitrate is checked, here too, it's checked if more than high/main tier max, but lowered to high tier only even if main.. ???)

so either checking
CHECK(param->rc.rateControlMode == X265_RC_CRF && param->rc.bStatRead && (param->rc.vbvMaxBitrate == 0 && (param->levelIdc == 85 || param->levelIdc == 0)),
"Constant rate-factor is incompatible with 2pass");
or execute this check after enforceLevel()

ps 2 I found that bEnableSlowFirstPass isn't enabled by crf mode and x265_param_apply_fastfirstpass() is called
As I got it, it should do a slow first pass, with output and in a 2nd pass selectivelly reencode only some parts/gops
http://forum.doom9.org/showpost.php?p=1747571&postcount=2922
but It will do a brand new encode again, and with different settings I doubt it will be so reliable

also what should occour when in the 2nd pass the vbvmaxrate is setted much lower than in the 1st one? like 50k and 1k?

[Atak_Snajpera]

Curious to the reasoning behind this? This increases file size substantially.

Better perceived quality in 2-pass mode (limited bitrate budget)
http://forum.doom9.org/showthread.php?p=1750936#post1750936

[forum king]

Hey all :)
needed some input if possible
can any of you great guys tell me these settings are from which encoder or GUI or whatever

wpp / ctu=64 / min-cu-size=8 / max-tu-size=32 / tu-intra-depth=1 / tu-inter-depth=1 / me=1 / subme=2 / merange=57 / no-rect / no-amp / max-merge=2 / temporal-mvp / no-early-skip / rdpenalty=0 / no-tskip / no-tskip-fast / strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / no-fast-intra / open-gop / no-temporal-layers / interlace=0 / keyint=250 / min-keyint=23 / scenecut=40 / rc-lookahead=20 / lookahead-slices=4 / bframes=4 / bframe-bias=0 / b-adapt=2 / ref=3 / limit-refs=3 / no-limit-modes / weightp / no-weightb / aq-mode=1 / qg-size=32 / aq-strength=1.00 / cbqpoffs=0 / crqpoffs=0 / rd=3 / psy-rd=0.30 / rdoq-level=0 / psy-rdoq=0.00 / signhide / deblock / sao / no-sao-non-deblock / b-pyramid / cutree / no-intra-refresh / rc=crf / crf=25.5 / qcomp=0.60 / qpmin=0 / qpmax=51 / qpstep=4 / ipratio=1.40 / pbratio=1.30

the other day just for the sake of making an x265 encode of a video file of mine to convert it for my Note 4 , i tried using these settings in megui x265 encoder , but no matter what i always got an error
at this point i dont know much about the HEVC ,
if its possible then can anyone kindly convert them for megui or suggest a similar preset.


Thanks

Edit : i think this is from ffmpeg

[LigH]

These are the encoding parameters from the inside of x265; they are not 1:1 equal to command line parameters provided to x265.exe from the outside... Just as an example, you can probably group several of these options to preset and tuning meta-parameters, and many will be defaults you could omit from a command line call.

I believe Selur wrote a tool which can calculate from internal parameters back to command lines. But I am not sure how up-to-date with x265 this tool is...

[forum king]

These are the encoding parameters from the inside of x265; they are not 1:1 equal to command line parameters provided to x265.exe from the outside... Just as an example, you can probably group several of these options to preset and tuning meta-parameters, and many will be defaults you could omit from a command line call.

I believe Selur wrote a tool which can calculate from internal parameters back to command lines. But I am not sure how up-to-date with x265 this tool is...

Thanks mate ,
But plz excuse my utter ignorance , can you may be point out which of these can be added to the custom field in megui x265 preset , i dont have a clue whatsoever about which will be accepted in megui and which wont.

would be grateful ,

Thanks

[LigH]

Why this effort? ... Do you want to try to copy options from one encoded movie to another? You should not do this. Your movie may need different options. You may prefer different options as compromise between encoding speed and quality retention. As a beginner, rely on "preset" and "tune" only, add VBV when compatibility to playback devices is required. To use more specific parameters, you really should know why they would be better than the preset defaults.

Sorry to disappoint you, but I will not spend a lot of time analyzing this internal parameter set. I see no use in doing so.

[divxmaster]

@forumking,
I suggest you use cmd.exe instead, I'm not even sure if megui can utilize x265 properly.
You will need to know how to create a .d2v and use ars calculator to work out your SAR.
if not, google it. I am presuming you are using a dvd vob.
I have done a lot of work optimizing for mobile devices, like your note 4.
I use command:
avs4x26x -L x265.exe --crf 28 --psy-rd 0.3 --preset slow --fps %fps% --b-intra --ref 5 --bframes 5 --max-merge 5 --nr-intra 400 --nr-inter 400 --sar %sar% --no-open-gop --min-keyint 23 --keyint 288 --deblock -1:-1 -o "output.hevc" "%source%.avs"

This works very well at super low bit rates. Again this is for mobile phone screen sizes only.
Change %fps%, %sar% and %source% to the correct values first.
If you need to deinterlace first, this is a whole lot more complicated.
--psy-rd 0.3 is for the newer 1.9 version. you don't need 2.0 psy-rd for small screens.

google how to use mp4box to turn the hevc into a mp4. Make sure you use 10bit x265.

Cheers
Divxmaster

[nandaku2]

@Motenai Yoda,

Yes, at present CRF is allowed in N-pass only when VBV is enabled (this condition should have been noted in my 1.9 release notes, apologies). We're working on some ratecontrol modifications which will remove this restriction.

[forum king]

Thanks a lot @Ligh and Divxmaster,
The thing is I was recently gifted a box set of prison break and a couple of miniseries,
A friend of a friend encoded the first episode and it looked awesome on my pc as well as my mobile, so to be able to watch em all on the go, I am trying to do this.
I am aware about megui only, that too just Elementary, that's was seeking help.

The guy who did it had shared the file like a couple of months back, at the time I didn't know how to play the X265, but recently when I tried I was surprised with the ratio. The guy is unreachable since then.

If I knew how to use x265 using cmd or some other application like ffmpeg, I would not have bothered you guys.

Still thanks a lot Divxmaster for the share, god bless mate.

Sent from my LGUS990 using Tapatalk

[Ma]

x265 code in many parts is exactly copy of x264 code. 28 Jan 2009 in x264 appears abs2 function -- http://git.videolan.org/?p=x264.git;a=commitdiff;h=0e43d5d995bb436a63934d70792e481770f406d3

After adopting this function to 10-bit encoding (http://git.videolan.org/?p=x264.git;a=commitdiff;h=8efd67c034190b415174fd03c3cfef4768345f11) it is without changes in x264 and x265 code.

The problem is that it calculates abs wrong. Consider pair (x=-1, y=0). This function returns pair (1, 1) instead of (1, 0).

[littlepox]

It seems v1.9 is less efficient especially when used in dual NUMAs(CPUs) system compared to v1.8. We deployed the same parameters (change in the presets incorporated so that every single parameter is the same), the speed (fps) has dropped by 15%.

Any idea why this happens?

x265 -D 10 --preset slower --crf 16.0 --ctu 32 --max-tu-size 16 --tu-intra-depth 3 --tu-inter-depth 3 --rdpenalty 2 --me 3 --subme 5 --merange 44 --b-intra --no-rect --no-amp --ref 5 --weightb --keyint 360 --min-keyint 1 --bframes 10 --aq-mode 1 --aq-strength 1.1 --rd 5 --psy-rd 0.8 --psy-rdoq 4.0 --rdoq-level 1 --no-sao --no-open-gop --rc-lookahead 80 --scenecut 40 --max-merge 4 --qcomp 0.80 --no-strong-intra-smoothing --input-depth 10 --deblock -2:-2 --qg-size 16 --vbv-bufsize 28000 --vbv-maxrate 25000 --limit-refs 0 --no-limit-modes

[MasterNobody]

x265 code in many parts is exactly copy of x264 code. 28 Jan 2009 in x264 appears abs2 function -- http://git.videolan.org/?p=x264.git;a=commitdiff;h=0e43d5d995bb436a63934d70792e481770f406d3

After adopting this function to 10-bit encoding (http://git.videolan.org/?p=x264.git;a=commitdiff;h=8efd67c034190b415174fd03c3cfef4768345f11) it is without changes in x264 and x265 code.

The problem is that it calculates abs wrong. Consider pair (x=-1, y=0). This function returns pair (1, 1) instead of (1, 0).
It calculates all correct if your interpretation of input is correct (https://mailman.videolan.org/pipermail/x264-devel/2014-December/010918.html)

[Ma]

It calculates all correct if your interpretation of input is correct (https://mailman.videolan.org/pipermail/x264-devel/2014-December/010918.html)

Question in this thread is OK, answer is not.

Function abs2 do:
(x, y) |-> { (abs(x), abs(y)) if x >= 0
{ (abs(x), abs(y+1)) if x < 0

It is crucial for x265 (and x264) because it takes part in SATD computations.

[LoRd_MuldeR]

It seems v1.9 is less efficient especially when used in dual NUMAs(CPUs) system compared to v1.8. We deployed the same parameters (change in the presets incorporated so that every single parameter is the same), the speed (fps) has dropped by 15%.

Any idea why this happens?

x265 -D 10 --preset slower --crf 16.0 --ctu 32 --max-tu-size 16 --tu-intra-depth 3 --tu-inter-depth 3 --rdpenalty 2 --me 3 --subme 5 --merange 44 --b-intra --no-rect --no-amp --ref 5 --weightb --keyint 360 --min-keyint 1 --bframes 10 --aq-mode 1 --aq-strength 1.1 --rd 5 --psy-rd 0.8 --psy-rdoq 4.0 --rdoq-level 1 --no-sao --no-open-gop --rc-lookahead 80 --scenecut 40 --max-merge 4 --qcomp 0.80 --no-strong-intra-smoothing --input-depth 10 --deblock -2:-2 --qg-size 16 --vbv-bufsize 28000 --vbv-maxrate 25000 --limit-refs 0 --no-limit-modes

x265 v1.9 has added new features and re-tuned the presets accordingly. This should give you an improved quality/speed ratio throughout the presets, but you can not assume that the absolute speed (or the absolute quality) is exactly the same as before for each preset. So, I think your test may be kind of misleading. In order to get "meaningful" results, you must either choose settings that give the same speed in v1.8 and v1.9 and then compare the resulting quality (at the same bitrate!), or you must choose settings that give the same quality (at the same bitrate!) in v1.8 and v1.9 and then compare the resulting speed. This is more difficult, yes. But, at the moment, you might be comparing apples and oranges...

[sneaker_ger]

He said he accounted for the preset changes.

Maybe related to that:
x265 version 1.9

- Multi-socket machines now use a single pool of threads that can work cross-socket.
http://x265.org/x265-version-1-9/


1.7 doc:
Default “”, one thread is allocated per detected hardware thread (logical CPU cores) and one thread pool per NUMA node.
http://x265.readthedocs.org/en/1.7/cli.html

current doc:
Default “”, one pool is created across all available NUMA nodes, with one thread allocated per detected hardware thread (logical CPU cores).
http://x265.readthedocs.org/en/default/cli.html

[MasterNobody]

Question in this thread is OK, answer is not.

Function abs2 do:
(x, y) |-> { (abs(x), abs(y)) if x >= 0
{ (abs(x), abs(y+1)) if x < 0

It is crucial for x265 (and x264) because it takes part in SATD computations.
You can compile and run test sample (http://pastie.org/10710948) yourself and result will be:
(-1, 0) -> 0xffffffffffffffff -> 0x0000000000000001

Update
Or you can compile and run test sample (http://pastebin.com/9EEd2nk9) yourself and result will be:
( -2, -2) -> 0xfffffffdfffffffe -> 0x0000000200000002 - > ( 2, 2)
( -2, -1) -> 0xfffffffefffffffe -> 0x0000000100000002 - > ( 2, 1)
( -2, 0) -> 0xfffffffffffffffe -> 0x0000000000000002 - > ( 2, 0)
( -2, 1) -> 0x00000000fffffffe -> 0x0000000100000002 - > ( 2, 1)
( -2, 2) -> 0x00000001fffffffe -> 0x0000000200000002 - > ( 2, 2)
( -1, -2) -> 0xfffffffdffffffff -> 0x0000000200000001 - > ( 1, 2)
( -1, -1) -> 0xfffffffeffffffff -> 0x0000000100000001 - > ( 1, 1)
( -1, 0) -> 0xffffffffffffffff -> 0x0000000000000001 - > ( 1, 0)
( -1, 1) -> 0x00000000ffffffff -> 0x0000000100000001 - > ( 1, 1)
( -1, 2) -> 0x00000001ffffffff -> 0x0000000200000001 - > ( 1, 2)
( 0, -2) -> 0xfffffffe00000000 -> 0x0000000200000000 - > ( 0, 2)
( 0, -1) -> 0xffffffff00000000 -> 0x0000000100000000 - > ( 0, 1)
( 0, 0) -> 000000000000000000 -> 000000000000000000 - > ( 0, 0)
( 0, 1) -> 0x0000000100000000 -> 0x0000000100000000 - > ( 0, 1)
( 0, 2) -> 0x0000000200000000 -> 0x0000000200000000 - > ( 0, 2)
( 1, -2) -> 0xfffffffe00000001 -> 0x0000000200000001 - > ( 1, 2)
( 1, -1) -> 0xffffffff00000001 -> 0x0000000100000001 - > ( 1, 1)
( 1, 0) -> 0x0000000000000001 -> 0x0000000000000001 - > ( 1, 0)
( 1, 1) -> 0x0000000100000001 -> 0x0000000100000001 - > ( 1, 1)
( 1, 2) -> 0x0000000200000001 -> 0x0000000200000001 - > ( 1, 2)
( 2, -2) -> 0xfffffffe00000002 -> 0x0000000200000002 - > ( 2, 2)
( 2, -1) -> 0xffffffff00000002 -> 0x0000000100000002 - > ( 2, 1)
( 2, 0) -> 0x0000000000000002 -> 0x0000000000000002 - > ( 2, 0)
( 2, 1) -> 0x0000000100000002 -> 0x0000000100000002 - > ( 2, 1)
( 2, 2) -> 0x0000000200000002 -> 0x0000000200000002 - > ( 2, 2)

[Ma]

You can compile and run test sample (http://pastie.org/10710948) yourself and result will be:
(-1, 0) -> 0xffffffffffffffff -> 0x0000000000000001

Thanks for this sample. If you change line 29 in your sample from sign-extended cast to zero-extended cast:
sum2_t sum2 = (sum2_t)(sum_t)x + ((sum2_t)y << BITS_PER_SUM);

it will be the case which I consider.

The problem is: what case is in x264/x265 code? Line 200 from file common/pixel.cpp (x265) is:
a0 = (pix1[0] - pix2[0]) + ((sum2_t)(pix1[4] - pix2[4]) << BITS_PER_SUM);
which is zero-extended cast.

Then it goes 2 times to HADAMARD4 macro. Now I don't know if it is OK or not -- I will look into it.

[MasterNobody]

The problem is: what case is in x264/x265 code? Line 200 from file common/pixel.cpp (x265) is:
a0 = (pix1[0] - pix2[0]) + ((sum2_t)(pix1[4] - pix2[4]) << BITS_PER_SUM);
which is zero-extended cast.
No. It is the same cast as in my code int -> sum2_t which is sign extended (because cast from signed int). And we get this int because we promote both pix1[0] and pix2[0] to (signed) int from pixel type.

And yes your addition of cast to sum_t will break it because you at first sign-extend it to sum_t and only then zero-extend (because sum_t is unsigned) to sum2_t.

[Ma]

No. It is the same cast as in my code int -> sum2_t which is sign extended (because cast from signed int). And we get this int because we promote both pix1[0] and pix2[0] to (signed) int from pixel type.

Thanks for explanation. I wrongly assumed that (pix1[0] - pix2[0]) is unsigned int.

[pradeeprama]

It seems v1.9 is less efficient especially when used in dual NUMAs(CPUs) system compared to v1.8. We deployed the same parameters (change in the presets incorporated so that every single parameter is the same), the speed (fps) has dropped by 15%.

Any idea why this happens?

x265 -D 10 --preset slower --crf 16.0 --ctu 32 --max-tu-size 16 --tu-intra-depth 3 --tu-inter-depth 3 --rdpenalty 2 --me 3 --subme 5 --merange 44 --b-intra --no-rect --no-amp --ref 5 --weightb --keyint 360 --min-keyint 1 --bframes 10 --aq-mode 1 --aq-strength 1.1 --rd 5 --psy-rd 0.8 --psy-rdoq 4.0 --rdoq-level 1 --no-sao --no-open-gop --rc-lookahead 80 --scenecut 40 --max-merge 4 --qcomp 0.80 --no-strong-intra-smoothing --input-depth 10 --deblock -2:-2 --qg-size 16 --vbv-bufsize 28000 --vbv-maxrate 25000 --limit-refs 0 --no-limit-modes

Is this on a server or on a desktop? If it is a multi-socketed server, can you try to explicitly specify --pools N,N where N is the # threads per socket if you're running on a dual-socket machine. In some faster presets with some videos, we've seen that explicitly setting the # pools and threads per pool helps performance; we launch a single pool with combined threads by default.

Also, while your command line calls preset slower, you seem to be turning off pretty much all features that are available in the slower presets to make it efficient (rect, amp, sao, reduced rdoq level). Perhaps you want to move to use a faster default preset that is more representative of the command line you're using? It looks like you're looking at something faster than ultrafast actually!

[x265_Project]

FYI - Pradeeprama is a manager on the x265 development team. Pradeep's team is responsible for testing, platform and performance optimization.

[littlepox]

Is this on a server or on a desktop? If it is a multi-socketed server, can you try to explicitly specify --pools N,N where N is the # threads per socket if you're running on a dual-socket machine. In some faster presets with some videos, we've seen that explicitly setting the # pools and threads per pool helps performance; we launch a single pool with combined threads by default.

Also, while your command line calls preset slower, you seem to be turning off pretty much all features that are available in the slower presets to make it efficient (rect, amp, sao, reduced rdoq level). Perhaps you want to move to use a faster default preset that is more representative of the command line you're using? It looks like you're looking at something faster than ultrafast actually!

The speed reduction is seen BOTH in desktop(4790K) and server(E5 2683v3 dual).
Much worse on the server, some times x265 only consumes one NUMA node, with another sleeping...It has been tested that with --pools "N,N" the issue can be solved, but still slower than v1.8+2

BTW, I'm not targeting ultrafast, I'm using --ref 5 --me star --subme 5 --rd 5. The parameters are heavily modified to fit high-quality 1080p anime encoding. rect and amp are disabled for being little contributing and (previously) too slow to bear(we are considering to bring them back recently with --limit-refs and --limit-mode, but at the beginning of the test we spotted the reduction on speed). sao and rdoq are tweaked due to visual quality adjustment.

[chenm001]

Thanks for explanation. I wrongly assumed that (pix1[0] - pix2[0]) is unsigned int.

http://en.cppreference.com/w/cpp/language/implicit_cast

(pix1[0] - pix2[0]) ---> (uint16_t - uint16_t) ---> (int32 - int32)

the final convert to sum2_t since second part is sum2_t

[x265_Project]

FYI - chenm001 is an architect on the x265 development team.

[Ma]

It seems v1.9 is less efficient especially when used in dual NUMAs(CPUs) system compared to v1.8. We deployed the same parameters (change in the presets incorporated so that every single parameter is the same), the speed (fps) has dropped by 15%.

I've checked speed of encoding on i5 3450S CPU (AVX level). The speed drop is from 7.36 to 6.73 fps -- about 9%. But the result of encoding is different -- you can check bitrate, PSNR and SSIM (in attached screen.txt). It is slower but the quality is better.

[littlepox]

I've checked speed of encoding on i5 3450S CPU (AVX level). The speed drop is from 7.36 to 6.73 fps -- about 9%. But the result of encoding is different -- you can check bitrate, PSNR and SSIM (in attached screen.txt). It is slower but the quality is better.

Thanks for verifying this for me.
This explanation makes sense, if they have improved the logic but code not as optimized as v1.8.

We are still left with the issue for multi-NUMA systems to worry.

[nandaku2]

Thanks for verifying this for me.
This explanation makes sense, if they have improved the logic but code not as optimized as v1.8.

We are still left the issue for multi-NUMA system to worry.

Please turn on limit-refs. You will retain most of the improved quality, and greatly improve speed.

[LigH]

I contacted a person who has access to a dual Xeon, I hope to get a reply soon...

[pradeeprama]

The speed reduction is seen BOTH in desktop(4790K) and server(E5 2683v3 dual).
Much worse on the server, some times x265 only consumes one NUMA node, with another sleeping...It has been tested that with --pools "N,N" the issue can be solved, but still slower than v1.8+2

This is strange. On our dual-Xeon E5-2699v3, I see nearly a 50% & 100% improvement in 4K encode speed at the default veryslow and ultrafast presets, respectively. On the 4790K, the improvement is a good 40-50%. These numbers are averaged across typical open-source videos. I wonder if it is something specific in your video, or settings that is causing such a big dip. Can you share your video with us?

Also, when you say that "some times x265 consumes only one NUMA node while the other is sleeping", is that in the middle of the execution or all the time? There was some initial problem with the implementation on Windows platform that was causing us to use only one NUMA node, but this was later fixed and I haven't seen any issues since then.

BTW, I'm not targeting ultrafast, I'm using --ref 5 --me star --subme 5 --rd 5. The parameters are heavily modified to fit high-quality 1080p anime encoding. rect and amp are disabled for being little contributing and (previously) too slow to bear(we are considering to bring them back recently with --limit-refs and --limit-mode, but at the beginning of the test we spotted the reduction on speed). sao and rdoq are tweaked due to visual quality adjustment.

Thanks for the clarification here - I missed some of your other parameters. Limit-refs and modes will considerably help you out as you have identified already

[littlepox]

This is strange. On our dual-Xeon E5-2699v3, I see nearly a 50% & 100% improvement in 4K encode speed at the default veryslow and ultrafast presets, respectively. On the 4790K, the improvement is a good 40-50%. These numbers are averaged across typical open-source videos. I wonder if it is something specific in your video, or settings that is causing such a big dip. Can you share your video with us?

uhmmm... I don't think I'm supposed to share them, since we are trying to backup some anime BDs. But the source should not be the key point. If you wish to test, find some fine-quality 1080p Japanese anime clips, or other lightly noisy 1080p film clips, those should do.

You observe the efficiency increase because you changed the default preset settings(ref, limit-refs, limit-modes), but here I have override every one of them, so that effectively you do NOT benefit from the change and force the comparison with exactly the same parameters.

As tested by Ma (http://forum.doom9.org/showthread.php?p=1756487#post1756487), It could be the logic is different so that you require more computations to reach a better quality, and the new codes have not been heavily optimized. If that's the case, it shall be fine.



Also, when you say that "some times x265 consumes only one NUMA node while the other is sleeping", is that in the middle of the execution or all the time? There was some initial problem with the implementation on Windows platform that was causing us to use only one NUMA node, but this was later fixed and I haven't seen any issues since then.


This happens randomly, (sometimes it utilizes both NUMA nodes, but other time it uses only one.) but if you are of bad luck, the asleep NUMA node sleeps all the time.

We are testing that on Win10. I'm asking my friend to do a few testing on Win7, hopefully I shall update this post later.
Confirmed, this is seen on windows 7 as well.

We are using the builds from http://www.msystem.waw.pl/x265/ , Stable branch VS 2015 clean builds, AVX2

[LigH]

I contacted a person who has access to a dual Xeon, I hope to get a reply soon...

May take a little longer, he would have to build for MacOS X.

[Ma]

Thanks for verifying this for me.
This explanation makes sense, if they have improved the logic but code not as optimized as v1.8.

We are still left with the issue for multi-NUMA systems to worry.

I can't help with multi-NUMA systems directly (I don't have any), but you could provide more detailed info.

I've compiled stable 1.8 & 1.9 version with '-DDETAILED_CU_STATS=ON' option:
www.msystem.waw.pl/x265/x265-1.8+2-1f0d4de-stable_vs2015-AVX2-detailed.7z
www.msystem.waw.pl/x265/x265-1.9+2-ee38630-stable_vs2015-AVX2-detailed.7z

You could use these versions with additional option '--log-level debug' on multi-NUMA system (encoding the same short sample) and then copy console window to clipboard (Right Click -> Select All -> ENTER). I think that it will be much easier for x265 team to find the source of this problem with detailed info.

[littlepox]

I can't help with multi-NUMA systems directly (I don't have any), but you could provide more detailed info.

I've compiled stable 1.8 & 1.9 version with '-DDETAILED_CU_STATS=ON' option:
www.msystem.waw.pl/x265/x265-1.8+2-1f0d4de-stable_vs2015-AVX2-detailed.7z
www.msystem.waw.pl/x265/x265-1.9+2-ee38630-stable_vs2015-AVX2-detailed.7z

You could use these versions with additional option '--log-level debug' on multi-NUMA system (encoding the same short sample) and then copy console window to clipboard (Right Click -> Select All -> ENTER). I think that it will be much easier for x265 team to find the source of this problem with detailed info.


Thank you for all this, and here is the test results(.csv with logging infomation):

http://1drv.ms/20m04GC

1.8_avx2 and 1.9_avx2 are tested with Core i7 4790K
speed reduction is about 5% (really small in this test...)

no_pools and pools_++ are tested with dual E5v3 (12C24T * 2)
no_pools is of the case that uses only one NUMA node, and pools_++ means we used --pools "+,+" to force it work properly.

the speed is about 2x difference.

Hope the above test result shall help the x265 team.

[Ma]

no_pools and pools_++ are tested with dual E5v3 (12C24T * 2)
no_pools is of the case that uses only one NUMA node, and pools_++ means we used --pools "+,+" to force it work properly.

the speed is about 2x difference.

Weird. Maybe it is only Windows specific behavior. I assume that x265 1.8 without specify any pools option works similar to ver. 1.9 with '--pools +,+' option?

[littlepox]

Weird. Maybe it is only Windows specific behavior. I assume that x265 1.8 without specify any pools option works similar to ver. 1.9 with '--pools +,+' option?

Yes. v1.8 works quite OK without specifying --pools.

[pingfr]

I must say I have a similar experience with a 5960X system (8c/16t) where I see x265.exe not even using 60% of the CPU "raw computing power".

Is this an intended behaviour or is there a way to force x265 aggressively on the 5960X hoping to achieve more fps crunching?

Edit: More details should they matter:

My x265 version:
x265 [info]: HEVC encoder version 1.9+9-8e093e85b9ab
x265 [info]: build info [Windows][GCC 5.3.0][64 bit] 8bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZ CNT BMI2

My command line:
avs4x265.exe -P x265.exe --psy-rd 2 --tune grain --crf 17 --preset veryslow --output %~n1.hevc %1

[Ma]

Yes. v1.8 works quite OK without specifying --pools.

I decided to find suspect commits that could be the cause of the problem. I found only one: 10983 -- https://bitbucket.org/multicoreware/x265/commits/e1adac00dce8e5641cbe9aec3d50a72261c308d9

It is my old enemy and this time I've killed it immediately. I've compiled a special build of ver. 1.9+2 stable -- without this one commit. Could you test this special build on multi-NUMA system?
www.msystem.waw.pl/x265/x265-1.9+2-ee38630-stable-10983_vs2015-AVX2.7z

[nevcairiel]

One guess would be that its designed around libNUMA and the Windows version of that code is just missing some smarter logic.
More specifically, from a quick look at the linked commit it requires a build targeted at Windows 7 and newer at compile time to even make use of the special windows logic (so no XP compat builds), would probably be smarter to dynamically load those functions when available so builds are more generically useful.

[pingfr]

@Ma: Tested the 1.9+2 stable you posted a few minutes ago, saw no noticeable changes on my end but then again my CPU despite being 8c/16t is a single physical CPU system.

[Ma]

I must say I have a similar experience with a 5960X system (8c/16t) where I see x265.exe not even using 60% of the CPU "raw computing power".

It could be normal -- you have 8 cores and x265 uses on your CPU 16 threads. All threads wants 100% computing power all the time, so the result should be about 50% (it is a bit better due to multithread CPU).

I'm curious what's happen when you add '--pools 8' options. It will be 8 threads on 8 cores with about 100% computing power or 8 threads on 4 cores with about 60% computing power?

[pingfr]

@Ma: with a --pools 8, it's even worse, the cores are used at 40% of their capacity only.

[Ma]

@Ma: with a --pools 8, it's even worse, the cores are used at 40% of their capacity only.

Do you refer to "detailed CU info" like in this example:
encoded 504 frames in 26.41s (19.09 fps), 4083.78 kb/s, Avg QP:37.82
x265 [info]: CU: %41.14 time spent in motion estimation, averaging 8.288 CU inter modes per CTU
x265 [info]: CU: Skipped motion searches per depth %8.78 %11.86 %19.99 %0.00
x265 [info]: CU: %02.60 time spent in intra analysis, averaging 5.693 Intra PUs per CTU
x265 [info]: CU: Skipped intra CUs at depth %-nan(ind) %63.49 %120.95
x265 [info]: CU: %22.33 time spent in inter RDO, measuring 19.088 inter/merge predictions per CTU
x265 [info]: CU: %05.56 time spent in intra RDO, measuring 16.963 intra predictions per CTU
x265 [info]: CU: %02.40 time spent in loop filters, average 0.381 ms per call
x265 [info]: CU: %00.12 time spent in weight analysis, average 0.926 ms per call
x265 [info]: CU: %15.49 time spent in slicetypeDecide (avg 96.802ms) and prelookahead (avg 2.980ms)
x265 [info]: CU: %10.36 time spent in other tasks
x265 [info]: CU: Intra RDO time per depth %00.00 %34.78 %18.75 %46.48
x265 [info]: CU: Intra RDO calls per depth %00.00 %03.17 %07.70 %89.13
x265 [info]: CU: Inter RDO time per depth %41.67 %32.64 %14.59 %11.09
x265 [info]: CU: Inter RDO calls per depth %09.58 %19.54 %25.49 %45.39
x265 [info]: CU: 120960 64X64 CTUs compressed in 95.965 seconds, 1260.460 CTUs per worker-second
x265 [info]: CU: 3.634 average worker utilization, %90.85 of theoretical maximum utilization
which on my i5 3450S CPU (4c/4t) is 90.85% or you observe "Task Manager" and you see about 40% on all 16 logical cores?
-----------------
I've tested '--pools 2' option on my 4c/4t CPU. Win 7 schedules this 2 threads on all 4 cores. So I assume that Windows is able to optimize threads on one CPU but probably can't/doesn't want move threads between CPU nodes.

[pingfr]

@Ma: I'm running Windows 7, using Core Temp gadget on the side which shows me the CPU usage and the temperature for each core.

During x265 usage with --preset veryslow, I see my temperatures rising as high as 79°C but the load per core is only roughly 60% each.

Thus I'm wondering if there's a parameter to force x265 to saturate cores to their full extent and therefore crunch fps faster maybe?

[pradeeprama]

I decided to find suspect commits that could be the cause of the problem. I found only one: 10983 -- https://bitbucket.org/multicoreware/x265/commits/e1adac00dce8e5641cbe9aec3d50a72261c308d9

It is my old enemy and this time I've killed it immediately. I've compiled a special build of ver. 1.9+2 stable -- without this one commit. Could you test this special build on multi-NUMA system?
www.msystem.waw.pl/x265/x265-1.9+2-ee38630-stable-10983_vs2015-AVX2.7z

Thanks for the break-down and the extensive testing. Even on the linux box, I've noticed that for some clips, explicitly specifying a --pools option gave much higher performance than not specifying the --pools version. However, through extensive testing on several machines, we noticed that not specifying the --pools option was by-and-large better and hence we decided to change the default behavior. The difference was primarily governed by what the work per thread to communication overhead balance was.

If specifying the --pools option gives you better performance, you may include it in your command line; the behavior of the thread pools will then go back to how it was in the versions under 1.8.

[pradeeprama]

@Ma: with a --pools 8, it's even worse, the cores are used at 40% of their capacity only.

Specifying --pools 8 in the command line for x265 on an 8c16t machine will result in using only 4 cores; each core will have 2 threads.

[Ma]

Now I'm lost. In ver. 1.9 source code I tried to find difference between not using pools option and '--pools +,+' option and it looks like there is no difference.

In littlepox data it is clear that the speed is only 50% without pools option.

@pradeeprama could you point what is different with '--pools +,+' option in source code?

[pradeeprama]

Now I'm lost. In ver. 1.9 source code I tried to find difference between not using pools option and '--pools +,+' option and it looks like there is no difference.

In littlepox data it is clear that the speed is only 50% without pools option.

@pradeeprama could you point what is different with '--pools +,+' option in source code?

@Ma, you are right - not using the --pools in the command line should be identical to using --pools +,+ for a system with 2 nodes; both should create one pool with # worker threads = sum of available logical cpus across both nodes.

@Littlepox - I'd misunderstood your problem earlier, my apologies. I saw your logs again and you can see that in both your runs, x265 creates one pool with 48 threads. Is there some RAM bottleneck in the system when you run for the first time because of which when you don't give --pools in the command line, you're working out of the disk and not the RAM? If you run the identical command line 3-4 times, do you consistently see a much lower FPS?

[pingfr]

Could any of you tell me which --pools option(s) should I use on a 8c/16t machine with 32GB of RAM?

Would be appreciated.

[pradeeprama]

Could any of you tell me which --pools option(s) should I use on a 8c/16t machine with 32GB of RAM?

Would be appreciated.

By default x265 should be able to find all 16 threads and use them for encoding; look at the console log for identifying how many threads were created. You shouldn't have to specify anything explicit.

[pingfr]

@pradeerprama: I see the console spouting out that is using 16 threads indeed but my concern is when looking in depths, the 8 cores are barely used/loaded with a load never topping 60% each core.

Is there a way to set the process to be more aggressively crunching fps?

[pradeeprama]

@pradeerprama: I see the console spouting out that is using 16 threads indeed but my concern is when looking in depths, the 8 cores are barely used/loaded with a load never topping 60% each core.

Is there a way to set the process to be more aggressively crunching fps?

I see that you're using the slower preset. Maybe you want to consider a quicker preset if you want higher fps? Of course, it is an efficiency trade-off though.

[pingfr]

@pradeeprama: I am using that preset because I am doing Blu-Ray source "archival". I am actually using the --preset veryslow at the moment. Picture quality is my top notch priority over anything else.

Off-topic question while you're here: I'm passing the --psy-rd 2 parameter, my question is: would passing --psy-rd 3 or --psy-rd 4 have any effect on perceptible quality when used with the veryslow preset?

[littlepox]

@Ma

My friend tested your new compile and it's functioning very well.


@pradeerprama

I do NOT think there are any bottlenecks. Nevertheless I'm requesting my friend to redo the test for a few more times, hopefully I shall update this post later.

Update: It is confirmed that the loss of speed and "sleeping" of one NUMA node is with HIGH probability, it's like >50% of chance we see it on native v1.9+2, WITHOUT --pools +,+

[Boulder]

@pradeeprama: are there any expected changes in rd-refine in the near future? When I posted a bug report at BitBucket, one comment mentioned that there are some enhancements to do as you were not yet happy with the output. I've put my x265 encodes on hold as rd-refine at least makes smaller files, and warrants further testing. I've not seen any test reports by anyone so far.

[Ma]

@Ma

My friend tested your new compile and it's functioning very well.

Thanks for this info. [...]

[littlepox]

Thanks for this info. Now we can try to shoot directly to the enemy.
I've compiled ver. 1.9+2 with one small change:
diff -r ee38630033b7 source/encoder/encoder.cpp
--- a/source/encoder/encoder.cpp Fri Feb 05 10:49:41 2016 +0530
+++ b/source/encoder/encoder.cpp Thu Feb 11 10:24:39 2016 +0100
@@ -102,6 +102,7 @@
}

bool allowPools = !p->numaPools || strcmp(p->numaPools, "none");
+ Sleep(500);

// Trim the thread pool if --wpp, --pme, and --pmode are disabled
if (!p->bEnableWavefront && !p->bDistributeModeAnalysis && !p->bDistributeMotionEstimation && !p->lookaheadSlices)

Could you test this new build?
www.msystem.waw.pl/x265/x265-1.9+2-ee38630-stable-sleep_vs2015-AVX2.7z

For this version, without specifying --pools, there is still the problem of speed reduction and an asleep NUMA node. all 48 threads are squeezed into 1 NUMA node, and the other not taking up computation.

log says "x265 [info]: Thread pool 0 using 48 threads on numa nodes 0,1"


however, with --pools, it works fine, two NUMA nodes share the work equally, and the log changed to:

x265 [info]: Thread pool 0 using 24 threads on numa nodes 0
x265 [info]: Thread pool 1 using 24 threads on numa nodes 1

[littlepox]

It seems a bit messy right now, I shall re-state the problem:

x265 1.8 works fine, the log says:

x265 [info]: Thread pool 0 using 24 threads on numa nodes 0
x265 [info]: Thread pool 1 using 24 threads on numa nodes 1


WITH PLAIN x265 1.9 build:

specifying --pools 24,24 all 48 threads are allocated into two thread pools, and the log says:
x265 [info]: Thread pool 0 using 24 threads on numa nodes 0
x265 [info]: Thread pool 1 using 24 threads on numa nodes 1



NOT specifying --pools 24,24 all 48 threads are allocated into one thread pool(reduction of speed), and the log says:
x265 [info]: Thread pool 0 using 48 threads on numa nodes 0,1

With MA's build: (http://forum.doom9.org/showthread.php?p=1756612#post1756612)

specifying --pools 24,24 all 48 threads are allocated into two thread pools, and the log says:
x265 [info]: Thread pool 0 using 24 threads on numa nodes 0
x265 [info]: Thread pool 1 using 24 threads on numa nodes 1

HOWEVER , the previous csv file says "x265 [info]: Thread pool 0 using 48 threads on numa nodes 0,1". The reason is unknown(may be wrong copy-paste, may be different behavior), here we've re-tested a few times so in this post whatever reported shall be correct.


NOT specifying --pools 24,24 all 48 threads are allocated into one thread pool(reduction of speed), and the log says:
x265 [info]: Thread pool 0 using 48 threads on numa nodes 0,1


In summary for x265 1.9:

with --pools 24,24:
x265 [info]: Thread pool 0 using 24 threads on numa nodes 0
x265 [info]: Thread pool 1 using 24 threads on numa nodes 1
http://img.2222.moe/images/2016/02/11/fixed.png



without --pools 24,24:
x265 [info]: Thread pool 0 using 48 threads on numa nodes 0,1
BUT numa node 1 is not utilized:
http://img.2222.moe/images/2016/02/11/wrong.png

BTW, since my friend uses such a powerful dual-socket computer, the encoding, even with --pools +,+, takes only ~50% of total CPU power. so It may be possible that x265/windows decide to allocate the work into one NUMA node and leave the other not involved. but the problem is that this strategy reduces speed.

[pingfr]

BTW, since my friend uses such a powerful dual-socket computer, the encoding, even with --pools +,+, takes only ~50% of total CPU power. so It may be possible that x265/windows decide to allocate the work into one NUMA node and leave the other not involved. but the problem is that this strategy reduces speed.

And despite my 5960X being single socket computer, I have exactly the same issue here (sorry to interfere in your posts).

My 8 cores are never used at more than 60% of their CPU power.

It would make sense that owners of "High-end machines" would want to harness the full power of their systems for x265 encoding and this is clearly not the case here.

[Ma]

For this version, without specifying --pools, there is still the problem of speed reduction and an asleep NUMA node. all 48 threads are squeezed into 1 NUMA node, and the other not taking up computation.

log says "x265 [info]: Thread pool 0 using 48 threads on numa nodes 0,1"


however, with --pools, it works fine, two NUMA nodes share the work equally, and the log changed to:

x265 [info]: Thread pool 0 using 24 threads on numa nodes 0
x265 [info]: Thread pool 1 using 24 threads on numa nodes 1

The second output is for '--pools 24,24' option or '--pools +,+' option? I try to understand what's going on with '--pools +,+' vs. without pools encoding.

[littlepox]

The second output is for '--pools 24,24' option or '--pools +,+' option? I try to understand what's going on with '--pools +,+' vs. without pools encoding.

epic fail...He found he was testing with pools 24,24 except for the one with csv output.

I'm requesting him to retest everything with pools ++.

Currently you can assume pools 24 24 gives u desired speed.

retesting with pools ++, the behavior is similar to without. the speed in BOTH cases are highly volatile, and all threads are squeezed into one NUMA nodes, significantly slower than pools 2424

[LigH]

My 8 cores are never used at more than 60% of their CPU power.

I believe this has been an issue for longer already, and the reason is the rather high level of dependencies between the threads, which is already known to be more a problem for HEVC than for AVC. It used to be a recommendation already months ago to rather run two conversions in parallel than to hope for a 100% utilized single conversion with many cores.

At least so I remember.

Issues with the balance of NUMA pools on multi-socket systems are technically a really different matter.

[Ma]

epic fail...He found he was testing with pools 24,24 except for the one with csv output.

I'm requesting him to retest everything with pools ++.

Currently you can assume pools 24 24 gives u desired speed.

retesting with pools ++, the behavior is similar to without. the speed in BOTH cases are highly volatile, and all threads are squeezed into one NUMA nodes, significantly slower than pools 2424

OK, so in csv output it was only unusual speedup for '--pools +,+' option. I think that it is the case which nevcairiel mentioned -- Linux is smarter than Windows in thread scheduling so we need separate logic for Linux (without change) and for Windows (no pools option and '--pools +,+' should act as '--pools 24,24' on your system).

[littlepox]

@Ma

Thanks for the explanation, now it seems we can just use the v1.9 you compiled without the commit, or manually add --pools "N,N" in the CLI. The rest shall be simple as you said to use separated logic on Win/Linux.


@pingfr

As far as from our experience, on a E5v3 12C24T, encoding 1080p with x265 generally consumes >95% usage, so it's strange for your case. A few reasons maybe:

1. How did you get the source? x265 can only encode at the rate it is given, if you are piping in a low speed (typically with complex avisynth scripts), this shall be expected.
2. what is the resolution you are coping with? if it is <=720p, this shall be explained as there are less work to be distributed.

[pingfr]

@littlepox: yes, the source is actually an already HEVC compressed stream 720p clip grabbed off youtube.

Are you implying that if I get a Blu-Ray 1080p raw uncompressed source to encode piped through avisynth then I might see an increased CPU usage/load?

[littlepox]

@littlepox: yes, the source is actually an already HEVC compressed stream 720p clip grabbed off youtube.

Are you implying that if I get a Blu-Ray 1080p raw uncompressed source to encode piped through avisynth then I might see an increased CPU usage/load?

Yes. under 1080p, there are more than 2x works to do per frame encode. with 720p, you probably have two choices:

1. run multiple instances, this is what we do when we are with >12C24T.
2. modify your settings to play-around with some parameters that could be bottleneck, unable to do multi-threading. As far as I can see, these settings shall help you out with little sacrifice or even better quality under 720p (remember x265 is tuned for 4K, so you do not need to follow exactly even for quality purposes):

--ctu 32 --lookahead-slices 4 --me-range 25

[pingfr]

@littlepox: I think I'll stay with the --ctu 64 --lookahead-slices 1 and --me-range 57 from the default's veryslow profile for now.

I'll just find a 1080p source material to encode tonight or something.

Will return and post any new findings as soon as I have anything new to share with you guys.

[pradeeprama]

@Ma

Thanks for the explanation, now it seems we can just use the v1.9 you compiled without the commit, or manually add --pools "N,N" in the CLI. The rest shall be simple as you said to use separated logic on Win/Linux.



Thanks for the extensive testing @littlepox & Ma.

In conclusion, it looks like using --pools +,+ with the x265+1.9 build spawns threads on both nodes, but the second node isn't seeing a lot of utilization. If you use --pools 24,24, again threads are spawned in both nodes and you explicitly see much higher utilization.

If this is correct, then the issue is one of the Windows scheduler not being intelligent enough to schedule things correctly. I will take a look at the Windows API again to see if there are some hints we can provide to make the scheduling better.

If this is not the problem, then I am lost in the messages :-(.

Also, @littlepox, if you're adding --pools 24,24 to your command line explicitly, there is no need to go for the binary that Ma generated - x265's default 1.9 binary should work just fine for you.

[littlepox]

Thanks for the extensive testing @littlepox & Ma.

In conclusion, it looks like using --pools +,+ with the x265+1.9 build spawns threads on both nodes, but the second node isn't seeing a lot of utilization. If you use --pools 24,24, again threads are spawned in both nodes and you explicitly see much higher utilization.

If this is correct, then the issue is one of the Windows scheduler not being intelligent enough to schedule things correctly. I will take a look at the Windows API again to see if there are some hints we can provide to make the scheduling better.

If this is not the problem, then I am lost in the messages :-(.

Also, @littlepox, if you're adding --pools 24,24 to your command line explicitly, there is no need to go for the binary that Ma generated - x265's default 1.9 binary should work just fine for you.

Exactly as you said. Also, we like to use MA's build because we do NOT need to add anything, it can automatically work for all hardware.

This is what we are expecting for improvements: x265 should always choose the way to maximize utility with other things equal; please find some way to cope with the windows API, if not, separation of the logic may be a solution.

[Ma]

In conclusion, it looks like using --pools +,+ with the x265+1.9 build spawns threads on both nodes, but the second node isn't seeing a lot of utilization. If you use --pools 24,24, again threads are spawned in both nodes and you explicitly see much higher utilization.

If this is correct, then the issue is one of the Windows scheduler not being intelligent enough to schedule things correctly. I will take a look at the Windows API again to see if there are some hints we can provide to make the scheduling better.

My understanding is: '--pools +,+' and no pools options works the same -- once fast on both nodes, once slow on one node (but more often slow); '--pools 24,24' options works fast on both nodes every time.

My feelings are: if Windows starts on both nodes it is fast to the end of encoding, if Windows starts on one node it is slow to the end of encoding -- maybe Windows doesn't want to move thread from one node to another if the thread is at 100% workload.

[Ma]

without --pools 24,24:
x265 [info]: Thread pool 0 using 48 threads on numa nodes 0,1
BUT numa node 1 is not utilized:
http://img.2222.moe/images/2016/02/11/wrong.png

At the time when this screenshot was taken was there any heavy process running (beside x265)? It looks like there are 6 to 10 threads on numa node 1 (beautifully distributed only at physical cores).

[littlepox]

At the time when this screenshot was taken was there any heavy process running (beside x265)? It looks like there are 6 to 10 threads on numa node 1 (beautifully distributed only at physical cores).

No, not any.

[Ma]

No, not any.

Thanks for info. It could be the case that "it's not a bug, it's a feature". x265 code looks OK, Windows schedules threads on both numa nodes but node 0 gets more threads (for example 42 threads to node 0 and 6 threads to node 1). x265 in that case works slow.
-------------
I think that the best solution is to leave threadpool code as is and only to change logic of allocating threads on numa nodes. First attempt in attachment, VS 2015 build of stable version 1.9+3 with new logic (only for Windows OS):
www.msystem.waw.pl/x265/x265-vs2015-stable-1.9+3.7z

[pradeeprama]

Thanks for info. It could be the case that "it's not a bug, it's a feature". x265 code looks OK, Windows schedules threads on both numa nodes but node 0 gets more threads (for example 42 threads to node 0 and 6 threads to node 1). x265 in that case works slow.
-------------
I think that the best solution is to leave threadpool code as is and only to change logic of allocating threads on numa nodes. First attempt in attachment, VS 2015 build of stable version 1.9+3 with new logic (only for Windows OS):
www.msystem.waw.pl/x265/x265-vs2015-stable-1.9+3.7z

Thanks for the patch Ma. I see that you're trying to resort --pools +,+ to effectively mean --pools 24,24 (or whatever the # threads per socket is) on Windows. For this use-case, I'm sure it'll work better but I'm not sure if this is universally applicable across other videos/machines. We will do more testing on a windows multi-socket system and report back with what we see. I am also a little uncomfortable keeping one meaning of --pools +,+ on linux and another on windows.

[LigH]

What exactly is the changed condition to enable 2-pass CRF? If I read the patches correctly: Use a VBV maximum bitrate in the 1st pass, so you can either let x265 calculate the optimal RF with VBV constraints to achieve a target size (in VBR mode), or use a specified RF with VBV constraints regardless of a target size (in VBV-CRF mode)?
__

New build with changed 2-pass VBV-CRF conditions:

x265 1.9+15-425b583f25db (https://www.mediafire.com/download/8f06cr79fe2besb/x265_1.9+15-425b583f25db.7z) (GCC 5.3.0)

[Ma]

I see that you're trying to resort --pools +,+ to effectively mean --pools 24,24 (or whatever the # threads per socket is) on Windows.

Yes, for '--pools +,+' and '--pools *' and no pools options it simulates '--pools 24,24' option. I think it is simple and more effective than current code and better than backout commit 10983.

I hope that you find better fix.

[qyot27]

I apologize for not following up on this much earlier, but this is a basic rundown of the issues when cross-compiling x265-yuuki currently:

input-filters.diff doesn't cleanly apply because the amount of fuzz got too large.

CMake has to be run twice to even detect that -DENABLE_LAVF has been turned on and get the locations for the libs. This is due to the position of the ENABLE_LAVF block in CMakeLists.txt
coming before the ENABLE_CLI block, but requiring the CLI is enabled. If the ENABLE_LAVF block is moved to a position after the CLI is turned on, or if the CLI requirement is removed
from the ENABLE_LAVF block, then it can detect the libs on the first try (although it seemed to cause other issues during building or linking).

There are problems linking against FFmpeg's private libs, especially when all of them are built static. zlib, bzip2, lzma, and iconv are all autodetected and used if present when FFmpeg is configured,
and these cause errors on link with the lavf module because the libs haven't been properly added to the linker list by cmake. To say nothing of when more expansive configurations with more
external libs are present, or for other options that are enabled by default (like what's brought in if the user doesn't explicitly use --disable-network). If the user explicitly disables these four libraries
and the network support when building FFmpeg, then x265 can link to them just fine, but otherwise it requires the user to manually append the list of linker libs to the failed link command, and this
then breaks the ability of ninja to do a proper install, even though the .exe is okay.

FFmpeg API usage needs to be updated. PIX_FMT_* needs to be changed to AV_PIX_FMT_* to allow compilation to proceed successfully, and even though it's just a warning, uses of deprecated
functions should be updated to non-deprecated ones (which x264 recently fixed for the lavf module there):

[78/88] Building CXX object CMakeFiles/cli.dir/input/lavf.cpp.obj
/home/qyot27/x265-build/x265-yuuki/source/input/lavf.cpp: In member function 'bool x265::LavfInput::readPicture(x265_picture&, x265::InputFileInfo*)':
/home/qyot27/x265-build/x265-yuuki/source/input/lavf.cpp:108:13: warning: 'void av_free_packet(AVPacket*)' is deprecated [-Wdeprecated-declarations]
av_free_packet(&pkt);
^
In file included from /usr/i686-w64-mingw32/include/libavformat/avformat.h:318:0,
from /home/qyot27/x265-build/x265-yuuki/source/input/lavf.h:12,
from /home/qyot27/x265-build/x265-yuuki/source/input/lavf.cpp:29:
/usr/i686-w64-mingw32/include/libavcodec/avcodec.h:4040:6: note: declared here
void av_free_packet(AVPacket *pkt);
^
/home/qyot27/x265-build/x265-yuuki/source/input/lavf.cpp:108:13: warning: 'void av_free_packet(AVPacket*)' is deprecated [-Wdeprecated-declarations]
av_free_packet(&pkt);
^
In file included from /usr/i686-w64-mingw32/include/libavformat/avformat.h:318:0,
from /home/qyot27/x265-build/x265-yuuki/source/input/lavf.h:12,
from /home/qyot27/x265-build/x265-yuuki/source/input/lavf.cpp:29:
/usr/i686-w64-mingw32/include/libavcodec/avcodec.h:4040:6: note: declared here
void av_free_packet(AVPacket *pkt);
^
/home/qyot27/x265-build/x265-yuuki/source/input/lavf.cpp:108:32: warning: 'void av_free_packet(AVPacket*)' is deprecated [-Wdeprecated-declarations]
av_free_packet(&pkt);
^
In file included from /usr/i686-w64-mingw32/include/libavformat/avformat.h:318:0,
from /home/qyot27/x265-build/x265-yuuki/source/input/lavf.h:12,
from /home/qyot27/x265-build/x265-yuuki/source/input/lavf.cpp:29:
/usr/i686-w64-mingw32/include/libavcodec/avcodec.h:4040:6: note: declared here
void av_free_packet(AVPacket *pkt);
^
[88/88] Linking CXX executable x265.exe
[8bit:$]

The ENABLE_LAVF block in input.cpp should really probably be changed to accept any file format lavf can. AviSynth was the main reason to argue it before, but really, only supporting *.mp4 and
*.mkv as input is too limiting (other containers and secondary extensions like *.m2ts, *.ts, *.mpg, *.ogv, *.webm, *.vob and several others are also found commonly enough to be nice to open).
The easiest solution would be to simply make the block into an else and the return statement, so anything that's not y4m or a pipe gets handled by lavf (and presumably lavf would error out in the
case that it can't open the file, although this probably needs more error-proofing in input/lavf.cpp unless it already does emit a clean error and not a crash).


The zimg filter also fails to build because of an error in zimgfilter.h:

[86/88] Building CXX object CMakeFiles/cli.dir/filters/filters.cpp.obj
FAILED: /usr/bin/i686-w64-mingw32-g++ -DENABLE_LAVF -DENABLE_LSMASH -DENABLE_MKV -DENABLE_ZIMG -DEXPORT_C_API=1 -DHAVE_INT_TYPES_H=1 -DHIGH_BIT_DEPTH=0 -DX265_ARCH_X86=1 -DX265_DEPTH=8 -DX265_NS=x265 -D_WIN32_WINNT=_WIN32_WINNT_WINXP -D_WIN32_WINNT_WIN7=0x0601 -D__STDC_LIMIT_MACROS=1 -mfpmath=sse -march=pentium3 -msse -mtune=pentium3 -O3 -DNDEBUG -I/home/qyot27/x265-build/x265-yuuki/source/. -I/home/qyot27/x265-build/x265-yuuki/source/common -I/home/qyot27/x265-build/x265-yuuki/source/encoder -I. -Wall -Wextra -Wshadow -std=gnu++98 -march=i686 -Wno-array-bounds -ffast-math -mstackrealign -fno-exceptions -MMD -MT CMakeFiles/cli.dir/filters/filters.cpp.obj -MF CMakeFiles/cli.dir/filters/filters.cpp.obj.d -o CMakeFiles/cli.dir/filters/filters.cpp.obj -c /home/qyot27/x265-build/x265-yuuki/source/filters/filters.cpp
In file included from /home/qyot27/x265-build/x265-yuuki/source/filters/filters.cpp:27:0:
/home/qyot27/x265-build/x265-yuuki/source/filters/zimgfilter.h:56:5: error: 'zimg_resize_context' does not name a type
zimg_resize_context* resizeCtx[3];
^
/home/qyot27/x265-build/x265-yuuki/source/filters/zimgfilter.h:57:5: error: 'zimg_depth_context' does not name a type
zimg_depth_context* depthCtx;
^
[86/88] Building CXX object CMakeFiles/cli.dir/x265.cpp.obj
ninja: build stopped: subcommand failed.
This is with a git build of zimg, so 2.0.4 or something. With the older 1.1.1 release (which I tested only because I thought maybe it might have been expecting that), there's even more compiler
noise thrown and the above errors still apply.


For reference, this is the set of instructions I'm following:
FFmpeg:
git clone git://source.ffmpeg.org/ffmpeg.git && \
cd ffmpeg && \
./configure --prefix=/usr/i686-w64-mingw32 --cross-prefix=i686-w64-mingw32- \
--enable-gpl --enable-version3 --disable-w32threads --enable-avresample \
--disable-encoders --disable-muxers --disable-doc --disable-debug \
--disable-devices --disable-avdevice --enable-avisynth --cpu=pentium3 \
--extra-cflags="-mfpmath=sse -march=pentium3 -msse -mtune=pentium3" \
--target-os=mingw32 --arch=x86 && \
make -j$(nproc) && \
sudo checkinstall --pkgname=ffmpeg-minimal-mingw --pkgversion="$(git \
rev-list --count HEAD)-g$(git rev-parse --short HEAD)-$(date --rfc-3339=date | \
sed 's/-//g')-git" --backup=no --deldoc=yes --delspec=yes --deldesc=yes \
--strip=yes --fstrans=no --default

Add --disable-network --disable-zlib --disable-bz2 --disable-lzma to keep FFmpeg from
calling in private libs, otherwise, see pkg-config note at the end.

mkdir -p ~/x265-build/12bit ~/x265-build/10bit ~/x265-build/8bit && \
cd ~/x265-build && \
hg qclone https://bitbucket.org/msg7086/x265-yuuki && \
cd x265-yuuki && \
hg qpush -a && \

Condense the ifdef ENABLE_LAVF block in source/input/input.cpp to:
#ifdef ENABLE_LAVF
else
return new LavfInput(info);
#endif

Correct uses of PIX_FMT_* to AV_PIX_FMT_*
sed -i 's/case PIX_FMT/case AV_PIX_FMT/g' source/input/lavf.cpp

cd ../12bit && \
cmake ../x265-yuuki/source -G "Ninja" -DCMAKE_INSTALL_PREFIX="$HOME/x265_build/x265-12bit" \
-DCMAKE_TOOLCHAIN_FILE="/usr/i686-w64-mingw32/toolchain-i686-w64-mingw32.cmake" \
-DCMAKE_CXX_FLAGS="-mfpmath=sse -march=pentium3 -msse -mtune=pentium3" \
-DHIGH_BIT_DEPTH:bool=on -DMAIN12:bool=on -DENABLE_SHARED:bool=off \
-DENABLE_ASSEMBLY:bool=off -DEXPORT_C_API:bool=off -DENABLE_CLI:bool=off \
[-DWINXP_SUPPORT:bool=on] && \
ninja && \
sudo checkinstall --pkgname=x265-main12-mingw --pkgversion="$(grep X265_VERSION \
build.ninja | sed 's/X265_VERSION=/\t/' | cut -f2 | sed 's/ /\t/g' | cut -f1)-$(date \
--rfc-3339=date | sed 's/-//g')-hg" --backup=no --deldoc=yes --delspec=yes \
--deldesc=yes --strip=yes --fstrans=no --default cp libx265.a /usr/i686-w64-mingw32/lib/libx265_main12.a && \
mv *.deb ~/mingw_debs

# Build 10-bit:
cd ../10bit && \
cmake ../x265-yuuki/source -G "Ninja" -DCMAKE_INSTALL_PREFIX="$HOME/x265_build/x265-10bit" \
-DCMAKE_TOOLCHAIN_FILE="/usr/i686-w64-mingw32/toolchain-i686-w64-mingw32.cmake" \
-DCMAKE_CXX_FLAGS="-mfpmath=sse -march=pentium3 -msse -mtune=pentium3" \
-DHIGH_BIT_DEPTH:bool=on -DENABLE_SHARED:bool=off -DENABLE_ASSEMBLY:bool=off \
-DEXPORT_C_API:bool=off -DENABLE_CLI:bool=off [-DWINXP_SUPPORT:bool=on] && \
ninja && \
sudo checkinstall --pkgname=x265-main10-mingw --pkgversion="$(grep X265_VERSION \
build.ninja | sed 's/X265_VERSION=/\t/' | cut -f2 | sed 's/ /\t/g' | cut -f1)-$(date \
--rfc-3339=date | sed 's/-//g')-hg" --backup=no --deldoc=yes --delspec=yes \
--deldesc=yes --strip=yes --fstrans=no --default cp libx265.a /usr/i686-w64-mingw32/lib/libx265_main10.a && \
mv *.deb ~/mingw_debs

# Only the .a files from 12-bit and 10-bit are installed to the system,
# to reduce the chances of conflicts.

# Build 8-bit:
cd ../8bit && \
cmake ../x265-yuuki/source -G "Ninja" -DCMAKE_INSTALL_PREFIX="/usr/i686-w64-mingw32" \
-DCMAKE_PREFIX_PATH="/usr/i686-w64-mingw32" \
-DCMAKE_TOOLCHAIN_FILE="/usr/i686-w64-mingw32/toolchain-i686-w64-mingw32.cmake" \
-DCMAKE_CXX_FLAGS="-mfpmath=sse -march=pentium3 -msse -mtune=pentium3" \
-DENABLE_SHARED:bool=off -DEXTRA_LINK_FLAGS=-L. -DLINKED_10BIT:bool=on -DLINKED_12BIT:bool=on \
-DEXTRA_LIB="/usr/i686-w64-mingw32/lib/libx265_main10.a;/usr/i686-w64-mingw32/lib/libx265_main12.a" \
-DENABLE_LSMASH:bool=on -DENABLE_MKV:bool=on -DENABLE_LAVF:bool=on [-DWINXP_SUPPORT:bool=on] && \

cmake ../x265-yuuki/source -G "Ninja" -DCMAKE_INSTALL_PREFIX="/usr/i686-w64-mingw32" \
-DCMAKE_PREFIX_PATH="/usr/i686-w64-mingw32" \
-DCMAKE_TOOLCHAIN_FILE="/usr/i686-w64-mingw32/toolchain-i686-w64-mingw32.cmake" \
-DCMAKE_CXX_FLAGS="-mfpmath=sse -march=pentium3 -msse -mtune=pentium3" \
-DENABLE_SHARED:bool=off -DEXTRA_LINK_FLAGS=-L. -DLINKED_10BIT:bool=on -DLINKED_12BIT:bool=on \
-DEXTRA_LIB="/usr/i686-w64-mingw32/lib/libx265_main10.a;/usr/i686-w64-mingw32/lib/libx265_main12.a" \
-DENABLE_LSMASH:bool=on -DENABLE_MKV:bool=on -DENABLE_LAVF:bool=on [-DWINXP_SUPPORT:bool=on] && \
ninja && \
sed -i 's/lx265/lx265 -lx265_main10 -lx265_main12/' x265.pc && \
sudo checkinstall --pkgname=x265-mingw --pkgversion="$(grep X265_VERSION \
build.ninja | sed 's/X265_VERSION=/\t/' | cut -f2 | sed 's/ /\t/g' | cut -f1)-$(date \
--rfc-3339=date | sed 's/-//g')-hg" --backup=no --deldoc=yes --delspec=yes \
--deldesc=yes --strip=yes --fstrans=no --default ninja install && \
mv *.deb ~/mingw_debs

CMake step needs to be run twice so FFmpeg is detected, but it only gets detected
if it's installed to the system mingw-w64 area. Then, if the zlib/bz2/lzma/iconv libs
and network support are still enabled, it fails to link. This can be fixed by appending the output of:
PKG_CONFIG_PATH=/usr/i686-w64-mingw32/lib/pkgconfig pkg-config --cflags --libs --static libavcodec libavformat libswscale

to the failed link command*. After this, the install step doesn't work, but x265.exe does.

*which, if following the FFmpeg instructs at the beginning, should be something like:
-I/usr/i686-w64-mingw32/include -lavcodec -lws2_32 -liconv -lm -llzma -lbz2 -lz -lpsapi -ladvapi32 -lshell32 -lswresample -lavutil -lavformat -pthread -lswscale -L/usr/i686-w64-mingw32/lib

sudo apt-get purge ffmpeg-minimal-mingw
(this needs to be purged to prevent it interfering with future ffmpeg or mpv builds)

[MeteorRain]

I apologize for not following up on this much earlier, but this is a basic rundown of the issues when cross-compiling x265-yuuki currently:

Thank you for all the efforts.

Please note that I have moved my repo to Github (https://github.com/msg7086/x265-Yuuki-Asuna) due to some personal reason.

And if you can provide some fixes, feel free to send a PR for me.

The zimg referenced is a pretty old version, mainly due to when I built this patch, it was still in 1.x . If you somehow port it to 2.x, please send a PR. I don't have much time to look into it for the moment.

My builds on Linux (and was successful) are compiled against:
zimg 1.1.1
ffmpeg 2.8.5-1+b1 (debian 9 sys pkg)
l-smash rev1384

[MeteorRain]

I apologize for not following up on this much earlier, but this is a basic rundown of the issues when cross-compiling x265-yuuki currently:

input-filters.diff doesn't cleanly apply because the amount of fuzz got too large.

Using HG is always a hassle for me. Moving to git and I'm feeling much better when dealing with patches.

CMake has to be run twice to even detect that -DENABLE_LAVF has been turned on and get the locations for the libs.

Thanks, this has been fixed and pushed.

There are problems linking against FFmpeg's private libs, especially when all of them are built static. zlib, bzip2, lzma, and iconv are all autodetected and used if present when FFmpeg is configured,
and these cause errors on link with the lavf module because the libs haven't been properly added to the linker list by cmake. To say nothing of when more expansive configurations with more
external libs are present, or for other options that are enabled by default (like what's brought in if the user doesn't explicitly use --disable-network). If the user explicitly disables these four libraries
and the network support when building FFmpeg, then x265 can link to them just fine, but otherwise it requires the user to manually append the list of linker libs to the failed link command, and this
then breaks the ability of ninja to do a proper install, even though the .exe is okay.

I guess, with the latest patch, cmake should be able to detect your pkgconfig files and automatically include these dependencies into consideration. If not, we'll see if someone will send a PR to fix it. I don't have much experience on cmake, sorry.

FFmpeg API usage needs to be updated. PIX_FMT_* needs to be changed to AV_PIX_FMT_* to allow compilation to proceed successfully, and even though it's just a warning, uses of deprecated
functions should be updated to non-deprecated ones (which x264 recently fixed for the lavf module there):

These have been fixed with latest patch. BTW I couldn't find the x264 fixes so I modified it according to your description.

The ENABLE_LAVF block in input.cpp should really probably be changed to accept any file format lavf can.

The LAVF part is ported poorly. Besides, I planned to have VFR support but didn't actually finished that. So I cannot guarantee that it's working well with various of types. I'd probably extend the list a bit, but I'd like to keep it minimal until someone has actually reviewed the code.

The zimg filter also fails to build because of an error in zimgfilter.h:

This is with a git build of zimg, so 2.0.4 or something. With the older 1.1.1 release (which I tested only because I thought maybe it might have been expecting that), there's even more compiler
noise thrown and the above errors still apply.

This is on my todo-list, but is also open for PR.

[qyot27]

Using HG is always a hassle for me. Moving to git and I'm feeling much better when dealing with patches.
I agree, I have practically zero working knowledge of Mercurial (enough to clone and look at the log, and I managed to figure out qrefresh/qpush when dealing with the patchsets). I actually was more or less ready to send a pull request on Github, but it looks like I'll need to review which changes can go through now.

I guess, with the latest patch, cmake should be able to detect your pkgconfig files and automatically include these dependencies into consideration. If not, we'll see if someone will send a PR to fix it. I don't have much experience on cmake, sorry.
The solution I'd ultimately came up with was to spin the case of a static FFmpeg off into its own option so that the existing ENABLE_LAVF continues to work as it always has with shared libs.

I'll test to see if the fix I'd implemented is irrelevant now.

These have been fixed with latest patch. BTW I couldn't find the x264 fixes so I modified it according to your description.
Huh, I thought I'd seen x264 stop complaining about using deprecated functions (av_free_packet, etc.) when building the lavf input. The pix_fmt IDs were separate from that.

[qyot27]

I've opened a pull request with two patches to address these points:
I guess, with the latest patch, cmake should be able to detect your pkgconfig files and automatically include these dependencies into consideration. If not, we'll see if someone will send a PR to fix it. I don't have much experience on cmake, sorry.
It's not so much that the pkgconfig files weren't detected, it was that it wasn't adding the content of Libs.private to the linker list. It was populating the right *_STATIC_* variables in CMakeCache.txt, though, so I was able to get around the problem by calling the variables directly.

The LAVF part is ported poorly. Besides, I planned to have VFR support but didn't actually finished that. So I cannot guarantee that it's working well with various of types. I'd probably extend the list a bit, but I'd like to keep it minimal until someone has actually reviewed the code.
Even though I'd simply prefer a wildcard, I went ahead and extended the list manually with the most common (or significant) extensions I could think of. The only one I forgot to include was *.webm, but considering those are just a subset of Matroska, it's not like the user can't just rename them *.mkv.

[MeteorRain]

I've opened a pull request with two patches to address these points:

Thanks, and that has been merged rebased. :thanks:

[n808]

Trying to encode a standard test sequence (BasketballDrill) using 2-pass and the latest 1.9 build ( as well as earlier builds), the 2nd pass creates a bitrate just a fraction of the requested..

My parameters are the same as I am using for x264:
x265 BasketballDrill_832x480_50.yuv --input-res 832x480 --frames 100 --preset slow --fps 50 -o bits --keyint 100 --bitrate 1000 --pass 1
x265 BasketballDrill_832x480_50.yuv --input-res 832x480 --frames 100 --preset slow --fps 50 -o bits --keyint 100 --bitrate 1000 --pass 2

Stats for the 2 passes:
1:encoded 100 frames in 2.94s (33.99 fps), 891.82 kb/s, Avg QP:36.8
2:encoded 100 frames in 3.46s (28.86 fps), 109.70 kb/s, Avg QP:51.00

There must be something basic I am missing..?

[benwaggoner]

Trying to encode a standard test sequence (BasketballDrill) using 2-pass and the latest 1.9 build ( as well as earlier builds), the 2nd pass creates a bitrate just a fraction of the requested..

My parameters are the same as I am using for x264:
x265 BasketballDrill_832x480_50.yuv --input-res 832x480 --frames 100 --preset slow --fps 50 -o bits --keyint 100 --bitrate 1000 --pass 1
x265 BasketballDrill_832x480_50.yuv --input-res 832x480 --frames 100 --preset slow --fps 50 -o bits --keyint 100 --bitrate 1000 --pass 2

Stats for the 2 passes:
1:encoded 100 frames in 2.94s (33.99 fps), 891.82 kb/s, Avg QP:36.8
2:encoded 100 frames in 3.46s (28.86 fps), 109.70 kb/s, Avg QP:51.00

There must be something basic I am missing..?
Encoding just 100 frames is just 2 seconds, way less than a GOP, and just 4x the slow's lookahead of 25 frames. How does the actual output look? You can output to bits_1p.hevc and bits_2p.hevc to compare.

I would expect the 2nd pass rate control to be a whole lot better than the 1st for such a short clip. If there's even a second of black leader, that'd throw everything off.

[n808]

It's a commonly used test sequence for standards development, including HEVC, so there's no black leader..

x264 encodes the same 100 frames with the exact same parameters like this:
1:encoded 100 frames, 237.53 fps, 885.64 kb/s
2:encoded 100 frames, 73.05 fps, 1045.64 kb/s

Something is not working right. x265 seems to clamp to QP to max 51. I may check if libx265 in ffmpeg works any better.

[LigH]

It would probably be useful to see the complete console output and maybe even a CSV log (level 1 or 2) of this second pass. There must be a reason why the target bitrate is apparently not relevant for x265 in the 2nd pass.
_

BTW:

For a more efficient command line, you may like to convert raw YUV to Y4M; this may be possible with y4mtools (https://github.com/concalma/y4mtools), MJPGTools (https://jadecolour.wordpress.com/2015/04/07/file-conversion-between-y4m-and-raw-yuv-format/) (Linux), AviSynth with RawSource26 (http://avisynth.nl/index.php/RawSource26) and avs2pipemod (http://avisynth.nl/index.php/Avs2pipemod), or ffmpeg (-f yuv4mpegpipe).

[n808]

I re-created the same results with a compressed version (x264 crf 10) available here (https://www.dropbox.com/s/6f8ssh22viycs2s/BasketballDrill_832x480_50_crf10.mkv?dl=0). if anyone else wants to try.

I converted this to Y4M using ffmpeg. Then ran the same two commands (adjusted to remove size and fps). The output in the 2nd pass is at the bottom. It's likely that "Error: 2pass curve failed to converge" may be causing the problem, but I don't know how to set parameters to avoid it. If I try higher bitrates, e.g 2000kbps, the error above is gone, but it still misses the mark completely (1795 kbps 1st pass / 789 kbps 2nd pass). Similarly, 2nd pass for for 4000 and 8000 kbps is much much worse than 1st pass, with a lower bitrate that requested.

P.S. I am using raw YUV due to some tools in my toolchain do not yet support Y4M.

====
y4m [info]: 832x480 fps 50/1 i420p8 sar 1:1 frames 0 - 99 of 100
raw [info]: output file: bit
x265 [info]: HEVC encoder version 1.9+32-c2228fb8151d
x265 [info]: build info [Windows][GCC 5.3.0][64 bit] 8bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x265 [info]: Main profile, Level-3.1 (Main tier)
x265 [info]: Thread pool created using 4 threads
x265 [info]: frame threads / pool features : 2 / wpp(8 rows)
x265 [warning]: Source height < 720p; disabling lookahead-slices
x265 [warning]: Error: 2pass curve failed to converge
x265 [warning]: target: 1000.00 kbit/s, expected: 1140.97 kbit/s, avg QP: 68.7127
x265 [warning]: try increasing target bitrate
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge : star / 57 / 3 / 3
x265 [info]: Keyframe min / max / scenecut : 10 / 100 / 40
x265 [info]: Lookahead / bframes / badapt : 25 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 0
x265 [info]: References / ref-limit cu / depth : 4 / 1 / 1
x265 [info]: AQ: mode / str / qg-size / cu-tree : 1 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress : ABR-1000 kbps / 0.60
x265 [info]: tools: rect limit-modes rd=4 psy-rd=2.00 rdoq=2 psy-rdoq=1.00
x265 [info]: tools: signhide tmvp strong-intra-smoothing deblock sao stats-read
[1.0%] 1/100 frames, 5.75 fps, 738.00 kb/s, eta 0:00:17
[10.0%] 10/100 frames, 22.52 fps, 166.76 kb/s, eta 0:00:03
[19.0%] 19/100 frames, 26.32 fps, 137.94 kb/s, eta 0:00:03
[27.0%] 27/100 frames, 27.69 fps, 129.30 kb/s, eta 0:00:02
[35.0%] 35/100 frames, 27.91 fps, 125.79 kb/s, eta 0:00:02
[43.0%] 43/100 frames, 28.33 fps, 125.13 kb/s, eta 0:00:02
[51.0%] 51/100 frames, 28.70 fps, 124.24 kb/s, eta 0:00:01
[59.0%] 59/100 frames, 28.99 fps, 122.76 kb/s, eta 0:00:01
[67.0%] 67/100 frames, 29.13 fps, 122.07 kb/s, eta 0:00:01
[75.0%] 75/100 frames, 29.41 fps, 121.13 kb/s, eta 0:00:00
[84.0%] 84/100 frames, 29.72 fps, 118.84 kb/s, eta 0:00:00
[92.0%] 92/100 frames, 29.89 fps, 117.43 kb/s, eta 0:00:00

x265 [info]: frame I: 1, Avg QP:51.00 kb/s: 308.80
x265 [info]: frame P: 24, Avg QP:51.00 kb/s: 183.78
x265 [info]: frame B: 75, Avg QP:51.00 kb/s: 83.48
x265 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x265 [info]: consecutive B-frames: 8.0% 0.0% 4.0% 60.0% 28.0%

encoded 100 frames in 3.29s (30.40 fps), 109.80 kb/s, Avg QP:51.00

[LigH]

x265 [warning]: Error: 2pass curve failed to converge
x265 [warning]: target: 1000.00 kbit/s, expected: 1140.97 kbit/s, avg QP: 68.7127
x265 [warning]: try increasing target bitrate

True, a strong hint, but directions are not yet clear to me. Apparently x265 believes that QP 51 is sufficient to reach the target bitrate, which is nonsense and results in a bitrate way below par. I will try to recreate your test, but don't have the same hardware (sometimes bugs are CPU instruction dependend, who knows...).

BTW: Is this BasketballDrill_832x480_50.yuv downladable somewhere? It's not in Derf's collection (https://media.xiph.org/video/derf/), unfortunately.
__

Trying similar parameters with a smaller file on an AMD Phenom-II X4, everything works well.

I believe a common reason for 2-pass issues was that a statistics file created in a 1st pas was not read correctly or misinterpreted... For now I have no better idea.
__

I can confirm this behaviour, using a CRF10 AVC-in-MKV source of BasketballDrill (https://www.dropbox.com/s/6f8ssh22viycs2s/BasketballDrill_832x480_50_crf10.mkv?dl=1) (uploaded a copy to my archive (https://www.mediafire.com/download/l6arqxn7kac8ar7/BasketballDrill_832x480_50_crf10.mkv)).

Furthermore, for different other sources, 2-pass VBR misses the target bitrate, the result has about 75-80%.

I will notify the x265 Developers mailing list about this case.
__

I was able to narrow it down between a few archived builds:

This issue was introduced between versions 1.8+167-e951ab673b1c and 1.8+201-769081eb5f4c.

[LigH]

Hum, funny stuff.

I received BasketballDrill_832x480_50.yuv again from staff of the Uni Hannover. It has originally 501 frames. Running them, the bitrate calculation converges.

So I have to wonder if the reason behind this issue is indeed that you encode only 100 frames with a keyframe distance of 100 frames.

Indeed: Adding "--frames 100 --keyint 100" to the command line provokes the miscalculation.

[n808]

@LigH: Thanks for the help! However, I still cannot get it to work properly, and I am little confused by a couple of your answers.

When I use all 501 frames, it somewhat improves, but 2nd pass is still much worse than 1st pass:
1: encoded 501 frames in 14.57s (34.38 fps), 1029.45 kb/s, Avg QP:35.94
2: encoded 501 frames in 31.30s (16.00 fps), 711.98 kb/s, Avg QP:38.16

When I encode the shortened 100 frames version (Y4M) with neither --frames, nor --keyint, the resulting bitrate is no better than with those parameters. So it seems 2 pass does not work on short clips at all. I tried other clips (100 frames) and the problem is the same.

>> Furthermore, for different other sources, 2-pass VBR misses the target bitrate, the result has about 75-80%.

OK. That's what I am seeing as well, for longer sequences and other bitrates. See 1st paragraph. It seems 1-pass is the only option for now.

>> Trying similar parameters with a smaller file on an AMD Phenom-II X4, everything works well.

Is this also the case for the command line parameters and clip in question (posted above)? How about the 75-80% 2nd pass issue in general?

(Yeah, these test sequences are hard to track down without the right connections...)

[LigH]

Remember, I found that this issue was introduced during a specific period of development. Now we have to give the developers some time to discover which change exactly caused it, so they can later fix it... But I hope I could help good enough to find the reason quickly now.

I believe the extreme result of being unable to converge always happens when you have only one GOP; when (what I guess may be the reason) the statistics of the first GOP are accidentally skipped, then everything is skipped because that was only one. The default keyframe interval is already more than 100 frames, so encoding only 100 frames will provoke it already without explicitly limiting the keyframe interval to only 100 frames, which could have been e.g. 250 or 300 frames without this parameter.

[n808]

Remember, I found that this issue was introduced during a specific period of development. Now we have to give the developers some time to discover which change exactly caused it, so they can later fix it...
Sure - that's expected of course. I was only trying to clarify what was still confusing to me, so thanks for explaining.

Here's what I think is the conclusion:

There is a bug where 2-pass will fail completely: encoding less than one GOP
There is a bug where 2-pass works very poorly, reaching only 75-80% of requested bitrate. I am not sure if it's possible to get 2-pass rate control to work in current releases.

[Ma]

x265-1.8+167:
i:\speed\2p>ffmpeg -i b.mkv -v warning -f yuv4mpegpipe - | x265-18+167 --y4m - -p slow --bitrate 1000 --keyint 100 -o w2.hevc --
pass 2
y4m [info]: 832x480 fps 50/1 i420p8 sar 1:1 unknown frame count
raw [info]: output file: w2.hevc
x265 [info]: HEVC encoder version 1.8+167-e951ab673b1c
x265 [info]: build info [Windows][MSVC 1900][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x265 [info]: Main 10 profile, Level-3.1 (Main tier)
x265 [info]: Thread pool created using 4 threads
x265 [info]: frame threads / pool features : 2 / wpp(8 rows)
x265 [warning]: Source height < 720p; disabling lookahead-slices
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge : star / 57 / 3 / 3
x265 [info]: Keyframe min / max / scenecut : 10 / 100 / 40
x265 [info]: Lookahead / bframes / badapt : 25 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 0
x265 [info]: References / ref-limit cu / depth : 3 / 0 / 0
x265 [info]: AQ: mode / str / qg-size / cu-tree : 1 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress : ABR-1000 kbps / 0.60
x265 [info]: tools: rect rd=4 psy-rd=0.30 rdoq=2 psy-rdoq=1.00 signhide tmvp
x265 [info]: tools: strong-intra-smoothing deblock sao stats-read
x265 [info]: frame I: 1, Avg QP:27.14 kb/s: 9306.40
x265 [info]: frame P: 24, Avg QP:30.98 kb/s: 2060.87
x265 [info]: frame B: 75, Avg QP:36.53 kb/s: 467.95
x265 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x265 [info]: consecutive B-frames: 8.0% 0.0% 4.0% 60.0% 28.0%

encoded 100 frames in 11.99s (8.34 fps), 938.64 kb/s, Avg QP:35.10


x265-1.8+168:
i:\speed\2p>ffmpeg -i b.mkv -v warning -f yuv4mpegpipe - | x265-18+168 --y4m - -p slow --bitrate 1000 --keyint 100 -o w2.hevc --
pass 2
y4m [info]: 832x480 fps 50/1 i420p8 sar 1:1 unknown frame count
raw [info]: output file: w2.hevc
x265 [info]: HEVC encoder version 1.8+168-73cc1cd4c7fc
x265 [info]: build info [Windows][MSVC 1900][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x265 [info]: Main 10 profile, Level-3.1 (Main tier)
x265 [info]: Thread pool created using 4 threads
x265 [info]: frame threads / pool features : 2 / wpp(8 rows)
x265 [warning]: Source height < 720p; disabling lookahead-slices
x265 [warning]: Error: 2pass curve failed to converge
x265 [warning]: target: 1000.00 kbit/s, expected: 1133.83 kbit/s, avg QP: 68.7129
x265 [warning]: try increasing target bitrate
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge : star / 57 / 3 / 3
x265 [info]: Keyframe min / max / scenecut : 10 / 100 / 40
x265 [info]: Lookahead / bframes / badapt : 25 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 0
x265 [info]: References / ref-limit cu / depth : 3 / 0 / 0
x265 [info]: AQ: mode / str / qg-size / cu-tree : 1 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress : ABR-1000 kbps / 0.60
x265 [info]: tools: rect rd=4 psy-rd=0.30 rdoq=2 psy-rdoq=1.00 signhide tmvp
x265 [info]: tools: strong-intra-smoothing deblock sao stats-read
x265 [info]: frame I: 1, Avg QP:51.00 kb/s: 358.00
x265 [info]: frame P: 24, Avg QP:51.00 kb/s: 194.60
x265 [info]: frame B: 75, Avg QP:51.00 kb/s: 88.01
x265 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x265 [info]: consecutive B-frames: 8.0% 0.0% 4.0% 60.0% 28.0%

encoded 100 frames in 5.98s (16.71 fps), 116.29 kb/s, Avg QP:51.00


So, the prime suspect is 73cc1cd (https://bitbucket.org/multicoreware/x265/commits/73cc1cd4c7fc0920a6b126fefebeea6269749498)

[LigH]

No surprise, Sherlock! :cool: Of course, the most important change in the rate control.

[Ma]

Finally there is a patch (https://patches.videolan.org/patch/12409/raw/) for 2-pass encoding.

x265-1.9+39+patch:i:\speed\2p>ffmpeg -i b.mkv -v warning -f yuv4mpegpipe - | x265-19+39p --y4m - -p slow --bitrate 1000 --keyint 100 -o w2.hevc --
pass 2
y4m [info]: 832x480 fps 50/1 i420p8 sar 1:1 unknown frame count
raw [info]: output file: w2.hevc
x265 [info]: HEVC encoder version 1.9+39-45c0dbd43dec
x265 [info]: build info [Windows][MSVC 1900][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x265 [info]: Main 10 profile, Level-3.1 (Main tier)
x265 [info]: Thread pool created using 4 threads
x265 [info]: frame threads / pool features : 2 / wpp(8 rows)
x265 [warning]: Source height < 720p; disabling lookahead-slices
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge : star / 57 / 3 / 3
x265 [info]: Keyframe min / max / scenecut : 10 / 100 / 40
x265 [info]: Lookahead / bframes / badapt : 25 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 0
x265 [info]: References / ref-limit cu / depth : 4 / 1 / 1
x265 [info]: AQ: mode / str / qg-size / cu-tree : 1 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress : ABR-1000 kbps / 0.60
x265 [info]: tools: rect limit-modes rd=4 psy-rd=2.00 rdoq=2 psy-rdoq=1.00
x265 [info]: tools: signhide tmvp strong-intra-smoothing deblock sao stats-read
x265 [info]: frame I: 1, Avg QP:27.62 kb/s: 9299.60
x265 [info]: frame P: 24, Avg QP:31.61 kb/s: 2019.28
x265 [info]: frame B: 75, Avg QP:37.20 kb/s: 474.77
x265 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x265 [info]: consecutive B-frames: 8.0% 0.0% 4.0% 60.0% 28.0%

encoded 100 frames in 10.88s (9.19 fps), 933.70 kb/s, Avg QP:35.76

It is working.

[LigH]

But not yet pushed ... I'll wait for a new release until it's confirmed and commited to the repository.

[n808]

Great news - Thanks!

[pingfr]

There are at least two interesting patches which yet have to be merged into the trunk as we speak:

https://patches.videolan.org/patch/12409/
https://patches.videolan.org/patch/12349/

I guess it's just a matter of time.

What I would love to see is a working patch to use avs as source and not only y4m or yuv so we don't have to rely on ffmpeg or avs4x265 anymore. ;)

[qyot27]

What I would love to see is a working patch to use avs as source and not only y4m or yuv so we don't have to rely on ffmpeg or avs4x265 anymore. ;)
See the LAVF patches in x265-yuuki (https://github.com/msg7086/x265-Yuuki-Asuna/commits/Yuuki); AviSynth is included in that now. From a philosophical standpoint, through LAVF is the cleanest way to do it as well, and you also get the benefit of not just AviSynth that way.

The only slight disadvantage is that the interleaved workaround for >8bit output used by Dither and f3kdb is probably not as easily able to fix in the LAVF input source as it is for x264's avs input (of which there is a patch to reconstitute the input video back to its proper bit depth and pretend its been at a higher bit depth all along). Kind of a minor issue compared to supporting it at all, though. In ffmpeg you can do it if you pipe rawvideo to a second instance of ffmpeg and set the input pix_fmt and resolution params correctly, but in a single instance I haven't found a solution yet (maybe something in libavfilter can do it, but that'd require x265 to also include an avfilter interface when using LAVF input).

[LazyNcoder]

Hi guys,
can anyone tell me what's the latest state of hardware accelerated x265 encoding?
Is there anyway to use OpenCL or any other kind of GPU encoding method in x265 encoding?

Thanks

[Darkangle]

I want to update the version of x265 in Staxrip to the latest version. Anyone know how to do that?

[sneaker_ger]

can anyone tell me what's the latest state of hardware accelerated x265 encoding?
Is there anyway to use OpenCL or any other kind of GPU encoding method in x265 encoding?
No, x265 is still a pure software encoder.

There are other encoders which use non-CPU hardware. See StaxRip or Hybrid GUIs, for example. (These may be faster but less effective than x265, i.e. x265 may have better quality at same filesize or lower filesize at same quality if you are willing to invest the time)

I want to update the version of x265 in Staxrip to the latest version. Anyone know how to do that?
Try replacing the integrated x265.exe with the multi-bitdepth version you can find here (http://forum.doom9.org/showpost.php?p=1757289&postcount=3262) (Win64\x265_ml.exe).

[Darkangle]

I think I didn't do it right. Could you be more specific? There are 2 folders in x265 64-Bit 8-Bit & 64-Bit 10-Bit. Each folder contains libx265.dll and x265.exe. So how can I manually update it?

[LigH]

You obviously don't use the latest release (https://github.com/stax76/staxrip/releases); version StaxRip_x64_1.3.1.7_beta (https://github.com/stax76/staxrip/releases/download/StaxRip_x64_1.3.1.7_beta/StaxRip_x64_1.3.1.7_beta.7z) uses only the multi-library EXE, not different specific bitdepth versions and separate DLLs anymore. Please clean up your installation.

In addition: Please wait with updating x265. The fix for the 2-pass VBR mode was not yet released to the official sources, only announced.

[sneaker_ger]

Either that or get yourself the single-bitdepth versions you need from e.g. https://encoder.pw.

[LigH]

I am confused by the naming. If Darkangle had different separate bitdepth versions, the DLLs would have different filenames still, they are bound to e.g. either libx265_main.dll for 8 bit or libx265_main10.dll for 10 bit precision in current versions.

[sneaker_ger]

Hmm, yeah. Should probably clean up and upgrade to the latest version like you said.

[Darkangle]

@LigH: thank you for the link.
Btw, is there anyway to make the the huge block less obvious when using ctu64 in x265? (Does it involve the settings of psy-rd, rdoq=2, psy-rdoq ?)

[LigH]

x265 1.9+54-291beccb6760 (https://www.mediafire.com/download/oze9eo3tclly9lk/x265_1.9+54-291beccb6760.7z) with fixed 2-pass VBR rate control.

Size 64 CTUs are subject to further optimizations when time permits, IIRC; an earlier attempt to interpolate them turned out to have side effects. Psy-RDO parameters may help. Using a convenient bitrate or quality target should help as well, in general.

[pingfr]

@LigH: Could you be more specific on the fore mentionned "side effects" please?

[LigH]

I refer to the analysis by Mandarinka (http://forum.doom9.org/showthread.php?p=1746309#post1746309), which caused (together with previous discussions about a loss of detail) disabling intra prediction for 64x64 CUs (http://forum.doom9.org/showthread.php?p=1746982#post1746982).

[Blowis]

Hi @Ligh download link x265 1.9+54-291beccb6760 dead: This download is currently unavailable.

[LigH]

Not my fault, I believe, I would expect a soon repair (if not, I may have to re-upload) ... but there is a copy attached in the VideoHelp forum (http://forum.videohelp.com/threads/357754-%5BHEVC%5D-x265-EXE-mingw-builds?p=2435148&viewfull=1#post2435148).

[pradeeprama]

Yes, for '--pools +,+' and '--pools *' and no pools options it simulates '--pools 24,24' option. I think it is simple and more effective than current code and better than backout commit 10983.

I hope that you find better fix.

We ran several tests on a dual-socket E5-2670 v3 machine running windows server encoding 1080p anime clips, and some 4K animation videos. Across the board we saw that performance when we don't specify the pools option on the command line (or equivalently specify --pools +,+) gave nearly better or close performance to specifying --pools 16,16. We did, as expected, notice that when a video was run for the first time, the fps is low because you end up incurring too many page faults but when you do repeated runs and access the video from the RAM, fps stabilizes.

Ma/littlepox: I suspect that the issue is something to do with either the warm-up issue as your first run always had no --pools option, or something with your server setup.

[LigH]

MediaFire download of v1.9+54 appears to be fixed.

[Ma]

Across the board we saw that performance when we don't specify the pools option on the command line (or equivalently specify --pools +,+) gave nearly better or close performance to specifying --pools 16,16. We did, as expected, notice that when a video was run for the first time, the fps is low because you end up incurring too many page faults but when you do repeated runs and access the video from the RAM, fps stabilizes.

Did you notice more threads on NUMA-node 0 than on NUMA-node 1? I think it is a problem that bothers littlepox.

[Blue_MiSfit]

Hey everyone - I'm working on some really demanding stuff that requires encoding 4096x4096p60. We have to decode in software on a Core i5.

Is there anything I can do to make the decode faster aside from using --tune fastdecode and limiting peak bitrate with VBV?

[nevcairiel]

Make sure WPP is on for better multithreading, I suppose.

[Blue_MiSfit]

indeed it is.


Disregard the below, this was operator error - I specified -r after -i



Also related - I encode elementary streams with x265.exe, then mux to mp4 with ffmpeg. The result (according to mediainfo) is VFR. How can I make this always constant frame rate? I want to limit the number of variables I'm looking at :)

[LigH]

MediaInfo reports videos as VFR already if just the container is able to support VFR (by assigning timestamps or durations to every frame). It doesn't scan the whole file to know if all frame durations are equal.

[x265_Project]

Hey everyone - I'm working on some really demanding stuff that requires encoding 4096x4096p60. We have to decode in software on a Core i5.

Is there anything I can do to make the decode faster aside from using --tune fastdecode and limiting peak bitrate with VBV?
Hey Derek,
Well, you can get a faster decoder; UHDcode. :) On a pure CPU software basis, it's slightly faster than FFMPEG/OpenHEVC (according to our internal testing). Plus, it's OpenCL accelerated, which gives another performance boost on capable platforms.

The challenge is primarily related to the bit rate. The more bits, the more entropy decoding, and that will become the bottleneck. On a Core i5-4760K desktop, at 10 Mbps, we can decode 4K24 or 4K30 content at around 60 FPS. This drops to around 35 FPS at 35 Mbps, and 28 FPS at 80 Mbps. 4K60 content should be a bit faster, as we should have more merge/skip blocks.

I'll email you to follow up. Tom

[Ely]

On a Core i5-4760K desktop, at 10 Mbps, we can decode 4K24 or 4K30 content at around 60 FPS.

I'm having trouble following.. Don't 4K24 and 4K30 mean 4K @ 24fps and 4K @ 30fps respectively ?

edit : nvm, I guess you meant you can decode 4K 10mbps at 60fps, but specifically naming 4K24 and 4K30 had me confused for a moment :P.

edit2 : nope, you were right to name everything like you did, since I presume that 4K24 @ 10mbps is different from 4K30 @ 10mbps. nvm what I said :D . I should think more before posting.

[MeteorRain]

edit2 : nope, you were right to name everything like you did, since I presume that 4K24 @ 10mbps is different from 4K30 @ 10mbps. nvm what I said :D . I should think more before posting.

Just for clarification,
4k24@10m -> 417kbit per frame
4k30@10m -> 333kbit per frame
4k60@20m -> 333kbit per frame, with more bitrate used on scene cut and more skip ctu/mb on p/b frames.

[littlepox]

We ran several tests on a dual-socket E5-2670 v3 machine running windows server encoding 1080p anime clips, and some 4K animation videos. Across the board we saw that performance when we don't specify the pools option on the command line (or equivalently specify --pools +,+) gave nearly better or close performance to specifying --pools 16,16. We did, as expected, notice that when a video was run for the first time, the fps is low because you end up incurring too many page faults but when you do repeated runs and access the video from the RAM, fps stabilizes.

Ma/littlepox: I suspect that the issue is something to do with either the warm-up issue as your first run always had no --pools option, or something with your server setup.

Acknowledged, thank you for your effort to investigate this; currently I think we can just set --pools "N,N" for our cases; it's not a big problem.

[pingfr]

I'm playing a bit around with various options of x265 and aiming at quality encodes, just had 2 quick questions:

1) In regards of the --lossless option, does it matter whether using --preset veryslow or --preset placebo in cunjunction with the --lossless parameter and does it make a slight difference if --lossless has "priority" over either veryslow or placebo presets?

2) In regards to --aq-mode, values can be set between 0 (disabled) up to value of 3. I can't fathom which value grants the best picture quality? is it 0 with AQ disabled completly? is it the default value of 1 or is it worth raising the value to 2 or even it's maximum at 3?

Once again: my main concern here is the resulted encoded quality, which needs to be top notch or as clear as the source (Blu-Ray discs in my case), neither the encoding speed nor the targetted file size matter at this point.

Thanks in advance.

Edit: My current favored parameters at the moment simply are "--psy-rd 2 --tune grain --crf 17 --preset placebo".

[x265_Project]

I'm playing a bit around with various options of x265 and aiming at quality encodes, just had 2 quick questions:

1) In regards of the --lossless option, does it matter whether using --preset veryslow or --preset placebo in cunjunction with the --lossless parameter and does it make a slight difference if --lossless has "priority" over either veryslow or placebo presets?

2) In regards to --aq-mode, values can be set between 0 (disabled) up to value of 3. I can't fathom which value grants the best picture quality? is it 0 with AQ disabled completly? is it the default value of 1 or is it worth raising the value to 2 or even it's maximum at 3?

Once again: my main concern here is the resulted encoded quality, which needs to be top notch or as clear as the source (Blu-Ray discs in my case), neither the encoding speed nor the targetted file size matter at this point.

Thanks in advance.

Edit: My current favored parameters at the moment simply are "--psy-rd 2 --tune grain --crf 17 --preset placebo".
It doesn't make any sense to take Blu-ray disc content (high bit rate H.264 or VC-1), and losslessly encode it to HEVC. The resulting file will be much larger, but the quality would be identical to what you started with.

AQ won't matter at all if you're encoding lossless, nor will any other "quality" algorithms like psy-rd, cu-tree, or tune settings.

AQ reduces the quality in areas of fast motion (where it will be hard to perceive details, and the picture may be blurred anyhow due to motion blur from the camera), increasing quality where you are more likely to notice it. The best AQ setting depends on the type of content you're encoding. With a lot of fast motion scenes, you might benefit from a higher AQ setting. Deepthi (Nandaku2) and other experts here may be able to give better guidance.

I wouldn't waste time using placebo mode.

[foxyshadis]

1) In regards of the --lossless option, does it matter whether using --preset veryslow or --preset placebo in cunjunction with the --lossless parameter and does it make a slight difference if --lossless has "priority" over either veryslow or placebo presets?

Presets are mostly speed/size tradeoffs, so in lossless, a slower preset just means a tiny bit smaller file. They mostly work that way in non-lossless, as well, though quality may shift slightly.

There's usually a sweet spot where the file size reduction isn't worth the extra time spent, depending on your application and CPU power, but placebo is probably going to be way over the line for a long time.

[MeteorRain]

There's usually a sweet spot where the file size reduction isn't worth the extra time spent, depending on your application and CPU power, but placebo is probably going to be way over the line for a long time.

Exactly. That being said, you'd spend 5x the time (probably couple hours or days) on encoding just get 1% smaller file size, or 1% quality improvement in return.

[pingfr]

AQ reduces the quality in areas of fast motion (where it will be hard to perceive details, and the picture may be blurred anyhow due to motion blur from the camera), increasing quality where you are more likely to notice it. The best AQ setting depends on the type of content you're encoding. With a lot of fast motion scenes, you might benefit from a higher AQ setting. Deepthi (Nandaku2) and other experts here may be able to give better guidance.

So I might as well encode with AQ disabled, (--aq-mode=0) if what I'm after is top notch quality above anything else?

[Ma]

So I might as well encode with AQ disabled, (--aq-mode=0) if what I'm after is top notch quality above anything else?

I tried to find right options to encode "Mad Max: Fury Road". I end up with '--preset veryslow --aq-mode 3 --deblock -1'.

You can make some tests encoding on short samples and compare results.

[littlepox]

IMHO, using parameters like --preset to trade time for quality is very inefficient. You are wasting visible things like hours of encode and dollars of electric bill for invisible things.
My own choice is to make heavy testing on rate-control parameters like aq, psy, qcomp, ipbratio. When combined properly, they can improve your quality dramatically without increasing your encoding time.

Sadly to say that currently the x265's default tuning is very poor especially in high bit-rate, that's why people saying 'I don't care bit-rate or encoding time, give me quality' shall typically use x264. However, my own test on parameter combinations have shown an comparable visual quality to x264, in anime encoding.

I have written some materials on how to systematically test for tuning options, but in Chinese:
http://vcb-s.nmm-hd.org/Dark%20Shrine/%5BVCB-Studio%5D%5B%E6%95%99%E7%A8%8B11%5D%E7%BC%96%E7%A0%81%E5%99%A8%E5%8F%82%E6%95%B0%E7%A0%94%E5%8F%91%E6%96%B9%E6%B3%95/

[pingfr]

In my case, the challenge in order to find the proper encoding parameters is associated with the fact I'm doing "batch encoding" over night using the server farm at work (thus, I'm not personally paying for the hardware costs nor the electricity bills).

But what is devastating is setting a batch with several encodes over night, only to come work next morning to realize encodes are all blocky, grainy, under optimized and such.

That is the reason why I'm looking for a top notch standard set of parameters I can pass to x265 without worrying too much and always use these "safe" parameters to every single encoded/archived movie.

Back in the x264 world for 1080p Blu-Ray quality content, my parameters always were --preset slow --subme 8 --bitrate 8000 --aq 1 --2 pass --ref 5.

But in the x265 world this seems rather unpractical, as the x264 params can't just be "carried over" from x264 to x265 hence my choice of --psy-rd 2 --tune grain --crf 17 --preset placebo.

[sneaker_ger]

So I might as well encode with AQ disabled, (--aq-mode=0) if what I'm after is top notch quality above anything else?
No, it's about bit distribution. Constant bitrate (on frame or even block level) does not equal top notch quality. Like x265_Project described it's supposed to reduce bitrate where you do not notice it and move the bits somewhere where you would notice the difference. Lossy encoding is all about human perception.

[pingfr]

No, it's about bit distribution. Constant bitrate (on frame or even block level) does not equal top notch quality. Like x265_Project described it's supposed to reduce bitrate where you do not notice it and move the bits somewhere where you would notice the difference. Lossy encoding is all about human perception.

So, should aq mode be raised to 3? or should it be kept at default value but then use --aq-strength 3.0?

Or is it --aq-mode 3 combined with --aq-strength 3.0 altogether?

[sneaker_ger]

Those are not settings where more = better. You have to test and tune them to your source material. If you don't want to spend any time testing you might want to simply stick to the default values.

[pingfr]

Default values being --aq-mode 1 --aq-strength 1.0 then?

[sneaker_ger]

Default being whatever x265 uses for the preset/tune combination you are using. These are subject to change.

[pingfr]

That doesn't leave much room for improvement optimization-wise.

[LigH]

As long as most of the room for optimizations is still in the responsibility of the developers, the testers will have to catch up. I still avoid talking about users here... ;)

[pingfr]

@LigH: What about power-users then? :p

[LigH]

Optimism FTW! :D

[benwaggoner]

So, should aq mode be raised to 3? or should it be kept at default value but then use --aq-strength 3.0?

Or is it --aq-mode 3 combined with --aq-strength 3.0 altogether?
You don't raise aq-mode, just set the one you want to have. The difference between 2 and 3 is that 3 shifts relatively more bits to darker areas of the frame, which results in better reproduction on LCD displays where subtle artifacts in blacks can be very visible. If everyone still used CRT, we wouldn't have an aq-mode 3 :). Perceptual uniformity based on gamma is a first order approximation, but not always accurate in the real world.

For another example, you wouldn't use aq-mode 3 with HDR encoding, since the PQ curve is more perceptually uniform than gamma, and there are relatively a whole lot more code values in black.

[pingfr]

So it is --aq-mode 3 with default --aq-strength 1.0 on 1080p/720p "quality" encodes from retail regular Blu-Ray sources?

The --aq-mode 3 would remove "ugly blocks" and "banding effects" that can be perceived in background darker areas of a scene? or the banding perceived in the sky or in the water/rain ripples or the color uniformity from say, the sea?

Do I get that right? :)

[benwaggoner]

So it is --aq-mode 3 with default --aq-strength 1.0 on 1080p/720p "quality" encodes from retail regular Blu-Ray sources?

The --aq-mode 3 would remove "ugly blocks" and "banding effects" that can be perceived in background darker areas of a scene? or the banding perceived in the sky or in the water/rain ripples or the color uniformity from say, the sea?

Do I get that right? :)
If the Blu-ray already had artifacts in black, it probably won't help much; maybe some if mixed with a 720p downscale. But aq-mode 3 can help avoid introducing artifacts in black due to recompressing to a lower bitrate.

It shouldn't have any impact on color uniformity; just on distribution of luma bits. If you want more bits in chroma, use negative values with cbqpoffs/crqpoffs

[sneaker_ger]

If you want to avoid banding try >= 10 bit encoding.

[benwaggoner]

If you want to avoid banding try >= 10 bit encoding.
Yes, that will help as well, if the presence of a 10-bit encoder can be counted on. Using aq-mode 3 can still be valuable, however, as detail and artifacts are generally more visible in the low luma range than high luma on typical displays using Rec. 709.

Note that HEVC is a lot better than H.264 when it comes to introducing banding, even in 8-bit. I'm more worried about artifacts and loss of shadow detail than classic banding.

[pingfr]

At the moment I'm only encoding Blu-Ray discs from 2007-2011, so from what I understood, a 8-bit encoder is appropriate.

Not sure what/if using a 10-bit (let alone 12-bit) encoder would serve any purpose at all?

[sneaker_ger]

10 bit encoding reduces rounding errors in the encoding algorithms so it is useful for avoiding banding even for 8 bit sources.

[benwaggoner]

At the moment I'm only encoding Blu-Ray discs from 2007-2011, so from what I understood, a 8-bit encoder is appropriate.

Not sure what/if using a 10-bit (let alone 12-bit) encoder would serve any purpose at all?
If you aren't doing any scaling or other image processing, yeah, you aren't likely to see any benefit to encoding in >8-bit with HEVC. And you'd take a compatibility and encode-time hit.

Things were different with H.264, where you could still see up to a 10% bitrate reduction, but HEVC seems to have solved that problem well in other ways.

[pingfr]

Wait, what? Are you saying that using a x265-10bit.exe with --aq-mode 3 as encoding parameters has "high chances" to improve perceptible subjective visual quality by reducing banding even if the source material is either a DVD from 2005 era or a rather old Blu-Ray from circa 2010 over a "regular" aka 8-bit x265.exe build?

Edit: I was replying to sneaker_ger, not to benwaggoner. Added for clarity. :)

[sneaker_ger]

Not reduce banding of the source, introduce less (new) banding than the 8 bit encoding. As you can see by benwaggoner's post some people think it's not useful with HEVC. I disagree but that's subjective. Test it and make your own opinion.

http://forum.doom9.org/showpost.php?p=1751744&postcount=3086

[benwaggoner]

Wait, what? Are you saying that using a x265-10bit.exe with --aq-mode 3 as encoding parameters has "high chances" to improve perceptible subjective visual quality by reducing banding even if the source material is either a DVD from 2005 era or a rather old Blu-Ray from circa 2010 over a "regular" aka 8-bit x265.exe build?
No, I'm saying that 8-bit v. 10-bit HEVC isn't likely to make a difference for an 8-bit source.

I am saying that aq-mode 3 is likely to allow for a lower ABR without visible artifacts. Really, think of it as a perceptual optimization for adaptive quant. In will quantize low luma blocks less than high luma blocks (based on DC coefficient? Not sure how it's implemented). This is more perceptually accurate, since the difference between adjoining blocks of Y'=16 and Y'=17 can be visible, much more so than with adjoining blocks of Y'=216 and Y'=217.

Video encoding still very much assumes the old presumption that the gamma of CRT displays is a perfect inverse match to the human visual system, and so most codecs weigh the difference between code values as identical error regardless of what the values are. And that is a decent first-order estimation. But with modern displays, we can see variation in black more than white, so it is more perceptually uniform to treat errors in low luma as a greater visual distortion than in high luma.

So, aq-mode 3 will use lower QP in darker areas, trading off higher QP in brighter areas. PSNR will go down, but perceptual quality will go up.

[pingfr]

@sneaker_ger: I can see --psyrd 2 and 10 bit as both clear benefits over a regular 8 bit encoder, let alone without --psyrd 2, but what kind of source material was this to begin with? was it an excerpt from a larger video clip?

[sneaker_ger]

The source is included in the post in case you are interested in using it for your own tests. (--psy-rd 2 is the default since x265 1.9, btw)

[pingfr]

@ben_waggoner: I know ben. I really was trying to reply to sneaker_ger on that one.

Looks like I'm definitely set on --preset placebo --crf 17 or 18 with --psy-rd 2 and --aq-mode 3 so far.

I will stay away from 10 bit or 12 bit encoders for now as there seems to be a "compatibility mismatch" issue going on, even though I can clearly see a *highly* perceptible visual improvement in regarding to what I call the "sky banding" or "deep sea banding" effects.

Anything else I should "turn on" or tweak at this point?

Always remember, my sources materials all are retail oldies Blu-Ray pre 2010~2011, neither the encoded .hevc output size nor the computational time spent to encode a movie matters at this point. Quality of the resulting .hevc output is my priority concern at this point.

Thanks to both of you. :)

[pingfr]

@sneaker_ger: So no need to specify --psy-rd 2.0 manually anymore with 1.9+73-6d06de58c316 and onwards?

[Boulder]

No, I'm saying that 8-bit v. 10-bit HEVC isn't likely to make a difference for an 8-bit source.What about the cases in which you have an 8-bit source but you process it as 16-bit and feed that to x265?
So, aq-mode 3 will use lower QP in darker areas, trading off higher QP in brighter areas. PSNR will go down, but perceptual quality will go up.Does this mean that aq-mode 3 is recommended as the general solution for regular movies (SD or HD)? I've been using the default aq-mode 1 so far.

[Boulder]

And you'd take a compatibility and encode-time hit.Wouldn't 10-bit HEVC actually be more compatible?

[pingfr]

@Boulder: Be compatible with...?

[sneaker_ger]

@Boulder:
More compatible than what? 10 bit AVC? Yes. 8 bit HEVC? No.
So far I have not seen any HEVC decoder that can decode 10 bit but not 8 bit.
But a lot of decoders can do 10 bit. ffmpeg/LAV, Windows 10, many TVs (10 bit HEVC is required for UltraHD), UltraHD BluRay players, Amazon Fire TV, recent top-smartphones. It's coming everywhere similar to how first AVC decoders were baseline profile with low resolution and today smartphones can decode untouched BluRay streams without breaking a sweat.

@sneaker_ger: So no need to specify --psy-rd 2.0 manually anymore with 1.9+73-6d06de58c316 and onwards?
Correct.

[pingfr]

@sneaker_ger: Thanks for the tip in regards to --psy-rd 2.0 being the expected default behaviour in 1.9+.

Do you know by any chance if there's any other params I should raise/use/tweak with my --preset placebo --crf 17 & --aq-mode 3 to at least be on par with what x264 offers in terms of "subjective quality"?

[LigH]

People are still addicted to placebos... :confused: — utility's best friend. :rolleyes:

[pingfr]

@LigH: The placebo preset is the *only* acceptable way to go for me until x265 can match x264 in terms of retaining the grain/details, see issue #122:

https://bitbucket.org/multicoreware/x265/issues/122/loss-of-details-compared-to-x264

So any added extra tweaks leaning towards optimization and subjective quality are a major concern. :)

Edit: Needless to say, the issue is still opened since april 2015... that's nearly a year ago.

[sneaker_ger]

Preset placebo does not magically fix the problems you encounter with other presets. If you think x264 is still better then the logical decision would be to stick to x264 for the time being.

[pingfr]

@sneaker_ger: Are you sure --psy-rd 2.0 is the expected default behaviour now?

avs4x265 [info]: "x265.exe" - --frames 6497 --fps 30/1 --input-res 1280x720 --input-csp i420 --crf 18 --preset placebo --tune grain --aq-mode 3 --output source.hevc
yuv [info]: 1280x720 fps 30/1 i420p8 unknown frame count
raw [info]: output file: source.hevc
x265 [info]: HEVC encoder version 1.9+73-6d06de58c316
x265 [info]: build info [Windows][GCC 5.3.0][64 bit] 8bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI2
x265 [info]: Main profile, Level-4 (Main tier)
x265 [info]: Thread pool created using 16 threads
x265 [info]: frame threads / pool features : 5 / wpp(12 rows)
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 4 inter / 4 intra
x265 [info]: ME / range / subpel / merge : star / 92 / 5 / 5
x265 [info]: Keyframe min / max / scenecut : 25 / 250 / 40
x265 [info]: Lookahead / bframes / badapt : 60 / 8 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 1
x265 [info]: References / ref-limit cu / depth : 5 / 0 / 0
x265 [info]: AQ: mode / str / qg-size / cu-tree : 3 / 0.3 / 32 / 1
x265 [info]: Rate Control / qCompress : CRF-18.0 / 0.80
x265 [info]: tools: rect amp rd=6 psy-rd=0.50 rdoq=2 psy-rdoq=10.00 tskip
x265 [info]: tools: signhide tmvp strong-intra-smoothing deblock(tC=-2:B=-2)
x265 [info]: tools: sao

I can see psyrd=0.50 default value, right there?

[sneaker_ger]

--tune grain overrides it.

[pingfr]

So are they mutually exclusive parameters? or should I bother enforcing both?

[sneaker_ger]

You can in turn override it by manually specifying --psy-rd 2. (Whether that makes sense is a different question. --tune grain will likely be overhauled soon, they have a patch waiting to be committed. But detail is not the same as grain anyways.)

[pingfr]

You can in turn override it by manually specifying --psy-rd 2.

Which is what I was doing until now and will continue to do so.

(Whether that makes sense is a different question.

Would you care to elaborate a bit more?

--tune grain will likely be overhauled soon, they have a patch waiting to be committed.

May I ask, is the patch publicly available and what does it changes/enhances?

But detail is not the same as grain anyways.)

I always was under the assumption that it was the case; grain = details?

[sneaker_ger]

Would you care to elaborate a bit more?
It's not far-stretched to assume the x265 devs chose that value for --tune grain for a reason. So I do not know whether it makes sense to combine --psy-rd 2 and --tune grain now or with the upcoming patch. And I mean "don't know" when I say "don't know", I'm not implying it's a bad idea. Maybe someone else can give you more insight.

May I ask, is the patch publicly available and what does it changes/enhances?
You can see it on the x265 devel mailing list. I suggest you wait until it has been committed.

I always was under the assumption that it was the case; grain = details?
Grain may be similar to detail but not all detail is grain. They are not equal.

[pingfr]

@sneaker_ger: Thank you for your answers. I'll be waiting until said patch has been committed to the stable tree and to see if they can shed some light on this whole grain =! details =! whatever sha-bang. :)

[pingfr]

Anyone would know what kind of decimal values can be passed to the --aq-strength parameter?

The reason why I'm asking this is, I've encoded a clip with --aq-strength 1.90 it looked flawless (placebo effect?), then re-encoded the same clip with --aq-strength 1.85 it started to show ugly blocks in the sky (placebo effect?)... then I am attempting with --aq-strength 1.88 but I see in the console it is reported as 1.9 value.

So what kind of decimal values can be passed to that parameter? 1.87 1.88? 1.899? 1.9000009009090? :)

[pingfr]

Also, an additional question here, I have all reasons to believe there is some sort of "bug" with the placebo preset.

Here are my findings:

x265-1.9+73-6d06de58c316 (8bit)
Used source sample: lighthouse_lossless.mp4 2.05GB
Sample grabbed from: https://mega.nz/#!osdxQbCR!vim8f5gAD5nf0w0jf-vEAA3mGySmEOoZQOH_GE3Z2uw
Encoded: The first 520 frames only out of the 2852 frames, enough to do a reliable test.

First doing an encode with the preset veryslow:

avs [info]: AviSynth 2.60 (ICL10)
avs [info]: Video colorspace: YV12
avs [info]: Video resolution: 1920x1080
avs [info]: Video framerate: 24/1
avs [info]: Video framecount: 2852
avs4x265 [info]: "x265.exe" - --frames 520 --fps 24/1 --input-res 1920x1080 --input-csp i420 --crf 18 --preset veryslow --psy-rd 2 --aq-mode 3 --aq-strength 1.75 --qcomp 0.68 --output source.hevc
yuv [info]: 1920x1080 fps 24/1 i420p8 unknown frame count
raw [info]: output file: source.hevc
x265 [info]: HEVC encoder version 1.9+73-6d06de58c316
x265 [info]: build info [Windows][GCC 5.3.0][64 bit] 8bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI2
x265 [info]: Main profile, Level-5 (Main tier)
x265 [info]: Thread pool created using 16 threads
x265 [info]: frame threads / pool features : 5 / wpp(17 rows)
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 3 inter / 3 intra
x265 [info]: ME / range / subpel / merge : star / 57 / 4 / 4
x265 [info]: Keyframe min / max / scenecut : 24 / 250 / 40
x265 [info]: Lookahead / bframes / badapt : 40 / 8 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 1
x265 [info]: References / ref-limit cu / depth : 5 / 0 / 1
x265 [info]: AQ: mode / str / qg-size / cu-tree : 3 / 1.8 / 32 / 1
x265 [info]: Rate Control / qCompress : CRF-18.0 / 0.68
x265 [info]: tools: rect amp limit-modes rd=6 psy-rd=2.00 rdoq=2 psy-rdoq=1.00
x265 [info]: tools: signhide tmvp b-intra strong-intra-smoothing deblock sao

x265 [info]: frame I: 4, Avg QP:10.15 kb/s: 28106.88
x265 [info]: frame P: 85, Avg QP:11.75 kb/s: 30704.38
x265 [info]: frame B: 431, Avg QP:17.04 kb/s: 7381.13
x265 [info]: Weighted P-Frames: Y:44.7% UV:43.5%
x265 [info]: Weighted B-Frames: Y:28.1% UV:24.8%
x265 [info]: consecutive B-frames: 4.5% 1.1% 2.2% 31.5% 7.9% 15.7% 5.6% 11.2% 20.2%

encoded 520 frames in 264.99s (1.96 fps), 11353.01 kb/s, Avg QP:16.12

Outcome: The resulted file is looking loss-less or so to speak to the human eye compared to the original uncompressed source, the 5960X crunched it pretty quickly at 1.96 fps under 265 seconds and the resulted file weights 29.3MB (30 750 859 bytes) and x265 reports an Avg QP of 16.12.

Now let's do it again with the placebo preset, shall we?

avs [info]: AviSynth 2.60 (ICL10)
avs [info]: Video colorspace: YV12
avs [info]: Video resolution: 1920x1080
avs [info]: Video framerate: 24/1
avs [info]: Video framecount: 2852
avs4x265 [info]: "x265.exe" - --frames 520 --fps 24/1 --input-res 1920x1080 --input-csp i420 --crf 18 --preset placebo --psy-rd 2 --aq-mode 3 --aq-strength 1.75 --qcomp 0.68 --output source.hevc
yuv [info]: 1920x1080 fps 24/1 i420p8 unknown frame count
raw [info]: output file: source.hevc
x265 [info]: HEVC encoder version 1.9+73-6d06de58c316
x265 [info]: build info [Windows][GCC 5.3.0][64 bit] 8bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI2
x265 [info]: Main profile, Level-5 (Main tier)
x265 [info]: Thread pool created using 16 threads
x265 [info]: frame threads / pool features : 5 / wpp(17 rows)
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 4 inter / 4 intra
x265 [info]: ME / range / subpel / merge : star / 92 / 5 / 5
x265 [info]: Keyframe min / max / scenecut : 24 / 250 / 40
x265 [info]: Lookahead / bframes / badapt : 60 / 8 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 1
x265 [info]: References / ref-limit cu / depth : 5 / 0 / 0
x265 [info]: AQ: mode / str / qg-size / cu-tree : 3 / 1.8 / 32 / 1
x265 [info]: Rate Control / qCompress : CRF-18.0 / 0.68
x265 [info]: tools: rect amp rd=6 psy-rd=2.00 rdoq=2 psy-rdoq=1.00 tskip
x265 [info]: tools: signhide tmvp b-intra strong-intra-smoothing deblock sao

x265 [info]: frame I: 4, Avg QP:10.14 kb/s: 28074.24
x265 [info]: frame P: 86, Avg QP:13.58 kb/s: 24380.51
x265 [info]: frame B: 430, Avg QP:21.51 kb/s: 6460.87
x265 [info]: Weighted P-Frames: Y:45.3% UV:43.0%
x265 [info]: Weighted B-Frames: Y:27.9% UV:24.4%
x265 [info]: consecutive B-frames: 4.4% 1.1% 2.2% 33.3% 5.6% 17.8% 4.4% 14.4% 16.7%

encoded 520 frames in 485.48s (1.07 fps), 9590.76 kb/s, Avg QP:20.11

Okay this is where things get weird and don't make sense: Sure it encoded much slower than the veryslow preset at the average rate of 1.07 fps for a grand total of 485s but this is where things get ugly; instead of yelding an encoded result that is supposedly better quality than a veryslow encode since we spent twice the amount of time to encode it, the quality is far far worse, there are lots of artefacts all over the frame, the Avg QP increased from 16.12 all the way up to 20.11 (from what I understood, the lower this value is, the better the quality should be) and there is where it gets really weird: the resulted file is much smaller, with a file weighting 24.7MB "only" (25 978 082 bytes), shouldn't it be the other way around since we specifically asked the encoder to spend more computing bits here and there?

So in this test that I was able to reproduce several times; placebo preset yields files that are smaller than veryslow, raises the Avg QP by roughly 5% (worse results), takes twice more time & ressources to encode than veryslow (counterproductive, really) and introduces banding, artefacts and compression "errors" here and there, rending a terrible quality encode over a much faster "simpler" preset.

Does that make sense to anyone here or have I missed something?

[LigH]

So what kind of decimal values can be passed to that parameter?

You can specify any precision. But the value may get rounded to a sensible precision either during the processing or during the report formatting. You may not notice an obvious difference between 1.8 and 1.9, so I guess the output format of this value in the log summary was reduced to one decimal just to limit the output width; the value used in the encoder may have a higher precision, but it won't matter much.

[TalasNetrag]

I have a warping (dont know how else to describe it) problem with x265 during motion.
This is the place to look out for:
https://www.mediafire.com/convkey/dccd/thvcezwop1f6ogc4g.jpg (http://www.mediafire.com/view/thvcezwop1f6ogc/scene1_x265_CRF18.png)

Using StaxRip: Quality: 18, Preset: Slower, Tune: None
scene1_x265_CRF18.mkv (http://www.mediafire.com/download/dp8fske9uz3nvnb/scene1_x265_CRF18.mkv)
The x264 encoder doesnt have this problem.
Quality: 18, Preset: Slower, Tune: Animation
scene1_x264_CRF18.mkv (http://www.mediafire.com/download/h8303jy3eyr00a5/scene1_x264_CRF18.mkv)
Both were encoded from this file.
scene1_x264_CRF0.mkv (http://www.mediafire.com/download/iriyiegs3n2ix3s/scene1_x264_CRF0.mkv)

[LigH]

Yes, a kind of "motion trails" with alternating distance, which seems to be related to mini-GOP ranges (means, the group of B frames between a pair of P, maybe I, frames).

[pingfr]

Anyone with insights regarding the "preset" issues I have reported last night?

[sneaker_ger]

the resulted file is much smaller, with a file weighting 24.7MB "only" (25 978 082 bytes), shouldn't it be the other way around since we specifically asked the encoder to spend more computing bits here and there?
What are "computing bits"?
I didn't see you telling x265 to create a bigger file. A slower preset means investing more computing time, not spending more bits. If anything, decreasing file sizes are expected with slower presets though crf does not guarantee same quality when used with a different set of options in the first place. So what you experienced may be totally normal though usually the name "placebo" should imply negligible differences in quality and bitrate compared to "veryslow". That said, I don't know if this sample is an edge case or if there actually is some underlying problem.

[littlepox]

@pingfr

try this and see whether you get any improvements.
ps: do NOT change any parameter, just copy and paste it:

--preset slower --ctu 32 --max-tu-size 16 --crf 18 --tu-intra-depth 2 --tu-inter-depth 2 --rdpenalty 2 --me 3 --subme 5 --merange 44 --b-intra --no-amp --ref 5 --weightb --keyint 360 --min-keyint 1 --bframes 8 --aq-mode 1 --aq-strength 1.0 --rd 5 --psy-rd 1.6 --psy-rdoq 8.0 --rdoq-level 1 --no-sao --no-open-gop --rc-lookahead 80 --max-merge 4 --qcomp 0.80 --no-strong-intra-smoothing --deblock -2:-2 --qg-size 16 --pbratio 1.2

See whether this helps you out or not.
This is based on the --tune film we have tested out. Again, do NOT modify any one of the above parameters otherwise you shall destroy the whole combination.

[pingfr]

@littlepox: Holy shit! The results are looking very good, the artefacts have disappeared, barely any banding in the sky, it encoded ultra fast (of course it's "only" the slower preset) and the result file was only 19MB big. :)

I'll try to see what can be tweaked further and/or more aggressively tomorrow, heading for my bed right now, it's past 3AM here, will get back to you tomorrow.

Thanks a ton! :)

[littlepox]

@littlepox: Holy shit! The results are looking very good, the artefacts have disappeared, barely any banding in the sky, it encoded ultra fast (of course it's "only" the slower preset) and the result file was only 19MB big. :)

I'll try to see what can be tweaked further and/or more aggressively tomorrow, heading for my bed right now, it's past 3AM here, will get back to you tomorrow.

Thanks a ton! :)

I'm using this to tell you that using --preset placebo to achieve quality expectation is a silly idea. No matter how slow you can tolerate, you can NEVER achieve any visible quality improvements.

The only solution is to explore rate-control parameters, which we have tested thousands of samples to offer you this combination. stick to it unless you have tested more.

[pingfr]

@littlepox: What about these settings combined with the --preset veryslow then? would it yield better perceptible quality... or at least a smaller out file? :)

[littlepox]

@littlepox: What about these settings combined with the --preset veryslow then? would it yield better perceptible quality... or at least a smaller out file? :)

Theoretically it should be, but we never spot any in our test, NEVER.

anyway, we have overrided a large number of preset-specified parameters like --ref --me --subme.... so --preset slower is not really "slower", but indeed it's running in a speed between slower and veryslow.

Be alerted that some parameters in veryslow/placebo actually DO hurt the quality, for example, --ctu 64 --max-tu-size 32. They create heavily blurring effect to wipe out details. We have reduced it to --ctu 32 --max-tu-size 16, which increases BOTH the speed and quality.

In short, NEVER hold a faith in --preset placebo. it's just a placebo, tasting sweet, no effect.

[pingfr]

@littlepox: Doing a full movie encode with your settings, really heading off to bed this time, will try another test as --preset veryslow afterwards see if it makes a smaller .hevc out file then I'll figure out which one I want to stick with... given the subjective quality are identical... indeed.

Good night.

Thanks a lot.

[nandaku2]

@pingfr

try this and see whether you get any improvements.
ps: do NOT change any parameter, just copy and paste it:

--preset slower --ctu 32 --max-tu-size 16 --crf 18 --tu-intra-depth 2 --tu-inter-depth 2 --rdpenalty 2 --me 3 --subme 5 --merange 44 --b-intra --no-amp --ref 5 --weightb --keyint 360 --min-keyint 1 --bframes 8 --aq-mode 1 --aq-strength 1.0 --rd 5 --psy-rd 1.6 --psy-rdoq 8.0 --rdoq-level 1 --no-sao --no-open-gop --rc-lookahead 80 --max-merge 4 --qcomp 0.80 --no-strong-intra-smoothing --deblock -2:-2 --qg-size 16 --pbratio 1.2

See whether this helps you out or not.
This is based on the --tune film we have tested out. Again, do NOT modify any one of the above parameters otherwise you shall destroy the whole combination.

Impressive! What kind of sources do you use this custom "tune film" on?

An improved tune grain is now available.

[luigizaninoni]

@pingfr

--preset slower --ctu 32 --max-tu-size 16 --crf 18 --tu-intra-depth 2 --tu-inter-depth 2 --rdpenalty 2 --me 3 --subme 5 --merange 44 --b-intra --no-amp --ref 5 --weightb --keyint 360 --min-keyint 1 --bframes 8 --aq-mode 1 --aq-strength 1.0 --rd 5 --psy-rd 1.6 --psy-rdoq 8.0 --rdoq-level 1 --no-sao --no-open-gop --rc-lookahead 80 --max-merge 4 --qcomp 0.80 --no-strong-intra-smoothing --deblock -2:-2 --qg-size 16 --pbratio 1.2


What modifications would you suggest for SD encoding ? Perhaps something like --crf 16 --ctu16 --max-tu-size 8 --merange 25 ?

[pingfr]

@littlepox: Good "morning".

x265 is still crunching, 46.9% done, eta: 11h30m.

At this rate, I will let you know the encoding results... most likely the next day. ;)

[littlepox]

Impressive! What kind of sources do you use this custom "tune film" on?

An improved tune grain is now available.

This is not the first time I showed this "--tune film". I have updated the previous post here with detailed explanation for the modifications: http://forum.doom9.org/showthread.php?t=172458

Basically, this applies to ALL sources with some level of noise/grain and you want to retain them for visual similarities between source and encode. It works especially well on recorded, high-quality sources like film BDs. Furthermore, this combination preserves details better than default or --tune grain. Our tests should have covered enough samples from different types.

We are mainly working on 1080p sources. No ideas about <720p or >1440p. Our primary focus is 10bit encoding, but I'd suppose with heavy grains, 8bit/10bit are not too much different for film sources.

The choice of parameters depend heavily on crf/bitrate. I would not suggest a similar combination if you are encoding around x265_crf=28. Currently, I'd only recommend to use it for crf=16~20, and even within this interval there are some minor but well-justified changes to make.

The settings can be modified to adapt less grainy sources to become a --tune animation, which is actually our primary focus since we are a team doing anime BDRips. It can also be adapted to a --tune grain by setting parameters favoring even more of grain retention, but we have not intensively tested about that given it brings edge artifacts raised from over-placing bits in flat areas and temporal artifacts raised from higher psy-values. We are not interested since it reduces overall quality, which is also true for x264.

This combination is updated with x265 v1.9 stable. I've notice that you are currently working on the RC issues for grainy sources, but I don't know how it is going to change the behavior of our settings. We plan to test further after v2.0. so @pingfr you are advised to use v1.9 stable and NOT the latest builds; we have not tested them so far.

I know you are going to have a new --tune grain, but previously with v1.9 stable, our --tune film outperform the official --tune grain with only 60% of bit-rate on grainy sources for grain retention. No offence, but the official tunings are really disappointing.

[littlepox]

What modifications would you suggest for SD encoding ? Perhaps something like --crf 16 --ctu16 --max-tu-size 8 --merange 25 ?

please use x264 --tune film --qcomp 0.75. This is the only suggestion I'll make.

[littlepox]

@littlepox: Good "morning".

x265 is still crunching, 46.9% done, eta: 11h30m.

At this rate, I will let you know the encoding results... most likely the next day. ;)

Good "evening".

Just repeat my previous hints here that you are advised to use x265 1.9 stable build (x265 1.9+1), which is the one we use to test this combination. We shall have no ideas about later builds until another scheduled testing after v2.0 is released.

I'm going to sleep in 2 hours, so I'd check you results when I wake up tomorrow.

[j1731630]

Hi! I have some simple question, which wont have a simple answer.
What software and settings should i use to encode video library(100gb mostly 8bit) into HEVC ?

Shotcut
Handbrake
Something else...

[Sharc]

please use x264 --tune film --qcomp 0.75. This is the only suggestion I'll make.
Isn't --qcomp effective for multipass only?

[LigH]

@ j1731630:

1. Do you really have to? You may save a little space (not certain!), but will lose some quality (quite certain), and spend some amount of electricity.

2. What runs on your machine (tell us some specs, especially OS and a little CPU/GPU details) and has a user interface you can handle.

[littlepox]

Isn't --qcomp effective for multipass only?

No. It is effective for all rc options.

[j1731630]

@ j1731630:

1. Do you really have to? You may save a little space (not certain!), but will lose some quality (quite certain), and spend some amount of electricity.

2. What runs on your machine (tell us some specs, especially OS and a little CPU/GPU details) and has a user interface you can handle.
1) I will do this only, if there is some settings which can provide minimum quality loss, and encode time wont be weeks per 1gb file.
2) MiddlePC, i5 4460, 12GB DDR3 1600, GTX 660. Win8.1_64

[LigH]

Alright, quite good hardware available.

Handbrake is a quite common tool, but a little more targeted towards portability across different OS'. There is also VidCoder as an additional UI, using Handbrake as converter engine, which is itself based on ffmpeg. You may also try Hybrid (by Selur) or TEncoder.

But there are more flexible tools specifically for Windows, using AviSynth rather than ffmpeg. One quite common converter of this family is StaxRip; MeGUI is a bit more technical, for advanced users. I bet I forgot about half a dozen more. But there are software archives with a lot of video converters, e.g. at VideoHelp.

[sneaker_ger]

An improved tune grain is now available.
I still get some weird effect:

Original:
http://abload.de/img/original_oduca.png
No tuning:
http://abload.de/img/no_tuning_q6uqn.png
Tune grain with weird effect:
http://abload.de/img/tune_grain_3luci.png

x265 1.9+86 10 bit 2pass, --preset slower (--tune grain) --bitrate 7000
Source and output files download (https://mega.nz/#F!51cRURzb!KtTjtCeP1e6rF9eb5tSAsA)

[pingfr]

@littlepox: The results are here. :)

First infos about the source:

General
Unique ID : 149648185676370562747073954039042049003 (0x709531337A3D6C98B6E41B9A40DA83EB)
Complete name : D:\source.mkv
Format : Matroska
Format version : Version 1
File size : 19.4 GiB
Duration : 2h 8mn
Overall bit rate mode : Variable
Overall bit rate : 21.6 Mbps
Encoded date : UTC 2016-03-10 02:21:51
Writing application : eac3to
Writing library : Haali DirectShow Matroska Muxer 1.13.138.14

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.1
Format settings, CABAC : Yes
Format settings, ReFrames : 4 frames
Format settings, GOP : M=3, N=24
Muxing mode : Container profile=@0.0
Codec ID : V_MPEG4/ISO/AVC
Duration : 2h 8mn
Bit rate mode : Variable
Bit rate : 21.2 Mbps
Maximum bit rate : 28.0 Mbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 23.976 fps
Standard : NTSC
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.426
Stream size : 19.1 GiB (98%)
Default : No
Forced : No
Color range : Limited
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709

Then the .avs script used:

LoadPlugin("C:\Program Files (x86)\MeGUI\tools\lsmash\LSMASHSource.dll")
LWLibavVideoSource("D:\source.mkv")
#deinterlace
crop(0, 20, 0, -20)
#resize
#denoise

Then the x265 parameters used:

@echo off
avs4x265.exe -P x265.exe --preset slower --ctu 32 --max-tu-size 16 --crf 18 --tu-intra-depth 2 --tu-inter-depth 2 --rdpenalty 2 --me 3 --subme 5 --merange 44 --b-intra --no-amp --ref 5 --weightb --keyint 360 --min-keyint 1 --bframes 8 --aq-mode 1 --aq-strength 1.0 --rd 5 --psy-rd 1.6 --psy-rdoq 8.0 --rdoq-level 1 --no-sao --no-open-gop --rc-lookahead 80 --max-merge 4 --qcomp 0.80 --no-strong-intra-smoothing --deblock -2:-2 --qg-size 16 --pbratio 1.2 --output slower.hevc %1
pause

Then what the log output spit out:

avs [info]: AviSynth 2.60 (ICL10)
avs [info]: Video colorspace: YV12
avs [info]: Video resolution: 1920x1040
avs [info]: Video framerate: 24000/1001
avs [info]: Video framecount: 185220
avs4x265 [info]: "x265.exe" - --frames 185220 --fps 24000/1001 --input-res 1920x1040 --input-csp i420 --preset slower --ctu 32 --max-tu-size 16 --crf 18 --tu-intra-depth 2 --tu-inter-depth 2 --rdpenalty 2 --me 3 --subme 5 --merange 44 --b-intra --no-amp --ref 5 --weightb --keyint 360 --min-keyint 1 --bframes 8 --aq-mode 1 --aq-strength 1.0 --rd 5 --psy-rd 1.6 --psy-rdoq 8.0 --rdoq-level 1 --no-sao --no-open-gop --rc-lookahead 80 --max-merge 4 --qcomp 0.80 --no-strong-intra-smoothing --deblock -2:-2 --qg-size 16 --pbratio 1.2 --output slower.hevc
yuv [info]: 1920x1040 fps 24000/1001 i420p8 unknown frame count
raw [info]: output file: slower.hevc
x265 [info]: HEVC encoder version 1.9+73-6d06de58c316
x265 [info]: build info [Windows][GCC 5.3.0][64 bit] 8bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI2
x265 [info]: Main profile, Level-5 (Main tier)
x265 [info]: Thread pool created using 16 threads
x265 [info]: frame threads / pool features : 5 / wpp(33 rows)
x265 [info]: Coding QT: max CU size, min CU size : 32 / 8
x265 [info]: Residual QT: max TU size, max depth : 16 / 2 inter / 2 intra
x265 [info]: ME / range / subpel / merge : star / 44 / 5 / 4
x265 [info]: Keyframe min / max / scenecut : 1 / 360 / 40
x265 [info]: Intra 32x32 TU penalty type : 2
x265 [info]: Lookahead / bframes / badapt : 80 / 8 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 1
x265 [info]: References / ref-limit cu / depth : 5 / 1 / 0
x265 [info]: AQ: mode / str / qg-size / cu-tree : 1 / 1.0 / 16 / 1
x265 [info]: Rate Control / qCompress : CRF-18.0 / 0.80
x265 [info]: tools: rect limit-modes rd=5 psy-rd=1.60 rdoq=1 psy-rdoq=8.00
x265 [info]: tools: signhide tmvp b-intra lslices=4 deblock(tC=-2:B=-2)

x265 [info]: frame I: 1520, Avg QP:14.06 kb/s: 27871.26
x265 [info]: frame P: 36198, Avg QP:16.17 kb/s: 18701.89
x265 [info]: frame B: 147502, Avg QP:18.18 kb/s: 7831.84
x265 [info]: Weighted P-Frames: Y:1.3% UV:0.9%
x265 [info]: Weighted B-Frames: Y:1.0% UV:0.7%
x265 [info]: consecutive B-frames: 7.5% 5.2% 6.5% 15.3% 7.1% 54.1% 2.9% 0.6% 0.8%

encoded 185220 frames in 77139.97s (2.40 fps), 10120.65 kb/s, Avg QP:17.76
Press a key to continue...

And the resulted file details from mediainfo / mpc-hc:

General
Format : HEVC
Format/Info : High Efficiency Video Coding
File size : 9.10 GiB
Writing library : x265 1.9+73-6d06de58c316:[Windows][GCC 5.3.0][64 bit] 8bit
Encoding settings : wpp / ctu=32 / min-cu-size=8 / max-tu-size=16 / tu-intra-depth=2 / tu-inter-depth=2 / me=3 / subme=5 / merange=44 / rect / no-amp / max-merge=4 / temporal-mvp / no-early-skip / rdpenalty=2 / no-tskip / no-tskip-fast / no-strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / no-fast-intra / no-open-gop / no-temporal-layers / interlace=0 / keyint=360 / min-keyint=1 / scenecut=40 / rc-lookahead=80 / lookahead-slices=4 / bframes=8 / bframe-bias=0 / b-adapt=2 / ref=5 / limit-refs=2 / limit-modes / weightp / weightb / aq-mode=1 / qg-size=16 / aq-strength=1.00 / cbqpoffs=0 / crqpoffs=0 / rd=5 / psy-rd=1.60 / rdoq-level=1 / psy-rdoq=8.00 / no-rd-refine / signhide / deblock=-2:-2 / no-sao / no-sao-non-deblock / b-pyramid / cutree / no-intra-refresh / rc=crf / crf=18.0 / qcomp=0.80 / qpmin=0 / qpmax=51 / qpstep=4 / ipratio=1.40 / pbratio=1.20

Video
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main@L5@Main
Width : 1 920 pixels
Height : 1 040 pixels
Display aspect ratio : 1.85:1
Frame rate : 23.976 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Writing library : x265 1.9+73-6d06de58c316:[Windows][GCC 5.3.0][64 bit] 8bit
Encoding settings : wpp / ctu=32 / min-cu-size=8 / max-tu-size=16 / tu-intra-depth=2 / tu-inter-depth=2 / me=3 / subme=5 / merange=44 / rect / no-amp / max-merge=4 / temporal-mvp / no-early-skip / rdpenalty=2 / no-tskip / no-tskip-fast / no-strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / no-fast-intra / no-open-gop / no-temporal-layers / interlace=0 / keyint=360 / min-keyint=1 / scenecut=40 / rc-lookahead=80 / lookahead-slices=4 / bframes=8 / bframe-bias=0 / b-adapt=2 / ref=5 / limit-refs=2 / limit-modes / weightp / weightb / aq-mode=1 / qg-size=16 / aq-strength=1.00 / cbqpoffs=0 / crqpoffs=0 / rd=5 / psy-rd=1.60 / rdoq-level=1 / psy-rdoq=8.00 / no-rd-refine / signhide / deblock=-2:-2 / no-sao / no-sao-non-deblock / b-pyramid / cutree / no-intra-refresh / rc=crf / crf=18.0 / qcomp=0.80 / qpmin=0 / qpmax=51 / qpstep=4 / ipratio=1.40 / pbratio=1.20

It doesn't tell the actual bitrate until merged into a .mkv container:

General
Unique ID : 226389536216421172588785103110168979842 (0xAA5109E78E571BDBACF0BFAA0BDD2982)
Complete name : D:\slower.mkv
Format : Matroska
Format version : Version 4 / Version 2
File size : 9.10 GiB
Duration : 2h 8mn
Overall bit rate : 10.1 Mbps
Encoded date : UTC 2016-03-11 00:01:03
Writing application : mkvmerge v8.9.0 ('Father Daughter') 64bit
Writing library : libebml v1.3.3 + libmatroska v1.4.4
DURATION : 02:08:45.218000000
NUMBER_OF_FRAMES : 185220
NUMBER_OF_BYTES : 9773771334
_STATISTICS_WRITING_APP : mkvmerge v8.9.0 ('Father Daughter') 64bit
_STATISTICS_WRITING_DATE_UTC : 2016-03-11 00:01:03
_STATISTICS_TAGS : BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES

Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main@L5@Main
Codec ID : V_MPEGH/ISO/HEVC
Duration : 2h 8mn
Bit rate : 9 922 Kbps
Width : 1 920 pixels
Height : 1 040 pixels
Display aspect ratio : 1.85:1
Frame rate mode : Constant
Frame rate : 23.976 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Bits/(Pixel*Frame) : 0.207
Stream size : 8.92 GiB (98%)
Writing library : x265 1.9+73-6d06de58c316:[Windows][GCC 5.3.0][64 bit] 8bit
Encoding settings : wpp / ctu=32 / min-cu-size=8 / max-tu-size=16 / tu-intra-depth=2 / tu-inter-depth=2 / me=3 / subme=5 / merange=44 / rect / no-amp / max-merge=4 / temporal-mvp / no-early-skip / rdpenalty=2 / no-tskip / no-tskip-fast / no-strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / no-fast-intra / no-open-gop / no-temporal-layers / interlace=0 / keyint=360 / min-keyint=1 / scenecut=40 / rc-lookahead=80 / lookahead-slices=4 / bframes=8 / bframe-bias=0 / b-adapt=2 / ref=5 / limit-refs=2 / limit-modes / weightp / weightb / aq-mode=1 / qg-size=16 / aq-strength=1.00 / cbqpoffs=0 / crqpoffs=0 / rd=5 / psy-rd=1.60 / rdoq-level=1 / psy-rdoq=8.00 / no-rd-refine / signhide / deblock=-2:-2 / no-sao / no-sao-non-deblock / b-pyramid / cutree / no-intra-refresh / rc=crf / crf=18.0 / qcomp=0.80 / qpmin=0 / qpmax=51 / qpstep=4 / ipratio=1.40 / pbratio=1.20
Default : Yes
Forced : No

I bolded the important parts for emphasis.

Properties of the .mkv file (and that's without any audio tracks added yet):

slower.mkv 9,10 GB (9.775.362.219 bytes)

So sure, the quality is top notch, it's hard to tell any differences with the original file grabbed straight from the source media which was about 19.4GB with an overall bitrate of 21.6 Mbps with a maximum bitrate peak of 28/0 Mbps but...

A such large file makes it unpractical for archiving purposes, let alone the fact it took no less than 77139 seconds to encode (1285 minutes or 21.43 hours, almost 22 hours!!!).

I was understanding that the main interest in x265/HEVC was to allow "high quality" content streaming/delivering/archiving with bitrate starved media/connections?

The very same movie in x264 with rather high-end settings weights only about 9.40GB once processed through x264 latest version... oh, and that's with a 2 hours DTS 5.1 track embedded within the matroska container versus here a 9.10 GB file *without* any audio tracks, just the video tracked processed through your settings.

What the hell? :)

Edit: Sorry my bad, I meant 9.40GB as x264 encode with audio track; not 6.40GB but with audio track still, versus a x265 encode of 9.10GB and no audio muxed yet. So my point stands still: x265 produced a file larger than x264 would have.

[littlepox]

x265 produced a file larger than x264 would have.

It takes long to explain, but you are comparing apples to oranges.

first of all, I can definitely give you another combination which is just 50% of the bit-rate. and I can give you another with double bit-rate but even more perfect of quality. I can even give you a third combination which encodes the movie into 1Mbps, but then you'd probably say "x265 produced a worse quality than x264 would have."

The point is that except for bit-rate, you must compare the visual quality as well, and the comparison must be rigorous and unambiguous.

We are often doing something like this: take the encoded samples with similar bitrate(<5% of difference), then we take ~20 random frames and compare it one by one:

32256 avc -1
16045 hevc 1
33030 hevc 1
20843 tie 0
05108 hevc 1
32023 hevc 1
15225 avc -1
13857 hevc 1
34702 hevc 1
15489 hevc 1
21942 hevc 1
33808 tie 0
29432 avc -1
03147 tie 0
04602 hevc 1
00156 avc -1
22487 hevc 1
33557 hevc 1
32219 avc -1
15616 avc -1

hevc 11 Mean 0.25
avc 6 SE 0.203586268
tie 3 t-value 1.227980663
P-value 89.70%

For this one, out of 20 random sampled frames, hevc looks better on 11 frames, its counter part wins 6, and 3 end up indistinguishable. Then some statistical computation tells you that you can say HEVC outperforms AVC in this test with 89.7% sure.

This is how we managed to get something better than the default tunings. Without such rigorous benchmarks, nothing can be concluded.

[pingfr]

Still doesn't change the fact the resulted file size makes it highly unpractical to archive as it is and the main key selling point of HEVC and x265 is "equal quality if not better than x264 at half the size/half the bitrate".

That's clearly not the case here.

Would you happen to have a set of parameters which can retain quality pretty well while cutting down on the final target encode file size?

Thanks a lot.

[littlepox]

A such large file makes it unpractical for archiving purposes, let alone the fact it took no less than 77139 seconds to encode (1285 minutes or 21.43 hours, almost 22 hours!!!).

Back to the point where you wish to backup your BDs, here are the suggestions:

1. Use 10bit x265 v1.9 stable. We have not tested further builds so we don't know what's going to happen. Furthermore, 10bit encoding gives you an unconditional, significant improvement.

2. To implement the above, just download http://www.msystem.waw.pl/x265/x265-1.9+5-20f14d7-stable_vs2015-AVX2.7z
upzip the x265-10b.exe, rename it to x265.exe, replace the one in your C:\Program Files (x86)\MeGUI\tools\x265\x64\x265.exe or whichever you were using as the encoder.

3. try the new combination:

-D 10 --preset slower --ctu 32 --max-tu-size 16 --crf 20 --tu-intra-depth 2 --tu-inter-depth 2 --rdpenalty 2 --me 3 --subme 5 --merange 44 --b-intra --no-rect --no-amp --ref 5 --weightb --keyint 360 --min-keyint 1 --bframes 8 --aq-mode 1 --aq-strength 1.0 --rd 5 --psy-rd 1.5 --psy-rdoq 5.0 --rdoq-level 1 --no-sao --no-open-gop --rc-lookahead 80 --max-merge 4 --qcomp 0.75 --no-strong-intra-smoothing --deblock -2:-2 --qg-size 16 --pbratio 1.2

[littlepox]

Still doesn't change the fact the resulted file size makes it highly unpractical to archive as it is and the main key selling point of HEVC and x265 is "equal quality if not better than x264 at half the size/half the bitrate".

That's clearly not the case here.

Would you happen to have a set of parameters which can retain quality pretty well while cutting down on the final target encode file size?

Thanks a lot.

See my replies above.

BTW, never trust those lies telling you that "x265 is equal quality if not better than x264 at half the size/half the bitrate". Out of so much we have tested, for high quality ripping, compared by visual quality, x265 takes ~90% of the bitrate to match up its counter part. If you just use official tunings, it's about ~160%, namely, x264 is equal quality if not better than x265 at half the size/half the bitrate, without highly professional tuning.

[pingfr]

Back to the point where you wish to backup your BDs, here are the suggestions:

1. Use 10bit x265 v1.9 stable. We have not tested further builds so we don't know what's going to happen. Furthermore, 10bit encoding gives you an unconditional, significant improvement.

Compatility-hit, no can do.

2. To implement the above, just download http://www.msystem.waw.pl/x265/x265-1.9+5-20f14d7-stable_vs2015-AVX2.7z
upzip the x265-10b.exe, rename it to x265.exe, replace the one in your C:\Program Files (x86)\MeGUI\tools\x265\x64\x265.exe or whichever you were using as the encoder.

Good guess. That's effectively where my x265.exe resides, I just appended the path to the system variables.

3. try the new combination:

-D 10 --preset slower --ctu 32 --max-tu-size 16 --crf 20 --tu-intra-depth 2 --tu-inter-depth 2 --rdpenalty 2 --me 3 --subme 5 --merange 44 --b-intra --no-rect --no-amp --ref 5 --weightb --keyint 360 --min-keyint 1 --bframes 8 --aq-mode 1 --aq-strength 1.0 --rd 5 --psy-rd 1.5 --psy-rdoq 5.0 --rdoq-level 1 --no-sao --no-open-gop --rc-lookahead 80 --max-merge 4 --qcomp 0.75 --no-strong-intra-smoothing --deblock -2:-2 --qg-size 16 --pbratio 1.2

-D 10 and --crf 20, that's a no go for me. The rest of the settings are worth exploring however when combined with a --crf 18.

Thank you for your time.

[pingfr]

See my replies above.

BTW, never trust those lies telling you that "x265 is equal quality if not better than x264 at half the size/half the bitrate". Out of so much we have tested, for high quality ripping, compared by visual quality, x265 takes ~90% of the bitrate to match up its counter part. If you just use official tunings, it's about ~160%, namely, x264 is equal quality if not better than x265 at half the size/half the bitrate, without highly professional tuning.

Then I think we have a problem here, msu.ru's HEVC Video Codecs Comparison along with quite a few different sites/blogs/articles on the web are claiming that x265/HEVC is vastly superior to x264 in situations where the bitrate is an issue and that x265 yields much smaller files to their x264 counterpart while retaining excellent quality if not better.

When you manage a library of 2600 movies like I do for an online public library (legal) every single megabyte worth of shaved off storage space is worth the effort re-encoding everything to x265... only if the yielded results effectively are smaller than their x264 counterpart that is.

[littlepox]

Compatility-hit, no can do.

All right, then just use x265-8b.exe. BTW our test was done for 10bit, so it's not going to be the best solution under 8bit.



-D 10 and --crf 20, that's a no go for me. The rest of the settings are worth exploring however when combined with a --crf 18.

Thank you for your time.

Bear in mind that --crf here is NOT telling you enough stories. Just compare this --crf 20 with a default --crf 16 and you shall probably prefer --crf 20 version. And the --crf in x265 should never be compared to its counterpart in x264, they just share a same name, that's all.

Furthermore, if you really wish to use --crf 18 with our suggestions, use the previous settings. Every individual parameter has its purpose and should work together as a whole.

[pingfr]

Bear in mind that --crf here is NOT telling you enough stories. Just compare this --crf 20 with a default --crf 16 and you shall probably prefer --crf 20 version.

Not exactly sure to understand what you actually meant here as the reason why would I prefer a --crf 20 encode over a --crf 16 one exactly?

And the --crf in x265 should never be compared to its counterpart in x264, they just share a same name, that's all.

And that sir, I think is one of the things about 54546543765 readers on this forum have been asking for since the very first page of this topic.

Devs have been more or less asked to give us a "table" of what to expect in terms of understanding/equaling/matching CRF values back and forth between x264 and x265.

x264 is nearly a decade old, video enthusiasts have been used for the past 10 years to use either 2-pass encoding (inherited from the XviD days and from the DivX days even before that) or use the CRF values from the very earliest x264 days, therefore the x265 devs should know that powerusers are not going to give up on that.

Good luck with convincing and explaining those users that crf 18 in x265 isn't equal to crf 18 in x264 and so forth.

The day we may see a conversion table added to official docs might actually change that, until then...

Furthermore, if you really wish to use --crf 18 with our suggestions, use the previous settings. Every individual parameter has its purpose and should work together as a whole

So which set of parameters should I use with a -D 8 --crf 18 if I would like to retain the same level of quality while cutting on the file size?

[littlepox]

Then I think we have a problem here, msu.ru's HEVC Video Codecs Comparison along with quite a few different sites/blogs/articles on the web are claiming that x265/HEVC is vastly superior to x264 in situations where the bitrate is an issue and that x265 yields much smaller files to their x264 counterpart while retaining excellent quality if not better.

When you manage a library of 2600 movies like I do for an online public library (legal) every single megabyte worth of shaved off storage space is worth the effort re-encoding everything to x265... only if the yielded results effectively are smaller than their x264 counterpart that is.

1. They are testing with objective benchmarks like psnr/ssim, NOT human eyes. With these digit benchmarks there are dozens of encoders claiming themselves better than x264 in the past a few years. For the most recent one, try daala;).

2. They are primarily focus on low bit-rate cases like crf=28. You can't even stand crf=20, so their conclusion makes no sense to you.

3. Indeed, the lower the bitrate, the better x265 performs. But that's not for the case of ripping so I'd not continue.

[littlepox]

Not exactly sure to understand what you actually meant here as the reason why would I prefer a --crf 20 encode over a --crf 16 one exactly?

OK my point is that --crf itself cannot determine the visual quality completely, you have to look at the other parameters it is using with.

just take the mature, well-recognized x264. try for your film sources:

x264 --preset veryslow --crf 19 --tune film --qcomp 0.7
x264 --preset veryslow --crf 16 --qcomp 0.4 --psy-rd 0.2:0 --aq-strength 0.3

guess what? the first encode with --crf 19 will surely looks better than the second --crf 16. Because the 2nd one, the rc parameters combined is so ill-conditioned.

Do this test and then think about the case, you can't even seek for consistency within x264, do you expect any consistency across encoders, with another set of highly customized settings?

[littlepox]

So which set of parameters should I use with a -D 8 --crf 18 if I would like to retain the same level of quality while cutting on the file size?

-D 8 --preset slower --ctu 32 --max-tu-size 16 --crf 19 --tu-intra-depth 2 --tu-inter-depth 2 --rdpenalty 2 --me 3 --subme 5 --merange 44 --b-intra --no-rect --no-amp --ref 5 --weightb --keyint 360 --min-keyint 1 --bframes 8 --aq-mode 1 --aq-strength 1.0 --rd 5 --psy-rd 1.5 --psy-rdoq 5.0 --rdoq-level 1 --no-sao --no-open-gop --rc-lookahead 80 --max-merge 4 --qcomp 0.8 --no-strong-intra-smoothing --deblock -2:-2 --qg-size 16 --pbratio 1.2

pretend that you don't know the crf, test about it, and give me your feedback.

[pingfr]

Gotcha.

Just like we did the experiment with a well provisioned --preset slower or --preset veryslow yielding better perceptible results over a "vanilla" un-tweaked --preset placebo.

Still, there is something I don't get. I've seen (sadly, pirated) content posted on public and popular peer-to-peer trackers for 2160p contents which was ripped from UHD 4k Blu-Ray discs even and let's say that a 4GB encoded file for an over 2 hours encoded fully action packed-action fast paced movie is... quite impressive.

I will not post such links here as I have no interest whatsoever in promoting piracy in any ways/forms/shapes however, I've seen the results myself they are quite... amazing.

Should I bother pasting the used parameters here from the file properties? Would this be of any use to you?

[littlepox]

Gotcha.

Just like we did the experiment with a well provisioned --preset slower or --preset veryslow yielding better perceptible results over a "vanilla" un-tweaked --preset placebo.

Still, there is something I don't get. I've seen (sadly, pirated) content posted on public and popular peer-to-peer trackers for 2160p contents which was ripped from UHD 4k Blu-Ray discs even and let's say that a 4GB encoded file for an over 2 hours encoded fully action packed-action fast paced movie is... quite impressive.

I will not post such links here as I have no interest whatsoever in promoting piracy in any ways/forms/shapes however, I've seen the results myself they are quite... amazing.

Should I bother pasting the used parameters here from the file properties? Would this be of any use to you?

I've seen some of the 4K BDRips, and surprisingly, some of them are using the parameters I posted here:
http://forum.doom9.org/showthread.php?t=172458
which used to be an earlier version of our "--tune film"

Of cause the probability that we are talking about the same encode is really rare. So I guess it could be:

1. You have not seen the sources. It could be that compared to the sources a lot of details are gone, you just don't know.

2. x265 itself is well-tuned for 4K contents with lower bit-rates. You aren't doing the same thing; you are working on 1080p high bit-rates.

3. They've got their own testing to replace the default tunings. Pls post the mediainfo here, see what I can read out of it.

[pingfr]

There you go:

Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : @L5@Main
Codec ID : V_MPEGH/ISO/HEVC
Duration : 2h 8mn
Width : 3 840 pixels
Height : 1 608 pixels
Display aspect ratio : 2.40:1
Frame rate mode : Constant
Frame rate : 23.976 fps
Color space : YUV
Chroma subsampling : 4:4:4
Bit depth : 10 bits
Writing library : x265 1.8+167-e951ab673b1c:[Windows][GCC 5.2.0][64 bit] 10bit
Encoding settings : wpp / ctu=32 / min-cu-size=16 / max-tu-size=32 / tu-intra-depth=1 / tu-inter-depth=1 / me=0 / subme=0 / merange=57 / no-rect / no-amp / max-merge=2 / temporal-mvp / early-skip / rdpenalty=0 / no-tskip / no-tskip-fast / strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / fast-intra / open-gop / no-temporal-layers / interlace=0 / keyint=250 / min-keyint=23 / scenecut=0 / rc-lookahead=5 / lookahead-slices=8 / bframes=3 / bframe-bias=0 / b-adapt=0 / ref=1 / limit-refs=0 / no-limit-modes / no-weightp / no-weightb / aq-mode=0 / qg-size=32 / aq-strength=0.00 / cbqpoffs=6 / crqpoffs=6 / rd=2 / psy-rd=0.30 / rdoq-level=0 / psy-rdoq=0.00 / no-signhide / deblock / no-sao / no-sao-non-deblock / b-pyramid / no-cutree / no-intra-refresh / rc=abr / bitrate=4300 / qcomp=0.60 / qpmin=0 / qpmax=51 / qpstep=4 / ipratio=1.40 / pbratio=1.30
Default : Yes
Forced : No

Resulted .mkv file size: 4.18 GB (4.494.613.248 bytes) and that's even with a freaking audio AAC track.

[littlepox]

There you go:

Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : @L5@Main
Codec ID : V_MPEGH/ISO/HEVC
Duration : 2h 8mn
Width : 3 840 pixels
Height : 1 608 pixels
Display aspect ratio : 2.40:1
Frame rate mode : Constant
Frame rate : 23.976 fps
Color space : YUV
Chroma subsampling : 4:4:4
Bit depth : 10 bits
Writing library : x265 1.8+167-e951ab673b1c:[Windows][GCC 5.2.0][64 bit] 10bit
Encoding settings : wpp / ctu=32 / min-cu-size=16 / max-tu-size=32 / tu-intra-depth=1 / tu-inter-depth=1 / me=0 / subme=0 / merange=57 / no-rect / no-amp / max-merge=2 / temporal-mvp / early-skip / rdpenalty=0 / no-tskip / no-tskip-fast / strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / fast-intra / open-gop / no-temporal-layers / interlace=0 / keyint=250 / min-keyint=23 / scenecut=0 / rc-lookahead=5 / lookahead-slices=8 / bframes=3 / bframe-bias=0 / b-adapt=0 / ref=1 / limit-refs=0 / no-limit-modes / no-weightp / no-weightb / aq-mode=0 / qg-size=32 / aq-strength=0.00 / cbqpoffs=6 / crqpoffs=6 / rd=2 / psy-rd=0.30 / rdoq-level=0 / psy-rdoq=0.00 / no-signhide / deblock / no-sao / no-sao-non-deblock / b-pyramid / no-cutree / no-intra-refresh / rc=abr / bitrate=4300 / qcomp=0.60 / qpmin=0 / qpmax=51 / qpstep=4 / ipratio=1.40 / pbratio=1.30
Default : Yes
Forced : No

Resulted .mkv file size: 4.18*GB (4.494.613.248 bytes) and that's even with a freaking audio AAC track.

first of all they are using 10bit, which is extremely powerful in low-bitrate since it is immune to banding/blocking artifacts.


second, they are using --preset ultrafast --bitrate 4300. The lowest preset you can ever set, without any tuning, and a fairly low bit-rate for 4K content.

I've done similar test before. The result is that x265-10bit will wash out every film grain or tiny details to give you a clean, blurry video. But after downscaled from 4K to 1080p, you don't feel any uncomfortable as if watching a DVD; you don't see banding/blocking/broken edge neither. This is exactly the power of HEVC-10bit under high resolution, extremely low bit-rates.

BUT this has NOTHING to do with 1080p, high-bitrate encoding.

[pingfr]

first of all they are using 10bit, which is extremely powerful in low-bitrate since it is immune to banding/blocking artifacts.

Alrighty.

second, they are using --preset ultrafast --bitrate 4300. The lowest preset you can ever set, without any tuning, and a fairly low bit-rate for 4K content.

I have noticed that, when manually comparing each parameter they passed to the encoder with the parameters found from the official preset page;

http://x265.readthedocs.org/en/default/presets.html

That's why it was confusing to begin with, that, coupled with the excessively low bitrate of 4300.

I've done similar test before. The result is that x265-10bit will wash out every film grain or tiny details to give you a clean, blurry video. But after downscaled from 4K to 1080p, you don't feel any uncomfortable as if watching a DVD; you don't see banding/blocking/broken edge neither. This is exactly the power of HEVC-10bit under high resolution, extremely low bit-rates.

BUT this has NOTHING to do with 1080p, high-bitrate encoding.

Time to find proper settings for 1080p high quality archival then.

Do you feel it would be possible at this point to find settings for x265 which would result in an encoded 1080p/720p contents weighting 25% less than it's x264 counterpart or am I being delusional here?

Edit; Also encoding a shorter movie (1h23m) with the latest settings you gave me 3 posts ago, it seems to encode much faster, ~6 fps versus ~2.4 fps, but then it could be because I switched from a 1920x1040 source to a 1920x800, less data to encode per frame = faster encoding I guess. ETA is: 6 hours. I will let you know the results whenever I get them done.

[littlepox]

Time to find proper settings for 1080p high quality archival then.


play around with :

--preset slower --ctu 32 --max-tu-size 16 --crf 19 --tu-intra-depth 2 --tu-inter-depth 2 --rdpenalty 2 --me 3 --subme 5 --merange 44 --no-rect --b-intra --no-amp --ref 5 --weightb --keyint 360 --min-keyint 1 --bframes 8 --aq-mode 1 --aq-strength 1.0 --rd 5 --psy-rd 1.5 --psy-rdoq 5.0 --rdoq-level 1 --no-sao --no-open-gop --rc-lookahead 80 --max-merge 4 --qcomp 0.8 --no-strong-intra-smoothing --deblock -2:-2 --qg-size 16 --pbratio 1.2

You can manually set crf to be within [18,20] for your preference. Too large or too small and this combination is not optimal anymore.



Do you feel it would be possible at this point to find settings for x265 which would result in an encoded 1080p/720p contents weighting 25% less than it's x264 counterpart or am I being delusional here?

2.5% is doable, our test suggests that on average x265 can save about ~10% of bitrate.

wait, 25%? don't even daydreaming:(...



Edit; Also encoding a shorter movie (1h23m) with the latest settings you gave me 3 posts ago, it seems to encode much faster, ~6 fps versus ~2.4 fps

I turned off --rect since you complained about the speed. --rect is a parameter which takes a lot of computation but the gain is negligible. It has little impact on quality or rate-control.

[pingfr]

play around with :

--preset slower --ctu 32 --max-tu-size 16 --crf 19 --tu-intra-depth 2 --tu-inter-depth 2 --rdpenalty 2 --me 3 --subme 5 --merange 44 --no-rect --b-intra --no-amp --ref 5 --weightb --keyint 360 --min-keyint 1 --bframes 8 --aq-mode 1 --aq-strength 1.0 --rd 5 --psy-rd 1.5 --psy-rdoq 5.0 --rdoq-level 1 --no-sao --no-open-gop --rc-lookahead 80 --max-merge 4 --qcomp 0.8 --no-strong-intra-smoothing --deblock -2:-2 --qg-size 16 --pbratio 1.2

You can manually set crf to be within [18,20] for your preference. Too large or too small and this combination is not optimal anymore.

Already running an encode with these settings as we speak:

--preset slower --ctu 32 --max-tu-size 16 --crf 18 --tu-intra-depth 2 --tu-inter-depth 2 --rdpenalty 2 --me 3 --subme 5 --merange 44 --b-intra --no-rect --no-amp --ref 5 --weightb --keyint 360 --min-keyint 1 --bframes 8 --aq-mode 1 --aq-strength 1.0 --rd 5 --psy-rd 1.5 --psy-rdoq 5.0 --rdoq-level 1 --no-sao --no-open-gop --rc-lookahead 80 --max-merge 4 --qcomp 0.8 --no-strong-intra-smoothing --deblock -2:-2 --qg-size 16 --pbratio 1.2

ETA used to be 6 hours'ish but now bumped up to 9 hours or so.

I'm exhausted and got a lot of work to do tomorrow, I will talk to you laters. You have no idea how grateful I am for your help.

Thank you, thank you, thank you.

[LigH]

One additional comment (I hope I did not miss it being mentioned already in the verbose discussion above):

If you have "perfect quality" original content (e.g. Blender render movies as PNG or even 16-bit-per-channel TIFF sequences), then you can compare the result one encoder created from the original content with the result another encoder created from the same original content. With some relaxed point of view, material encoded with a very generous bitrate and quantization fine enough to "guarantee visual transparency" may be considered as "quasi original"; Blu-ray content should match this criterion if the production studio did not mess it with nerdy blindness.

But if your so-called "original" content is not really original, but already compressed with a lossy format, then it doesn't make much sense to compare a re-encode of this already lossy content passing another lossy encode. This will introduce a cascading issue: The next encoder will have to spend more bitrate to resemble the already present encoding artifacts produced by the previous encoder due to its rate control limits. Thus, no big surprise that the re-encoded result can become larger and still look worse.

[pingfr]

@littlepox: The results are here.

Source first:

General
Unique ID : 40485131295552032955658755792816885178 (0x1E75271526187B453EC961B7C6A0CDBA)
Complete name : D:\source.mkv
Format : Matroska
Format version : Version 1
File size : 15.3 GiB
Duration : 1h 23mn
Overall bit rate : 26.1 Mbps
Encoded date : UTC 2016-03-11 02:00:12
Writing application : eac3to
Writing library : Haali DirectShow Matroska Muxer 1.13.138.14

Video
ID : 1
Format : VC-1
Format profile : Advanced@L3
Codec ID : V_MS/VFW/FOURCC / WVC1
Codec ID/Hint : Microsoft
Duration : 1h 23mn
Bit rate : 25.6 Mbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 24.000 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.514
Stream size : 15.0 GiB (98%)
Default : No
Forced : No

Encoding parameters were:

@echo off
avs4x265.exe -P x265.exe --preset slower --ctu 32 --max-tu-size 16 --crf 18 --tu-intra-depth 2 --tu-inter-depth 2 --rdpenalty 2 --me 3 --subme 5 --merange 44 --b-intra --no-rect --no-amp --ref 5 --weightb --keyint 360 --min-keyint 1 --bframes 8 --aq-mode 1 --aq-strength 1.0 --rd 5 --psy-rd 1.5 --psy-rdoq 5.0 --rdoq-level 1 --no-sao --no-open-gop --rc-lookahead 80 --max-merge 4 --qcomp 0.8 --no-strong-intra-smoothing --deblock -2:-2 --qg-size 16 --pbratio 1.2 --output out.hevc %1
pause

The x265.exe output log:

avs [info]: AviSynth 2.60 (ICL10)
avs [info]: Video colorspace: YV12
avs [info]: Video resolution: 1920x800
avs [info]: Video framerate: 24/1
avs [info]: Video framecount: 120888
avs4x265 [info]: "x265.exe" - --frames 120888 --fps 24/1 --input-res 1920x800 --input-csp i420 --preset slower --ctu 32 --max-tu-size 16 --crf 18 --tu-intra-depth 2 --tu-inter-depth 2 --rdpenalty 2 --me 3 --subme 5 --merange 44 --b-intra --no-rect --no-amp --ref 5 --weightb --keyint 360 --min-keyint 1 --bframes 8 --aq-mode 1 --aq-strength 1.0 --rd 5 --psy-rd 1.5 --psy-rdoq 5.0 --rdoq-level 1 --no-sao --no-open-gop --rc-lookahead 80 --max-merge 4 --qcomp 0.8 --no-strong-intra-smoothing --deblock -2:-2 --qg-size 16 --pbratio 1.2 --output out.hevc
yuv [info]: 1920x800 fps 24/1 i420p8 unknown frame count
raw [info]: output file: out.hevc
x265 [info]: HEVC encoder version 1.9+73-6d06de58c316
x265 [info]: build info [Windows][GCC 5.3.0][64 bit] 8bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI2
x265 [info]: Main profile, Level-4 (Main tier)
x265 [info]: Thread pool created using 16 threads
x265 [info]: frame threads / pool features : 5 / wpp(25 rows)
x265 [info]: Coding QT: max CU size, min CU size : 32 / 8
x265 [info]: Residual QT: max TU size, max depth : 16 / 2 inter / 2 intra
x265 [info]: ME / range / subpel / merge : star / 44 / 5 / 4
x265 [info]: Keyframe min / max / scenecut : 1 / 360 / 40
x265 [info]: Intra 32x32 TU penalty type : 2
x265 [info]: Lookahead / bframes / badapt : 80 / 8 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 1
x265 [info]: References / ref-limit cu / depth : 5 / 1 / 0
x265 [info]: AQ: mode / str / qg-size / cu-tree : 1 / 1.0 / 16 / 1
x265 [info]: Rate Control / qCompress : CRF-18.0 / 0.80
x265 [info]: tools: limit-modes rd=5 psy-rd=1.50 rdoq=1 psy-rdoq=5.00 signhide
x265 [info]: tools: tmvp b-intra lslices=4 deblock(tC=-2:B=-2)

x265 [info]: frame I: 1037, Avg QP:14.18 kb/s: 28314.02
x265 [info]: frame P: 24299, Avg QP:16.47 kb/s: 15315.20
x265 [info]: frame B: 95552, Avg QP:18.73 kb/s: 7015.12
x265 [info]: Weighted P-Frames: Y:2.6% UV:1.4%
x265 [info]: Weighted B-Frames: Y:1.8% UV:1.2%
x265 [info]: consecutive B-frames: 13.7% 7.0% 8.6% 15.7% 8.9% 25.7% 7.7% 6.1% 6.7%

encoded 120888 frames in 31095.92s (3.89 fps), 8866.18 kb/s, Avg QP:18.23
Press a key to continue...

Results are:

General
Format : HEVC
Format/Info : High Efficiency Video Coding
File size : 5.20 GiB
Writing library : x265 1.9+73-6d06de58c316:[Windows][GCC 5.3.0][64 bit] 8bit
Encoding settings : wpp / ctu=32 / min-cu-size=8 / max-tu-size=16 / tu-intra-depth=2 / tu-inter-depth=2 / me=3 / subme=5 / merange=44 / no-rect / no-amp / max-merge=4 / temporal-mvp / no-early-skip / rdpenalty=2 / no-tskip / no-tskip-fast / no-strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / no-fast-intra / no-open-gop / no-temporal-layers / interlace=0 / keyint=360 / min-keyint=1 / scenecut=40 / rc-lookahead=80 / lookahead-slices=4 / bframes=8 / bframe-bias=0 / b-adapt=2 / ref=5 / limit-refs=2 / limit-modes / weightp / weightb / aq-mode=1 / qg-size=16 / aq-strength=1.00 / cbqpoffs=0 / crqpoffs=0 / rd=5 / psy-rd=1.50 / rdoq-level=1 / psy-rdoq=5.00 / no-rd-refine / signhide / deblock=-2:-2 / no-sao / no-sao-non-deblock / b-pyramid / cutree / no-intra-refresh / rc=crf / crf=18.0 / qcomp=0.80 / qpmin=0 / qpmax=51 / qpstep=4 / ipratio=1.40 / pbratio=1.20

Video
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main@L4@Main
Width : 1 920 pixels
Height : 800 pixels
Display aspect ratio : 2.40:1
Frame rate : 24.000 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Writing library : x265 1.9+73-6d06de58c316:[Windows][GCC 5.3.0][64 bit] 8bit
Encoding settings : wpp / ctu=32 / min-cu-size=8 / max-tu-size=16 / tu-intra-depth=2 / tu-inter-depth=2 / me=3 / subme=5 / merange=44 / no-rect / no-amp / max-merge=4 / temporal-mvp / no-early-skip / rdpenalty=2 / no-tskip / no-tskip-fast / no-strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / no-fast-intra / no-open-gop / no-temporal-layers / interlace=0 / keyint=360 / min-keyint=1 / scenecut=40 / rc-lookahead=80 / lookahead-slices=4 / bframes=8 / bframe-bias=0 / b-adapt=2 / ref=5 / limit-refs=2 / limit-modes / weightp / weightb / aq-mode=1 / qg-size=16 / aq-strength=1.00 / cbqpoffs=0 / crqpoffs=0 / rd=5 / psy-rd=1.50 / rdoq-level=1 / psy-rdoq=5.00 / no-rd-refine / signhide / deblock=-2:-2 / no-sao / no-sao-non-deblock / b-pyramid / cutree / no-intra-refresh / rc=crf / crf=18.0 / qcomp=0.80 / qpmin=0 / qpmax=51 / qpstep=4 / ipratio=1.40 / pbratio=1.20

Then results in a matroska container to grasp the average bitrate from mediainfo/mpc-hc:

General
Unique ID : 255088647150482398718149168969974570018 (0xBFE84951A4CB2B07B9B78C63B4105C22)
Complete name : D:\out.mkv
Format : Matroska
Format version : Version 4 / Version 2
File size : 5.20 GiB
Duration : 1h 23mn
Overall bit rate : 8 869 Kbps
Encoded date : UTC 2016-03-11 10:58:24
Writing application : mkvmerge v8.9.0 ('Father Daughter') 64bit
Writing library : libebml v1.3.3 + libmatroska v1.4.4
DURATION : 01:23:57.000000000
NUMBER_OF_FRAMES : 120888
NUMBER_OF_BYTES : 5582851354
_STATISTICS_WRITING_APP : mkvmerge v8.9.0 ('Father Daughter') 64bit
_STATISTICS_WRITING_DATE_UTC : 2016-03-11 10:58:24
_STATISTICS_TAGS : BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES

Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main@L4@Main
Codec ID : V_MPEGH/ISO/HEVC
Duration : 1h 23mn
Bit rate : 8 692 Kbps
Width : 1 920 pixels
Height : 800 pixels
Display aspect ratio : 2.40:1
Frame rate mode : Constant
Frame rate : 24.000 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Bits/(Pixel*Frame) : 0.236
Stream size : 5.10 GiB (98%)
Writing library : x265 1.9+73-6d06de58c316:[Windows][GCC 5.3.0][64 bit] 8bit
Encoding settings : wpp / ctu=32 / min-cu-size=8 / max-tu-size=16 / tu-intra-depth=2 / tu-inter-depth=2 / me=3 / subme=5 / merange=44 / no-rect / no-amp / max-merge=4 / temporal-mvp / no-early-skip / rdpenalty=2 / no-tskip / no-tskip-fast / no-strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / no-fast-intra / no-open-gop / no-temporal-layers / interlace=0 / keyint=360 / min-keyint=1 / scenecut=40 / rc-lookahead=80 / lookahead-slices=4 / bframes=8 / bframe-bias=0 / b-adapt=2 / ref=5 / limit-refs=2 / limit-modes / weightp / weightb / aq-mode=1 / qg-size=16 / aq-strength=1.00 / cbqpoffs=0 / crqpoffs=0 / rd=5 / psy-rd=1.50 / rdoq-level=1 / psy-rdoq=5.00 / no-rd-refine / signhide / deblock=-2:-2 / no-sao / no-sao-non-deblock / b-pyramid / cutree / no-intra-refresh / rc=crf / crf=18.0 / qcomp=0.80 / qpmin=0 / qpmax=51 / qpstep=4 / ipratio=1.40 / pbratio=1.20
Default : Yes
Forced : No

My conclusions are:

The source video was 15.3GB big (16.441.757.770 bytes) with an average bitrate of 25.6 Mbps, we are ending up with a x265 compressed video file which weights 5.19GB (5.582.852.322 bytes) so that's nearly a 1/3rd of the original so that's quite good, visual quality seemed rather decent in indoor scenes, looked rather "not so HD content looking anymore but acceptable still" in outdoor scenes. It took no less than 31095 seconds (8 hours and 38 minutes roughly, which is acceptable by my standards (once again though, the movie length is only 1h23m including intro & credits which both compress very fast and very well and the frame size is 1920x800 "only"), however the problem I see is:

Once the video gets merged with it's 910MB DTS track, we end up with a movie that's ready for "archiving" at the size of 6.08GB (6.534.470.512 bytes) while it's x264 counterpart yielded a file that was 6.56 GB (7.041.497.802 bytes).

My final conclusion is; Arguably the x264 encode seemed slightly better than the x265 in terms of visual quality on outdoor scenes and we computed this for roughly 8 hours 40 minutes only to shave off... 507MB (507.027.290 bytes).

Overall would I say this was this worth it? Hard to tell. It would be worth it if the quality would be on par and I would have shaved off at least 1GB of data compared to a x264 encode and of course kept identical quality.

In our case, I have over 2600 movies to re-encode and archive for our online digital library, given we could shave off 1GB per movie/encode, that's potentially 2.6TB of data saved up, right there... at the expense of power consumption bill and patience, lots of it.

I'm doing another test with the same parameters just with --preset veryslow to see if I can get the file slightly smaller. Will post results as soon as I have them.

Thank you again for your attention.

[littlepox]

I'm doing another test with the same parameters just with --preset veryslow to see if I can get the file slightly smaller. Will post results as soon as I have them.

Thank you again for your attention.

Be careful when you are switching to --preset veryslow because I have manually set a lot parameters.

This is what you should do for the --preset veryslow (otherwise the actual parameters shall be rather identical)

--preset veryslow --ctu 32 --max-tu-size 16 --crf 18 --tu-intra-depth 2 --tu-inter-depth 2 --rdpenalty 2 --keyint 360 --min-keyint 1 --bframes 8 --aq-mode 1 --aq-strength 1.0 --rd 5 --psy-rd 1.5 --psy-rdoq 5.0 --rdoq-level 1 --no-sao --no-open-gop --rc-lookahead 80 --max-merge 4 --qcomp 0.8 --no-strong-intra-smoothing --deblock -2:-2 --qg-size 16 --pbratio 1.2

Just be patient.

[pingfr]

Unfortunately I am out at the moment and have neither physical nor intranet access to the encoding box as we speak.

I shall return in a few hours, will do a test with your newest params once I get back.

Thanks!

[LigH]

Some more bitrate control enhancements in x265 1.9+88-b6d8e66e7f71 (https://www.mediafire.com/download/0kv17e8g4vmy2rj/x265_1.9+88-b6d8e66e7f71.7z)

[pingfr]

@littlepox: Just got back, will interrupt current encoding and restart a newer one with the latest parameters you gave me a few minutes ago.

@LigH: Looking at the source https://bitbucket.org/multicoreware/x265/commits/all I don't see any bitrate control related changes, which commit are you refering to?

[LigH]

Some are abbreviated, some a bit indirect ... e.g. "rc: change reencode position for cappedvbr" ("rc" = "rate control"; cappedvbr ~ VBV limited VBR with instantly re-encoded GOPs), or the tweaks for "--tune grain"; even "sao: Use qp of encoded CU, instead of slice qp" (using a better quantization base value should improve the bitrate distribution slightly).

[TalasNetrag]

Yes, a kind of "motion trails" with alternating distance, which seems to be related to mini-GOP ranges (means, the group of B frames between a pair of P, maybe I, frames).

I tried playing aroung with the minGOP and maxGOP settings. minGOP didnt do anything, maxGOP also didnt do much until I set it to 0, with also introduced a lot of noise.

[LigH]

Both options are mostly unrelated to this issue. They set the limits of distances between I frames. But the issue is in the B frames, I would guess. I doubt you can fix that easily with any available CLI options, maybe except some which are B frame related, e.g. limit the maximum number of consecutive B frames (but that would decrease the overall efficiency), or alter the use of B frame pyramids (that may be interesting).

[MeteorRain]

Once the video gets merged with it's 910MB DTS track, we end up ...

Hmm? Why using the crappy 1509k DTS?

If you are going to play with h/w decoding, go with 640k AC3.
If going s/w way, ~400k AAC or even OPUS.

Using DTS to me is like encoding your bluray with MPEG-1.

[benwaggoner]

Some are abbreviated, some a bit indirect ... e.g. "rc: change reencode position for cappedvbr" ("rc" = "rate control"; cappedvbr ~ VBV limited VBR with instantly re-encoded GOPs), or the tweaks for "--tune grain"; even "sao: Use qp of encoded CU, instead of slice qp" (using a better quantization base value should improve the bitrate distribution slightly).
And --tune grain now includes --rc-grain, a whole new rate control mode for grainy content. Seems like a "1.9.5" update just happened.

https://bitbucket.org/multicoreware/x265/commits/578c7f12b7f4ab51b5635fc0acfd89e057630623
https://bitbucket.org/multicoreware/x265/commits/54d2625ae6a18acd471708b78a3ee8d5357d2fbc
https://bitbucket.org/multicoreware/x265/commits/305a1272a412e6da50544c151820631b319de1cc

I'll try to test today or over the weekend, if I can get either of my 16-core workstations to start booting again (they were all good two days ago!).

Are those having quality issues using CRF and RC at the same time trying the new 2-pass CRF reencode mode? That should help quality when high complexity results in the VBV constraining CRF.

[benwaggoner]

...
This is based on the --tune film we have tested out. Again, do NOT modify any one of the above parameters otherwise you shall destroy the whole combination.
Has anyone tried doing a separate "tune animation." Based on x264, I'd think optimal cel animation settings would be substantially different than the film ones. Although there may be some overlap in constraining unit size and such.

[pingfr]

@littlepox: ETA is still roughly 8 hours, this encode test should be done around 4:00AM UTC+1. Until then, don't hold your breath. :p

(Regardless of the results, once this encode is done, I'll upgrade to the latest 1.9+88 as benwaggoner more or less tagged it as a "1.9.5").

[TalasNetrag]

Both options are mostly unrelated to this issue. They set the limits of distances between I frames. But the issue is in the B frames, I would guess. I doubt you can fix that easily with any available CLI options, maybe except some which are B frame related, e.g. limit the maximum number of consecutive B frames (but that would decrease the overall efficiency), or alter the use of B frame pyramids (that may be interesting).

*sails over the head* I'll just make lossless backup with x264 and try newer versions of x265 when they come out.

[littlepox]

@littlepox: ETA is still roughly 8 hours, this encode test should be done around 4:00AM UTC+1. Until then, don't hold your breath. :p

(Regardless of the results, once this encode is done, I'll upgrade to the latest 1.9+88 as benwaggoner more or less tagged it as a "1.9.5").

I've said this combination is tested with v1.9+1. It is NOT going to be optimal anymore for later versions.

[roo1234]

Has anyone tried doing a separate "tune animation." Based on x264, I'd think optimal cel animation settings would be substantially different than the film ones. Although there may be some overlap in constraining unit size and such.
I'm really interested in some animation tune for really low bitrates. I mean low, like 100-200kbps for mobile content. HEVC seems killer for that.

[pingfr]

@littlepox: The newest results are here. :)

The source is identical so we might as well refer to the other post if needed.

The newest parameters just so we agree on what we're talking about:

@echo off
avs4x265.exe -P x265.exe --preset veryslow --ctu 32 --max-tu-size 16 --crf 18 --tu-intra-depth 2 --tu-inter-depth 2 --rdpenalty 2 --keyint 360 --min-keyint 1 --bframes 8 --aq-mode 1 --aq-strength 1.0 --rd 5 --psy-rd 1.5 --psy-rdoq 5.0 --rdoq-level 1 --no-sao --no-open-gop --rc-lookahead 80 --max-merge 4 --qcomp 0.8 --no-strong-intra-smoothing --deblock -2:-2 --qg-size 16 --pbratio 1.2 --output newparams.hevc %1
pause

The console log output for reference:

avs [info]: AviSynth 2.60 (ICL10)
avs [info]: Video colorspace: YV12
avs [info]: Video resolution: 1920x800
avs [info]: Video framerate: 24/1
avs [info]: Video framecount: 120888
avs4x265 [info]: "x265.exe" - --frames 120888 --fps 24/1 --input-res 1920x800 --input-csp i420 --preset veryslow --ctu 32 --max-tu-size 16 --crf 18 --tu-intra-depth 2 --tu-inter-depth 2 --rdpenalty 2 --keyint 360 --min-keyint 1 --bframes 8 --aq-mode 1 --aq-strength 1.0 --rd 5 --psy-rd 1.5 --psy-rdoq 5.0 --rdoq-level 1 --no-sao --no-open-gop --rc-lookahead 80 --max-merge 4 --qcomp 0.8 --no-strong-intra-smoothing --deblock -2:-2 --qg-size 16 --pbratio 1.2 --output newparams.hevc
yuv [info]: 1920x800 fps 24/1 i420p8 unknown frame count
raw [info]: output file: newparams.hevc
x265 [info]: HEVC encoder version 1.9+73-6d06de58c316
x265 [info]: build info [Windows][GCC 5.3.0][64 bit] 8bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI2
x265 [info]: Main profile, Level-4 (Main tier)
x265 [info]: Thread pool created using 16 threads
x265 [info]: frame threads / pool features : 5 / wpp(25 rows)
x265 [info]: Coding QT: max CU size, min CU size : 32 / 8
x265 [info]: Residual QT: max TU size, max depth : 16 / 2 inter / 2 intra
x265 [info]: ME / range / subpel / merge : star / 57 / 4 / 4
x265 [info]: Keyframe min / max / scenecut : 1 / 360 / 40
x265 [info]: Intra 32x32 TU penalty type : 2
x265 [info]: Lookahead / bframes / badapt : 80 / 8 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 1
x265 [info]: References / ref-limit cu / depth : 5 / 0 / 1
x265 [info]: AQ: mode / str / qg-size / cu-tree : 1 / 1.0 / 16 / 1
x265 [info]: Rate Control / qCompress : CRF-18.0 / 0.80
x265 [info]: tools: rect amp limit-modes rd=5 psy-rd=1.50 rdoq=1 psy-rdoq=5.00
x265 [info]: tools: signhide tmvp b-intra deblock(tC=-2:B=-2)

x265 [info]: frame I: 1035, Avg QP:14.19 kb/s: 28438.99
x265 [info]: frame P: 23470, Avg QP:16.46 kb/s: 15379.48
x265 [info]: frame B: 96383, Avg QP:18.74 kb/s: 7048.38
x265 [info]: Weighted P-Frames: Y:2.7% UV:1.4%
x265 [info]: Weighted B-Frames: Y:1.8% UV:1.2%
x265 [info]: consecutive B-frames: 12.4% 5.8% 7.7% 15.8% 9.2% 27.2% 8.1% 6.7% 7.1%

encoded 120888 frames in 43255.58s (2.79 fps), 8848.98 kb/s, Avg QP:18.26
Press a key to continue...

And the resulted file size: 5.18GB (5.572.021.384 bytes).

With --preset veryslow and the newest parameters, we merely shaved off 11MB extra over the previous encode with --preset slower, not that impressive.

Also, it took no less than 43255 seconds to encode, in human time, it's 720 minutes or 12 hours and 55 minutes, almost 13 hours only to shave off 11MB extra, that's negligible.

I think for now I'll stick to the previous encode with --preset slower over this --preset veryslow non-sense, the other encode took "only" 8 hours and half.

Also for what it's worth:

Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main@L4@Main
Codec ID : V_MPEGH/ISO/HEVC
Duration : 1h 23mn
Bit rate : 8 675 Kbps
Width : 1 920 pixels
Height : 800 pixels
Display aspect ratio : 2.40:1
Frame rate mode : Constant
Frame rate : 24.000 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Bits/(Pixel*Frame) : 0.235
Stream size : 5.09 GiB (98%)
Writing library : x265 1.9+73-6d06de58c316:[Windows][GCC 5.3.0][64 bit] 8bit
Encoding settings : wpp / ctu=32 / min-cu-size=8 / max-tu-size=16 / tu-intra-depth=2 / tu-inter-depth=2 / me=3 / subme=4 / merange=57 / rect / amp / max-merge=4 / temporal-mvp / no-early-skip / rdpenalty=2 / no-tskip / no-tskip-fast / no-strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / no-fast-intra / no-open-gop / no-temporal-layers / interlace=0 / keyint=360 / min-keyint=1 / scenecut=40 / rc-lookahead=80 / lookahead-slices=0 / bframes=8 / bframe-bias=0 / b-adapt=2 / ref=5 / limit-refs=1 / limit-modes / weightp / weightb / aq-mode=1 / qg-size=16 / aq-strength=1.00 / cbqpoffs=0 / crqpoffs=0 / rd=5 / psy-rd=1.50 / rdoq-level=1 / psy-rdoq=5.00 / no-rd-refine / signhide / deblock=-2:-2 / no-sao / no-sao-non-deblock / b-pyramid / cutree / no-intra-refresh / rc=crf / crf=18.0 / qcomp=0.80 / qpmin=0 / qpmax=51 / qpstep=4 / ipratio=1.40 / pbratio=1.20
Default : Yes
Forced : No

Thank you for your time and efforts. Cheers.

Good night to you.

Edit: Oh also, the previous --preset slower encode was probably better as MPC-HC reported a bitrate of 8 692 Kbps versus 8 675 Kbps here. It's not that big of a difference but every little bit helps I guess (pun intended), that of course... and the 8h38m encode time vs. 12h55m, I can't possibly justify keeping a server's ressources maxed out for nearly 5 hours more just to shave off 11MB.

[djesteban]

@ foxy etc.: Can you confirm that zones don't work for n-pass, --pass 3|2?

I am having trouble with this too... and trouble finding answers.
I was able to make zones work in CRF mode, but it doesn't seem to have any effect on 2pass encodes.

Can anyone confirm this?!

[littlepox]

Has anyone tried doing a separate "tune animation." Based on x264, I'd think optimal cel animation settings would be substantially different than the film ones. Although there may be some overlap in constraining unit size and such.

We have that as well. For anime contents basically you wish to encourage bits to be invested in the edge areas, and that can be done by tweaking psy and aq.


Will probably do another extensive test and post our suggestions here after v2.0. Our current parameters are updated with v1.9 stable.

[pingfr]

@littlepox: No feedback or conclusions on the last encode that ended up a few hours ago? :p

[littlepox]

@littlepox: No feedback or conclusions on the last encode that ended up a few hours ago? :p

No. This was just a repeat of our previous tests suggesting higher presets shall only waste your time with little improvement.

[pingfr]

@littlepox: And it turned out we both came to the same conclusions, more or less. :p

I'm currently pondering whether should I request more funds from our IT department boss to upgrade to a dual Xeon E5-26xx v3 right away or if I should instead wait a few extra months/weeks for Intel to "release" their upcoming and highly anticipated newer Broadwell-EP v4 lines of Xeon CPUs.

It would help me decide if I knew what kind of fps crunching improvement I would get... but meh, I disgress.

Thanks for all the fish!

[Boulder]

Out of interest, do you have to keep everything at the original resolution? Upon playback, a huge amount of Blu-rays look just fine, if not the same as the original, when downscaled to 720p with some sharpening.

[x265_Project]

I'm currently pondering whether should I request more funds from our IT department boss to upgrade to a dual Xeon E5-26xx v3 right away or if I should instead wait a few extra months/weeks for Intel to "release" their upcoming and highly anticipated newer Broadwell-EP v4 lines of Xeon CPUs.

v4 Xeons (Broadwell) use the same basic CPU architecture as v3 (Haswell), so you shouldn't expect to see much in the way of instructions-per-clock improvement. You will be able to get up to 22 cores per chip with v4 Xeons, but it's likely that the price per core will probably be similar for a while after the v4 release. v5 Xeons (Skylake) will support AVX-512 and higher internal ring bandwidth, but I don't expect to see them until late 2016 or early 2017.

[pingfr]

@x265_Project: Hey Tom, good to have you here.

So basically what you're saying is, that a dual v3 encoding box will more or less yield identical encoding speed (fps crunching) to a dual v4 Broadwell-EP because there aren't any newer set of instruction introduced?

[x265_Project]

@x265_Project: Hey Tom, good to have you here.

So basically what you're saying is, that a dual v3 encoding box will more or less yield identical encoding speed (fps crunching) to a dual v4 Broadwell-EP because there aren't any newer set of instruction introduced?
Hey... I'm always here... but I'm letting our experts, nandaku2 and pradeeprama handle more of the technical questions.

That's right... Broadwell is basically a die shrink of Haswell, going from 22 nm to 14 nm. As with any new chip, there will be refinements in timings and power utilization... but the logic units and architecture are basically the same, and there are no new instructions in Broadwell. Because of the die shrink, however, (according to leaked information (http://wccftech.com/intel-broadwellep-xeon-e52600-v4-skus-leaked/)) there will be larger Xeons available with more cores.

On the consumer side there were more changes... primarily bigger, better GPUs (Intel Gen Graphics).

[LigH]

I am having trouble with this too... and trouble finding answers.
I was able to make zones work in CRF mode, but it doesn't seem to have any effect on 2pass encodes.

Can anyone confirm this?!

I just tested the Sintel trailer in a smaller size (640x272) in 3 passes with verbose CSV log files. Plotting the average QP per frame (in encoded order only, displayed order is too hard to reconstruct in Excel) reveals a quite confusing behaviour in "--pass 2", while it looked about as expected in "--pass 3". Developers got details via mailing list.

[aegisofrime]

Hey... I'm always here... but I'm letting our experts, nandaku2 and pradeeprama handle more of the technical questions.

That's right... Broadwell is basically a die shrink of Haswell, going from 22 nm to 14 nm. As with any new chip, there will be refinements in timings and power utilization... but the logic units and architecture are basically the same, and there are no new instructions in Broadwell. Because of the die shrink, however, (according to leaked information (http://wccftech.com/intel-broadwellep-xeon-e52600-v4-skus-leaked/)) there will be larger Xeons available with more cores.

On the consumer side there were more changes... primarily bigger, better GPUs (Intel Gen Graphics).

While we are on this topic, what would you say would be the performance increase upgrading from a i7-4770 to a i7-6700?

[x265_Project]

While we are on this topic, what would you say would be the performance increase upgrading from a i7-4770 to a i7-6700?
For most scenarios, the Skylake i7-6700 is about 20% faster, due to the higher clock speed. For 4K ultrafast it is more than 2x faster, due to higher internal memory bandwidth (which bottlenecks the Haswell i7-4770).

[pingfr]

Since we're talking hardware, I'll drop my experiments here:

In the lab we're using a 5960X (8C/16T) which has it's stock speed clocked at 3GHz with a Turbo speed of 3.5GHz, I instantly overclocked it to 4GHz with a very basic cooler and without any extra tweaks (I could most likely push it farther towards the 4.5GHz even 4.7GHz clamp with proper watercooling, which in my case can't be justified in a professionnal environment).

During our crunching tests at the 4GHz frequency, I use a reference short 5 minutes (11862 frames@50fps) Ultra HD clip with the nasty resolution of 3840x2160; with the following parameters on x265-1.9+88 (10 bits):

avs4x265.exe -P x265-10b.exe -D 10 --input-depth 10 --fps 50 --input-res 3840x2160 --preset ultrafast --no-rect --crf 18 --output output.hevc %1

We're getting a 8.80 fps crunching speed, rendering the 11862 frames encoded more or less under 22 minutes.

This sunday I was granted temporary access to a "supposedly" high-end machine running a single Xeon E5 2695v3 at stock 2.2GHz (could not be overclocked at all, wasn't allowed to play around with it's BIOS or anything) and let's just say that I was REALLY REALLY disappointed.

During my tests, the Xeon E5 2695v3 running at full speed (2.2GHz) with all cores saturated at 100% at all time, gave us crunching speeds of 4.5fps which seemed abysmally low for a production server that was far more costly than our little lab's i7 5960X.

I gathered that a CPU running stable at 4GHz will pretty much "own" a 2.2GHz clocked CPU, but at the same time, that Xeon is supposed to sport 14 cores and no less than 28 threads which I initially thought would grant faster speeds over an i7 HEDT sporting "only" 8 cores and "only" 16 threads and therefore "make up" for the slower clocking speeds.

Could it be related to the fact we only found out in the end that Xeon CPU was an Engineering Sample leaked from Intel and therefore could be slower/crippled/lacking features over a retail CPU?

Who knows.

As far as I'm concerned: i7 5960X (8C/16T) OC'ed at 4GHz > Xeon E5-2695v3 ((ES) 14C/28T) at 2.2GHz any time, any day, anywhere.

[djesteban]

I just tested the Sintel trailer in a smaller size (640x272) in 3 passes with verbose CSV log files. Plotting the average QP per frame (in encoded order only, displayed order is too hard to reconstruct in Excel) reveals a quite confusing behaviour in "--pass 2", while it looked about as expected in "--pass 3". Developers got details via mailing list.

LigH, I have a thread dedicated to this problem here (http://forum.doom9.org/showthread.php?t=173290).
Would be nice if you could post your result there also to attract more attention on the issue.

I'll make my own test on my side and post my result there. If there's indeed an issue, I'll log a bug on bitbucket I guess...

[x265_Project]

In the lab we're using a 5960X (8C/16T) which has it's stock speed clocked at 3GHz with a Turbo speed of 3.5GHz, I instantly overclocked it to 4GHz with a very basic cooler and without any extra tweaks...

During our crunching tests at the 4GHz frequency, I use a reference short 5 minutes (11862 frames@50fps) Ultra HD clip with the nasty resolution of 3840x2160; with the following parameters on x265-1.9+88 (10 bits):

avs4x265.exe -P x265-10b.exe -D 10 --input-depth 10 --fps 50 --input-res 3840x2160 --preset ultrafast --no-rect --crf 18 --output output.hevc %1

We're getting a 8.80 fps crunching speed, rendering the 11862 frames encoded more or less under 22 minutes.

This sunday I was granted temporary access to a "supposedly" high-end machine running a single Xeon E5 2695v3 at stock 2.2GHz (could not be overclocked at all, wasn't allowed to play around with it's BIOS or anything) and let's just say that I was REALLY REALLY disappointed.

During my tests, the Xeon E5 2695v3 running at full speed (2.2GHz) with all cores saturated at 100% at all time, gave us crunching speeds of 4.5fps which seemed abysmally low for a production server that was far more costly than our little lab's i7 5960X.

I gathered that a CPU running stable at 4GHz will pretty much "own" a 2.2GHz clocked CPU, but at the same time, that Xeon is supposed to sport 14 cores and no less than 28 threads which I initially thought would grant faster speeds over an i7 HEDT sporting "only" 8 cores and "only" 16 threads and therefore "make up" for the slower clocking speeds.

As far as I'm concerned: i7 5960X (8C/16T) OC'ed at 4GHz > Xeon E5-2695v3 ((ES) 14C/28T) at 2.2GHz any time, any day, anywhere.
You're seeing Amdahl's Law (https://en.wikipedia.org/wiki/Amdahl's_law) in action. While we've done everything possible to parallelize x265, there are certain functions that are inherently serial (CABAC encoding and decoding, for example). Even for the operations we've parallelized, there are often dependencies on other threads.

For example, if you are running 8 threads on a quad-core machine, most threads will be busy most of the time. Occasionally, one or more of the 8 threads will be waiting for the result from another thread, and so it will be stalled. If you double the power of your machine (run a 5960x with 8 cores/16 threads), you will not get 2x the performance, due to Amdahl's Law. Now, the number of threads that might be waiting for another thread to finish goes up. In x265 we have multiple frame encoders, which call row encoders, which call CTU encoders. Let's say one of the CTU encoders encounters some complex video, and it doesn't easily find good prediction matches in its reference frames. It will take longer than average to encode, and it will stall the row encoder, which may stall other row encoders that depend on that CTU being done (so that it can be referenced).

So, it is an ongoing challenge to get the highest possible effective utilization from many-core machines. We have a private commercial encoding library called UHDkit that can break the incoming video into chunks, encoding each chunk with a separate instance of x265. In this way we can gain higher effective utilization on many-core machines, such as the dual Xeon E5-2699 v3 (2 x 18 cores = 36 cores / 72 threads). The downside is added latency for live encoding, as you have to queue up enough video to keep all encoder instances busy.

[pingfr]

We have a private commercial encoding library called UHDkit that can break the incoming video into chunks, encoding each chunk with a separate instance of x265.

Hey Tom,

Sounds good but on our little test-bed project, I already fought hard with my department's manager and the IT department's boss/office chief to allocate funds to acquire a 5960X...

In the coming days I will have to fight again to convince them to either allocate even more extra funds or to grab a dual E5-26XX-v3 server from the next IT department...

So now if on top of that, I also have to convince them to acquire expensive licenses for a private commercial encoding library... my head will be on a pike, literally. :)

But yes, this definitely looks sexy:

http://www.multicorewareinc.com/wp-content/uploads/2015/04/x265_88FPS.jpg

:drool:

[stax76]

StaxRip should now be up-to-date in regard of x265, changes in the last build are:



larger custom command line TextBox
new switch added --rc-grain
tune grain defaults updated
the encoding options dialogs for x265, AMD, Intel and NVIDIA have now an option to display the full command line
removed one switch that was accidentally added twice :-)
x265 1.9+88


Download: http://forum.doom9.org/showthread.php?p=1760640#post1760640

http://oi64.tinypic.com/c26id.jpg

[pingfr]

@stax76: Congrats! (I'm more of a MeGUI guy myself but nevertheless, grats to you!). :p

Edit: Also good to see those settings littlepox gave us a few days ago are being added as a "custom" profile to StaxRip, way to go! :)

[benwaggoner]

Will probably do another extensive test and post our suggestions here after v2.0. Our current parameters are updated with v1.9 stable.
I don't know there's any indication that 1.9 will be followed by a "super major" 2.0 release, other than base 10 :). Next up might be 1.10 AFAIK. Or maybe 1.A if it turns out MCW was using hex all along...

1.9 is a good release, and I don't think there's any reason to wait to make animation tuning. And the earlier it is worked on, the more feedback can go into the next version. I'd love to see a proper --tune animation and --tune film in there .

[LigH]

@ benwaggoner:

The x265 team already documented that they will prefer a continuous decimal version progression. After v1.9 will quite certainly follow v2.0 without any "super major" speciality attribute, v1.0 wasn't special either after v0.9.

[pingfr]

@LigH: How about... x265-v2.0-reloaded-extended-unleashed-extreme-edition-super-combo-deluxe+1 ? :)

[littlepox]

I don't know there's any indication that 1.9 will be followed by a "super major" 2.0 release, other than base 10 :). Next up might be 1.10 AFAIK. Or maybe 1.A if it turns out MCW was using hex all along...

1.9 is a good release, and I don't think there's any reason to wait to make animation tuning. And the earlier it is worked on, the more feedback can go into the next version. I'd love to see a proper --tune animation and --tune film in there .

I mean currently they are doing quite a lot in RC adjustments. Our combination works quite different under 1.9 stable and the latest builds. So we might just wait until v2.0, when the change is more or less verified and settled.

For people interested in --tune animation, here it is:

--ctu 32 --max-tu-size 16 --tu-intra-depth 2 --tu-inter-depth 2 --rdpenalty 2 --weightb --keyint 360 --min-keyint 1 --aq-mode 1 --aq-strength 1.1 --rd 5 --psy-rd 1.5 --psy-rdoq 3.0 --pbratio 1.2 --rdoq-level 1 --no-sao --no-open-gop --rc-lookahead 80 --scenecut 40 --max-merge 4 --qcomp 0.8 --no-strong-intra-smoothing --deblock -2:-2 --qg-size 16

if your ref is <3, pls increase it by 1.
if your bframes is <4, pls increase it by 2.


BTW, the definition of "animation" is quite ambiguous. We assume you are referring to those with sharp edges, simple colors, and very tiny grains.
However, with developments recently in Japan, this is also "animation": http://img.2222.moe/images/2016/02/23/21114.png

Should you are dealing with anime of high quality (usually BD source) and you wish to retain the high quality (typically with x264_crf<17), you are more encouraged to use our --tune film.

[benwaggoner]

I mean currently they are doing quite a lot in RC adjustments. Our combination works quite different under 1.9 stable and the latest builds. So we might just wait until v2.0, when the change is more or less verified and settled.

For people interested in --tune animation, here it is:

--ctu 32 --max-tu-size 16 --tu-intra-depth 2 --tu-inter-depth 2 --rdpenalty 2 --ref 5 --weightb --keyint 360 --min-keyint 1 --bframes 8 --aq-mode 1 --aq-strength 1.1 --rd 5 --psy-rd 1.5 --psy-rdoq 3.0 --pbratio 1.2 --rdoq-level 1 --no-sao --no-open-gop --rc-lookahead 80 --scenecut 40 --max-merge 4 --qcomp 0.8 --no-strong-intra-smoothing --deblock -2:-2 --qg-size 16
No --qg-size? I would have guessed that 16 would be helpful to push down quant changes to small blocks that would contain lines.

BTW, the definition of "animation" is quite ambiguous. We assume you are referring to those with sharp edges, simple colors, and very tiny grains.
However, with developments recently in Japan, this is also "animation": http://img.2222.moe/images/2016/02/23/21114.png

Should you are dealing with anime of high quality (usually BD source) and you wish to retain the high quality (typically with x264_crf<17), you are more encouraged to use our --tune film.
I was thinking traditional cel animation. I agree that the introduction of continuous tones like in your image would make it more like film content in terms of adaptive quant requirements. The lack of motion blur and large areas that are identical between frames will remain a significant difference, although I'm not sure if they would impact tuning. More likely, they just will make encoding more efficient.

[littlepox]

No --qg-size? I would have guessed that 16 would be helpful to push down quant changes to small blocks that would contain lines.

check the last one.
Also I removed the hard coded --ref and --bframes


I was thinking traditional cel animation. I agree that the introduction of continuous tones like in your image would make it more like film content in terms of adaptive quant requirements. The lack of motion blur and large areas that are identical between frames will remain a significant difference, although I'm not sure if they would impact tuning. More likely, they just will make encoding more efficient.

We are actually doing the opposite types, which, at least in high quality, even for x264 we are using a tuning more close to --tune film.

[x265_Project]

@LigH: How about... x265-v2.0-reloaded-extended-unleashed-extreme-edition-super-combo-deluxe+1 ? :)

That's got my vote!

[pingfr]

That's got my vote!

Glad you like it, watch out tho: I might ask for royalties fees if you ever go with that one! :devil:

[pingfr]

@x265_Project: Contemplating the idea of grabbing 2x E5-2687W v3 (10C/20T at 3.1GHz with Turbo at 3.5GHz).

http://ark.intel.com/products/81909/Intel-Xeon-Processor-E5-2687W-v3-25M-Cache-3_10-GHz

But unsure if any other CPU combo would be any faster...

Also a test that might somehow be relevant: http://www.anandtech.com/show/8730/intel-haswellep-xeon-14-core-review-e52695-v3-and-e52697-v3/3

Any ideas on that matter?

[Atak_Snajpera]

@LigH: How about... x265-v2.0-reloaded-extended-unleashed-extreme-edition-super-combo-deluxe+1 ?
You forgot ULTRA and TURBO words.

[LigH]

*LigH prepares a "Bullsh.../Buzzword Bingo" sheet...*

[x265_Project]

You forgot ULTRA and TURBO words.
Not to mention "Titanium Edition"

[x265_Project]

@x265_Project: Contemplating the idea of grabbing 2x E5-2687W v3 (10C/20T at 3.1GHz with Turbo at 3.5GHz).

http://ark.intel.com/products/81909/Intel-Xeon-Processor-E5-2687W-v3-25M-Cache-3_10-GHz

But unsure if any other CPU combo would be any faster...

Also a test that might somehow be relevant: http://www.anandtech.com/show/8730/intel-haswellep-xeon-14-core-review-e52695-v3-and-e52697-v3/3

Any ideas on that matter?
Again, it depends on whether you are running a single x265 instance, or a transcode pipeline that supports multiple parallel instances. If you're running one x265 instance, lean towards faster single-threaded performance. If you're running UHDkit, get the most total compute power for your dollars.

The Anandtech review from 2014 is interesting, but it's a bit dated, given all of the AVX2 acceleration we did last year, and other algorithmic improvements.

[pingfr]

You forgot ULTRA and TURBO words.

I actually did not forget those... I was saving them up for the next milestone! :devil:

x265-2.1-reloaded-extended-unleashed-extreme-edition-super-combo-deluxe-ultra-turbo-titanium-edition+1 :devil:

[pingfr]

@x265_Project: Hey Tom.

In my case it's a single x265.exe running at once, no parallel encoding (yet), so in this case Xeon E5-2687W v3 (10C/20T@3.1GHz/3.5GHz) would "win" the encoding race versus a Xeon E5-2697 v3 (14C/28T@2.6GHz/3.6GHz) because the turbo feature actually "flattens" (don't know the technical term) each core to the stock base frequency when all cores are properly equally saturated.

So ideally it's a matter of comparing 10 cores back to 3.1GHz for the E5-2687W v3 against 14 cores back to 2.6GHz for the Xeon E5-2697 v3... hopefully I got it right.

Summarized: it's more efficient to have 10 cores running at 500MHz faster than having more (in this case, 4 extras) 14 cores running at slower speeds? (pardon the poor wording).

Speaking of the "devil"... is there any official pricing for UHDkit licensing available to the public/general audience?

[Stacey Spears]

What is the process to request a feature for x265? e.g. I have all of the UHD BD constraints and would like to know how to get them on the todo list for x265. Assuming there is interest. I would like to use it to encode my next disc. I used x264 on the current test/calibration disc.

[x265_Project]

What is the process to request a feature for x265? e.g. I have all of the UHD BD constraints and would like to know how to get them on the todo list for x265. Assuming there is interest. I would like to use it to encode my next disc. I used x264 on the current test/calibration disc.
Hi Stacey,
The official way to request a new feature is through our Bitbucket Issues tracker (https://bitbucket.org/multicoreware/x265/issues?status=new&status=open). UHD-BD compatibility is something we've been working on. One of our developers submitted a patch, and it should soon be supported in the development builds. I'll check on it for you.

[Stacey Spears]

Thank you for the quick response. Would be happy to test the patch with some of the BD authoring tools if I could get a build. Would also like to make sure the developer has all of the data needed. I have it all in a PDF. I can PM you the PDF.

After I submitted my feature request, I noticed someone else already requested it. (maxqp) So I added additional comments to the existing request and you can close my new one if you like.

[Ely]

Any idea what encoder is used by professionals when authoring current UDH BD ?

[LigH]

There is already a number of HEVC products (https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding_implementations_and_products). Among the more famous vendors are: Ericsson, Vanguard, Fraunhofer HHI, MainConcept, Ateme (who cooperated with DivX Networks already), NHK (Japan Broadcasting Corp.) + Mitsubishi Electrics ... Ittiam Systems from India is not so famous to me, but they are developing an own HEVC encoder already for longer. And don't we forget GPU vendors like Nvidia.

So it seems that there is already a quite generous range to choose from. I guess the MainConcept SDK is probably among the best selling software solutions. Hardware encoders (usually focused on realtime encoding with lower complexity) are probably sold by general and mobile-affine electronic companies.

[Stacey Spears]

Any idea what encoder is used by professionals when authoring current UDH BD ?

Several UHD BD titles have been authored using Ateme. Biggest issue right now is that it does not offer segment re-encoding.

[2themax]

Several UHD BD titles have been authored using Ateme. Biggest issue right now is that it does not offer segment re-encoding.
Sirius Pixels can't get their encoder to market fast enough to solve that problem.

[pingfr]

Hey guys.

I'm facing a small dilemma here, which would be faster for x265 (no parallel encoding), between these 2 architectures:

2*E5 26XX v3 8C/16T@2.9GHz (35M cache L3) vs 2*E5 26XX v3 12C/24T@2.8GHz (30M cache L3).

If anyone with moderate x86-64 architecture knowledge could shed some light, it would be greatly appreciated, thanks.

[Atak_Snajpera]

8C vs 12C??? Choice is obvious.

[littlepox]

8C*2 vs 12C*2
No parallel encoding
Then it is not obvious anymore.
The answer, as far as I have tested, heavily depends on the source and your settings.
Without further infomation I'd suggest 12C since the frequence lead for 8C is so small.

[Atak_Snajpera]

12C@2.8Ghz will always be faster than 8C@2.9GHz. Period. Ofcourse speed-up will be much less than 1.5x.

[LigH]

x265 1.9+96-b09998b1256e (https://www.mediafire.com/download/npaaaw9gdfaoeln/x265_1.9+96-b09998b1256e.7z) should have fixed the bitrate control for zones.

[pingfr]

@Atak_Snajpera & @littlepox: Thanks for the heads up guys, I shall follow your advices and cancel my pending order of 2* 8C@2.9GHz and head for the 2* 12C@2.8GHz route. :)

[x265_Project]

Hey guys.

I'm facing a small dilemma here, which would be faster for x265 (no parallel encoding), between these 2 architectures:

2*E5 26XX v3 8C/16T@2.9GHz (35M cache L3) vs 2*E5 26XX v3 12C/24T@2.8GHz (30M cache L3).

If anyone with moderate x86-64 architecture knowledge could shed some light, it would be greatly appreciated, thanks.
Agreed... the extra cores will be a bigger benefit than 100 MHz of clock and 5 MB of cache.

[pingfr]

Agreed... the extra cores will be a bigger benefit than 100 MHz of clock and 5 MB of cache.

Thanks a bunch! much.. err.. "love". :)

[benwaggoner]

Agreed... the extra cores will be a bigger benefit than 100 MHz of clock and 5 MB of cache.
IF the output frame size is large enough. A 640x272 might not see any benefit.

[pingfr]

@benwaggoner: Mostly encoding 1920x1040, 1920x868, 1920x808, 1920x800 then a couple of a 1280x560 and 1280x532 cropped material IIRC.

[Grojm]

Any news on this issue? https://bitbucket.org/multicoreware/x265/issues/214/ghosting-artefacts-even-with-low-crf-when
This is a tremendous bug leading to a huge impact on video quality. It has been reported months ago. But little attention by the developers.

[LigH]

I just checked the small samples ({u}hd177.y4m) from the BitBucket issue tracker with x265 v1.9+96 (all presets incl. "-D 10 --crf 15" as documented); indeed, there are choppy/flickering/inconsistent motions for small objects. Preset "slower" appears to produce the most obvious issues. I tested with several parameters which should change the behaviour of mini-GOPs (B-frame bias; B-frame pyramid; weighted P/B frames); the effect did not disappear.

For some reason, I don't see any "Reply" button in the BitBucket issue tracker, despite being logged in; is there a user/project permission required? Or maybe it's just incompatible with Opera 12, or I'm missing a JavaScript import via proxy. It works in Pale Moon.

[littlepox]

I just checked the small samples ({u}hd177.y4m) from the BitBucket issue tracker with x265 v1.9+96 (all presets incl. "-D 10 --crf 15" as documented); indeed, there are choppy/flickering/inconsistent motions for small objects. Preset "slower" appears to produce the most obvious issues. I tested with several parameters which should change the behaviour of mini-GOPs (B-frame bias; B-frame pyramid; weighted P/B frames); the effect did not disappear.



try --qcomp 0.8 --psy-rd 0.3

[LigH]

@ littlepox:

Nope, always flickering still. Archived result. (http://www.ligh.de/tmp/uhd177_slower.7z)

[pingfr]

@ littlepox:

Nope, always flickering still. Archived result. (http://www.ligh.de/tmp/uhd177_slower.7z)

For what it's worth, can't see much either... I mean 384x160 all pixellated garbage... could use some 576p material at least. :p

[Ma]

Could someone tell me where in source code is function for choose object/shape that will be moved?

[Grojm]

For what it's worth, can't see much either... I mean 384x160 all pixellated garbage... could use some 576p material at least. :p

Try the original Star Wars example, attached in comment #4 on bitbucket. It's 720p and the effect is notable at normal resolutions.

[LigH]

I'll try this, but because I have no AVX capable CPU, it will take a longer while to test that...

[pingfr]

I'll try this, but because I have no AVX capable CPU, it will take a longer while to test that...

Hey LigH,

Any ideas of "how longer" are we talking about here for a CPU without AVX and AVX 2.0 instruction sets compared to a CPU with said instructions?

In terms of either time spent/percentages/fps, any rough guesses will do.

Just out of curiosity.

[LigH]

I don't know, because I can't compare. If you have a CPU which supports better instructions, you can disable their use. But not vice-versa.

With an AMD Phenom-II, the maximum supported instruction set is "SSE2-Fast". Not even SSE3 or SSSE4.

[pingfr]

Hey guys,

Having a few issues to get x265 to compile on Windows systems here.

- SourceTree is installed & I cloned the repo succesfully.
- YASM 1.30 is installed as well and is in the path (in fact I just copy'ed over in %windir%).
- GCC is installed (gcc version 5.3.0 (x86_64-posix-seh-rev0, Built by MinGW-W64 project).
- CMake 3.5.0 is installed as well (and cmake-gui as well).

I have succesfully cloned the repo but in the build subfolder I only see arm-linux, linux, msys, vc9 vc10 vc11 vc12 and xcode sub-subdirs.

Excuse me if I'm a bit "clueless" here, but I don't see a gcc subdir and can't locate a CMakeLists.txt to use with cmake-gui tool either.

If anyone could enlight me...

[LigH]

To compile for Windows, you would probably prefer to set up an MSYS/MinGW environment, as described in the VideoHelp forum (http://forum.videohelp.com/forums/11-Video-Conversion) posts about this topic ([HEVC] x265.EXE: mingw builds (http://forum.videohelp.com/threads/357754-%5BHEVC%5D-x265-EXE-mingw-builds) | x265 HEVC Encoder (http://forum.videohelp.com/threads/360069-x265-HEVC-Encoder)). You can easily download a whole prepared MSYS environment from XhmikosR (http://xhmikosr.1f0.de/tools/msys/); use a link to the included msys.bat with the parameter "-mintty" to call MinTTY as shell, it is a lot more user friendly than sh and also supports LOCALE (e.g. a translation for accent chars and umlauts). Then you can call the shell scripts inside the ~/x265/build/msys directory (~ is your home directory where you are logged in when calling the MSYS shell), after cloning x265 from the BitBucket repo there using hg.

Another approach is using jb-alvarado's media-autobuild_suite (https://github.com/jb-alvarado/media-autobuild_suite), set up to only build x265 (or in addition to ffmpeg, as you like).

[pingfr]

Moi@X99 ~/x265/x265/build/msys
$ cmake -G "MSYS Makefiles" ../../source && cmake-gui ../../source
-- cmake version 3.5.0
-- The C compiler identification is GNU 5.3.0
-- The CXX compiler identification is GNU 5.3.0
-- Check for working C compiler: C:/MSYS/mingw/bin/gcc.exe
-- Check for working C compiler: C:/MSYS/mingw/bin/gcc.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: C:/MSYS/mingw/bin/g++.exe
-- Check for working CXX compiler: C:/MSYS/mingw/bin/g++.exe -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detected x86 target processor
-- Looking for include file inttypes.h
-- Looking for include file inttypes.h - found
-- Performing Test CC_HAS_NO_STRICT_OVERFLOW
-- Performing Test CC_HAS_NO_STRICT_OVERFLOW - Success
-- Performing Test CC_HAS_NO_NARROWING
-- Performing Test CC_HAS_NO_NARROWING - Success
-- Performing Test CC_HAS_NO_ARRAY_BOUNDS
-- Performing Test CC_HAS_NO_ARRAY_BOUNDS - Success
-- Performing Test CC_HAS_FAST_MATH
-- Performing Test CC_HAS_FAST_MATH - Success
-- Performing Test CC_HAS_STACK_REALIGN
-- Performing Test CC_HAS_STACK_REALIGN - Success
-- Performing Test CC_HAS_FNO_EXCEPTIONS_FLAG
-- Performing Test CC_HAS_FNO_EXCEPTIONS_FLAG - Success
-- Found yasm: C:/MSYS/bin/yasm.exe (found version "1.3.0")
-- Found Yasm 1.3.0 to build assembly primitives
-- hg found at C:/Program Files/TortoiseHg/hg.exe
-- x265 version 1.9+100-ed744166c37e
-- The ASM_YASM compiler identification is unknown
-- Found assembler: C:/MSYS/bin/yasm.exe
-- Looking for strtok_r
-- Looking for strtok_r - found
-- Looking for include file getopt.h
-- Looking for include file getopt.h - found
CMake Error at CMakeLists.txt:621 (LIST):
list sub-command REMOVE_DUPLICATES requires list to be present.


CMake Warning (dev) at CMakeLists.txt:622 (set):
Cannot set "PLATFORM_LIBS": current scope has no parent.
This warning is for project developers. Use -Wno-dev to suppress it.

-- Configuring incomplete, errors occurred!
See also "C:/MSYS/home/Moi/x265/x265/build/msys/CMakeFiles/CMakeOutput.log".
See also "C:/MSYS/home/Moi/x265/x265/build/msys/CMakeFiles/CMakeError.log".

Moi@X99 ~/

[LigH]

This forum's bbCode subset offers a CODE tag to present such pre-formatted results in a monospaced font.
__

It looks not bad yet, up to the point that there may be a bug in the preparation of the CMake directives. I'll try to confirm that.
__

OK, I have that warning about PLATFORM_LIBS too, but not about REMOVE_DUPLICATES (still using CMake 3.4.1).

[pingfr]

Don't bother, I've switched to media-autobuild_suite, it's still updating lots of missing packages but seems to be fine so far.

[LigH]

Sometimes you just get an unfortunate moment with a buggy source state. Next week it may already be fixed.

[qyot27]

The REMOVE_DUPLICATES error also shows up for me when cross-compiling under Lubuntu 16.04 Beta 1, which has CMake 3.3.2 or 3.2.2* (I can't make heads nor tails of that back-asswards version string (http://packages.ubuntu.com/xenial/cmake)...pick just one, guys). It's safe to just comment that one line out and carry on, as a band-aid before it gets fixed upstream. It seems like it's only there to eliminate duplicate copies of dependent libs from all getting linked in, or something.

*although I'm also pretty sure it's also CMake 3.5.0, since I remember seeing '3.5' or something similar when configuring the project.

[LigH]

I'm confident x265_Project will read here soon. I believe to remember that another fix in this area was already proposed (only use PLATFORM_LIBS where useful)...

[nandaku2]

I'm confident x265_Project will read here soon. I believe to remember that another fix in this area was already proposed (only use PLATFORM_LIBS where useful)...

Sorry, folks. PLATFORM_LIBS fix patch pushed.

[LigH]

Experimental UHD Bluray support (--uhd-bd) in x265 1.9+106-c8ec86965e54 (https://www.mediafire.com/download/b11kgzti9w4gglu/x265_1.9+106-c8ec86965e54.7z); is there anyone who can test and confirm?

[sneaker_ger]

BluRay patch:
if ((p->sourceWidth != 1920 && p->sourceWidth != 3840) || (p->sourceHeight != 1080 && p->sourceHeight != 1088 && p->sourceHeight != 2160))
1957
+ {
1958
+ x265_log(p, X265_LOG_ERROR, "uhd-bd: Supported resolutions are 1920x1080, 1920x1088 and 3840x2160\n");
1959
+ disableUhdBd = 1;
1960
+ }
1920x1088 is only allowed with conformance window cropping to 1920x1080 (like on regular BluRay using H.264). Since x265 does not allow to manually set conformance window cropping this should not be allowed. (but 1920x1080 input with --min-cu-size 16 should be possible)

[sneaker_ger]

/edit:
ok, hrd is working. I didn't turn vbv on and it wasn't checked by --uhd-bd.
1. But with vbv without --level there is an actual error: general_level_idc is set to 0 which isn't valid at all. Log shows strange error:
x265 [info]: Main 10 profile, Level-1.9+106-c8ec86965e54 (Main tier)
The problem does not occur if you manually specify --level 5.1.

2. x265 tries no lowers vbv from 100000 to 40000, even though the log says it is using high tier. This can be worked around again by manually specifying --high-tier.

(There's more)

[x265_Project]

BluRay patch:
if ((p->sourceWidth != 1920 && p->sourceWidth != 3840) || (p->sourceHeight != 1080 && p->sourceHeight != 1088 && p->sourceHeight != 2160))
1957
+ {
1958
+ x265_log(p, X265_LOG_ERROR, "uhd-bd: Supported resolutions are 1920x1080, 1920x1088 and 3840x2160\n");
1959
+ disableUhdBd = 1;
1960
+ }
1920x1088 is only allowed with conformance window cropping to 1920x1080 (like on regular BluRay using H.264). Since x265 does not allow to manually set conformance window cropping this should not be allowed. (but 1920x1080 input with --min-cu-size 16 should be possible)
Thanks. Do you want to sign a Contributor License Agreement (https://bitbucket.org/multicoreware/x265/downloads/x265ContributorAgreement.pdf) and submit this patch?

[sneaker_ger]

You misunderstood. It's not my patch, it is a quote from your code.

[x265_Project]

You misunderstood. It's not my patch, it is a quote from your code.
Oh... sorry. I see what you're saying in the original message. We'll take a look at this.

[J1Man]

Hi Guys. I am trying to encode 16bit Yuv 4:2:0 source material into 10bit hevc using x265 10bit build. Should I convert 16bit input into 10bit (via third party tools) before I send it to x265? Or is it better to let x265 handle 16bit to 10bit conversion?

I did some visual comparisons but I could not decide which method is the best, quality wise.

Thanks.

[littlepox]

Hi Guys. I am trying to encode 16bit Yuv 4:2:0 source material into 10bit hevc using x265 10bit build. Should I convert 16bit input into 10bit (via third party tools) before I send it to x265? Or is it better to let x265 handle 16bit to 10bit conversion?

I did some visual comparisons but I could not decide which method is the best, quality wise.

Thanks.

I don't think there is any significant difference. For me, I'd just use --dither --input-depth 16 and that's fine.

[jlpsvk]

hi all, what would be better in terms of speed of encoding (at same settings) - i7-3930K (6C/12T with 4x4GB RAM Quad-Channel) or i7-4790K (4C/8T with 2x8GB RAM Dual-Channel)? Thanks for suggestion. I have both, but want to keep one for work, and one for encoding with x265.

[sneaker_ger]

Zond (free demo available here (http://www.dektec.com/products/applications/Zond/)) reports errors when using HRD:
Frame num Offset Source Description
0 0x00000459 Conformance to HEVC specification SEI: When an SEI NAL unit containing an active parameter sets SEI message is present in an access unit, it shall be the first SEI NAL unit that follows the prevVclNalUnitInAu of the SEI NAL unit and precedes the nextVclNalUnitInAu of the SEI NAL unit (Section D.3.1)
0 0x00000462 Conformance to HEVC specification SEI: When a non-nested buffering period SEI message is present in an access unit, it shall not follow any other SEI message that follows the prevVclNalUnitInAu of the buffering period SEI message and precedes the nextVclNalUnitInAu of the buffering period SEI message, other than an active parameter sets SEI message (Section D.3.1)
0 0x00000472 Conformance to HEVC specification SEI: When a non-nested picture timing SEI message is present in an access unit, it shall not follow any other SEI message that follows the prevVclNalUnitInAu of the picture timing SEI message and precedes the nextVclNalUnitInAu of the picture timing SEI message, other than an active parameter sets SEI message or a non-nested buffering period SEI message (Section D.3.1)
Example command:
x265 "input" -o "output" --level 5.1 --high-tier --vbv-maxrate 100000 --vbv-bufsize 100000 --hrd

[sneaker_ger]

Specific sample can trigger more errors:
Summary
Errors count 12
Different errors count 5
Frames count with errors 3
Frame num Offset Source Description
0 0x00000466 Conformance to HEVC specification SEI: When an SEI NAL unit containing an active parameter sets SEI message is present in an access unit, it shall be the first SEI NAL unit that follows the prevVclNalUnitInAu of the SEI NAL unit and precedes the nextVclNalUnitInAu of the SEI NAL unit (Section D.3.1)
0 0x0000046f Conformance to HEVC specification SEI: When a non-nested buffering period SEI message is present in an access unit, it shall not follow any other SEI message that follows the prevVclNalUnitInAu of the buffering period SEI message and precedes the nextVclNalUnitInAu of the buffering period SEI message, other than an active parameter sets SEI message (Section D.3.1)
0 0x0000047e Conformance to HEVC specification SEI: When a non-nested picture timing SEI message is present in an access unit, it shall not follow any other SEI message that follows the prevVclNalUnitInAu of the picture timing SEI message and precedes the nextVclNalUnitInAu of the picture timing SEI message, other than an active parameter sets SEI message or a non-nested buffering period SEI message (Section D.3.1)
2 0x00000bd5 Conformance to HEVC specification SEI: When an SEI NAL unit containing an active parameter sets SEI message is present in an access unit, it shall be the first SEI NAL unit that follows the prevVclNalUnitInAu of the SEI NAL unit and precedes the nextVclNalUnitInAu of the SEI NAL unit (Section D.3.1)
2 0x00000bde Conformance to HEVC specification SEI: When a non-nested buffering period SEI message is present in an access unit, it shall not follow any other SEI message that follows the prevVclNalUnitInAu of the buffering period SEI message and precedes the nextVclNalUnitInAu of the buffering period SEI message, other than an active parameter sets SEI message (Section D.3.1)
2 0x00000bed Conformance to HEVC specification SEI: When a non-nested picture timing SEI message is present in an access unit, it shall not follow any other SEI message that follows the prevVclNalUnitInAu of the picture timing SEI message and precedes the nextVclNalUnitInAu of the picture timing SEI message, other than an active parameter sets SEI message or a non-nested buffering period SEI message (Section D.3.1)
2 0x00000bd5 Conformance to HEVC specification SEI: An SEI NAL unit containing an active parameter sets SEI message shall contain only one active parameter sets SEI message (Section D.3.1)
25 0x0000ea1c Conformance to HEVC specification SEI: When an SEI NAL unit containing an active parameter sets SEI message is present in an access unit, it shall be the first SEI NAL unit that follows the prevVclNalUnitInAu of the SEI NAL unit and precedes the nextVclNalUnitInAu of the SEI NAL unit (Section D.3.1)
25 0x0000ea25 Conformance to HEVC specification SEI: The value of nal_initial_cpb_removal_delay[0] shall be less than or equal to 90000 * ( CpbSize[0] / BitRate[ 0 ] ), the time-equivalent of the CPB size in 90 kHz clock units. But it is equal to 90001, CpbSize[ 0 ] is equal to 100000000, BitRate [ 0 ] is equal to 100000000 (section D.3.2)
25 0x0000ea25 Conformance to HEVC specification SEI: When a non-nested buffering period SEI message is present in an access unit, it shall not follow any other SEI message that follows the prevVclNalUnitInAu of the buffering period SEI message and precedes the nextVclNalUnitInAu of the buffering period SEI message, other than an active parameter sets SEI message (Section D.3.1)
25 0x0000ea34 Conformance to HEVC specification SEI: When a non-nested picture timing SEI message is present in an access unit, it shall not follow any other SEI message that follows the prevVclNalUnitInAu of the picture timing SEI message and precedes the nextVclNalUnitInAu of the picture timing SEI message, other than an active parameter sets SEI message or a non-nested buffering period SEI message (Section D.3.1)
25 0x0000ea1c Conformance to HEVC specification SEI: An SEI NAL unit containing an active parameter sets SEI message shall contain only one active parameter sets SEI message (Section D.3.1)
x265_main10.exe source.y4m --uhd-bd --level 5.1 --high-tier --vbv-maxrate 100000 --vbv-bufsize 100000 --colorprim 1 --colormatrix 1 --transfer 1 -o uhd-bd_test.265
Package with input and output file plus log: download (http://217.160.126.132/x265_uhdbd_hrd_errors.7z)

[Boulder]

I don't think there is any significant difference. For me, I'd just use --dither --input-depth 16 and that's fine.Some time ago, I tried using --dither when the output was 10bit. The encoder didn't complain but the output was garbage.

Then I checked the docs and it says:
Only applicable when the input bit depth is larger than 8bits and internal bit depth is 8bits.

Maybe there should be a sanity check?

[Motenai Yoda]

Some time ago, I tried using --dither when the output was 10bit. The encoder didn't complain but the output was garbage.

Then I checked the docs and it says:
Only applicable when the input bit depth is larger than 8bits and internal bit depth is 8bits.

Maybe there should be a sanity check?

from what I've seen for anything but 8 bit the dither isn't applied, but it discard some bit, ie from 16 to 10 looks like it discard first 6 (or 8) bits

[LigH]

@ J1Man:

10 bit precision of internal parameters in the HEVC stream, after a transformation to the frequency domain, are not related to 10 bit precision of raw RGB or YUV video "pixels"/components.

[MeteorRain]

I did some visual comparisons but I could not decide which method is the best, quality wise.
Even if there is any difference, it will be so small that your monitor cannot even show.

I won't be worry about that at all.

[pingfr]

I'm in the process of building a x265 dedicated encoding/testbox but still hesitating between two architectures, at the moment I'm leaning towards the quad socket but could use more input again from you guys, so here goes:

- On one side we have a dual socketed system, we'll call it "duo" for the sake of the argument. It is built on a high-end consumer/enthusiast motherboard ASUS Z10PE-D16 WS which benefits from latest generation DDR4, the processors are two latest generation Xeon E5-2699v4 CPUs (retail, not ES) which Intel is about to release to the public in a few days; for the sake of comparison, they are 22 cores, 44 threads each, they run at 2.3GHz stock speed with a Turbo speed of 3.6GHz and of course they feature the AVX2 instruction set.

So in a dual socket system, properly configured, properly cooled, etc, we are looking at a total of 44 cores for 88 threads.

- On the opposite side we have a quad socketed system, we'll call it "quad" for the sake of the argument. It uses a true professional server-grade board Supermicro X10QBL which unfortunately is limited to DDR3, the processors are four Xeon E7-4890v2 CPUs (they are not retail, they are Engineering Samples) they were introduced Q1 2014 to the public; for the sake of comparison, each CPUs are sporting 15 cores, 30 threads, they run at 2.8GHz stock speed (higher than the CPUs from the first machine) with a Turbo Speed of 3.4GHz (lower than the CPUs from the first machine), as they are older CPUs they lack latest instruction sets such as AVX2 but still feature the AVX one.

So in the present configuration in a quad socket system, we are looking at a grand total of 60 cores for no less than 120 threads.

The other last and minor drawbacks aside the lack of AVX2 instruction set is the "limited" 32 PCI lanes coupled with a restricting operating system requirement forcing the switch to either Linux or Windows Server 2012 (or 2016 beta) as my good ole' Windows 7 can't see quad socketed configurations.

So there you have it:

- A newer dual socketed system with all the nice and latest features from the fourth upcoming generation of Xeon E5's for a total of 44 cores/88 threads.

Versus:

- A slightly older quad socketed system from two generations ago which trades off the AVX2/40~32 PCI lanes for much more cores/threads with a total of 60 cores/120 threads.

With all that being said, my only interest being faster x265 encoding, which route should I take?

Go for 4-way E7/AVX 60c/120t from 2 generations ago which still "packs a punch" or rather aim for modern 2-way E5/AVX2 44c/88t from this upcoming generation which seems just "meh, whatever" to me?

Thanks.

Edit: Added two relevant links but they should be taken with a pinch of salt as it's E5 v3, I have access to E5 v4 processors in my case.

http://www.spec.org/cpu2006/results/res2015q1/cpu2006-20150209-34979.html

http://www.spec.org/cpu2006/results/res2014q1/cpu2006-20140210-28464.html

[MeteorRain]

pingfr: I think you should first set up the goal. Are you looking for an ultimate box regardless of the price? Or are you looking to have more performance per certain amount of money?

The fastest way to do the encoding is to have a cluster of servers, and dispatch tasks to be executed in parallel.

[x265_Project]

I'm in the process of building a x265 dedicated encoding/testbox but still hesitating between two architectures, at the moment I'm leaning towards the quad socket but could use more input again from you guys, so here goes:

- On one side we have a dual socketed system, we'll call it "duo" for the sake of the argument. It is built on a high-end consumer/enthusiast motherboard ASUS Z10PE-D16 WS which benefits from latest generation DDR4, the processors are two latest generation Xeon E5-2699v4 CPUs (retail, not ES) which Intel is about to release to the public in a few days; for the sake of comparison, they are 22 cores, 44 threads each, they run at 2.3GHz stock speed with a Turbo speed of 3.6GHz and of course they feature the AVX2 instruction set.

So in a dual socket system, properly configured, properly cooled, etc, we are looking at a total of 44 cores for 88 threads.

- On the opposite side we have a quad socketed system, we'll call it "quad" for the sake of the argument. It uses a true professional server-grade board Supermicro X10QBL which unfortunately is limited to DDR3, the processors are four Xeon E7-4890v2 CPUs (they are not retail, they are Engineering Samples) they were introduced Q1 2014 to the public; for the sake of comparison, each CPUs are sporting 15 cores, 30 threads, they run at 2.8GHz stock speed (higher than the CPUs from the first machine) with a Turbo Speed of 3.4GHz (lower than the CPUs from the first machine), as they are older CPUs they lack latest instruction sets such as AVX2 but still feature the AVX one.

So in the present configuration in a quad socket system, we are looking at a grand total of 60 cores for no less than 120 threads.

The other last and minor drawbacks aside the lack of AVX2 instruction set is the "limited" 32 PCI lanes coupled with a restricting operating system requirement forcing the switch to either Linux or Windows Server 2012 (or 2016 beta) as my good ole' Windows 7 can't see quad socketed configurations.

So there you have it:

- A newer dual socketed system with all the nice and latest features from the fourth upcoming generation of Xeon E5's for a total of 44 cores/88 threads.

Versus:

- A slightly older quad socketed system from two generations ago which trades off the AVX2/40~32 PCI lanes for much more cores/threads with a total of 60 cores/120 threads.

With all that being said, my only interest being faster x265 encoding, which route should I take?

Go for 4-way E7/AVX 60c/120t from 2 generations ago which still "packs a punch" or rather aim for modern 2-way E5/AVX2 44c/88t from this upcoming generation which seems just "meh, whatever" to me?

Thanks.

Edit: Added two relevant links but they should be taken with a pinch of salt as it's E5 v3, I have access to E5 v4 processors in my case.

http://www.spec.org/cpu2006/results/res2015q1/cpu2006-20150209-34979.html

http://www.spec.org/cpu2006/results/res2014q1/cpu2006-20140210-28464.html
60 Ivy Bridge Xeon cores running at 2.8 GHz has more x265 compute power than 44 Broadwell Xeon cores running at 2.3 GHz. Forget about turbo clock speeds... if you're running x265 you will hit the thermal limits of your chips, and there will be no turbo boost.

Again, one x265 instance can't keep that many threads working efficiently, even with all of the parallelism that we've implemented. You will need to run multiple instances in parallel to fully exploit either of these many-core servers. This could either be multiple videos, or multiple chunks of the same video.

[zioneed]

Hi all,
I am encoding a lot a of TV series, shrinking from BD.
1080p to 720p.
So far the quality is pretty good with below settings, via MEGui; what I'd need is some hints if these are the best settings for preserving as much quality as possible.
Clearly it's a matter of compromises.

Const. Quality, preset medium.
additional commands :program --crf 17.0 --limit-refs=3 --no-sao --no-deblock --qcomp=0.9 --early-skip --qg-size=32 --psy-rd=2 --rdpenalty=2 --limit-modes --vbv-maxrate=5000 --vbv-bufsize=5000 --rdoq-level=2 --tu-intra-depth=3 --output "output" "input"

These settings are giving good results in terms of speed VS quality
but have a couple of doubts
1) What is the relationship between qcomp and bitrate?
2) QG Size is set correctly? Spedifically the original BD videos are a bit blocky (Heroes tv series) and my undesranding is that deblock option activated (together with AQ mode) would make the results a bit blurry.

So please can anyone help and give some hints about what could be set differently or if there's any problem with my preset?

Many thanks in advance!!!

[Motenai Yoda]

Const. Quality, preset medium.
additional commands :program --crf 17.0 --limit-refs=3 --no-sao --no-deblock --qcomp=0.9 --early-skip --qg-size=32 --psy-rd=2 --rdpenalty=2 --limit-modes --vbv-maxrate=5000 --vbv-bufsize=5000 --rdoq-level=2 --tu-intra-depth=3 --output "output" "input"

1) What is the relationship between qcomp and bitrate?
2) QG Size is set correctly? Spedifically the original BD videos are a bit blocky (Heroes tv series) and my undesranding is that deblock option activated (together with AQ mode) would make the results a bit blurry.

I would set deblock to at least -2:-1 or -1:-1, disabling it isn't a good idea, and qg-size to 16, also why vbv to 5Mbps? maybe increase ref to 6 and rdpenality to 1 if you need it

1- qcomp "adjust" how variable the bitrate would be, 0 = costant bitrate, 1 = costant quantizer, usually the .6 default value is good, if you have a mid/high complex video with few low complex scenes setting it to .5 will help to bump those low complex scenes quality, otherwise, with a mid/low complex and few high complex scenes raise it up to .7 or .8
2- maybe you should deblock first, like with deblock_qed

[zioneed]

I would set deblock to at least -2:-1 or -1:-1, disabling it isn't a good idea, and qg-size to 16, also why vbv to 5Mbps? maybe increase ref to 6 and rdpenality to 1 if you need it

1- qcomp "adjust" how variable the bitrate would be, 0 = costant bitrate, 1 = costant quantizer, usually the .6 default value is good, if you have a mid/high complex video with few low complex scenes setting it to .5 will help to bump those low complex scenes quality, otherwise, with a mid/low complex and few high complex scenes raise it up to .7 or .8
2- maybe you should deblock first, like with deblock_qed

Many thanks for the reply!!!
Sorry but questions that might seem dumb, but HEVC is still a bit of a bet for me, and trying to deal with some tuning. The goal is keep a "good" image quality and avoid blocky images. Keeping sharpness as much as possible at the same time.
Been reading a lot lately on specific topics, but I'm still confused about some options :)
Back to your kind reply:
1) will try a sample with deblock -1-1
2) qcomp: not clear how it works, sorry. Assumed that usually scenes are quite complex and full of action which value works best? (in theory, I know there's no definitive answer on that). The higher the better for action scenes? Default is 0.6 if I'm not mistaken, hence raising to 0.8/9 would be any good?
3) will modify qgsize as per your suggestion.
4) "also why vbv to 5Mbps? maybe increase ref to 6 and rdpenality to 1 if you need it": can you please explain this? I have tried different settings for bitrate and between 4 and 5 k is giving best result with my settings. by "ref" you mean "limit-refs"?
Thanks a lot again, the technical discussions and topics are always exhaustive and clear, but it's hard to get a hold on interactions between different setting :)

Btw, what do you mean by "deblock_qed"?

[LigH]

Btw, what do you mean by "deblock_qed"?

http://avisynth.nl/index.php/Deblock_QED (discussion (http://forum.doom9.org/showthread.php?t=154777))

[Motenai Yoda]

2- qcomp when you raise or decrease bitrate (or crf) it will be "spreaded" to all the video, if you have only few high complex scenes most of the bits go to the others "yet-good-quality" scenes, higher qcomp allow the codec to assign more bit on the hot ones, viceversa with few low complex scenes the codec will assign too few bits and you have to lower it to "spread" better the bits.

4- vbv didn't controll bitrate, it restrict the operating space of the codec, 5Mbps is a low value for vbv, usless you are targeting to a streaming purpose you can raise them to 10Mbps or even more (level 3.1 main tier, the lowest for 720p, require 10Mbps, 4.0 (1080p) tier main 12Mbps and high 30Mbps)
If you are aiming to a specific bitrate mode you should go with 2 pass abr ratecontrol not the crf one.

ref are for reference frames, default 3

deblock_qed is an avisynth function wrote by Didée, and further modified by others, which aim to adaptively reduce blockness artifacts
description and dependecies http://avisynth.nl/index.php/Deblock_QED
lastest version http://forum.doom9.org/showpost.php?p=1697386&postcount=13

[jlpsvk]

why is x265 about 50% slower with --pmode than without it? on 6 core i7-3930K?

[Motenai Yoda]

@jlpsvk read the docs http://x265.readthedocs.org/en/default/cli.html#cmdoption--pmode

[pingfr]

pingfr: I think you should first set up the goal. Are you looking for an ultimate box regardless of the price? Or are you looking to have more performance per certain amount of money?

Alrighty, here it goes, here's a bit of lecture for you:

I actually never took the proper time to introduce myself but, I work as a support staff/library clerk for a major city in France (read: we're in charge of roughly 1 million citizen).

In my department, we are handling responsabilities ranging from managing a book loans system to managing DVDs and Blu-Rays loans and rentals to our citizens.

As the time passed over the years, we realized that with our 7 days turn-over system, we, the library staffers came to the conclusion we need at least 3 copies on every single movie we ever have owned/on our catalog at any given time.

Now if you're a video enthusiast like me, you are aware that owning every Blu-Ray movie ever released, is nearly impossible, we're talking about roughly 5000 physical discs, multiplied by 3x, based on our 7 days rotation "loan and must return within 7 days" system, that's a total of no less than 15000 discs.

Now with that set aside, both our local and nation-wide broadband penetration statistics have shown that over the years, it is now possible to have fiber at home with unlimited download caps for say, less than 25$/month.

So we came to the conclusion, as we were running out of physical space (read: proper office space) we should probably "digitize" all our DVD and Blu-Ray contents to simple files and maybe broadcast them over the internet instead of loaning the physical the discs back and forth between users.

Now, as you've probably guessed by my posts history, I'm not really a fully highly tech savvy person and I don't know all the details regarding the streaming bit/part (not our department), however as the source discs are in my department, I have been tasked to "digitize" all our content and render them into nearly "acceptable" quality digital content.

I at first went the x264 route but I've been more or less been told by our other IT departments, based on their calculations, x264 encoded files with an average bitrate of 4k for 720p contents and 8k for 1080p would have a huge impact on the network's performance and bandwidth use (huge costs for us).

It was stated at that this point, the trade ratio for bandwidth expenses (not even counting the hardware costs and the encoding expenses/time spent:electricity bill) wasn't worth the effort.

But at the same time, I've been told that if I could find a way to retain the same type of subjective visual quality with a lower bitrate, read; lower storage and bandwidth costs altogether, that it would be "green lighted" as a project.

Keeping in mind, that this is taxpayer money we're talking about here, as such: every single dollar/euro spent has to be properly justified and allocated.

In the very end, this is naturally how x265 came into play; I am constantly looking for ways to reduce our encoding file sizes while retaining "upper" quality and saving as much as we can on the bandwidth costs/fees.

Now that was for the professional side of things.

Now for the personal bit, I'm also a very geeky person and always am interested in newer technologies... I remember when the very first MP3 format came out shortly followed by the original DivX 3.11, back in the days, everyone around me was going bat-shit crazy! but again I disgress. ;)

So now on the personal side of things, I'm also interested in x265 encoding, but that's just a "side-project" or a "hobby" as I can put it.

Over the past months, I've been saving personal funds here and there, shaving rough edges on my personal budget and have saved roughly $3k USD, my intent is to be on the lookout for the cheapest, refurbished, second-hand hardware, Engineering Sample deals on eBay and such sites and pretty much build the "best and fastest" encoding box for me, my own personal toybox at home based on a restricted $3k budget.

Of course the machine wouldn't just be a pure dedicated x265 box, it has to be my own personal computer at home, but you get the idea, build something that is strong enough to encode things "mildly rapidly" at the cheapest cost ratio that would fit in my budget which is again roughly 3k $USD.

So there you have it; grabbing CPUs from eBays, motherboard and other parts from shady retailers, DDR3 ECC memory sticks from retired or "defunct" servers from work, cook it all together and hope for the best.

Sorry if that was lengthy, but yeah, at least you guys know me a bit better now, understand what is my approach and are aware what kind of machine I'm building here.

As always, any feedback (even negative as long as it brings something to the table) is appreciated. :)

The fastest way to do the encoding is to have a cluster of servers, and dispatch tasks to be executed in parallel.

As it stands, best machines I can get remote access to are 4x Dell R930 from work from the IT department, but I've been warned they can be accessed at odd times (mostly night time), whenever they aren't in heavy use by the departments they belong to and they have extra spare ressources. I don't even know if they are part of a cluster or they're standalone machines, but requesting a "privilege" access to them shouldn't be much of an issue.

As of today, so far, all I have access to is a i7 5960X at work to do remote encoding and an i7 6700k at home, my own personnal machine... and let's just say this; for everyday usage it's fine, but when it comes to x265 (with the quality parameters littlepox kindly cooked us), it just doesn't cut it, at all, so at least on the personal side of things, I'm just throwing more hardware ressources at it (read: money), hoping it will "fix it", but then again at the very end of the day, keep in mind it's just a (costly) hobby. :)

[pingfr]

60 Ivy Bridge Xeon cores running at 2.8 GHz has more x265 compute power than 44 Broadwell Xeon cores running at 2.3 GHz.

Hey Tom thanks for your input!

So if I get you right, regardless 60 Ivy Bridge Xeon cores at 2.8GHz *without* AVX2, BMI, LZCNT instruction sets still provide more x265 compute power than 44 Broadwell Xeon cores at 2.3GHz even if they do have AVX2, BMI, LZCNT to compensate for the lower core/thread count?

Forget about turbo clock speeds... if you're running x265 you will hit the thermal limits of your chips, and there will be no turbo boost.

This one machine I'm building is a personnal system for my own use at home, do you think agressive watercooling could help with lowering the chips temp and therefore retain turbo boost active or is that a "helpless cause" to you?

You will need to run multiple instances in parallel to fully exploit either of these many-core servers. This could either be multiple videos, or multiple chunks of the same video.

Regarding a multiple chunks encoding, is there a more or less "official" and practical solution/kit from MulticoreWare or do we still have to rely on programs such as Ripbot264 etc?

Thanks for your feedback.

[nevcairiel]

This one machine I'm building is a personnal system for my own use at home, do you think agressive watercooling could help with lowering the chips temp and therefore retain turbo boost active or is that a "helpless cause" to you?

You don't even need water cooling for this, a good air cooler can keep CPUs in full turbo at all times as well - of course such huge air coolers don't find their way into server chassis, since they are too big.

[pingfr]

You don't even need water cooling for this, a good air cooler can keep CPUs in full turbo at all times as well - of course such huge air coolers don't find their way into server chassis, since they are too big.

It looks like I'm gonna have to "hack" a case chassis anyways, this one is for my own use at home sitting under my desk, if I'm effectively going for a quad E7 v2, the only reliable motherboard I found is from Supermicro, X10QBL and it would seem the board itself is using a proprietary format (16.79" x 16.4" (42.6cm x 41.7cm)).

http://www.supermicro.com/products/motherboard/Xeon/C600/X10QBL.cfm

Not sure what kind of case will fit this baby, but it doesn't look seem most standard ATX cases would do the trick.

So yeah, chassis "hacking", case air cooler "hacking" and water cooling "hacking".

Sounds like my kind of fun. :)

[nevcairiel]

For a quad setup water cooling is probably best indeed, as 4 decent air coolers would have trouble fitting on there!

[pingfr]

For a quad setup water cooling is probably best indeed, as 4 decent air coolers would have trouble fitting on there!

Thanks nevcairiel. :D

[zioneed]

2- qcomp when you raise or decrease bitrate (or crf) it will be "spreaded" to all the video, if you have only few high complex scenes most of the bits go to the others "yet-good-quality" scenes, higher qcomp allow the codec to assign more bit on the hot ones, viceversa with few low complex scenes the codec will assign too few bits and you have to lower it to "spread" better the bits.

4- vbv didn't controll bitrate, it restrict the operating space of the codec, 5Mbps is a low value for vbv, usless you are targeting to a streaming purpose you can raise them to 10Mbps or even more (level 3.1 main tier, the lowest for 720p, require 10Mbps, 4.0 (1080p) tier main 12Mbps and high 30Mbps)
If you are aiming to a specific bitrate mode you should go with 2 pass abr ratecontrol not the crf one.

ref are for reference frames, default 3

deblock_qed is an avisynth function wrote by Didée, and further modified by others, which aim to adaptively reduce blockness artifacts
description and dependecies http://avisynth.nl/index.php/Deblock_QED
lastest version http://forum.doom9.org/showpost.php?p=1697386&postcount=13

Thanks, was just checking avisynth but have no clue how to use it, will pass :)
More than bitrate I'm interested in spreading it properly across different scenes, tryng to find out best settings to encode properly bot action scenes and slow scenes. Size really doesn't matter much since I'm more into finding a good compromise between quality and encoding speed.
If I might ask one last question (since have read quite a lot and still have no final answer on this):
what's the role of this part?
I have set up " --psy-rd=1 --psy-rdoq 1.10 --rdpenalty=2 --rdoq-level 2". This according to different posts read here. The results are quite good in terms of quality but wondering if there's anything in these options which might fight.
Thanks again!

[jlpsvk]

this is x264:
http://s12.postimg.org/4r368zrl9/SW_x264.png

and this x265:
http://s8.postimg.org/sldo3qj51/SW_x265.png

No matter what I do, the transition on the banner (from red to black - highlighted) is smoother on x264. On x265 you the the squares after the close-up. What do you advice?

My settings:
--crf 22 --preset slow --output-depth 10 --tu-intra-depth 2 --rdoq-level 1 --qg-size 16 --ipratio 1.3 --pbratio 1.2 --max-merge 2 --rc-lookahead 60 --ref 3 --min-keyint 23 --keyint 240 --colorprim bt709 --colormatrix bt709 --transfer bt709 --deblock -3:-3 --psy-rdoq 5

[MeteorRain]

Keeping in mind, that this is taxpayer money we're talking about here, as such: every single dollar/euro spent has to be properly justified and allocated.

So you have 2 options for the maximum performance per money,
1) i7-4790 + B85 mobo + 8GB RAM + PSU
2) 2xE5-2670 (Used) + C602 mobo + Some ECC RAM (Used) + Good PSU

First choice is about $450 without HDD, monitor and computer case.
Second is about $600 without these craps.
So if you have, let's say, 5 bluray to encode, distribute them to 5 different servers is going to maximize the speed. Given $3000, running 5 i7s will be faster than a single E7, let alone E7 being much more expensive.

The whole point is that, you'll be paying a lot more for the high density (the amount of space on the rack).

[pingfr]

So you have 2 options for the maximum performance per money,
1) i7-4790 + B85 mobo + 8GB RAM + PSU
2) 2xE5-2670 (Used) + C602 mobo + Some ECC RAM (Used) + Good PSU

First choice is about $450 without HDD, monitor and computer case.
Second is about $600 without these craps.
So if you have, let's say, 5 bluray to encode, distribute them to 5 different servers is going to maximize the speed. Given $3000, running 5 i7s will be faster than a single E7, let alone E7 being much more expensive.

The whole point is that, you'll be paying a lot more for the high density (the amount of space on the rack).

I think you misread it real fast.

Regarding my interest in regards to x265 in general, there are two things;

Professionally, like, at my work, as my job, it's impossible for me to bring-in "fleaky hardware" from dubitous sources without proper invoices, warranties or without a corporate contract directly from HP, Dell or any other major retailers because it's actually a budget that has to be allocated and voted in city hall's court. Which implies, what hardware I will eventually get my hands on depends on things that are out of my control.

Now what I have complete control over is what I do with my personnal funds, at home, as a leisure/hobby and what I have at the moment is a 3k budget to use as I see it at my own will and I have all reasons to believe, my best option at home is a Quad Xeon E7 v2 since I'm acquiring parts for reaaaaaaallly cheap compared to the official fares.

But then again if someone has an idea of "what performs better than what for cheaper than whatever else grabbed from over here rather than over there", please let me know. I'm all ears. ;)

[kypec]

Professionally, like, at my work, as my job, it's impossible for me to bring-in "fleaky hardware" from dubitous sources without proper invoices, warranties or without a corporate contract directly from HP, Dell or any other major retailers because it's actually a budget that has to be allocated and voted in city hall's court. Which implies, what hardware I will eventually get my hands on depends on things that are out of my control.

Maybe I'll go a bit off topic here but did you also consider HEVC licensing fees (http://forum.doom9.org/showthread.php?t=172387) apart from technological (better compression ratio) aspects of choosing HEVC over AVC? I don't want to spoil your fun and really hope that your project of digital movie library ends up successfully but I think you should engage lawyers as well into the process before spending too much time & electricity on encoding the content... ;)

[MeteorRain]

I think you misread it real fast.

Regarding my interest in regards to x265 in general, there are two things;

Professionally, like, at my work, as my job, it's impossible for me to bring-in "fleaky hardware" from dubitous sources without proper invoices, warranties or without a corporate contract directly from HP, Dell or any other major retailers because it's actually a budget that has to be allocated and voted in city hall's court. Which implies, what hardware I will eventually get my hands on depends on things that are out of my control.

Now what I have complete control over is what I do with my personnal funds, at home, as a leisure/hobby and what I have at the moment is a 3k budget to use as I see it at my own will and I have all reasons to believe, my best option at home is a Quad Xeon E7 v2 since I'm acquiring parts for reaaaaaaallly cheap compared to the official fares.

But then again if someone has an idea of "what performs better than what for cheaper than whatever else grabbed from over here rather than over there", please let me know. I'm all ears. ;)

Sorry for misreading your post.
For the professional part, unless you are running out of physical space, I'd not recommend using rack servers as they tend to be more expensive and less cost-effective. Buying multiple dell or hp i7s business level computer could be a good use of money, compared to those E5s (2x+ expensive with almost same performance).

For the hobby part, if you can get dirty cheap E7 then go ahead. The E5 that I mentioned was also a quite cheap option for you. Just take care of the noise control and power consumption.
For performance wise, google search "passmark <whatever cpu here>" and you can compare the score easily.

[pingfr]

Sorry for misreading your post.

No problems, no harm, I know you were trying to be helpful. :)

For the hobby part, if you can get dirty cheap E7 then go ahead.

400$ per CPU, not sure if that's "dirt cheap" but we're talking about E7 Xeons here at the fastest/highest frequency ones from that generation.

For performance wise, google search "passmark <whatever cpu here>" and you can compare the score easily.

Passmark tends to be "unreliable" specially if you're looking at multi sockets benchs, on the contrary I believe SPEC benchs are more reliable.

Regardless, thanks a lot for your input. :)

[MeteorRain]

400$ per CPU, not sure if that's "dirt cheap" but we're talking about E7 Xeons here at the fastest/highest frequency ones from that generation.

That's cheap enough. Usually you can only get an i7 with that money.

Regardless, thanks a lot for your input. :)

You're welcome. Hope it helps ;)

[pingfr]

That's cheap enough. Usually you can only get an i7 with that money.

It's really a bargain, otherwise I wouldn't even bother to begin with.

It seems the best CPU power route from the worst to the best is:

E5-2699 v3 x2 -> E7-4890 v2 x4 = E7-4850 v3 x4 -> E5-4669 v3 x4 -> E7-8895 v3 x8 -> Nothing yet? maybe some high-end E5 or E7 v4 around Q3-Q4 2016 -> E5 or E7 v5 with AVX512 H2-2017.

You're welcome. Hope it helps ;)

Anything helps. :)

[MeteorRain]

It seems the best CPU power route from the worst to the best is:

I actually don't have much idea on how much performance can you get with one of these. I'd personally go with cluster solution where it fits me better.

[foxyshadis]

Another datapoint you might be interested in: VMWare, both ESXi and Workstation, gives you native encoding speed. Benchmarks of KVM and Xen show the same. Virtualbox you take a hit, unfortunately, but other virtual solutions allow 99%+ native speed in x264 and x265. (One of the benefits of heavy number crunching is that you don't need to call into the kernel or do I/O much.) You could use that to set up a test cluster for working out how a real cluster would perform at work, and get experience in setting it up.

I know Multicoreware has demonstrated a cluster, but I don't know how polished vs spit and duct tape it is, since they don't seem to offer it for sale. Aside from that, the only one I know of is lancoder (https://github.com/jdupl/lancoder), though there's x264farm (http://forum.doom9.org/showthread.php?t=117889) & ELDER (https://forum.doom9.org/showthread.php?t=102119) that might be modified to call x265 instead of x264, and Media Encoding Cluster (http://www.codergrid.de) that uses ffmpeg. None of those have been touched since 2009-2010 though, still a bit of a gap for someone who wants to come in and fill it.

Maybe you can ask Netflix if they're willing to share part of their solution? You never know.

[2themax]

Experimental UHD Bluray support (--uhd-bd) in x265 1.9+106-c8ec86965e54 (https://www.mediafire.com/download/b11kgzti9w4gglu/x265_1.9+106-c8ec86965e54.7z); is there anyone who can test and confirm?
This is very close to working. The only error I am getting is that GOPs are longer than they are specified to be.

[sneaker_ger]

x265 command-line? x265 log? Exact error message of your authoring software? (What software?)

[Vesdaris]

Forget about turbo clock speeds... if you're running x265 you will hit the thermal limits of your chips, and there will be no turbo boost.


Why would he hit thermal limits having a proper cooling?

I can run my Ivy bridge 3770k @4.5 on air using a pretty high for these CPUs voltage (1.33V under 100% load -> unfortunately I haven't won the silicon lottery) and I still don't hit thermal limits. Ever.

[foxyshadis]

Why would he hit thermal limits having a proper cooling?

I can run my Ivy bridge 3770k @4.5 on air using a pretty high for these CPUs voltage (1.33V under 100% load -> unfortunately I haven't won the silicon lottery) and I still don't hit thermal limits. Ever.

I think the initial assumption was that it would be a pizza box (rack mount), which are much, much harder to cool that a standalone server, because there just isn't enough room. You don't see too many quad sockets that aren't racked these days. Only a heavy-duty watercooling can keep those running at full-tilt, and even that's not easy in the confines of the rack. Since it'll be standalone, though, it'll be loud but easily capable of being cooled.

[x265_Project]

Why would he hit thermal limits having a proper cooling?

I can run my Ivy bridge 3770k @4.5 on air using a pretty high for these CPUs voltage (1.33V under 100% load -> unfortunately I haven't won the silicon lottery) and I still don't hit thermal limits. Ever.
I'm guessing that you hit thermal throttling all the time, but you aren't aware of it. Thermal management is all done automatically. You would have to use an advanced performance profiling tool like Intel's Performance Counter Monitor to log the clock speed in small increments.

Your Core i7-3770K has a TDP of 77 watts, or roughly 19 watts per core. Many-core Xeons have higher TDPs, but much lower TDP per core (hence, the slower clock speeds). Fully saturating them with compute-intensive applications will heat them up good, and you are very unlikely to have enough thermal headroom to allow turbo mode to be engaged.

[Vesdaris]

I'm guessing that you hit thermal throttling all the time, but you aren't aware of it.



You are guessing wrong.:) I'm always running a monitoring tool(hwinfo). I always stay below my CPU's thermal limits and I have turbo clock at it's max (clocks & performance).

[pingfr]

No fighting here boys! Keep it cool. (You see what I did there?). :)

[MeteorRain]

You are guessing wrong.:) I'm always running a monitoring tool(hwinfo). I always stay below my CPU's thermal limits and I have turbo clock at it's max (clocks & performance).

I guess the TDP is the maximum generated thermal, not the maximum evacuated thermal. So in most case if you max out your CPU, it will (regardless of cooler) limit itself to a certain point.
I know that some mobo can crack it a bit to have turbo boost running at all time. In this case the only requirement would be a decent sink and cooler.

[benwaggoner]

Passmark tends to be "unreliable" specially if you're looking at multi sockets benchs, on the contrary I believe SPEC benchs are more reliable.

Regardless, thanks a lot for your input. :)
Also, x265 makes heavy use of AVX2 instructions. So perf-per-clock gets a lot better with a chip that uses that. I wouldn't suggest buying anything pre-AVX for use with x265.

[benwaggoner]

I guess the TDP is the maximum generated thermal, not the maximum evacuated thermal. So in most case if you max out your CPU, it will (regardless of cooler) limit itself to a certain point.
I know that some mobo can crack it a bit to have turbo boost running at all time. In this case the only requirement would be a decent sink and cooler.
And we also know that lots of sustained multicore AVX2 operations (heavily used to good effect by x265) can also trigger thermal throttling, apparently irrespective to the level of cooling in action. I know MCW has been looking at this.

Even with that, we see big throughput increases (>50%) on c4.8xlarge instances versus c3.8xlarge instances for x265 encoding with recent builds.

C3.8xlarge: E5-2680v2 (Ivy Bridge), logical CPUs=32
C4.8xlarge: E5-2666v3 (Haswell), logical CPUs=36

While there are a few more cores, it's mainly going from v2 to v3.

[x265_Project]

You are guessing wrong.:) I'm always running a monitoring tool(hwinfo). I always stay below my CPU's thermal limits and I have turbo clock at it's max (clocks & performance).
I certainly could be wrong, but I'm not sure that HWInfo would be the definitive way to know if there is any thermal throttling going on. I know you can trust Intel's Performance Counter Monitor to tell you what's really going on. We've built support for PCM into x265, so if you can get a build you can run it alongside x265.

[2themax]

x265 command-line? x265 log? Exact error message of your authoring software? (What software?)

Here's the command-line:
x265-64bit-10bit-2016-03-31.exe --preset veryslow --output "outpt.hevc" --input "input.yuv" --input-res 3840x2160 --input-csp i420 --fps 24 --profile main10 --level-idc 51 --high-tier --open-gop --keyint 24 --min-keyint 1 --bitrate 75000 --vbv-maxrate 95000 --vbv-bufsize 100000 --pass 1 --colorprim bt709 --transfer bt709 --colormatrix bt709 --hrd --aud --sar 1:1 --output-depth 10 --uhd-bd --repeat-headers --no-scenecut --rc-lookahead 20
And here's the error being thrown by the Panasonic UHD verifier:
The number of frames displayed in a GOP exceeds the maximum number defined in Table 9-97.

Actual number of frames in a GOP= 27
frame_rate_value = 24
Maximum number of frames displayed in a GOP = 24

[LigH]

So despite using the option --keyint 24 the verifier found a longer GOP?

[MasterNobody]

May be it is confusion if GOP length should be in presentation order or decode order. For x264 (H.264) at least there was difference because default is presentation order and --bluray-compat case needed decode order (bluray order). This difference in GOP length exist only for open GOP.

[x265_Project]

And here's the error being thrown by the Panasonic UHD verifier:
The number of frames displayed in a GOP exceeds the maximum number defined in Table 9-97.

Actual number of frames in a GOP= 27
frame_rate_value = 24
Maximum number of frames displayed in a GOP = 24
This is great feedback. Please email me so that we can connect directly to debug this issue. I just sent you a private message with my email address. Thanks, Tom

[2themax]

So despite using the option --keyint 24 the verifier found a longer GOP?
Yes. I tried adding --no-scenecut and --rc-lookahead 20 thinking it might change the encode but it did not.

[x265_Project]

Yes. I tried adding --no-scenecut and --rc-lookahead 20 thinking it might change the encode but it did not.
Thanks for connecting via email. Our engineers will take a look at this today, and we'll follow up here and if needed (for sharing confidential files or information) privately.

I suspect that --open-gop may be an issue, but again, we'll see what our team thinks. We may need a frame-by-frame log file (--csv) to see the exact GOP structure and references you produced for this file.

[nandaku2]

Thanks for connecting via email. Our engineers will take a look at this today, and we'll follow up here and if needed (for sharing confidential files or information) privately.

I suspect that --open-gop may be an issue, but again, we'll see what our team thinks. We may need a frame-by-frame log file (--csv) to see the exact GOP structure and references you produced for this file.

Yes, open-gop is the issue. The reason the Panasonic UHD Verifier shows 27 frames is because 3 frames after the I frame refer to frames in the previous GOP. Can you test with --no-open-gop?

The UHD-BD documentation was not specific about enforcing closed GOPs.

[sneaker_ger]

OpenGOP is allowed according to the white paper on the BluRay website. Chapter 3.2 lists GOP restrictions.
http://www.blu-raydisc.com/assets/Downloadablefile/BD-ROM_Part3_V3.0_WhitePaper_150724.pdf

[zerowalker]

Anyone know how to set up Matrix, Transfer, Range etc on x265 in ffmpeg?

[2themax]

Yes, open-gop is the issue. The reason the Panasonic UHD Verifier shows 27 frames is because 3 frames after the I frame refer to frames in the previous GOP. Can you test with --no-open-gop?

The UHD-BD documentation was not specific about enforcing closed GOPs.
Setting it to --no-open-gop did indeed fix the problem for now. The encode passes without any error.

[a5180007]

The reason the Panasonic UHD Verifier shows 27 frames is because 3 frames after the I frame refer to frames in the previous GOP.

Did you mean before the I frame?

[foxyshadis]

OpenGOP is allowed according to the white paper on the BluRay website. Chapter 3.2 lists GOP restrictions.
http://www.blu-raydisc.com/assets/Downloadablefile/BD-ROM_Part3_V3.0_WhitePaper_150724.pdf

Open GOP is allowed, but that just means any frames that are "open" still come off the total frames in the GOP. Still can't add up to more than 1 second of frames. I'm sure x265 will fix the accounting shortly.

[Motenai Yoda]

It's right that in x265, with preset medium and rdoq-level=2, is psy-rdoq 0.00?
The docs says it should be 1.0 and it isn't changed by preset

[Ma]

It's right that in x265, with preset medium and rdoq-level=2, is psy-rdoq 0.00?
The docs says it should be 1.0 and it isn't changed by preset

Yes, there is the difference.

In docs there is 1.0:
--psy-rdoq (http://x265.readthedocs.org/en/default/cli.html?highlight=psy-rdoq#cmdoption--psy-rdoq)

In source code there is 0.0:
void x265_param_default(x265_param* param) @ param.cpp (https://bitbucket.org/multicoreware/x265/src/5b01678f6fb4e89e23cd41295592a9aa5d51d4ba/source/common/param.cpp?at=default&fileviewer=file-view-default#param.cpp-189)

1.0 is assigned from preset 'slow' to 'placebo'. I don't know if it is OK.

[x265_Project]

Yes, there is the difference.

In docs there is 1.0:
--psy-rdoq (http://x265.readthedocs.org/en/default/cli.html?highlight=psy-rdoq#cmdoption--psy-rdoq)

In source code there is 0.0:
void x265_param_default(x265_param* param) @ param.cpp (https://bitbucket.org/multicoreware/x265/src/5b01678f6fb4e89e23cd41295592a9aa5d51d4ba/source/common/param.cpp?at=default&fileviewer=file-view-default#param.cpp-189)

1.0 is assigned from preset 'slow' to 'placebo'. I don't know if it is OK.
It's been a while, but if I recall correctly Psy-RDOQ only works with RD level 4 and above. Our preset documentation (http://x265.readthedocs.org/en/default/presets.html) matches the code in param.cpp.

[burfadel]

It's been a while, but if I recall correctly Psy-RDOQ only works with RD level 4 and above. Our preset documentation (http://x265.readthedocs.org/en/default/presets.html) matches the code in param.cpp.

So therefore there is a difference betwee RD 3 and 4, with 4 allowing Psy-RDOQ?

I am currently using:

--rd 4
--tu-intra-depth 3
--rdoq-level 1
--early-skip
--fast-intra
--b-intra
--tskip
--tskip-fast
--limit-modes
--aq-mode 3
--qg-size 16
--me star
--merange 25
--max-merge 3
--weightb
--bframes 6
--rc-lookahead 40
--ref 6

I've found using --me star to be effectively faster than --me umh, but provide better quality. psy-rdoq 1 does increase the file size a little, but visual quality wise it does seem to produce a better image, much in line with what using --rd 5 does but considerably faster, with a similar file size output. Well worth it I think.

--merange 25 is faster than the default. If the video is say, 480P, you wouldn't need as much --merange as you would with say, 1080P or 2160P. --merange 25 is great at 480P, and when you think of it, it makes sense. 25 at 480P covers the same distance as what 56.25 (so 57) does at 1080P. Maybe there should be an autoscaler in x265 to adjust this automatically based on resolution, like I suggested above.

--early-skip, --fast-intra, --tskip, --tskip-fast, --limit-modes all improve speed without being detrimental to quality, as far as I see it anyway :). In combination with the other settings encoding is both fast and high quality. I chose --tu-intra-depth 3 because it does seem to be a little more efficient without impacting on performance too much, likewise with --maxmerge 3. I chose --rc-lookahead as 40, I see it as a slightly beneficial, and I figure it make sense being the same as scenecut. --ref 6 and --bframes 6 is just what I chose. Nice balance of efficiency versus speed. For most content is seems using 5 bframes is most efficient in terms of the stats at the end showing consecutive b-frame usage. Yes, 5, so I chose 6 for that little extra :).

A little more controversial may be my use of --nr-intra 400 and --nr-inter 400. I didn't list those above, but I do use them. I use these because it seems to get rid of some low level noise that affects compressibility, but doesn't impede too much on the output picture. For the same file size, you can lower the CRF (keeping in mind decimal CRF is allowed), and get effectively the same file size output. The quality gained by the lower CRF overcomes any loss as a result of the noise reduction, at least in my opinion.

So, there are my settings. I get very fast compressing speed with them, and a high quality output at a decent file size.

[x265_Project]

So therefore there is a difference between RD 3 and 4, with 4 allowing Psy-RDOQ?

Yes. Confirmed.

From x265.h, line 973...
/* Strength of psycho-visual optimizations in quantization. Only has an effect in presets which use RDOQ (rd-levels 4 and 5). The value must be between 0 and 50, 1.0 is typical. Default 1.0 */
double psyRdoq;

We'll update the documentation (http://x265.readthedocs.org/en/default/cli.html#cmdoption--psy-rdoq)to clarify this.

[Motenai Yoda]

Yes. Confirmed.

From x265.h, line 973...
/* Strength of psycho-visual optimizations in quantization. Only has an effect in presets which use RDOQ (rd-levels 4 and 5). The value must be between 0 and 50, 1.0 is typical. Default 1.0 */
double psyRdoq;
But rd-levels 3 and 4 aren't the same? also if it isn't used why to put it at 0?

It's an year old thing 75c019f (https://bitbucket.org/multicoreware/x265/commits/75c019fe32f8296d176dce676129bc20c3c467a8)
3 RDO mode and split decisions, chroma residual used for sa8d
4 Currently same as 3

[x265_Project]

But rd-levels 3 and 4 aren't the same? also if it isn't used why to put it at 0?

I think this is another opportunity for improvement of our documentation. Rate-Distortion Optimized Quantization (RDOQ) isn't performed until RD-level 4. We can't psy optimize RDOQ until we start doing RDOQ.

[burfadel]

The documentation still says 4 is the same as 3. Funnily enough though, when I was playing around with the settings I originally chose --rd 5 and rd-refine, as the picture seems to retain some of the texture better. It made for slow encoding though, so I dropped it to 4 and tried psy-rdoq. I never actually tried to use psy-rdoq with rd 3. psy-rdoq I feel is a good thing to use, it seems to maintain picture texture a bit better. It doesn't make the flatter surfaces too smooth. It's more a visual thing, maybe psychological... it seems to be more noticeable in the picture than with PSNR or SSIM. I'm just glad I spent the time to refine my settings, I'm very happy with the output and speed of encoding with the settings I'm using now (as listed in my previous post).

[nandaku2]

Open GOP is allowed, but that just means any frames that are "open" still come off the total frames in the GOP. Still can't add up to more than 1 second of frames. I'm sure x265 will fix the accounting shortly.

I dont think we want to change the definition of GOPsize - which currently means distance between 2 consecutive I/IDR frames, to include number of "open" frames.

If the UHD-BD spec has such a distinction, the best thing to do is probably enforce no-open-gop, imho.

[nandaku2]

I think this is another opportunity for improvement of our documentation. Rate-Distortion Optimized Quantization (RDOQ) isn't performed until RD-level 4. We can't psy optimize RDOQ until we start doing RDOQ.

Fixed the comment in x265.h. Many moons ago, rd-level 4 was rd-level 3 + rdoq and rd-level 6 was rd-level 5 + rdoq. Later we separated rdoq-level from rd-level, so now rd-level 3=4 and 5=6. And of course, we dont want to break our users' scripts by cleaning this down to 4 rd-levels.

Psy-rdoq will be in effect if rdoq-level is non-zero. It doesnt depend on rd-level.

[Motenai Yoda]

Instead I think that comment and that assignment are some leftovers of the 1.3.

I can't find any selection about rdLevel >(=) 4, < 4, > 3 or <= 3
only < 3, >= 3, <= 4 or < 5

edit: preceded by nandaku

ps @x265_Project sorry the board was getting me a db error so I ended to send you an pm, I saw it only now.

[kamineko]

We have that as well. For anime contents basically you wish to encourage bits to be invested in the edge areas, and that can be done by tweaking psy and aq.


Will probably do another extensive test and post our suggestions here after v2.0. Our current parameters are updated with v1.9 stable.

Looking forward to your recommendations after 2.0! What would you use as current parameters for archival of anime content for SD and HD (Bluray)? I am looking for something which is good quality for a home theater setup and a seasoned viewer, but still reasonable in terms of time and storage demands.

I know that there are huge differences in source material; something like Fuujin Monogatari (http://i.imgur.com/SpVZRC8.jpg) from DVD is completely different to Ao no Exorcist - Movie (http://i.imgur.com/jVR1VLG.jpg) from the BluRay. And that's probably different to Western 3D animation from Pixar and Disney as well.

Not to mention rotoscoped stuff like Ajin or Aku no Hana.

Does any of this make a difference for your encoding? Any recommendations?

[jlpsvk]

this is x264:
http://s12.postimg.org/4r368zrl9/SW_x264.png

and this x265:
http://s8.postimg.org/sldo3qj51/SW_x265.png

No matter what I do, the transition on the banner (from red to black - highlighted) is smoother on x264. On x265 you the the squares after the close-up. What do you advice?

My settings:
--crf 22 --preset slow --output-depth 10 --tu-intra-depth 2 --rdoq-level 1 --qg-size 16 --ipratio 1.3 --pbratio 1.2 --max-merge 2 --rc-lookahead 60 --ref 3 --min-keyint 23 --keyint 240 --colorprim bt709 --colormatrix bt709 --transfer bt709 --deblock -3:-3 --psy-rdoq 5

Anyone?

[huhn]

hard to judge it is not the same frame.

[jlpsvk]

know it's not the same frame, but the difference is obvious...no matter the frame. :(

[LigH]

Several fixes regarding UHD-BD, dithering of different color depths, and documentation clarifications:

x265 1.9+125-40afead3177d (https://www.mediafire.com/download/v435ptuzuvnf12c/x265_1.9+125-40afead3177d.7z)

[burfadel]

I notice in the development log the patch:
[x265] [PATCH] analysis: introduce analysis refinement level for analysis save/load feature

There seems to be a mistake in it!

In the top part, it says:
Higher the value, Lesser the information stored/reused, Slower the encode. Default 5.

This is backed up in the table below it, which shows the default level of 5 just storing 'Lookahead information', with 4 storing additionally Intra and Inter modes, ref's, and mode 3 storing rec-amp additionally (level 2 and 1 same as 3).

However, down the bottom of the patch in the command line help part:
--analysis-refine-level <1..5>Level of information stored/reused in analysis save/load mode 1:least....5:most.

That is back to front to the information at the top of the patch. I assume the reason for not just using what would be '1' according to the table as the default is memory use? In any case, just thought I'd mention it since it could be easily overlooked.

[2themax]

Several fixes regarding UHD-BD, dithering of different color depths, and documentation clarifications:

x265 1.9+125-40afead3177d (https://www.mediafire.com/download/v435ptuzuvnf12c/x265_1.9+125-40afead3177d.7z)
Does it address open GOP for UHD-BD?

[Ma]

Does it address open GOP for UHD-BD?

Not yet. The latest opinion about this bug from nandaku2 (x265 boss) is in message:
http://forum.doom9.org/showthread.php?p=1763866#post1763866

[LigH]

Not yet, I believe. I don't remember a patch specific to changing the count of frames per GOP in an open-GOP setup. There was just a removed allowance for 1088 lines with cropping.

See yourself: x265 commit log (https://bitbucket.org/multicoreware/x265/commits/)

[LigH]

P.S.: Closed GOPs will get forced for UHD-BD compatibility (https://bitbucket.org/multicoreware/x265/commits/e7d937ad1ea341eeebd210188e08540ab6104fef), maybe until a more elaborate logic might be programmed for the open GOP case.

[x265_Project]

P.S.: Closed GOPs will get forced for UHD-BD compatibility (https://bitbucket.org/multicoreware/x265/commits/e7d937ad1ea341eeebd210188e08540ab6104fef), maybe until a more elaborate logic might be programmed for the open GOP case.

At this point, we don't see the benefit of a smaller GOP size with open GOP, versus a larger GOP size. Does anyone really think this is a necessary capability?

[LigH]

As far as I remember, closed GOPs have slight advantages also regarding "seeking everywhere" and supporting speed playback features, already in times of the DVD Video with MPEG-2, and also on Blu-ray with AVC. Even though closed GOPs may not be necessary, they may be useful, and they are probably quite usual in commercial productions. Compatibility already required a few trade-offs in compression efficiency, open GOPs will be a marginal addend in the whole sum.

[hajj_3]

PowerDVD 16 is out today. Anyone fancy comparing the h265 decoder with the one from v15? V16 does not support UHD bluray discs btw, we don't know whether support will be added in a patch or if it will come in v17.

[x265_Project]

PowerDVD 16 is out today. Anyone fancy comparing the h265 decoder with the one from v15? V16 does not support UHD bluray discs btw, we don't know whether support will be added in a patch or if it will come in v17.
It remains to be seen if the security requirements for UHD-BD will allow a PC software player application.

[LigH]

x265 1.9+140-34a3d35c5f97 (https://www.mediafire.com/download/t7819gofasbr955/x265_1.9+140-34a3d35c5f97.7z): UHD-BD compatibility forces closed GOPs; --limit-refs clarified to be on/off; and a bunch of fixes and speed-ups (ARM implementations don't matter for x86 platform)

[RainyDog]

Can anyone explain the effects of --tu-inter-depth and --tu-intra-depth in layman terms for me please?

I'm currently encoding old ish TV series from DVD whilst experimenting with some settings in the process. It's been a while since I last tried x265 and I'm pleasantly surprised by what speeds I can get on my i7 3770k with a bit of settings juggling.

But I've just stumbled across that increasing both --tu-inter-depth and --tu-intra-depth from 1 to 2 causes a 15% reduction in bitrate with all other settings left the same. Is really that much of an efficiency gain or is it actually having a negative impact somewhere? The DVD's are PAL 720x576 so is resolution a factor? Perhaps 1 is best for SD, 2 for HD and 3 is for 4k? Going from 2 to 3 barely has any effect but the reduction in bitrate and file size when increasing from 1 to 2 is quite alarming!

Speed doesn't seem much different when set to 1, 2 or 3 but that might be because it's chugging along at a lower bitrate when set to 2 or 3.

Main settings I've settled on otherwise (for 576p resolution) are preset medium but with the following tweaks :-

--CRF 21 --ctu 16 --max-tu-size 8 --qg-size 16 --early-skip --b-intra --limitmodes --weightb --me star --merange 25 --max-merge 3 --subme 3 --ref 5 --bframes 5 --rc-lookahead 40

Thanks.

[benwaggoner]

Can anyone explain the effects of --tu-inter-depth and --tu-intra-depth in layman terms for me please?

I'm currently encoding old ish TV series from DVD whilst experimenting with some settings in the process. It's been a while since I last tried x265 and I'm pleasantly surprised by what speeds I can get on my i7 3770k with a bit of settings juggling.

But I've just stumbled across that increasing both --tu-inter-depth and --tu-intra-depth from 1 to 2 causes a 15% reduction in bitrate with all other settings left the same. Is really that much of an efficiency gain or is it actually having a negative impact somewhere? The DVD's are PAL 720x576 so is resolution a factor? Perhaps 1 is best for SD, 2 for HD and 3 is for 4k? Going from 2 to 3 barely has any effect but the reduction in bitrate and file size when increasing from 1 to 2 is quite alarming!

Speed doesn't seem much different when set to 1, 2 or 3 but that might be because it's chugging along at a lower bitrate when set to 2 or 3.

Main settings I've settled on otherwise (for 576p resolution) are preset medium but with the following tweaks :-

--CRF 21 --ctu 16 --max-tu-size 8 --qg-size 16 --early-skip --b-intra --limitmodes --weightb --me star --merange 25 --max-merge 3 --subme 3 --ref 5 --bframes 5 --rc-lookahead 40

Thanks.
Given a --max-tu-size 8 I don't know what --tu-*-depth can really do, and am surprised it would have any impact. From 8x8 it can only really go down one level to 4x4. Although if it's coming from a max cu-size of 32x32, to get to 8x8 you'd need to go down several levels to get to that size. Perhaps the tu-depth modes start from max TU size?

That's a pretty complex command line with some unusual settings. Perhaps try --preset slower --crf 18? A lower CRF plus more standard settings may give a good quality/size balance. If it's grainy source, maybe add in --tune grain.

[RainyDog]

Given a --max-tu-size 8 I don't know what --tu-*-depth can really do, and am surprised it would have any impact. From 8x8 it can only really go down one level to 4x4. Although if it's coming from a max cu-size of 32x32, to get to 8x8 you'd need to go down several levels to get to that size. Perhaps the tu-depth modes start from max TU size?

Thanks for the response. I'll have another play around with them again, perhaps increase intra and not inter and visa versa. And compare all the resulting encodes by eye. I was just taken back by the file size reduction from only changing those two settings. Too good to be true, didn't seem right at all.

That's a pretty complex command line with some unusual settings. Perhaps try --preset slower --crf 18? A lower CRF plus more standard settings may give a good quality/size balance. If it's grainy source, maybe add in --tune grain.

Not really from what I can gather. It's all pretty much based on what burfadel and littlepox have been recommending over the past few pages with a couple of other tweaks. --max-tu-size 16 seems to be the way to go for 1080p encodes at present so --max-tu-size 8 makes sense for 480-576p resolutions.

[burfadel]

I'm still a little confused about RD 3 and 4. Are they supposed to be the same or not?

The outputs for RD 3 and 4 are NOT the same consistently. In theory, if they were the same the only difference should be the info part of the output, with one showing RD 3 and the other 4. This is not the case, the outputs are actually different. I normally have RDOQ on level 1, but I disabled it for this comparison. Regardless of whether it is on or off, the output is different between RD 3 and 4. All settings are identical apart from RD 3 and 4. Also, the results are consistent across encoding the same file and same settings (just to show it's not a memory error or errata causing the difference):

RD 3:
x265 [info]: frame I: 15, Avg QP:18.01 kb/s: 10311.83
x265 [info]: frame P: 272, Avg QP:19.71 kb/s: 2785.46
x265 [info]: frame B: 923, Avg QP:26.51 kb/s: 384.35
x265 [info]: Weighted P-Frames: Y:2.6% UV:2.2%
x265 [info]: Weighted B-Frames: Y:1.1% UV:1.1%
x265 [info]: consecutive B-frames: 9.1% 2.4% 3.8% 49.8% 15.7% 15.7% 3.5%
encoded 1210 frames in 38.71s (31.26 fps), 1047.17 kb/s, Avg QP:24.87

RD 4:
x265 [info]: frame I: 15, Avg QP:18.01 kb/s: 10324.29
x265 [info]: frame P: 272, Avg QP:19.71 kb/s: 2787.77
x265 [info]: frame B: 923, Avg QP:26.49 kb/s: 385.29
x265 [info]: Weighted P-Frames: Y:2.6% UV:2.2%
x265 [info]: Weighted B-Frames: Y:1.1% UV:1.1%
x265 [info]: consecutive B-frames: 9.1% 2.4% 3.8% 49.8% 15.7% 15.7% 3.5%
encoded 1210 frames in 38.21s (31.67 fps), 1048.56 kb/s, Avg QP:24.86

They are similar, but NOT identical. This is using the settings I listed a few days ago, except for not using RDOQ which I normally have on. Since that was said to be the difference at one point, I disabled it to show the difference is elsewhere.

If they are meant to be the same, there is obviously a bug somewhere in x265. Something is definitely being done differently between RD 3 and 4!

The version used is:
x265 [info]: HEVC encoder version 1.9+140-34a3d35c5f97

[Ma]

I'm still a little confused about RD 3 and 4. Are they supposed to be the same or not?

They are not the same. There is only one difference in source code:
https://bitbucket.org/multicoreware/x265/src/34a3d35c5f97c7ecf76bbb3311a9bbb66db0d695/source/encoder/search.cpp?at=default&fileviewer=file-view-default#search.cpp-1479
-- for '--rd 4' maxCandCount has bigger value.

I don't know if they supposed to be the same.

[burfadel]

Yes. Confirmed.

From x265.h, line 973...
/* Strength of psycho-visual optimizations in quantization. Only has an effect in presets which use RDOQ (rd-levels 4 and 5). The value must be between 0 and 50, 1.0 is typical. Default 1.0 */
double psyRdoq;

We'll update the documentation (http://x265.readthedocs.org/en/default/cli.html#cmdoption--psy-rdoq)to clarify this.

But rd-levels 3 and 4 aren't the same? also if it isn't used why to put it at 0?

It's an year old thing 75c019f (https://bitbucket.org/multicoreware/x265/commits/75c019fe32f8296d176dce676129bc20c3c467a8)

I think this is another opportunity for improvement of our documentation. Rate-Distortion Optimized Quantization (RDOQ) isn't performed until RD-level 4. We can't psy optimize RDOQ until we start doing RDOQ.

Fixed the comment in x265.h. Many moons ago, rd-level 4 was rd-level 3 + rdoq and rd-level 6 was rd-level 5 + rdoq. Later we separated rdoq-level from rd-level, so now rd-level 3=4 and 5=6. And of course, we dont want to break our users' scripts by cleaning this down to 4 rd-levels.

Psy-rdoq will be in effect if rdoq-level is non-zero. It doesnt depend on rd-level.

So the consensus was that it came back to RD level 4 being the same as 3. Since it isn't, then the information should be further clarified.

From the x265 code you linked, this comment:
Find the top maxCandCount candidate modes with cost within 25% of best or among the most probable modes. maxCandCount is derived from the rdLevel and depth. In general we want to try more modes at slower RD levels and at higher depths

So, it makes sense in that RD 4 has a different output to RD 3, as more modes would have been tried. In theory therefore, 4 should be more efficient than 3, which I believe is the case (albeit slight). The speed difference for me when averaged out over different sections is non-existent, or at least not measurable, so RD 4 will remain my choice :).

[Motenai Yoda]

Well with --rd 4 it return a pretty identical stream, maybe some not deterministic stuff?

x265 [info]: frame I: 19, Avg QP:21.97 kb/s: 20434.61 PSNR Mean: Y:42.274 U:44.137 V:47.052 SSIM Mean: 0.957082 (13.674dB)
x265 [info]: frame P: 421, Avg QP:23.05 kb/s: 13604.57 PSNR Mean: Y:42.211 U:43.094 V:46.043 SSIM Mean: 0.962831 (14.298dB)
x265 [info]: frame B: 1706, Avg QP:26.91 kb/s: 2479.44 PSNR Mean: Y:40.630 U:42.025 V:45.616 SSIM Mean: 0.953919 (13.365dB)
x265 [info]: Weighted P-Frames: Y:34.7% UV:28.5%
x265 [info]: consecutive B-frames: 6.4% 2.3% 3.2% 26.1% 17.5% 38.6% 4.1% 1.6% 0.2%

encoded 2146 frames in 523.56s (4.10 fps), 4820.92 kb/s, Avg QP:26.11, Global PSNR: 41.712, SSIM Mean Y: 0.9556955 (13.536 dB)

x265 [info]: frame I: 19, Avg QP:21.97 kb/s: 20421.66 PSNR Mean: Y:42.276 U:44.138 V:47.054 SSIM Mean: 0.957075 (13.673dB)
x265 [info]: frame P: 421, Avg QP:23.05 kb/s: 13604.11 PSNR Mean: Y:42.212 U:43.092 V:46.043 SSIM Mean: 0.962820 (14.297dB)
x265 [info]: frame B: 1706, Avg QP:26.89 kb/s: 2480.82 PSNR Mean: Y:40.630 U:42.024 V:45.616 SSIM Mean: 0.953914 (13.364dB)
x265 [info]: Weighted P-Frames: Y:34.7% UV:28.5%
x265 [info]: consecutive B-frames: 6.4% 2.3% 3.2% 26.1% 17.5% 38.6% 4.1% 1.6% 0.2%

encoded 2146 frames in 518.16s (4.14 fps), 4821.82 kb/s, Avg QP:26.10, Global PSNR: 41.712, SSIM Mean Y: 0.9556887 (13.535 dB)

[burfadel]

Yes, very, very similar but not the same. It would also vary depending on content.

[x265_Project]

We'll be in Las Vegas next week for the 2016 NAB Show. I'll be giving a talk on HEVC Encoding on Monday, and we'll have a demo suite set up. If you're going and want to meet up, please send me a message.

Tom

nandaku2 and pradeeprama optimizing our live 4K encoding demo...
https://www.facebook.com/x265project/photos/a.1413021475581600.1073741826.1395701047313643/1691877084362703/?type=3&theater

[Magik Mark]

Hi there!

Is there any professional software that is using this project? May we know there company name so we could evaluate on how they are applying this technology to their product

Thanks a lot


Sent from my iPhone using Tapatalk

[x265_Project]

Hi there!
Is there any professional software that is using this project? May we know there company name so we could evaluate on how they are applying this technology to their product

In addition to FFMPEG, VLC, Handbrake, StaxRip, MeGUI, and many other open source software applications, x265 powers commercial solutions from leading encoder vendors like Telestream, BBright, Sorenson, Pegasys, and many others (most of which are confidential at this time), and many leading streaming video services and cloud encoding services.

[Magik Mark]

In addition to FFMPEG, VLC, Handbrake, StaxRip, MeGUI, and many other open source software applications, x265 powers commercial solutions from leading encoder vendors like Telestream, BBright, Sorenson, Pegasys, and many others (most of which are confidential at this time), and many leading streaming video services and cloud encoding services.



Thank you for this. Tried Sovenson. Unfortunately, they don't update their hevc feature. Lacks a lot of features

Can you recommend a professional software. Similar to squeeze desktop pro that uses all the features of this project

Thanks

[JohnLai]

In addition to FFMPEG, VLC, Handbrake, StaxRip, MeGUI, and many other open source software applications, x265 powers commercial solutions from leading encoder vendors like Telestream, BBright, Sorenson, Pegasys, and many others (most of which are confidential at this time), and many leading streaming video services and cloud encoding services.

Interesting........what are commercial features not offered by opensource software other than Graphical User Interface?

[fauxreaper]

https://bitbucket.org/multicoreware/x265/issues/214/ghosting-artefacts-even-with-low-crf-when

Can other people test --no-cutree to see if they find less ghosting on their encodes?

[Ma]

https://bitbucket.org/multicoreware/x265/issues/214/ghosting-artefacts-even-with-low-crf-when

Can other people test --no-cutree to see if they find less ghosting on their encodes?

Yes, it is much better with --no-cutree (almost OK).

[kolak]

Interesting........what are commercial features not offered by opensource software other than Graphical User Interface?

None...

It's more about things which you can do with this software. I don't think x265 engine itself is any better.

[stax76]

while running an automated test to find out if StaxRip needs to be updated I found out there are switches not listed under --help

--bframe-bias
--cbqpoffs
--chromaloc
--colormatrix
--cplxblur
--crf-max
--crf-min
--crqpoffs
--dither
--hash
--interlace
--ipratio
--lookahead-slices
--lossless
--nr-inter
--nr-intra
--pbratio
--qblur
--qcomp
--qp
--qpstep
--sao-non-deblock
--temporal-mvp
--tskip
--tskip-fast
--y4m

[sneaker_ger]

Second to last sentence when using --help:
Use --log-level full --help for a full listing

[benwaggoner]

Second to last sentence when using --help:
And better yet, the only good codec documentation that has ever existed:

http://x265.readthedocs.org/en/default/

[James Freeman]

Hi, I'm new at this.

Should I start from --preset slower (or slow?), --crf 18, as suggested by benwaggoner and test from there?
The resulting bitrate in average is 10,000 kb/s (3840x2160, 10bit 25fps), and looks good to me.
I can see Slower adds sao deblocking at the end, is it beneficial?

Is there any important settings I should be aware of for a casual user?
I saw --tune grain, if the video is from a film.

[birdie]

You should almost always choose the slowest encoding preset you can bear (aside from "placebo" which is called this way for a reason).

[James Freeman]

Thanks.

I can't find a intermediate codec that can do 10bit 4:2:2 AND 4K in AVI.
Any suggestion?

[birdie]

Um, do you mean container? Or encoder? The best encoder has always been ffmpeg.

[MeteorRain]

Thanks.

I can't find a intermediate codec that can do 10bit 4:2:2 AND 4K in AVI.
Any suggestion?

Try main422-10 profile.

Also AVI is a tricky choice. Like fit a car with some bicycle tires.

[Magik Mark]

Is it advisable to finish encoding first before using the computer for other applications? Will the quality of the encodes affected if cpu load starts shifting for other apps?


Sent from my iPhone using Tapatalk

[birdie]

Encoding/decoding are a strict mathematical process. In theory, if your CPU/RAM/HDD don't falter during work, the output will always be the same regardless of your workflow. Make sure you run your encoder at the lowest CPU priority of you may experience significant lags.

[James Freeman]

Try main422-10 profile.
Also AVI is a tricky choice. Like fit a car with some bicycle tires.

main422-10 is restricted to 1080p in After Effects, plus the file size is HUGE.

Anyway I found a AVI lossless encoder named Lagarith, it makes the file around 20 times smaller in 4:2:0 8bit, than Uncompressed RGB 8bit.
I can export straight to YV12 (4:2:0) YUY2 (4:2:2) or RGB in any resolution.

Another thing I found that AE dithers down from any higher bit depth, so the 32bit workflow is dithered down to 8bit at export.
My experience with madVR tells my that properly dithered 8bit is indistinguishable from 12bit (as read by i1 Display Pro).

When I encode with x265 I specify --input-depth 10, I guess that a dithered 8bit file inside a 10bit file would be perfectly fine.
Not true 10bit but still smooth as silk.

[Motenai Yoda]

mmm I'm not sure but IIRC 32bit in AE refers to all channel in a RGB behaviour, which is roughtly the same as 8bit in the single channel YUV one.
Also Lagarith is very slow with HD stuff, way better UtVideo or MagicYUV

[James Freeman]

mmm I'm not sure but IIRC 32bit in AE refers to all channel in a RGB behaviour, which is roughtly the same as 8bit in the single channel YUV one.
Also Lagarith is very slow with HD stuff, way better UtVideo or MagicYUV

No, 32bit Floating per channel, there is also 16bit and 8bit.
AE dithers down to 8bit if the workflow is in higher bit depth.

UtVideo, is EXACTLY what I needed, fast and efficient. thank you!

AE will not let me export as 10bit (with any codec that is not from adobe) in AVI no matter what.
Dithered 8bit looks as good as 12bit anyway.

[LigH]

Starting from a crf+preset combo is certainly a good idea. The developers spend some efforts in making presets a good all-purpose set of parameters. But slow presets will cost a lot of time, and even more if your CPU does not support AVX(2).

The "grain" tuning is meant to retain especially strong grain (think of "300"). Test on your own whether or not it is required and good-looking for your material... and expect some bitrate demand.

[dipje]

Thanks.

I can't find a intermediate codec that can do 10bit 4:2:2 AND 4K in AVI.
Any suggestion?

There are threads about questions like that. In .AVI is a problem though. Personally I'm using Vapoursynth and AfterEffects (and other tools) and happily go in and out through things like ProRes, DNXHD, DPX sequences and / or .EXR with PIZ compression.

Can't Cineform do 4K? I know it can do yuv422p10.. (just tested, my After Effects can export Cineform in 4K). Grassvaley HQX also seems to work in 4K alright.

It are both codecs that can be stored in AVI, although I can't think of any program that can interface properly with it. You could try going from Vapoursynth YUV422P10 (V210 fourcc) open it with virtualdub and from there save in one of two formats.

But honestly, there is a reason that all video editing packages seem to prefer Mov for this stuff. Avi is tricky enough already to store modern 8-bit stuff in, let alone 10p, 12p, 12p + alpha or other more 'cinema' style formats.

[James Freeman]

I realized that HEVC main10 output chroma subsampling has to be equal to the input, in raw YUV (uncompressed), so it has to be 10bit 4:2:0 uncompressed first.

If it is Lossless (compressed) I have to push it to HEVC through Avisynth with avs4x265 so only AVI is optional.
I want to export from AE in 10bit 4:2:0 4K (P010), MOV or AVI so that I can encode with HEVC.

Any idea?

[benwaggoner]

There are threads about questions like that. In .AVI is a problem though. Personally I'm using Vapoursynth and AfterEffects (and other tools) and happily go in and out through things like ProRes, DNXHD, DPX sequences and / or .EXR with PIZ compression.

Can't Cineform do 4K? I know it can do yuv422p10.. (just tested, my After Effects can export Cineform in 4K). Grassvaley HQX also seems to work in 4K alright.

It are both codecs that can be stored in AVI, although I can't think of any program that can interface properly with it. You could try going from Vapoursynth YUV422P10 (V210 fourcc) open it with virtualdub and from there save in one of two formats.

But honestly, there is a reason that all video editing packages seem to prefer Mov for this stuff. Avi is tricky enough already to store modern 8-bit stuff in, let alone 10p, 12p, 12p + alpha or other more 'cinema' style formats.
The AVI format itself is fine for this kind of use, and has the advantage of being simpler, unlike QuickTime files which in theory are Turing Complete :). And with the deprecation of QuickTime for Windows with a serious security flaw in the final version, Everyone not on Mac is going to have to use other implementations.

I've had AVI files >200 GB without any issues.

v210 is always good. Cineform absolutely works for UHD. Sometimes the stock DirectShow version doesn't work well, but the BlackMagic codec version does, and is broadly compatible. DNxHR is promising, and doesn't have the locked-down frame rates and frame sizes of DNxHD.

What we're lacking is a good codec that supports all the HDR metadata. The only thing I know of is just using a high-bitrate HEVC. HEVC-Intra with WPP could make a very good mezzanine codec: 10 and 12 bit support, and better efficiency than anything else. With WPP reasonably fast decoding on multicore systems, and with hardware decoders coming it'll be blazing fast. I can certainly get the same quality as ProRes at half the bitrate.

[stax76]

It seems x265 fails on Unicode file names, no problem with x264, QSVEncC, NVEncC or ffmpeg.


------------------------------------------------------------
Encoding video using x265 1.9+140 x64 multi lib 8/10/12 bit
------------------------------------------------------------

@echo off
CHCP 65001
"C:\Program Files (x86)\VapourSynth\core64\vspipe.exe" D:\Пред_temp\Пред.vpy - --y4m | D:\Projekte\GitHub\staxrip\bin\Apps\x265\x265_ml.exe --crf 22 --frames 1301 --y4m --output D:\Пред_temp\Пред_out.hevc -

cmd.exe /C call "D:\Пред_temp\Пред_encode.bat"

Aktive Codepage: 65001.
y4m [info]: 1280x720 fps 30000/1001 i420p8 unknown frame count
x265 [error]: failed to open output file <D:\????_temp\????_out.hevc> for writing
Error: fwrite() call failed when writing frame: 1, plane: 0, line: 11, errno: 22
Output 9 frames in 0.04 seconds (218.50 fps)

------------------------------------------------------------
Error Encoding video using x265 1.9+140 x64 multi lib 8/10/12 bit
------------------------------------------------------------

Encoding video using x265 1.9+140 x64 multi lib 8/10/12 bit failed with exit code: 1 (0x1)

The exit code might be a system error code: STATUS_WAIT_1

The exit code might be a system error code: Unzulässige Funktion.

Aktive Codepage: 65001.
y4m [info]: 1280x720 fps 30000/1001 i420p8 unknown frame count
x265 [error]: failed to open output file <D:\????_temp\????_out.hevc> for writing
Error: fwrite() call failed when writing frame: 1, plane: 0, line: 11, errno: 22
Output 9 frames in 0.04 seconds (218.50 fps)

[f81ccx]

Has anyone been able to get thread pool paramaters to work with ffmpeg?

I've tried passing in x265-params pools=16,16 as well as pools=+,+, but I only ever get a single thread pool instead of the 2 pools that I expect.

Machine:

Intel Xeon E5-2670 x2, Fedora 23. Lastest static build of ffmpeg that includes x265 version 1.9+141-02d79be487d7

Command line:

ffmpeg -i source.mov -c:v libx265 -preset veryslow -x265-params crf=20:pools=16,16 test.mkv


Output:
x265 [info]: HEVC encoder version 1.9+141-02d79be487d7
x265 [info]: build info [Linux][GCC 5.3.1][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x265 [info]: Main 4:2:2 10 profile, Level-4 (Main tier)
x265 [info]: Thread pool created using 16 threads
x265 [info]: frame threads / pool features : 6 / wpp(17 rows)
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 3 inter / 3 intra
x265 [info]: ME / range / subpel / merge : star / 57 / 4 / 4
x265 [info]: Keyframe min / max / scenecut : 24 / 250 / 40
x265 [info]: Lookahead / bframes / badapt : 40 / 8 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 1
x265 [info]: References / ref-limit cu / depth : 5 / off / on
x265 [info]: AQ: mode / str / qg-size / cu-tree : 1 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress : CRF-20.0 / 0.60
x265 [info]: tools: rect amp limit-modes rd=6 psy-rd=2.00 rdoq=2 psy-rdoq=1.00
x265 [info]: tools: signhide tmvp b-intra strong-intra-smoothing deblock sao

[LigH]

Welcome to doom9...

So you have only one single physical CPU? I believe thread pools are meant to support the physical separation of multi-socket systems, but may require explicit NUMA support, depending on the OS.

[Ma]

Has anyone been able to get thread pool paramaters to work with ffmpeg?
[...]
Intel Xeon E5-2670 x2, Fedora 23. Lastest static build of ffmpeg that includes x265 version 1.9+141-02d79be487d7

For Linux you should compile x265 with ENABLE_LIBNUMA (it is ON by default).

[f81ccx]

Welcome to doom9...

So you have only one single physical CPU? I believe thread pools are meant to support the physical separation of multi-socket systems, but may require explicit NUMA support, depending on the OS.

My system is a Dual Xeon E5-2670.

I'll have a go at building ffmpeg from scratch, making sure libnuma is included.

[Ma]

It seems x265 fails on Unicode file names, no problem with x264, QSVEncC, NVEncC or ffmpeg.

Yes, it fails. You can try attached patch for output file.

[stax76]

Do you by chance have a 8/10/12 bit multilib build?

[Ma]

Do you by chance have a 8/10/12 bit multilib build?

www.msystem.waw.pl/x265/utf16_output.7z

It is only output file fixed (remains: input y4m + yuv and stat file).

[stax76]

www.msystem.waw.pl/x265/utf16_output.7z

It is only output file fixed (remains: input y4m + yuv and stat file).

works great, thanks!

[Magik Mark]

I have a single Xeon 2695v3 CPU. May I ask which among the x265 build is best for my CPU? Is it the ICC build? GCC 5.3 / 6? AVX2?

Thanks

[x265_Project]

I have a single Xeon 2695v3 CPU. May I ask which among the x265 build is best for my CPU? Is it the ICC build? GCC 5.3 / 6? AVX2?

Thanks
x265 has over a thousand kernels that have been hand-optimized with the latest SIMD instruction sets (SSE2, SSE3, SSE4, AVX, AVX2). Your compiler won't change these kernels one bit. For the rest of the code, there are very minor differences in performance due to different compilers (ICC vs GCC).

[Ma]

I've made extended version of Unicode patch for Windows -- now output file works and stat file(s). Console info is also fixed including error messages, example:
i:\vs\x265\ma\XP>x265 --pass 1 --bitrate 4500 --stat Łojezu_Łotr-jeden?.stat /speed/720p50_parkrun_ter.y4m -o W_żółtych_płomieni
ach_liści.hevc
y4m [info]: 1280x720 fps 50/1 i420p8 sar 1:1 frames 0 - 503 of 504
raw [info]: output file: W_żółtych_płomieniach_liści.hevc
x265 [info]: HEVC encoder version 1.9+147-19cced21060f
x265 [info]: build info [Windows][GCC 6.1.0][64 bit] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x265 [info]: Main profile, Level-4 (Main tier)
x265 [info]: Thread pool created using 4 threads
x265 [info]: frame threads / pool features : 2 / wpp(12 rows)
x265 [error]: can't open stats file Łojezu_Łotr-jeden?.stat.temp
x265 [error]: failed to open encoder


Sample multilib build (WinXP compatible, 64-bit and 32-bit + patch):
www.msystem.waw.pl/x265/utf16_output-stat.7z

If there will be no errors, tomorrow I will send this patch to devel-list.

[stax76]

@MA

The first was perfect, this one encodes but the console output looks wrong:

https://i.imgur.com/DUOqiXY.png

My code works for all other tools:


Overloads Sub Encode(passName As String, batchCode As String)
batchCode = "@echo off" + BR + "CHCP 65001" + BR + batchCode
Dim batchPath = p.TempDir + p.TargetFile.Base + "_encode.bat"
File.WriteAllText(batchPath, batchCode, New UTF8Encoding(False)) 'UTF8 without BOM

Using proc As New Proc
proc.Init(passName)
proc.Encoding = Encoding.UTF8
proc.File = "cmd.exe"
proc.Arguments = "/C call """ + batchPath + """"
proc.Start()
End Using
End Sub

[Ma]

@stax76
Thanks for info! I've changed output mode for stderr from _O_U16TEXT to _O_U8TEXT. New test build (only 64-bit + patch):
www.msystem.waw.pl/x265/utf16_output-stat2.7z

[stax76]

@Ma

Thanks for the test build, I tested various things without noticing any problem!

[pingfr]

The true power of the Open Source. :)

[x265_Project]

The true power of the Open Source. :)

It's unbeatable. Thanks guys. :)

[Ma]

Patch which allows Unicode filenames in Windows (for now output file and stat files) has been sent
https://patches.videolan.org/patch/13109/

Next we can fix input files.
------------------------
GCC on old mingw-w64 not works with this code. First workaround is:
diff -r 00ea3784bd36 source/common/common.cpp
--- a/source/common/common.cpp Thu Apr 28 09:59:30 2016 +0200
+++ b/source/common/common.cpp Fri Apr 29 03:12:31 2016 +0200
@@ -185,7 +185,7 @@
MultiByteToWideChar(CP_UTF8, 0, buffer, -1, buf_utf16, sizeof(buf_utf16)/sizeof(wchar_t));
fflush(stderr);
int oldmode = _setmode(_fileno(stderr), _O_U8TEXT);
- fwprintf(stderr, L"%ls", buf_utf16); // WARNING: due to bug in msvcrt.dll fputws doesn't work in mingw/gcc
+ fwprintf_s(stderr, L"%ls", buf_utf16); // WARNING: due to bug in msvcrt.dll fputws doesn't work in mingw/gcc
fflush(stderr);
_setmode(_fileno(stderr), oldmode);
}Under investigation...

[Ma]

Could someone with WinXP test this 32-bit version of x265:
www.msystem.waw.pl/x265/32bit-xp.zip

I've tested on Win7 64bit and Vista 32bit -- works. I don't have WinXP.
It should work and should be line "raw [info]:"
y4m [info]: 1280x720 fps 50/1 i420p8 sar 1:1 frames 0 - 503 of 504
raw [info]: output file: Żółć.hevc
x265 [info]: HEVC encoder version 1.9+150-00ea3784bd36
x265 [info]: build info [Windows][GCC 5.3.0][32 bit] 8bit

This is due a bug in old mingw-w64 (v4 for example) in function fwprintf, so we can try fwprintf_s instead.
-------------------
Finally I give up with mingw-w64 (for ver. 3 I can't write working code), so it is easier switch to Windows API. If someone wants to try new code:
hg import --no-commit https://patches.videolan.org/patch/13138/raw/

[pie]

Could someone with WinXP test this 32-bit version of x265:
www.msystem.waw.pl/x265/32bit-xp.zip



Doesn't work:
http://i.imgur.com/v4bLZpq.png

[Ma]

Doesn't work:
http://i.imgur.com/v4bLZpq.png

Thanks!

I've prepared new samples compiled with patch
https://patches.videolan.org/patch/13138/
Could you test this on WinXP (on 99% should work, both compiled by GCC 4.8.3 and GCC 5.3):
www.msystem.waw.pl/x265/32bit-xp2.zip

[pie]

Thanks!
Could you test this on WinXP (on 99% should work, both compiled by GCC 4.8.3 and GCC 5.3):
www.msystem.waw.pl/x265/32bit-xp2.zip


Both versions seem to work.

raw [info]: output file: Zólc.hevc

[Ma]

Both versions seem to work.

raw [info]: output file: Zólc.hevc

Thanks! So we should make a fix to mingw-w64 code (but this may take a long while). For now it is better to apply 13138 patch.
---------------
After investigation -- mingw-w64 code looks OK, x265 v1.9+150 code looks OK, the bug is in msvcrt.dll. Almost all mingw-w64 stdio code is written with use of fputwc/fputc functions from msvcrt.dll. fputwc function from msvcrt.dll doesn't work with _O_U8TEXT mode (works only with _O_U16TEXT mode). If you build x265 v1.9+150 by mingw-w64/gcc linked to msvcrt.dll, there will be no "raw [info]" line in console. If you link to msvcr110.dll instead of msvcrt.dll, all is working (without any changes in x265 code nor mingw-w64 code). Example www.msystem.waw.pl/x265/gcc483-xp.7z -- x265n.exe is linked to msvcr110.dll and works OK without any patches.

[James Freeman]

I realized that HEVC main10 output chroma subsampling has to be equal to the input, in raw YUV (uncompressed), so it has to be 10bit 4:2:0 uncompressed first.

If it is Lossless (compressed) I have to push it to HEVC through Avisynth with avs4x265 so only AVI is optional.
I want to export from AE in 10bit 4:2:0 4K (P010), MOV or AVI so that I can encode with HEVC.

Any idea?

Bump this.

Any idea how I can push a EXR sequence (individual file fer frame) to x265?
I tried everything in after effects but can't generate a 10bit 4:2:0 4K MOV or AVI file for x265 Main10.

Any suggestion to go from After Effects workflow to 10bit 4:2:0 4K is much appreciated.

[James Freeman]

Okay, Ihave managed to maintain a 10bit 4K workflow from start to finish.

First I export to PNG image sequence in 16bit (smallest file format compared to TIFF, DPX, EXR).
Next I use ffmpeg PNG image sequence to Y4M raw in yuv420p10, 16bit RGB full range PNG converted and dithered down to 10bit YUV 4:2:0 Limited range by ffmpeg with 1:1 color accuracy.
8 seconds Y4M (192 frames) = 4GB !!

Anyone knows a better way to keep a 4:2:0 10bit 4K workflow in a smaller footprint?
For test patterns it is pointless to use more than 1 fps, so the Y4M is proportionally smaller since it is uncompressed.

For anyone interested, here are my steps:
ffmpeg.exe -framerate 24 -i "E:\Sequence\img-%%05d.png" -strict -1 -pix_fmt yuv420p10 -vf colormatrix=bt601:bt709 "E:\x265\RAW.Y4M"

x265.exe --input-depth 10 --uhd-bd --preset slow --crf 18 --colorprim bt2020 --transfer smpte-st-2084 --colormatrix bt2020nc --master-display "G(8500,39850)B(6550,2300)R(35400,14600)WP(15635,16450)L(10000000,1)" --max-cll "0,0" --output "E:\x265\output.hevc" "E:\x265\RAW.y4m"

mp4box.exe -add E:\X265\output.hevc -new E:\X265\output.mp4

[LigH]

I guess ffmpeg could read image sequences and pipe Y4M. But no guarantees, and no idea about quirks how to manage the color depths.

[James Freeman]

What do you mean "no guarantee" and "quirks how to manage the color depths"?
Can you be more specific?

[LigH]

I cannot guarantee that ffmpeg can handle 16 bit per component PNG.

If it can, I don't know which command line options are required to convert to 10 bit per component YUV 4:2:0.

But it should be worth discovering if it works, and how.

[sneaker_ger]

Piping is simple. Replace ffmpeg output and x264 input file name with a "-", signal both to use y4m (since they don't have a file name they cannot guess from the file extension) and write the commands in a single line connect with a pipe symbol "|".
ffmpeg.exe -framerate 24 -i "E:\Sequence\img-%%05d.png" -strict -1 -pix_fmt yuv420p10 -f YUV4MPEGPIPE - | x265.exe --input-depth 10 --uhd-bd --preset slow --crf 18 --colorprim bt2020 --transfer smpte-st-2084 --colormatrix bt2020nc --master-display "G(8500,39850)B(6550,2300)R(35400,14600)WP(15635,16450)L(10000000,1)" --max-cll "0,0" --output "E:\x265\output.hevc" - --y4m

The color thing might indeed not be trivial at all. PNG is RGB not YUV.

[James Freeman]

It definitely works and the conversion from 16bit RGB Full range to 10bit YUV 4:2:0 Limited range is flawless.
The Y4M 10bit is properly dithered by ffmpeg and not simply truncated.
A smooth 16bit grey ramp looks absolutely identical after the conversion.

But if you find flaws, this will be informative to us all.

[James Freeman]

Piping is simple. Replace ffmpeg output and x264 input file name with a "-", signal both to use y4m (since they don't have a file name they cannot guess from the file extension) and write the commands in a single line connect with a pipe symbol "|".
ffmpeg.exe -framerate 24 -i "E:\Sequence\img-%%05d.png" -strict -1 -pix_fmt yuv420p10 -f YUV4MPEGPIPE - | x265.exe --input-depth 10 --uhd-bd --preset slow --crf 18 --colorprim bt2020 --transfer smpte-st-2084 --colormatrix bt2020nc --master-display "G(8500,39850)B(6550,2300)R(35400,14600)WP(15635,16450)L(10000000,1)" --max-cll "0,0" --output "E:\x265\output.hevc" - --y4m

The color thing might indeed not be trivial at all. PNG is RGB not YUV.

You mean I don't have to create a HUGE y4m file before going to x265?
Why that is absolutely PERFECT!

I'm off to try it.

[sneaker_ger]

It definitely works and the conversion from 16bit RGB Full range to 10bit YUV 4:2:0 Limited range is flawless.
The Y4M 10bit is properly dithered by ffmpeg and not simply truncated.
But we are not just talking about range and dithering. You want bt.2020. You want HDR. You want non-mpeg2-chroma position. I highly doubt this is achieved with your simple ffmpeg command.

[James Freeman]

The PNG images are already in 2020 and HDR, they are exported from After Effects with Color space and range conversion with official Rec.2020 ST.2084 (PQ) profiles and edited with 1000 nit in mind.
Think of the PNG files as though I just graded a movie on a Rec.2020 ST.2084 monitor.

I don't know about chroma position but it looks right to me on default settings.

EDIT:
The pipe works perfectly, you saved me a lot of effort, thank you!

[Jamaika]

mp4box.exe -add E:\X265\output.hevc -new E:\X265\output.mp4
Correct entry:;)
mp4box.exe -new -info -add E:\X265\output.h265:mpeg4:noedit:trailing:xps_inband E:\X265\output.mp4
or
mkvmerge.exe --ui-language en --fourcc 0:HEVC --cues 0:iframes "E:\X265\output.h265" --track-order 0:0 --disable-track-statistics-tags --output "E:\X265\output.mkv"

[James Freeman]

I had to add "-vf colormatrix=bt601:bt709" to ffmpeg so the RGB -> YUV color conversion will be correct.
Now it looks and measures 1:1 to the RGB.

EDIT Important!
The "-vf colormatrix=bt601:bt709" should be changed to "-vf scale=out_color_matrix=bt709" because the colormatrix command is only 8bit and it dithers down to 8bit.
While the scale=out_color_matrix=bt709 command bypasses any conversion while retains correct chroma values between RGB to YUV conversion.

[sneaker_ger]

mkvmerge.exe --ui-language en --fourcc 0:HEVC --cues 0:iframes "E:\X265\output.h265" --track-order 0:0 --disable-track-statistics-tags --output "E:\X265\output.mkv"
Don't give people funny ideas on how to use mkvmerge. A simple "mkvmerge -o output.mkv input.hevc" is totally fine. I hope your mp4box example is better, I don't know enough about mp4box to judge it.

[James Freeman]

Okay I run a single batch file that does everything in one click:
ffmpeg.exe -framerate 24 -i "E:\x265\Sequence\%%05d.png" -strict -1 -pix_fmt yuv420p10 -vf colormatrix=bt601:bt709 -f YUV4MPEGPIPE - | x265.exe --input-depth 10 --uhd-bd --preset slow --crf 18 --colorprim bt2020 --transfer smpte-st-2084 --colormatrix bt2020nc --master-display "G(8500,39850)B(6550,2300)R(35400,14600)WP(15635,16450)L(10000000,1)" --max-cll "0,0" --output "E:\x265\output.hevc" - --y4m
mp4box.exe -add E:\X265\output.hevc -new E:\X265\output.mp4
pause
Thanks sneaker_ger.

EDIT Important!
The "-vf colormatrix=bt601:bt709" should be changed to "-vf scale=out_color_matrix=bt709" because the colormatrix command is only 8bit and it dithers down to 8bit.
While the scale=out_color_matrix=bt709 command bypasses any conversion while retains correct chroma values between RGB to YUV conversion.

[surami]

James, please write your post into to the 4K HDR encoding topic too and post the example files.

Ps.: As I have time I will try this with Cineform RGBA 12bit files (instead of pngs), I read somewhere that ffmpeg supports them already... but if somebody knows more please say something.

[James Freeman]

Alright surami.

Basically What I need now is the smallest high bitdepth Image file that After Effects will render the fastest.
Cineform RGBA 12bit is smaller that PNG 16bit?

[surami]

I need to test too what you did, what was your exact visual source, settings, etc., let's continue in the other topic, if that is good for you and so poeple could follow there the process.

[James Freeman]

Alright moving there, I'll reserve the x265 HEVC question to this thread.

[Jamaika]

I hope your mp4box example is better, I don't know enough about mp4box to judge it.
MP4Box recently is more stable. Although HEVC codec without command ":xps_inband" has a black screen. Player reads it as 8bit.
I use .h265 in MP4Box because of .hevc I had problems previously.
I had to add "-vf colormatrix=bt601:bt709" to ffmpeg so the RGB -> YUV color conversion will be correct.
For what purpose?

[James Freeman]

Actually everything is A OK with the mp4box I'm using with the very basic settings.
Right image and 10bit.

[James Freeman]

After few tests, OpenEXR sequence is actually smaller and faster to render than PNG, and it is in floating point too.
And most importantly ffmpeg can work with EXR.

I had to add "-vf colormatrix=bt601:bt709" to ffmpeg so the RGB -> YUV color conversion will be correct.
For what purpose?

Without this command the last few steps of Chroma are clipped (shifted).
I read somewhere that ffmpeg treats RGB as 601 so this conversion has to occur for the colors to be right in 709.
After encoding I compared with the original RGB image and indeed the colors are accurate 1:1 which is important when creating calibration/measurement patterns.
http://forum.videohelp.com/threads/366469-x264-Which-colorspace-is-best-for-input?p=2340122&viewfull=1

EDIT Important!
The "-vf colormatrix=bt601:bt709" should be changed to "-vf scale=out_color_matrix=bt709" because the colormatrix command is only 8bit and it dithers down to 8bit.
While the scale=out_color_matrix=bt709 command bypasses any conversion while retains correct chroma values between RGB to YUV conversion.

[LigH]

If you have an ffmpeg with included libx265, you may not need to pipe to a separate x265 encoder, not even create a raw *.h265 video first. But anyway, remultiplexing to a complete MP4 container using MP4Box or L-SMASH is still recommendable, I am not sure how complete the MP4 container created by ffmpeg is (it used to be adversely arranged, due to its "filter" / FIFO behaviour: attributes only known at the end of the conversion have to be written to the end of the output).

[benwaggoner]

Alright surami.

Basically What I need now is the smallest high bitdepth Image file that After Effects will render the fastest.
Cineform RGBA 12bit is smaller that PNG 16bit?
I would stay away from an RGBA codec if at all possible, since that'll force another two color transforms. I'd want to render in 32-bit float to a YUV color space. There's no reason not to render to 10-bit if you're already in PQ space with 2020 primaries. You only need 12-bit if targeting Rec. 2020 12-bit HDR. Note both 12-bit 2020 and HDR-10 require non-gamma luma curves, and I'm not sure how to get the Adobe products to do those natively.

Looking at what Adobe Media Encoder supports, high quality options include:

DNxHR
Cineform YUV 10-bit
JPEG 2000 MXF OP1a YUV 4:2:2 10-bit


Blackmagic v210 certainly works as well, although is huge.

If you can get native OpenEXR to work as you describe, with the correct luma curve, that seems pretty optimal to me.

In practice, the best HDR-10 mezz format is likely a high quality HEVC, as that's the only bitstream format that can include all the HDR-10 metadata, including MaxFALL and MaxCLL. And it can do chromaloc 2, if you can find a tool that'll actually do the 0.25 pel chroma sample vertical offset. That might require a whole new color space format in ffmpeg.

[Magik Mark]

Guys,

Can you help me with my ghost problem. Maybe a missing switch? Doing x265 10bit 2pass via staxrip

http://s32.postimg.org/nr161hk4h/Capture.jpg (http://postimg.org/image/nr161hk4h/)

[fauxreaper]

Guys,

Can you help me with my ghost problem. Maybe a missing switch? Doing x265 10bit 2pass via staxrip

http://s32.postimg.org/nr161hk4h/Capture.jpg (http://postimg.org/image/nr161hk4h/)

There is some discussion about this in https://bitbucket.org/multicoreware/x265/issues/214/ghosting-artefacts-even-with-low-crf-when.

[Magik Mark]

There is some discussion about this in https://bitbucket.org/multicoreware/x265/issues/214/ghosting-artefacts-even-with-low-crf-when.

This one is happening using 2pass. I didn't have this problem before. I think it may have something to do with some switches being change when x265 was updated.

I'm comparing my encodes with others, the only difference that I could see right now are bitrate and "tune->grain". I'm inclined it may have something to do with the latter. Maybe the developers can take a look at it.

[Jamaika]

I read somewhere that ffmpeg treats RGB as 601 so this conversion has to occur for the colors to be right in 709.
I am surprised by this limitation. How to cut off the gamut colors to the BT601 isn't regain it when converting to BT2020.
http://www.earthboundlight.com/phototips/in-camera-color-space-for-raw-shooters.html
http://www.laserfocusworld.com/content/dam/lfw/print-articles/2014/04/1404LFW06f3.jpg
PS It also means that the software isn't for this conversion.):

[Ma]

Can you help me with my ghost problem.

You can try --no-cutree option.

[Magik Mark]

You can try --no-cutree option.

"Tune->Grain" --no-cutree has been activated. Still "Ghosting"

x265 1.9+147-19cced21060f:[Windows][GCC 6.1.0][64 bit] 10bit
Encoding settings : wpp / ctu=64 / min-cu-size=8 / max-tu-size=32 / tu-intra-depth=1 / tu-inter-depth=1 / me=1 / subme=2 / merange=57 / no-rect / no-amp / max-merge=2 / temporal-mvp / no-early-skip / rdpenalty=0 / no-tskip / no-tskip-fast / strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / no-fast-intra / open-gop / no-temporal-layers / interlace=0 / keyint=250 / min-keyint=23 / scenecut=40 / rc-lookahead=20 / lookahead-slices=6 / bframes=4 / bframe-bias=0 / b-adapt=2 / ref=3 / limit-refs=3 / no-limit-modes / weightp / no-weightb / aq-mode=0 / qg-size=64 / aq-strength=0.00 / cbqpoffs=0 / crqpoffs=0 / rd=3 / psy-rd=2.00 / rdoq-level=0 / psy-rdoq=0.00 / no-rd-refine / signhide / deblock=0:0 / sao / no-sao-non-deblock / b-pyramid / no-cutree / no-intra-refresh / rc=2 / pass / bitrate=1483 / qcomp=0.60 / qpmin=0 / qpmax=51 / qpstep=1 / cplxblur=20.0 / qblur=0.5 / ipratio=1.10 / pbratio=1.00

[LigH]

x265 1.9+150-00ea3784bd36 (https://www.mediafire.com/download/5td3190cir9dbbe/x265_1.9+150-00ea3784bd36.7z): Unicode filenames support for output and stats; and some rate control improvements

[Magik Mark]

x265 1.9+150-00ea3784bd36 (https://www.mediafire.com/download/5td3190cir9dbbe/x265_1.9+150-00ea3784bd36.7z): Unicode filenames support for output and stats; and some rate control improvements



Ghosting has been addressed?

[LigH]

My post was not a reply to your post.

Ah, by the way: Your x265 was compiled with GCC 6.1.0; this new version has different defaults, I would not be surprised if x265 did not yet ensure to be still compatible. May or may not be related to your ghosting sample. Anyway, if the reason was already found, I guess we would have read about a fix. But not in the last 10 commits.

[littlepox]

I don't think there is an easy solution to the ghosting effect; it is not a single bug, but symptom resulted from the entire RC scheme of x265, which heavily relies on comprehensive macroblock-based temporal prediction and strong psy tuning.

We might need to see tons of "RC improvements" before we get it done, with some sacrifice in other aspects.

[hajj_3]

a new VP9 encoder called Eve is out, it claims to be much better than x264, no info about how it compares to x265, thought you guys would be interested: https://blogs.gnome.org/rbultje/2016/05/02/the-worlds-best-vp9-encoder-eve-2/

[Magik Mark]

My post was not a reply to your post.

Ah, by the way: Your x265 was compiled with GCC 6.1.0; this new version has different defaults, I would not be surprised if x265 did not yet ensure to be still compatible. May or may not be related to your ghosting sample. Anyway, if the reason was already found, I guess we would have read about a fix. But not in the last 10 commits.

What I did right now is turned off "Tune->Grain" then activate --no-cutree option. So far it's looking good.

This started happening during the last 3-4 update of x265. I do not now if staxrip has messed up the switches or it's a new bug in the encoder

[Atak_Snajpera]

a new VP9 encoder called Eve is out, it claims to be much better than x264, no info about how it compares to x265, thought you guys would be interested: https://blogs.gnome.org/rbultje/2016/05/02/the-worlds-best-vp9-encoder-eve-2/

How many times I heard that some codec was suppose to be better than x264 ;) Even x265 in some situations looks worse than x264.
Most users here already know than PSNR and SSIM values mean nothing for perceived quality.

[mastrboy]

How many times I heard that some codec was suppose to be better than x264 ;) Even x265 in some situations looks worse than x264.
Most users here already know than PSNR and SSIM values mean nothing for perceived quality.

Have to agree here, x265 slightly beats x264 on low bitrates, but x264 is still the champ in medium/high bitrate encodes.
All other codecs seems visually soft/blurred compared to x264.

[LigH]

Well, you might have heard of the 100th birthday of Claude Shannon. His work is not completely unrelated, even though people may argue that it is more related to lossless compression. The concept of an entropy as threshold of information may be rather soft than hard in case of lossy compression. Still, you will have to sacrifice details for a convenient remaining quality, and even though you can optimize the choice which kind of loss is how annoying, there will always be objective loss, and there will always be different people with different opinions how annoying this loss feels to each of them.

Brief: Don't expect miracles about a matter of taste.

[Magik Mark]

x265 1.9+150-00ea3784bd36 (https://www.mediafire.com/download/5td3190cir9dbbe/x265_1.9+150-00ea3784bd36.7z): Unicode filenames support for output and stats; and some rate control improvements



I could confirm that encoding fps is a lot stable

I could further confirm that at least on my part that, --rc grain causes ghosting sporadically. Turning this off fixes it

[nandaku2]

I could confirm that encoding fps is a lot stable

I could further confirm that at least on my part that, --rc grain causes ghosting sporadically. Turning this off fixes it

We just found a bug with tune grain and 2-pass. Fixing.

[Magik Mark]

We just found a bug with tune grain and 2-pass. Fixing.



Thanks a lot!

[Motenai Yoda]

If the chroma qp offsets are decreased a bit when psy-rd and psy-rdoq are used, as x264 do, will be nice

[littlepox]

If the chroma qp offsets are decreased a bit when psy-rd and psy-rdoq are used, as x264 do, will be nice

Agreed. We've seen some cases for anime, where the colored edges are sharp, and x265 produce some chroma edge ringing/haloing.

Setting the two offsets to be ~-2 alleviates the problem.

[gamebox]

Some unrelated questions :)

- is some sort of Trellis quantisation step planned for x265? Is it missing for the time being only because it is not (yet) coded?

- can someone explain to me what --sao-non-deblock does? The encode I'm doing right now using this switch seems sharper than previous ones, but I changed other options too (AQ/RD strength). Sharpness I see also comes with more ringing/blocking noise, primarily around strong (high contrast) edges - it is (visually) much closer to x264, so SAO seems suppressed. Deblock is set to -1 -1, both for this encode and the one I did before without this switch

- does --no-strong-intra-smoothing work only on 64px CTUs - I enabled it for an encode limited to 32px CTUs, but it seems ignored by encoder

Thanks in advance, and keep up the great work :)

[eclipse98]

Hi Everybody,

I am getting some bad "color bleeding", that's what I think it is but I think a more accurate description is "color trailing". It gets pretty bad and mostly occurring when the object that leaves the color trail (car, road sign) passes in front of bare trees or dry grass.

Original footage is UHD, Sony AX53 Handycam, XAVC 100mbps shot from moving car (camera mounted outside) - resized to HD and encoded with x265 5mbps very slow preset (StaxRip, HEVC encoder version 1.9+150-00ea3784bd36). I tried different presets (medium-very slow) and problem occurs in each of them. It seems a bit better with SSIM tune, but still clearly present.

Attached is zoomed in image where a trail of blue/purple spots left by passing car can be clearly seen:
https://drive.google.com/open?id=0B8kqMOSsi4R-SElnSzBBNkV2NXc

Tested with h264 and it does not have the same problem.

Link to shared folder that has screenshot from original footage (no spots), h264 video, x265 video and zoomed in x265 version:
https://drive.google.com/folderview?id=0B8kqMOSsi4R-ZFFGQ0xFdkp1VGs&usp=sharing

Would appreciate your help if there is any setting I can try that might help with getting rid of this issue.

Thanks for your help, Cheers !

[Jamaika]

Hi elipse98
Maybe I shouldn't write, but also once had a problem with color XAVC.
Which did you use the decoder for codec H264?

Depending NLE editor may be different shades of decoding.
http://filmowiec.pl/forum/viewtopic.php?f=12&t=25255&start=195
Video Bitrate 5Mbps far too small for the dynamic films FullHD.

It is worth remembering about parameters XAVC.
Color primaries : BT.709
Transfer characteristics : IEC 61966-2-4
Matrix coefficients : BT.709

[eclipse98]

Hi elipse98
Maybe I shouldn't write, but also once had a problem with color XAVC.
Which did you use the decoder for codec H264?

Depending NLE editor may be different shades of decoding.
http://filmowiec.pl/forum/viewtopic.php?f=12&t=25255&start=195
Video Bitrate 5Mbps far too small for the dynamic films FullHD.

It is worth remembering about parameters XAVC.
Color primaries : BT.709
Transfer characteristics : IEC 61966-2-4
Matrix coefficients : BT.709

Hi Jamaika,

I am not using any NLE editor, pretty much loading source XAVC video directly with StaxRip and choosing Automatic AviSynth+ filter, it picks LSMASHVideoSource, can this be an issue ? Do you think if I convert XAVC with some intermediate codec (Avid) it might help ?

For 264 I am using x264 r2692 8-Bit x64 'very slow' profile, 2 pass encoding using StaxRip (same LSMASHVideoSource filter, same 5mbps bit rate).

Yes, 5mbps is on low side, but this is a requirement for streaming, so not much choice here, it is accepted that I am not shooting for BD quality.

I would not necessarily call it far too small either, the reason I am using x265 is because it excels at low bit rate compared to x264. I made some incredible x265 encodes at 5mbps (same type of dynamic footage) - overall very happy until this problem appeared.

I uploaded a very short video to shared folder where you can see trailing color problem - I am not sure if it's a color issue - as you can see in the video the bushes in front of the car do not exhibit any artifacts, but you can clearly see them behind the car :(

Thanks for your help !

[fauxreaper]

Hi Everybody,

I am getting some bad "color bleeding", that's what I think it is but I think a more accurate description is "color trailing". It gets pretty bad and mostly occurring when the object that leaves the color trail (car, road sign) passes in front of bare trees or dry grass.

Original footage is UHD, Sony AX53 Handycam, XAVC 100mbps shot from moving car (camera mounted outside) - resized to HD and encoded with x265 5mbps very slow preset (StaxRip, HEVC encoder version 1.9+150-00ea3784bd36). I tried different presets (medium-very slow) and problem occurs in each of them. It seems a bit better with SSIM tune, but still clearly present.

Attached is zoomed in image where a trail of blue/purple spots left by passing car can be clearly seen:
https://lh5.googleusercontent.com/RwLLOJuFPWELc0-YsswBN0MCpkDyTf4vnulzVCNoLkqlYMzmvJsOzCKSs_hSSYOAHYTeMw=w1896-h873

Tested with h264 and it does not have the same problem.

Link to shared folder that has screenshot from original footage (no spots), h264 video, x265 video and zoomed in x265 version:
https://drive.google.com/folderview?id=0B8kqMOSsi4R-ZFFGQ0xFdkp1VGs&usp=sharing

Would appreciate your help if there is any setting I can try that might help with getting rid of this issue.

Thanks for your help, Cheers !

Can you test a x265 encode with "--no-cutree" option? Maybe these spots are just ghosting.

[eclipse98]

Can you test a x265 encode with "--no-cutree" option? Maybe these spots are just ghosting.

Fauxreaper,

"--no-cutree" made it much worse, more spots and now they are glowing with every color of the rainbow :(

Here are encode settings:

x265 [info]: HEVC encoder version 1.9+150-00ea3784bd36
x265 [info]: build info [Windows][GCC 5.3.0][64 bit] 8bit+10bit+12bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX LZCNT
x265 [info]: Main profile, Level-4 (Main tier)
x265 [info]: Thread pool created using 8 threads
x265 [info]: frame threads / pool features : 3 / wpp(17 rows)
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge : star / 57 / 3 / 3
x265 [info]: Keyframe min / max / scenecut : 25 / 250 / 40
x265 [info]: Lookahead / bframes / badapt : 25 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 0
x265 [info]: References / ref-limit cu / depth : 4 / on / on
x265 [info]: AQ: mode / str / qg-size / cu-tree : 1 / 1.0 / 32 / 0
x265 [info]: Rate Control / qCompress : ABR-5336 kbps / 0.60
x265 [info]: tools: rect limit-modes rd=4 psy-rd=2.00 rdoq=2 psy-rdoq=1.00
x265 [info]: tools: signhide tmvp strong-intra-smoothing lslices=4 deblock sao
x265 [info]: tools: stats-read

[Ma]

"--no-cutree" made it much worse

You can try '--cbqpoffs -2 --crqpoffs -2'.

[eclipse98]

You can try '--cbqpoffs -2 --crqpoffs -2'.

Ma, slightly better, maybe 20% or so, but still persists. I tried '--cbqpoffs -4 --crqpoffs -4' and it returned back to original. You think experimenting with different values might yield some improvements ?

Thanks !

[Ma]

Ma, slightly better, maybe 20% or so, but still persists. I tried '--cbqpoffs -4 --crqpoffs -4' and it returned back to original. You think experimenting with different values might yield some improvements ?

You can try -1 or -3 values. You can also try '--aq-mode 3' option.

[Jamaika]

Hi Jamaika,

I uploaded a very short video to shared folder where you can see trailing color problem - I am not sure if it's a color issue - as you can see in the video the bushes in front of the car do not exhibit any artifacts, but you can clearly see them behind the car :(
Add the original.

PS These are the ghosts. On my decoder MPC-BE the spots of color they are in other places.

[Ma]

For volunteers I have x265 build with Unicode support for input files to test (yuv and y4m) -- www.msystem.waw.pl/x265/utf-input.7z (diff file in archive)

Example (from Total Commander):x265.exe "żółć-liść-będąc-23甜蜜的一天(玛丽亚.凯丽&男人男孩).y4m" -o o.hevc

[eclipse98]

Add the original.

PS These are the ghosts. On my decoder MPC-BE the spots of color they are in other places.

Screenshots were taken from different encode, that is probably why spots of color are in different places.

I uploaded a 90 sec segment of original video (original.mp4, XAVC, 100mbps) as well as new encode from this segment (x265.mp4, slow preset, 3 pass, 5mbps). You can see color ghosts left by car at 0:13, 0:18, 0:35, 0:44, 0:57. By road signs at 01:20 (on left), 1:34 and 1:44 (on the right). None of those artifacts exist in original video.

https://drive.google.com/folderview?id=0B8kqMOSsi4R-ZFFGQ0xFdkp1VGs&usp=sharing

So, is it possible to get rid of them ?

Thanks !

btw. Sorry to hijack the thread, I thought it was relevant to x265 as a possible issue. If you think otherwise please let me know and I'll start a new thread specific to this issue.

[Motenai Yoda]

try raising qcomp to .75~.8 and cbqpoffs/crqpoffs to -2

[eclipse98]

try raising qcomp to .75~.8 and cbqpoffs/crqpoffs to -2

--bitrate 5000 --preset slow --qcomp 0.75 --cbqpoffs -2 --crqpoffs -2

Doesn't seem to make any difference.

You can try -1 or -3 values. You can also try '--aq-mode 3' option.

Tried different values and aq-mode 3, didn't work either :(

[Jamaika]

The problems are with the SAO.

My attempt to film processing:
https://www.sendspace.com/filegroup/W3SSQSOzcOkRC8NxqCCdQ7wn9gLOXuaRRNvFBAzseMl5hp8ABy05nGTfmeml27dD1IzOnJ%2BKKrwlMCB35dd9o%2FHCHZGW5y0sHsm9MM%2BGPqwyShIA369DbjYPMLX4LFOt

[eclipse98]

The problems are with the SAO.

My attempt to film processing:
https://www.sendspace.com/filegroup/W3SSQSOzcOkRC8NxqCCdQ7wn9gLOXuaRRNvFBAzseMl5hp8ABy05nGTfmeml27dD1IzOnJ%2BKKrwlMCB35dd9o%2FHCHZGW5y0sHsm9MM%2BGPqwyShIA369DbjYPMLX4LFOt

Jamaika,

Thank you so much for testing this, appreciate it. I am looking at your settings and will replicate them in my tests. In addition to no-sao, I see other settings that differ from mine - you also use a different scaler (Lanczos vs Bicubic), not sure if it makes any difference.

I'll update once I have some test results !

Cheers !

[x265_Project]

Some patches were just committed that address some of the issues reported here... especially with respect to tune grain. We look forward to your feedback.

[ndkamal]

I made some tests with x265 V19 R165 and tune grain, and the result is really impressive. This version retains more grain than x264 (at equivalent bitrate). Very good job. I hope there will be a significant progress in normal mode (without tune grain), in the next months.

[Magik Mark]

I made some tests with x265 V19 R165 and tune grain, and the result is really impressive. This version retains more grain than x264 (at equivalent bitrate). Very good job. I hope there will be a significant progress in normal mode (without tune grain), in the next months.



Can you post a before & after image? Thanks

[nandaku2]

Guys,

Can you help me with my ghost problem. Maybe a missing switch? Doing x265 10bit 2pass via staxrip

http://s32.postimg.org/nr161hk4h/Capture.jpg (http://postimg.org/image/nr161hk4h/)

Hi, is this clip a grainy one (heavy film grain, ie)? If so, you may want to check with the new tune grain setting.

[Jamaika]

The problem hasn't been solved.
To test, I used material from the drone. Cloudy sky, the sun's rays reflecting in the water. At some point drone changes its position and is another angle of sunlight. And here curiosity. Where are the frames B all is well. Next frame "I" and there are violets.
Excision of extra functions give nothing "--no-amp --no-temporal-mvp --no-signhide --no-strong-intra-smoothing --no-b-intros". The use of "open-gop" also doesn't give results.
As for the focus. It seems to me that better falls x264. There is no pulsation between sharp and blurred places between frames.

PS I realize that we are talking about cheap cameras with poor optics.

Edit: It isn't satisfied with the settings tune grain.
https://github.com/videolan/x265/commit/69bb8ea79f7db1ed6c35216ad399b95f2e37d46f
First, I have problems with:
*: Option: `--aq-mode` 0
*: Option: `--cutree` 0
*: Option: `--qpstep` 1
Here there are large pixels.
I don't know what's going on with recursion-skip` 1. The inclusion doesn't improve quality. For me it is worse.
Settings:
*: Option: `--sao` 0
*: Option: `--psy-rd` 4.0
*: Option: `--psy-rdoq` 10.0
They are better than mine --psy-rd` 2.0 --psy-rdoq` 50.0
Use of me = umh causes more blemishes. Therefore, the above badly written.
The advertised the function 'lanczos' is bullshit. How would I not configured and so is the 'bicubic'.
My best settings for version 1.9 + 167 (veryslow):
--psy-rd 4.00 --psy-rdoq 10.00 --no-sao --high-tier --no-recursion-skip --keyint 60 --bitrate 6000 --vbv-bufsize 10000 --vbv-maxrate 10000
PS These settings in version 1.9 + 168 already look worse for the same film.
https://www.sendspace.com/file/inmvs6

[LigH]

x265 1.9+167-bebae72f9db7 (http://www.mediafire.com/download/6i1cizeppw4191t/x265_1.9+167-bebae72f9db7.7z) provides a few changes regarding the grain handling, and a new tweak:

--[no-]recursion-skip Enable early exit from recursion. Default enabled

[benwaggoner]

x265 1.9+167-bebae72f9db7 (http://www.mediafire.com/download/6i1cizeppw4191t/x265_1.9+167-bebae72f9db7.7z) provides a few changes regarding the grain handling, and a new tweak:

--[no-]recursion-skip Enable early exit from recursion. Default enabled
I think these are some actually pretty huge changes in regards to grain. It would be a great build for everyone struggling with detail and grain issues to retest with, particularly but not exclusively, with --tune grain.

MultiCoreWare could presumably use a lot of feedback on the results here.

[x265_Project]

MultiCoreWare could presumably use a lot of feedback on the results here.

Yes, please.

[fauxreaper]

I've tested --no-recursion-skip on a short sample that was posted on devel list (https://mailman.videolan.org/pipermail/x265-devel/2016-May/010350.html) and the results are good.


Reduced ghosting (but ghosting still exists)
Increased visual quality
Reduced bitrate (1/3 less bitrate on same crf)


Encode gets 50% slower.

[littlepox]

Encode gets 50% slower.

ROFT

But at least they made the breakthrough, good job.

[ndkamal]

Can you post a before & after image? Thanks

I have posted some pictures in "Comparisons of x264 vs x265 " section forum

[pingfr]

@littlepox: Would you say it is safe and a quality benefit/visual benefit to use your custom parameters you cooked us since 1.9+1 with the newest --no-recursion-skip feature/tweak they added in 1.9+167?

Something along the lines of:

--preset veryslow --ctu 32 --max-tu-size 16 --crf 18 --no-recursion-skip --tu-intra-depth 2 --tu-inter-depth 2 --rdpenalty 2 --me 3 --subme 5 --merange 44 --b-intra --no-rect --no-amp --ref 5 --weightb --keyint 360 --min-keyint 1 --bframes 8 --aq-mode 1 --aq-strength 1.0 --rd 5 --psy-rd 1.6 --psy-rdoq 8.0 --rdoq-level 1 --no-sao --no-open-gop --rc-lookahead 80 --max-merge 4 --qcomp 0.80 --no-strong-intra-smoothing --deblock -2:-2 --qg-size 16 --pbratio 1.2

:)

Edit: Typo, I meant 1.9+167 not 1.9+197. :p

[littlepox]

@littlepox: Would you say it is safe and a quality benefit/visual benefit to use your custom parameters you cooked us since 1.9+1 with the newest --no-recursion-skip feature/tweak they added in 1.9+197?

Something along the lines of:

--preset veryslow --ctu 32 --max-tu-size 16 --crf 18 --no-recursion-skip --tu-intra-depth 2 --tu-inter-depth 2 --rdpenalty 2 --me 3 --subme 5 --merange 44 --b-intra --no-rect --no-amp --ref 5 --weightb --keyint 360 --min-keyint 1 --bframes 8 --aq-mode 1 --aq-strength 1.0 --rd 5 --psy-rd 1.6 --psy-rdoq 8.0 --rdoq-level 1 --no-sao --no-open-gop --rc-lookahead 80 --max-merge 4 --qcomp 0.80 --no-strong-intra-smoothing --deblock -2:-2 --qg-size 16 --pbratio 1.2

:)

I would say it's unsafe for any version newer than 1.9+10

[pingfr]

@littlepox: Alrighty, that makes sense. Let's wait on newer visually optimzed 1080p and 720p custom profiles from you for 1.9+167++ then. :p

Edit: Typo, I meant 1.9+167 not 1.9+197. :p

[pistacho]

Bug report:

x265 builds 1.9.150 and newer crashes if filenames have accentuated characters on Spanish Windows (CMD with codepage 850 Multilingual Latin I). Probably the problem occurs with other languages as well.

Example:

With build 1.9.169 crashes:

http://115.imagebam.com/download/TWUxm1XWC0LbQkJlBmNTDw/48371/483702433/169.png

Same command line with build 1.9.149 works fine:

http://116.imagebam.com/download/V08y8a6Z5JNu0vli0-upvA/48371/483702432/149.png

Problem exist from this commit: https://bitbucket.org/multicoreware/x265/commits/00ea3784bd36c164c5f799c998d7a09f2cb244bf

Thanks!

[Ma]

x265 builds 1.9.150 and newer crashes if filenames have accentuated characters on Spanish Windows

'--qpfile' is not converted to Unicode yet. Fix should be soon. Thanks for report.
-----------------
Patch file for '--qpfile' and '--csv' attached.
-----------------
There are 2 problems with '--qpfile' -- now it doesn't support Unicode filenames and if you specify nonexistent file GCC build of x265 hangs, VS 2015 build of x265 exits with wired error messages.

So in patch 13370 (https://patches.videolan.org/patch/13370/) Unicode filenames are working with '--qpfile' and '--csv' options and if you specify nonexistent file in '--qpfile' option, x265 displays error message and encode without hangs.

For tests I've compiled multilib version of x265 1.9+169 + 13370 (https://patches.videolan.org/patch/13370/) + 13312 (https://patches.videolan.org/patch/13312/) patches -- www.msystem.waw.pl/x265/x265-1.9+169+input+qpfile+csv.7z

[pistacho]

I confirm that now works again with accented characters. Also if --qpfile file name not exists error message is displayed but continues encoding (not hangs anymore).

Thanks!

[Grojm]

I've tested --no-recursion-skip on a short sample that was posted on devel list (https://mailman.videolan.org/pipermail/x265-devel/2016-May/010350.html) and the results are good.


Reduced ghosting (but ghosting still exists)
Increased visual quality
Reduced bitrate (1/3 less bitrate on same crf)


Encode gets 50% slower.

I have compared the results from the mailing list and the no-recursion-skip version is only slightly better, the issue is far from solved. Also, the no-recursion-skip version was 10% larger, and encoding time goes up. Instead of finding workaround with only minor improvement but at the cost of major slowdown, they should focus on finding the root of this issue. x264 is completely free from this artifact, so ghosting is by no means a tradeoff you have to make for motion based video compression.

[littlepox]

As clear as I remember, x264 used to have all the same blames: slow, blurry, banding/blocking, motion artifacts... It took years to mature, not over a single night.

It shall be same for x265, which is mostly a brand new encoder whose internal logic has changed almost completely. Do not expect miracles out of it to be an immediate, all-round upgrade for x264.

I'm always emphasizing that currently we prefer x264 over x265, NOT because x265 is poor, but x264 is just incredibly good. Do you blame Korean/Japanese/Singaporean table tennis players for their losing to Chinese in competitions? If you ever compare x265 to all other encoders like Adobe H264, Quicktime, libvpx... You'll realize that it's far outstanding.

Thanks again for all the great work of the developers; keep it up.

[benwaggoner]

I have compared the results from the mailing list and the no-recursion-skip version is only slightly better, the issue is far from solved. Also, the no-recursion-skip version was 10% larger, and encoding time goes up. Instead of finding workaround with only minor improvement but the cost of mayor slowdown, they should focus on finding the root of this issue. x264 is completely free from this artifact, so ghosting is by no means a tradeoff you have to make for motion based video compression.
Don't assume that the very first check-in of a new feature includes full optimization or full quality tuning. x265 has gotten a lot better and a lot faster over the last 18 months!

For something as far along as x265, having any single new feature that offers an obvious visual quality improvement is a huge accomplishment. x264 hasn't had one of those since, sheesh, mbtree? Most codec development is about squeezing out a bunch of discreet <<1% compression efficiency gains.

[littlepox]

Don't assume that the very first check-in of a new feature includes full optimization or full quality tuning. x265 has gotten a lot better and a lot faster over the last 18 months!

For something as far along as x265, having any single new feature that offers an obvious visual quality improvement is a huge accomplishment. x264 hasn't had one of those since, sheesh, mbtree? Most codec development is about squeezing out a bunch of discreet <<1% compression efficiency gains.

Hmm...recently x264 has added aq-mode=3 into the vanilla build which gives some significant improvements. However that had been in unofficial builds (like tMod or kmod) for years.

[mandarinka]

Hmm...recently x264 has added aq-mode=3 into the vanilla build which gives some significant improvements. However that had been in unofficial builds (like tMod or kmod) for years.

Whenever I tried that, I wasn't able to find the proper strength setting (that would look better than mode 1 strength 0.8 which I use for cel anime). What strengths are you using?
(Sorry for off-topic.)

[littlepox]

Whenever I tried that, I wasn't able to find the proper strength setting (that would look better than mode 1 strength 0.8 which I use for cel anime). What strengths are you using?
(Sorry for off-topic.)

I'd often pick aq=3:0.8 for high quality jap anime BDRip for crf≈16, 10bit x264.

Not so sure for your case; it's possible aq=1 is still relatively better. try strength=0.7?

[jlpsvk]

I've tested --no-recursion-skip on a short sample that was posted on devel list (https://mailman.videolan.org/pipermail/x265-devel/2016-May/010350.html) and the results are good.


Reduced ghosting (but ghosting still exists)
Increased visual quality
Reduced bitrate (1/3 less bitrate on same crf)


Encode gets 50% slower.


points 1,2 and 4 are correct. but don't know where you get 1/3 bitrate saving. encoded 2 movies, and have only about 130kbps save on the same settings and crf.

[RainyDog]

Hmm...recently x264 has added aq-mode=3 into the vanilla build which gives some significant improvements. However that had been in unofficial builds (like tMod or kmod) for years.

I thought you'd said that aq-mode 3 was a waste of bits in your x265 tune-film thread (http://forum.doom9.org/showthread.php?t=172458) littlepox? Or is it better suited to x264?

[littlepox]

I thought you'd said that aq-mode 3 was a waste of bits in your x265 tune-film thread (http://forum.doom9.org/showthread.php?t=172458) littlepox? Or is it better suited to x264?

Why would you even link them together?
The truth is aq3 is well suited in x264 but ill suited in x265.

[fauxreaper]

points 1,2 and 4 are correct. but don't know where you get 1/3 bitrate saving. encoded 2 movies, and have only about 130kbps save on the same settings and crf.

I used --bframes 8 on that sample. I encoded some bigger samples of anime videos (10-15 minute length) and bitrate reduction with --no-recursion-skip is smaller.

[EncodedMango]

Hello, say I want to encode a video without having to create a raw YUV/Y4M file first, how do I pipe it using only ffmpeg without the x265.exe?
ffmpeg -i "INPUT.avi" -c:v libx265 -preset medium -x265-params crf=20 -pix_fmt yuv420p -c:a libmp3lame -q:a 3 "OUTPUT.mp4"

Assuming YUV4MPEGPIPE is used, could someone tell me how using the above command? Thanks.
(Parameters might be incorrect, it's just to give an example)

[sneaker_ger]

I don't understand. You want to use a pipe but not use x265.exe? Then what do you need the pipe for?
You can either:
a.) use a pipe to send ffmpeg.exe y4m output to x265.exe or
b.) use integrated libx265 of ffmpeg.exe. (Like in your example)

[mandarinka]

x265 1.9+167-bebae72f9db7 (http://www.mediafire.com/download/6i1cizeppw4191t/x265_1.9+167-bebae72f9db7.7z) provides a few changes regarding the grain handling, and a new tweak:

--[no-]recursion-skip Enable early exit from recursion. Default enabled

Is --no-recursion-skip changing how analysis is done at slow settings (veryslow, placebo)?

At first I thought that --recursion-skip is the new functionality and --no-recursion-skip just disables it (and since it is auto set to disabled on RD 6, there would be no change in behaviour from prior state on slowest setting). But I am not so sure now. Does --no-recursion-skip change output compared to builds before the whole feature was introduced?

[fauxreaper]

Is --no-recursion-skip changing how analysis is done at slow settings (veryslow, placebo)?

At first I thought that --recursion-skip is the new functionality and --no-recursion-skip just disables it (and since it is auto set to disabled on RD 6, there would be no change in behaviour from prior state on slowest setting). But I am not so sure now. Does --no-recursion-skip change output compared to builds before the whole feature was introduced?

--no-recursion-skip does more analysis, its a new functionality. --recursion-skip is how analysis worked before.

[mandarinka]

Thanks, I guess I'll have to go back to that painful testing, then.

Damn, I just finished two weeks of encoding.

[Jamaika]

Colleagues clarify to me
How is it the function of the limit-ref?
In my "veryslow" is disabled. In the latest table is one.
https://github.com/videolan/x265/commit/3545d4c0a6e1a9ee3af5a714e99317f7b39bf6cf

[foxyshadis]

Colleagues clarify to me
How is it the function of the limit-ref?
In my "veryslow" is disabled. In the latest table is one.
https://github.com/videolan/x265/commit/3545d4c0a6e1a9ee3af5a714e99317f7b39bf6cf

Then you should update your x265, it's been 1 for veryslow since December.

[EncodedMango]

I don't understand. You want to use a pipe but not use x265.exe? Then what do you need the pipe for?
You can either:
a.) use a pipe to send ffmpeg.exe y4m output to x265.exe or
b.) use integrated libx265 of ffmpeg.exe. (Like in your example)
Okay I might have mixed something up, earlier(many many months ago) when I tried using x265 from ffmpeg, it never worked because it wasn't accepting anything other than Y4M/YUV(?).

It seems to be working directly now. I might have confused x265.exe and ffmpeg here. My mistake sorry.

[Magik Mark]

Build 170 --> 2 pass 10bit x265 "tune none" preset medium

First pass --> 5fps slower
Second pass --> same


Sent from my iPhone using Tapatalk

[x265_Project]

I hope all of you have been doing some encodes with the latest development build of x265. The changes made one week ago were fairly important. We're interested in your feedback on the visual quality of x265 encodes at all presets, compared with earlier builds. --preset veryslow and placebo (with --no-skip-recursion) will show the biggest change, but all presets were affected.

Personally, I'm seeing the best encoding quality I've ever seen from x265, particularly on very challenging content (water, grainy content). I'm also noticing that --preset placebo will now show a meaningful difference from --preset veryslow.

[Magik Mark]

May we ask where to find the default switches of all presets so we could update our GUI. Thanks a lot


Sent from my iPhone using Tapatalk

[x265_Project]

May we ask where to find the default switches of all presets so we could update our GUI. Thanks a lot


http://x265.readthedocs.io/en/default/presets.html

or in the x265 source...
x265 / doc / reST / presets.rst

[Jamaika]

For veryslow is a big improvement. (: However, the figures are even shades of purple.
Codec 1.9+170 (GCC 6.1.1) worked faster because he has now turned 'limit-ref = 1'. However, this message doesn't show.
x265 [info]: References / ref-limit cu / depth : 5 / off / on
As for the 'tune grain'. I understand that this is an option only for the animation, pass = 2,3 and crf. I will not be spoken.

[Ma]

Build 170 --> 2 pass 10bit x265 "tune none" preset medium

First pass --> 5fps slower
Second pass --> same

Could you specify 5fps slower than what version? Could you specify command lines to reproduce?

[tObber166]

1.9+144 vs 1.9+169, I've noticed a slowdown about 0.9 fps on a 1080p clip

preset medium --crf 17.0 --ref 5 --vbv-bufsize 20000 --vbv-maxrate 20000 --rc-lookahead=25 --me 3 --psy-rd 1.0 --no-sao

[x265_Project]

1.9+144 vs 1.9+169, I've noticed a slowdown about 0.9 fps on a 1080p clip

preset medium --crf 17.0 --ref 5 --vbv-bufsize 20000 --vbv-maxrate 20000 --rc-lookahead=25 --me 3 --psy-rd 1.0 --no-sao

Absolute numbers are not very helpful. Can you describe the effect on performance in relative terms?

Did you notice a meaningful improvement in visual quality?

[tObber166]

Absolute numbers are not very helpful. Can you describe the effect on performance in relative terms?

Did you notice a meaningful improvement in visual quality?


-Quality wise, I have not noticed any difference.

-Average bitrate is pretty much the same on both encodes.

-Final size pretty much the same. 10kb smaller on +169


pic: 144 vs 169

http://s32.postimg.org/ss435edoh/144_vs_169.jpg (http://postimg.org/image/ss435edoh/)

[LigH]

Still, is it 0.9 fps slower than 20 fps before, or 0.9 fps slower than 2 fps before?

OK, 12.57 : 13.50 ~ 93.1%

[Motenai Yoda]

it's 12.97 vs 12.06

[Ma]

1.9+144 vs 1.9+169, I've noticed a slowdown about 0.9 fps on a 1080p clip

preset medium --crf 17.0 --ref 5 --vbv-bufsize 20000 --vbv-maxrate 20000 --rc-lookahead=25 --me 3 --psy-rd 1.0 --no-sao

I can confirm the slow-down. I compared ver. 1.9+144 vs 1.9+170 vs 1.9+183 with your options. Each version produces different output file.

In VS 2015, AVX-CPU, clean builds line (x265vsNNNac) I have relative encoding times:
1.9+144 | 100.0%
1.9+170 | 105.7%
1.9+183 | 107.2%

In GCC 6 line (x265gcNNN) I have relative encoding times:
1.9+144 | 100.0%
1.9+170 | 105.7%
1.9+183 | 107.0%

In 1.9+183 version I have relative (to different builds) encoding times:
100.0% | x265vs183ac- VS 2015, AVX, clean
099.9% | x265vs183a - VS 2015, AVX
100.8% | x265vs183 -- VS 2015
101.6% | x265gc183a - GCC 6.1.1, AVX
102.3% | x265gc183 -- GCC 6.1.1

Full data in attached screen.txt

[pingfr]

I must ask what is the intent of the changes pushed in commit 1b2b912?

What does this actually improve/implies for the *end-user in terms of decoding ressources/visual quality?

*end-user = not the person doing the encode, rather the person watching said encoded results.

[jlpsvk]

I used --bframes 8 on that sample. I encoded some bigger samples of anime videos (10-15 minute length) and bitrate reduction with --no-recursion-skip is smaller.

I also encoded with --bframe 8 and also with --qpstep 8. And I encoded the whole movies.

[nandaku2]

I must ask what is the intent of the changes pushed in commit 1b2b912?

What does this actually improve/implies for the *end-user in terms of decoding ressources/visual quality?

*end-user = not the person doing the encode, rather the person watching said encoded results.

Hi,
For multi-pass, the best encode/visual quality is observed when the settings are as similar as possible across passes. Earlier, pass 1 had the turbo settings enabled by default, which means the encodes in each pass were substantially different (even though the actual commandlines were similar). This was causing lower quality multi-pass encodes, and needless confusion for users.

[burfadel]

-Quality wise, I have not noticed any difference.

-Average bitrate is pretty much the same on both encodes.

-Final size pretty much the same. 10kb smaller on +169


pic: 144 vs 169

http://s32.postimg.org/ss435edoh/144_vs_169.jpg (http://postimg.org/image/ss435edoh/)

Try the settings below and see if it is any better quality, without losing too much on the encoding time side of things. I've put a comment in italics next to them separated by ///, don't copy those if you copy the settings :).


--output-depth 10 /// your output depth is currently 8, ideally 8 would only be used if the target device only supports 8 and not 10 bit, which is unlikely at that profile level?...
--rd 4 /// as I came across a couple of weeks ago, 4 isn't the same as 3, it's a little more accurate and potentially higher quality output without noticeably affecting encode speed. --rd 5 provides a nicer picture but is much slower. Documentation says 4 is the same as 3 still, it is NOT. If you don't believe it, do identical encodes, one with 3 and the other 4 and compare the states, like the kb/s for p, b and I frames etc. If they are truly identical all stats should be exactly the same, as with the file size output. The only difference between the two files is the stat info where one will be 3 and the other 4, which wouldn't affect file size
--tu-intra-depth 4 /// slower than default 1 obviously, but --tu-intra-depth doesn't affect speed too much unlike --tu-inter-depth
--rdoq-level 1 /// in my opinion this provides a noticeable PQ improvement, almost like using --rd 5 without as much of the speed penalty!
--early-skip /// performance recovery without noticeable PQ detriment
--fast-intra /// performance recovery without noticeable PQ detriment
--b-intra /// improved PQ, normally on profile slower and above
--tskip /// performance recovery without noticeable PQ detriment
--tskip-fast /// performance recovery without noticeable PQ detriment
--limit-modes /// only set for profiles 6, 7, 8 by default, speed improvment...
--qg-size 16 /// quality setting
--me star /// you already use this, default for profile slow and above. Considering its speed I'm surprised it's not used in place of hex as default
--max-merge 3 /// default for slow mode
--weightb /// default for profile slower and above. Interestingly enough the default for x264 is having this on at medium!
--aq-mode 2 /// [I]this is probably a bit of a controversial setting, I feel there is an improvement. Try it and compare. Don't just compare the foreground, compare the background and more flat areas etc


File size will be different. --rdoq-level 1 increases output file size a little, however the quality difference is perceptively greater than what you would fine with using a lower CRF to achieve the same file size output. If you don't like the file size, you can raise your CRF to get closer to the same file size. I feel by doing so, the PQ is still improved. If you don't believe there is any difference, try a higher CRF like 22 and then compare the two different settings.

You may want to try a slightly higher CRF as the file size will be bigger with the use of Variance AQ. Try 17.5 instead of 17, I believe it will sitll produce worthwhile benefit.

[Motenai Yoda]

--rd 4 /// as I came across a couple of weeks ago, 4 isn't the same as 3, it's a little more accurate and potentially higher quality output without noticeably affecting encode speed. --rd 5 provides a nicer picture but is much slower. Documentation says 4 is the same as 3 still, it is NOT. If you don't believe it, do identical encodes, one with 3 and the other 4 and compare the states, like the kb/s for p, b and I frames etc. If they are truly identical all stats should be exactly the same, as with the file size output. The only difference between the two files is the stat info where one will be 3 and the other 4, which wouldn't affect file size
as even the devs wrote 4 and 3 are the same, in the code there isn't any if of conditional about 3 or 4, also the same encode give little differences coz non deterministic things (you can do many times and get differents results with the same settings), all metrics like average quantizer, ssim and psnr are exactly the same.

rd-level 1 give you better quality for bigger size, generally, for the same size, rd-level 2 is better than 1.

[burfadel]

It's a tough call between --rdoq-level 1 and --rdoq-level 2. Level 2 does mean the output is a little smaller, however I do think you lose a little bit of detail compare to 1 when you look at all aspects of the picture.

At level 1 rate-distortion cost is used to find optimal rounding values for each level (and allows psy-rdoq to be effective). It trades-off the signaling cost of the coefficient vs its post-inverse quant distortion from the pre-quant coefficient. When --psy-rdoq is enabled, this formula is biased in favor of more energy in the residual (larger coefficient absolute levels)

At level 2 rate-distortion cost is used to make decimate decisions on each 4x4 coding group, including the cost of signaling the group within the group bitmap. If the total distortion of not signaling the entire coding group is less than the rate cost, the block is decimated. Next, it applies rate-distortion cost analysis to the last non-zero coefficient, which can result in many (or all) of the coding groups being decimated. Psy-rdoq is less effective at preserving energy when RDOQ is at level 2, since it only has influence over the level distortion costs.
http://x265.readthedocs.io/en/latest/cli.html#cmdoption--rdoq-level

Going by that, wouldn't it mean level 2 isn't as good as level 1 in some aspects? I even tried manipulating the CRF (by lowering it slightly, since you can use decimals) to get bascally the exact same output file size. I still believe even by doing so that level 1 retains some detail more effectively. It may be that for flat images like typical animation that level 2 is generally 'better' as you say, but in a high detail, texture clip with some fine noise that I used, level 1 was more effective. I tried several different encodes, and subjectively I felt that level 1 did a nicer job.

[LigH]

x265 1.9+183-4723933fdec9 (https://www.mediafire.com/download/8o1l8spi8i5fcop/x265_1.9+183-4723933fdec9.7z): --slow-firstpass is now enabled as default; quantization is clipped in 2-pass with grain tuning; --qpfile and --csv support Unicode too.

[Jamaika]

I have a question. I use three codecs X265 1.9+183 different authors (WAW/Ligh /Selur) and received three different results in the binary HEX. (Veryslow/tune grain /hightier/pass2)
Is it possible?

[LigH]

A binary comparison of encoded HEVC video streams doesn't make much sense. I believe there is a chance that the output is not deterministic, but the differences may not be relevant for the image reconstruction. And if you have containers around the video stream, it is almost certain that there will be differences in some header areas, which are not at all related to the image reconstruction (e.g. "junk" chunks).

[Jamaika]

Thanks. Now I know what means non deterministic.
I can only say that for 220 frames is the difference 23kb file size.(6000kbps,25fps)

[Ma]

I use three codecs X265 1.9+183 different authors (WAW/Ligh /Selur) and received three different results in the binary HEX. (Veryslow/tune grain /hightier/pass2)
Is it possible?

Yes, it is. Problem is known -- you can read last sentence at page https://bitbucket.org/sborho/test-harness

If you read second sentence in Performance TODOs at page https://bitbucket.org/multicoreware/x265/wiki/TODO it will be the solution.

There is even a patch proposal -- https://patches.videolan.org/patch/13186/

At page www.msystem.waw.pl/x265 for GCC builds I've weakened optimization level (from -O3 to -O2) for file encoder/sao.cpp and all builds from this site should produce bit exact result (with exception for compiler name).

[Jamaika]

At page www.msystem.waw.pl/x265 for GCC builds I've weakened optimization level (from -O3 to -O2) for file encoder/sao.cpp and all builds from this site should produce bit exact result (with exception for compiler name).
SAO is reportedly off for tune grain.:)

[Ma]

SAO is reportedly off for tune grain.:)

I don't use '--tune grain', so maybe my builds produces the same output only with "my" settings. Could you specify your command line? I will check if there is something else to do for bit exact output.

[Jamaika]

ffmpeg.exe -loglevel verbose -i "orginal.MP4" -r 25000/1000 -ss 14.320 -vf scale=1920x1080:flags=lanczos+accurate_rnd,format=yuv420p -an -sn -f yuv4mpegpipe - |
x265_ml.exe --y4m --info --limit-modes --tune grain --preset veryslow --high-tier --no-open-gop --no-hrd --input-depth 8 --input-res 1920x1080 --input-csp i420
--output-depth 8 --fps 25000/1000 --keyint 60 --bitrate 6000 --vbv-bufsize 10000 --vbv-maxrate 10000 --pass 2 --stats x265_2pass_1.log --colormatrix bt709 --colorprim bt709 --transfer bt709 --output "film1.h265" -

[Ma]

Thanks! I confirm that VS 2015, GCC 6.1 and GCC 6.1 32-bit builds produce different outputs with your settings. So I'm starting investigation...

[nandaku2]

Thanks! I confirm that VS 2015, GCC 6.1 and GCC 6.1 32-bit builds produce different outputs with your settings. So I'm starting investigation...

Any VBV encode is non-deterministic (same binary, same commandline run twice on the same machine) will produce slightly different results, due to non-deterministic thread synchronization of wpp rows.

That apart, different machines will produce different results (different #threads and sync points), different compilers will produce different results (due to differences in FP operations). Apart from these, our automated tests track all output changing commits in the test-harness repo.

[Jamaika]

Any VBV encode is non-deterministic (same binary, same commandline run twice on the same machine) will produce slightly different results, due to non-deterministic thread synchronization of wpp rows.
You're right. VBV can be turned off and the file size is deterministic. However, I am surprised that the error tolerance file size for low VBV is ±23kb.
That apart, different machines will produce different results (different #threads and sync points), different compilers will produce different results (due to differences in FP operations). Apart from these, our automated tests track all output changing commits in the test-harness repo.
I understand that this means the old processors with threads=4 throw out. HEVC is too large differences in results. I think it is impossible to compare films eg. in amounts streaks or grains.

[nandaku2]

You're right. VBV can be turned off and the file size is deterministic. However, I am surprised that the error tolerance file size for low VBV is ±23kb.

The file size as a metric is meaningless. The percentage change in bitrate is designed to be really low.

I understand that this means the old processors with threads=4 throw out. HEVC is too large differences in results.

What are you comparing against?

[Jamaika]

The percentage change in bitrate is designed to be really low.
Indeed, you're right. It can be the difference in file size, but the bitrate allocation is the same.;)

Insert two pictures.
http://i63.tinypic.com/t7fo90.png
http://i67.tinypic.com/2dsh1lc.png

PS The percentage change in bitrate one frame I is 0%.
The percentage change in bitrate one frame P is 0.59%.(x47)
The percentage change in bitrate one frame B is 0.26%.(x168)

You're right. VBV can be turned off and the file size is deterministic. However, I am surprised that the error tolerance file size for low VBV is ±23kb.
Sorry. I add bufsize 10000kbps but codec change parameter to 33000kbps. My reasoning was from outer space.

PS After many tests, I reach to the conclusion that for vbv=30000kbps codec LigH something wrong counts. The rest accepted.

[Selur]

Got a dual Xeon E5640 system and I wonder what I could do to boost the cpu usage during x265 encoding

atm. I use:
265 --preset slow --pme --input - --output-depth 10 --y4m --profile main10 --no-high-tier --level-idc 4.1 --amp --no-open-gop --weightb --crf 18.00 --psy-rdoq 15.00 --vbv-maxrate 20000 --vbv-bufsize 20000 --range limited --colormatrix bt709 --output "D:\09_02_06_7310_02.265"
and run two encodes in parallel, but cpu usage still is just around 65%. Input and output are inside a RAM disk and decoding also isn't the bottleneck, so the problem is with the x265 settings. (I get ~5fps per encoder instance)

adding different --pools settings just caused the encoding to be even slower.
using different --frame-threads counts doesn't seem to help either
using --pmode additionally to --pme doesn't help
using msvc instead of mingw builds doesn't help (version I use is 1.9+183)

-> Is there some recommendations on what to do, to utilize the CPU on multi socket systems better?

Cu Selur

Ps.: I know that the system isn't new and only MMX2 SSE2Fast SSSE3 SSE4.2 optimization is used, but it's really bugging me that the only way to get a proper cpu usage and cumulative speed is to run 3 or more encodes at a time,.. (nowadays I only encode one file a week or so ;))

[MeteorRain]

Got a dual Xeon E5640 system and I wonder what I could do to boost the cpu usage during x265 encoding

atm. I use:
265 --preset slow --pme --input - --output-depth 10 --y4m --profile main10 --no-high-tier --level-idc 4.1 --amp --no-open-gop --weightb --crf 18.00 --psy-rdoq 15.00 --vbv-maxrate 20000 --vbv-bufsize 20000 --range limited --colormatrix bt709 --output "D:\09_02_06_7310_02.265"
and run two encodes in parallel, but cpu usage still is just around 65%. Input and output are inside a RAM disk and decoding also isn't the bottleneck, so the problem is with the x265 settings. (I get ~5fps per encoder instance)

1. What's your source? Is it source related performance issue?
2. Here on i7, it maxes out my 4c8t so easily on -F 1 --pools 6, with pme and pmode off. Doesn't make sense for 65% on parallel encoding.

[Selur]

Source is HD, the source resolution I feed to x265 an SD resolution. Source decoding isn't the issue since I can easily run multiple decoder instances and they all still produce 200fps with no real cpu usage.
On my i7 4770k, cpu usage is maxed out too and encoding speed is way higher, but it's a single socket system and a lot newer. ;)

[LigH]

@MeteorRain: But you don't have any dual-socket mainboard with two separate CPUs? That's the point of interest here.

@ Selur: Frame dimensions will be interesting here. I remember from a console notification that some features are disabled for small frame heights, probably because it wouldn't make sense to divide the frame into such small slices.

[Selur]

@LigH: afaik lookahead-slices should be the only thing that gets disabled with source height < 720p.
Typically the resolution I use is 704x576 or 704x480.
Using ~35% per encode with the given command line simple seems way to low for me. (and no, on a single socket system this issue does not appear, all my single socket system always have 85+% usage for a single encode using the same settings,... )
My single socket systems also use way more cpu optimizations, since the cpus are newer (typically MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI2), so it might be that some CPU features are needed to properly use x265,...

[MeteorRain]

@MeteorRain: But you don't have any dual-socket mainboard with two separate CPUs? That's the point of interest here.
True.

So, @Selur, maybe you can try to disable one socket using CPU affinity, and see how it goes?

Options,
1) --pools +,-
2) Change the cmd.exe affinity to only use first sockets, then from that cmd window, run your command with --pools 8
3) Unplug one CPU and see how it works

[Selur]

@MetorRain: and to avoid some blind testing I thought it might be easier to ask here, since iirc a bunch of other users here use multi socket systems and probably encode a bit more then I do nowadays ;)

---

Only thing that seems to help with the cpu usage, is to use a 720p or higher resolution where the --lookahead-slices are not disabled,.. :(

[pradeeprama]

True.

So, @Selur, maybe you can try to disable one socket using CPU affinity, and see how it goes?

Options,
1) --pools +,-
2) Change the cmd.exe affinity to only use first sockets, then from that cmd window, run your command with --pools 8
3) Unplug one CPU and see how it works

We recently found some issues with the way in which x265 runs on multi-socket windows machines (I'm assuming you're using windows). There is an outstanding patch that should fix this problem - https://patches.videolan.org/patch/13436/. Can you see if this helps you out? You should see better multi-socket utilization.

Pradeep.

[Selur]

Yes, I'm on Windows. -> will try the patch
failed to apply the patch, so would be nice if someone could make a numa enabled build with the patch applied,..

[Ma]

Yes, I'm on Windows. -> will try the patch
failed to apply the patch, so would be nice if someone could make a numa enabled build with the patch applied,..

There is a small bug in this patch. Instead of:
int popCount(uint64_t x)
should be:
static int popCount(uint64_t x)

Compiled multilib version: www.msystem.waw.pl/x265/x265-1.9+183+p13436+static.7z

[Selur]

Thanks Ma!
Using that version cpu usage fluctuates a lot more and when running two encodes in parallel average boost usage to 85%+ :)

[pingfr]

Looks like we're getting closer and closer to a newer milestone release to me once that NUMA patch gets merged in the main tree.

- Presets tinkered a bit further: Done.
- Implementation of the --[no-]recursion-skip and --slow-firstpass features: Done.
- Tune grain improvements: Done.
- Docs cleaned out: Done.
- Multi-pass tweaks: Done.
- Full UNICODE support: Done.
- NUMA systems optimization: Done.

Now all we need is to get rid of that nasty "purple ghosting" effect users have reported in the previous pages and it sounds like we have a good candidate for a x265-2.0 release. ;)

That's just an assumption from my end, nothing else. It would just seem silly to bump the 1.9 version further forever "a la" x265-1.9+64829173.

Time to let go 1.9 and shoot for x265-2.0 boys! :)

[Leo 69]

The codec got much better with the latest release in terms of visual quality but the encoding speed slowed down drastically. Fully keeping the original film grain and sharpness is still an issue with any settings one can imagine.

[littlepox]

Looks like we're getting closer and closer to a newer milestone release to me once that NUMA patch gets merged in the main tree.

- Presets tinkered a bit further: Done.
- Implementation of the --[no-]recursion-skip and --slow-firstpass features: Done.
- Tune grain improvements: Done.
- Docs cleaned out: Done.
- Multi-pass tweaks: Done.
- Full UNICODE support: Done.
- NUMA systems optimization: Done.

Now all we need is to get rid of that nasty "purple ghosting" effect users have reported in the previous pages and it sounds like we have a good candidate for a x265-2.0 release. ;)

That's just an assumption from my end, nothing else. It would just seem silly to bump the 1.9 version further forever "a la" x265-1.9+64829173.

Time to let go 1.9 and shoot for x265-2.0 boys! :)

waiting for the v2.0 build as well, when the recent changes should be settled to some stable implements. Then we are ready to do our 4th round of massive tuning test.

[pingfr]

waiting for the v2.0 build as well, when the recent changes should be settled to some stable implements. Then we are ready to do our 4th round of massive tuning test.

I think I can safely say on the behalf of the community, we can't wait for your 4th round of tests. :D

[nandaku2]

Looks like we're getting closer and closer to a newer milestone release to me once that NUMA patch gets merged in the main tree.

- Presets tinkered a bit further: Done.
- Implementation of the --[no-]recursion-skip and --slow-firstpass features: Done.
- Tune grain improvements: Done.
- Docs cleaned out: Done.
- Multi-pass tweaks: Done.
- Full UNICODE support: Done.
- NUMA systems optimization: Done.

Now all we need is to get rid of that nasty "purple ghosting" effect users have reported in the previous pages and it sounds like we have a good candidate for a x265-2.0 release. ;)

That's just an assumption from my end, nothing else. It would just seem silly to bump the 1.9 version further forever "a la" x265-1.9+64829173.

Time to let go 1.9 and shoot for x265-2.0 boys! :)

Yes - this is the direction. We're also investigating a couple more quality issues in veryslow, as well as SAO.

I can't seem to locate any ghosting artifact issues posted, which is not a sub-720p resolution (in which case, folks, please consider lowering max-CTU). Can someone help me out here?

[pingfr]

I can't seem to locate any ghosting artifact issues posted, which is not a sub-720p resolution (in which case, folks, please consider lowering max-CTU). Can someone help me out here?

I think it is firstly posted around the page 183'ish of this thread?

Also see this specific post:

http://forum.doom9.org/showpost.php?p=1766393&postcount=3660

There appears to be some "ghosting effect" reported.

[LigH]

I believe you misunderstood, possibly due to the comma: nandaku2 believes that there is no report of ghosting artefacts with at least some HD resolution; all ghosting reports seem to address lower resolutions.

The only report I remember where some motion related issue was reported for a higher resolution video was the StarWars clip where a star blinked when the text started to scroll by.

[pingfr]

Oh, ermph. Was only trying to help. :)

[littlepox]

For resolution <= 720p, never a single case out of our hundreds of tests ever suggest that x265 outperforms x264 by any significant measure; most of the tests prove x264 to be a much better choice, at least before x265 v1.9.

[Jamaika]

The only report I remember where some motion related issue was reported for a higher resolution video was the StarWars clip where a star blinked when the text started to scroll by.
The worst are large transparent subtitles with some effects, but not nitpicking at this stage of development.

PS The smaller the image, the less one can see.;)

[Selur]

@jamaika: It would probably help if the developers hat a sample of an input file to reproduce the issue. ;)

[roo1234]

For resolution <= 720p, never a single case out of our hundreds of tests ever suggest that x265 outperforms x264 by any significant measure; most of the tests prove x264 to be a much better choice, at least before x265 v1.9.
X265 clearly outperforms x264 in very low bitrates/perceived quality

[MeteorRain]

X265 clearly outperforms x264 in very low bitrates/perceived quality

Well, I assume LP was talking about higher bitrate where quality alone matters. On very low bitrate, x265 might fit better.

[littlepox]

X265 clearly outperforms x264 in very low bitrates/perceived quality

In 2016, very low bitrate @ 720p is NOT a significant measure.

[Jamaika]

@jamaika: It would probably help if the developers hat a sample of an input file to reproduce the issue. ;)
It's not a problem for me. I try not use transparent subtitle.

For the inquisitive.
https://www.sendspace.com/file/3ijlto

[roo1234]

In 2016, very low bitrate @ 720p is NOT a significant measure.
You wrote <=720, which means all resolutions until 720. X265 is Winner in that.

[fauxreaper]

There's artifacts in sample posted on https://mailman.videolan.org/pipermail/x265-devel/2016-May/010350.html even with --no-recursion-skip. You just need to encode this sample video with subtitles to see the artifacts.

[nandaku2]

There's artifacts in sample posted on https://mailman.videolan.org/pipermail/x265-devel/2016-May/010350.html even with --no-recursion-skip. You just need to encode this sample video with subtitles to see the artifacts.

Thanks - this is the one I was looking for.

[littlepox]

You wrote <=720, which means all resolutions until 720. X265 is Winner in that.

I mean very few people would like a bitrate-starving 720p nowadays, like 1000Kbps, which is even lower than the online videos.

Even x265 is indeed the winner, so what? With all the FHD/UHD monitors and TB sized hard drive, we are not in the rmvb/xvid->x264 era. If you can only do 720p, at least feed it with enough bitrate and retain those fragile details as much as possible.

[pingfr]

@littlepox: For me, the real argument to favor x265 over x264 at the moment is because there has been very little dev/work done on x264 over the past months. At this point it feels like x264 dev has halted or at least reached a point there is very little optimization left to do.

On the contrary, x265 is the "next gen" encoder and it's free, open source and well discussed/dev/maintained.

It's safe to say there is still a lot of headroom ahead of ourselves to improve the compression:quality factor even if that is at the cost of time:cpu-cycles spent either encoding or decoding.

x264/AVC/VC-1 is a thing of the past. HD/1080p/Blu-Ray is the past.
x265/HEVC is the way to go. UHD/4K/Ultra Blu-Ray is the future.

[roo1234]

I mean very few people would like a bitrate-starving 720p nowadays, like 1000Kbps, which is even lower than the online videos.

Even x265 is indeed the winner, so what? With all the FHD/UHD monitors and TB sized hard drive, we are not in the rmvb/xvid->x264 era. If you can only do 720p, at least feed it with enough bitrate and retain those fragile details as much as possible.
Why most only think of local files? Ok, we have terabytes of cheap storage, but what if youre on a low bandwidth and would like an online video, etc? Bandwidth savings mean $$$ and accessibility to masses. Thats why lossy codecs are evolving, doing more with less.
Well, that and a better efficiency for higher res.

[Jamaika]

I mean very few people would like a bitrate-starving 720p nowadays, like 1000Kbps, which is even lower than the online videos.
Oh, they would be lose count. First, you must have a well-recorded video, a free codec is a addition. In Webcamming sports and 3000kbps is not enough.

[benwaggoner]

Got a dual Xeon E5640 system and I wonder what I could do to boost the cpu usage during x265 encoding

atm. I use:
265 --preset slow --pme --input - --output-depth 10 --y4m --profile main10 --no-high-tier --level-idc 4.1 --amp --no-open-gop --weightb --crf 18.00 --psy-rdoq 15.00 --vbv-maxrate 20000 --vbv-bufsize 20000 --range limited --colormatrix bt709 --output "D:\09_02_06_7310_02.265"
and run two encodes in parallel, but cpu usage still is just around 65%. Input and output are inside a RAM disk and decoding also isn't the bottleneck, so the problem is with the x265 settings. (I get ~5fps per encoder instance)

adding different --pools settings just caused the encoding to be even slower.
using different --frame-threads counts doesn't seem to help either
using --pmode additionally to --pme doesn't help
using msvc instead of mingw builds doesn't help (version I use is 1.9+183)

-> Is there some recommendations on what to do, to utilize the CPU on multi socket systems better?

I think that --pme is the likely culprit. You need a LOT of threads for that to do anything useful, and in typical use I've seen it actually slow things down. Generally --pmode is much more useful on a dual Xeon.

[benwaggoner]

I can't seem to locate any ghosting artifact issues posted, which is not a sub-720p resolution (in which case, folks, please consider lowering max-CTU). Can someone help me out here?
If the optimal max-CTU value changes based on resolution, can x265 use a different default value based on the encoded resolution? In general, the defaults and presets should take advantage of whatever encoder-time state information is available to better tune default encodes.

[benwaggoner]

In 2016, very low bitrate @ 720p is NOT a significant measure.
Perhaps not for your use case, but it certainly matters for a LOT of use cases. For adaptive streaming, a good very low bitrate 720p means that low-bandwidth clients could now get HD instead of SD.

[littlepox]

Perhaps not for your use case, but it certainly matters for a LOT of use cases. For adaptive streaming, a good very low bitrate 720p means that low-bandwidth clients could now get HD instead of SD.

I'm not saying that low bitrate encoding is insignificant, I'm trying to say that currently for x265 to demonstrate some advantage, the bitrate must be RIDICULOUSLY low so that it's even unpractical for online videos. In that situation the video looks too blurry and washed to be classified as "720p HD", and at that bitrate, x264@480p looks no worse even with upscale.

It's possible in the future that x265 will make it acceptable to human eyes, but not now, not before x265 v1.9.

[Magik Mark]

May we ask what we should focused on testing build 192? Thanks


Sent from my iPhone using Tapatalk

[eclipse98]

Yes - this is the direction. We're also investigating a couple more quality issues in veryslow, as well as SAO.

I can't seem to locate any ghosting artifact issues posted, which is not a sub-720p resolution (in which case, folks, please consider lowering max-CTU). Can someone help me out here?

Nandaku2, I posted an issue with color ghosting here: https://forum.doom9.org/showpost.php?p=1767169&postcount=3681

It has screenshots as well as original video source to re-produce the issue - all my attempts to get rid of these artifacts have failed, I tried pretty much everything. The only way to reduce artifacts by 30-40% is to use SSIM tune at expense of visual quality.

It gets worse though, in addition to bare trees/bushes, I have observed these artifacts on pine trees, rocky mountains and trees with leaves too (to a lesser degree). I can provide more samples if needed.

Cheers !

[x265_Project]

I'm not saying that low bitrate encoding is insignificant, I'm trying to say that currently for x265 to demonstrate some advantage, the bitrate must be RIDICULOUSLY low so that it's even unpractical for online videos. In that situation the video looks too blurry and washed to be classified as "720p HD", and at that bitrate, x264@480p looks no worse even with upscale.

It's possible in the future that x265 will make it acceptable to human eyes, but not now, not before x265 v1.9.

It frustrates us when we see someone say that x265 is no better than x264 for typical video resolutions and bit rates. That's not our experience, or the experience of any of the dozens of video experts at the companies and organizations we work with. We take it as a challenge. And so, we challenge you or anyone else to show us what you're seeing.

Please point us to a good test sequence that x264 encodes better than x265, and let us know the bit rate you prefer. Not something that was already compressed to consumer streaming video bit rates (which already has H.264 compression artifacts) - a real uncompressed or very lightly compressed (very high bit rate) video test sequence, like one of the videos posted on media.xiph.org (or CableLabs, Elemental, Harmonic, etc.). Any test video, at any bit rate. Tell us your preferred x264 settings also, and we'll encode to the same bit rate with x265 and let everyone judge which encode is better.

Before you run your own tests, be sure you're using the latest development build of x265. Littlepox - I know you have your own favorite x265 command-line recipe, but after May 12th, things changed, and I think that recipe won't deliver the best visual quality. I suggest that you start with default settings for --preset veryslow, and if you have time, also try --preset placebo (which is now noticeably better than veryslow). If you want to run faster than veryslow, also try adding --no-recursion-skip to your command line.

[littlepox]

It frustrates us when we see someone say that x265 is no better than x264 for typical video resolutions and bit rates. That's not our experience, or the experience of any of the dozens of video experts at the companies and organizations we work with. We take it as a challenge. And so, we challenge you or anyone else to show us what you're seeing.

Please point us to a good test sequence that x264 encodes better than x265, and let us know the bit rate you prefer. Not something that was already compressed to consumer streaming video bit rates (which already has H.264 compression artifacts) - a real uncompressed or very lightly compressed (very high bit rate) video test sequence, like one of the videos posted on media.xiph.org (or CableLabs, Elemental, Harmonic, etc.). Any test video, at any bit rate. Tell us your preferred x264 settings also, and we'll encode to the same bit rate with x265 and let everyone judge which encode is better.

Before you run your own tests, be sure you're using the latest development build of x265. Littlepox - I know you have your own favorite x265 command-line recipe, but after May 12th, things changed, and I think that recipe won't deliver the best visual quality. I suggest that you start with default settings for --preset veryslow, and if you have time, also try --preset placebo (which is now noticeably better than veryslow). If you want to run faster than veryslow, also try adding --no-recursion-skip to your command line.

I see I've offended you, my apologize. I don't mean to sneer at x265 nor the developers.

Our testcases are carefully taken from commercial BluRay Disc sources so none of them are heavily compressed. I'm asking my teammate to prepare a 720p one(properly down-scaled) that is short enough and it shouldn't violate the forum policies to be posted. Meanwhile, an x264 benchmark will also be available. We shall get back to you in a few days.

The three massive tuning tests we've done before is very expensive to carry(>500 encodes per time) so we only do that for stable builds; the latest one is with v1.9, and the next one shall be with v2.0. Indeed we don't have a good idea about the most updated version; but we shall be the first to cheer for your guys if major breakthrough is seen next time.

Last but not least, don't expect many users to tolerate --placebo. It's not pragmatic for daily use. The slowest one we are able to accept would be veryslow, if not deciding to use x264 when speed is a concern.

[LigH]

Just as a suggestion for a commonly available and uncompressed, "Creative Commons" licensed source: "Tears of Steel" should provide a good variety of different scenes, from stills to heavy action. The cartoonish credits may have their own challenge for an encoder possibly optimized for real-world footage.

[x265_Project]

I see I've offended you, my apologize. I don't mean to sneer at x265 nor the developers.
No apologies necessary - I'm not offended. We want your feedback, good or bad. We want to address any concerns head-on. x265 has all of the coding tools of x264, and many, many more. In other words, there is never any reason why x265 should be inferior to x264. In the worst case, x265 could encode the video with the exact same frame types, block structures, motion vectors and modes. But thanks to the HEVC standard, x265 has many other options to choose from, including larger block sizes. If there are any areas where x265 is not equal to or better than x264, we need to understand and fix them. If this is a myth, we want to bust it.

[Jamaika]

It gets worse though, in addition to bare trees/bushes, I have observed these artifacts on pine trees, rocky mountains and trees with leaves too (to a lesser degree). I can provide more samples if needed.
I think you will have to wait. The patches to the codec are tested so maybe something after the holidays to improve. I present only two pictures 6000kbps option 'best' latest codec Mainconcept (surcharge) and 'veryslow' X265. There is a difference, and that's all. Cyberlink and Corel have even poorer HEVC encoders. Only preset medium. Get worse using X265 on Google.
Mainconcept 6000kbps, 1920x1080, bframes=3, best, pass=1
http://i65.tinypic.com/295wtv9.png
X265 6000kbps, 1920x1080, bframes=5, veryslow, pass=2
http://i67.tinypic.com/33yj314.png

What can I require?

[pingfr]

The three massive tuning tests we've done before is very expensive to carry(>500 encodes per time) so we only do that for stable builds; the latest one is with v1.9, and the next one shall be with v2.0. Indeed we don't have a good idea about the most updated version; but we shall be the first to cheer for your guys if major breakthrough is seen next time.

Which is why I believe you should wait for a 2.0 official release before running any further tests since they are expensive to carry as you've stated.

By then you should be able to compare original vs. 1.9+1 encode vs. 2.0 encodes and I believe this is where we'll see most of the improvements between 1.9 and 2.0, from there you should also be able to compare your best tuning results (from 2.0, I believe) vs any x264 encodes of the same segment.

My two cents.

[pingfr]

If there are any areas where x265 is not equal to or better than x264, we need to understand and fix them. If this is a myth, we want to bust it.

On point. Couldn't have said it better myself.

[x265_Project]

Just as a suggestion for a commonly available and uncompressed, "Creative Commons" licensed source: "Tears of Steel" should provide a good variety of different scenes, from stills to heavy action. The cartoonish credits may have their own challenge for an encoder possibly optimized for real-world footage.
Yes, it's definitely one of the better test sequences, although it doesn't have any particularly challenging scenes. The higher the original quality, the better (although we need to show that we can handle grainy/noisy content also). There are a bunch of new 4K test sequences on https://media.xiph.org/video/derf/ contributed by Netflix that are excellent 10 bit content. Amazon also contributed high quality gaming captures from Twitch.

[eclipse98]

I think you will have to wait. The patches to the codec are tested so maybe something after the holidays to improve. I present only two pictures 6000kbps option 'best' latest codec Mainconcept (surcharge) and 'veryslow' X265. There is a difference, and that's all. Cyberlink and Corel have even poorer HEVC encoders. Only preset medium. Get worse using X265 on Google.

I can wait, no problem - just wanted to make sure developers are aware of the issue so it can get addressed whenever it becomes a higher priority. As I said before, I am very happy with x265 video quality, your Mainconcept screenshots confirm it.

I also tested Nvidia x265 encoder and it produces even worse color ghosting. I also tested similar footage shot in 1080 rather than 4K and there appears to be no issue, it seems to be 4K related.

Cheers !

[LigH]

There are a bunch of new 4K test sequences on https://media.xiph.org/video/derf/ contributed by Netflix that are excellent 10 bit content.

Wonderful. Rollercoaster videos always used to be challenging, now in UHD even. :cool:

[Selur]

@x265_Project: are there any plans to further improve the x265 performance on multi socket systems when encoding SD content? (for SD cpu usage&speed became better with https://patches.videolan.org/patch/13436/ but it still doesn't seem good,...)

[mandarinka]

@x265_Project: are there any plans to further improve the x265 performance on multi socket systems when encoding SD content? (for SD cpu usage&speed became better with https://patches.videolan.org/patch/13436/ but it still doesn't seem good,...)

For such usage, I think you should probably consider doing multiple encodes at once.

[x265_Project]

@x265_Project: are there any plans to further improve the x265 performance on multi socket systems when encoding SD content? (for SD cpu usage&speed became better with https://patches.videolan.org/patch/13436/ but it still doesn't seem good,...)

We're always working to improve performance, but with a single instance of x265 we run into Amdahl's law. We can only parallelize so much. With SD encodes, there isn't enough parallel work to keep many cores/threads busy, even with frame parallelism. So, that's one of the reasons we developed UHDkit, which can break a single encode into many chunks, and encode the chunks in parallel. If you're encoding many different videos, you can do them all in parallel. Just be sure to pin each encode to a different thread pool using our pools feature.

[pingfr]

We're always working to improve performance, but with a single instance of x265 we run into Amdahl's law. We can only parallelize so much. With SD encodes, there isn't enough parallel work to keep many cores/threads busy, even with frame parallelism. So, that's one of the reasons we developed UHDkit, which can break a single encode into many chunks, and encode the chunks in parallel. If you're encoding many different videos, you can do them all in parallel. Just be sure to pin each encode to a different thread pool using our pools feature.

May I ask what OS does UHDKit runs on natively? I would assume Linux?

Any public pricing models available?

Cheers.

[LigH]

but with a single instance of x265 we run into Amdahl's law.

Thank you, now I know the "RTFM term" to smash against the forehead of people who keep complaining about a "low CPU utilization" ;) ;)

I already suspected that the more complex an algorithm gets, the more restricted parallelizability gets as well (because many intermediate results have to be collected to a final result). You confirmed this assumption here.
__

P.S.:

Matheusz just mentioned in the mailinglist that there are some quirks regarding DLLs and compilers and speed ... not all compilers can handle optimization of multilib builds correctly, so in general, using a build with separate encoder library DLLs per bitdepth should be the fastest solution. On top, GCC 6.1 seems to be faster for Win32 8-bit, but GCC 5.3 for 10-bit and 12-bit code.

I won't be able to use several compiler versions easily, therefore I will keep building with GCC 5.3.

To be able to place both 32-bit and 64-bit builds in the same directory, the scripts will soon produce a new naming pattern, libx265-32_main[10|12].dll for separate Win32 DLLs.

[Motenai Yoda]

Thank you, now I know the "RTFM term" to smash against the forehead of people who keep complaining about a "low CPU utilization"
Nop, Amdahl's law says that optimizing a part x of a program y will make y faster in proportion to how much time y spent on x.

ie a program with function a() and function b(), where a() take 80% of time and b() 20%, optimizing b() to run twice faster will make the program run in 80%+(20%/2) = 90% the time not 50%.

In this case x265's unparallelizable parts will slowing down so much any further (if) possible optimization of the parallelized parts, or parallelizations, will give negible speedup with SD content.

[x265_Project]

Nop, Amdahl's law says that optimizing a part x of a program y will make y faster in proportion to how much time y spent on x.

ie a program with function a() and function b(), where a() take 80% of time and b() 20%, optimizing b() to run twice faster will make the program run in 80%+(20%/2) = 90% the time not 50%.

In this case x265's unparallelizable parts will slowing down so much any further (if) possible optimization of the parallelized parts, or parallelizations, will give negible speedup with SD content.
That's right. Amdahl's law tells you that if you have an algorithm that is 100% parallelizable, you can speed this up proportionally by adding more processor cores. But when you have both serial and parallel operations, some threads will end up waiting for the information they need from another thread that isn't finished, and adding more threads ends up having diminishing returns.

There are many routines involved in HEVC encoding that are serial in nature. For example, to find the true "cost" (# of bits) of a candidate encoding mode for a block, we have to encode and decode the predicted block, calculate the residual error (the difference between the source block and the predicted block), calculate the discrete cosine transform of the residual error, quantize the transformed residual error, and compress the encoded result with CABAC entropy coding. All in series. This can't be further parallelized. CABAC encoding itself is an inherently serial process.

Thanks to Wavefront Parallel Processing, we can encode multiple rows of blocks in parallel, and thanks to x265's frame parallelism, we encode multiple frames in parallel. But the number of rows per frame is limited by the frame size. x265 can operate on more rows per frame with larger frames.

[LigH]

x265 1.9+200-6098ba3e0cf16b11 (https://www.mediafire.com/download/fckfmb1eiol4idp/x265_1.9+200-6098ba3e0cf16b11.7z) (oh, revision hashes are longer now): some thread pool changes, git version ID and other fixes

[Ma]

oh, revision hashes are longer now

x264 has 7 chars long revision hash, x265 had 12 chars long, now it is 16 chars long. On page https://bitbucket.org/multicoreware/x265/commits/all there are 7 chars long revision hashes (and it is enough).

I think we should switch to 7 chars long revision hashes in x265.

[RiCON]

I don't know why I thought {node|short} gave 16-char long hash, but yeah, it should be 12 character instead.
It shouldn't be 7 characters because that can be mistaken with git short form and git/hg commits have no relation whatsoever.

[LigH]

A patch was offered today which reduces the hash length to 12 again.

[RiCON]

I did, yes. After reading the replies here.

[LigH]

Oh, that was you. :)

[pingfr]

Hey guys,

Development appears to be quiet since commit ad961f5 on the 30/05... I mean sure I can see a few minor commits here and there, but they are mostly "minor fixes" such as warning silencing, clarifications, commit hash fixing, etc.

I'm wondering what would be the "next" step or what's the "direction" at the moment from the MulticoreWare team, specifically if anything in the "visual optimization" field of expertise or "compression efficiency" is either in the works or in the pipeline? :)

Cheers.

[x265_Project]

Hey guys,

Development appears to be quiet since commit ad961f5 on the 30/05... I mean sure I can see a few minor commits here and there, but they are mostly "minor fixes" such as warning silencing, clarifications, commit hash fixing, etc.

I'm wondering what would be the "next" step or what's the "direction" at the moment from the MulticoreWare team, specifically if anything in the "visual optimization" field of expertise or "compression efficiency" is either in the works or in the pipeline? :)

Cheers.
There are some optimizations to SAO in development that should be ready this month. Longer term there are a number of visual quality and performance improvements we're working on, but for now we'd prefer to keep our powder dry, so we can make a bigger bang when we have some results to show.

[pingfr]

There are some optimizations to SAO in development that should be ready this month. Longer term there are a number of visual quality and performance improvements we're working on, but for now we'd prefer to keep our powder dry, so we can make a bigger bang when we have some results to show.

Amazing, thanks for the heads up! :)

[filler56789]

.....

PS: Isn't "(now it only enables B GOP structure)" on x265 help outdated

It is the same as version 0.3 when you didn't have b-adapt...

Yes$S... HEVC encoder version 1.9+215-78ffb67a844e still says the same thing.

x265 --help

[Barough]

There are some optimizations to SAO in development that should be ready this month. Longer term there are a number of visual quality and performance improvements we're working on, but for now we'd prefer to keep our powder dry, so we can make a bigger bang when we have some results to show.

That's gr8 info/news. Thnx. :)

[filler56789]

Very-well, my next MinGW builds will not say:

"(now it only enables B GOP structure)"

Sometimes Wikipedia is right: be bold :)

[innosia]

Thanks guys for all of the efforts

I am noob here, can I ask the cli syntax to run x265 to convert mkv to mkv for smaller CRF for tune animation

[LigH]

Unfortunately, a "vanilla" build of x265 can neither read video input from an MKV directly, nor create the encoded result inside an MKV container.

You will need some helping frameserver script (usually AviSynth) to decode the video from your source MKV, using avs4x26x or similar piping helper tools, and will get a raw HEVC video stream you have to multiplex later with e.g. mkvmerge.

Your only way to achieve your desired workflow may be using ffmpeg with libx265 as video encoder. Probably something like:

ffmpeg -i input.mkv -vcodec libx265 -crf 20 -preset slow -tune animation -acodec copy output.mkv

[innosia]

Unfortunately, a "vanilla" build of x265 can neither read video input from an MKV directly, nor create the encoded result inside an MKV container.

You will need some helping frameserver script (usually AviSynth) to decode the video from your source MKV, using avs4x26x or similar piping helper tools, and will get a raw HEVC video stream you have to multiplex later with e.g. mkvmerge.

Your only way to achieve your desired workflow may be using ffmpeg with libx265 as video encoder. Probably something like:

ffmpeg -i input.mkv -vcodec libx265 -crf 20 -preset slow -tune animation -acodec copy output.mkv

thanks LigH for pointing it out
will try and let you know if it works

thanks again

[innosia]

hi i found some guide
but it was 8 bit ffmpeg

can i confirm the page : https://ffmpeg.zeranoe.com/builds/ (64 bit static) has 10 bit? and how to specify for 10 bit depth?

[Ma]

can i confirm the page : https://ffmpeg.zeranoe.com/builds/ (64 bit static) has 10 bit? and how to specify for 10 bit depth?

It has only 8-bit x265 stable 1.9 (outdated). You can start with 100 frames sample like this:
ffmpeg -i ../original.mkv -ss 50 -frames 100 -v error -f yuv4mpegpipe - | x265-10b --y4m - -p slow --crf 24 o.hevc
where x265-10b is current (1.9+217) 10-bit x265.

[LigH]

All libx265 options not explicitly exposed by ffmpeg have to be added in internal parameter format (not x265 CLI format) following the ffmpeg '-x265-params' parameter:

ffmpeg -i input.mkv -vcodec libx265 -crf 20 -preset slow -tune animation -x265-params output-depth=10 -acodec copy output.mkv

Possible problem: x265 does not offer a tuning "animation" yet. Without a tuning parameter it does something, let's see if it used 10 bit depth...

No. It recognizes neither "-x265-params output-depth=10" nor "-x265-params D=10" nor "-x265-params profile=main10".

BTW, I tested with an ffmpeg build created with jb-alvarado's media-autobuild_suite. It may contain multiple x265 libraries with several bit depths. But their selection does not seem to work.

[easyfab]

You can try ffmpeg with -c:v libx265 -pix_fmt yuv420p10le perhaps

[LigH]

The problem is that the mentioned parameters are not recognized:

...
[libx265 @ 000000000294d940] Unknown option: {output-depth|D|profile}.
x265 [info]: HEVC encoder version 1.9+192-aeade2e8d8688ebf
x265 [info]: build info [Windows][GCC 5.3.0][64 bit] 8bit+10bit+12bit
...

So, libx265 inside ffmpeg reports providing three bitdepth libraries, but I cannot find a recognized parameter to select one specific out of the three.

Piping Y4M from ffmpeg to x265 CLI is probably the best OS-independent way to get high bitdepth video streams. They just have to be multiplexed afterwards.
__

The suggestion by easyfab was confirmed in IRC. Testing...

OK, profile "Main 10" is selected, so this appears to be a correct suggestion. Tanks, easyfab!

[Ma]

So, libx265 inside ffmpeg reports providing three bitdepth libraries

Are you sure that you are writing about Zeranoe FFmpeg Build?

[qyot27]

The problem is that the mentioned parameters are not recognized:

...
[libx265 @ 000000000294d940] Unknown option: {output-depth|D|profile}.
x265 [info]: HEVC encoder version 1.9+192-aeade2e8d8688ebf
x265 [info]: build info [Windows][GCC 5.3.0][64 bit] 8bit+10bit+12bit
...

So, libx265 inside ffmpeg reports providing three bitdepth libraries, but I cannot find a recognized parameter to select one specific out of the three.
You're supposed to use -pix_fmt, because that's how FFmpeg selects the format, and when used inside FFmpeg, libx265 works in direct I/O mode - give it 8-bit, it outputs 8-bit. Give it 10-bit, it outputs 10-bit*, etc. Overriding that has to be done by the part of the toolchain that resamples formats, and that's FFmpeg, not libx265.

*So long as it's a multilib libx265, anyway.

[LigH]

Are you sure that you are writing about Zeranoe FFmpeg Build?

No, as I added later:

BTW, I tested with an ffmpeg build created with jb-alvarado's media-autobuild_suite. It may contain multiple x265 libraries with several bit depths.

[innosia]

Thanks all and ligh for confirming that ffmpeg+libx265 does not support converting from 8bit to 10bit (my source is 8 bit).

My only option will be below

It has only 8-bit x265 stable 1.9 (outdated). You can start with 100 frames sample like this:
ffmpeg -i ../original.mkv -ss 50 -frames 100 -v error -f yuv4mpegpipe - | x265-10b --y4m - -p slow --crf 24 o.hevc
where x265-10b is current (1.9+217) 10-bit x265.

May I know
1. Where to get x265-10b.exe? I can't compile myself coz a noob here
2. How to convert o.hevc back to mkv?

Thank you

[LigH]

1. There are several sources for both "multilib" builds (one EXE with all three bitdepth variants of the encoder included, which you can select by the parameter -D) and separate builds; my MinGW / GCC packages contain both kinds, more links to different builds can be found e.g. in the VideoHelp forum thread (http://forum.videohelp.com/threads/357754-%5BHEVC%5D-x265-EXE-mingw-builds).

2. Multiplex it using mkvtoolnix (with mkvmerge as CLI tool, or with the formerly usual MKVmergeGUI or the current MKVToolnixGUI as GUI application) or maybe with ffmpeg too. This is not a "conversion", just wrapping the raw HEVC video stream into a Matroska container (which may then also contain more than just the video stream, like an additional audio stream).
__

P.S. 1.: x265 1.9+217-626fcbac7ffb (https://www.mediafire.com/download/1l666559dr8u691/x265_1.9+217-626fcbac7ffb.7z)

[innosia]

1. There are several sources for both "multilib" builds (one EXE with all three bitdepth variants of the encoder included, which you can select by the parameter -D) and separate builds; my MinGW / GCC packages contain both kinds, more links to different builds can be found e.g. in the VideoHelp forum thread (http://forum.videohelp.com/threads/357754-%5BHEVC%5D-x265-EXE-mingw-builds).

2. Multiplex it using mkvtoolnix (with mkvmerge as CLI tool, or with the formerly usual MKVmergeGUI or the current MKVToolnixGUI as GUI application) or maybe with ffmpeg too. This is not a "conversion", just wrapping the raw HEVC video stream into a Matroska container (which may then also contain more than just the video stream, like an additional audio stream).
__

P.S. 1.: x265 1.9+217-626fcbac7ffb (https://www.mediafire.com/download/1l666559dr8u691/x265_1.9+217-626fcbac7ffb.7z)

Thanks Ligh, it works!
currently i am experimenting with the speed vs quality i can get, my cpu took more than 2 hours encoding 200MB 9 mins animation so i terminate it. I used preset veryslow

ffmpeg -i %1 -an -me_method umh -b_strategy 2 -subq 10 -refs 10 -v error -weightp 2 -trellis 2 -qmin 10 -qmax 81 -f yuv4mpegpipe - | x265_main12 --y4m - --level 5.1 --preset veryslow --min-keyint 12 --scenecut 40 --bframes 10 --qcomp 0.6 --rc-lookahead 60 --aq-mode 1 --aq-strength 0.8 --merange 24 --psy-rd 0.60 --crf %5 --output %2

will probably tweak for faster encode.
i appreciate if you can help?

[Jamaika]

Which of the following web pages is current? Maybe there are some new?
http://x265.ru/builds/
https://encoder.pw/
http://msystem.waw.pl/x265/
https://github.com/videolan/x265/commits/master

[x265_Project]

Which of the following web pages is current? Maybe there are some new?
http://x265.ru/builds/
https://encoder.pw/
http://msystem.waw.pl/x265/
https://github.com/videolan/x265/commits/master
The videolan github should be a mirror of our main repository... https://bitbucket.org/multicoreware/x265

None of the other sites are supported by us.

[filler56789]

Which of the following web pages is current? Maybe there are some new?
http://x265.ru/builds/
https://encoder.pw/
http://msystem.waw.pl/x265/
https://github.com/videolan/x265/commits/master

Just compare the builds they offer with the list below:

https://bitbucket.org/multicoreware/x265/commits/

encoder.pw and x265.ru do NOT contain the builds based on the most recent commits.
waw.pl apparently is up-to-date, but their content is a mess :scared:

[Jamaika]

The videolan github should be a mirror of our main repository... https://bitbucket.org/multicoreware/x265

Hmm... At the github I have a patch from 7 June, Videolan 24 June, Bitbucket 16 June. Hence my question.
https://patches.videolan.org/project/x265-devel/list/
https://bitbucket.org/multicoreware/x265/commits/branch/default

[LigH]

You forgot my build archive at MediaFire (https://www.mediafire.com/folder/6lfp2jlygogwa/HEVC); but I will not build daily or even automatically, only after a few important commits, more or less weekly.

[Jamaika]

I didn't forget. You are here from the beginning.
He should ask: Do users LigH, El Heggunte, Selur, others have full accreditation, and are now confident creators codec X265?
encoder.pw and x265.ru do NOT contain the builds based on the most recent commits.
We not only Selur's codecs are also outdated.(GNU 5.3.0)

[kypec]

He should ask: Do users LigH, El Heggunte, Selur, others have full accreditation, and are now confident creators codec X265?

They're not creators, just builders. They compile the source code which is being written (created, developed) by MulticoreWare programmers so that we (=regular users) can use x265 on our systems and encode video streams.

[Ma]

1. Where to get x265-10b.exe? I can't compile myself coz a noob here
2. How to convert o.hevc back to mkv?

You need 3 EXE files:
1) ffmpeg.exe from https://ffmpeg.zeranoe.com/builds/
2) x265-10b.exe from http://www.msystem.waw.pl/x265/ (first table, ver. 1.9+217, Minimum CPU arch AVX)
3) mkvtoolnix-gui.exe from http://www.fosshub.com/MKVToolNix.html (portable 64bit)

Let your original MKV file will be org.mkv
First you encode video:
ffmpeg -i org.mkv -v error -vf scale=384:-2 -f yuv4mpegpipe - | x265-10b --y4m - -p0 new.hevc
After a while (should be ultra fast) you will have new.hevc file with encoded video.

Then you start mkvtoolnix-gui.exe.
1) press button "+ Add source files" and choose your original org.mkv
2) second time press button "+ Add source files" and choose your encoded new.hevc
3) in window "Tracks, chapters and tags:" please move last track "MPEG-H/HEVC/h.265" to second place from top (by mouse)
4) uncheck original video track, it should looks like in attached picture
5) press "Start muxing" button
You will have "org (1).mkv" file with replaced video track. If you do this, you can see if audio is sync with video (video quality should be very low). If all is OK, you can remove '-vf scale=384:-2' from ffmpeg options and instead of '-p0' in x265 options you can insert '-p8 --crf 20' and encode to better quality (but very slow).

[MeteorRain]

I also compile a modded version including various input and output options.

[Selur]

We not only Selur's codecs are also outdated.(GNU 5.3.0)
I normally build fresh versions when:
a. something interesting was committed
b. I release a new Hybrid version
but not sure if I won't stop building myself since the mingw builds keep on randomly crashing on my i7 and on newer Xeons,... (guess there is still something off with MinGW or the way https://github.com/jb-alvarado/media-autobuild_suite builds the binaries,.. MSVC builds seem to work fine)

[Leo 69]

Dear development team,

Has there been any good progress lately? When x265 can be expected to produce as sharp encodes as x264 does?
Thanks a lot!

[microchip8]

Dear development team,

Has there been any good progress lately? When x265 can be expected to produce as sharp encodes as x264 does?
Thanks a lot!

I did recently an encode to test the advancements so far in x265. Honestly, I'm very disappointed when it comes to retaining detail (and a bit of noise).

The same encode done with x264 looks far and I repeat FAR better than the one done with x265.

I usually know what I do when configuring the options, but I wasn't able to match the quality of the x264 encode thus far.

So I'd say it'll probably take 1-2 years for x265 to come on par with x264

[burfadel]

Try some custom options:
--rdoq-level 1
--output-depth 10
--rd 4
--tu-intra-depth 3
--early-skip
--fast-intra
--b-intra
--tskip
--tskip-fast
--limit-modes
--aq-mode 2
--qg-size 16
--me star
--max-merge 3
--weightb
--bframes 6
--ref 6
--rc-lookahead 40

Note the first entry, I use the level 1 despite people saying level 2 produces more ideal results. I agree with the help:
Psy-rdoq is less effective at preserving energy when RDOQ is at level 2, since it only has influence over the level distortion costs.

I find level 1 produces the best results. --tu-intra-depth 3 is good and there is little speed penalty (only on intra, not on inter), setting it higher than 1 on inter has little improvement at the expense of encode speed. --early-skip, --fast-intra, --tskip, --tskip-fast, and --limit-modes are all performance options with little (none for me) discernible difference in picture quality. --b-intra, --aq-mode 2, --qg-size 16, --max-merge 3, --weightb, and --rc-lookahead 40 are all quality and/or efficiency related. --bframes 6 and --ref 6 seem the most ideal for me. Any higher than about 6 b-frames their usage drops very significantly. --me star seems the highest performing motion search algorithm in both quality and is fast. It really makes UMH and even HEX redundant due to the quality of the output and speed.

Note that I have --merange set to 25 for speed at 480P content. If encoding to a higher resolution then a higher merange would be beneficial. I did read there are SAO optimisations etc in the works, they should further improve the results.

Try those settings and see how the output compares. I mean just those settings, nothing else except for stating the CRF. The CRF can also be in decimal form, such as 20.5. These settings seem to work well together for not only a fast encode but great quality ouput. If you're game try say, 400 for --nr-intra and --nr-inter. It might sound counterintuitive for what you intend, but it does reduce the file size a bit such that you can use a slightly lower CRF. The lower CRF then could produce more likeable results. Try it out on a clip, one with both of those set to 0 and another with them both set to 400 but lower the CRF by say, 0.5 and see the output quality and file size. You may be even able to lower the CRF even a tad more than that.

[microchip8]

@burfadel

Thanks for the options. I'll try them but what I used for the test encode is very close to what you provided, except for the *--skip ones which I disabled. I will do some more tweaking here...

[Motenai Yoda]

just use --no-sao --deblock -2:-2 --rdoq-level 2 or 1 --psy-rdoq 1.5 or 2

@burfadel level 1 give better result for the same crf, but also a bigger file, at the same bitrate level 2 generally looks better.
also note, on presets lower than slow, without setting --psy-rdoq it's defaulted to 0 and --rdoq-level don't do anything.

[filler56789]

commit 17c0c875f27d @ http://forum.videohelp.com/threads/357754-%5BHEVC%5D-x265-EXE-mingw-builds?p=2450950&viewfull=1#post2450950

Only the x64 binary is "multilib", the x86 one is 8-bit only, and is "XP-compatible" (or at least I think so, I don't have an XP machine anymore).

[Jamaika]

I found a codec creators. I should probably use them.
https://builds.x265.eu/

[hector1980]

Hi everyone i use MeGui for my X264 encoding i was wondering if there is an encoding app like Megui with that much options for Hevc X265?! Is meGui the best in X265 or there are apps with better features for x265?!

[LigH]

MeGUI did not yet provide a lot of options for x265 because x265 still changes a lot (and its developer seems to be not very active a.t.m.). Other tools like StaxRip x64 and Hybrid may provide more detailed and current GUI options. Also there is MuldeR's Simple x264/x265 launcher; but beware, it will not create any AviSynth script to let x265 read media files.

[hector1980]

So Staxrip and Hybrid gives away best features for X265?!
By the way is there any AVISYNTH for hevc X265 or not??

[LigH]

I don't know which is "the best". Maybe one I did not yet hear of. I use batch files most usually, and I do not even use x265 regularly yet, only for testing.

Unmodified x265 builds can not yet load AviSynth scripts directly, only YUV (raw without, or with YUV4MPEG header); you will usually pipe from a tool which can read AviSynth scripts (e.g. avs4x26x) to x265. But there may be x265 builds modified to include additional input modules.

The x in x265 is lower case. HEVC is all upper case.

[burfadel]

just use --no-sao --deblock -2:-2 --rdoq-level 2 or 1 --psy-rdoq 1.5 or 2

@burfadel level 1 give better result for the same crf, but also a bigger file, at the same bitrate level 2 generally looks better.
also note, on presets lower than slow, without setting --psy-rdoq it's defaulted to 0 and --rdoq-level don't do anything.

I did some testing, we're both 'right' :). Without changing any other settings, --rdoq-level 1 is nicer than 2, but 2 results in smaller file. I increased the -psy-rdoq to 1.28 which equates to generally the same file size (1.3 is fractionally larger), and --rdoq-level 2 was nicer whilst maintaining the same file size as --rdoq-level 1.

So, --rdoq-level 2 by itself isn't ideal, therefore I stand by my previous settings with the following alterations:

--rdoq-level 2 (changed from 1)
--psy-rdoq 1.28 (up from the default of 1.00)

[stax76]

MeGUI did not yet provide a lot of options for x265 because x265 still changes a lot (and its developer seems to be not very active a.t.m.). Other tools like StaxRip x64 and Hybrid may provide more detailed and current GUI options. Also there is MuldeR's Simple x264/x265 launcher; but beware, it will not create any AviSynth script to let x265 read media files.

It don't has anything to do with x265 is still changing but rather with GUI building is very tedious.

As example my x264 dialog is about 4000 lines of code for about 100 switches and everything is handcrafted. I had to go to 7 different code locations when I was adding a new switch.

When x265 came up I had better coding skills and better tools available, it was clear I'm not going again into the major trouble of building and maintaining something tedious like my x264 dialog, I had few choices, make only a CLI solution, I toy solution, or think about how can I add full support without 95% of the hassle I dealt in the x264 dialog with. My solution was a GUI framework were the GUI is done from code, that's why all new GUIs look very similar, they are all build without form designer. The second part is a command line framework where the GUI, GUI logic, command line definition, command line logic and command line generation and persistence, is handles, everything is integrated and automated.

This resulted in instead on 7 lines needed to edit for every switch only a single line needed for the majority of switches. The entire x265 dialog has a code size 3-4 time smaller then the x264 dialog while supporting about more 30 switches .

The end result is my definitions look as simple as this:


New OptionParam With {.Switch = "--fps", .Text = "FPS:", .Options = {"Automatic", "24", "24000/1001", "25", "30000/1001", "50", "60000/1001"}},
New NumParam With {.Switch = "--seek", .Text = "Seek:"},
New BoolParam With {.Switch = "--dither", .Text = "Dither (High Quality Downscaling)"})


Building the frameworks took much time but it was also much fun being innovative work instead of drudge work like the x264 dialog, it's also fun to add new codecs and switches, things take now minutes instead of hours.

It don't make sense to add all switches from an encoder because some switches are not relevant for instance audio switches which handles the GUI on it's own, from all switches I think they are relevant StaxRip has a 100% coverage for x265, QSVEncC and NVEnC, it has a automated test built in that analyses x265/QSVEncC/NVEnC help file and source code to detect new, changed and removed switches.

I'm waiting for new codecs to be added because it will take only minutes and will be fun.

[Motenai Yoda]

I increased the -psy-rdoq to 1.28 which equates to generally the same file size
I was rather talking about decrease crf

[burfadel]

I know, I just wrote what looked more effective :).

[Grojm]

Seems they are preparing another x265 stable release: https://bitbucket.org/multicoreware/x265/commits/

However, this (major / showstopping) ghosting / flickering bug seems still to be present. The only thing they have provided so far is this placebo "no recursion skip" switch which does not reduce the ghosting / flickering in a significant manner, as I described before: http://forum.doom9.org/showpost.php?p=1767841&postcount=3713

Seems I have to wait another year until I can replace x264 with x265 for my movie projects...

[LigH]

And we have to hear the same complaints for another year... ;)

But we should already know: Implementing features they get paid for is their priority, and fixing issues you don't really know the reason of takes more time.

I won't lose confidence, though, that issues will be fixed. And it will be easier with more obvious and reproduceable samples.

[nevcairiel]

However, this (major / showstopping) ghosting / flickering bug seems still to be present. The only thing they have provided so far is this placebo "no recursion skip" switch which does not reduce the ghosting / flickering in a significant manner

Did they actually say this is supposed to directly help with that? I didn't read that, but I may have missed it. Otherwise, not everything revolves around one particular issue, that flag probably has a bunch of other uses ;)
Note that --no-recursion-skip was renamed to --no-rskip

[pingfr]

Merge with default; prep for 2.0

It's coming! :D

[x265_Project]

Implementing features they get paid for is their priority, and fixing issues you don't really know the reason of takes more time.

To be clear, encoding efficiency and quality are our first priority. We cannot afford to come in 2nd place to any other encoder.

[Grojm]

Did they actually say this is supposed to directly help with that? I didn't read that, but I may have missed it. Otherwise, not everything revolves around one particular issue, that flag probably has a bunch of other uses ;)
Note that --no-recursion-skip was renamed to --no-rskip

You are right, only some forum users claimed that. The x265 team just claimed that with this switch "visual quality is greatly improved" (without mentioning in which way the quality is improved): https://www.facebook.com/x265project/posts/1703598466523898?comment_id=1703672846516460&comment_tracking=%7B%22tn%22%3A%22R%22%7D

[fauxreaper]

Multi-level rskip is a good improvement: has less compression artifacts and better quality than old --rskip.

[LigH]

x265 1.9+227-836a870ba76b (https://www.mediafire.com/download/qqkrtwb2sy8vu8e/x265_1.9+227-836a870ba76b.7z): merge with stable

[shinchiro]

I noticed about ~8% speed drop in commit 836a870 compare to 626fcbac7ffb under similar settings. Is rskip going to cost that much speed?

[LigH]

Rather surprising; I would expect skipping a few recursion steps speeding up the encoding instead.

[stax76]

maybe somebody has a idea on following error:

http://pastebin.com/bGKH1yLZ


C:\Stax\Apps\ffmpeg\ffmpeg.exe -i "D:\Video\The Dead Pool 1988_temp\The Dead Pool 1988_new.avs"
-f yuv4mpegpipe -pix_fmt yuv420p -loglevel error - | C:\Stax\Apps\x265\x265_ml.exe --pass 2
--bitrate 1861 --output-depth 10 --aq-mode 3 --bframes 8 --frames 131224 --y4m
--stats "D:\Video\The Dead Pool 1988_temp\The Dead Pool 1988_new.stats"
--output "D:\Video\The Dead Pool 1988_temp\The Dead Pool 1988_new_out.hevc" -

y4m [info]: 1920x1072 fps 24000/1001 i420p8 unknown frame count
raw [info]: output file: D:\Video\The Dead Pool 1988_temp\The Dead Pool 1988_new_out.hevc
x265 [info]: HEVC encoder version 1.9+140-34a3d35c5f97
x265 [info]: build info [Windows][GCC 5.3.0][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI2
x265 [info]: Main 10 profile, Level-4 (Main tier)
x265 [info]: Thread pool created using 8 threads
x265 [info]: frame threads / pool features : 3 / wpp(17 rows)
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge : hex / 57 / 2 / 2
x265 [info]: Keyframe min / max / scenecut : 23 / 250 / 40
x265 [info]: Lookahead / bframes / badapt : 20 / 8 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 0
x265 [info]: References / ref-limit cu / depth : 3 / on / on
x265 [info]: AQ: mode / str / qg-size / cu-tree : 3 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress : ABR-1861 kbps / 0.60
x265 [info]: tools: rd=3 psy-rd=2.00 signhide tmvp strong-intra-smoothing
x265 [info]: tools: lslices=6 deblock sao stats-read
av_interleaved_write_frame(): Broken pipe
Error writing trailer of pipe:: Broken pipe

[LoRd_MuldeR]

I think the error message "av_interleaved_write_frame(): Broken pipe" originates from the FFmpeg process.

You will usually get this error after the application on the "output" side of the pipe has terminated unexpectedly, so the pipe is now broken and thus FFmpeg (which is on the "input" side of that pipe) now fails to write to the pipe.

My guess would be that the x265 process has crashed. Either that, or it already failed to even start with encoding.

[LigH]

If it happens at the end, I'd wonder if the movie has as many frames as delivered in the command line; if it happens quite immediately, maybe there is something with the stats? And probably somewhere in the middle ... hmm, Selur mentioned unexpected crashes in MinGW builds created by media-autobuild_suite; which x265.exe build do you use?

[pradeeprama]

I noticed about ~8% speed drop in commit 836a870 compare to 626fcbac7ffb under similar settings. Is rskip going to cost that much speed?

Can you please share your command line for encoding? Also, do you see an improvement in quality?

[shinchiro]

..do you see an improvement in quality?
yes..there's some quality improvement even in low bitrate. kudos for that :D

Cli (626fcbac7ffb):
ffmpeg -i input.mkv -f yuv4mpegpipe -pix_fmt yuv420p - | x265.exe --y4m --bitrate 500 --preset veryslow --ctu 32 --max-tu-size 16 --tu-intra-depth 2 --tu-inter-depth 2 --me umh --rdpenalty 1 --psy-rd 2.5 --recursion-skip --rc-lookahead 90 --aq-mode 2 --ref 3 --deblock -3:-3 --no-rect --no-amp --no-b-intra --no-weightb --no-psy-rdoq --no-rdoq-level --no-strong-intra-smoothing --no-sao --output output.mkv -

Cli (836a870): (rskip is enabled by default in preset veryslow)
ffmpeg -i input.mkv -f yuv4mpegpipe -pix_fmt yuv420p - | x265.exe --y4m --bitrate 500 --preset veryslow --ctu 32 --max-tu-size 16 --tu-intra-depth 2 --tu-inter-depth 2 --me umh --rdpenalty 1 --psy-rd 2.5 --rc-lookahead 90 --aq-mode 2 --ref 3 --deblock -3:-3 --no-rect --no-amp --no-b-intra --no-weightb --no-psy-rdoq --no-rdoq-level --no-strong-intra-smoothing --no-sao --output output.mkv -

[Barough]

Have also noticed a speed drop here. Commits compared, 78ffb67 with 836a870. ~7%

Preset : Medium
Custom Command line :
--me 3 --ref 4 --aq-mode 2 --aq-strength 3 --rdoq-level 1 --psy-rdoq 5 --rc-lookahead 80 --no-rskip

[LigH]

Better compare either the same build with one option enabled and disabled, or two builds with both the same options; you may be surprised how much more changed between two releases, and then you can't decide if the reason of an output or speed difference is this one option you changed, or all the other code differences...

In such a case where the meaning of an option changed slightly, one may need to compare four tests (both builds with option enabled and disabled).

[Barough]

I have compared between 2 specific commits using the same options and using the same test files. Have run the tests 3 times each with the commits i mentioned above above and the latest commit is a little slower.

Got a friend 2 do some quick tests also and he say the same thing about the latest commit. It's slower.

[nandaku2]

Yes. At preset veryslow, if you compare the older --recursion-skip and the current --rskip (enabled by default), you will find a small loss in speed, but a huge gain in quality. Essentially, the newer rskip aims to keep the same quality as the older no-recursion-skip, without the 40%+ performance loss.

There should be no impact at medium preset though...

[littlepox]

Yes. At preset veryslow, if you compare the older --recursion-skip and the current --rskip (enabled by default), you will find a small loss in speed, but a huge gain in quality. Essentially, the newer rskip aims to keep the same quality as the older no-recursion-skip, without the 40%+ performance loss.

There should be no impact at medium preset though...

I'm a bit confused here; let's assume we use the newest stable build:

1. At preset veryslow, is --rskip set by default?
2. At preset placebo, is --no-rskip set by default?
3. At preset slower and slow, --rskip gives a huge quality gain with only a little speed decrease, true or false?
4. It is not recommended to set --no-rskip unless one don't care about time but only quality, true or false?

[LigH]

To avoid doubts, you should publish tables of results (build / options / speed), not only interpretations.

[Ma]

I'm a bit confused here; let's assume we use the newest stable build:

1. At preset veryslow, is --rskip set by default?
2. At preset placebo, is --no-rskip set by default?
3. At preset slower and slow, --rskip gives a huge quality gain with only a little speed decrease, true or false?
4. It is not recommended to set --no-rskip unless one don't care about time but only quality, true or false?

1. --rskip is set as default (with exception of placebo preset): https://bitbucket.org/multicoreware/x265/src/d574d3d5b9cf7d06113b49af60448d1de08e0e67/source/common/param.cpp?at=stable&fileviewer=file-view-default#param.cpp-167

2. At preset placebo --no-rskip is as default: https://bitbucket.org/multicoreware/x265/src/d574d3d5b9cf7d06113b49af60448d1de08e0e67/source/common/param.cpp?at=stable&fileviewer=file-view-default#param.cpp-414

[nevcairiel]

1. --rskip is set as default (with exception of placebo preset): https://bitbucket.org/multicoreware/x265/src/d574d3d5b9cf7d06113b49af60448d1de08e0e67/source/common/param.cpp?at=stable&fileviewer=file-view-default#param.cpp-167

2. At preset placebo --no-rskip is as default: https://bitbucket.org/multicoreware/x265/src/d574d3d5b9cf7d06113b49af60448d1de08e0e67/source/common/param.cpp?at=stable&fileviewer=file-view-default#param.cpp-414

To add to that, --no-rskip is also used for --tune grain.

[Barough]

Yes. At preset veryslow, if you compare the older --recursion-skip and the current --rskip (enabled by default), you will find a small loss in speed, but a huge gain in quality. Essentially, the newer rskip aims to keep the same quality as the older no-recursion-skip, without the 40%+ performance loss.

There should be no impact at medium preset though...


Well from my and my m8's tests so is there an impact on the performance when using --no-rskip with Present Medium. The loss on our tests is 3-4 FPS.

Haven't tested with the latest commit, ie a932b43 but the loss is definitely there on 1.9+227.

[littlepox]

Well, the speed loss came from the extra work in 1.9+227, described as
"analysis: compute Inter2Nx2N after merge in analysis mode=load if reuse mode chosen is skip"

[uneedme]

maybe somebody has a idea on following error:

http://pastebin.com/bGKH1yLZ


C:\Stax\Apps\ffmpeg\ffmpeg.exe -i "D:\Video\The Dead Pool 1988_temp\The Dead Pool 1988_new.avs"
-f yuv4mpegpipe -pix_fmt yuv420p -loglevel error - | C:\Stax\Apps\x265\x265_ml.exe --pass 2
--bitrate 1861 --output-depth 10 --aq-mode 3 --bframes 8 --frames 131224 --y4m
--stats "D:\Video\The Dead Pool 1988_temp\The Dead Pool 1988_new.stats"
--output "D:\Video\The Dead Pool 1988_temp\The Dead Pool 1988_new_out.hevc" -

y4m [info]: 1920x1072 fps 24000/1001 i420p8 unknown frame count
raw [info]: output file: D:\Video\The Dead Pool 1988_temp\The Dead Pool 1988_new_out.hevc
x265 [info]: HEVC encoder version 1.9+140-34a3d35c5f97
x265 [info]: build info [Windows][GCC 5.3.0][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX AVX2 FMA3 LZCNT BMI2
x265 [info]: Main 10 profile, Level-4 (Main tier)
x265 [info]: Thread pool created using 8 threads
x265 [info]: frame threads / pool features : 3 / wpp(17 rows)
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge : hex / 57 / 2 / 2
x265 [info]: Keyframe min / max / scenecut : 23 / 250 / 40
x265 [info]: Lookahead / bframes / badapt : 20 / 8 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 0
x265 [info]: References / ref-limit cu / depth : 3 / on / on
x265 [info]: AQ: mode / str / qg-size / cu-tree : 3 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress : ABR-1861 kbps / 0.60
x265 [info]: tools: rd=3 psy-rd=2.00 signhide tmvp strong-intra-smoothing
x265 [info]: tools: lslices=6 deblock sao stats-read
av_interleaved_write_frame(): Broken pipe
Error writing trailer of pipe:: Broken pipe



might x265 can not receive the y4m stream as a 10bit input

try add -strict -1 to set ffmpeg not to compiling strictly to the limitations.

or even use raw format (need to specify all basic perferences)

ffmpeg -i XXXXXX.XXX -an -q:v 0 -f rawvideo -pix_fmt yuv420p10le -vf scale=out_color_matrix=btX0XX -r XXXXX/100X -s XXXXxXXXX -strict -1 -| (to piping out/stdout?)

replace XXX with your proper values......


-pix_fmt yuv420p12le 12bit? -pix_fmt yuv420p16le 16bit?

[benwaggoner]

might x265 can not receive the y4m stream as a 10bit input

try add -strict -1 to set ffmpeg not to compiling strictly to the limitations.
RAW is a total pain in the arse due to its lack of metadata. Here's an example string I use for 10-bit encoding. And x265 definitely can take 10-bit .y4m input.

http://test-benwagg-materials.s3.amazonaws.com/CES/grd015_sdr_25M_10bit_crf16.mp4?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJFI2EXAAIFKNS4UA%2F20141231%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20141231T182706Z&X-Amz-Expires=604793&X-Amz-SignedHeaders=Host&X-Amz-Signature=3e70789fce79c84b390d8a8f03e1e8ec722f27ffa1999bb8fbe3ec6360eca0a0

ffmpeg.exe -i "input.mov" -pix_fmt yuv420p10le -strict -1 -f yuv4mpegpipe - | x265_16bit.exe - --y4m

And you just stick the normal x265 parameters after that.

Note you need to be using a 16-bit build or the multilib build to encode 10-bit.

[uneedme]

RAW is a total pain in the arse due to its lack of metadata. Here's an example string I use for 10-bit encoding. And x265 definitely can take 10-bit .y4m input.

http://test-benwagg-materials.s3.amazonaws.com/CES/grd015_sdr_25M_10bit_crf16.mp4?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJFI2EXAAIFKNS4UA%2F20141231%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20141231T182706Z&X-Amz-Expires=604793&X-Amz-SignedHeaders=Host&X-Amz-Signature=3e70789fce79c84b390d8a8f03e1e8ec722f27ffa1999bb8fbe3ec6360eca0a0

ffmpeg.exe -i "input.mov" -pix_fmt yuv420p10le -strict -1 -f yuv4mpegpipe - | x265_16bit.exe - --y4m

And you just stick the normal x265 parameters after that.

Note you need to be using a 16-bit build or the multilib build to encode 10-bit.


LOL...

Should you specify Arguments there is no pain in the Ass, it's in your fingers... lol

[benwaggoner]

LOL...

Should you specify Arguments there is no pain in the Ass, it's in your fingers... lol
...assuming I can remember what each .y4m was in height, width, fps, and color space, and don't mind adjusting my scripts to match those all the times.

Which I don't remember, and do mind :). My napkin estimate for the number of unique combinations we commonly use is... 1440!

[uneedme]

...assuming I can remember what each .y4m was in height, width, fps, and color space, and don't mind adjusting my scripts to match those all the times.

Which I don't remember, and do mind :). My napkin estimate for the number of unique combinations we commonly use is... 1440!

err...... so your napkinS are full of tiny-handwrited-digitS......

err......it is good for your handwriting...... LOL...... Have you finally found out your bank card pin? (I thought you mean that) Sorry, calculated out......

Interpol (or propably FBI) will collect those tissues sercetly and analysis them for the portland local intel for sure...... LOL

Do you recall the guys you meet daily ever with vigilant eyes.......that you feel wired......the FBI is working......

[littlepox]

A beta version of the new --tune film based on the most up-to-date stable 10-bit build:

--ctu 32 --max-tu-size 16 --crf 18 --pbratio 1.2 --cbqpoffs -3 --crqpoffs -3 --no-sao --subme 3 --b-intra --no-amp --weightb --aq-mode 3 --aq-strength 0.9 --rd 5 --psy-rd 2.0 --psy-rdoq 3.0 --rdoq-level 2 --rc-lookahead 80 --qcomp 0.65 --no-strong-intra-smoothing

Tweak the crf between 16~20
Comments and feedback are welcome.

[Ma]

I've updated VS 2015 to update 3 and realize that 12-bit x265 compiled with options
set CXXFLAGS=/arch:AVX /GS- /GL
hangs at beginning (before encoding, illegal instruction). VS 2015 update 2 with the same options works. VS 2015 update 3 with this options works for 8-bit and 10-bit x265.

If I compile by VS 2015u3 in debug mode it works. It could be that update 3 to VS 2015 is not perfect.

[Motenai Yoda]

Comments and feedback are welcome.
I'm not sure some speed/quality parameter like --rd 5 should be changed with a tune
Of which entity is the improvement of --rc-lookahead 80 and --no-strong-intra-smoothing on film based content?
also difference (in quality) between --rdpenalty 1/2 and --ctu 32 --max-tu-size 16?

[littlepox]

I'm not sure some speed/quality parameter like --rd 5 should be changed with a tune
Of which entity is the improvement of --rc-lookahead 80 and --no-strong-intra-smoothing on film based content?
also difference (in quality) between --rdpenalty 1/2 and --ctu 32 --max-tu-size 16?

I forgot to mention, one should use preset medium ~ veryslow for this preset. It is difficult to get a tuning for all presets, all bit-rate levels, all resolutions, so we focus on a most used, specific setting: preset slow, --crf 18, 10bit 1080p.

At this level, we believe some preset options should be tuned for a better quality/speed purposes. For example:

--ctu 32 --max-tu-size 16: Though this pair does not give a significant quality gain as before(due to improvement on the logic about handling large CUs/TUs), It loses nothing in quality during the test (to be specific, It shows a very insignificant quality gain), and it still boosts the encoding speed and multi-threading. So we use it primarily for speed and secondarily for quality.

--no-sao: sao is perhaps the worst setting in x265, making the default settings as blurred as RealVideo. Removing it gives a far better detail retention.

--rdoq-level 1: In our test for the same bitrate, --rdoq-level 1&2 does not show significant difference(or 1 is slightly better than 2 for detail retention). Since 1 is probably faster than 2, we use 1.

--subme 3: this is to ensure that chroma residual is used in ME analysis, which we found to be very useful.

--rd 5: same reason, we find it more useful than costly.

--rc-lookahead 80: same reason. You may lower down that a bit, but please at least set it to be 50.

--no-strong-intra-smoothing: In the test setting, very few cases you need to smooth the boundary of large blocks, as we focus on mid-high bit-rate with 10bit. So the smoothing gives a negative return. BTW, coupled with --ctu 32 --max-tu-size 16, this option is not even relevant. Just in case someone still allow --ctu 64 --max-tu-size 32.



This setting is still in its beta stage, we plan to refine it later with more test settings.

[burfadel]

--rdoq-level 1: In our test for the same bitrate, --rdoq-level 1&2 does not show significant difference(or 1 is slightly better than 2 for detail retention). Since 1 is probably faster than 2, we use 1.

I found the same myself, in that with the other settings set the same, --rdoq-level 1 seemed to retain the detail more effectively. You should notice however that the bitrate is lower with --rdoq-level 2 than 1. I found that using --psy-rdoq 1.28 the file size was roughly the same over a range of test samples I did, and the quality ended up being nicer than --rdoq-level 1.

I looked into this because I mentioned previously --psy-rdoq 1 looked nicer, but it was pointed out to me the file size is smaller. They mentioned that at the same bitrate, level 2 was nicer and I agree with that :). I found raising the --psy-rdoq to 1.28 achieved this and was nicer than lowering the CRF by 0.1 until a similar file size was found.

Additionally, I found --me star to be the most effective performance and quality wise, however hex is the default at medium preset. I would suggest using -me star if using the medium preset with your suggested settings.

In any case, try a sample clip with --rdoq-level 2 --psy-rdoq 1.28 --me star compared to the same clip with the settings you suggested and see the difference :). The file size when using --psy-rdoq 1.28 with --rdoq-level 2 will likely be fairly similar to the file size of --rdoq-level 1 and --psy-rdoq 1 (default setting).

I also notice you have --rd 5, it is nicer but costly speed wise. On top of --rd 5 you can use --rd-refine but that is very costly performance wise. I found in some respects --rdoq-level 2 and --psy-rdoq 1.28 with --rd 4 produces similar results. Of course, using those with --rd 5 and --rd-refine is even more effective, but very costly performance wise! I found --rdoq-level 2 and --psy-rdoq 1.28 retained the texture of almost flat areas more effectively than default settings, much like --rd 5 and --rd-refine does as well. If performance isn't an issue, use all four settings together!

[burfadel]

I've updated VS 2015 to update 3 and realize that 12-bit x265 compiled with options
set CXXFLAGS=/arch:AVX /GS- /GL
hangs at beginning (before encoding, illegal instruction). VS 2015 update 2 with the same options works. VS 2015 update 3 with this options works for 8-bit and 10-bit x265.

If I compile by VS 2015u3 in debug mode it works. It could be that update 3 to VS 2015 is not perfect.

I have heard of inconsistent behaviour with a pre-update 3, but post-update 2 version of the runtimes, although the latest runtimes seem to be okay. I realise the compiler is separate to the runtimes, just thought I'd mention it. It's the reason why I haven't updated my installer, although I think the latest runtimes at least are okay now...

Ironic that they're pushing for VS2015 (version 15) now that VS15 (which will likely end up being called Visual Studio 2017) is in preview and can be tried. Currently it uses the same runtimes as VS2015.
https://www.visualstudio.com/en-us/downloads/visual-studio-next-downloads-vs.aspx

[littlepox]

Strange, we managed to keep a similar bitrate(<5% of diff) and compare the results.

Doing further tests on rdoq-level; will update this post later.

Update: change rdoq-level 1 to 2; Since it demonstrated some advantage with more testcases.

[burfadel]

Did you try bumping the --psy-rdoq a little when using --rdoq-level 2?

[littlepox]

Did you try bumping the --psy-rdoq a little when using --rdoq-level 2?

I tried two methods:

decreasing crf a bit
increasing rdoq a bit

Both of them seem better than the --rdoq-level 1, but two of them are sort of indistinguishable.

[RainyDog]

Thanks for your updated --tune-film recommendations littlepox. Your thread from last year was what made me transition to x265.

The biggest change that sticks out to me with these new beta tunings is the change from --qcomp 0.8 to 0.65. Can you elaborate on the reason for this please?

[littlepox]

Thanks for your updated --tune-film recommendations littlepox. Your thread from last year was what made me transition to x265.

The biggest change that sticks out to me with these new beta tunings is the change from --qcomp 0.8 to 0.65. Can you elaborate on the reason for this please?

Previously increasing qcomp did show a lot of advantage, but now it seems less attractive. We'd better increase psy or decrease crf instead of raising qcomp, at least with the current testcases.

[RainyDog]

Previously increasing qcomp did show a lot of advantage, but now it seems less attractive. We'd better increase psy or decrease crf instead of raising qcomp, at least with the current testcases.

Thanks. I'll mess about with lowering qcomp back down from 0.8 and decreasing CRF values to compensate too.

But do you not also find that higher qcomp equals less aggressive cutree and that also to be beneficial?

[littlepox]

Thanks. I'll mess about with lowering qcomp back down from 0.8 and decreasing CRF values to compensate too.

But do you not also find that higher qcomp equals less aggressive cutree and that also to be beneficial?


It is beneficial, but not as beneficial as crf and psy.

[LigH]

Version 2.0 was just announced in the mailing list. I'll expect a verbose post here soon...

[littlepox]

Out of curiosity, when are they going to disable the infamous Smoothing-All-Objects (sao) in the default setting?

It kills nearly every tiny detail with no bit-rate advantage, even at very low quality/bitrate.

[x265_Project]

x265 version 2.0 has been released. This release supports many new features as well as support for ARM assembly optimizations for most basic pixel and ME operations, as well as SAO cleanups and a fully tested reconfigure functionality.

Full documentation is available at http://x265.readthedocs.org/en/stable/

=========================================== New Features =========================================

• uhd-bd: Enforce Ultra-HD Blu-ray Disc parameters (overrides any other settings)
• rskip: Enables skipping recursion to analyze lower CU sizes using heuristics at different rd-levels. Provides good visual quality gains at the highest quality presets.
• rc-grain: Enables a new rate control mode specifically for grainy content. Strictly prevents QP oscillations within and between frames to avoid grain fluctuations.
• tune grain: A fully refactored and improved option to encode film grain content including QP control as well as analysis options.
• asm: ARM assembly is now enabled by default, native or cross compiled builds supported on armv6 and later systems.
==================================== API and Key Behaviour Changes ==================================
• x265_rc_stats added to x265_picture, containing all RC decision points for that frame
• PTL: high tier is now allowed by default, chosen only if necessary
• multi-pass: First pass now uses slow-firstpass by default, enabling better RC decisions in future passes
• pools: fix behaviour on multi-socketed Windows systems, provide more flexibility in determining thread and pool counts
• ABR: improve bits allocation in the first few frames, abr reset, vbv and cutree improved
=============================================== Misc ==============================================
• An SSIM calculation bug was corrected

[littlepox]

Another problem with x265 in its current mid-high quality encoding, is the blurring effect in chroma planes. x264 by default uses chroma-qp-offset -2 for yuv420&422 content and this offset is further enhanced with high psy settings. The logic behind is with chroma sub-sampling, details in the chroma planes are fragile, and psy works primarily on luma planes thus favoring more bits to be distributed to luma plane. Thus x264 uses chroma-qp-offset to get the bits back to chroma planes.

In x265, there seems no such mechanics. That's why we'd recommend to tweak -3 -3 manually for cbqoffset and crqoffset. Even so, x265 still blurs up the chroma plane as if it used to be at v1.x, only leaving major edges survive. I hope we see some improvements there in the future.

[Khun_Doug]

[QUOTE=x265_Project;1773941]x265 version 2.0 has been released. This release supports many new features as well as support for ARM assembly optimizations for most basic pixel and ME operations, as well as SAO cleanups and a fully tested reconfigure functionality.

I'm new at posting to the forum so please be patient if this doesn't submit correctly or isn't formatted properly.

Does anyone know when will a release of ffmpeg be available with the X265 2.x release included? I've been testing Hybrid with limited success, and VirtualDub using external encoders with better success. But myffmpeg and VidCoder seem the best for creating X265 videos. But myffmpeg is tweaky about the version of ffmpeg and some of the recent versions do not work. I'm anxious to see (and use) the new version of X265.

[Leo 69]

@Khun_Doug

Use StaxRip. That's an awesome piece of software, a lot more customizable than VidCoder and has a user-friendly interface. You can change the default x265.exe to the newest one and enjoy all the new features.

[Motenai Yoda]

Does anyone know when will a release of ffmpeg be available with the X265 2.x release included?
here (expire in 2 days) http://expirebox.com/download/f89c6d593bcb32e6335bb791f33b64a5.html

[littlepox]

x265 v1.0 provided an HEVC encoder with basic usage;
x265 v2.0 now finally replaces x264 as the best encoder in visual quality(with customized tunings).

As far as from our recent tests, the improvement is the perhaps the most impressive among all version jumps.
Keep up the good work developers, you are creating another incredible miracle.

Maybe in x265 v3.0, we can see the 50% promise to be realized. Now it's sort of half way there; our test suggest currently x265 saves about 25% of the bitrate compared to x264.

[nakTT]

x265 v1.0 provided an HEVC encoder with basic usage;
x265 v2.0 now finally replaces x264 as the best encoder in visual quality(with customized tunings).

As far as from our recent tests, the improvement is the perhaps the most impressive among all version jumps.
Keep up the good work developers, you are creating another incredible miracle.
Glad to hear that. Can't wait to get home and give it a try.

By the way, is there any noticeable improvement between this early 2.0 to the late 1.9 (e.g: 1.9+229)?

[LigH]

Not really a lot. In a short range around a new tag, there are usually only small fixes, cleanups, clarifications in the documentation, to "stabilize" the development status. But see for yourself in the commit log:

https://bitbucket.org/multicoreware/x265/commits/all

There is no increment count, so you will have to look for the hashes to know which of them was 1.9+229.

[nakTT]

Not really a lot. In a short range around a new tag, there are usually only small fixes, cleanups, clarifications in the documentation, to "stabilize" the development status. But see for yourself in the commit log:

https://bitbucket.org/multicoreware/x265/commits/all

There is no increment count, so you will have to look for the hashes to know which of them was 1.9+229.
Thank you very much for the reply and the advice.

[birdie]

So what are those magic settings to make x265 2.0 produce an equally sharp output with reduced bitrate vs. x264? I don't care performance/speed even if takes up to 20 times longer to encode vs x264. ;-)

Also I'd love to see a comparison with VP9.

[luigizaninoni]

I'd like to ask a question about early-skip parameter. I usually use slower preset with minor tweaks. As the name says, encodings are rather slow. However, I noticed that if I enable early-skip switch, speed gets a nice bump of about 40%. The penalty is a somewhat larger file (perhaps 2% or 3%). So my question is: does early-skip impact only on file size (I am more than willing to trade speed for a little more bitrate) ? Or do I also suffer a significant quality loss ? (to my untrained eyes, difference in quality is minimal, but someone with more experience could give some better advice)

[jlpsvk]

LigH, waiting for your 2.x build. :) When can we wait it?

[ndkamal]

A beta version of the new --tune film based on the most up-to-date stable 10-bit build:

--ctu 32 --max-tu-size 16 --crf 18 --pbratio 1.2 --cbqpoffs -3 --crqpoffs -3 --no-sao --subme 3 --b-intra --no-amp --weightb --aq-mode 3 --aq-strength 0.9 --rd 5 --psy-rd 2.0 --psy-rdoq 3.0 --rdoq-level 2 --rc-lookahead 80 --qcomp 0.65 --no-strong-intra-smoothing

Tweak the crf between 16~20
Comments and feedback are welcome.



Hi Littlepox,

I have made some tests with the beta version of the new --tune film, and the result is very good near from tune film from x264. This beta version work best on 10-bit than 8-bit version of the encoder. I hope than new tune film will be incorporate soon in the release of x265.

[mandarinka]

Another problem with x265 in its current mid-high quality encoding, is the blurring effect in chroma planes. x264 by default uses chroma-qp-offset -2 for yuv420&422 content and this offset is further enhanced with high psy settings. The logic behind is with chroma sub-sampling, details in the chroma planes are fragile, and psy works primarily on luma planes thus favoring more bits to be distributed to luma plane. Thus x264 uses chroma-qp-offset to get the bits back to chroma planes.

In x265, there seems no such mechanics. That's why we'd recommend to tweak -3 -3 manually for cbqoffset and crqoffset. Even so, x265 still blurs up the chroma plane as if it used to be at v1.x, only leaving major edges survive. I hope we see some improvements there in the future.

Thanks for pointing out that. I think I noticed said thing, but for some reason it didn't occur to me that I should try to offset the QP, then. Oh well.

[littlepox]

Several interesting facts about three different aq modes:

aq-mode = 1 is the safest choice. It is intended for high and stable quality encoding with 10-bit precision.

aq-mode = 2 is the most efficient choice. It gives less bits in high motion scenes compared to aq-mode 1, and it saves a lot of bit-rate, when you can choose to further lower down your crf or increase your psy. Human eyes are less sensitive to high motion scenes, so you can only spot its donwsides if you watch the video frame by frame. Recommend if you are using 10-bit, and encoding at a sweet-pot bit-rate.

aq-mode = 3 is the choice for 8-bit encoding, as well as low-bitrate 10 bit encoding. It distributes bits to dark areas, where color banding and blocking is most visible. Even in 10-bit encoding, if bit-rate is low, major distortions and detail loss are going to appear in dark site, and aq-mode=3 makes the best outcome. When used alone it significantly boosts the bit-rate, so you'd need to lower down the strength a bit(~0.2) and increase your crf for the size control.

The exact bitrate/crf intervals for each of them is circumstantial, we probably need further research on that.

[herbert]

Thank you for your continued testing, littlepox.

I tried your most recent settings on a sample and found the encoding times to be less than practical. After some tinkering the biggest culprits seem to be --b-intra and --rd 5, each reducing encoding speed by a third.

Have you compared the cost/benefit of --b-intra and --rd >3 and would you consider them mandatory for a possible --tune film setting?

[littlepox]

Thank you for your continued testing, littlepox.

I tried your most recent settings on a sample and found the encoding times to be less than practical. After some tinkering the biggest culprits seem to be --b-intra and --rd 5, each reducing encoding speed by a third.

Have you compared the cost/benefit of --b-intra and --rd >3 and would you consider them mandatory for a possible --tune film setting?

The benchmark for the trade-off parameters are x265.exe --tune film(without --rd --subme....) --preset veryslow. If you use --tune film --preset medium, you are almost as good as using --tune film --preset veryslow. The difference in bit-rate and visual quality is within a given threshold.

So, you should realize that this --tune film is NOT meant for fast encoding use-cases. To sacrifice little from --preset veryslow, it has to be running at a speed as slow as --preset slow.

Then, our test suggest --rd 5 and --b-intra is a must.

As I've said, making a --tune film for all use-cases is tough and inefficient. We'd rather now focus on the ripping use-case:

1. You can tolerate the speed to some degree, i.e, --preset slow and slower;
2. You are not too concerned about compatibility, i.e, you choose 10-bit encoding.
3. You are encoding for mid-high level of quality, i.e, crf=16..20


When speed is a concern, you may lower down --rd to 4; it is still quite acceptable. However do NOT use 3 since then the psy-RDOQ is disabled.
--b-intra is fairly powerful in certain cases, enabling b-frames to cope with "stranger" blocks. Coupled with --pbratio 1.2 they boost the quality of bframes which takes up 70+% of total frames. Disabling is NOT recommended.

[LigH]

There is an addition to the v2.0 announcement:

Addendum: x265 is capable of triggering this Centos kernel bug (https://bugs.centos.org/view.php?id=8371) causing a deadlock, mostly observed on large multi-socketed servers. Please update your kernels to kernel-2.6.32-504 or above to avoid that risk.

Apart from that...

LigH, waiting for your 2.x build. :)

x265 2.0+2-70581d6cd065 (https://www.mediafire.com/download/uvcmk0gu0stqed2/x265_2.0+2-70581d6cd065.7z) (MSYS/MinGW, GCC 5.3.0, 32+64bit 8/10/12bit EXEs+DLLs + multilib EXEs)

[Barough]

x265-v2.0+4-43ca554799c2 (http://mediaupload.us/asb7k60l96km) (MSYS/MinGW, GCC 5.4.0, 32 & 64bit 8/10/12bit multilib EXEs)

[stax76]

@echo off
C:\StaxRip\Apps\ffmpeg\ffmpeg.exe -i "D:\~Conv Temp\~Temp\Død.Kalm.x264_temp\Død.Kalm.x264.x265.avs" -f yuv4mpegpipe -pix_fmt yuv420p -loglevel error - | C:\StaxRip\Apps\x265\x265.exe --crf 22 --frames 1339 --y4m --output "D:\~Conv Temp\~Temp\Død.Kalm.x264_temp\Død.Kalm.x264.x265_out.hevc" -

cmd.exe /C call "D:\~Conv Temp\~Temp\Død.Kalm.x264_temp\Død.Kalm.x264.x265_encode.bat"

D:\~Conv Temp\~Temp\D¢d.Kalm.x264_temp\D¢d.Kalm.x264.x265.avs: No such file or directory
x265 [error]: unable to open input file <->


I've a bug report from a USA Win 10 user, he gets an error from x265 when his source file contains 'ø', I'm a German Win 10 user and it works fine here, isn't both USA and Germany using CP 1252? With CP 1252 the char 'ø' is on position 248.

https://en.wikipedia.org/wiki/Windows-1252

[MeteorRain]

Still waiting for the github mirror to be updated. When that is done, I'll compile my 2.0 builds.

[MeteorRain]

C:\StaxRip\Apps\ffmpeg\ffmpeg.exe -i "D:\~Conv Temp\~Temp\Død.Kalm.x264_temp\Død.Kalm.x264.x265.avs" -f yuv4mpegpipe -pix_fmt yuv420p -loglevel error - | C:\StaxRip\Apps\x265\x265.exe --crf 22 --frames 1339 --y4m --output "D:\~Conv Temp\~Temp\Død.Kalm.x264_temp\Død.Kalm.x264.x265_out.hevc" -
he gets an error from x265

I guess you should report that to ffmpeg?

[benwaggoner]

The benchmark for the trade-off parameters are x265.exe --tune film(without --rd --subme....) --preset veryslow. If you use --tune film --preset medium, you are almost as good as using --tune film --preset veryslow. The difference in bit-rate and visual quality is within a given threshold.
To clarify, you're referring to --tune grain in x265, right? There isn't a --tune film equivalent in x265 yet, nor --tune animation.

[filler56789]

I guess you should report that to ffmpeg?

I agree. avs2yuv works fine here with a file named "267øtest.avs". OS = US-English x64 Windows 7.

[littlepox]

To clarify, you're referring to --tune grain in x265, right? There isn't a --tune film equivalent in x265 yet, nor --tune animation.

No, I mean when I tested those trade-off parameters like rd=5/subme=3/b-intra, I set a comparison using the settings as in --preset veryslow, together with my other tunings like psy 2.0:3.0, aq 3:0.9...

veryslow uses subme=4 which I found to be inefficient, I change it to 3;
veryslow uses b-intra which is indeed useful, I keep it;
......
That's why in this --tune film you see some parameters typically in --preset. I believe they are necessary to keep the quality not degraded.

[stax76]

Problem was I was using CP 850 for batch files for any locale, it should be fixed using System.Console.InputEncoding.CodePage instead.

https://de.wikipedia.org/wiki/Codepage_850

[Khun_Doug]

When will the --help be updated to show the "--tune film" option? None of the 2.x encoders I have tested show this option.

Doug

[x265_Project]

When will the --help be updated to show the "--tune film" option? None of the 2.x encoders I have tested show this option.

Doug

x265 does not have a "--tune film".

[littlepox]

Just to clarify, the above discussion is not any official "--tune film", it is only "fan-made" customized tuning for film clips, and it is still fairly limited and immature.

[brumsky]

Several interesting facts about three different aq modes:

aq-mode = 1 is the safest choice. It is intended for high and stable quality encoding with 10-bit precision.

aq-mode = 2 is the most efficient choice. It gives less bits in high motion scenes compared to aq-mode 1, and it saves a lot of bit-rate, when you can choose to further lower down your crf or increase your psy. Human eyes are less sensitive to high motion scenes, so you can only spot its donwsides if you watch the video frame by frame. Recommend if you are using 10-bit, and encoding at a sweet-pot bit-rate.

aq-mode = 3 is the choice for 8-bit encoding, as well as low-bitrate 10 bit encoding. It distributes bits to dark areas, where color banding and blocking is most visible. Even in 10-bit encoding, if bit-rate is low, major distortions and detail loss are going to appear in dark site, and aq-mode=3 makes the best outcome. When used alone it significantly boosts the bit-rate, so you'd need to lower down the strength a bit(~0.2) and increase your crf for the size control.

The exact bitrate/crf intervals for each of them is circumstantial, we probably need further research on that.

Littlepox, what are the suggested CRF ranges for each of the modes?

Also, on another subject. I've been noticing issues with dark areas flickering. Kinda of a fussy look. It is difficult to describe.

Also, I noticed on stripped or checkered patterns a flickering effect as well, both in light and dark scenes.

Here are the settings I was using and noticed the issue.

--crf 21 --profile main10 --output-depth 10 --ctu 32 --max-tu-size 16 --bframes 8 --rc-lookahead 80 --scenecut 40 --ref 5 --limit-refs 0 --me 3 --merange 44 --subme 5 --no-rect --no-amp --limit-modes --max-merge 4 --no-early-skip --b-intra --no-sao --signhide --weightp --weightb --aq-mode 1 --aq-strength 1.0 --cutree --rd 4 --tu-intra-depth 3 --tu-inter-depth 3 --psy-rd 1.5 --psy-rdoq 5.0 --rdoq-level 2 --lookahead-slices 4 --qcomp 0.65 --no-strong-intra-smoothing --deblock -2:-2 --qg-size 32 --pmode

I've changed several of the settings after following this thread. :)

Thanks!

[Leo 69]

@brumsky

"Also, I noticed on stripped or checkered patterns a flickering effect as well, both in light and dark scenes."

Did you try lowering CRF to, say, 18? 21 is bad for my eyes, personally.

[Khun_Doug]

Just to clarify, the above discussion is not any official "--tune film", it is only "fan-made" customized tuning for film clips, and it is still fairly limited and immature.

Thanks for the clarification on the "fan made" option still in beta. I'll re-read some of the suggested settings and may be able to add some feedback. My primary focus is material from the 80's and 90's with a smattering of B&W classics. Almost all before HD filming was more prevalent so plenty of grain in all of these.

[RainyDog]

A beta version of the new --tune film based on the most up-to-date stable 10-bit build:

--ctu 32 --max-tu-size 16 --crf 18 --pbratio 1.2 --cbqpoffs -3 --crqpoffs -3 --no-sao --subme 3 --b-intra --no-amp --weightb --aq-mode 3 --aq-strength 0.9 --rd 5 --psy-rd 2.0 --psy-rdoq 3.0 --rdoq-level 2 --rc-lookahead 80 --qcomp 0.65 --no-strong-intra-smoothing

Tweak the crf between 16~20
Comments and feedback are welcome.

Littlepox, I've noticed that you're now only recommending to turn --amp off and leave --rect on. Previously you said that neither seemed to have any visible use so recommended to turn both off to save encoding time.

Are you now seeing a benefit to --rect then? Thanks.

[littlepox]

--rect is a bit useful so we recommend it at --preset slower/veryslow/placebo. However we do not think it valuable @ slow.
--amp, on the contrary, is only recommended @ --preset placebo; i.e., you have enough time to waste.


For those who give your feedback, kindly mention which bit-depth are you using. It is very common to have color-banding @ 8-bit without proper pre-processing; However if the source quality is good enough, massive color-banding and blocking should not appear in 10-bit encoding. Currently our focus is on the 10-bit ones.

[brumsky]

@brumsky

"Also, I noticed on stripped or checkered patterns a flickering effect as well, both in light and dark scenes."

Did you try lowering CRF to, say, 18? 21 is bad for my eyes, personally.

Leo 69, No I haven't tried CRF 18. I was hoping to get a smaller file size in general. I was hoping there was another setting I could use to assist with issues like this.

Littlepox post about aq-modes and how mode 3 give dark areas a higher preference for the dark scenes. This made me wonder if that could assist me with the issue I am seeing.

It looks like the picture becomes fussy or a significant amount of noise is added when it "flickers" - mostly in the dark scenes.

[Leo 69]

@brumsky

Can you provide a problematic sample? I'd be glad to play with it.

[pingfr]

Congratulations on the 2.0 milestone release.

I have a question for the devs so it might not be the best place to ask, maybe the Maling List is a better place, but;

Would there be any significant improvement in using such instructions sets: pclmulqdq (and maybe rdrand to some lesser extent?)

https://en.wikipedia.org/wiki/CLMUL_instruction_set

CLMUL instruction set is present in modern x86 CPUs since Ivy Bridge more or less and serves as an accelerator for GCM.

If the x265 code can take advantage of the CLMUL set we might even see a fps/efficiency/speed crunching increase. ;-)

My 2 cents.

[nevcairiel]

Would there be any significant improvement in using such instructions sets: pclmulqdq (and maybe rdrand to some lesser extent?)

x265 already uses all sorts of enhanced instruction sets from SSE to AVX2.
Note that carry-less multiplication has only very specific uses (mostly in crypto), but if it were to solve a specific problem more efficiently I'm sure they would use it.

[pingfr]

x265 already uses all sorts of enhanced instruction sets from SSE to AVX2.

That I am fully aware of the use of SSE, AVX and AVX2.

Intel's Kaby Lake (Xeon only) next generation which is due sometime late 2016/Q1 2017 is supposed to bring AVX512 to the table, I wonder how long will it take until x265 can take advantage of that instruction set? ;)

Note that carry-less multiplication has only very specific uses (mostly in crypto), but if it were to solve a specific problem more efficiently I'm sure they would use it.

To be honest with you, I wasn't even sure of what that instruction set does until late last night when I upgraded my Filezilla client and saw the rather extensive list of instruction sets it benefits of on my Macbook Air 11" (Mid 2013-i7 4650U).

So yeah, there you have it.

Edit: My Filezilla seems to use SSE SSE2 SSSE3 SSE4.1 SSE4.2 AVX AVX2 AES PCLMULQDQ RDRND BMI2 BMI2. (no typo from me, it states BMI2 twice).

[lvqcl]

until late last night when I upgraded my Filezilla client and saw the rather extensive list of instruction sets it benefits of on my Macbook Air 11" (Mid 2013-i7 4650U).

So yeah, there you have it.

Edit: My Filezilla seems to use SSE SSE2 SSSE3 SSE4.1 SSE4.2 AVX AVX2 AES PCLMULQDQ RDRND BMI2 BMI2. (no typo from me, it states BMI2 twice).

Filezilla recognizes them, but it doesn't mean that in benefits from them.

[littlepox]

--tune film Update:

--crf 18 --pbratio 1.2 --cbqpoffs -3 --crqpoffs -3 --no-sao --subme 3 --b-intra --no-amp --weightb --aq-mode 3 --aq-strength 0.9 --rd 4 --psy-rd 2.0 --psy-rdoq 3.0 --rdoq-level 2 --rc-lookahead 80 --qcomp 0.65 --no-strong-intra-smoothing

Remove --ctu 32 --max-tu-size 16; without these two we see some wired artifacts reduced in high motion, flat areas.
Decrease --rd 5 to 4 since too many complains about the speed. Seriously, you guys should try NVEncC/QSVEncC by rigaya, which encodes H.265 at lightening speed with acceptable quality.

[pingfr]

Filezilla recognizes them, but it doesn't mean that in benefits from them.

Considering Filezilla handles SFTP FTP over SSH and FTP over TLS it would make sense it recognizes and benefits from PCLMULQDQ.

[pingfr]

Decrease --rd 5 to 4 since too many complains about the speed.

Do you have any kind of statistics regarding the encoding speed delta between a --rd 5 and an --rd 4 encode of an identical sample?

What's the offset and what trade-in or trade-out it implies/correlation in regards to resulted subjective visual quality and finalized file size?

PS: I'm drunk as fuck, hope I make sense.

[littlepox]

Do you have any kind of statistics regarding the encoding speed delta between a --rd 5 and an --rd 4 encode of an identical sample?

What's the offset and what trade-in or trade-out it implies/correlation in regards to resulted subjective visual quality and finalized file size?

PS: I'm drunk as fuck, hope I make sense.

Speed increases about 1/4, or even higher.
Quality decrease is hard to quantify with limited samples and observations, but still fairly acceptable; untrained eyes are probably not going to spot any significant difference.

[NikosD]

Seriously, you guys should try NVEncC by rigaya, which encodes H.265 at lightening speed with acceptable quality.

And for Skylake owners I would suggest QSVEncC by rigaya.

Unfortunately I'm not one of them but judging from the speed and quality of my Haswell's H.264 HW encoding, Skylake's H.265 HW encoding should be faster and with better quality than Nvidia's.

Nvidia's H.265 HW encoder lacks b frame support too.

[JohnLai]

Seriously, you guys should try NVEncC by rigaya, which encodes H.265 at lightening speed with acceptable quality.

I would suggest using Intel Skylake QSV HEVC instead. QSV HEVC encoding has more options (eg, QSV ICQ rather than NVENC CQP rate control, did I mention Skylake QSV has B-Frame support while Nvidia GPU, yeah, Pascal included....doesn't even support HEVC B-Frame encoding? Oh, only Pascal will get SAO support, Maxwell will not......and maximum CU size still limited to 32x32 for Pascal [Hardware limitation] )

How do I know? Because I asked Nvidia and got replies?

[NikosD]

I posted just one sec before you.

What about speed comparison between those two HW H.265 encoders ?

And file size ?

[JohnLai]

I posted just one sec before you.

What about speed comparison between those two HW H.265 encoders ?

And file size ?


T_T......Don't have Skylake.....Can't test nor compare.
BTW, NikosD, you got Skylake right? Can post 1 minute 1920x1080 QSV HEVC encoded sample? I wanna check something with the bitstream.

And we are getting off-topic now.....PM?

[NikosD]

No. .I wrote that in my previous post.

I only have Haswell.

[JohnLai]

No. .I wrote that in my previous post.

I only have Haswell.

I see....cause I wanna verify type of Intra + Inter PU sizes, SAO and 64X64 CU size support for QSV HEVC encode.

NVENC HEVC shows ;

Intra PU sizes
4x4
8x8
16x16
32x32

Inter PU sizes
8x8
8x16
8x32
16x8
16x16
16x32
24x32
32x8
32x16
32x24
32x32

Meanwhile, x265 (Very Fast preset) shows ;
Intra PU sizes
4x4
8x8
16x16
32x32

Inter PU sizes
8x8
16x16
32x32
64x64

I wonder why NVENC and x265 has different statistic for Inter PU Sizes. Make me wonder about QSV HEVC too....

Edit:
Now, extra statistic from NVENC and X265 very fast preset, ignore the last values, just wanna show what hevc standard and features being used by hardware and software encoders;

NVENC HEVC

0x00000058 Slice I, IDR_W_RADL 0 (60510)
nal_unit_header
forbidden_zero_bit 0
nal_unit_type 19
nuh_layer_id 0
nuh_temporal_id_plus1 1
first_slice_segment_in_pic_flag 1
no_output_of_prior_pics_flag 0
slice_pic_parameter_set_id 0
slice_type 2
slice_qp_delta -9
slice_loop_filter_across_slices_enabled_flag 1

0x0000ecb6 Slice P, TRAIL_R 1 (390)
nal_unit_header
forbidden_zero_bit 0
nal_unit_type 1
nuh_layer_id 0
nuh_temporal_id_plus1 1
first_slice_segment_in_pic_flag 1
slice_pic_parameter_set_id 0
slice_type 1
slice_pic_order_cnt_lsb 1
short_term_ref_pic_set_sps_flag 0
inter_ref_pic_set_prediction_flag 0
num_negative_pics 1
num_positive_pics 0
delta_poc_s0_minus1[0] 0
used_by_curr_pic_s0_flag[0] 1
num_ref_idx_active_override_flag 0
cabac_init_flag 0
five_minus_max_num_merge_cand 0
slice_qp_delta -5
slice_loop_filter_across_slices_enabled_flag 1


x265 Very Fast preset with --aq-strength 0.6 --bframes 16 --rc-lookahead 21 --ref 5 --psy-rd 0.3

0x0000046a Slice I, IDR_W_RADL 0 (90185)
nal_unit_header
forbidden_zero_bit 0
nal_unit_type 19
nuh_layer_id 0
nuh_temporal_id_plus1 1
first_slice_segment_in_pic_flag 1
no_output_of_prior_pics_flag 0
slice_pic_parameter_set_id 0
slice_type 2
slice_sao_luma_flag 1
slice_sao_chroma_flag 1
slice_qp_delta -3
slice_loop_filter_across_slices_enabled_flag 1
num_entry_point_offsets 16
offset_len_minus1 12
entry_point_offset_minus1[0] 6288
entry_point_offset_minus1[1] 5988
entry_point_offset_minus1[2] 5103
entry_point_offset_minus1[3] 5581
entry_point_offset_minus1[4] 6711
entry_point_offset_minus1[5] 4489
entry_point_offset_minus1[6] 3397
entry_point_offset_minus1[7] 3618
entry_point_offset_minus1[8] 3997
entry_point_offset_minus1[9] 4412
entry_point_offset_minus1[10] 5210
entry_point_offset_minus1[11] 6472
entry_point_offset_minus1[12] 6287
entry_point_offset_minus1[13] 5020
entry_point_offset_minus1[14] 5821
entry_point_offset_minus1[15] 5984

0x000164b3 Slice P, TRAIL_R 1 (213)
nal_unit_header
forbidden_zero_bit 0
nal_unit_type 1
nuh_layer_id 0
nuh_temporal_id_plus1 1
first_slice_segment_in_pic_flag 1
slice_pic_parameter_set_id 0
slice_type 1
slice_pic_order_cnt_lsb 17
short_term_ref_pic_set_sps_flag 0
num_negative_pics 1
num_positive_pics 0
delta_poc_s0_minus1[0] 16
used_by_curr_pic_s0_flag[0] 1
slice_temporal_mvp_enabled_flag 1
slice_sao_luma_flag 1
slice_sao_chroma_flag 1
num_ref_idx_active_override_flag 0
luma_log2_weight_denom 7
delta_chroma_log2_weight_denom -1
luma_weight_l0_flag[0] 0
chroma_weight_l0_flag[0] 0
five_minus_max_num_merge_cand 3
slice_qp_delta -3
slice_loop_filter_across_slices_enabled_flag 1
num_entry_point_offsets 16
offset_len_minus1 4
entry_point_offset_minus1[0] 18
entry_point_offset_minus1[1] 11
entry_point_offset_minus1[2] 12
entry_point_offset_minus1[3] 11
entry_point_offset_minus1[4] 14
entry_point_offset_minus1[5] 9
entry_point_offset_minus1[6] 8
entry_point_offset_minus1[7] 7
entry_point_offset_minus1[8] 6
entry_point_offset_minus1[9] 4
entry_point_offset_minus1[10] 8
entry_point_offset_minus1[11] 11
entry_point_offset_minus1[12] 11
entry_point_offset_minus1[13] 8
entry_point_offset_minus1[14] 9
entry_point_offset_minus1[15] 14

0x00016588 Slice B, TRAIL_R 2 (125)
nal_unit_header
forbidden_zero_bit 0
nal_unit_type 1
nuh_layer_id 0
nuh_temporal_id_plus1 1
first_slice_segment_in_pic_flag 1
slice_pic_parameter_set_id 0
slice_type 0
slice_pic_order_cnt_lsb 9
short_term_ref_pic_set_sps_flag 0
num_negative_pics 1
num_positive_pics 1
delta_poc_s0_minus1[0] 8
used_by_curr_pic_s0_flag[0] 1
delta_poc_s1_minus1[1] 7
used_by_curr_pic_s1_flag[1] 1
slice_temporal_mvp_enabled_flag 1
slice_sao_luma_flag 1
slice_sao_chroma_flag 1
num_ref_idx_active_override_flag 0
mvd_l1_zero_flag 0
collocated_from_l0_flag 0
five_minus_max_num_merge_cand 3
slice_qp_delta -2
slice_loop_filter_across_slices_enabled_flag 1
num_entry_point_offsets 16
offset_len_minus1 3
entry_point_offset_minus1[0] 10
entry_point_offset_minus1[1] 6
entry_point_offset_minus1[2] 5
entry_point_offset_minus1[3] 6
entry_point_offset_minus1[4] 6
entry_point_offset_minus1[5] 5
entry_point_offset_minus1[6] 4
entry_point_offset_minus1[7] 4
entry_point_offset_minus1[8] 3
entry_point_offset_minus1[9] 3
entry_point_offset_minus1[10] 4
entry_point_offset_minus1[11] 4
entry_point_offset_minus1[12] 4
entry_point_offset_minus1[13] 5
entry_point_offset_minus1[14] 4
entry_point_offset_minus1[15] 6


See how many missing features from Nvenc compared to x265?

....now if only someone can post QSV HEVC sample....

[x265_Project]

Intel's Kaby Lake (Xeon only) next generation which is due sometime late 2016/Q1 2017 is supposed to bring AVX512 to the table, I wonder how long will it take until x265 can take advantage of that instruction set? ;)
My understanding is that AVX512 will only be available in Skylake Xeons (code named Purley). Support for AVX512 in x265 depends on when we have access to working samples, and enough funding to cover the optimization effort. Of course, contributions are also welcomed.

If I understand correctly, Kaby Lake is a consumer (Core i7, i5, i3, etc.) refresh of Skylake, with more powerful graphics and Quicksync hardware. It won't support AVX512. From news reports, Purley Xeons that support AVX512 will be fairly massive, using a new LGA 3647 socket.
http://www.tomshardware.com/news/intel-xeon-skylake-purley-cpu,31980.html

[littlepox]

I'm not so concerned about speed yet. For x264/x265, speed, quality and bit-rate are always linked together, having one improved automatically improves the other two:

Previously we used almost --preset slower to guarantee the quality/bit-rate, but now since our test suggest x265 has improved a lot, after some tuning we decided to adopt --preset slow, now we can encode faster, smaller and better (in quality) than before.

Given the current situation of x265, the focus should still be the quality optimization and tuning exploration, at least my team and I find it most useful and efficient in practice.

[pingfr]

My understanding is that AVX512 will only be available in Skylake Xeons (code named Purley).

Yuppers.

Want bleeding top-end features? Pay top notch dollars for 'em or get shafted.

<rant>
This is what happens when large companies such as Intel get monopoly.
</rant>

[pingfr]

I'm not so concerned about speed yet. For x264/x265, speed, quality and bit-rate are always linked together, having one improved automatically improves the other two:

Previously we used almost --preset slower to guarantee the quality/bit-rate, but now since our test suggest x265 has improved a lot, after some tuning we decided to adopt --preset slow, now we can encode faster, smaller and better (in quality) than before.

Given the current situation of x265, the focus should still be the quality optimization and tuning exploration, at least my team and I find it most useful and efficient in practice.

A typical movie is usually shot at 23fps to 25fps, I think it's safe to say all of us would be satisfied with x265 the day we manage to encode a movie at the 25fps rate with either --preset slower/veryslow/placebo with a 25% to 50% compression ratio over a x264 counterpart encode.

Yeah okay I think I'm dreaming awake here.. :)

[littlepox]

A typical movie is usually shot at 23fps to 25fps, I think it's safe to say all of us would be satisfied with x265 the day we manage to encode a movie at the 25fps rate with either --preset slower/veryslow/placebo with a 25% to 50% compression ratio over a x264 counterpart encode.

Yeah okay I think I'm dreaming awake here.. :)

Try XviD or Easy Real Producer(RV10 encoder), I think the ultra settings can do 20+fps easily with i7 CPUs, and it is superior than its MPEG2 counterparts with 20+% of compression ratio.

But are you willing to use them seriously nowadays?:p

[x265_Project]

I see....cause I wanna verify type of Intra + Inter PU sizes, SAO and 64X64 CU size support for QSV HEVC encode.

NVENC HEVC shows ;

Intra PU sizes
4x4
8x8
16x16
32x32

Inter PU sizes
8x8
8x16
8x32
16x8
16x16
16x32
24x32
32x8
32x16
32x24
32x32

Meanwhile, x265 (Very Fast preset) shows ;
Intra PU sizes
4x4
8x8
16x16
32x32

Inter PU sizes
8x8
16x16
32x32
64x64

I wonder why NVENC and x265 has different statistic for Inter PU Sizes. Make me wonder about QSV HEVC too....

CU, TU and PU sizes are controlled by several x265 parameters.
--ctu specifies the maximum CTU size. Default is 64x64. Making --ctu smaller limits the range of CTU sizes that x265 will evaluate and encode, allowing encoding to go faster, but decreasing encoding efficiency.
--min-ctu specifies the minimum CTU size. Default is 8x8. Making --min-ctu larger limits the range of CTU sizes that x265 will evaluate and encode, allowing encoding to go faster, but decreasing encoding efficiency.
--rect allows x265 to analyze and encode rectangular partitions (Nx2N or 2NxN - 16x32, 32x16, 8x16, 16x8, etc.) Default is --no-rect, meaning that x265 will only evaluate square partitions.
--amp allows x265 to analyze and encode asymmetric partitions (2N×nU, 2N×nD, nL×2N, and nR×2N - 32x8, 24x32, 32x24, 32,x8, etc.)
--tu-inter-depth determines how far down the quad tree x265 will analyze and encode inter-coded prediction units (PUs). A higher value will allow smaller prediction units to be evaluated and encoded, providing for higher encoding efficiency, but taking longer to evaluate.
--tu-intra-depth determines how far down the quad tree x265 will analyze and encode intra-coded prediction units (PUs). A higher value will allow smaller prediction units to be evaluated and encoded, providing for higher encoding efficiency, but taking longer to evaluate.

--rect and --amp are off for --preset veryfast. If you analyze the results of x265 --preset veryslow or placebo, you'll see the full range of partition shapes and sizes.

For more information on how HEVC blocks are partitioned, see http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.352.1947&rep=rep1&type=pdf

[littlepox]

CU, TU and PU sizes are controlled by several x265 parameters.
--ctu specifies the maximum CTU size. Default is 64x64. Making --ctu smaller limits the range of CTU sizes that x265 will evaluate and encode, allowing encoding to go faster, but decreasing encoding efficiency.
--min-ctu specifies the minimum CTU size. Default is 8x8. Making --min-ctu larger limits the range of CTU sizes that x265 will evaluate and encode, allowing encoding to go faster, but decreasing encoding efficiency.
--rect allows x265 to analyze and encode rectangular partitions (Nx2N or 2NxN - 16x32, 32x16, 8x16, 16x8, etc.) Default is --no-rect, meaning that x265 will only evaluate square partitions.
--amp allows x265 to analyze and encode asymmetric partitions (2N×nU, 2N×nD, nL×2N, and nR×2N - 32x8, 24x32, 32x24, 32,x8, etc.)
--tu-inter-depth determines how far down the quad tree x265 will analyze and encode inter-coded prediction units (PUs). A higher value will allow smaller prediction units to be evaluated and encoded, providing for higher encoding efficiency, but taking longer to evaluate.
--tu-intra-depth determines how far down the quad tree x265 will analyze and encode intra-coded prediction units (PUs). A higher value will allow smaller prediction units to be evaluated and encoded, providing for higher encoding efficiency, but taking longer to evaluate.

--rect and --amp are off for --preset veryfast. If you analyze the results of x265 --preset veryslow or placebo, you'll see the full range of partition shapes and sizes.

Speaking of these, recently our test seem to find some wired artifacts by limiting --ctu 32 --max-tu-size 16; it is not likely due to any "encoding efficiency decrease".

We're testing further on that and probably report an issue if we are ready to get the problems specified.

[burfadel]

AVX-512 refers to several groups of instructions. AVX-512 will be available on Cannonlake supposedly, but only some groups. Zen or Zen+ May introduce other features.

[x265_Project]

Yuppers.
Want bleeding top-end features? Pay top notch dollars for 'em or get shafted.

Implementing support for a new Single Instruction Multiple Data (SIMD) instruction set isn't just a "feature", as it is in software. It isn't trivial to design and implement a new SIMD instruction set. It involves building logic units with a whole new architecture that can operate on data words that are twice as wide as before (512 bits instead of 256 bits), in a single clock cycle! This is hard stuff, man. It takes a lot of silicon, and a lot of power to do twice as much work per clock cycle as you were able to do in the previous generation.

[JohnLai]

CU, TU and PU sizes are controlled by several x265 parameters.
--ctu specifies the maximum CTU size. Default is 64x64. Making --ctu smaller limits the range of CTU sizes that x265 will evaluate and encode, allowing encoding to go faster, but decreasing encoding efficiency.
--min-ctu specifies the minimum CTU size. Default is 8x8. Making --min-ctu larger limits the range of CTU sizes that x265 will evaluate and encode, allowing encoding to go faster, but decreasing encoding efficiency.
--rect allows x265 to analyze and encode rectangular partitions (Nx2N or 2NxN - 16x32, 32x16, 8x16, 16x8, etc.) Default is --no-rect, meaning that x265 will only evaluate square partitions.
--amp allows x265 to analyze and encode asymmetric partitions (2N×nU, 2N×nD, nL×2N, and nR×2N - 32x8, 24x32, 32x24, 32,x8, etc.)
--tu-inter-depth determines how far down the quad tree x265 will analyze and encode inter-coded prediction units (PUs). A higher value will allow smaller prediction units to be evaluated and encoded, providing for higher encoding efficiency, but taking longer to evaluate.
--tu-intra-depth determines how far down the quad tree x265 will analyze and encode intra-coded prediction units (PUs). A higher value will allow smaller prediction units to be evaluated and encoded, providing for higher encoding efficiency, but taking longer to evaluate.

--rect and --amp are off for --preset veryfast. If you analyze the results of x265 --preset veryslow or placebo, you'll see the full range of partition shapes and sizes.

Oh....I see...using preset very slow (or --rect --amp) does shows all the shapes and sizes. It is really slow though....1 fps...ouch. I guess there is tradeoff for speed/quality/efficiency.

Interestingly....for simple comparison with nvenc (all other things being equal), even without all those SAO, --rect, --amp and B-frame being used, x265 still outperform NVENC in term of quality per bit in CQP mode (except the encoding speed XD)

[Khun_Doug]

XVID nowadays, with HD source?? Just say no. Really.

So on this matter of speed, I have been testing some of the various suggestions for the as yet not implemented --tune film recommendations. I'm using an i7 3.4 Ghz with RAID data space and SSD for OS, and AMD HD 7850 GPU. Generally I have never seen this machine be slow. And it will be replaced with something faster sometime this year, retired to be a Kodi media center box.

I tried encoding a 4:32 color HD sample. In X264 using CRF 20, preset slow and tune film, the encode was 9 minutes. Using X265 2.0+4 64 bit 10 bit, this same clip using CRF 20 and preset slow took 45 minutes. When I added some of the suggested tweaks the encode jumped to more than 80 minutes. I watched the encoder report fps of ~ 1.5.

I can safely say that, visually, the X264 and X265 (without the extra tweaks) at CRF 20 looked so nearly identical that the difference was trivial, to my eyes. I tried using 2 pass encodes and noticed the same thing. But the encode speeds were so slow using X265 that I estimated encoding the full HD source would take roughly 48 hours.

If it matters what tools I am using, I have been staying with Hybrid for the X265 tests since it allows me to use whatever .exe I choose. Handbrake is my favorite for X264 HD encodes, but doesn't have the latest X265 libraries.

[herbert]

Thanks again, littlepox.

Depending on the chosen speed preset the following options get enabled or disabled:

--no-rskip --limit-modes --rect

Do you have any data as to the usefulness of the above options yet?

[mandarinka]

https://mailman.videolan.org/pipermail/x265-devel/2016-July/010532.html

--qpmin and --qpmax are coming it seems. Thanks for that, MCW, I used the latter option a lot with x264 (and requested it for x265). Not sure when I'll get to try how useful it is for high quality encodes these days though, it will be tough (and less time than in the past for that...). Too bad that libavcodec doesn't support QP visualizations for HEVC like with H.264 I think. The free stream analyser IIRC also doesn't show block QPs.

[benwaggoner]

A typical movie is usually shot at 23fps to 25fps, I think it's safe to say all of us would be satisfied with x265 the day we manage to encode a movie at the 25fps rate with either --preset slower/veryslow/placebo with a 25% to 50% compression ratio over a x264 counterpart encode.
Film==24p. It might get slowed down to 23.976 for NTSC or sped up to 25p for PAL, but 99.99% of all "filmed" content is shot 24p.

Are you talking 1080p24 above? I believe slower in realtime is certainly achievable on a modern multicore CPU. It might require a dual socket system and AVX2 instructions.

[pingfr]

Are you talking 1080p24 above? I believe slower in realtime is certainly achievable on a modern multicore CPU. It might require a dual socket system and AVX2 instructions.

Welp, I'm on a quad multi socketed CPU system (with AVX but without AVX2) and God no, no matter what preset I use, I can see two things happening:

- Average speed is about 15fps, nowhere near real time encoding can be achieved.
- The core/threads are under-utilized by a single running instance of x265.

And yes, source content is 1080p23.976, preset used is either slower or veryslow using littlepox's homebrewn "--tune film" parameters and x265 is 1.9+200'ish.

It's neither impossible to saturate the 60 available cores nor impossible to do real time encoding.

However, as I either get rid of littlepox's tweaks or start using faster presets such as medium, I see faster encoding and can witness core utilization/load going up dramatically, real time is most likely do'able at veryfast ultrafast presets... at the cost of quality.

In that case, the resulted encode looks far worse though.

If it was just me, I'd stress that further NUMA optimization (or a full rewrite and a different approach) would be a necessity to fully harness the potential power of x265, but that's just me.

[microchip8]

@pingfr

Have you tried using pme and pmode ?

[benwaggoner]

Welp, I'm on a quad multi socketed CPU system (with AVX but without AVX2) and God no, no matter what preset I use, I can see two things happening:

- Average speed is about 15fps, nowhere near real time encoding can be achieved.
- The core/threads are under-utilized by a single running instance of x265.
Quad-socket is definitely sub-optimal for a single 1080p stream. Dual-sockets will easily saturate the threads and offer better per-core performance, so net throughput is quite a bit better. Some of the newer 8-core single socket enthusiast Broadwell parts might actually be the optimal for throughput at 1080p.

As was suggested --pmode will almost certainly help in your use case, and perhaps even --pme with that many threads. Even when encoding on my quad-core Haswell laptop, I need to use --pmode to get fast performance for sub-SD frame sizes at the higher presets.

If it was just me, I'd stress that further NUMA optimization (or a full rewrite and a different approach) would be a necessity to fully harness the potential power of x265, but that's just me.
x265 is way better NUMA optimized than x264, and has a lot of features to manage NUMA well, especially when doing parallel encoding.

It's just that a lot of parallelization options wind up having a quality impact, so they're off in the higher presets.

[littlepox]

Thanks again, littlepox.

Depending on the chosen speed preset the following options get enabled or disabled:

--no-rskip --limit-modes --rect

Do you have any data as to the usefulness of the above options yet?

--limit-modes is enabled by default at slower or above;
--no-rskip is intolerably slow with little improvement.

[littlepox]

--tune film Update:

--crf 18 --ctu 32 --pbratio 1.2 --cbqpoffs -3 --crqpoffs -3 --no-sao --subme 3 --b-intra --no-amp --weightb --aq-mode 3 --aq-strength 0.9 --rd 4 --psy-rd 2.5 --psy-rdoq 4.0 --rdoq-level 2 --rc-lookahead 80 --qcomp 0.65 --no-strong-intra-smoothing --limit-modes

Reintroduce --ctu 32 since further tests suggest the buggy term is --max-tu-size 16, while --ctu 32 is innocent.
Increase psy 2.0:3.0 to 2.5:4.0 which should further reduce blurriness, enhancing visual quality. (bit-rate should increase as well, but worthy)
Add --limit-modes in case someone enables --rect at preset medium (which I strongly DISRECOMMEND; --rect should only be used at preset slower or above)

At the same time, here we go for a --tune animation:

--crf 18 --ctu 32 --ref 4 --bframes 6 --pbratio 1.2 --cbqpoffs -3 --crqpoffs -3 --no-sao --subme 3 --b-intra --no-amp --weightb --aq-mode 3 --aq-strength 0.8 --rd 4 --psy-rd 1.8 --psy-rdoq 1.5 --rdoq-level 2 --rc-lookahead 80 --qcomp 0.65 --no-strong-intra-smoothing --limit-modes

[mandarinka]

What sorts of artifacts do you get from --max-tu-size 16? I encoded some stuff with may/june builds so maybe I could look for them.

[Barough]

x265-v2.0+5-98a948623fdc (http://www91.zippyshare.com/v/MZaPppbS/file.html) (MSYS/MinGW, GCC 5.4.0, 32 & 64bit 8/10/12bit multilib EXEs)

[benwaggoner]

--tune film Update:

--crf 18 --ctu 32 --pbratio 1.2 --cbqpoffs -3 --crqpoffs -3 --no-sao --subme 3 --b-intra --no-amp --weightb --aq-mode 3 --aq-strength 0.9 --rd 4 --psy-rd 2.5 --psy-rdoq 4.0 --rdoq-level 2 --rc-lookahead 80 --qcomp 0.65 --no-strong-intra-smoothing --limit-modes

Reintroduce --ctu 32 since further tests suggest the buggy term is --max-tu-size 16, while --ctu 32 is innocent.
Increase psy 2.0:3.0 to 2.5:4.0 which should further reduce blurriness, enhancing visual quality. (bit-rate should increase as well, but worthy)
Add --limit-modes in case someone enables --rect at preset medium (which I strongly DISRECOMMEND; --rect should only be used at preset slower or above)

For film, what about --ctu 64 --rdpenalty 1 --qg-size 32? That'll still allow those big 32x32 intra blocks, but bias strongly against them so they only get used when they really pay off, and still give you adaptive quant at the 32x32 level.

As for --limit-modes, that only does something when --amp or --rect are on anyway, which is make them a lot faster with a tiny reduction in efficiency.

Also, note there's an issue in --aq-mode 3 currently which causes it to use WAY more bits in CRF mode than --aq-mode 2.

[benwaggoner]

What sorts of artifacts do you get from --max-tu-size 16? I encoded some stuff with may/june builds so maybe I could look for them.
I wouldn't expect artifacts so much as reduced compression efficiency.

[K.i.N.G]

Wow, that '--no-sao' switch drasticly increased detail (grain) retention for me!

Anyone more knowledgeable on here can tell me what it does... Google results get kinda too technical for me (i dont understand the terminologie)

[RainyDog]

Also, note there's an issue in --aq-mode 3 currently which causes it to use WAY more bits in CRF mode than --aq-mode 2.

Hasn't --aq-mode 3 always been like this even with x264?

[RainyDog]

For film, what about --ctu 64 --rdpenalty 1 --qg-size 32? That'll still allow those big 32x32 intra blocks, but bias strongly against them so they only get used when they really pay off, and still give you adaptive quant at the 32x32 level.

Also note that littlepox's --tune-film is geared for 1080p.

[gamebox]

@K.i.N.G: SAO is an encoding step that tries to compensate against some typical encoding artifacts seen in most MPEG compression technologies to date (h265 as well), by altering the image after it is compressed. It changes the way mostly edges look, trying to "iron" (smooth) some defects introduced by compression - the ones that are "expected" to be present in those areas. It does a very good job, but sadly seems to eliminate a lot of details by error too, as it can not define compression defects in an image "precisely" (it would take a lot of data to describe them), so it only has some "general impression" or "expectation" about the way they should look.

A small question about x265 "blurriness" most people complain about - has anyone tried pre-filtering the video before sending it to x265, by applying some sharpening to counter-effect the issue? To me, problem seems motion estimation related - defining further steps in sub-pixel motion precision over previous technologies increased efficiency but introduced many calculation steps that ended up smoothing the pixels too much. That's just an amateur guess, I might be wrong. The issue of x265 sharpness seems to influence "detail clarity" as well as (or even more than) "edge contrast" - it seems to need something like HDR filter in preprocessing as well to improve detail "amplitudes" by actually "over-accenting" them before compression takes place.

[Barough]

x265-v2.0+8-669dc9bfe7eb (http://www70.zippyshare.com/v/BEulNFhn/file.html) (MSYS/MinGW, GCC 5.4.0, 32 & 64bit 8/10/12bit multilib EXEs)

[K.i.N.G]

@K.i.N.G: SAO is an encoding step that tries to compensate against some typical encoding artifacts seen in most MPEG compression technologies to date (h265 as well), by altering the image after it is compressed. It changes the way mostly edges look, trying to "iron" (smooth) some defects introduced by compression - the ones that are "expected" to be present in those areas. It does a very good job, but sadly seems to eliminate a lot of details by error too, as it can not define compression defects in an image "precisely" (it would take a lot of data to describe them), so it only has some "general impression" or "expectation" about the way they should look.

A small question about x265 "blurriness" most people complain about - has anyone tried pre-filtering the video before sending it to x265, by applying some sharpening to counter-effect the issue? To me, problem seems motion estimation related - defining further steps in sub-pixel motion precision over previous technologies increased efficiency but introduced many calculation steps that ended up smoothing the pixels too much. That's just an amateur guess, I might be wrong. The issue of x265 sharpness seems to influence "detail clarity" as well as (or even more than) "edge contrast" - it seems to need something like HDR filter in preprocessing as well to improve detail "amplitudes" by actually "over-accenting" them before compression takes place.

Hey thanks for the very clear explanation. :)

I dont know about other ppl, but to my eyes, without SAO it still looks allot better than x264 at the same bitrate (arround 4500kb/s) and I dont notice any prominent artifacts (much less than the x264 encode for sure). At least, on my laptop screen it does... I'll be testing it later on my TV.

To me it looks like the SAO switch could use some more dev-tweaking and/or should only be used with very low bitrates... It is way to aggressive right now, imho.

Im using a slightly tuned 'slower' preset so it is allot slower than x264 (it's encoding 1080p at 1.34fps right now).

I'll be testing some more and try to get it faster. If things keep looking this promessing then this will probably make me switch over to x265.

Im using version 1.9+200-6098ba3e0cf16b11 (with staxrip).
Id like to test v2.0 but im still waiting for staxrip to update and have optimized UI for v2.0

[Barough]

Regarding --ag-mode 3 so do i have to say that it can do more 'harm' then good some times. Have noticed on and off on some of my darker test videos that at the edges of dark & very dark areas there can be some kind of 'ghosting'. With -aq-mode 2 so have there been no 'ghosting' like 60 % of the time.

Preset : Medium
CRF : 19-22
Custom command lines used :
--me 3 --ref 4 --aq-mode 2 --aq-strength 2 --rdoq-level 1 --psy-rdoq 4 --rc-lookahead 80
--me 3 --ref 4 --ctu 32 --no-sao --aq-mode 3 --aq-strength 2 --rdoq-level 1 --psy-rdoq 4 --rc-lookahead 80

[gamebox]

@ K.I.N.G. You're welcome :)

You will not notice compression artifacts in x265 as often and as easily as in x264 partially because x265 encodes video differently. Artifacts in 64x64 and 32x32 image blocks are different than those in x264, slightly "dissolved" and sometimes covering greater areas.

Apart from that, your bitrate can be considered "generous", and not many artifacts should appear. I typically encode at bitrates closer to limits of "acceptable" image quality and "sane" quants - namely 3 Mbps for 1080p - and I could clearly see benefits from SAO. SAO will probably be very useful for broadcasting and streaming as it will "mask" disturbing artifacting during high motion scenes, when limits of data throughput force encoder to temporarily use extreme levels of compression.

[littlepox]

OK here is a good demo about x265's max-tu-size 16 bug:

https://www.dropbox.com/sh/slftjn7meozs1f3/AADokCEEhMm_aW4IEE-V6P3ba?dl=0

source.mkv is the source with 99 frames; denoted as source.
dft_20.hevc is encoded with --preset slow --no-sao --crf 20; denoted as A
dft_19.5_maxtu16.hevc is encoded with --preset slow --no-sao --crf 19.5 --max-tu-size 16; denoted as B

The size difference of them is 0.6%.

In many frames (e.g, frame 17,21,63,83...) You see unpleasant patterns in the flat area of B, like worms on the wall, especially noticable when the grain is lost and the area is clean. In A the situation is much better.

This is A(part of frame 83):
http://img.2222.moe/images/2016/07/19/A.png

This is B(part of frame 83):
http://img.2222.moe/images/2016/07/19/B.png

Asking my friend to drop an issue to the https://bitbucket.org/multicoreware/x265/issues?status=new&status=open (done)

[littlepox]

For film, what about --ctu 64 --rdpenalty 1 --qg-size 32? That'll still allow those big 32x32 intra blocks, but bias strongly against them so they only get used when they really pay off, and still give you adaptive quant at the 32x32 level.

After clearing the --max-tu-size bug, we tested various possibilities like --ctu 32, --ctu 64 + --rdpenalty 0/1/2
They are indifferent both quality-wise and size-wise, at least with our test settings.
The only difference is --ctu 32 is faster and better paralleled; that's why it is reintroduced.


Also, note there's an issue in --aq-mode 3 currently which causes it to use WAY more bits in CRF mode than --aq-mode 2.
That's why we suggest to tweak the aq-strength -0.2 for aqmode=3 and +0.2 for aqmode=2. With reduced strength, the bit-rate increase is acceptable and worthy.


Regarding --ag-mode 3 so do i have to say that it can do more 'harm' then good some times. Have noticed on and off on some of my darker test videos that at the edges of dark & very dark areas there can be some kind of 'ghosting'. With -aq-mode 2 so have there been no 'ghosting' like 60 % of the time.

Preset : Medium
CRF : 19-22
Custom command lines used :
--me 3 --ref 4 --aq-mode 2 --aq-strength 2 --rdoq-level 1 --psy-rdoq 4 --rc-lookahead 80
--me 3 --ref 4 --ctu 32 --no-sao --aq-mode 3 --aq-strength 2 --rdoq-level 1 --psy-rdoq 4 --rc-lookahead 80


having aq-strength outside [0.5,1.5] is probably a bad idea, aq-strength=2 for mode=3 is WAY too large.
Also, you need to control your variables. I don't see it reasonable to use different sao settings for the test.

[Barough]

having aq-strength outside [0.5,1.5] is probably a bad idea, aq-strength=2 for mode=3 is WAY too large.
Also, you need to control your variables. I don't see it reasonable to use different sao settings for the test.

Still doing tests here. Have lowered the aq-strength to between 1.2 and 1.5. Will do a full movie encode tonight of a dark movie. Will see what my bookshelf have 2 offer.

Also waiting for a m8 that work with video to get back to me. He's not into HEVC so much yet as i understand but his knowledge in the world of video encoding is good.

[littlepox]

Still doing tests here. Have lowered the aq-strength to between 1.2 and 1.5. Will do a full movie encode tonight of a dark movie. Will see what my bookshelf have 2 offer.

Also waiting for a m8 that work with video to get back to me. He's not into HEVC so much yet as i understand but his knowledge in the world of video encoding is good.

I think 1.2~1.5 is still too high for aqmode 3. For me, I'd use it as 0.9.

[divxmaster]

@K.i.N.G: SAO is an encoding step that tries to compensate against some typical encoding artifacts seen in most MPEG compression technologies to date (h265 as well), by altering the image after it is compressed. It changes the way mostly edges look, trying to "iron" (smooth) some defects introduced by compression - the ones that are "expected" to be present in those areas. It does a very good job, but sadly seems to eliminate a lot of details by error too, as it can not define compression defects in an image "precisely" (it would take a lot of data to describe them), so it only has some "general impression" or "expectation" about the way they should look.

A small question about x265 "blurriness" most people complain about - has anyone tried pre-filtering the video before sending it to x265, by applying some sharpening to counter-effect the issue? To me, problem seems motion estimation related - defining further steps in sub-pixel motion precision over previous technologies increased efficiency but introduced many calculation steps that ended up smoothing the pixels too much. That's just an amateur guess, I might be wrong. The issue of x265 sharpness seems to influence "detail clarity" as well as (or even more than) "edge contrast" - it seems to need something like HDR filter in preprocessing as well to improve detail "amplitudes" by actually "over-accenting" them before compression takes place.

@gamebox, I have tested sharpening before processing, and it makes a huge difference. Ironically it is through QTGMC in vapoursynth (I have just noticed that a few select cgi scenes in later seasons of ds9 are interlaced...ick!)
The side effect of the qtgmc deinterlacing is a strong sharpen and also seems to be colour enhancement. This does increase bitrate +25%, so I increased CRF from 21 to 22, but it still looks way better than no qtgmc/sharpen and is slightly lower bitrate (5%) as a bonus.
Specifically ds9 really benefits from degrain also 'vid = haf.SMDegrain(vid, tr=3,thSAD=300,RefineMotion=True,contrasharp=True,pel=2)'
params are --preset slow --rskip --crf 22 --no-sao --aq-mode 2 --rdoq-level 2 --psy-rd 2 --psy-rdoq 1.1 --qg-size 32 --tu-intra-depth 3 --merange 27 --weightb --early-skip --fast-intra --b-intra --tskip --tskip-fast --ref 6 --bframes 6 --max-merge 5 --min-keyint 23 --keyint 288 --deblock -1:-1 --no-open-gop

[nakTT]

x265-v2.0+8-669dc9bfe7eb (http://www70.zippyshare.com/v/BEulNFhn/file.html) (MSYS/MinGW, GCC 5.4.0, 32 & 64bit 8/10/12bit multilib EXEs)
Hi,

Is this the same as the one that can be found in the webpage below?

https://builds.x265.eu

[Barough]

Hi,

Is this the same as the one that can be found in the webpage below?

https://builds.x265.eu
No. It's 'my' compiles made through media-autobuild suite.

Sent from my Samsung Galaxy S7 edge via Tapatalk