x265 HEVC Encoder

[Tommy Carrot]

Have you tried set the arguments to high detail preserved ones.Film grain, a special noise, can be preserved, which means the fine details can be reserved as well......
It doesn't matter which parameters i change (i've tried all kind of settings), x265 is not able reach transparency where x264 can.
x265 is designed to encode up to 4k even 8k output, the pores is no need to be detailed as much as x264 does on the common scale......
H.265 was designed to have better efficiency than h.264 in all cases. It's true it has bigger advantage at higher res, but there is no reason it should be worse at 720p or even at sd videos.
Currently, because of the hardware limitation, we compare the x265 fast-encode with x264 placebo......

All you need is to find the right tuning parameters...... to early to coffin the newbies.....
No, i compared x264 veryslow and placebo with x265 veryslow and placebo presets (or even slower settings manually). Again, it doesn't matter which settings i used, x265 cannot compete with x264 for high quality encodings. This can easily change as the development progresses, but the fact is, X265 is simply not properly tuned for high quality purposes yet.

[mandarinka]

I'd say that grain tends to bring up false details and it also makes the image appear sharper than it actually is. Noise is a different thing, it can be a problem.

It'S the other way around. Noisy/dirty picture looks natural, but as soon as you put a video through a denoising filter, everything starts to look like ass ugly plastic.

[Ajvar]

I spent hours to decide how exactly I want to encode my L4D2 gameplay library and while most of the time I compared HEVC I also made two x264 encoding and I do approve that x265 doesn't keep micro textures as good as x264.
x264 -precet placebo, CRF23, FullHD (http://i60.fastpic.ru/big/2015/0219/7b/11d0e1e8560d257a98a6bdb04695637b.png)
x265 -preset very slow, 2pass bitrate of resulted x264, PsyRD-0,7. (http://i60.fastpic.ru/big/2015/0219/7a/e5efb8d4aef0b94521e3a08ad5230b7a.png) Check the wall. x264 retained more details than x265 on same bitrate. Maybe that is because of Psy-RD?
I also encoded into CRF23 and it was WAY-WAY blurred speaking about not just micro details but almost all. However you can't see this in fast action.

Anyway I will do HEVC CRF 22.8 Psy-RD 0.7 FullHD

[foxyshadis]

That explains it then, thanks for the info!

Looks like Zeranoe is updated to 1.5 now.

[x265_Project]

The question still remains: is it even possible to tune x265 towards better grain/noise/whatever retention or is H.265 video just designed so that it is more like an alternative to x264 instead of a pure successor? There doesn't seem to be a clear answer to that anywhere.

Yes, it is possible to tune x265 towards better detail retention. Use a higher quality preset like veryslow, and increase psy-rd strength. But as others have pointed out, the best way to retain details is to use a high enough bit rate. As the bit rate goes lower, something's got to give.

I should point out that the kind of film grain that our high-end customers (movie studios, movie streaming services, etc.) see has much more detail (and energy) than anything a typical consumer user has ever seen. Some of the new 4K uncompressed content that we've seen from classic films ('90s on back to the '30s) has unbelievable levels of film grain (the higher the film scanner resolution, the more grain detail that shows up). If you're dealing with film grain from a DVD or Blu-ray rip, it's already been low-pass filtered by the first noise reduction/encoding process, and so it isn't nearly as challenging as high res film grain from freshly scanned film negatives.

[uneedme]

I spent hours to decide how exactly I want to encode my L4D2 gameplay library and while most of the time I compared HEVC I also made two x264 encoding and I do approve that x265 doesn't keep micro textures as good as x264.
x264 -precet placebo, CRF23, FullHD (http://i60.fastpic.ru/big/2015/0219/7b/11d0e1e8560d257a98a6bdb04695637b.png)
x265 -preset very slow, 2pass bitrate of resulted x264, PsyRD-0,7. (http://i60.fastpic.ru/big/2015/0219/7a/e5efb8d4aef0b94521e3a08ad5230b7a.png) Check the wall. x264 retained more details than x265 on same bitrate. Maybe that is because of Psy-RD?
I also encoded into CRF23 and it was WAY-WAY blurred speaking about not just micro details but almost all. However you can't see this in fast action.

Anyway I will do HEVC CRF 22.8 Psy-RD 0.7 FullHD


I tried and found the deblock [-3 and below] and rdoq [+30 and above] affect the deblurring much (my personal opinion......)

http://pan.baidu.com/s/1jGh5K1K

[Boulder]

Yes, it is possible to tune x265 towards better detail retention. Use a higher quality preset like veryslow, and increase psy-rd strength. But as others have pointed out, the best way to retain details is to use a high enough bit rate. As the bit rate goes lower, something's got to give.Unfortunately, the high enough bitrate is probably more than what x264 requires. I did a short test: encoded the x264 clip in CRF mode and then did a 2-pass encode with x265 with the same avg bitrate.

To me it seems that the I-frames look quite similar but the one after the I-frame is much softer with x265.

x264 : https://drive.google.com/open?id=0BzeF_1syecQwX28xMEV0aVBzZ2s&authuser=0
x265 : https://drive.google.com/open?id=0BzeF_1syecQwTHZuaXFuSlRha2M&authuser=0
x265_tunegrain : https://drive.google.com/open?id=0BzeF_1syecQwTHZuaXFuSlRha2M&authuser=0


x264 settings : c:\x264\x264.exe --stdin y4m - --sar 1:1 --level 4.1 --colormatrix "bt709" --colorprim "bt709" --transfer "bt709" --preset veryslow --crf 18.5 --output f:\temp\captures\x264.h264

x265 settings : c:\x265\x265.exe --pass 2 --y4m --input - --sar 1:1 --colormatrix "bt709" --colorprim "bt709" --transfer "bt709" --preset slow --bitrate 5403 --output f:\temp\captures\x265_tunegrain.h265

(or x265 with --tune grain)

Source cropped and resized to 1280x528 before encoding.
I guess I've been asking if it's possible to combine the grain/detail retention of x264 with the more advanced motion search etc. capabilities of x265. Is this achievable in the future or are the differences too big between the codecs?

[uneedme]

uncompressed content that we've seen from classic films ('90s on back to the '30s) has unbelievable levels of film grain (the higher the film scanner resolution, the more grain detail that shows up). If you're dealing with film grain from a DVD or Blu-ray rip, it's already been low-pass filtered by the first noise reduction/encoding process, and so it isn't nearly as challenging as high res film grain from freshly scanned film negatives.

Grain is Celluloid film's natural beauty. Alot film makers use grain as an expression. you can denoise it, but you cant kill the mood of grain effect......

[grumpy]

I should point out that the kind of film grain that our high-end customers (movie studios, movie streaming services, etc.) see has much more detail (and energy) than anything a typical consumer user has ever seen. Some of the new 4K uncompressed content that we've seen from classic films ('90s on back to the '30s) has unbelievable levels of film grain (the higher the film scanner resolution, the more grain detail that shows up). If you're dealing with film grain from a DVD or Blu-ray rip, it's already been low-pass filtered by the first noise reduction/encoding process, and so it isn't nearly as challenging as high res film grain from freshly scanned film negatives.
If a source is already compromised doesn't that only make it more important to accurately persevere the relatively little detail that it still has?

The codec will see it as more compressible, but you should force it to use more?

[Ma]

I've tested many x265 16bpp builds and I've noticed that the output file (encoded movie) isn't the same for all builds (all with the same settings with --no-info option and 10bpp output).

If you build x265 encoder with GCC with HIGH_BIT_DEPTH option and CMAKE_BUILD_TYPE=Debug (instead of Release) the encoded file is identical to output chromashift's build highbitdepth-msvc2012-64. The output (encoded movie) is indentical if you build with GCC with CMAKE_BUILD_TYPE=Release and -O2 optimize option instead of default -O3.

If you build with GCC with -O3 option the output file (encoded movie) is different and is changing with GCC versions (492-64bit != 482-64bit != 482-32bit).

Is it normal that output file (with --no-info option) differ with compiler options? Maybe the source code shouldn't be compile with -O3 option?

[LoRd_MuldeR]

I've tested many x265 16bpp builds and I've noticed that the output file (encoded movie) isn't the same for all builds (all with the same settings with --no-info option and 10bpp output).

If you build x265 encoder with GCC with HIGH_BIT_DEPTH option and CMAKE_BUILD_TYPE=Debug (instead of Release) the encoded file is identical to output chromashift's build highbitdepth-msvc2012-64. The output (encoded movie) is indentical if you build with GCC with CMAKE_BUILD_TYPE=Release and -O2 optimize option instead of default -O3.

If you build with GCC with -O3 option the output file (encoded movie) is different and is changing with GCC versions (492-64bit != 482-64bit != 482-32bit).

Is it normal that output file (with --no-info option) differ with compiler options? Maybe the source code shouldn't be compile with -O3 option?

First of all, is x265 even supposed to be deterministic (with default settings), i.e. is it supposed to always produce the bit-identical output for the same input? I know that x264 is supposed to be deterministic - but only unless you either pass the "--non-deterministic" option or you enable VBV (note that VBV is inherently nondeterministic, so this is not a bug). There also have been bugs in the past that caused nondeterminism.

Secondly, how did you determine that the outputs differ? Did you only compare the file hashes or did you actually see a difference? Or did you even get decoding errors with one version? Keep in mind that, just because two outputs differ, this doesn't necessarily mean one is "correct" and the other is "wrong". After all, there is an infinite number of "permissible" ways to encode the same source video.

Finally, higher optimization levels enable more "aggressive" optimizations. This raises the chance that something breaks. But it's also possible that, at the higher optimization levels, the compiler generates code that, while not being "wrong" or "broken", produces slightly different output! For example, floating point math is never "100% accurate", due to the limited FP precision (usually 32- or 64-Bit). Thus, in "strict" FP mode, there are tight rules on the rounding of intermediate FP results, in order to make sure that the final result is reproducible. In "fast" FP mode, however, these rules are softened in order to speed-up computations. Thus, "fast" and "strict" mode can produce different results. But, is "strict" more accurate? Not necessarily!

[Ma]

First of all, is x265 even supposed to be deterministic (with default settings), i.e. is it supposed to always produce the bit-identical output for the same input?

I don't know. I prefer deterministic behaviour. If different output it is a bug in source code, we should fight with this bug. If it is by design, I will accept that.

Secondly, how did you determine that the outputs differ? Did you only compare the file hashes or did you actually see a difference? Or did you even get decoding errors with one version?

I've binary compared output files. Output from Intel compiler build was bigger size, all others was the same size (but slightly differ). I've watched all outputs and didn't see any differences nor any decoding errors.

I want to choose the fastest build of 16bpp x265 and was surprised with the fact that the output files was different (with --no-info option).

[x265_Project]

First of all, is x265 even supposed to be deterministic (with default settings), i.e. is it supposed to always produce the bit-identical output for the same input? I know that x264 is supposed to be deterministic - but only unless you either pass the "--non-deterministic" option or you enable VBV (note that VBV is inherently nondeterministic, so this is not a bug). There also have been bugs in the past that caused nondeterminism.

Your hunch is correct. ABR (and CQP and CRF) output is expected to be deterministic with any number of frame encoders. The output of -F2 may be different than the output of -F3 but they will both be deterministic.

The only feature which is deliberately non-deterministic is VBV, which makes rate control decisions mid-frame based on the other partially completed frames. Here even with -F1 VBV is most likely non-deterministic because of WPP parallelism.


Secondly, how did you determine that the outputs differ? Did you only compare the file hashes or did you actually see a difference? Or did you even get decoding errors with one version? Keep in mind that, just because two outputs differ, this doesn't necessarily mean one is "correct" and the other is "wrong". After all, there is an infinite number of "permissible" ways to encode the same source video.

Finally, higher optimization levels enable more "aggressive" optimizations. This raises the chance that something breaks. But it's also possible that, at the higher optimization levels, the compiler generates code that, while not being "wrong" or "broken", produces slightly different output! For example, floating point math is never "100% accurate", due to the limited FP precision (usually 32- or 64-Bit). Thus, in "strict" FP mode, there are tight rules on the rounding of intermediate FP results, in order to make sure that the final result is reproducible. In "fast" FP mode, however, these rules are softened in order to speed-up computations. Thus, "fast" and "strict" mode can produce different results. But, is "strict" more accurate? Not necessarily!

Different compilers can generate different outputs. We only guarantee determinism across encodes made by the same binary, with the same command line options, and without VBV.

Output differences between compilers are likely caused by floating point math divergence in rate control, but we have not verified this.

[phate89]

Yes, it is possible to tune x265 towards better detail retention. Use a higher quality preset like veryslow, and increase psy-rd strength. But as others have pointed out, the best way to retain details is to use a high enough bit rate. As the bit rate goes lower, something's got to give.


I think what everyone is asking is not just pure grain retention. Everyone here knows that with grain you need to give something (so way higher bitrates or slower presets) to have accurate compression.
But the question is:
x265 has the same tools plus other more advanced than x264. Can we expect to get x265 at least as good as x264 with fine details and grain? what's the limit that stops x265 to be as good as x264 at the moment in this scenario?
Because right now I think everyone agree that x264 behave way better at the same bitrate with these 2 cases

[xooyoozoo]

The more precise question is probably "why hasn't x265's psy matched x264's psy".

Core prediction and content weighing tools (CUTree and AQ) perform as well versus x264's equivalence as you'd expect.

[mandarinka]

From my testing, x265 seemed to have problems keeping grain/texture in large 32x32 (it probably happened in 64x64 ones too) blocks. For some reason, in such CUs, it would code almost zero residual*. I didn't investigate it much, but interestingly the problem didn't go away if you disabled 32x32 and larger coding units. So it probably wasn't use of the large blocks as such, but something with analysis...
Damn, I'll have to look into it properly and bugreport some day.

* In stream analyzer you could see that while 16x16 and smaller blocks had normal grainy residual, 32x32 and 64x64 had the residual completely flat without grain.

[Ajvar]

x264 is like grain: gives artifacts but it contains useful details. It's like bad jpeg. While x265 blurs it, bigger bitrate - less blur but blur anyway: like downsampled and then oversampled again.
It would be good if it was possible to move slider details per X bitrate/blur per X bitrate.

Anyway x265 is BETTER for watching video in playback 24fps and above while if you want to KEEP ALL DETAILS in each frame then it's better x264 for now.

I wish new codec would mix both: give details until it is possible but if artifacts become less informative and more... well.. artifacts... it should start blurring like it does now.

[foxyshadis]

From my testing, x265 seemed to have problems keeping grain/texture in large 32x32 (it probably happened in 64x64 ones too) blocks. For some reason, in such CUs, it would code almost zero residual*. I didn't investigate it much, but interestingly the problem didn't go away if you disabled 32x32 and larger coding units. So it probably wasn't use of the large blocks as such, but something with analysis...
Damn, I'll have to look into it properly and bugreport some day.

* In stream analyzer you could see that while 16x16 and smaller blocks had normal grainy residual, 32x32 and 64x64 had the residual completely flat without grain.

Interesting, that would explain why there's so much more detail in really busy areas than simpler parts. The algorithms for testing blocks must have a pretty high threshold before they'll code a residual. It'd be nice to be able to adjust the skipping, even if that lowers quality elsewhere.

[EncodedMango]

I'm curious, is there a specific reason as to why the encoder needs to be provided YUV or Y4M files instead of managing the source itself from common containers? It has reached v1.5 and this is something that's been there for a very long time.

[x265_Project]

I'm curious, is there a specific reason as to why the encoder needs to be provided YUV or Y4M files instead of managing the source itself from common containers? It has reached v1.5 and this is something that's been there for a very long time.
Strictly speaking, a video encoder library accepts only uncompressed video frames as input. Accepting anything else would mean that the other format would first need to be decoded (to uncompressed YUV video). Combining decoder and encoder libraries you can create a transcoding application. Decoders for common input video formats are available from other open source projects. x265 is focused 100% on HEVC encoding. If you're looking for a transcoding application, there are many available that use x265 including FFMPEG, Handbrake, Hybrid, IFME, MeGUI, and StaxRip.

[benwaggoner]

If a source is already compromised doesn't that only make it more important to accurately persevere the relatively little detail that it still has?

The codec will see it as more compressible, but you should force it to use more?
The problem with old 4K masters isn't that there is relatively little detail. There's a LOT of detail. However, it's almost entirely film grain. Imagine blowing up a clean 720p movie to 3840x2160 and then layering a really strong noise synthesis filter on top of it. The energy in the frame is almost all film grain, and very little anything else.

And I'd argue this isn't an accurate experience to what the filmmakers saw or intended. Prints, projectors, and screens of the day certainly weren't capable of getting all the detail into the eyes of the viewer. We see movies with all kind of defects in special effects as well that would have been invisible in an 80's movie theater but which are painfully obvious on a 4K still frame on a high quality TV.

Remember that pausing wasn't even possible on celluloid in theaters, because the sustained heat would burn through the film.

So, the grain that a really high quality pin registered negative scan isn't grain that anyone who made the movie or saw it in theaters ever experienced.

The questions of "artistic intent" and "original experience" turn out to be complex and slippery in practice. No one would argue that we should be synthesizing projector noise or the slightly random positioning per frame caused by the pins not always lining up perfectly. Or flashing each frame three times for 1/144th of a second instead of leaving it on for the full 1/24th of a second. Etcetera.

Would David Lean have used modern fine-grain film stocks if they were available? Would he have shot day-for-night if he had access to modern CCD-based digital cameras? And should that matter? Is it more accurate for Bridge on the River Kwai to retain it's old chemical-based day-for-night shots that look very unrealistic, or to use modern digital effects to make those shots look like it was actually night, as intended?

Should The Philadelphia Story overwhelm viewers by accurately delivering all the grain in the negative, or filter out noise to what would have been seen on the fresh print at opening night at the Ziegfeld? Or something else?

[EncodedMango]

Strictly speaking, a video encoder library accepts only uncompressed video frames as input. Accepting anything else would mean that the other format would first need to be decoded (to uncompressed YUV video). Combining decoder and encoder libraries you can create a transcoding application. Decoders for common input video formats are available from other open source projects. x265 is focused 100% on HEVC encoding. If you're looking for a transcoding application, there are many available that use x265 including FFMPEG, Handbrake, Hybrid, IFME, MeGUI, and StaxRip.
Thanks.

I'm not looking into any HEVC encoders per se, I already use x265 builds provided in this thread or ffmpeg, if needed but I'm not encoding anything to H265 outside of testing yet.

Maybe I am spoiled by x264 transcoders that quietly do avs2yuv and I don't notice

[LigH]

Well, x265 is still so much in the development of the encoder kernel, that there is still "no spare time" (so to say) to add additional decoders to the binary. First the job, then the fun... ;)

Instead, x264 is already more or less "done", so it also had enough time to get some convenience additions, like a native AviSynth interface.

[mandarinka]

Well, ffm2s and libavformat (or what it is) inputs is probably one thing that could be ported over from x264 cleanly/without much changes in functionality needed, if somebody wants to make such patch.

For me, avs2yuv is enough of a solution.

[LigH]

And possibly slightly more convenient at the command line: avs4x26x.

[benwaggoner]

I got a little bored/overcaffeinated this weekend and did a x264 v. x265 test using ultra-placebo settings. It's a pretty definitive demonstration that at a low enough bitrate HEVC can blow H.264 out of the water, even with content with some detail/grain.

http://forum.doom9.org/showthread.php?p=1710758#post1710758

To push x265 beyond placebo, I added:

--cu-lossless, just in case lossless was better in some cases
-F 1, as sometimes frame parallelism can cause intra-frame rate control changes, and why risk it.
--subme 7, because placebo only uses 5
--tskip, because maybe sometimes that mode could be better
--ref 6, as that is the max for my frame size at target level, and placebo only goes up to 5
--rc-lookahead 96, as that's my max GOP duration, and placebo only goes up to 60


Unrelated to quality, I also used --pmode, so it wouldn't run glacially slow using up <20% CPU on my 16-core workstation. Killing frame parallelism is bad for parallelism, and pmode helps get some of it back (bouncing around 50-75% for me). --pme is really parallel, and didn't help throughput.

I doubt the above actually added that much improvement, but the test was all about what was theoretically possible. And I figured it might be useful to call out the knobs that go up to >11.

I saved my log file. I should probably do a straight placebo version and compare results.

[Lyris]

The problem with old 4K masters isn't that there is relatively little detail. There's a LOT of detail. However, it's almost entirely film grain. Imagine blowing up a clean 720p movie to 3840x2160 and then layering a really strong noise synthesis filter on top of it. The energy in the frame is almost all film grain, and very little anything else.

And I'd argue this isn't an accurate experience to what the filmmakers saw or intended. Prints, projectors, and screens of the day certainly weren't capable of getting all the detail into the eyes of the viewer. We see movies with all kind of defects in special effects as well that would have been invisible in an 80's movie theater but which are painfully obvious on a 4K still frame on a high quality TV.

Remember that pausing wasn't even possible on celluloid in theaters, because the sustained heat would burn through the film.

So, the grain that a really high quality pin registered negative scan isn't grain that anyone who made the movie or saw it in theaters ever experienced.

The questions of "artistic intent" and "original experience" turn out to be complex and slippery in practice. No one would argue that we should be synthesizing projector noise or the slightly random positioning per frame caused by the pins not always lining up perfectly. Or flashing each frame three times for 1/144th of a second instead of leaving it on for the full 1/24th of a second. Etcetera.

Would David Lean have used modern fine-grain film stocks if they were available? Would he have shot day-for-night if he had access to modern CCD-based digital cameras? And should that matter? Is it more accurate for Bridge on the River Kwai to retain it's old chemical-based day-for-night shots that look very unrealistic, or to use modern digital effects to make those shots look like it was actually night, as intended?

Should The Philadelphia Story overwhelm viewers by accurately delivering all the grain in the negative, or filter out noise to what would have been seen on the fresh print at opening night at the Ziegfeld? Or something else?

All very valid points, but for me, the deal breaker is that noise and grain reduction is often overdone and even done lightly, it still leaves artefacts behind. (See the otherwise great 1080p BD of Jaws and the "swimming grain" artefacts, or the "powdery grain" on The Godfather - although having said that, combining multiple film sources of various generations is a very good justification for grain reduction).

So while I fully agree that the amount (and sharpness) of grain we end up seeing isn't what was intended (and speaking of what was intended, as you touched on, nobody would vote to watch something degraded to the quality of a theatrical print), it's better than the processed alternative.

[benwaggoner]

All very valid points, but for me, the deal breaker is that noise and grain reduction is often overdone and even done lightly, it still leaves artefacts behind. (See the otherwise great 1080p BD of Jaws and the "swimming grain" artefacts, or the "powdery grain" on The Godfather - although having said that, combining multiple film sources of various generations is a very good justification for grain reduction).

So while I fully agree that the amount (and sharpness) of grain we end up seeing isn't what was intended (and speaking of what was intended, as you touched on, nobody would vote to watch something degraded to the quality of a theatrical print), it's better than the processed alternative.
There's a sliding scale from "all the grain in the 4K scan" to "all grain removed." The most accurate to creative intent and original experience is somewhere in the middle.

Also, there's a scale between "really good noise reduction that just makes it like the grain wasn't so strong" to "really bad noise reduction that leaves artifacts and throws out real non-grain detail." The optimal experience is at the first.

My go-to example for horribly jarring grain removal was the original Pinocchio Blu-ray. The algorithm wasn't properly motion adaptive, so static shots were grain free, but whenever anything moved grain popped into existence for the duration of the motion.

The culprit there wasn't that noise reduction was used, but that an inappropriate algorithm was used (perhaps one not tuned for cel animation?).

Getting back to x265 and encoding, ALL compression algorithms are in large part low-pass filters, and will soften out high-frequency spatial and temporal details like grain whenever there aren't enough bits available. How that gets tuned is a challenge. And the art is often not in "preserving" grain but in reproducing the general energy and feel of the original grain. If you do a frame-by-frame diff between source and encode, grain generally won't be identical. So codecs that are lauded for grain preservation are likely doing more grain emulation in practice. The basic idea of preserving source energy is what --psy-rdoq does in x265 sort of what psy-trellis does in x264.

Since grain is going to get removed/changed anyway, my preference is that the creative decision of what to remove and what to keep gets made at the mastering stage as much as possible, and in the codec as little as possible. And rest assured, you do NOT want all the grain in a high quality film scan. Think how noisy a RAW file shot in low light is. It's not "accurate" to the original image, creative intent, original experience, or anything. And it gets worse the better the scan.

[Fantasy]

Hello,
I use Megui to encode video.
I try to use 2nd pass with x265 with command "--pass 2" but it doesnt work.
Can I use x265 2nd pass with megui?
What have i do to encode 2nd pass with x265?
Thanks

Edit: I found out that i have to do first pass and then 2nd pass:
OK. Variables that you supply are shown in capital letters...

x265 --input SOURCEFILE --input-res RES --fps FPS --bitrate BITRATE -p PRESET --pass 1 --slow-firstpass --stats STATSFILE.LOG OUTPUT1.hevc

x265 --input SOURCEFILE --input-res RES --fps FPS --bitrate BITRATE -p PRESET --pass 2 --stats STATSFILE.LOG OUTPUT2.hevc


So, for example...
x265 --input test.yuv --input-res 1920x1080 --fps 30 --bitrate 5000 -p slow --pass 1 --slow-firstpass --stats test_stats.log test-firstpass.hevc

x265 --input test.yuv --input-res 1920x1080 --fps 30 --bitrate 5000 -p slow --pass 2 --stats test_stats.log test.hevc

I am testing now. I use medium preset for both pass. But it tooks too long for me to do first pass. So now I have questions:
If I use -p fast or -p faster for the first pass, will the quality worse or will the file bigger with constant bitrate?
What is the best way to do first pass?

[x265_Project]

Hello,
I am testing now. I use medium preset for both pass. But it tooks too long for me to do first pass. So now I have questions:
If I use -p fast or -p faster for the first pass, will the quality worse or will the file bigger with constant bitrate?
What is the best way to do first pass?
A slow first pass isn't going to buy you much. Try removing this option from your first pass, and it will automatically run much faster. This is much better than changing the performance preset for the first pass, as different presets use different parameters that can affect the GOP structure (like the max # of consecutive b-frames allowed), and the GOP structure must be identical in both passes. So, if you use a faster preset in the first pass, it could limit compression efficiency in the second pass.

[LigH]

The new builds support NUMA (http://en.wikipedia.org/wiki/Non-uniform_memory_access) thread pools (enjoy your "render farm") :D

x265 1.5+77-87173d41df87 (http://www.mediafire.com/download/uqmuc9bt4720mvk/x265_1.5+77-87173d41df87.7z)

[Fantasy]

Thanks for your answer.
I have now problem with first pass: only 50 % of the frames were encoded.
My video has 60 fps. I also try to use command
--fps 60
but it doesn't work. :(

[LigH]

The 'fps' switch will not have any impact on the number of frames encoded, it only tells the encoder about the duration each frame will be visible so it can possibly adjust psycho-visual features, and maybe flag the output if the output supports frame rate flags at all.

Does x265 detect the expected number of frames but stops unexpectedly after half the number? But ... if you feed a raw YUV file without supplying the 'frames' parameter, x265 may not be able to detect the number at all, just encode to the EOF.

[Fantasy]

Like I said I am using Megui and x265 stops by half the number and only when I add --pass 1 to command. I want to use 2nd pass because I can chose the bitrate and it is better then 1 pass (ABR Mode).

I am using "--crf" now without "--pass" and x265 still works over half the number. waiting now, 50 min to finish...

[Music Rockz]

I have error with MeGUI x265...When encodes it shows error...!

Error:
Standard Stream Error

What could be the issue?

No problem with Avs script! Checked with AvsPmod!

[LigH]

Uh, guys, you are mixing MeGUI and x265 now. If there is no error starting the conversion from a command line or in a batch file (maybe using avs4x26x in case of an AviSynth script as source), it doesn't belong to this thread, but instead there may be an issue with MeGUI.

I just tested a fast 2-pass encode of a Y4M file with x265 1.5+77 using a batch, and there is no issue, the 1st pass processes all frames.

[Fantasy]

My video is a .mp4 or a .mkv file. How can i convert to .y4m or compatible input type for x265?
I try this:
LoadPlugin ("C:\Program Files (x86)\AviSynth 2.5\plugins\DirectShowSource.dll")
file = DirectShowSource("C:\Users\...\video.mp4", fps=59.940, audio=false, convertfps=true).AssumeFPS(60000,1001)

F:\MeGUI_2418_x86\tools\x265\x64\x265.exe --input file --input-depth 8 --input-res 1920x1080 --fps 60 --bitrate 25000 -p medium --pass 1 --stats test_stats.log test-firstpass.hevc
F:\MeGUI_2418_x86\tools\x265\x64\x265.exe --input file --input-depth 8 --input-res 1920x1080 --fps 60 --bitrate 25000 -p medium --pass 2 --stats test_stats.log test.hevc
but batch not runs.

[sneaker_ger]

Script:
DirectShowSource("C:\Users\...\video.mp4", fps=59.940, audio=false, convertfps=true).AssumeFPS(60000,1001)

command:
avs2pipemod.exe -y4mp script.avs | F:\MeGUI_2418_x86\tools\x265\x64\x265.exe - --y4m --bitrate 25000 --pass 1 -o test-firstpass.hevc
avs2pipemod.exe -y4mp script.avs | F:\MeGUI_2418_x86\tools\x265\x64\x265.exe - --y4m --bitrate 25000 --pass 2 -o test.hevc
avs2pipemod download (http://www.mediafire.com/download/alg424t3kx7ak5d/avs2pipemod-0.4.2m.7z)

Or you can use a tool like ffmpeg (http://ffmpeg.zeranoe.com/builds/) if you don't need any AviSynth processing (no script file needed):
ffmpeg.exe -i "C:\Users\...\video.mp4" -f yuv4mpegpipe - | F:\MeGUI_2418_x86\tools\x265\x64\x265.exe - --y4m --bitrate 25000 --pass 1 -o test-firstpass.hevc
ffmpeg.exe -i "C:\Users\...\video.mp4" -f yuv4mpegpipe - | F:\MeGUI_2418_x86\tools\x265\x64\x265.exe - --y4m --bitrate 25000 --pass 2 -o test.hevc

(I removed some parts of your command that are not necessary)

[LigH]

That's because the first half is an AviSynth script, not a batch. The first two lines could not be executed on a command line prompt, so they can't in a batch file either.

Besides, DirectShowSource is always my last choice, only chosen if all other possible source plugins failed. DirectShow as such has a priority to play video fast, even if that means to reduce quality or even skip frames, except you really know your installed filters and how to tweak them. Instead, native AviSynth decoders (like FFMS2 / L-SMASH Source / DGDecNV / DGDecIM) have a priority to provide exact decoding results, even if that may take a bit more time.

Is it at all necessary to change the frame rate to "ntsc_double" (FPS preset (http://avisynth.nl/index.php/FPS))? Does your original video not already have that frame rate?
__

P.S.: Instead of avs2pipemod, why not avs4x26x (http://forum.doom9.org/showthread.php?t=162656)?

avs4x26x -L F:\MeGUI_2418_x86\tools\x265\x64\x265.exe --bitrate 25000 --pass 1 -o test-firstpass.hevc script.avs
avs4x26x -L F:\MeGUI_2418_x86\tools\x265\x64\x265.exe --bitrate 25000 --pass 2 -o test.hevc script.avs

It may even support the MP4 or MKV source more or less directly, creating the required script on-the-fly, based on the extension and available source plugins.

[Selur]

--pools <string>, --numa-pools <string>
Comma seperated list of threads per NUMA node. If "none", then no worker pools are created and only frame parallelism is possible. If NULL or "" (default) x265 will use all available threads on each NUMA node-

'+' is a special value indicating all cores detected on the node
'*' is a special value indicating all cores detected on the node and all remaining nodes
'-' is a special value indicating no cores on the node, same as '0'

example strings for a 4-node system:
"" - default, unspecified, all numa nodes are used for thread pools
"*" - same as default
"none" - no thread pools are created, only frame parallelism possible
"-" - same as "none"
"10" - allocate one pool, using up to 10 cores on node 0
"-,+" - allocate one pool, using all cores on node 1
"+,-,+" - allocate two pools, using all cores on nodes 0 and 2
"+,-,+,-" - allocate two pools, using all cores on nodes 0 and 2
"-,*" - allocate three pools, using all cores on nodes 1, 2 and 3
"8,8,8,8" - allocate four pools with up to 8 threads in each pool

The total number of threads will be determined by the number of threads assigned to all nodes. The worker threads will each be given affinity for
their node, they will not be allowed to migrate between nodes, but they will be allowed to move between CPU cores within their node.

If the three pool features: `--wpp`, `--pmode` and `--pme` are all disabled, then `--pools` is ignored and no thread pools are created.

If "none" is specified, then all three of the thread pool features are implicitly disabled.

Multiple thread pools will be allocated for any NUMA node with more than 64 logical CPU cores. But any given thread pool will always use at most one NUMA node.
Frame encoders are distributed between the available thread pools, and the encoder will never generate more thread pools than `--frame-threads`. The pools are used for WPP and for distributed analysis and motion search.

Default "", one thread is allocated per detected hardware thread (logical CPU cores) and one thread pool per NUMA node.
source: https://bitbucket.org/multicoreware/x265/commits/7252c10278a1e1aeddbe489180aad35196262919#chg-doc/reST/cli.rst

-> anyone played around with this and can recommend some values for normal (dual/quad/hex/octa-core) users?

[LigH]

I wonder if that is at all useful on only one PC. If I understood the Wikipedia explanation at all, NUMA is about per-CPU local memory, which is not very usual in a PC.

[xooyoozoo]

I wonder if that is at all useful on only one PC. If I understood the Wikipedia explanation at all, NUMA is about per-CPU local memory, which is not very usual in a PC.

Dual-sockets are common enough in editing/rendering rigs, and the NUMA barrier had always been a problem for most encoders. AFAIK, x264 itself never benefited from an extra CPU and was sometimes worse.

Anything with more than 2 sockets is unlikely except in a multi-user server sense, though I wouldn't mind seeing some numbers for a 8-socket Xeon-EX encode.

The only place that I've ever seen trying to benchmark both x265 and multi-socket systems is Anandtech. However, I recall them using x265 with Hybrid, both of which have rapid development cycles, without ever listing version numbers...

[LigH]

My first thoughts went into the direction of super computers with processor card slots...

[x265_Project]

I wonder if that is at all useful on only one PC. If I understood the Wikipedia explanation at all, NUMA is about per-CPU local memory, which is not very usual in a PC.
NUMA awareness is useful on multi-socket systems (workstations and servers). If your PC motherboard has only one processor socket (like all consumer PCs), this feature isn't relevant to you.

By scheduling similar work on a particular NUMA node (CPU), we can maximize the possibility that data needed by a particular thread will be found on locally (in the cache of that processor), and not on another NUMA node (which takes much longer to retrieve than if the data is in the local cache).

[Selur]

NUMA awareness is useful on multi-socket systems (workstations and servers). If your PC motherboard has only one processor socket (like all consumer PCs), this feature isn't relevant to you.
Okay,... then why remove the old '--threads X' ?
Threading, performance:
--pools <integer,...> Comma separated thread count per thread pool (pool per NUMA node)
'-' implies no threads on node, '+' implies one thread per core on node
-F/--frame-threads <integer> Number of concurrently encoded frames. 0: auto-determined by core count
--[no-]wpp Enable Wavefront Parallel Processing. Default enabled
--[no-]pmode Parallel mode analysis. Default disabled
--[no-]pme Parallel motion estimation. Default disabled
--[no-]asm <bool|int|string> Override CPU detection. Default: autosource: x265 command line help

[EncodedMango]

I was looking at an x265 encode someone made and was impressed with the results(quality:filesize ratio), unfortunately I don't know the person so couldn't ask for the commandline nor do I have a way to contact whoever made it. Is someone 'well versed' in the default parameters to discern what exactly were the arguments supplied?

Taken from MediaInfo:
Encoding settings: wpp / ctu=64 / tu-intra-depth=1 / tu-inter-depth=1 / me=3 / subme=3 / merange=57 / rect / no-amp / max-merge=3 / temporal-mvp / no-early-skip / no-fast-cbf / rdpenalty=0 / no-tskip / no-tskip-fast / strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / no-fast-intra / open-gop / interlace=0 / keyint=240 / min-keyint=24 / scenecut=40 / rc-lookahead=25 / bframes=4 / bframe-bias=0 / b-adapt=2 / ref=3 / weightp / no-weightb / aq-mode=2 / aq-strength=1.00 / cbqpoffs=0 / crqpoffs=0 / rd=4 / psy-rd=0.30 / psy-rdoq=1.00 / signhide / lft / sao / no-sao-non-deblock / b-pyramid / cutree / rc=crf / crf=25.0 / qcomp=0.60 / qpmin=0 / qpmax=51 / qpstep=4 / ipratio=1.40 / pbratio=1.30

I can obviously see it's 25 CRF and a psy-rd of 0.3 but other than that I can't say if there's any 'custom' parameters out of the ordinary.

[x265_Project]

I was looking at an x265 encode someone made and was impressed with the results(quality:filesize ratio), unfortunately I don't know the person so couldn't ask for the commandline nor do I have a way to contact whoever made it. Is someone 'well versed' in the default parameters to discern what exactly were the arguments supplied?

Taken from MediaInfo:


I can obviously see it's 25 CRF and a psy-rd of 0.3 but other than that I can't say if there's any 'custom' parameters out of the ordinary.
See http://x265.readthedocs.org/en/default/presets.html
It looks like --preset slow --aq-mode 2 --crf 25

[x265_Project]

Okay,... then why remove the old '--threads X' ?

On typical single-socket systems --pool N will do the same thing that --threads N did prior to the change, but obviously --pool can do a lot more than --threads ever could.

A large impetus to remove x265 --threads was that it has never matched the behavior of x264 --threads. x264's --threads configured frame threads, unless you were using slice threads, in which case it configured the number of slices. x265 --threads has never had the same effect as x264 --threads and we saw users were confused by this.

So now when some x264 user starts to run x265, if they try to configure the number of frame threads with x265 --threads they will learn that is not the right param, they will read the docs, and discover that our threading strategy uses --frame-threads N and --pool M,O,P. Failing the first time is better than getting a bad result with x265 while thinking that you're using optimal settings on your machine.

[LigH]

I believe this thread pooling could be a feature the x264 might be interested in for backporting. If AVC encoding would profit just as much. Who knows if there are power computers with more than 2 CPUs in a "render farm".

[benwaggoner]

I believe this thread pooling could be a feature the x264 might be interested in for backporting. If AVC encoding would profit just as much. Who knows if there are power computers with more than 2 CPUs in a "render farm".
Lots of encoding gets done in AWS. All the "8xlarge" instance types are dual socket. The new c4.8xlarge has 18 cores (36 hyperthreaded) of Broadwell, and thus AVX2.

http://aws.amazon.com/ec2/instance-types/

Generally dual socket is the sweet spot, as quad-socket systems tend to run at a lower clock speed, and are a lot more expensive in MIPS/$. But they might be useful for higher quality real-time UHD encoding or something.

FWIW, My primary workstation has been always been dual-socket since the late 90's. Even if it doesn't always make a single encoding job go faster, it helps with video processing in general, and when doing multiple encodes at once.