x265 HEVC Encoder

[LigH]

The tool "strings" just filters longer sequences of readable characters, without knowing if it is actually text meant to be read by humans. "Random" sequences of data bytes "incidently" making readable character sequences pass it too.

[xooyoozoo]

Multi-pass encoding is now enabled through the cli.

Based on a few tests, the SSIM of CRF ~= Pass2 >= Pass1, so it seems to be working fine. However, "turbo first pass" doesn't exist yet.

Yes, 2 pass encoding is enabled in the Dev build, but there are a few bugs that we are aware of and working on (dealing with compatibility with rate control options used in the first or second pass). Detailed feedback (ideally, with source clips available to us and steps to reproduce the results) is welcomed.

Tom

[Sagittaire]

Originally Posted by xooyoozoo

Multi-pass encoding is now enabled through the cli.

Based on a few tests, the SSIM of CRF ~= Pass2 >= Pass1, so it seems to be working fine. However, "turbo first pass" doesn't exist yet.

well you can make yourself "turbo first pass". I make several test and that work very well ...

[Selur]

I make several test and that work very well ...

What options did you disable/change during 1st pass?

----
btw. 2pass + qpfile -> crash (https://bitbucket.org/multicoreware/...frame-type-not)

[Sagittaire]

Originally Posted by Selur

What options did you disable/change during 1st pass?

----
btw. 2pass + qpfile -> crash (https://bitbucket.org/multicoreware/...frame-type-not)

I make that for good first pass speed:

x265.exe --input hp.yuv --output crf23aq.265 --input-res 720x304 --fps 25 --pass 1 --stats hp.log --bitrate 450 
--preset fast --me dia --ref 1 --aq-mode 0 --aq-strength 1.0 --psy-rd 0.0 --bframes 3 --b-adapt 2 --min-keyint 1 --ipratio 1.2 --pbratio 1.2 --psnr

Only frame decision is important (b-adapt 2). You can use very fast quality decision without problem (me, subme, ref, rd ...).

[Selur]

thanks

[Sagittaire]

well some test with 2 pass mode:

- fast first pass mode with crf
- Npass with fast, medium, slow, slower, veryslow and placebo preset

|--------------|----------|----------|-----------|
| fast         | 449 kbps | 42.97 dB | 13.98 fps |
| medium       | 449 kbps | 43.10 dB | 12.13 fps |
| slow         | 449 kbps | 43.35 dB |  3.40 fps |
| slower       | 448 kbps | 43.72 dB |  1.22 fps |
| veryslow     | 449 kbps | 43.74 dB |  0.75 fps |
| placebo      | 449 kbps | 43.75 dB |  0.47 fps |
| balanced     | 449 kbps | 43.31 dB |  7.93 fps |
|--------------|----------|----------|-----------|

Conclusion:

- 2 pass mode work very well even if you use fast first pass.
- On my system (c2d Q6600) you have abyssal speed delta between medium and slow preset. It's certainely in the most important speed zone because BD Ripp 1080p will be done in this speed zone with performant CPU system. I propose alternative "balanced" preset setting in this really important zone.

balanced = slower + rd 4 + no-rect + no-amp + me hex

Originally Posted by Sagittaire

well some test with 2 pass mode:

- fast first pass mode with crf
- Npass with fast, medium, slow, slower, veryslow and placebo preset

|--------------|----------|----------|-----------|
| fast         | 449 kbps | 42.97 dB | 13.98 fps |
| medium       | 449 kbps | 43.10 dB | 12.13 fps |
| slow         | 449 kbps | 43.35 dB |  3.40 fps |
| slower       | 448 kbps | 43.72 dB |  1.22 fps |
| veryslow     | 449 kbps | 43.74 dB |  0.75 fps |
| placebo      | 449 kbps | 43.75 dB |  0.47 fps |
| balanced     | 449 kbps | 43.31 dB |  7.93 fps |
|--------------|----------|----------|-----------|

Conclusion:

- 2 pass mode work very well even if you use fast first pass.
- On my system (c2d Q6600) you have abyssal speed delta between medium and slow preset. It's certainly in the most important speed zone because BD Ripp 1080p will be done in this speed zone with performant CPU system. I propose alternative "balanced" preset setting in this really important zone.

balanced = slower + rd 4 + no-rect + no-amp + me hex

Hi Sagittaire,
Thanks for the feedback. The big difference between medium and slow is that medium doesn't test rectangular blocks (--no-rect). Once you turn on --rect, you greatly increase the number of possible ways to partition a 64x64 CU. You can see what I mean if you look at the top of page 3 of this paper... https://www.ic.tu-berlin.de/fileadmi...12_HEVC-BM.pdf. At the first level of the quad-tree, you can partition a CU 8 different ways. With --no-rect, this drops to only 2 possibilities. The same is true at every level of the quad-tree except when you get down to 8x8 pixel blocks.

So, once you add --rect, there is a big increase in the number of possible partitions that must be checked to see which results in the most efficient way to encode the CU.

[foxyshadis]

Originally Posted by x265_Project

Hi Sagittaire,
Thanks for the feedback. The big difference between medium and slow is that medium doesn't test rectangular blocks (--no-rect). Once you turn on --rect, you greatly increase the number of possible ways to partition a 64x64 CU. You can see what I mean if you look at the top of page 3 of this paper... https://www.ic.tu-berlin.de/fileadmi...12_HEVC-BM.pdf. At the first level of the quad-tree, you can partition a CU 8 different ways. With --no-rect, this drops to only 2 possibilities. The same is true at every level of the quad-tree except when you get down to 8x8 pixel blocks.

So, once you add --rect, there is a big increase in the number of possible partitions that must be checked to see which results in the most efficient way to encode the CU.

Maybe --rect could become a 0-3 argument, each testing more levels and partition possibilities at each level? Or make it depend on the rd level? Similar to how Steve talked about minimizing the number of intra tests on the mailing list a few days ago.

Originally Posted by foxyshadis

Maybe --rect could become a 0-3 argument, each testing more levels and partition possibilities at each level? Or make it depend on the rd level? Similar to how Steve talked about minimizing the number of intra tests on the mailing list a few days ago.

The same thought occurred to me as I typed the response above. I didn't want to raise the possibility publicly without checking on the scope of effort, so I emailed our Dev team asking if --rect 1 could be limited to 2N x N rectangles , with --rect 2 adding all of the 2N x n rectangles. We certainly have lots of other higher priorities right now, but let's see what they say in terms of the complexity that this would create and the scope of effort. It seems logical that it would create an intermediate level of partitioning that would enable a smaller step along the speed vs. efficiency tradeoff curve.

[fumoffu]

I have a question (I think it has been touched before but that was a long time ago).
Why merange is so high in all presets? Does it work the same as x264 merange?
because as you can find on some forums and also trough testing:
"...a large merange is basically useless. merange is the distance to search from the best predictor, not the maximum motion vector range. The best predictor is generally close enough to the correct MV that a large search radius is a waste of time."
and so in x264 in almost all presets merange is only 16 and changing it doesn't really do much except increasing encoding time...
So what is the reason for setting it so high in x265?

Originally Posted by foxyshadis

Maybe --rect could become a 0-3 argument, each testing more levels and partition possibilities at each level? Or make it depend on the rd level? Similar to how Steve talked about minimizing the number of intra tests on the mailing list a few days ago.

Our dev leads reminded me that --amp controls whether x265 checks the additional n x 2N rectangular partition possibilities. However, even with --amp we only ever check at most one AMP prediction, based on the results of the N x 2N rectangular predictions.

So, there are no intermediate levels that don't already exist.

There are other dials we can turn to spread out the distribution of performance presets, but we have some additional coding tools that aren't on by default yet (psy-rd and psy-rdoq), and we need to figure out where they should come into play, and the impact on performance and visual quality. We don't want to change things too often, so it's better to do this all at once.

Tom

[upyzl]

Originally Posted by fumoffu

I have a question (I think it has been touched before but that was a long time ago).
Why merange is so high in all presets? Does it work the same as x264 merange?
because as you can find on some forums and also trough testing:
"...a large merange is basically useless. merange is the distance to search from the best predictor, not the maximum motion vector range. The best predictor is generally close enough to the correct MV that a large search radius is a waste of time."
and so in x264 in almost all presets merange is only 16 and changing it doesn't really do much except increasing encoding time...
So what is the reason for setting it so high in x265?

I'm not expert on it, but AFAIK, H.264/AVC coding unit should be MacroBlock, which size is 16x16; and H.265/HEVC coding unit is CTU, which default is 64x64

and here is the doc from MCW x265 Team:
http://x265.readthedocs.org/en/defau...ption--merange

Originally Posted by fumoffu

I have a question (I think it has been touched before but that was a long time ago).
Why merange is so high in all presets? Does it work the same as x264 merange?
because as you can find on some forums and also trough testing:
"...a large merange is basically useless. merange is the distance to search from the best predictor, not the maximum motion vector range. The best predictor is generally close enough to the correct MV that a large search radius is a waste of time."
and so in x264 in almost all presets merange is only 16 and changing it doesn't really do much except increasing encoding time...
So what is the reason for setting it so high in x265?

When we created our performance presets we ran many tests to see the effect on performance and quality for each parameter. Reducing the merange at most presets didn't produce any meaningful improvement on performance. It does have a slightly negative impact on encoding efficiency.

It's easy to run your own tests to verify this. Just run multiple encodes of the same clip with --ssim on (to measure quality), varying --merange for each test run. Your feedback and test results are welcomed!

[Kurtnoise]

Originally Posted by x265_Project

Yes, 2 pass encoding is enabled in the Dev build, but there are a few bugs that we are aware of and working on (dealing with compatibility with rate control options used in the first or second pass). Detailed feedback (ideally, with source clips available to us and steps to reproduce the results) is welcomed.

Using any presets lower than "Fast" crash on the 2nd pass, here (well...not a crash but a lot of warnings/errors :

x265 [warning]: specified frame type is not compatible with max B-frames
x265 [warning]: specified frame type is not compatible with max B-frames
x265 [error]: slice=P but 2pass stats say B
x265 [error]: slice=P but 2pass stats say B
x265 [warning]: specified frame type is not compatible with max B-frames

Are you able to reproduce this ?

Currently, I'm using version 1.2+349-8bab on Windows 8.1 64 bits - 8bpp.

I tried different sources with different length and different bitrates but the issue still appears...

--preset superfast --pass 1/2 --bitrate 2500 --stats "file.stats" --output "output.hevc" "input.avs"

[Sagittaire]

Perhaps that preset under "fast" have lower bframe number. try to have same frametype structure in first and second pass (min key frame, max key frame, bframe number, pyramidal bframe, closed or open GOP ...).

[foxyshadis]

Kurt, are you sure the stats file got updated when you ran the first pass again? You'd get those errors if you'd run a first pass with --preset medium or above and a second pass with --preset fast or lower.

Originally Posted by Kurtnoise

Using any presets lower than "Fast" crash on the 2nd pass, here (well...not a crash but a lot of warnings/errors :




Are you able to reproduce this ?

Currently, I'm using version 1.2+349-8bab on Windows 8.1 64 bits - 8bpp.

I tried different sources with different length and different bitrates but the issue still appears...

--preset superfast --pass 1/2 --bitrate 2500 --stats "file.stats" --output "output.hevc" "input.avs"

The --bframes setting must match between the 1st and 2nd pass. So, if you want to run a faster first pass using a performance preset that uses a different --bframes value, just add --bframes to the command line for the 1st pass, and use the value associated with the setting you will use for your 2nd pass.
You can find the bframes setting for each performance preset here... http://x265.readthedocs.org/en/default/presets.html

We're also aware of an issue with 2 pass and presets faster, superfast or ultrafast. Currently you don't get any B frames in the 2nd pass, making the 2nd pass less efficient than the first. The team is working to fix this.

[LigH]

A patch has just been submitted to the mailing list.