Log in

View Full Version : x265 HEVC Encoder


Pages : 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 [23] 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197

LigH
22nd July 2014, 10:28
The tool "strings" just filters longer sequences of readable characters, without knowing if it is actually text meant to be read by humans. "Random" sequences of data bytes "incidently" making readable character sequences pass it too.

xooyoozoo
24th July 2014, 23:19
Multi-pass encoding is now enabled through the cli.

Based on a few tests, the SSIM of CRF ~= Pass2 >= Pass1, so it seems to be working fine. However, "turbo first pass" doesn't exist yet.

x265_Project
25th July 2014, 00:04
Yes, 2 pass encoding is enabled in the Dev build, but there are a few bugs that we are aware of and working on (dealing with compatibility with rate control options used in the first or second pass). Detailed feedback (ideally, with source clips available to us and steps to reproduce the results) is welcomed.

Tom

Sagittaire
26th July 2014, 11:51
Multi-pass encoding is now enabled through the cli.

Based on a few tests, the SSIM of CRF ~= Pass2 >= Pass1, so it seems to be working fine. However, "turbo first pass" doesn't exist yet.

well you can make yourself "turbo first pass". I make several test and that work very well ...

Selur
26th July 2014, 15:03
I make several test and that work very well ...
What options did you disable/change during 1st pass?

----
btw. 2pass + qpfile -> crash (https://bitbucket.org/multicoreware/x265/issue/67/2pass-qpfile-specified-frame-type-not)

Sagittaire
26th July 2014, 15:18
What options did you disable/change during 1st pass?

----
btw. 2pass + qpfile -> crash (https://bitbucket.org/multicoreware/x265/issue/67/2pass-qpfile-specified-frame-type-not)

I make that for good first pass speed:

x265.exe --input hp.yuv --output crf23aq.265 --input-res 720x304 --fps 25 --pass 1 --stats hp.log --bitrate 450
--preset fast --me dia --ref 1 --aq-mode 0 --aq-strength 1.0 --psy-rd 0.0 --bframes 3 --b-adapt 2 --min-keyint 1 --ipratio 1.2 --pbratio 1.2 --psnr

Only frame decision is important (b-adapt 2). You can use very fast quality decision without problem (me, subme, ref, rd ...).

Selur
26th July 2014, 17:19
thanks

Sagittaire
26th July 2014, 18:31
well some test with 2 pass mode:

- fast first pass mode with crf
- Npass with fast, medium, slow, slower, veryslow and placebo preset

|--------------|----------|----------|-----------|
| fast | 449 kbps | 42.97 dB | 13.98 fps |
| medium | 449 kbps | 43.10 dB | 12.13 fps |
| slow | 449 kbps | 43.35 dB | 3.40 fps |
| slower | 448 kbps | 43.72 dB | 1.22 fps |
| veryslow | 449 kbps | 43.74 dB | 0.75 fps |
| placebo | 449 kbps | 43.75 dB | 0.47 fps |
| balanced | 449 kbps | 43.31 dB | 7.93 fps |
|--------------|----------|----------|-----------|

Conclusion:

- 2 pass mode work very well even if you use fast first pass.
- On my system (c2d Q6600) you have abyssal speed delta between medium and slow preset. It's certainely in the most important speed zone because BD Ripp 1080p will be done in this speed zone with performant CPU system. I propose alternative "balanced" preset setting in this really important zone.

balanced = slower + rd 4 + no-rect + no-amp + me hex

x265_Project
27th July 2014, 03:35
well some test with 2 pass mode:

- fast first pass mode with crf
- Npass with fast, medium, slow, slower, veryslow and placebo preset

|--------------|----------|----------|-----------|
| fast | 449 kbps | 42.97 dB | 13.98 fps |
| medium | 449 kbps | 43.10 dB | 12.13 fps |
| slow | 449 kbps | 43.35 dB | 3.40 fps |
| slower | 448 kbps | 43.72 dB | 1.22 fps |
| veryslow | 449 kbps | 43.74 dB | 0.75 fps |
| placebo | 449 kbps | 43.75 dB | 0.47 fps |
| balanced | 449 kbps | 43.31 dB | 7.93 fps |
|--------------|----------|----------|-----------|

Conclusion:

- 2 pass mode work very well even if you use fast first pass.
- On my system (c2d Q6600) you have abyssal speed delta between medium and slow preset. It's certainly in the most important speed zone because BD Ripp 1080p will be done in this speed zone with performant CPU system. I propose alternative "balanced" preset setting in this really important zone.

balanced = slower + rd 4 + no-rect + no-amp + me hex
Hi Sagittaire,
Thanks for the feedback. The big difference between medium and slow is that medium doesn't test rectangular blocks (--no-rect). Once you turn on --rect, you greatly increase the number of possible ways to partition a 64x64 CU. You can see what I mean if you look at the top of page 3 of this paper... https://www.ic.tu-berlin.de/fileadmin/fg121/Source-Coding_WS12/selected-readings/2012_12_HEVC-BM.pdf. At the first level of the quad-tree, you can partition a CU 8 different ways. With --no-rect, this drops to only 2 possibilities. The same is true at every level of the quad-tree except when you get down to 8x8 pixel blocks.

So, once you add --rect, there is a big increase in the number of possible partitions that must be checked to see which results in the most efficient way to encode the CU.

foxyshadis
27th July 2014, 04:09
Hi Sagittaire,
Thanks for the feedback. The big difference between medium and slow is that medium doesn't test rectangular blocks (--no-rect). Once you turn on --rect, you greatly increase the number of possible ways to partition a 64x64 CU. You can see what I mean if you look at the top of page 3 of this paper... https://www.ic.tu-berlin.de/fileadmin/fg121/Source-Coding_WS12/selected-readings/2012_12_HEVC-BM.pdf. At the first level of the quad-tree, you can partition a CU 8 different ways. With --no-rect, this drops to only 2 possibilities. The same is true at every level of the quad-tree except when you get down to 8x8 pixel blocks.

So, once you add --rect, there is a big increase in the number of possible partitions that must be checked to see which results in the most efficient way to encode the CU.

Maybe --rect could become a 0-3 argument, each testing more levels and partition possibilities at each level? Or make it depend on the rd level? Similar to how Steve talked about minimizing the number of intra tests on the mailing list a few days ago.

x265_Project
27th July 2014, 05:19
Maybe --rect could become a 0-3 argument, each testing more levels and partition possibilities at each level? Or make it depend on the rd level? Similar to how Steve talked about minimizing the number of intra tests on the mailing list a few days ago.
The same thought occurred to me as I typed the response above. I didn't want to raise the possibility publicly without checking on the scope of effort, so I emailed our Dev team asking if --rect 1 could be limited to 2N x N rectangles , with --rect 2 adding all of the 2N x n rectangles. We certainly have lots of other higher priorities right now, but let's see what they say in terms of the complexity that this would create and the scope of effort. It seems logical that it would create an intermediate level of partitioning that would enable a smaller step along the speed vs. efficiency tradeoff curve.

fumoffu
29th July 2014, 02:56
I have a question (I think it has been touched before but that was a long time ago).
Why merange is so high in all presets? Does it work the same as x264 merange?
because as you can find on some forums and also trough testing:
"...a large merange is basically useless. merange is the distance to search from the best predictor, not the maximum motion vector range. The best predictor is generally close enough to the correct MV that a large search radius is a waste of time."
and so in x264 in almost all presets merange is only 16 and changing it doesn't really do much except increasing encoding time...
So what is the reason for setting it so high in x265?

x265_Project
29th July 2014, 03:27
Maybe --rect could become a 0-3 argument, each testing more levels and partition possibilities at each level? Or make it depend on the rd level? Similar to how Steve talked about minimizing the number of intra tests on the mailing list a few days ago.
Our dev leads reminded me that --amp controls whether x265 checks the additional n x 2N rectangular partition possibilities. However, even with --amp we only ever check at most one AMP prediction, based on the results of the N x 2N rectangular predictions.

So, there are no intermediate levels that don't already exist.

There are other dials we can turn to spread out the distribution of performance presets, but we have some additional coding tools that aren't on by default yet (psy-rd and psy-rdoq), and we need to figure out where they should come into play, and the impact on performance and visual quality. We don't want to change things too often, so it's better to do this all at once.

Tom

upyzl
29th July 2014, 03:43
I have a question (I think it has been touched before but that was a long time ago).
Why merange is so high in all presets? Does it work the same as x264 merange?
because as you can find on some forums and also trough testing:
"...a large merange is basically useless. merange is the distance to search from the best predictor, not the maximum motion vector range. The best predictor is generally close enough to the correct MV that a large search radius is a waste of time."
and so in x264 in almost all presets merange is only 16 and changing it doesn't really do much except increasing encoding time...
So what is the reason for setting it so high in x265?

I'm not expert on it, but AFAIK, H.264/AVC coding unit should be MacroBlock, which size is 16x16; and H.265/HEVC coding unit is CTU, which default is 64x64

and here is the doc from MCW x265 Team:
http://x265.readthedocs.org/en/default/cli.html#cmdoption--merange

x265_Project
29th July 2014, 04:52
I have a question (I think it has been touched before but that was a long time ago).
Why merange is so high in all presets? Does it work the same as x264 merange?
because as you can find on some forums and also trough testing:
"...a large merange is basically useless. merange is the distance to search from the best predictor, not the maximum motion vector range. The best predictor is generally close enough to the correct MV that a large search radius is a waste of time."
and so in x264 in almost all presets merange is only 16 and changing it doesn't really do much except increasing encoding time...
So what is the reason for setting it so high in x265?
When we created our performance presets we ran many tests to see the effect on performance and quality for each parameter. Reducing the merange at most presets didn't produce any meaningful improvement on performance. It does have a slightly negative impact on encoding efficiency.

It's easy to run your own tests to verify this. Just run multiple encodes of the same clip with --ssim on (to measure quality), varying --merange for each test run. Your feedback and test results are welcomed!

Kurtnoise
29th July 2014, 10:01
Yes, 2 pass encoding is enabled in the Dev build, but there are a few bugs that we are aware of and working on (dealing with compatibility with rate control options used in the first or second pass). Detailed feedback (ideally, with source clips available to us and steps to reproduce the results) is welcomed.

Using any presets lower than "Fast" crash on the 2nd pass, here (well...not a crash but a lot of warnings/errors :


x265 [warning]: specified frame type is not compatible with max B-frames
x265 [warning]: specified frame type is not compatible with max B-frames
x265 [error]: slice=P but 2pass stats say B
x265 [error]: slice=P but 2pass stats say B
x265 [warning]: specified frame type is not compatible with max B-frames



Are you able to reproduce this ?

Currently, I'm using version 1.2+349-8bab on Windows 8.1 64 bits - 8bpp.

I tried different sources with different length and different bitrates but the issue still appears...

--preset superfast --pass 1/2 --bitrate 2500 --stats "file.stats" --output "output.hevc" "input.avs"

Sagittaire
29th July 2014, 10:40
Perhaps that preset under "fast" have lower bframe number. try to have same frametype structure in first and second pass (min key frame, max key frame, bframe number, pyramidal bframe, closed or open GOP ...).

foxyshadis
29th July 2014, 16:42
Kurt, are you sure the stats file got updated when you ran the first pass again? You'd get those errors if you'd run a first pass with --preset medium or above and a second pass with --preset fast or lower.

x265_Project
29th July 2014, 18:19
Using any presets lower than "Fast" crash on the 2nd pass, here (well...not a crash but a lot of warnings/errors :




Are you able to reproduce this ?

Currently, I'm using version 1.2+349-8bab on Windows 8.1 64 bits - 8bpp.

I tried different sources with different length and different bitrates but the issue still appears...

--preset superfast --pass 1/2 --bitrate 2500 --stats "file.stats" --output "output.hevc" "input.avs"

The --bframes setting must match between the 1st and 2nd pass. So, if you want to run a faster first pass using a performance preset that uses a different --bframes value, just add --bframes to the command line for the 1st pass, and use the value associated with the setting you will use for your 2nd pass.
You can find the bframes setting for each performance preset here... http://x265.readthedocs.org/en/default/presets.html

We're also aware of an issue with 2 pass and presets faster, superfast or ultrafast. Currently you don't get any B frames in the 2nd pass, making the 2nd pass less efficient than the first. The team is working to fix this.

LigH
29th July 2014, 18:50
A patch has just been submitted to the mailing list.

Selur
29th July 2014, 23:26
may be that patch will also help with my issue that using qpfile caused similar output&crashes

Kurtnoise
30th July 2014, 08:13
We're also aware of an issue with 2 pass and presets faster, superfast or ultrafast. Currently you don't get any B frames in the 2nd pass, making the 2nd pass less efficient than the first. The team is working to fix this.
That patch fixed my issue...:thanks:

x265_Project
30th July 2014, 16:35
We've checked in Psy-RDOQ. At this point, you should consider this feature highly experimental. We are performing tests to determine the scaling factor that is needed. Ongoing work includes Psy-RDOQ performance optimization.

As always, your test results (with full command line, --log 3 or 4 csv file with --SSIM on and ideally, a link to the publicly available source clip) are welcomed.

Tom

upyzl
1st August 2014, 03:53
MPC-BE 1.4.3 5157+ (MediaInfo 0.7.69+ svn rev.6373) dev (nightly builds (https://sourceforge.net/projects/mpcbe/files/MPC-BE/Nightly%20Builds%20%28from%20svn%20trunk%29/)) has supported new x265 SEI:
General
Complete name : D:\enc\test_265.mp4
Format : MPEG-4
Format profile : Base Media / Version 2
Codec ID : mp42
File size : 13.5 MiB
Duration : 1mn 31s
Overall bit rate mode : Variable
Overall bit rate : 1 237 Kbps
Encoded date : UTC 2014-08-01 01:50:26
Tagged date : UTC 2014-08-01 01:50:26
Writing application : qaac 2.35, CoreAudioToolbox 7.9.8.5, AAC-HE Encoder, CVBR 80kbps, Quality 96
HDCD : 0
VALID_BITS : 16

Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main@L4.0
Codec ID : hvc1
Codec ID/Info : High Efficiency Video Coding
Duration : 1mn 31s
Bit rate : 1 153 Kbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 23.976 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Bits/(Pixel*Frame) : 0.023
Stream size : 12.5 MiB (93%)
Writing library : x265 1.2+413-e85b0aaa64e4:[Windows][ICC 1400][64 bit]
Encoding settings : wpp / ctu=64 / tu-intra-depth=1 / tu-inter-depth=1 / me=1 / subme=2 / merange=57 / no-rect / no-amp / max-merge=2 / no-early-skip / no-fast-cbf / rdpenalty=0 / no-tskip / no-tskip-fast / strong-intra-smoothing / no-lossless / no-cu-lossless / no-constrained-intra / open-gop / interlace=0 / keyint=250 / min-keyint=23 / scenecut=40 / rc-lookahead=20 / bframes=4 / bframe-bias=0 / b-adapt=2 / ref=3 / weightp / no-weightb / aq-mode=2 / aq-strength=1.00 / cbqpoffs=0 / crqpoffs=0 / rd=3 / signhide / lft / sao / sao-lcu-bounds=0 / sao-lcu-opt=1 / b-pyramid / cutree / rc=crf / crf=24.0 / qcomp=0.60 / qpmin=0 / qpmax=51 / qpstep=4 / ipratio=1.40 / pbratio=1.30
Encoded date : UTC 2014-08-01 01:50:26
Tagged date : UTC 2014-08-01 01:50:26

Audio
ID : 2
Format : AAC
Format/Info : Advanced Audio Codec
Format profile : HE-AAC / LC
Codec ID : 40
Duration : 1mn 31s
Bit rate mode : Variable
Bit rate : 82.1 Kbps
Channel(s) : 2 channels
Channel positions : Front: L R
Sampling rate : 48.0 KHz / 24.0 KHz
Compression mode : Lossy
Stream size : 914 KiB (7%)
Encoded date : UTC 2014-08-01 01:50:26
Tagged date : UTC 2014-08-01 01:50:26

sneaker_ger
2nd August 2014, 07:57
x264 is trademarked by VideoLan, a non-profit organisation. x265 is(or is trying to be) trademarked by multicoreware, a profit seeking business. The difference is in ways subtle, such as multicoreware trying to use their trademark to prevent people from releasing x265 builds.
VideoLAN has registered x265 as well, at least for the internal market of the European Union.

Procrastinating
2nd August 2014, 08:06
sneaker_ger, you may have accidentally posted to the wrong thread.

In any case, Do we have a clear idea of how much benefit RDOQ will bring to x265? That is, do we have a sure idea whether RDOQ will bring significant benefits over RD? Or will we just wait and see.

foxyshadis
3rd August 2014, 12:35
sneaker_ger, you may have accidentally posted to the wrong thread.

In any case, Do we have a clear idea of how much benefit RDOQ will bring to x265? That is, do we have a sure idea whether RDOQ will bring significant benefits over RD? Or will we just wait and see.

RDOQ beats plain RD. psy-rdoq? Who knows! Test hard! It's bash things against the wall time to see what shakes out.

LigH
3rd August 2014, 16:54
Yo, check this out:

I encoded a series of small tests with "Kristen and Sara" 720p (@ varun); already reduced strengths show quite obviously a "bleeding" of energy from areas with fine details (hair) into areas with no details (blue background), causing "wobbling background" artefacts. A later analysis should show if this is caused rather by Psy-RD or Psy-RDOQ, or only in combination of both...

KS_psy.7z (https://www.mediafire.com/download/q2db5j6x5c41i4j/KS_psy.7z) in HEVC (https://www.mediafire.com/folder/6lfp2jlygogwa/HEVC)/samples (https://www.mediafire.com/folder/ldwl20fppplbx/samples)

Sagittaire
4th August 2014, 10:10
rdoq seem produce major visual improvement for park joy sample. Test in progress.

xooyoozoo
4th August 2014, 10:49
Did a quick test on a moderately grainy clip with (arbitrarily chosen) 0.5-3.0 psyrd-psyrdoq, and I think things are looking much better! My main nitpick is that chroma noise is a bit blatant.

Done on x265@8edb2a5 with turbo 1st pass diff from mailing list applied. Used 2pass and preset slower.
Bitstreams and side-by-side composition (https://www.mediafire.com/folder/22jrkonb4d2wb/2014-08-04-tales-hevc).
Some snapshots: 1 (http://i.imgur.com/eqQ5JZH.png), 2 (http://i.imgur.com/PC6auZY.png), 3 (http://i.imgur.com/UG1t7St.png).

The source is first 800 frames of the 80 mbits prores download from here (https://vimeo.com/83184433) (might need to be logged in).

sneaker_ger
4th August 2014, 11:13
Did a quick test on a moderately grainy clip with (arbitrarily chosen) 0.5-3.0 psyrd-psyrdoq
I thought 2.0 was the limit?

Btw, are psyrd and psyrdoq mutually exclusive?

foxyshadis
4th August 2014, 23:19
I thought 2.0 was the limit?

Btw, are psyrd and psyrdoq mutually exclusive?

2.0 is the limit for psy-rd, but psy-rdoq is unbound while they tweak it. They're complimentary, psy-rdoq is akin to x264's psy-trellis.

Atak_Snajpera
8th August 2014, 16:50
Is there any reason why x265 stats file is written in such odd way without proper line breaks ?

x265 stats file
http://i.cubeupload.com/AphD7Q.png

x264 stats file
http://i.cubeupload.com/dAN8CO.png

sneaker_ger
8th August 2014, 16:54
x265 produces a newline (LF) after each frame for me just like x264. :confused:

Atak_Snajpera
8th August 2014, 16:56
x265 produces a newline (LF) after each frame for me just like x264. :confused:

show me :)

sneaker_ger
8th August 2014, 17:00
http://abload.de/img/x265_2pass_stats_newlu2inf.png
(Using this build (http://chromashift.org/x265_builds/x265-1.2.474-highbitdepth-icl14.0-64.7z))

Atak_Snajpera
8th August 2014, 17:02
I use latest from here http://builds.x265.eu/ 1.2+496

benwaggoner
8th August 2014, 17:39
Is there any reason why x265 stats file is written in such odd way without proper line breaks ?
I've not had any issues with line breaks.

I VASTLY prefer the x265 CSV style to the weird turgid x264 style. Anything that can read CSV (like Excel) can easily import the x265 .stats files, while a bunch of annoying parsing is required to turn the x264 .stats files into easily analyzed data. Also, all per-item prefixes make the x264 .stats files way bigger.

I hope the CSV style will get backported to x264, at least as an option.

Atak_Snajpera
8th August 2014, 17:43
I've not had any issues with line breaks.

I VASTLY prefer the x265 CSV style to the weird turgid x264 style. Anything that can read CSV (like Excel) can easily import the x265 .stats files, while a bunch of annoying parsing is required to turn the x264 .stats files into easily analyzed data. Also, all per-item prefixes make the x264 .stats files way bigger.

I hope the CSV style will get backported to x264, at least as an option.

what version of x265 do you use?

Atak_Snajpera
9th August 2014, 12:40
Ok. It turns out that builds on http://builds.x265.eu/ are broken. I used older build from chromashift and log now has correct line breaks.

BTW. Noobish question here. Is this normal that those numbers are not in order like in x264?

x265 stats
http://i.cubeupload.com/JS3eko.png

x264 stats
http://i.cubeupload.com/QiyomG.png

x265_Project
9th August 2014, 17:14
BTW. Noobish question here. Is this normal that those numbers are not in order like in x264?

Can you help us understand what you are doing with the stats file? Are you trying to parse this file for some reason? x265 has a comma-separated value (--csv filename.csv ) log file function that should provide all of the information you would normally need.

Atak_Snajpera
9th August 2014, 19:32
I read stat file in order to verify how many frames have been actually processed by encoder in first pass. Then I inject that value to second pass as --frames xxxxxx switch. Why am I doing this? FFmpegSource() for some unknown reason sometimes likes to discard last frame. Movie initially is detected by decoder as let's say 1000 frames but encoder processes in first pass only 999 frames. With x264 stat file I just have to read last line in order to extract number of processed frames. With x265 I've encountered two issues. First latest builds from http://builds.x265.eu/ do not have proper line breaks. Older version from chromashift.org works fine. Then I've noticed that last line in x265 stat file does not always contain what I want. From technical point of view this is not a big problem for me but still I'm curious why values in x265's stats file do not act like in older brother x264.

x265_Project
9th August 2014, 23:13
I read stat file in order to verify how many frames have been actually processed by encoder in first pass. Then I inject that value to second pass as --frames xxxxxx switch. Why am I doing this? FFmpegSource() for some unknown reason sometimes likes to discard last frame. Movie initially is detected by decoder as let's say 1000 frames but encoder processes in first pass only 999 frames. With x264 stat file I just have to read last line in order to extract number of processed frames. With x265 I've encountered two issues. First latest builds from http://builds.x265.eu/ do not have proper line breaks. Older version from chromashift.org works fine. Then I've noticed that last line in x265 stat file does not always contain what I want. From technical point of view this is not a big problem for me but still I'm curious why values in x265's stats file do not act like in older brother x264.
I'll have to check with our engineers for the answer.

x265's --csv function with --log 3 or 4 will produce a log file with the frame count that shouldn't be too difficult to parse (although the frame count you are looking for will be in the 4 lines before the last line in the file). And, of course, add 1 to the frame number which is at the beginning of the (last-4) line in the csv log file, as frames are numbered starting from 0.

Atak_Snajpera
10th August 2014, 12:36
It turns out that even latest version from http://chromashift.org/x265_builds/ does not have line breaks in stat file. Why have you removed that? If it ain't broke, don't fix it ;)

a5180007
10th August 2014, 17:04
@Atak_Snajpera : it comes from commit 02d805e (https://bitbucket.org/multicoreware/x265/commits/02d805ee3d387366db2222ce0f1cc379629a6790), "\n" has been omitted to obtain a CSV file.

Just replace ";" with ";\n" in file with Notepad++ to have the old presentation.

x265_Project
10th August 2014, 18:44
It turns out that even latest version from http://chromashift.org/x265_builds/ does not have line breaks in stat file. Why have you removed that? If it ain't broke, don't fix it ;)
Our developer (Aarthi) responded "Sorry, it was my mistake! I accidentally ommitted LF from the end of print statment when trying to refactor! It doesn't break the build as we only look for ';' to obtain the next frame's info from stats file - so I didn't notice it earlier. l'll send that fix now." She has already submitted a patch to the x265-devel mailing list.

I also asked "Do you know why x265 occasionally writes stats out of frame order?"

Aarthi explained "Yes, this happens at the very end of the encode sometimes, when a lot of frame threads are used. For the last N frames (N = # frame threads), we don't wait until the RateControlEnd() of previous frame is called before calling the same for current frame. The encoder order block on RateControlEnd() is not imposed as there are no more frames in the input queue and this won't affect the flow. So, the frame stats for frame N can be printed before frame N-1. This doesn't affect 2 pass rate control, since in the second pass we index the frame data in the array based on the POC number obtained from each line of the stats file only."

LigH
11th August 2014, 05:59
x265 1.2+510-2bdcfcc1bb33 (https://www.mediafire.com/download/2ir3t5s7i659c3b/x265_1.2+510-2bdcfcc1bb33.7z) adds the linebreak in stats files back.

a5180007
12th August 2014, 09:37
This doesn't affect 2 pass rate control, since in the second pass we index the frame data in the array based on the POC number obtained from each line of the stats file only."

Not sure I understand this. How can the POC be correct if the decoding order is not in order, especially when the 2nd pass takes line number and not decoding number ?

greenfountain
12th August 2014, 11:31
How can the POC be correct if the decoding order is not in order, especially when the 2nd pass takes line number and not decoding number ?

in : -> poc or display order. out : ->encode order. the actual encoding is done in order; only the printf to stat file is out of order for the last few frames whenever Frame parallelism is enabled.
That being said, in the second pass, we assign the sliceTypes for each frame based to the data read from the stats file for the respective poc and the encode order is re-constructed within each lookahead window similar to first pass based on the slicetype assigned.

x265_Project
12th August 2014, 15:34
Not sure I understand this. How can the POC be correct if the decoding order is not in order, especially when the 2nd pass takes line number and not decoding number ?

The 2nd pass doesn't just read the stats file line by line, assuming that stats are written in encode order. x265 finds the correct line for the frame it is encoding.