So the problem is only at 10bit encoding.
My emulation of 10bit encoding with your CPU (SSE4.2 level) on i5 3450S, x265- is compiled without /GS- /GL options, x265 with:
Code:
i:\speed\1.9+141>x265- --asm=SSE4.2 ../ducks_take_off_1080p50.y4m w.hevc
y4m [info]: 1920x1080 fps 50/1 i420p8 sar 1:1 frames 0 - 499 of 500
raw [info]: output file: w.hevc
x265 [info]: HEVC encoder version 1.9+141-02d79be487d7
x265 [info]: build info [Windows][MSVC 1900][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2
x265 [info]: Main 10 profile, Level-4.1 (Main tier)
x265 [info]: Thread pool created using 4 threads
x265 [info]: frame threads / pool features : 2 / wpp(17 rows)
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge : hex / 57 / 2 / 2
x265 [info]: Keyframe min / max / scenecut : 25 / 250 / 40
x265 [info]: Lookahead / bframes / badapt : 20 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 0
x265 [info]: References / ref-limit cu / depth : 3 / on / on
x265 [info]: AQ: mode / str / qg-size / cu-tree : 1 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress : CRF-28.0 / 0.60
x265 [info]: tools: rd=3 psy-rd=2.00 signhide tmvp strong-intra-smoothing
x265 [info]: tools: lslices=6 deblock sao
x265 [info]: frame I: 2, Avg QP:35.89 kb/s: 36347.20
x265 [info]: frame P: 123, Avg QP:36.65 kb/s: 27657.64
x265 [info]: frame B: 375, Avg QP:39.35 kb/s: 4933.84
x265 [info]: Weighted P-Frames: Y:18.7% UV:12.2%
x265 [info]: consecutive B-frames: 0.8% 0.0% 0.0% 96.8% 2.4%
encoded 500 frames in 85.48s (5.85 fps), 10649.55 kb/s, Avg QP:38.67
i:\speed\1.9+141>x265 --asm=SSE4.2 ../ducks_take_off_1080p50.y4m w.hevc
y4m [info]: 1920x1080 fps 50/1 i420p8 sar 1:1 frames 0 - 499 of 500
raw [info]: output file: w.hevc
x265 [info]: HEVC encoder version 1.9+141-02d79be487d7
x265 [info]: build info [Windows][MSVC 1900][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2
x265 [info]: Main 10 profile, Level-4.1 (Main tier)
x265 [info]: Thread pool created using 4 threads
x265 [info]: frame threads / pool features : 2 / wpp(17 rows)
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge : hex / 57 / 2 / 2
x265 [info]: Keyframe min / max / scenecut : 25 / 250 / 40
x265 [info]: Lookahead / bframes / badapt : 20 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 0
x265 [info]: References / ref-limit cu / depth : 3 / on / on
x265 [info]: AQ: mode / str / qg-size / cu-tree : 1 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress : CRF-28.0 / 0.60
x265 [info]: tools: rd=3 psy-rd=2.00 signhide tmvp strong-intra-smoothing
x265 [info]: tools: lslices=6 deblock sao
x265 [info]: frame I: 2, Avg QP:35.89 kb/s: 36347.20
x265 [info]: frame P: 123, Avg QP:36.65 kb/s: 27657.64
x265 [info]: frame B: 375, Avg QP:39.35 kb/s: 4933.84
x265 [info]: Weighted P-Frames: Y:18.7% UV:12.2%
x265 [info]: consecutive B-frames: 0.8% 0.0% 0.0% 96.8% 2.4%
encoded 500 frames in 84.64s (5.91 fps), 10649.55 kb/s, Avg QP:38.67
VS 2015 builds compiled with /GS- /GL options you can download from
www.msystem.waw.pl/x265