Starange glitch in x265 encoded video

[hellgauss]

I encoded a video file and I obtain a strange glitch using x265. This is the 'broken' part (7 seconds, the glitch is in the last second), extracted with virtualdub2 in direct stream copy.

https://limewire.com/d/ijLko#vUofJ5b2FA

The problem does not occour on every player, it seems to be related to hardware decoding on windows pc and integrated intel graphics, but I'm not 100% sure. On TV via usb the file is OK.

I tryed to reencode the video but the same file is generated, as well as I checked for errors using virtualdub2 or ffmpeg ( -v error ) and the stream seems to be ok

Do someone else experienced similar issue? Is the file hevc-compliant?

Thanks you for any feedback.

[Selur]

Does play fine on MacOS (Player: mpv)
looking the clip with mediainfo:

Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main 10@L4@Main
Codec ID : V_MPEGH/ISO/HEVC
Duration : 8 s 716 ms
Bit rate : 5 319 kb/s
Width : 1 280 pixels
Height : 720 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0 (Type 0)
Bit depth : 10 bits
Bits/(Pixel*Frame) : 0.241
Stream size : 5.53 MiB (98%)
Writing library : x265 4.1:[Windows][GCC 10.2.0][64 bit] 10bit
Encoding settings : cpuid=1111039 / frame-threads=1 / numa-pools=none / no-wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=1280x720 / interlace=0 / total-frames=0 / level-idc=40 / high-tier=1 / uhd-bd=0 / ref=6 / no-allow-non-conformance / no-repeat-headers / annexb / no-aud / no-eob / no-eos / no-hrd / info / hash=0 / temporal-layers=0 / open-gop / min-keyint=23 / keyint=250 / gop-lookahead=0 / bframes=16 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=99 / lookahead-slices=0 / scenecut=42 / no-hist-scenecut / radl=0 / no-splice / no-intra-refresh / ctu=64 / min-cu-size=8 / rect / amp / max-tu-size=32 / tu-inter-depth=4 / tu-intra-depth=4 / limit-tu=0 / rdoq-level=2 / dynamic-rd=0.00 / no-ssim-rd / signhide / tskip / nr-intra=10 / nr-inter=25 / no-constrained-intra / strong-intra-smoothing / max-merge=5 / limit-refs=0 / no-limit-modes / me=3 / subme=4 / merange=58 / temporal-mvp / no-frame-dup / no-hme / weightp / weightb / no-analyze-src-pics / deblock=1:1 / sao / no-sao-non-deblock / rd=6 / selective-sao=2 / no-early-skip / rskip / rskip-edge-threshold=0.020000 / no-fast-intra / no-tskip-fast / no-cu-lossless / b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=0.70 / psy-rdoq=0.80 / no-rd-refine / no-lossless / cbqpoffs=1 / crqpoffs=1 / rc=crf / crf=18.6 / qcomp=0.59 / qpstep=4 / stats-write=0 / stats-read=0 / vbv-maxrate=8000 / vbv-bufsize=12000 / vbv-init=0.9 / min-vbv-fullness=50.0 / max-vbv-fullness=80.0 / crf-max=0.0 / crf-min=0.0 / ipratio=1.40 / pbratio=1.30 / aq-mode=3 / aq-strength=0.70 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=1 / overscan=0 / videoformat=5 / range=0 / colorprim=1 / transfer=1 / colormatrix=1 / chromaloc=1 / chromaloc-top=0 / chromaloc-bottom=0 / display-window=0 / cll=0,0 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / no-opt-qp-pps / no-opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / no-opt-cu-delta-qp / no-aq-motion / no-hdr10 / no-hdr10-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=0 / analysis-save-reuse-level=0 / analysis-load-reuse-level=0 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=1 / refine-ctu-distortion=0 / limit-sao / ctu-info=0 / no-lowpass-dct / refine-analysis-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei / no-hevc-aq / no-svt / no-field / qp-adaptation-range=1.00 / scenecut-aware-qp=0conformance-window-offsets / right=0 / bottom=0 / decoder-max-rate=0 / no-vbv-live-multi-pass / no-mcstf / no-sbrc / no-frame-rc

one of these:
... / ref=6 / ... /bframes=16 / b-adapt=2 / b-pyramid / ...
is likely the troublemaker for some hardware decoders.
I would try:
1. disabling b-pyramid
2. lowering the b-frame count to maybe 6
3. lower ref count to maybe 4

Cu Selur

[rwill]

I am not so sure.

I can reproduce it with DXVA (Windows Video Hardware Stuff API) on an NVidia card.

It kinda looks like this using the reference software:


Layer 0 POC 255 TId: 0 ( p-SLICE, QP 24 ) [DT 0.154] [L0 245 240 235 230 225 223 ] [L1 ] [:,(unk)]
Layer 0 POC 250 TId: 0 ( b-SLICE, QP 25 ) [DT 0.147] [L0 245 240 235 230 223 ] [L1 255 ] [:,(unk)]
Layer 0 POC 246 TId: 0 ( b-SLICE, QP 26 ) [DT 0.175] [L0 245 240 235 230 ] [L1 250 255 ] [:,(unk)]
Layer 0 POC 247 TId: 0 ( b-SLICE, QP 26 ) [DT 0.164] [L0 245 240 235 230 ] [L1 250 255 ] [:,(unk)]
Layer 0 POC 248 TId: 0 ( b-SLICE, QP 26 ) [DT 0.177] [L0 245 240 235 230 ] [L1 250 255 ] [:,(unk)]
Layer 0 POC 249 TId: 0 ( b-SLICE, QP 26 ) [DT 0.166] [L0 245 240 235 230 ] [L1 250 255 ] [:,(unk)]
Layer 0 POC 251 TId: 0 ( b-SLICE, QP 26 ) [DT 0.158] [L0 250 245 240 235 230 ] [L1 255 ] [:,(unk)]
Layer 0 POC 252 TId: 0 ( b-SLICE, QP 26 ) [DT 0.173] [L0 250 245 240 235 230 ] [L1 255 ] [:,(unk)]
Layer 0 POC 253 TId: 0 ( b-SLICE, QP 26 ) [DT 0.173] [L0 250 245 240 235 230 ] [L1 255 ] [:,(unk)]
Layer 0 POC 254 TId: 0 ( b-SLICE, QP 26 ) [DT 0.160] [L0 250 245 240 235 230 ] [L1 255 ] [:,(unk)]
Layer 0 POC 259 TId: 0 ( p-SLICE, QP 24 ) [DT 0.183] [L0 255 250 245 240 235 230 ] [L1 ] [:,(unk)] ** here the glitch starts


DXVA duplicates the picture with POC 255 (picture 181 of your sample) into POC 259 (picture 185 of your sample), at least thats the output. This breaks subsequent decoding order motion compensation. I am not so sure whats going on because I cannot look into the DXVA workings. If you can reproduce it with Intel it might be a Microsoft thing. But then again maybe x265 is doing some weired buffer stuff, I cannot really tell, I am no decoder guy.

Software decoders output the correct decoded picture and work correctly, at least the ones I tested.

[hellgauss]

Thanks for your replies!

@Selur
I do not think that is the problem. I had always used that preset with very minor tuning for 720p encodes and never had problems. Also, for best compatibility, I used level 4.0. If options are not compatible with level, x265 reports a warning. I'm 99.99% sure it is an encoder or a decoder bug.

@rwill
I did not fully understand the log, but from your answer it seems that it is probably a decoder bug.

[nevcairiel]

I tried different hardware and different APIs to access the hardware, and all tested configurations show the same issue, which rules out the problem being a particular video driver, or the software feeding the data to the driver, unless they happen to have the exact same bug.
So the most likely issue is that the stream somehow violates the conformance in some way that hardware is much more susceptible to then a software decoder.

[Ritsuka]

Could it be this one https://bitbucket.org/multicoreware/x265_git/commits/1e993ee88063103904867ced35f7fcfc8e0748cf ?
Does it happens with a x265 from the main git branch?

[hellgauss]

I used Animmouse ffmpeg stable autobuild of 7 march 25 , 64 bit.
https://github.com/AnimMouse/ffmpeg-stable-autobuild/releases/tag/2025-03-07-17-13-6ae81d8-7.1.1

[Z2697]

It's not the problem with weighted prediction, just checked.
Can you "hint" the way to get the input video, for testing?

(does this not violate forum rule? :p)
This is probably very source-specific so there's no better way I think.

[hellgauss]

The source is copyrighted material, I think it is reasonably ok to share here 7 second of encoded video for technical discussion (e.g. I see a lot of screenshots on the board), but I prefer not to share the source. And if you change the source a little bit the bug goes away, you should catch the same GOP and key frame, so it is useless to share a source sample.

The origianal is Uma Musume Pretty Derby, Season 3 Ep 10, @8Mbits perhaps you can somewhat find it somewhere. The glitch is at 20:45. Please note that a few seconds before there is a nearly identical scene which is encoded/decoded well.

Command line used:
ffmpeg -threads 1 -y -hide_banner -nostdin -stats_period 15 -bitexact -i "u.mkv" -map 0 -profile:v main10 -vf "removegrain=1:1:1,format=yuv420p10le,scale=1280:720:flags=bicublin:param0=0.125:param1=0.4375" -sws_flags accurate_rnd -c:v libx265 -preset veryslow -crf 18.6 -x265-params "psy-rd=0.7:psy-rdoq=0.8:aq-mode=3:aq-strength=0.7:deblock=1,1:nr-intra=10:nr-inter=25:tu-intra-depth=4:tu-inter-depth=4:merange=58:ref=6:rc-lookahead=99:bframes=16:rd=6:rskip=2:rskip-edge-threshold=2:cbqpoffs=1:crqpoffs=1:scenecut=42:scenecut-bias=5.2:qcomp=0.59:tskip=1:tskip-fast=0:limit-sao=1:selective-sao=2:vbv-maxrate=8000:vbv-bufsize=12000:frame-threads=1:pools=none:no-wpp=1:level-idc=40:high-tier=1:colorprim=1:colormatrix=1:transfer=1:range=limited" -bitexact "out.mp4"

I performed another encode with the official BtbN build and it is ok. However frames size are slightly different in the whole encode. My opinion is that it is a bug that manifest only in very specific circumstances combo. Next days I will try to change some parameters and to reproduce the glitch on a short sample.

[jpsdr]

Just for information, played the test file your provided on a HW player : Zappity Neo, didn't notice any glitch.

[benwaggoner]

Try bframes=8. There aren't any presets that use more than that in any encoder I am aware of, so it'll be a much less tested scenario. And I doubt >8 is providing much extra compression efficiency for you. Although grain-free anime is the scenario where it would be most likely to have any positive impact. That may be exposing a rare bug, as most real-world content wouldn't actually use >8 consecutive b-frames even if the bframe value is >8.

[hellgauss]

Actually slower + anime leads to brames=10. That clip contains maximum 9 bframes. And in slow and clean anime, or anime with somewhat "repeated" frames, they are often used.

The specific problem for me is no longer for that video, since I performed a test encode with another build and it is ok. But I'm interested to know if it is an encoder bug (i.e. the stream is not compliant with hevc or with level) or decoder bug. If it is an encoder bug, I assume that it can be present even in the official build, which simply avoided the specific circumstances of animmouse build.

It is interesting to notice that two hardware decoder (my tv and jpsdr's Zappity) are ok. I still bet for decoder bug, but I'll do some tests when I have time.

[benwaggoner]

Actually slower + anime leads to bframes=10. That clip contains maximum 9 bframes. And in slow and clean anime, or anime with somewhat "repeated" frames, they are often used.
Yeah, cel animation is the primo use case for very high bframe counts. But that doesn't mean they've been tested thoroughly. If you have to debug this again sometime, you might try setting it to 8 and seeing if it changes anything. It likely won't, but was the only thing that stood out for me a little.

The specific problem for me is no longer for that video, since I performed a test encode with another build and it is ok. But I'm interested to know if it is an encoder bug (i.e. the stream is not compliant with hevc or with level) or decoder bug. If it is an encoder bug, I assume that it can be present even in the official build, which simply avoided the specific circumstances of animmouse build.

It is interesting to notice that two hardware decoder (my tv and jpsdr's Zappity) are ok. I still bet for decoder bug, but I'll do some tests when I have time.
rwill above identified a POC issue that seems a likely culprit, and may violate the spec in some small way.

Real-world decoders generally get lots of fuzz testing with a variety of non-compliant streams, so they are often able to pay things that aren't perfectly spec legal. Making a decoder crash in an interesting way has been a big source of security holes over the decades, so they really work to make sure all kinds of wrong things either still work or result in a safe exit and error message/black frame.

[rwill]

rwill above identified a POC issue that seems a likely culprit, and may violate the spec in some small way.

Real-world decoders generally get lots of fuzz testing with a variety of non-compliant streams, so they are often able to pay things that aren't perfectly spec legal. Making a decoder crash in an interesting way has been a big source of security holes over the decades, so they really work to make sure all kinds of wrong things either still work or result in a safe exit and error message/black frame.

Its not really a POC issue and I am still somewhat puzzled. It looks like the decoder copies the first reference frame from L0 into the glitch picture framebuffer and does nothing else. The first glitch picture itself looks normal so there are no decoding artifacts or so, its just the wrong picture. Subsequent pictures are broken because the glitch picture is the wrong picture and so motion compensation fails. I checked the decoded picture buffer management and picture indices look ok to me.

Might be that something triggers some sort of error concealment. But I have not found out what that might be. The picture triggering the issue is a single P slice, so really simple, I do not know yet what could go wrong there that might trip off hardware decoders but not software decoders.

[benwaggoner]

Might be that something triggers some sort of error concealment. But I have not found out what that might be. The picture triggering the issue is a single P slice, so really simple, I do not know yet what could go wrong there that might trip off hardware decoders but not software decoders.
Yeah, when some decoders work and others don't it is often because of error concealment.

Have you decoded just that p-slice, and the correct one, and compared them visually? Might be a clue in there.

[rwill]

Have you decoded just that p-slice, and the correct one, and compared them visually? Might be a clue in there.

Yes, they are bit identical.

[Z2697]

Have we tested this on AMD and Intel decoders?

Zen4 igpu decoder seems fine...
An old i5 with HD 630 shows different looking artifact. It copied a shifted version of frame 176 into a partially decoded frame 185. (176 is next to 181 in decoding order)
That's interesting, maybe the block where it stops decoding correctly is the "offending" block.
And an interesting error message (when using FFmpeg -c:v hevc_qsv for decoding)
[hevc_qsv @ 000001dbf267fec0] A decode call did not consume any data: expect more data at input (-10)

[jpsdr]

No one has some kind of H265 compliant data stream analyzer tool ?

[Z2697]

I'd say HM is good enough...
Having a lot more assertions in the code.

(JM? not so good.
https://web.archive.org/web/20100228010450/http://x264dev.multimedia.cx/?p=212
Edit/Note: It seems that earlier we were testing with a stream that was actually invalid; the reference decoder didn’t complain, but after putting it through a stream validator, it found an obvious issue (which we fixed in x264 r1342). As such, the list is a lot shorter now.
Also weighted prediction? LOL
That's a slightly different bug, I think, HEVC changed how chroma offset is coded)

[GeoffreyA]

Have we tested this on AMD and Intel decoders?

Zen4 igpu decoder seems fine...
An old i5 with HD 630 shows different looking artifact. It copied a shifted version of frame 176 into a partially decoded frame 185. (176 is next to 181 in decoding order)
That's interesting, maybe the block where it stops decoding correctly is the "offending" block.
And an interesting error message (when using FFmpeg -c:v hevc_qsv for decoding)
[hevc_qsv @ 000001dbf267fec0] A decode call did not consume any data: expect more data at input (-10)

Shows up on Arc B580 hardware decoding as well. Software decoding is all right.

[SquallMX]

I had a similar issue (https://forum.doom9.org/showthread.php?t=185706) (hardware decoding on nVidia was buggy, software decoding was OK), the culprit was "--rskip 2". Using "--rskip 1" fixed the issue but increased encoding time a lot.

[Z2697]

I don't think so... rskip 2 is not that different from rskip 1, it just uses different metric to make skip decision.

[rwill]

Have we tested this on AMD and Intel decoders?

Zen4 igpu decoder seems fine...
An old i5 with HD 630 shows different looking artifact. It copied a shifted version of frame 176 into a partially decoded frame 185. (176 is next to 181 in decoding order)
That's interesting, maybe the block where it stops decoding correctly is the "offending" block.
And an interesting error message (when using FFmpeg -c:v hevc_qsv for decoding)
[hevc_qsv @ 000001dbf267fec0] A decode call did not consume any data: expect more data at input (-10)

Can you upload the .yuv output from the i5 quicksync somewhere? I don't have any Intel systems available currently.

[Z2697]

Can you upload the .yuv output from the i5 quicksync somewhere? I don't have any Intel systems available currently.

I have it losslessly compressed in H264
https://workupload.com/file/VgTXNyAtrbF

[rwill]

Thanks Z2697, this got me further. I hope I got the numbers right here.

The block in question, POC 259, CTU 73 is a single 64x64 INTER_2Nx2N coded CU.

Due to certain mechanics (motion vector predictor POC scaling) the motion vector predictor (MVP) ends up being:
X= -32768, Y= -19437
with the alternative (not used) being
X= -32768, Y= -19506

So x265 codecs the motion vector difference as
MVD X= 32768, Y= 18385

so the final MV in the decoder is
MVD X= 0, Y= -1052

The element of a MVP can take the range of -32768 to 32767, so X is at the maximum here.

It appears the NVidia and Intel implementation gets confused by the large MVD X, which would take 17 bit to represent if signed, I don't know how they have implemented it in hardware though.

The picture where the glitch starts is the only picture in the sequence with such a large MVD.

I further inspected SquallMX his stream and there his glitch picture is also the only picture in the stream with a MVD greater or equal 2^15.

I suggest the x265 ppl to limit their motion vectors to sane ranges so hellgaus can try again. Letting MVs get out of hand is just bad style, even if it saves some bits.

[hellgauss]

Thanks for nice reverse engineering, although I did not understand everything! Sometimes bugs are very subtle to find.

In the end, is the stream hevc-compliant and it was decoder issue? For motion range, I replaced merange=57 with merange=58, I read in reference that "If the search range were any larger than this, another CTU row of latency would be required for reference frames." (but I do not know what does it means). For "hex" -me, the limit is 57, but i read that it can be raised to 58 for "star".

[rwill]

Well, to me it looks like a decoder issue. There is also currently nothing you can do with x265 configurations to avoid it.

I quickly scanned the standard for some motion vector delta limit but was not able to find something specific. There is an equation that puts the final motion vector in the range -2^15 to (2^15)-1 but there does not seem to be a limit on the coded delta. The delta just gets wrapped each 2^16 so internaly only the lower 16 bits need to be kept of the coded MVD I guess. Maybe this is where things go wrong, I don't know.

[Z2697]

Well, to me it looks like a decoder issue. There is also currently nothing you can do with x265 configurations to avoid it.

Disable temporal-mvp? :devil:

[Z2697]

So x265 codecs the motion vector difference as
MVD X= 32768, Y= 18385


The MVD X I got from HM decoder is -32768...
Which means the MVP is 32768? Int16 Overflow?
Either way, one of them is 32768. And is greater than maximum signed 16bit integer.

Wait... the sign is probably saved separately in the bitstream...


Void TDecSbac::parseMvd( TComDataCU* pcCU, UInt uiAbsPartIdx, UInt uiPartIdx, UInt uiDepth, RefPicList eRefList )
{
UInt uiSymbol;
UInt uiHorAbs;
UInt uiVerAbs;
UInt uiHorSign = 0;
UInt uiVerSign = 0;
...


I'd say this is completely decoders' fault.

[Z2697]

If the MVP is causing problem, maybe they stored the value in a single signed int16 internally, the problem is more complicated than MVD, right?

I mean, if the encoder should do workaround for the decoders' fault.
What are you gonna do? Going all the way back to the reference block and redo its motion search?

[GeoffreyA]

Well, if it's a decoder fault, I'd be happy to submit the issue on the Intel side. Hopefully, something can be done in the driver. Perhaps somebody else can tackle Nvidia.

[rwill]

I did some more MVD experiments:

https://drive.google.com/drive/folders/1jlpHekxYG_ZL7-OLTl3rYQ_0GtfcRqIG?usp=sharing

The problem gets triggered even with the small MVPs here, so MVP value seems unrelated.

MVD in short, for my GTX 1650 (don't judge plz): -0x8000 works, 0x7fff works, -0x8001 does not work, 0x8000 does not work.

The quick fix I think that can be applied to an encoder is to cast the MVD to int16 and back to int32 before writing.

[Z2697]

MVD in short, for my GTX 1650 (don't judge plz): -0x8000 works, 0x7fff works, -0x8001 does not work, 0x8000 does not work.

But is the MVD in the sample 32768 or -32768?
Perhaps MVP and MVD both can trigger the bug?

How are you able to make them arbitrary numbers?

[rwill]

But is the MVD in the sample 32768 or -32768?
Perhaps MVP and MVD both can trigger the bug?

How are you able to make them arbitrary numbers?

In hellgauss his sample? 32768 (positive).

For my experiments I modified an encoder to output such MVDs.

[Z2697]

In hellgauss his sample? 32768 (positive).

For my experiments I modified an encoder to output such MVDs.

I can figure that out... I'm asking how to modify the encoder, I know it's your company's proprietary encoder, but is there a general idea that can be reproduced in x265?

[rwill]

Rule of three: motion vector = 0x8000 + MVP.

Somewhere after the inter search. Best done with 2Nx2N inter at depth = 0.

[Z2697]

In hellgauss his sample? 32768 (positive).

For my experiments I modified an encoder to output such MVDs.

This thing here in HM really tells me that the MVD is -32768...
Am I misunderstanding something?
https://vcgit.hhi.fraunhofer.de/jvet/HM/-/blob/master/source/Lib/TLibDecoder/TDecEntropy.cpp?ref_type=heads#L279

[rwill]

This thing here in HM really tells me that the MVD is -32768...
Am I misunderstanding something?
https://vcgit.hhi.fraunhofer.de/jvet/HM/-/blob/master/source/Lib/TLibDecoder/TDecEntropy.cpp?ref_type=heads#L279

Thats because it returns the value of a motion vector which class is defined here:

https://vcgit.hhi.fraunhofer.de/jvet/HM/-/blob/master/source/Lib/TLibCommon/TComMv.h?ref_type=heads#L51

where the X and Y are defined as a 16 bit Short.

One has to check where the Mvd is set at

https://vcgit.hhi.fraunhofer.de/jvet/HM/-/blob/master/source/Lib/TLibDecoder/TDecSbac.cpp?ref_type=heads#L844

[Z2697]

I understand it now... so HM and FFmpeg also store the internal MVD as int16, and -32768 is the overflowed result of the actual 32768.
But how do they get the decoding result right? I'm confused.

And now it's x265's fault to produce such value?

[rwill]

A motion vector element is always in the range -2^15 to (2^15)-1.

Its in the standard at:

8.5.3.2 Derivation process for motion vector components and reference indices
8.5.3.2.1 General
Equation 8-94 to 8-97


Which more or less defines that motion vector elements are stored in int16.


uLX[ 0 ] = ( mvpLX[ 0 ] + mvdLX[ 0 ] + 2^16 ) % 2^16
mvLX[ 0 ] = ( uLX[ 0 ] >= 2^15 ) ? ( uLX[ 0 ] − 2^16 ) : uLX[ 0 ]


So for the final motion vector element X mvLX[ 0 ] it does not matter if a 0x8000 MVD overflowed or not because:

( ( mvpLX[ 0 ] + 0x8000 + 2^16 ) % 2^16 ) == ( ( mvpLX[ 0 ] + -0x8000 + 2^16 ) % 2^16 )


So the issue most likely can be avoided by
- Clipping motion vectors to sane ranges
- Casting the MVD to "int16" before writing

But as I wrote above, I have not found any actual limits regarding the MVDs in the bitstream in the standard.

[GeoffreyA]

With the view that this is a decoder issue, I posted the bug on Intel's side:

https://github.com/IGCIT/Intel-GPU-Community-Issue-Tracker-IGCIT/issues/1153

[Z2697]

I think this is both decoder and encoder issue :p

[GeoffreyA]

I think this is both decoder and encoder issue :p

Oh, well, let's see what happens :)

[Z2697]

There is some sort of limit in x265... is this not enough?
Or is it something with the specific binary? I can't reproduce the Uma Musume bug in my test build of x265 stable branch, and can reproduce using the FFmpeg from AnimMouse.
https://bitbucket.org/multicoreware/x265_git/src/cd4f0d6e9d30766f5e2be89dab2e6809f3b76507/source/encoder/search.cpp#lines-4523:4527

[GeoffreyA]

What changes are in AnimMouse's version?

[hellgauss]

As far as I can tell, it takes some time to receipt every changes/improvements of the official ffmpeg build. I use it because it is compiled with libfdk for aac encode. The building process seems to be automated.

Perhaps the bug pointed by Ritsuka at the beginning of the thread has still not be fixed in Animmouse build?

PS sorry for not being more precise, but I am not so skilled with git, builds, scrutinize complex code etc... I'm just able to do basic things in c and other languages.

[GeoffreyA]

Yes, it could be something fixed, in x265, in the past few months. I was under the impression that AnimMouse's build was a fork.

[Z2697]

Yes, the weighted prediction bug is not fixed in current AnimMouse build because it uses stable branch of x265, which basically is the "lastest release version".
There haven't been a release after that bug fix.
But, that bug is not what caused your problem.

There's no modification being done in that build, other than a CMake patch.
https://github.com/rdp/ffmpeg-windows-build-helpers/blob/083f55b7cbad2fb4339dca9ef26fe4f647abeeb4/patches/x265_x86_noasm_fix.patch

But some compiler bug is not impossible...
(For example if you disable too many compiler cpu flag in a X86 build, the ratecontrol is completely fried)

[hellgauss]

I noticed that the bug occurs at POC=256. If I use trace_header BSF in ffmpeg, POC is reset at zero after 255 (it seems to be only one byte).

Perhaps some weird things happens in the decoder which overwrites some carry flag?

[benwaggoner]

I noticed that the bug occurs at POC=256. If I use trace_header BSF in ffmpeg, POC is reset at zero after 255 (it seems to be only one byte).

Perhaps some weird things happens in the decoder which overwrites some carry flag?
If it is supposed to be 0-255 not 1-256, a 256 could be an issue.

If so, that's one of the classic programming bugs.

[rwill]

I noticed that the bug occurs at POC=256. If I use trace_header BSF in ffmpeg, POC is reset at zero after 255 (it seems to be only one byte).

Perhaps some weird things happens in the decoder which overwrites some carry flag?

Yeah, must be that one number you can view with ffmpeg and not one of the million numbers you cannot. /s

[rwill]

There is some sort of limit in x265... is this not enough?
Or is it something with the specific binary? I can't reproduce the Uma Musume bug in my test build of x265 stable branch, and can reproduce using the FFmpeg from AnimMouse.
https://bitbucket.org/multicoreware/x265_git/src/cd4f0d6e9d30766f5e2be89dab2e6809f3b76507/source/encoder/search.cpp#lines-4523:4527

The limits in place, and the way this function you linked is called, are quite ineffective for motion vector clipping.

And you probably cannot reproduce the issue because its one of those "when the stars align" things. You need the same input, same platform, and same configuration to even have a chance to repro.

[Z2697]

The bug "starts in display order" at POC 256, but in reality it starts at POC 259 in decoding order.

[GeoffreyA]

Felipe from Intel would like to know if anyone tested this on the RX 7000 and 9000 series (RDNA 3 and 4)? If someone could check, that would be great.

https://github.com/IGCIT/Intel-GPU-Community-Issue-Tracker-IGCIT/issues/1153#issuecomment-3033059227

[Z2697]

I'm planning to buy a RX 9070 XT but that's not gonna happen very soon...
If this is fixable at the driver level that's a good news.

[GeoffreyA]

That's a potent GPU.

[benwaggoner]

The limits in place, and the way this function you linked is called, are quite ineffective for motion vector clipping.

And you probably cannot reproduce the issue because its one of those "when the stars align" things. You need the same input, same platform, and same configuration to even have a chance to repro.
Definitely file it as a bug with MultiCoreWare!

[GeoffreyA]

Definitely file it as a bug with MultiCoreWare!

That seems to be best.

Intel's analysis says that frame 182 is corrupted, and AMD and software decoding are handling it differently. Any further questions of them before closing the ticket?

https://github.com/IGCIT/Intel-GPU-Community-Issue-Tracker-IGCIT/issues/1153#issuecomment-3049365819

[hellgauss]

Quite cryptic:
"Based on the analysis this is a decode issue. The first corrupted frame is 182"

Perhaps Z2697 is right and it is both an encoder and decoder issue :)

[benwaggoner]

Quite cryptic:
"Based on the analysis this is a decode issue. The first corrupted frame is 182"

Perhaps Z2697 is right and it is both an encoder and decoder issue :)
Lots of things are both, yeah. Although the standard really defines conformance of the encoded bitstream, so getting that right is the most important. How a decoder deals with malformed bitstreams is undefined, although it is always welcome when they do so gracefully.

[Z2697]

7.4.9.9 Motion vector difference semantics
...
The value of MvdLX[ x0 ][ y0 ][ compIdx ] shall be in the range of −215
to 215 − 1, inclusive.


I think this is where the range of MVD is specified, and it is [-2^15, 2^15-1].
The pure text quote is hard to read, so I suggest go read the PDF.

[Z2697]

Are we going to send a patch to multicoreware?

[Z2697]

That seems to be best.

Intel's analysis says that frame 182 is corrupted, and AMD and software decoding are handling it differently. Any further questions of them before closing the ticket?

https://github.com/IGCIT/Intel-GPU-Community-Issue-Tracker-IGCIT/issues/1153#issuecomment-3049365819

The bitstream violates the specification, yes.
But if the decoding process is done exactly according to the specification, the result is "not affected" in this case.

[GeoffreyA]

The bitstream violates the specification, yes.
But if the decoding process is done exactly according to the specification, the result is "not affected".

I see. So their decoding process is not following the specification, exactly?

Are we going to send a patch to multicoreware?

Why not? :)

[Z2697]

I see. So their decoding process is not following the specification, exactly?

Can't be sure, without the knowledge about their internal design...
But very likely.

https://forum.doom9.org/showthread.php?p=2020241#post2020241
This kind of decoding process looks very like a low-hanging fruit for optimization.

[GeoffreyA]

Can't be sure, without the knowledge about their internal design...
But very likely.

https://forum.doom9.org/showthread.php?p=2020241#post2020241
This kind of decoding process looks very like a low-hanging fruit for optimization.

Yes, my thought is, optimisation.

I wonder, though, if this is locked down in the hardware, or if it can be changed in the driver. Intel does seem to use a generalised encoding-decoding engine, like a miniature video CPU, so the logic might be in firmware.

[rwill]

I see. So their decoding process is not following the specification, exactly?


It is according to the specification as it supports the full legal value range.

I wonder why I have missed the range limit in 7.4.9.9 when reading the standard.. too much selective reading caused by social media probably.

[Z2697]

Of course, the culprit is the encoder encodes illegal values.
And that decoding process is over-engineered.
We can't blame the decoders really.

But the evidence does show that they probably optimized the over-engineered process out.

[Z2697]

It is according to the specification as it supports the full legal value range.

I wonder why I have missed the range limit in 7.4.9.9 when reading the standard.. too much selective reading caused by social media probably.
Those things are scattered all around like hell...

Can I add you as a co-author in the patch?
Or maybe you want to send the patch?

[jpsdr]

If you're doing a patch, i'm interested by it to apply asap on my build, not to have to wait a long time it goes to main branch.

[rwill]

Those things are scattered all around like hell...

Can I add you as a co-author in the patch?
Or maybe you want to send the patch?

I would like to stay out of it for political reasons... ;D

[Z2697]

My patch is gonna be look like this:

diff --git a/source/encoder/motion.cpp b/source/encoder/motion.cpp
index 86f413c3d..a3fe0619d 100644
--- a/source/encoder/motion.cpp
+++ b/source/encoder/motion.cpp
@@ -1604,6 +1604,17 @@ me_hex2:
bmv = MV(0, 0);
}

+ // to ensure the mvdLX is in the range of [-2^15, 2^15-1]
+ MV clipmin((int32_t) -(1<<15), (int32_t) -(1<<15));
+ MV clipmax((int32_t) (1<<15) - 1, (int32_t) (1<<15) - 1);
+ clipmin += qmvp;
+ clipmax += qmvp;
+ if (!bmv.checkRange(clipmin, clipmax))
+ {
+ bmv = bmv.clipped(clipmin, clipmax);
+ bcost = subpelCompare(ref, bmv, satd) + mvcost(bmv);
+ }
+
x265_emms();
outQMv = bmv;
return bcost;



I chose to do it in motionEstimate because I want to update the cost as well... however it's probably not that important after all.
If the clipping is done in the predInterSearch we can probably save some CPU cycles.
What's your thought?

[jpsdr]

I don't have the knowledge for technical answer, so just some thought.

If you wanted to update the cost, it's probably because you thought that, even if impact is very small, the result will be more accurate, so globaly it will produce a better result, even if it's minimal.
As i'm looking for "best" result, i don't care saving some CPU cycles if solution is better than another. Even if the "better" is very small, sometimes it's the sum of several very small things which begins to produce a finaly not so much small "better".

[Z2697]

Uh, it's just my assumption that the cost will be slightly different.
But maybe it will be too small to make any change in the RDO process.
And given that this bug is only occured like, once in a million encodes, as far as I can tell / estimate from the fact that "this is the first time I saw it", I think I'd rather save that CPU resource.
(I didn't actually watched a million x265 encoded videos though ;) )

So here's the predInterSearch version:
diff --git a/source/encoder/search.cpp b/source/encoder/search.cpp
index 0522f52cc..5f3fca5b6 100644
--- a/source/encoder/search.cpp
+++ b/source/encoder/search.cpp
@@ -2543,6 +2543,12 @@ void Search::predInterSearch(Mode& interMode, const CUGeom& cuGeom, bool bChroma
}
}

+ // to ensure the mvdLX is in the range of [-2^15, 2^15-1]
+ MV clipmin((int32_t) -(1<<15) , (int32_t) -(1<<15) );
+ MV clipmax((int32_t) (1<<15) - 1, (int32_t) (1<<15) - 1);
+ bestME[0].mv = bestME[0].mv.clipped(bestME[0].mvp + clipmin, bestME[0].mvp + clipmax);
+ bestME[1].mv = bestME[1].mv.clipped(bestME[1].mvp + clipmin, bestME[1].mvp + clipmax);
+
/* Bi-directional prediction */
MotionData bidir[2];
uint32_t bidirCost = MAX_UINT;
@@ -2648,6 +2654,11 @@ void Search::predInterSearch(Mode& interMode, const CUGeom& cuGeom, bool bChroma
bidirBits = bits0 + bits1 + m_listSelBits[2] - (m_listSelBits[0] + m_listSelBits[1]);
}
}
+ // to ensure the mvdLX is in the range of [-2^15, 2^15-1]
+ MV clipmin((int32_t) -(1<<15) , (int32_t) -(1<<15) );
+ MV clipmax((int32_t) (1<<15) - 1, (int32_t) (1<<15) - 1);
+ bidir[0].mv = bidir[0].mv.clipped(bidir[0].mvp + clipmin, bidir[0].mvp + clipmax);
+ bidir[1].mv = bidir[1].mv.clipped(bidir[1].mvp + clipmin, bidir[1].mvp + clipmax);
}

/* select best option and store into CU */

[GeoffreyA]

MV is a class with no dynamic allocation?

I think the motionEstimate() version looks cleaner and simpler.

[Z2697]

MV is a class with no dynamic allocation?

I think the motionEstimate() version looks cleaner and simpler.

What do we need dynamic allocation for, in this case?

2 versions are basically the same thing, in different places.

[GeoffreyA]

What do we need dynamic allocation for, in this case?

Just asking. I don't know what the definition is.

[Z2697]

The patch is sent.
I can't help but wondering, since the values we've seen are only overflowed by 1, is there some value (constant) just off by one, and we can tweak it to make the problem not happening in the first place?

[rwill]

The patch is sent.
I can't help but wondering, since the values we've seen are only overflowed by 1, is there some value (constant) just off by one, and we can tweak it to make the problem not happening in the first place?

Well we just had 1 case here where it was 1 too large. SquallMX his sample had the MVD at 0x8004, so 5 too large. So its just a coincidence.

[rwill]

Regarding your patch...

The MVP can only be between -2^15 and (2^15)-1.
Clipping the MV to MVP +/- 2^15 kind of limits the range an MV can have.

For example if the MVP X is -2^15 the X maximum is -1. So if MVP ever gets strange the ME will have a hard time to re-sanitize the MV vector field.

Might I suggest a clipping range of +/- 49152 ( 2^15 + 2^14 ) and do this additional step:


void Entropy::codeMvd(const CUData& cu, uint32_t absPartIdx, int list)
...
const int hor = mvd.x & 0xffff;
const int ver = mvd.y & 0xffff;


It should work because the ( MVP + MVD ) does the 16 bit wraparound in the decoder.

[Z2697]

Regarding your patch...

The MVP can only be between -2^15 and (2^15)-1.
Clipping the MV to MVP +/- 2^15 kind of limits the range an MV can have.

For example if the MVP X is -2^15 the X maximum is -1. So if MVP ever gets strange the ME will have a hard time to re-sanitize the MV vector field.

Might I suggest a clipping range of +/- 49152 ( 2^15 + 2^14 ) and do this additional step:


void Entropy::codeMvd(const CUData& cu, uint32_t absPartIdx, int list)
...
const int hor = mvd.x & 0xffff;
const int ver = mvd.y & 0xffff;


It should work because the ( MVP + MVD ) does the 16 bit wraparound in the decoder.

This actually makes the things much worse, if I'm not implementing it wrong:

diff --git a/source/encoder/entropy.cpp b/source/encoder/entropy.cpp
index ece8d6bc9..3826c31a7 100644
--- a/source/encoder/entropy.cpp
+++ b/source/encoder/entropy.cpp
@@ -2104,8 +2104,8 @@ void Entropy::codeRefFrmIdx(const CUData& cu, uint32_t absPartIdx, int list)
void Entropy::codeMvd(const CUData& cu, uint32_t absPartIdx, int list)
{
const MV& mvd = cu.m_mvd[list][absPartIdx];
- const int hor = mvd.x;
- const int ver = mvd.y;
+ const int hor = mvd.x & 0xffff;
+ const int ver = mvd.y & 0xffff;

encodeBin(hor != 0 ? 1 : 0, m_contextState[OFF_MV_RES_CTX]);
encodeBin(ver != 0 ? 1 : 0, m_contextState[OFF_MV_RES_CTX]);
diff --git a/source/encoder/search.cpp b/source/encoder/search.cpp
index 2a324700d..295cec52c 100644
--- a/source/encoder/search.cpp
+++ b/source/encoder/search.cpp
@@ -2545,6 +2545,12 @@ void Search::predInterSearch(Mode& interMode, const CUGeom& cuGeom, bool bChroma
}
}

+ // to ensure the mvdLX is in the range of [-2^15, 2^15-1]
+ MV clipmin((int32_t) -(1<<15) - (1<<14) , (int32_t) -(1<<15) - (1<<14) );
+ MV clipmax((int32_t) (1<<15) + (1<<14) - 1, (int32_t) (1<<15) + (1<<14) - 1);
+ bestME[0].mv = bestME[0].mv.clipped(bestME[0].mvp + clipmin, bestME[0].mvp + clipmax);
+ bestME[1].mv = bestME[1].mv.clipped(bestME[1].mvp + clipmin, bestME[1].mvp + clipmax);
+
/* Bi-directional prediction */
MotionData bidir[2];
uint32_t bidirCost = MAX_UINT;
@@ -2650,6 +2656,11 @@ void Search::predInterSearch(Mode& interMode, const CUGeom& cuGeom, bool bChroma
bidirBits = bits0 + bits1 + m_listSelBits[2] - (m_listSelBits[0] + m_listSelBits[1]);
}
}
+ // to ensure the mvdLX is in the range of [-2^15, 2^15-1]
+ MV clipmin((int32_t) -(1<<15) - (1<<14) , (int32_t) -(1<<15) - (1<<14) );
+ MV clipmax((int32_t) (1<<15) + (1<<14) - 1, (int32_t) (1<<15) + (1<<14) - 1);
+ bidir[0].mv = bidir[0].mv.clipped(bidir[0].mvp + clipmin, bidir[0].mvp + clipmax);
+ bidir[1].mv = bidir[1].mv.clipped(bidir[1].mvp + clipmin, bidir[1].mvp + clipmax);
}

/* select best option and store into CU */



I don't understand why...

[rwill]

This actually makes the things much worse, if I'm not implementing it wrong:

+ const int hor = mvd.x & 0xffff;
+ const int ver = mvd.y & 0xffff;


..

I don't understand why...

That is because I am actually an idiot. The int type of hor and ver is still 32bit, so maybe instead of my suggestion to do "& 0xffff", which will break things for sure, maybe try "const signed short" instead of "const int" .. or something like that. Maybe then it works ...

[jpsdr]

What is the type of mvd.x (or mvd.y) ?
Is it int (meaning 32 bits or 64 bits acording if the OS target is 32 or 64 bits) ?
And exactly what do you want to do with the "& 0xFFFF" ?
You realize in that case +65535 or -1 will produce the same result (if it's int) ?
(Unless mvd.x is only positive... That i don't know).

[GeoffreyA]

What is the type of mvd.x (or mvd.y) ?
Is it int (meaning 32 bits or 64 bits acording if the OS target is 32 or 64 bits) ?
And exactly what do you want to do with the "& 0xFFFF" ?
You realize in that case +65535 or -1 will produce the same result (if it's int) ?
(Unless mvd.x is only positive... That i don't know).

They are int32_t: https://bitbucket.org/multicoreware/x265_git/src/cd4f0d6e9d30766f5e2be89dab2e6809f3b76507/source/common/mv.h#lines-42

I assume it's to keep the lower 16 bits.

[Z2697]

I don't see the reason behind this extra step...
Or the extended clipping range...

[rwill]

If the MVP is -32768 (what we had here in the issue stream), if you clip to int16_t search range around the MVP, you can at most reach MV = -1 but not 0 to 32767.

So the search range has to be extended to something sane like +49152 to reach more positive values around the CU that is coded.

Then regarding coding the MVD, the HEVC standard says:

uLX[ 0 ] = ( mvpLX[ 0 ] + mvdLX[ 0 ] + 2^16 ) % 2^16
mvLX[ 0 ] = ( uLX[ 0 ] >= 2^15 ) ? ( uLX[ 0 ] − 2^16 ) : uLX[ 0 ]


So lets take for example MVP = -32768 and we want to code an MV of 1024, the MVD is 33.792 which, if reduced to signed 16 bit, is -31744.

uLX = ( -32768 + -31744 + 2^16 ) % 2^16 = 1024.

This also works the other way around with MVP 32767 and we want to code an MV of -1024, then the MVD is -33791 which, if reduced to signed 16 bit, is 31745.

uLX = ( 32767 + 31745 + 2^16 ) % 2^16 = 64512
mvLX = 64511 >= 32768 ? ==> 64512 - 2^16 = -1024

That is what I tried to do but I have a hard time reading that x265 code.

[jpsdr]

If i want to use the first motion.cpp version of the patch, do i also need to expand the clipping values ?

[Z2697]

Is it safe to just not doing the clipping part?

[Z2697]

So... since this works, Nvidia and Intel are not doing the decoding process differently, they just did some sanity check first.
So this is a encoder bug 100%.

Sorry Nvidia and Intel :o

[rwill]

Is it safe to just not doing the clipping part?

MCW should know best but I guess the guys that did the original architecture and implementation are no longer around so my guess is no one knows.

Instead of clipping to (MVP -2^15 / (2^15)-1) maybe clip to absolute (-2^15 / (2^15)-1) around the CU? These should be always reachable... but I would start the motion estimation with that constraint and don't add it as an afterthought after.

[Z2697]

There is a limit about max mv length in setSearchRange

/* Clip search range to signaled maximum MV length.
* We do not support this VUI field being changed from the default */
const int maxMvLen = (1 << 15) - 1;
mvmin.x = X265_MAX(mvmin.x, -maxMvLen);
mvmin.y = X265_MAX(mvmin.y, -maxMvLen);
mvmax.x = X265_MIN(mvmax.x, maxMvLen);
mvmax.y = X265_MIN(mvmax.y, maxMvLen);


I don't know how reliable is this, because these mvmin mvmax values clearly have been breached in this case (they do work normally most of the time)
POC=930 CTU=73 mvp=-32768,-16141 outmv=1,-1052 mvmin=-903,-263 mvmax=-903,-263

EDIT: Now I think of it, if the MV is exceeding that range, it's a different (and worse?) problem. Let's just fking go! (for now).

[jpsdr]

I don't understand what is the trouble here.
Is it because it's difficult to find where exactly do it in the code architecture ?
Even if the spec tells exactly what to restrein ?

Edit:
I didn't see an issue opened on the bitbucket.
Maybe you should cancel your patch, and just describe precisaly the issue on a ticket on the bitbucket, if not with a video sample, at least with some results like the previous post.

And i'm asking again, is the first patch you made in motion.cpp not finaly the good place ?

I mean, if spec said -32768/+32767 but for any reason at the place you clip you have to clip to higher values, maybe it means in that case the clipping is not at the right place ?

[Z2697]

I think the current method, casting the mvd to int16_t at entropy coding stage, is "as painless as it goes".

[GeoffreyA]

I think the current method, casting the mvd to int16_t at entropy coding stage, is "as painless as it goes".

I don't know encoder design or the x265 code, but wouldn't the "correct" way be to alter the code where the motion vector is calculated (i.e., that code is the culprit), clamping it there, and wouldn't clamping or casting late in the pipeline cause undefined results if earlier stages used the original value?

[Z2697]

I don't know encoder design or the x265 code, but wouldn't the "correct" way be to alter the code where the motion vector is calculated (i.e., that code is the culprit), clamping it there, and wouldn't clamping or casting late in the pipeline cause undefined results if earlier stages used the original value?

I honestly can't figure out where this bug starts.
(so I just created an issue in bitbucket but still awaits approval)

But talking about the difference made by casting it, there's really none.
The values here are, I'll say they are what will be seen by the decoder, and only the decoder will see them, and as we have discussed about, the decoding process makes correct and identical results.
(yes, there're flags to be set according to the value, and the final thing gets to the bitstream is "abs" and "minus2", but they are nothing to worry about, they are just describing this value)

Here is a reproduced encode and a "fixed" encode, please test.
https://workupload.com/file/d9D48eUYudr

(yes, we have to run that exact github action workflow to get a binary that can trigger the bug by the following command, the x265 parameters are translated from the FFmpeg style from the OP, it's really rare)

ffmpeg-test -ss 20:10 -to 21:00 -threads 1 -y -hide_banner -nostdin -bitexact -i "u.mkv" -map 0:v -profile:v main10 -vf "removegrain=1:1:1,format=yuv420p10le,scale=1280:720:flags=bicublin:param0=0.125:param1=0.4375" -sws_flags accurate_rnd -f yuv4mpegpipe -strict -2 -fps_mode vfr - | x265-ghaction --input - --y4m -D10 -o out.265 -pveryslow --crf 18.6 --psy-rd 0.7 --psy-rdoq 0.8 --aq-mode 3 --aq-strength 0.7 --deblock 1,1 --nr-intra 10 --nr-inter 25 --tu-intra-depth 4 --tu-inter-depth 4 --merange 58 --ref 6 --rc-lookahead 99 -b16 --rd 6 --rskip 2 --rskip-edge-threshold 2 --cbqpoffs 1 --crqpoffs 1 --scenecut 42 --scenecut-bias 5.2 --qcomp 0.59 --tskip --no-tskip-fast --limit-sao --selective-sao 2 --vbv-maxrate 8000 --vbv-bufsize 12000 --frame-threads 1 --pools none --no-wpp --level 40 --high-tier --colorprim 1 --colormatrix 1 --transfer 1 --range limited --chromaloc 0

[GeoffreyA]

That makes sense. If nothing is accessing the values apart from the decoder, then where to clamp or cast would be driven by convenience.

Would the x264 source provide guidance on how to handle this? Considering that the logic should be the same but different ranges.

[Z2697]

H.264 doesn't have temporal mv prediction, so probably no.

[GeoffreyA]

Here is a reproduced encode and a "fixed" encode, please test.
https://workupload.com/file/d9D48eUYudr

(yes, we have to run that exact github action workflow to get a binary that can trigger the bug by the following command, the x265 parameters are translated from the FFmpeg style from the OP, it's really rare)

Tested it, and "out-4.1-casting" shows no artefacts. "out-4.1" does.

Oh, boy, there's anime on everything, even running :)

[jpsdr]

I think the current method, casting the mvd to int16_t at entropy coding stage, is "as painless as it goes".

So you means, that in entropy.cpp just changing in Entropy::codeMvd:

const int hor = mvd.x;
const int ver = mvd.y;

by

const int16_t hor = (int16_t)mvd.x;
const int16_t ver = (int16_t)mvd.y;

and nothing else, is how you made your casting video ?
And it seems at least for GeoffreyA it made a difference.

[jpsdr]

Oh, boy, there's anime on everything

You have no idea !!

[GeoffreyA]

You have no idea !!

Though I'm fond of anime, there are branches I've never come across, so it's always a surprise. However, almost always, a world more delightful than our own, full of comical but endearing characters.

[Z2697]

I have a question, does this casting from int32 to int16 behave the same across all platforms that can run x265?
Maybe we should add some preprocessor directives if it doesn't.
Or something like mvd.x > 32767 ? mvd.x % 32768 - 32768 : mvd.x < -32768 ? mvd.x % 32768 + 32768 : mvd.x

[jpsdr]

If behavoir would change, it would be more compiler related to platform related.
But... for the few i know, the C definition is very stric and has a very good coverage, there is very few holes.
So there is a big change that this case is covered by the C definition, and the compiler must comply to it (otherwise it's considered a bug in the compiler), so whatever the compiler used, it will produce the same result, whatever the platform.
The real question should be : is this casting produce what we want to do ?
For the few people who have tested it, it seems it solved the issue without creating others, so for now we should consider the answer a "yes".

[Ritsuka]

As far as I know only float to int casts work differently, from Apple docs: https://developer.apple.com/documentation/apple-silicon/addressing-architectural-differences-in-your-macos-code#Audit-Code-that-Contains-Float-to-Int-Conversions