x265 junk location

[javlak]

Can I ask if anyone knows where the junk (video headers/footers/info sections) is located in raw x265 video files?

Thanks in advance!

Related to: x264 junk location (https://forum.doom9.org/showthread.php?t=170818)

[x265_Project]

Can I ask if anyone knows where the junk (video headers/footers/info sections) is located in raw x265 video files?

Thanks in advance!
Chapter 6 of the HEVC specification describes the syntax of the Bitstream and picture formats.

Supplemental enhancement information (SEI) is defined in Annex D of the HEVC specification. Video Usability Information (VUI) is defined in Annex E. Generally, SEI and VUI is not mandatory in the bitstream, but good encoders like x265 support SEI+VUI.

[javlak]

Thank you very much for the pointer. Although that was very close to the info I was looking for, let me be clearer as to what I want to achieve, as my interest is not to create an x265 decoder to be required to go that deep into HEVC specs (I think).

Say that I want to encode a video using x265. But I'm not too keen on leaving my PC working on 100% (or close) CPU usage for extended periods of time so I encode just the first 1000 frames of a video. Next day I encode the next 1000 frames and so on.

In theory, I could just concatenate all these files to one file with no problem, but I'd have some extra overhead due to the extra (unnecessary I guess) SEIs and VUIs. That's what I'm interested in; avoiding those "extra" recurring bits of info. They don't hurt, but not all of them are really necessary after the concatenation. They should be in the first video but I guess it would be more efficient to extract them from the rest of the video chunks for efficiency before the concatenation. Perhaps too much work for not that big of a difference in file size, but too many chunks per encoding session, and a few concatenated videos later it would show.

So I guess what I'm looking for is the location of the VUI and SEI, provided that I have (obviously) selected --no-repeat-headers, in a raw x265 file.

Thanks for your patience.

[nevcairiel]

raw x265 files don't have any headers, they only have a raw HEVC bitstream. If you want to remove SEIs, you need to parse the HEVC bitstream and remove the appropriate NALUs.
However, thats kinda extreme effort to save a couple bytes, and for a stream to be perfectly playable (especially seekable), you should have repeating headers in it anyway.

The exact same applies to x264.

[foxyshadis]

It's better to talk about H.264 and HEVC files in this case; there's nothing special about x265's. You really should just create a NAL parser, it's dirt simple and most languages already have libraries available to do it. (Older H.264 parsers will require minor tweaks.) Actually parsing NALUs is a much more complicated task, if you start from scratch, but it would be enough to simply find SEIs, SPSs, and PPSs, (and VPSs in HEVC), and verify that they're exactly byte-identical to the preceding equivalent NALU before throwing them away, or there's a strong chance your video won't play. The first byte of every NALU is the type.

In H.264, you'd want to look for 6, 7, or 8, after masking off the top three bits. Might have to drop End of Seq/End of Stream NALUs (10/11) if present as well.
In HEVC, you'd shift down by one bit, and then look for 32, 33, 34, 39, or 40. Again, End NALUs might be present and are 36/37.

I sure won't guarantee this will work right away, but if you really want to save every possible byte, that's the way to do it.

[LigH]

You can [Pause] a process running in a command line window. But it will only react on this key each time it outputs text.

You can also "suspend" a process with an advanced task manager like ProcessExplorer.

You may even be able to suspend the whole PC and be lucky that the process continues after waking it up.

But programming an encoder being able to continue an interrupted encode by storing the whole encoder state would be most elaborate.

[Daemon404]

But programming an encoder being able to continue an interrupted encode by storing the whole encoder state would be most elaborate.

Using libx265's API? Not at all. Just stop feeding the API frames after a captured key press.

Of course, the idea of building a small util to do it is pretty silly. Just set the process affinity to one core or do what you stated above.

[LigH]

The problem would be the resuming if the process quit. You should not shut down the PC after suspending the encode.

[Daemon404]

The problem would be the resuming if the process quit. You should not shut down the PC after suspending the encode.

I didn't see anyone mentioning shutting down. Only pausing.

[mandarinka]

Suspend to disk is however a safe thing to do.
You don't even need to use the pause button - suspending is transparent for the application (assuming there is no potentially unsafe GPGPU code, or that the suspend/resume don't disturb your source).