Multicore video encoding

[mustardman]

I have searched aplenty, on this forum and others, and found nothing - so please don't flame me (too bad) if this has been previously discussed!

I know there are issues around multithreading an encode as things have to happen in a very specific order for anything other than straight I-frames (eg: DV, MJPEG). However, a lot of CPUs now have multiple cores, and effectively, all but one remains idle for an encode task. It bugs me that video encoding is performed using just a single core. (Although each 'core' of the CPU varies its' load, the total always remains at about "1").

Now, perhaps others far cleverer than I have already thought of this, but what if an encode task is broken down into really large chunks, rather than the appraoch that people seen to be stuck at - encoding a stream with multiiple threads/cores (admittedly, an insolvable problem).

A lot of encoding tasks are done on an existing, and complete, file - not a stream (unless you are into live broadcasting, in which case encoding at more than real time is not necessary anyway). What if the file is broken into peieces? Say, for a quad-core CPU, the file is broken into four pieces. Each piece is encoded separately, and encoding the file becomes four times faster.

The rub? Where on earth would you cut the file so the GOPs align when the task is complete? What about the cuts are done completely arbitarily. Since every encoding 'engine' is working to the same algorithm, a scene cut that forces a new GOP occuring close to the start of 'block 2' can be matched with a GOP that occurs from an over-run of 'block 1' into 'block 2'.

IE: There will be some overlaps between each block, but the overlaps should be small, particulary if the file to be encoded is large.

Is something like this possible, or am I delusional?

Cheers,
MM.

[LoRd_MuldeR]

I know there are issues around multithreading an encode as things have to happen in a very specific order for anything other than straight I-frames (eg: DV, MJPEG). However, a lot of CPUs now have multiple cores, and effectively, all but one remains idle for an encode task. It bugs me that video encoding is performed using just a single core. (Although each 'core' of the CPU varies its' load, the total always remains at about "1").

That's not correct. Nowadays many (most?) video encoders support multi-threading just fine! Most notably x264 and x265 ;)

While there are dependencies between frames, this doesn't mean you cannot encode multiple frames in parallel. It even works with P- and B-Frames! If you encode multiple frames in parallel, each thread can only make reference to those parts of its reference frames that have been completed already, obviously. Also, the threads that process those reference frame should always be a few macro-block rows "ahead", so the thread that processes the current frame has enough reference data available to work with. But if things are synchronized properly, this all is not a problem. For example, x264 supports frame-based multi-threading (i.e. multi-threading by processing multiple frames in parallel) since around 2006. And it scales very well!

In addition to that, modern video compression standards, such as H.264/AVC, allow for breaking down a frame into multiple "slices". One of the reasons to use multiple slices (per frame) is that they can be processed in parallel. So even if you only process one frame at a time, parallelization definitely still is possible! Last but not least, H.265/HEVC has introduces a feature called "Wave Front Processing", which allows for processing multiple macro-block rows in parallel. Nice animation here (http://www.parabolaresearch.com/blog/2013-12-01-hevc-wavefront-animation.html)!

[mustardman]

Hmm, yeah. My bad.

I should have checked the age of my encoder! A later one certainly utilises much more of the available processing power - although interestingly still not "all she's got cap'n".

Your explanation helped a quite a bit, and the animation you directed me to is so very cool! The explanation on that page, although concise, does leave a bit of room for improvement.

I guess a lot has evolved over the last few years.

MM

[LoRd_MuldeR]

Your explanation helped a quite a bit, and the animation you directed me to is so very cool! The explanation on that page, although concise, does leave a bit of room for improvement.

Again: The basic idea about frame-based multi-threading is that you create one worker thread for each frame to be processed, so multiple frames can be processed in parallel. Or in other words: Multiple threads run in parallel and each of these threads processes one frame. The challenge here is that a thread (frame) might need to make reference to another frame which is not 100% complete yet – because the reference frame is currently being processed, in parallel, by its own thread. However that is not really an unsolvable problem! We just have to make sure that each thread (frame) makes reference only to those parts of the reference frame(s) that are completed already. x264 shows that this method works great :)

If you want to learn more about frame-based multi-threading in x264 and how it compares to slice-based multi-threading, see the document here:
http://git.videolan.org/?p=x264.git;a=blob_plain;f=doc/threads.txt

The new "Wave Front Processing" (that is what the above animation is about), introduced H.265/HEVC, is something different than frame-based multi-threading. It allows for multi-threading within one frame:
https://sites.google.com/site/hevcwppp/


I should have checked the age of my encoder! A later one certainly utilises much more of the available processing power - although interestingly still not "all she's got cap'n".

There are many possible reasons why you don't see "100% CPU utilization" while encoding. One of which is an input bottleneck!

For example, if your input is delivered by Avisynth and your (single-threaded) Avisynth script cannot deliver the frames fast enough to the encoder, then the encoder becomes "idle" (waiting for more input) very often...