CUDA encoders for H.264, hows the quality?

[Chengbin]

I'm very excited about CUDA and it's ability to encode videos to H.264 very quickly. I'm wondering about quality. Do you think the quality would be worse, about the same or (preferably) better? I don't care about time(though that would be icing on the cake), but quality is absolute first. Is there any information about the quality of the conversion? Is it possible to be supported by GUIs such as AutoMKV (a newer version, edited to support CUDA, of course). Most importantly, does it support my 8800GT?

[Dark Shikari]

There is no publicly available CUDA-supporting encoder if I recall correctly; there's two GPU-based solutions that are being promoted (though aren't released yet, I think). RapiHD is an intra-only solution designed probably for high-speed high-quality high-bitrate recording of film footage/similar. Elemental is promoting their own one (which has shown up on Anandtech/similar), but I'm somewhat underwhelmed by their results, since speed-wise x264 can beat it and I highly doubt they're using high-quality settings when promoting how fast they can go ;)

GPU encoding has a lot of potential, but it has a lot of weaknesses too. Its a bit like programming for a Cell or an FPGA, except exponentially more of a nightmare.

[Inventive Software]

I smell a potential final-year project coming on in the next year or 2 if nothing viable turns up in that time. Would be good to use the PS3's hardware properly for encoding, and I can see GPU encoding taking off if the feature sets are right. :)

[tre31]

Toshiba already announced a laptop with a Cell DSP chip (similar too their previously announced FPGA/PCI-e solution - which I've heard nothing of since, would be good if they just bought out a PCI-e solution, people would snap it up like hot cakes) on it specifically for encoding/decoding tasks.

Announcements are all well and good, but its actual products that we want.... (yes patience is a virtue ...lol)

[lexor]

Would be good to use the PS3's hardware properly for encoding, and I can see GPU encoding taking off if the feature sets are right. :)

It would, but it won't happen to the full extent. If you don't shell out for a PS3 dev kit, you can't get at the GPU. It can only be done from PS3's firmware, Linux doesn't get access to the GPU and has to use software rendering mode to render graphics on Cell.

[Dark Shikari]

It would, but it won't happen to the full extent. If you don't shell out for a PS3 dev kit, you can't get at the GPU. It can only be done from PS3's firmware, Linux doesn't get access to the GPU and has to use software rendering mode to render graphics on Cell.And apparently the Cell is not very impressive either. From what I heard, an IBM team wrote an encoder from the ground up for the Cell and optimized it the best they could, but still needed 2.5 out of 8 cores to do realtime 1080p on extremely fast and crappy settings.

x264 can do realtime 1080p24 on a single 3Ghz core of a Core 2 on its fastest and crappiest settings, so those numbers aren't really impressive.

[audyovydeo]

From what I heard, an IBM team wrote an encoder from the ground up for the Cell and optimized it the best they could, but still needed 2.5 out of 8 cores to do realtime 1080p on extremely fast and crappy settings.

Are you talking about this ?

"White Paper
In this paper we present an implementation of an H.264 video encoding algorithm on a Cell Broadband Engine (CBE), for the application of high-quality video surveillance. The proposed system aims to encode three channels of a standarddefinition (720 × 480) video stream at 30 frames per second with a target bit-rate of 2 Mbps. The presented encoder is compliant with the main-profile of the H.264 standard, and uses a learning-theoretic mode selection algorithm as an alternative to brute-force rate-distortion optimized mode selection, enabling significantly reduced computational complexity. The CBE offers an aggregate of 204.8 GFlops of computing power, at 3.2 GHz, in 8 Synergistic Processor Elements (SPEs), each with 128-bit wide vector processing capability. The SPEs are under the control of a central Power Processor Element (PPE) which has its own 128-bit vector processing unit and all units are connected by an on-chip broadband bus with 25.6 GB/s bandwidth capacity and an I/O bus providing 50 GB/s. This combination of processing units and high-speed internal buses is ideally suited for the target application of multi-channel real-time H.264 video encoding. The proposed system employed only the standard tools provided with the CBE toolkit, without resorting to customized assembly level programming."

Revision Date: 31/10/05

source : http://www-01.ibm.com/chips/techlib/techlib.nsf/techdocs/B5A1F97CAB641365872570AB00594459


or this : http://www-03.ibm.com/technology/cell/pdf/DCD_solution_brief1.6.pdf ???


cheers
audyovydeo

[Dark Shikari]

Are you talking about this ?

...

standarddefinition (720 × 480) video stream at 30 frames per second with a target bit-rate of 2 Mbps.No, I'm not talking about that. I said 1080p, not SD :p

[akupenguin]

uses a learning-theoretic mode selection algorithm as an alternative to brute-force rate-distortion optimized mode selection, enabling significantly reduced computational complexity.
Translation: "We threw out all the structure of the problem and gave it to a black-box optimizer, and then we had to compare it against brute-force because it didn't fare well against anyone else's ad-hoc heuristics." ;)

[audyovydeo]

No, I'm not talking about that. I said 1080p, not SD :p

!! The linked PDF actually dated 1998 !!!

cheers
audyovydeo

[lexor]

Translation: "We threw out all the structure of the problem and gave it to a black-box optimizer, and then we had to compare it against brute-force because it didn't fare well against anyone else's ad-hoc heuristics."

I think you are being unfair. And this isn't the first time you've made that argument. How does x264 with all it's great (on x86) algorithms run on Cell? Like shit. That's because hand written asm goes out of the window, and gcc fails oh-so-hard at generating code for Cell (more so than for anything else). You can't expect IBM, or any other research team for that matter, to start implementing heuristics to compare. They just implemented the easy brute-force method so that when others implement their algorithms on Cell they can compare to it as well and know how their heuristic is doing compared to IBM's.

Similar point goes to DS, I doubt very much they've created a full featured encoder. They did it in terms (and time frame) of research project, now comapre that to how long x264 took to get to where it is, and it is still being improved. I think criticizing IBM's effort here for being weak is like criticizing midgets for being short.

[Gabriel_Bouvigne]

From what I heard, an IBM team wrote an encoder from the ground up for the Cell and optimized it the best they could
I think that it was in fact written by Vanguard

[Inventive Software]

You know what? GPUs these days are pushing the Teraflop barrier, and we're struggling to use them to our advantage. Not so long ago, 100 billion operations per second (100 G-ops) was really something to shout about... http://www.youtube.com/watch?v=ldiYYJNnQUk

[708145]

Hi folks,

[I]
...
The proposed system employed only the standard tools provided with the CBE toolkit, without resorting to customized assembly level programming."


Well the CBE toolkit includes assembly optimized functions in a library. What they did not do is use additional assembly optimization.

And as already pointed out the algorithms and heuristics have to be tuned to/for the CELL architecture. These heuristics will be very different from x86 optimizations!
The main problem is that the people optimizing and tuning for CELL are by no means video experts. But nonetheless there is work going on towards good encoders on CELL.

A side note:
Despite the number 1998 in the filename (which is a counter not the year) the PDF is from 2005.
[Jagmohan 05] A. Jagmohan, B. Paulovicks, V. Sheinin, H.
Yeo, “H.264 Video Encoding Algorithm on Cell Processor,”
GSPx 2005.

bis besser,
T0B1A5

[7oby]

The proposed system employed only the standard tools provided with the CBE toolkit, without resorting to customized assembly level programming."

I'd also like to add that jumping to assembly isn't the first optimization you want to do. Probably posted elsewhere a couple of times, but I think this

http://www.hotchips.org/archives/hc17/2_Mon/HC17.S1/HC17.S1T4.pdf

is a somewhat impressing demonstration of the profiling tools for Cell. It shows at the example of a real H.264 encoder how to balance all the constraints like DMA latency, number of registers, branch misses ... and using pipeplining to load the SPEs with sufficient work.

Cell development environment looks very mature compared to CUDA. Cell should be less painful. A Playstation3 could be a very powerful BluRay transcoder, besides being not exactly legal.

[MfA]

Cell is different than a GPU ... a lot different. Good programmers can get good utilization out of it for nearly any type of algorithm (including branch heavy stuff).

[CruNcher]

Hehe the Elemental Guys trying themselves in Advertising ;)
http://www.badaboomit.com/

also Cyberlinks PowerDirector is gonna support ATIs 4x00 series AVT http://www.cyberlink.com/eng/press_room/view_1756.html

I wonder on which H.264 SDK it's based i remember their old H.264 encoder was based on Moonlights very old one and later i think the Mainconcept SDK has been used, it wouldn't be bad if the Cyberlink Engineers added GPU support to that now or maybe it comes from Elecard/Mainconcept directly, tough they should first get Decoding on the GPU work as it was the case in the last Elecard Beta Decoders (before they joined Mainconcept it worked) instead of concentrating so much on GPU Encoding now (just my 2 cents) ;)

And last but not least ATI updated their AViVo Transcoder Software Core Recently (i guess it's now on the level that AMDs Encoder showed in the MSU test, tough maybe it's even a little better now) but it's still impossible to set any tools everything is still preseted :)

It's hard to judge Elementals Quality and Watt efficiency in Encoding on the GPU compared to CPU by all the Reviews that are out currently and done with the 2 min restricted Alpha Press Demo. As long as it doesn't hit Doom9 or it's Community Members (which is really surprising that he didn't get a Press Copy to test, i would guess Elemental fears the Doom9 judgment when comparing against the X264 optimizations and a good balanced set of Device Profiles for it hehe).
Also no Encoded stream has been released yet to analyze the tools that get used for the different Device Profiles Elemental decided to use.
I think full judgment and in depth comparison vs X264 can only be conducted as soon as the Premiere Plugin (with the whole setting featureset) gets released.

@Inventive Software
This Voxel Space Rendering is pretty well usage off the new Teraflop Power if you ask me
http://farm4.static.flickr.com/3148/2585016023_341219cf32_b.jpg
http://farm4.static.flickr.com/3085/2585849022_2dc42296fe_b.jpg
think about in the future not Broadcasting Video but High Compressed 3D assets :)

Heres some AMD Press Material about AVT and all the new Scaling stuff (most probably now lanczosresize via the driver (shader) directly) (Careful this is Business representation stuff no real technical information here it gives only clues and exaggerating of facts is common in such marketing Material) (hmm never saw any information about ATIs Premiere Encoding Plugin interesting must have been a test for API implementation for 3rdparty Developers)

http://static.pcinpact.com/pdf/docs/06DaveBaumann-HDMedia-Final.pdf

[metaxaos]

So why just not to switch x264 to use CUDA in next version? Community doesn't have NV cards?
It is really strange to have a method to generally boost up perf in times but not to use it.
I am even agree to pay for that improved x264.

[Dark Shikari]

So why just not to switch x264 to use CUDA in next version? Community doesn't have NV cards?Want to rewrite the entire encoder completely using the most nightmarish API I've ever seen in my life? Patches are welcome.I am even agree to pay for that improved x264.Given my experience so far in trying to port the motion search to CUDA, and Avail's hiring of a contractor to attempt to do so, I'd put the quote for porting the whole encoder somewhere on the level of a few million dollars... if you can even find people willing and able to do it.

[metaxaos]

But what exactly is so bad that you can call it "nightmarish"? I'm not a specialist but AFAIR CUDA is almost standart C plus compiler. Is it really so way-more ass-pain programming for GPUs than for multy-CPUs?

[Dark Shikari]

But what exactly is so bad that you can call it "nightmarish"? I'm not a specialist but AFAIR CUDA is almost standart C plus compiler. Is it really so way-more ass-pain programming for GPUs than for multy-CPUs?CUDA being "just like standard C" is true, but only syntactically--the threading model is what makes it very difficult to use.

[Oxygen]

From what i remember, there is at least one CUDA encoder, Badaboom.

From the quick feedback from Damien Triolet from hardware.fr, there is at least one big limitation, it can't do a lower than 2Mbits encode, and it would be especially made for 'not too large resolutions' aka 640x365 for example for the iphone, generating a 2.4Mbits encoding at default quality (while any *good* encoder would be able to sustain a 700kbits for that resolution).

Consequence, a GTX280 is on pair with a QX9700 for the same resolution (but 2Mbits for Badaboom, and 700kbits for the cpu-based encoder) while giving similair results

Badaboom (GTX 280) : 25s
Videora (QX9770) : 28s

With 2mbps vs 768 kbps as for the bitrate and a 480x320 resolution used.

[Dark Shikari]

From what i remember, there is at least one CUDA encoder, Badaboom.Isn't Badaboom Baseline-only (and maybe even intra-only if I remember correctly), thus making it entirely useless?

[Oxygen]

Isn't Badaboom Baseline-only (and maybe even intra-only if I remember correctly), thus making it entirely useless?

Probably, i haven't search more on that matter, and i'm just the messanger as they say ;)

But the feedback seems to prove it is indeed useless...

[Gabriel_Bouvigne]

Isn't Badaboom Baseline-only (and maybe even intra-only if I remember correctly), thus making it entirely useless?
If I remember well, it's high profile...BUT...
...
...no cabac.

So it's able to use 8x8 transforms, but is constrained to use cavlc (for the "pro" version).

[Revgen]

According to NVidia, the new GeForce 200's allow CUDA much more flexibility when it comes to FPU intensive apps like encoders.

[akupenguin]

FPU intensive apps like encoders.
When was the last time you saw a video codec that used floating-point?

[Revgen]

When was the last time you saw a video codec that used floating-point?

That's why I wrote "According to Nvidia". :)

[zzzpt]

I'm waiting for ATI- GPU encoding. This will be a great breakthrough.
It will reduce the encoding time to 1x real time.

CyberLink PowerDVD and PowerDirector Support ATI Radeon™ HD 4800 Graphics Series from AMD,
Delivering Optimized HD Playback and Accelerated Video Transcoding

Expecting an answer from Cyberlink forum:
http://forum.cyberlink.com/forum/posts/list/3307.page


Cant wait :D

[LoRd_MuldeR]

It will reduce the encoding time to 1x real time.

For what resolution, what framerate, what bitrate and what quality ???????????

Unless it produces quality equal to x264 (at same average bitrate), the speed says nothing - absolutely nothing...

[Dark Shikari]

1x realtime? So if you pair up half a dozen of them you can compete with a single Core 2 CPU?

[zzzpt]

For what resolution, what framerate, what bitrate and what quality ???????????

Unless it produces quality equal to x264 (at same average bitrate) the speed says nothing, absolutely nothing...

Well I'll wait and see too.
But comparing with the decoding, i don't see why the encoding shouldn't be the same.

The new graphic cards are very powerful, we just have to take advantage of them :p

[BlackSharkfr]

Cyberlink doesn't provide GPU encoding. They only use the GPU for decoding.
It's made to accelerate Blu-Ray to xxx (ipod?) transcoding.
It's all in the press release just 3 posts above yours. You have to read between lines, and not invent one you'd love to see in there. (it happens to me too sometimes)

[Comatose]

I don't think we can expect to see a quality and affordable GPU based encoder at least for the next few years, considering what Dark Shikari said about the difficulty (and subsequently, the cost) of implementing it.

[Caroliano]

But comparing with the decoding, i don't see why the encoding shouldn't be the same.
AFAIK the GPU decoding is not implemented in general purpose shadders, but in specialized units, like UVD. And an decoder is many times simplier than an (good quality) encoder.

[ajp_anton]

Would it be possible to make certain parts of x264 GPU compatible?

[Sagekilla]

It's possible, but it's very difficult to code functions that run fast in the C-like language that nvidia uses.

I remember looking on the nvidia CUDA boards and seeing Dark Shikari have issues with getting SAD to run fast enough. IIRC, the CPU version -still- ran faster.

[Sharktooth]

AFAIK the GPU decoding is not implemented in general purpose shadders, but in specialized units, like UVD. And an decoder is many times simplier than an (good quality) encoder.
encoding is a different thing.

[7oby]

Given my experience so far in trying to port the motion search to CUDA, and Avail's hiring of a contractor to attempt to do so, I'd put the quote for porting the whole encoder somewhere on the level of a few million dollars...

You tried back in December '07 (http://forums.nvidia.com/lofiversion/index.php?t53172.html) to port some ME functionality to CUDA. Trying to do something like that:

http://img93.imageshack.us/img93/5816/h264withgpudc5.png

Straight forward ESA is simpler to implement compared to SEA. But even in the case that ESA on GPU will perform much worse than the x264 SEA CPU algorithm, the overall performance should be still faster with a GPU coprocessor. ME makes up quiete a reasonable slice of the overall computation time and GPU-ME will free some CPU cycles. It should be possible to compensate any latency involved in CPU <-> GPU communication by some small fixed size buffers.

Besides CUDA having an inferior API and besides some serious constraints regarding wraps and threads in CUDA itself: What are the real challenges here? Nobody wants to rewrite the entire x264 encoder. Or is the approach sketched above impractical due to too high dependencies between ME and other components, which result in a miserable utilization on all collaborating units? And all efforts to get a better utilization (raise no. of CPU threads to raise the number of non blocked threads, introduction of some buffers here and there) will have a negative impact on the maintainability of the CPU-only x264 encoder?

Would really like to hear something about that. I've been asking something similar here (http://forum.doom9.org/showthread.php?p=1151031#post1151031), but didn't get an answer. Maybe your experiences have been posted elsewhere, then I'd appreciate a link.

[Dark Shikari]

Straight forward ESA is simpler to implement compared to SEA. But even in the case that ESA on GPU will perform much worse than the x264 SEA CPU algorithm, the overall performance should be still faster with a GPU coprocessor. ME makes up quiete a reasonable slice of the overall computation time and GPU-ME will free some CPU cycles. It should be possible to compensate any latency involved in CPU <-> GPU communication by some small fixed size buffers.

Besides CUDA having an inferior API and besides some serious constraints regarding wraps and threads in CUDA itself: What are the real challenges here? Nobody wants to rewrite the entire x264 encoder. Or is the approach sketched above impractical due to too high dependencies between ME and other components, which result in a miserable utilization on all collaborating units? And all efforts to get a better utilization (raise no. of CPU threads to raise the number of non blocked threads, introduction of some buffers here and there) will have a negative impact on the maintainability of the CPU-only x264 encoder?

Would really like to hear something about that. I've been asking something similar here (http://forum.doom9.org/showthread.php?p=1151031#post1151031), but didn't get an answer. Maybe your experiences have been posted elsewhere, then I'd appreciate a link.The problem is that the latency is high enough that you must do one of three things:

1. Move all analysis to the GPU (yeah, right).
2. Separate a part of the analysis to make it frame-based instead of block-based. Move this to the GPU.
3. Reduce quality by moving something to the GPU that normally requires information from neighboring blocks, and take a shortcut to avoid requiring that information.

The problem is that every single macroblock needs information from its previously encoded neighbors. You can avoid this, but at a quality penalty. One example is intra analysis, where you can do the analysis using source pixels instead of reconstructed, but it loses a significant amount of quality.

And with the motion search, its probably not worth moving it to the graphics card unless you're doing exhaustive... in which case the CPU seems to be faster anyways.

[Sharktooth]

i would wait for a common API. i mean, microsoft is working at it in DX11 but im sure there will be some open common APIs in the future, so to code generic functions that can work on evey cards that comes with shaders.
also larrabee would definatly be easier to program due to its x86 code, "1" may be even worth a try coz it comes in the form of a multi-core co-processor...

[7oby]

I know you're very ambitious and you consider moving major analysis parts to the GPU.

I was really only thinking of doing motion search on the GPU first and if at all move on from there.

And with the motion search, its probably not worth moving it to the graphics card unless you're doing exhaustive... in which case the CPU seems to be faster anyways.

The model I had in mind was the following. The numbers are completely made up and may be far from reality:

If I assume 20% of cpu cycles to SEA motion search (maybe too much) the picture looks like this:
http://img230.imageshack.us/img230/6775/h264cpuonlyvl9.png

If I further assume:
. CPU with SEA is 4 times faster than GPU with ESA
. communication overhead is 10% on CPU side (and maybe also some on GPU side)
then the picture looks like this:
http://img388.imageshack.us/img388/5094/h264cpugpucombinedni9.png

The CPU + GPU version is 13,6% faster (100% : 88% = 1,136).

Encoding efficiency in terms of watt / fps may be a complete disaster, however I consider 13,6% performance increase given the not too difficult extension already remarkable compared to other achievable performance gains.

You surely have an opinion on that - given the time you spent with x264. Maybe the 20% are too much, but also 10% SEA and a smaller amount of communication overhead would result in some performance gain.

[Dark Shikari]

But you have another problem with separating the motion search onto the GPU; in x264, qpel-RD (subme7) is done at the end of encoding every inter block. This means that since future blocks' motion vectors are based on past blocks' motion vectors due to the bit cost of MVs, the only way to get truly 100% accurate motion searching is to do the motion search on a per-macroblock level knowing the exact MV used by the previous blocks.

Thus, even an exhaustive search from the GPU would have to be refined on the CPU afterwards for maximum effectiveness.

[7oby]

i would wait for a common API.

Maybe you're right. However there will be CUDA, Microsoft Shader Model 5.0 in DX11, OpenCL, BrookGPU.

I'm oversimplifying when saying once you don't optimize too much on the GPU side, you might get away with a rather straight forward refactoring.

You're definitely right in saying it's less work to port anything to Larrabee. But when will it be available? 2009/2010. At what price range? Meanwhile we'll get Nehalem for free:

http://images.anandtech.com/graphs/nehalempreview_060508030043/17017.png
http://www.anandtech.com/cpuchipsets/intel/showdoc.aspx?i=3326&p=6

[Sharktooth]

well, you're right too.
however, i just said that since ppl is speaking of CUDA all the way, but CUDA is a proprietary solution.
an open standard (like openCL) is much better coz would ensure things will run on any platform and any hardware.
i dont doubt about nehalem processing power but since there is the tendency to have really fast co-processors (look at the latest AMD videocards... more than 1TFLOP of computational power, intel's larrabee, etc) why not try to exploit them?

[Mangix]

AMD Fusion is the exploitation

[MfA]

But you have another problem with separating the motion search onto the GPU; in x264, qpel-RD (subme7) is done at the end of encoding every inter block. This means that since future blocks' motion vectors are based on past blocks' motion vectors due to the bit cost of MVs, the only way to get truly 100% accurate motion searching is to do the motion search on a per-macroblock level knowing the exact MV used by the previous blocks.
Finding the local minimum purely based on the preceding MV ain't all that either. The true (but still local) minimum depends on all the MVs in the GOP, they are all independent. It's a trellis search problem as usual (albeit in this case the trellis search is simply not tracteable).

How much time does a really good pre-pass ME with long search range take? It doesn't really need to be RD optimized, as long as it gives true motion it should be good enough. Pre-pass ME seems a more realistic use of the GPU.

[Dark Shikari]

Finding the local minimum purely based on the preceding MV ain't all that either. The true (but still local) minimum depends on all the MVs in the GOP, they are all independent. It's a trellis search problem as usual (albeit in this case the trellis search is simply not tracteable).This is true; the ideal way to do it is an iterative search aka Snow.

How much time does a really good pre-pass ME with long search range take? It doesn't really need to be RD optimized, as long as it gives true motion it should be good enough. Pre-pass ME seems a more realistic use of the GPU.The point is that the CPU is much more efficient at an exhaustive search than a GPU--and from what I've found, faster search algorithms tend to be quite difficult to implement using CUDA.