x264 OpenCL

[Gser]

Yup crashes on win7 x64, 5870 as well. Catalyst 11.4

[Snowknight26]

You're 12 versions behind on your GPU drivers.

[duncanvdlinden]

Ok it working now :) :
Uninstalled 12.5 beta (including APP SDK 2.6)
Installed 12.4
installed App SDK 2.6

Testing now...

[Gser]

You're 12 versions behind on your GPU drivers.

Typo, just updated them today. I'm at 12.4.

[duncanvdlinden]

If you had app SDK 2.7 before, try to uninstall everything from AMD, the drivers and the SDK (it's all in one and the same uninstaller), if you have multiple GPU's disable the others. Then, install catalyst 12.3, and APP SDK 2.6. Then it works on my system.

After that i have enable my onboard GPU again (use it for Intel Quicksync), the OpenCL X264 still working.

I don't know if all the things I mention makes sense, but that's wat I did and work with an AMD HD6770.

[Snake91]

mmm will there be opencl boost in crf mode too?

[deekey777]

GPUGPU, may it be implemented via OpenCL or via CUDA or via something else, is not the "magic" speed booster that many people seems to think. Just throwing your CPU code on the GPU won't work. Even if you manage to port your CPU code and make it run on the GPU, it will be slow like hell. Getting calculations done one the GPU fast usually requires completely new ideas/algorithms - and that's exactly where all those so-called "GPU Encoders" sacrifice quality for speed to get some nice FPS numbers for the marketing department! And even if you mange to get your calculations done on the GPU fast - and with "fast" I mean significant faster than on a state-of-the-art the CPU - then it's still not guaranteed that you'll see an overall speed-up! Uploading and downloading the data to/from the "device" (graphics card memory) causes a lot of overhead. Only if you still save time including that overhead then you might be able to see some overall speed-up...

In other words: It's very easy to port your CPU-based software to the GPU and get something that runs a lot slower than the original. At the same time getting something that actually runs faster is very difficult and sometimes impossible!

(There are some calculations that are "sequential" by nature and therefore will never run fast on a massively parallel processor, such as a GPU)

So, it's time for PCIe 3.0?
http://www.hardwaresecrets.com/article/AMD-Radeon-HD-7970-Video-Card-Review/1458/13

[LoRd_MuldeR]

Well, PCIe 3.0 might be able to speed-up the upload/download delays quite a bit. Still, having to upload all data from CPU memory to GPU memory before you can access it in your GPU program ("kernel") and later downloading all the results again is orders of magnitude slower than accessing the data right away from CPU memory. PCIe 3.0 won't change this fundamentally, I think. Also the "Unified Addressing" now available in CUDA, which means you don't have to distinguish between device pointers and host pointers, may be a simplification for the programmer, but it doesn't change the fact that all data needs to be transferred over the Bus at some point in time. Furthermore, the "on board" (or "on CPU") GPU's may come at the advantage that they use the main memory as graphics memory, so ideally nothing needs to be copied between different memories. But on the other hand the CPU's main-memory interface may now become the bottleneck, if both, the CPU itself and its built-in GPU, heavily access the main memory. Last but not least all "on board" (or "on CPU") graphics chips are relatively weak GPU's. They have become better recently, yes. Still even a cheap dedicated Graphics Cards easily rules out the built-in CPU graphics chip. High-end cards are from another world. After all you won't be able to benefit from GPUPU, unless your GPU program ("kernel") runs sufficiently long to "hide" the upload/download delays. Some "Pipelining" might also help...

[sborho]

You need a fairly recent discrete GPU to get really obvious performance improvements, and even then, the best it can do is make the lookahead fast. If the lookahead was not the bottleneck, the encode will not go a lot faster (you only get the off-load benefit of doing the work on the GPU)

For AMD GPUs, you need the latest Catalyst drivers (at least 12.3) to get some required compiler fixes. There is a known issue that seems to affect many Radeon HD 6xxx GPUs that causes driver resets when the intra analysis kernel runs. AMD is investigating.

[Audionut]

and even then, the best it can do is make the lookahead fast.

There's probably a reason it's called OpenCL lookahead. ;)

[hajj_3]

i wonder if these intel docs just released will help us with utilising intel's chips to do hardware encoding: http://hardware.slashdot.org/story/12/06/23/1635238/intel-releases-ivy-bridge-programming-docs-under-cc-license

[CruNcher]

There's probably a reason it's called OpenCL lookahead. ;)

Yes though AMD advertises it like it's the holly grail in the press ;)

looks like the guys who implemented it are currently on a OpenCL run http://www.multicorewareinc.com/index.php?option=com_content&view=article&id=76&Itemid=86

Though im really surprised that AMD didn't acquired them yet seeing how much work they do for them though they seem to position in the middle and leave cuda and opencl left and right of them Dancing on both Parties ;)


So i did my own OpenCL lookahead benchmarks

460 GTX

No OpenCL 64 = 120W = 103 fps = 52 sec = SSIM Mean Y:0.9658581

OpenCL 64 = 175W = 125 fps = 42 sec = SSIM Mean Y:0.9655919 / PSNR Global 39.658

+ 55W = + 22 fps = - 10 sec (30% GPU Utilization) (130% Overall System Utilization)


X264 High Speed Quality (HSQ) 32bit = 125W = 190 fps = 28 sec = PSNR Global 39.534 / 40.097 (100% Overall System Utilization)

X264 High Speed Quality (HSQ) 64bit = 125W = 194 fps = 27 sec = PSNR Global 39.534 / 40.097 (100% Overall System Utilization)

Nvidia Cuda Encoder 32 bit = 180W = 193 fps = 27 sec = PSNR Global 39.649 = (110% Overall System Utilization)

[fields_g]

i wonder if these intel docs just released will help us with utilising intel's chips to do hardware encoding: http://hardware.slashdot.org/story/12/06/23/1635238/intel-releases-ivy-bridge-programming-docs-under-cc-license

That's why I just came back to this thread... wondering if the new docs were any use.

Direct Link: http://intellinuxgraphics.org/documentation.html

[hajj_3]

Does anyone know about any new progress on OpenCL x264 encoding support?

[Filker]

Does anyone know about any new progress on OpenCL x264 encoding support?

"A publicly available, OpenCL-enabled version of x264 should be out as early as next month. Luckily, we didn't have to wait—we secured a build of Handbrake that includes a pre-release, hardware-accelerated x264 encoder, and we've posted the results alongside our data from MediaEspresso and MediaConverter."

http://techreport.com/articles.x/23324

[Bloax]

It's a bit silly they used a very high motion shot for comparing reasons, unsurprisingly it looked the worst in x264, because it's actually smart enough not to allocate too many bits to a part you won't be able to really follow in detail.

[Keiyakusha]

It's a bit silly they used a very high motion shot for comparing reasons, unsurprisingly it looked the worst in x264, because it's actually smart enough not to allocate too many bits to a part you won't be able to really follow in detail.
BTW this x264 behavior really bugs me. Lets say I can spare a lot of bitrate to make everything look perfect. When most of the movie IS perfect - parts with high motion look too bad they even have blocks. Of course I can't see this is motion, only in artificial comparison. If I will rise bitrate to make fast motion look good too, other parts recieve overkill amount of bitrate so now look not only good but virtually lossless... If this is what CRF brings to us, maybe its time to go qp? ^__^

[Dark Shikari]

The problem is that x264 doesn't know what "perfect" is, since it doesn't have any code to do something on the order of a "just-noticeable differences" metric. It only knows varying levels of quality, not what the cutoff is for "you can't see any difference". And "you can't see any difference" is going to vary a lot depending on how you watch the video, too. If it did, it would be able to cap the quality in areas of the video that are lower-complexity, freeing up bits for the other areas.

[SassBot]

Of course I can't see this is motion, only in artificial comparison.

So then what exactly is the problem? Do you routinely watch movies by single stepping?

[Keiyakusha]

So then what exactly is the problem? Do you routinely watch movies by single stepping?
I don't watch anything I'm backuping. If I do some encoding - I watched this already and maybe even more than once. The goal is to make it look as close to the original as possible.
And here is my observation (not a complaint or something):
every time when I pick x264 settings in a way that scenes with fast motion will look like original (minus some loses in grain and stuff) - other more static parts, which are usually most of the show recieve overkill amount of bitrate.
If something will happen with original BD, I don't want to have backup with some parts of it in lower quality regardless of can I see it or not.
EDIT: of course i can use qpfile or something but this is too much work.

[Dark Shikari]

If you want to reduce that effect, raise qcomp. Watch out: it'll make the lower complexity areas worse, since the bits don't come from nowhere.

[turab]

I don't watch anything I'm backuping. If I do some encoding - I watched this already and maybe even more than once. The goal is to make it look as close to the original as possible.
And here is my observation (not a complaint or something):
every time when I pick x264 settings in a way that scenes with fast motion will look like original (minus some loses in grain and stuff) - other more static parts, which are usually most of the show recieve overkill amount of bitrate.
If something will happen with original BD, I don't want to have backup with some parts of it in lower quality regardless of can I see it or not.
EDIT: of course i can use qpfile or something but this is too much work.
If you want every frame to have the same quality regardless of motion, I believe --no-mbtree will work.

[Bloax]

If you want every frame to have the same quality regardless of motion, I believe --no-mbtree will work.
I think that would be --q-comp 1

[Andouille]

--q-comp 1turns completely off constant quality by turning on constant quantizer.
Who really wants this ??

[mandarinka]

It won't force constant quantizer, AQ will still be enabled IIRC.
I never tried that, I do encode with raised qcomp (default is 0.6) often, though. With mbtree, it helps more often than with pre-mbtree rate-control (IIRC).

[zerowalker]

Can i get the latest x264 build with Opencl x64 :)?
If it´s available.

[Filker]

benchmark of x264 opencl accelerated version on Trinity A10M vs i5.

https://reviews.handbrake.fr/r/328/

https://forum.handbrake.fr/viewtopic.php?f=11&t=24966&p=115091&hilit=opencl#p114793
" The executive summary is that an i5 CPU unaccelerated beats an accelerated amd trinity in nearly all categories. And the only workloads where accelerated trinity beats unaccelerated trinity by any significant margin is when downscaling is being done during the encode."

I hope the top of the line A10 Desktop gets better results.

[deadrats]

with regards to all the OCL benchmarks it seems to me that the testers are missing the point of having the lookahead function hw accelerated.

as a general rule of thumb setting the lookahead to higher values results in higher quality and/or stability as per:

http://mewiki.project357.com/wiki/X264_Settings#rc-lookahead

with normal software based lookahead setting this value to the max 250 slows down the encoding by quite a bit. since this is an easily parallelizable function and thus ideally suitable for gpu acceleration, an OCL enabled x264 build will allow end users to max out this setting, thus increasing the quality of the encode, without having to worry about the performance impact.

basically the proper way to test/see the benefits of OCL lookahead is to run 2 test encodes, where the exact same settings are used (including rc_lookahead=250) and compare the encode times (and resulting quality) with a pure software build of x264 and the OCL enabled build of x264.

i think we will find that the speed discrepancy between the 2 builds will be significant.

[Dark Shikari]

Setting lookahead higher than the defaults is generally useless for compression.

[deadrats]

Setting lookahead higher than the defaults is generally useless for compression.

what about quality?

[Asmodian]

In this context compression is synonymous with quality.

[deadrats]

this then begs the obvious question: if compressibility/quality isn't affected by higher lookahead values, then why bother coding them into the encoder in the first place?

this question applies to a whole bunch of x264 settings, the wiki is full of "this setting should be left at default" or "there's no need to change these settings", well why did the developers code the settings and include them as customizable parameters if they should be touched? just hard code the settings and be done with it.

[Dark Shikari]

rc-lookahead isn't even listed on the main help page. You don't have to touch it, and it's not out in the open to be messed with. Just because it's there doesn't mean you have to put your fingers in it without understanding it first (following that logic would quickly lead to a lack of fingers).

The lookahead is lowered by many faster presets in order to improve speed. We can't simply eliminate it as a parameter.

Not every option is applicable to every use case. It's simply not reasonable to remove all the options that aren't relevant to yours.

[deadrats]

side note, does subme 11 (full rd) not work? i wanted to test it out and set it within the latest video mastering works and xmedia recode but when i analyzed the results with mediainfo it said that subme=9 was used.

i know for subme=10, trellis=2 and aq-mode must be greater than 0 but does that also apply to subme 11?

[Dark Shikari]

Yes, all subme settings are a superset of lower subme settings.

[hajj_3]

@Dark Shikari, do you have an eta as to when OpenCL support will land in a stable build of x264?

[Dark Shikari]

I have no idea. A month, 6 months, never, depending on when people get their act together. Currently, the person responsible for the patch is working on it entirely in his free time, because the contract to write it apparently made no stipulations about actually getting it committed and finished.

[Filker]

http://astrataro.wordpress.com/

x264_rev2216+686_tMod-opencl.7z (test only)

"OpenCL version need --opencl to enable use of opencl lookahead. I did not test it, as it seems broken on my CCC 12.8. The last driver I successfully run it was CCC 12.3 ( or NVidia cards ), compiled with APP v2.6, but this version is compiled with APP v2.7."

Anyone tested it yet?

[sneaker_ger]

I can confirm that it crashes the GPU driver on Catalyst 12.8, HD 5850, Win7 x64, just like the older build...

[Atak_Snajpera]

the whole opencl is a pure madness for programmer. similar problems people have with flac cl . sometimes it works and sometime does not work. developer had to implement many workarounds for specific gpu cards.

[cyberbeing]

Anyone tested it yet?

Works fine on my NVIDIA GT440 DDR5 (864Mhz Core / 1728Mhz RAM), but opencl was ~13% slower at defaults settings on 720p, compared to CPU-only on my i5-3570K @4.4Ghz with near-identical PSNR & SSIM results. GPU load with opencl was 50-80% throughout encoding, but I guess the GT440 DDR5 is just too slow to see any benefit?

[Hiritsuki]

@cyberbeing
maybe it's need more faster gpu card.
I'll test it with GTX670 tomorow.

[Keiyakusha]

On nvidia 5xx generation there is zero benefits too, results vary from the same speed as core I7, to a bit (but just a bit, like 5% max) slower. I don't see it will be better than that, because it doesn't uses much gpu processing power... It just not very optimized or reached some design limits... However I don't know, with latest nvidia drivers opencl apps don't work for me at all :D

[06_taro]

opencl lookahead is implemented in opencl 1.1, and Catalyst 12.4 seems to have made some changes for opencl 1.1->1.2, not sure if it causes this failure or not.

[deadrats]

Works fine on my NVIDIA GT440 DDR5 (864Mhz Core / 1728Mhz RAM), but opencl was ~13% slower at defaults settings on 720p, compared to CPU-only on my i5-3570K @4.4Ghz with near-identical PSNR & SSIM results. GPU load with opencl was 50-80% throughout encoding, but I guess the GT440 DDR5 is just too slow to see any benefit?

really? you're comparing a gt440@864 mhz against a i5-3570@4.4 ghz and you're surprised that the software only version is faster? i'm surprised that the gpu version was only 13% slower.

i would expect a fair test using a better matched video card to be more than just a bit faster and i would expect to really see the difference if you used a lookahead value of 60 frames.

btw, how are you guys testing this build, from the CLI?

[the_weirdo]

really? you're comparing a gt440@864 mhz against a i5-3570@4.4 ghz and you're surprised that the software only version is faster? i'm surprised that the gpu version was only 13% slower.

Oh, that're you again! I'm not going to argue with you, actually I shouldn't, because I know how it'll be ended :sly: However, once again, you don't have any clues from what you say. GPU and CPU have different architectures, and so they also operate differently. That means you're comparing Orange with Apple (uh, I mean Apple the fruit). According to your logic, do you know GT 440 has 96 stream processors (CUDA cores) while i5-3570 has only 4 cores? And you expect we have to compare a 864 Mhz CPU against a 864 Mhz GPU?

I'll stop here and don't expect I'll reply your arguments in this thread again, 'cause like I've said, I shouldn't argue with you.

[Hiritsuki]

ok ... today i'd try it with my gtx480
it's around 6% faster than no gpu acceleration

my pc now is
980X@4.2Ghz
DDR3-1600@24GB@9-11-10-24 1T
N480GTX Lightning Default clock

that my test , the source is same.

my usual setting with opencl
http://i.imgur.com/vaLFk.png

my usual setting without opencl
http://i.imgur.com/RVxY4.png

set placebo with opencl
http://i.imgur.com/6kiDV.png

set placebo without opencl
http://i.imgur.com/R6m0z.png

[deadrats]

According to your logic, do you know GT 440 has 96 stream processors (CUDA cores) while i5-3570 has only 4 cores? And you expect we have to compare a 864 Mhz CPU against a 864 Mhz GPU?

you do know that the term "CUDA core" is primarily a marketing term and that GPU cores are simplistic ALU's whereas cpu cores are ALU's coupled to hybrid SIMD/FP units with either exclusive or inclusive cache hierarchies?

do you also know that Intel and AMD cpu's are primarily CISC architectures whereas gpu's are primarily RISC architectures.

my point was hat he was testing a lower end video card against a upper middle range cpu and harping on the fact that the gpu version was 13% slower.

@Hiritsuki
that's not bad, a 6% speed up for what is essentially "free" and you have an overclocked 6 core beast, a guy with a low end dual core will probably see a much bigger speed up and he doesn't have to spend any dough on new hardware. i see no reason why someone wouldn't use the ocl version as long as they have a decent video card.

[cyberbeing]

really? you're comparing a gt440@864 mhz against a i5-3570@4.4 ghz and you're surprised that the software only version is faster?
I would have expected lookhead offloaded to the GPU to result in the CPU being able to dedicate resources to other encoding tasks and speed up encoding. It would seem that there must be a GPU<->CPU communication penalty, which is greater with the GT440 DDR5 than the speed gain from GPU lookahead. This is after-all a low-end fermi GPU which has poor performance to begin with.

i would expect a fair test using a better matched video card to be more than just a bit faster and i would expect to really see the difference if you used a lookahead value of 60 frames.

I don't have a 'better matched' video card to test, but I ran a couple more CPU-only encodes on the same source with higher --rc-lookahead values:

CPU-only default settings with --rc-lookahead 60 was 11% faster than --opencl
CPU-only default settings with --rc-lookahead 250 was 2% faster than --opencl
CPU-only --preset veryslow --crf 16 was 16% faster than --opencl

Anybody have have a Kepler NVIDIA GPU + overclocked Ivy Bridge which they'd be willing to test?

[Hiritsuki]

that's my conclusion..
It's seem usage rate too low to make faster.