Plea- transcode with SandyBridge

[emueyes]

Hi all

This is such a general question that I'm embarrassed to ask, more that the fact that I can't just figure it out myself

I want to take large - say 25-30G - mkv files, and slim them down to around the 15G mark. Put another was, I can't tell the difference between a 35Mbps stream and a 15Mbps one. Except by looking at the filesize.

I'd like to be able to use the vaunted graphics on my i3 2100 to help, and have tried and tried; Media Espresso just crashes. Media Encoder can't find the encoder part of the cpu (that is odd, and I hope just an anomaly). Whatever I've used, I can't find a decent transcoder. People are happy to list packages that support HD graphics, but which won't necessarily work.

Is there a package that will use Intel HD2000 gfx hardware on h264 video in m2ts and mkv containers ?

I know there are a great many good packages out there, mkvtools, tsmuxer, I just can't find the right one, or the right method, and every time I come back to this problem I go away unhappy.

Please, any help welcome
Rob

[aegisofrime]

Starting with the simple possibilities, have you installed drivers for the HD2000? Also, are you using a discrete graphics card?

[hello_hello]

Isn't using a GPU for encoding generally inferior in quality to x264/CPU encoding?

[jaju123]

You need a z68 motherboard or h67 to use quick sync. I think only z68 supports it with a discrete gpu.

Sent from my Nexus S using Tapatalk 2

[TheFluff]

gpu accelerated encoding is still mostly a gimmick. use x264 like everyone else, it's not like you have a slow cpu

[emueyes]

I'm sorry I wasn't more specific in my original post, it was a silhouette of a question with the lights out. I have an Intel DH67GD motherboard (H67 chipset) with an i3 2100 CPU, am using the onboard HD2000 graphics and whatever Intel drivers were current when I built it last November. It has 8G of generic GSkill RAM and an Asus Xonar soundcard.

Intel's specification of the CPU are here http://ark.intel.com/products/53422/

The differences between the h67 and p67 were in clock adjustments; I can't overclock my h67 and had no desire to.

The encode/decode parts of the chip do work, and they work very quickly. Media Espresso flies along, till it hits the glass building.

I'd have to differ on the future of GPU processing, albeit without all the facts at hand, since I've never been able to transcode anything using it. However, a if Nvidia have been promoting CUDA so succesfully for years, and Intel think it important enough to play catchup then I presume it something to be worth investigating.

I have to wonder about the smoke and mirrors though. NVidia's own Badaboom was spectacularly unstable. I don't know. It might just be hard to write code for GPUs.

[nm]

I'd have to differ on the future of GPU processing, albeit without all the facts at hand, since I've never been able to transcode anything using it. However, a if Nvidia have been promoting CUDA so succesfully for years, and Intel think it important enough to play catchup then I presume it something to be worth investigating.

Nvidia hasn't really been promoting GPU encoding until the latest Kepler GPUs were launched. Those cards include a hardware encoder that is comparable to Intel's Sandy Bridge Quick Sync. I'd say that it's Nvidia who is playing catchup here.

The Badaboom CUDA encoder from Elemental Technologies got some (undeserved) attention and it made GPU encoding a hot topic. But it still can't match good CPU encoders at quality, speed and energy efficiency.

It might just be hard to write code for GPUs.

It's hard to write an efficient GPGPU H.264 encoder for current GPU architectures. But Quick Sync and other hardware solutions are different since the hardware is specifically designed for the task.

Still, Quick Sync only gives about the same quality as x264 --preset superfast, which is usually bottlenecked by slow input (software decoding and filtering). Since you are trying to save space by re-encoding, Quick Sync isn't for you. x264 with default settings should be pretty fast and the results will be much better. I'd guesstimate 20 % to 40 % file size savings over Quick Sync at same quality.

Actually you might not save any space at all with Quick Sync if you aim for visual transparency compared to the original Blu-ray.

[emueyes]

Nvidia hasn't really been promoting GPU encoding until the latest Kepler GPUs were launched. Those cards include a hardware encoder that is comparable to Intel's Sandy Bridge Quick Sync. I'd say that it's Nvidia who is playing catchup here.


I'd have to disagree; it's not relevant to this discussion, but Tesla, and now Kepler GPUs have been used in supercomputers for a long time, check supercomputers.org for example.

CUDA has been use in personal software too, and is fast, reliable and gives good results. Pretty much all PC video software will use it, from players to editors.



Still, Quick Sync only gives about the same quality as x264 --preset superfast, which is usually bottlenecked by slow input (software decoding and filtering). Since you are trying to save space by re-encoding, Quick Sync isn't for you. x264 with default settings should be pretty fast and the results will be much better. I'd guesstimate 20 % to 40 % file size savings over Quick Sync at same quality.



This all sounds a bit vague to me. Saying that "Quick Sync only gives about the same quality as x264 --preset superfast" doesn't make much sense. It is the software that runs on QuickSync hardware that determines the quality of the output, not the hardware on which it runs. In particular, the hardware is designed to both encode and decode. But there's a dearth of software for it.

My original post was on just that topic - what can I get to use QuickSync properly?

I think there may be a little wizardry in x264, it does remind me of my early Unix days (yeah, I used to be a programmer turned sysadmin. Again, not relevant). Or perhaps, quite the point. On one hand, I'm sick of having to learn the dozens of different switches to programs, while at the same time realising that h264 encoding is enourmously complex and don't necessarily want a program with one button on it.

I think people have realised that: some things like MakeMKV and BDClown cover up the complexity nicely. Yes, I know they do different things, but both could/would benefit from hardware acceleration I think.

As far as quality is concerned, I suppose I'm getting older, but I can't tell the difference between a 35Mbps stream and a 20Mbps encode of the same content.

The software manufacturers are making it hard for themselves; I tried a trial version of Arcsoft Media last night. It takes a bit over an hour to transcode an mkv from MakeMKV to an mp4 with a 15Mbps bitrate, which looks fine on a 42" TV, but it only encodes a crappy 128kbps stereo soundtrack.

[vivan]

CUDA has been use in personal software too, and is fast, reliable and gives good results. Pretty much all PC video software will use it, from players to editors.What software? Which real (and good) results? Adobe mercury playback engine is useless.
All CUDA encoders are so awful not because of poor software, but because of GPU's architecture.

This all sounds a bit vague to me. Saying that "Quick Sync only gives about the same quality as x264 --preset superfast" doesn't make much sense. It is the software that runs on QuickSync hardware that determines the quality of the output, not the hardware on which it runs. In particular, the hardware is designed to both encode and decode. But there's a dearth of software for it.Blaming software doesn't make any sense, since QuickSync is ASIC.

[emueyes]

After a quick survey, it seems that software Bluray for PCs will all use CUDA, the AMD equivalent and some the Intel equivalent. I don't know if that makes them 'good' enough, but I doubt companies would sell software that is purposely inferior to a competitor's.

The compute cores from all the manufacturers are not just ASIC, they are designed to be GPGPU devices because certain types of algorithms run very well on that hardware, originally designed for graphics use.

Come to think of it, SSE and AVX silicon could be regarded as ASIC black boxes, too.

I'm not quite sure how me asking about software for Intel HD graphics has me now defending CUDA; the whole point seems to have been lost, and that `wizardly` cabal similar to the Unix gurus' appears to have risen. I can't hear the difference between a $1000 piece of speaker wire and a $2 one, but some can, and enjoy hours talking about it.

[Atak_Snajpera]

x264 thesedays is very easy to use due to included decoders and presets.

x264.exe input.mkv --crf 20 output.264

still too complicated?

[nm]

I'd have to disagree; it's not relevant to this discussion, but Tesla, and now Kepler GPUs have been used in supercomputers for a long time, check supercomputers.org for example.

CUDA has been use in personal software too, and is fast, reliable and gives good results. Pretty much all PC video software will use it, from players to editors.

[...]

After a quick survey, it seems that software Bluray for PCs will all use CUDA, the AMD equivalent and some the Intel equivalent. I don't know if that makes them 'good' enough, but I doubt companies would sell software that is purposely inferior to a competitor's.

CUDA and other GPGPU frameworks can be used to speed up many things, including video filtering in players and editors. But in this thread we are only talking about video encoding and so far all the GPGPU video encoders have been useless.

However, Quick Sync and Kepler's encoder are a different case. They can actually be useful for really fast but low quality encoding. According to MSU's seventh H.264 codec comparison (http://compression.ru/video/codec_comparison/h264_2011/) (see the PDF (http://compression.ru/video/codec_comparison/h264_2011/mpeg-4_avc_h264_video_codecs_comparison.pdf), page 99), Intel's best settings for their Quick Sync -based encoder could give about the same quality as x264 with very fast and low quality presets (--preset superfast and --preset veryfast). On the Core i7 2600 CPU they tested on, x264's --preset superfast was also as fast as Quick Sync. As a home assignment, take a look at those figures and see how well CUDA encoders did in that test.

MSU doesn't tell which Quick Sync encoder they used, but they worked with Intel engineers to come up with good settings. Besides, all available implementations use Intel's SDK, so they are basically the same thing decorated with different user interfaces.


The compute cores from all the manufacturers are not just ASIC, they are designed to be GPGPU devices because certain types of algorithms run very well on that hardware, originally designed for graphics use.

We weren't talking about GPGPU here but the hardware encoder part of Quick Sync, which is quite different beast and has nothing to do with GPU architectures. Rumors tell that it's partly programmable, but since it's completely controlled by Intel, it could as well be fixed in silicon.



As far as quality is concerned, I suppose I'm getting older, but I can't tell the difference between a 35Mbps stream and a 20Mbps encode of the same content.

And was that 20 Mbps video encoded with Quick Sync? I think not.

[CruNcher]

There is also the Special situation off High Bitrate and Cabac ;)

We weren't talking about GPGPU here but the hardware encoder part of Quick Sync, which is quite different beast and has nothing to do with GPU architectures.

In terms of Quicksync it's not correct it's a hybrid kind of thing (but yeah CPU Utilization is generally on a very low level around 10% @ max) dunno about Nvidias new one but i guess it works similiar very Interesting AMDs becomes though that is the least tested yet also because practically 0 implementation exist that uses the new 7th Generation features ;)

The new results are expected to come in May im pretty sure we gonna see AMDs 7th Generation Reference results then also.

[hello_hello]

This all sounds a bit vague to me. Saying that "Quick Sync only gives about the same quality as x264 --preset superfast" doesn't make much sense. It is the software that runs on QuickSync hardware that determines the quality of the output, not the hardware on which it runs.

I think that's the point. Using encoder "A", "B" and "C" won't all give you the same quality at the same bitrate. Currently the x264 encoder is the one by which all others are measured.
Seventh MPEG-4 AVC/H.264 Video Codecs Comparison - Standard Version (http://compression.ru/video/codec_comparison/h264_2011/)

Now personally I think the following comparison article has it's faults, and I've not read it completely thoroughly, but the gist of the conclusion is probably correct. And it's interesting as they provide "mouseover" screenshots which allow you to compare encoder outputs.
H.264 encoding - CPU vs GPU: Nvidia CUDA, AMD Stream, Intel MediaSDK and x264 (http://www.behardware.com/articles/828-27/h-264-encoding-cpu-vs-gpu-nvidia-cuda-amd-stream-intel-mediasdk-and-x264.html)
It’s difficult to provide a succinct conclusion to so many tests, but one point does stand out: GPU acceleration of H.264 transcoding isn’t on a par with encoding carried out by CPUs.
What these solutions bring most of all is frustration. Whether NVIDIA, AMD or Intel solutions, rapidity has been accentuated to the detriment of quality. It's rather surprising to see that with software such as MediaConverter from Arcsoft or MediaEspresso from Cyberlink, CPU encoders systematically give a better result in terms of quality than the integrated GPU encoders!

Anyway, there's two other encoders used for testing in that article (aside from the one you've already tried) which support QuickSync, if that's what you're after. There's two others listed on Intel's (http://www.intel.com/content/www/us/en/architecture-and-technology/quick-sync-video/quick-sync-video-general.html) site and apparently DVDFab (http://www.dvdfab.com/intel-quick-sync.htm) now supports hardware encoding. I've read a few complaints regarding the quality of DVDFab's output in other forums though, but I have no idea if that's necessarily hardware encoding related.

Diary Of An x264 Developer
Why so many H.264 encoders are bad (http://x264dev.multimedia.cx/archives/164)
If one works long enough with a large number of H.264 encoders, one might notice that a large number of them are pretty much awful. This of course shouldn’t be a surprise: Sturgeon’s Law says that “90% of everything is crap”.

[nm]

In terms of Quicksync it's not correct it's a hybrid kind of thing

Ah, I forgot that they may be using the programmable GPU units for motion estimation. IIRC they actually have special hardware (a "Media Sampler") within the GPU for this purpose.

[fields_g]

I'd have to disagree; it's not relevant to this discussion, but Tesla, and now Kepler GPUs have been used in supercomputers for a long time, check supercomputers.org for example.

CUDA has been use in personal software too, and is fast, reliable and gives good results. Pretty much all PC video software will use it, from players to editors.



I think there is some confusion about how a Graphics card can help encoding/decoding. I hope others might comment if there is misinformation here.

1) First there is Hardware assisted decoding. This goes back to the days of DVD drives being added to computers. No fancy GPUs needed, just a decoder engine to offload CPU usage. Newer Graphics cards not only do MPEG2, but VC1 and H.264, as well as imaging tasks such as color space conversions, deinterlacing, etc.

2) The GPGPU cores can be programmed to do the encoding (CUDA). They excel becuase they have many parallel processors, but are hurt due to their limited instruction set. Sure CUDA can be made into quazi-supercomputers, but the tasks that they can accomplish well are limited. These limitations effect the encoding efficiency of CUDA H.264 encoding.

3) Specialized hardwared encoding engines (Quick Sync) are being attached after the hardware decoder (see #1). They are designed only to encode video. There are is a Programmable Execution Unit Array portion, but I am unsure of how flexible the programming is. Threre are few programs that can even access Quick Sync, and as far as I know, they all output Main Profile, loosing coding efficiency.

Now when we talk about hardware encoding (2&3), you must compare speed and bitrate efficiency. What I know about hardware encoding is from this forum and internet articles, but what I have gathered, is that CUDA and Quick Sync is fast, but the bitrate efficiency is lacking compared to a CPU encoder (CUDA < Quck Sync < CPU).

Since your goal is reduced sized transparent copies, Hardware encoding is not going to help you. Perhaps in the future there may be better hardware encoders and software that access those encoders, but until then x264 is your best bet. Choose the preset that matches your desired speed/efficiency tradeoff.

[emueyes]

So, the answer to my original question

"Is there a package that will use Intel HD2000 gfx hardware on h264 video in m2ts and mkv containers ?"


would be no, if I want decent quality. Though that becomes subjective at some point like the audio cable I was talking about. Not quite the right metaphor, really, any little artifact in video hits me like a misspelled word. But it does seem to be a little fanatical to rubbish every GPU solution as fast but of mediocre quality more than once in the same thread.

As for simplicity, as I said, I work(ad) as a programmer. I'd just hoped we got over that. Even by ignoring the de/remuxing a file, the quoted x264 commandline usage couldn't be considered difficult. Anyone could write a command like that, eh?


I'm quite unhappy about all this, of course. I had really hoped that transcoding could be both fast and good.

Intel have also changed the graphics in the Ivy Bridge chips, so that may change things, I hope.

[nm]

So, the answer to my original question

"Is there a package that will use Intel HD2000 gfx hardware on h264 video in m2ts and mkv containers ?"


would be no, if I want decent quality.

Yep.

But it does seem to be a little fanatical to rubbish every GPU solution as fast but of mediocre quality more than once in the same thread.

We stated the current situation. Every six months someone comes here to ask about GPU encoding support in x264 and the answer has been the same for a couple years now: it's probably not as useful as you think but patches are welcome. Even if it's possible to make a good GPGPU encoder that runs on current CUDA or OpenCL -compatible cards, it's certainly not easy and the speedup over pure CPU encoding won't be huge.

Every GPGPU encoder so far has been rubbish. They aren't even that fast compared to CPU encoders when both are run at comparable low quality settings. Quick Sync is currently the only useful consumer GPU/hardware solution but it gives much worse quality than x264 at mediocre settings. Hopefully Ivy Bridge improves the quality side.

[turab]

For what it's worth, there was a plan to support Quick Sync on x264 (see: http://mailman.videolan.org/pipermail/x264-devel/2011-April/008388.html), but I think the problem is that Media SDK is only available on Windows.

Hopefully Ivy Bridge improves the quality side.
I don't think the ASIC has anything to do with how a file is encoded. It's up to the software implementation using Quick Sync as far as I know.

[MasterNobody]

For what it's worth, there was a plan to support Quick Sync on x264 (see: http://mailman.videolan.org/pipermail/x264-devel/2011-April/008388.html), but I think the problem is that Media SDK is only available on Windows.
Did you read all epithets for Intel in this email? It was clear after that Intel wouldn't provide low-level access to ASIC for us and Intel Media SDK is no go because it useless for x264 (it is as attaching black-box encoder which is not x264 and so don't produce desired quality).

I don't think the ASIC has anything to do with how a file is encoded. It's up to the software implementation using Quick Sync as far as I know.
In this case ASIC is fully responsible for result of encoding (how a file is encoded) because it is not really programmable through Intel Media SDK which has only few options to change.

[turab]

Did you read all epithets for Intel in this email? It was clear after that Intel wouldn't provide low-level access to ASIC for us and Intel Media SDK is no go because it useless for x264 (it is as attaching black-box encoder which is not x264 and so don't produce desired quality).

In this case ASIC is fully responsible for result of encoding (how a file is encoded) because it is not really programmable through Intel Media SDK which has only few options to change.
I couldn't make out what the issue was from that vague email ("waiting on Intel" doesn't tell me much about the problem). I understand now, thanks. Is there any information regarding how Quick Sync works at the hardware level? Maybe it's not as limited as Media SDK makes it out to be?

[emueyes]

OK, though I haven't spent a lot of time looking at side-by-side video from different encoders, either CPU or GPU assisted, I'll just accept that for best quality a wholly software transcode is going to show the best results.

I guess the process is then going to be demux->x264 and eac3to, remux; it's a bit hard to draw flowcharts in ascii.

I'd want better control and ease of use, so if I'm scripting the process I'd probably find a machine and run Linux, which at least has bash instead of the execrable DOS cmdline. AutoHotKey does help that a lot though, as does TCC.

I've seen a few frontends available too, Handbrake, RipBot264, a few others I can't remember. Most of them seem to insist on having exact versions of other software, located just where it wants. ClownBD was nice, but doesn't do what I want here.

Any suggestion there, then, about automating the process? It'd actually be MakeMKV as the first step in that flowchart, creating the MKV in the first place. The 'disk drives are cheap' axiom doesn't hold true anymore, they aren't cheap, and neither are SATA ports, and if I can halve the size of a file without discernibly degrading it then I'd be happy.

[nm]

I've seen a few frontends available too, Handbrake, RipBot264, a few others I can't remember. Most of them seem to insist on having exact versions of other software, located just where it wants.

HandBrake doesn't depend on any external binaries. It also has a nice and simple GUI, IMO.

[Atak_Snajpera]

But only in ripbot264 you can encode using multiple pcs (Distributed Encoding Mode)

http://i.imgur.com/ozg2d.png

[CruNcher]

I couldn't make out what the issue was from that vague email ("waiting on Intel" doesn't tell me much about the problem). I understand now, thanks. Is there any information regarding how Quick Sync works at the hardware level? Maybe it's not as limited as Media SDK makes it out to be?

It's practical a trade secret though they still improve it on the Software side @ least you can see progress in terms of RC continuously.