x264 and Intel's Media Engine / Sandy Bridge

[deadrats]

Originally Posted by kieranrk

Have a guess what kind of bitrates iTunes uses?

don't tell me they're 4 mb/s 720p? omg, what a rip off!

[mariush]

deadrats, bitrate is not really a good indication of quality of some material.

It depends on the compression technology, the features the codec has and lots of other things.

For example, I could capture a 1 hour video at 720p or 1080p of me typing something on my desktop or playing a game like Transport Tycoon Deluxe (which has plenty of repetitive textures and is 2D) and then I could get great results encoding lossless this at about 100-200kbps using Camtasia's Screen Codec or anything else designed for this, but I would probably be upset with the quality of h264 at 500 kbps.

I could also film 2 hours of a presentation or a speech, and I could get away with 2mbps at 720p because it's just a person speaking on stage and not much movement, so h264 could really shine.... on the other hand if it's a football match then yes, you would need plenty of bitrate.

You're saying 720x480p MPEG2 was at 9 mbps - this is an old codec, with some features that would bring more quality per bitrate restricted and with other features limited, due to the DVD specification.

I think that you'd agree with me that were you to encode this 720x480 content with plain old MPEG, you'd probably need 15-35 mbps to retain the quality. So if the bitrate went down from 15-35 to about 10 when going from MPEG to MPEG2, why is it so hard to believe you can encode 30% more at 50% less bitrate and get great quality, from the h264 codec?

720p at 4 mbps can give you excellent quality, if those videos are encoded using quality settings and a decent encoder, which isn't hampered by settings to decode fast (cavlc vs cabac), to lower buffering time and so on.

For example, Youtube's quality could be way better at the bitrates they use, but they try to design their encoding presets so that a 720p or an 1080p can be decoded on various game consoles, telephones and low performance pcs.

[Didée]

Continuation of "Beating a Dead Horse" ....

Originally Posted by deadrats

in fact i said that if one so desired one could find some screenshots to support the claim that one was slightly better than the other.

Except for the fact that you don't really need to "search" in order to "cherry pick" a frame that makes CUDA look bad. The truth is much more simple - Cuda IS bad.

Perhaps you're not fully aware of the technicals behind. It is not that encoding per GPU would be inherently fast because GPUs are oh so fast and oh so parallel. Fact is that GPUs can do certain operations at extraordinary speeds .... most of which, alas, do not apply to video encoding.

The *major* reason why GPU encoders are as fast as they are is: usage of very simple algorithms. If you do only very few computations, then you can compute very fast. THAT is what CUDA encoder is doing. And that's what the result is looking like.

Take a 2600K, and let x264 encode some 720p with "--superfast". You'll surely get more than 150fps. Well possible that you might see 200fps.
And it will still look better than a CUDA encode. Yes, it will. And surely it will not look worse than QuickSync. Probably still better.

(Yes, the last paragraph is pure speculation. But you can call me back on this by the time when the real tests will come up. With pleasure.)

lastly, i noticed no one commented on the media sdk encode, which is ultimately the topic of this thread and comparison.

Looking from far, it doesn't fall apart in the same obvious way as the CUDA encoding. But still ...

Extension to the above flip-flop:


And this took roughly 4 times longer to encode than x264? Interesting!

[aegisofrime]

Originally Posted by Didée

And this took roughly 4 times longer to encode than x264? Interesting!

I'm guessing that's because deadrats doesn't actually have a Sandy Bridge CPU. I wonder what happens when Intel Media SDK runs on a non-Sandy Bridge CPU, and whether the algorithm is similar to Quick Sync.

[deadrats]

Originally Posted by Didée

Continuation of "Beating a Dead Horse"

perhaps you're a sadomasochist with a bestiality fetish, i believe they have a cure for that now.

Except for the fact that you don't really need to "search" in order to "cherry pick" a frame that makes CUDA look bad. The truth is much more simple - Cuda IS bad.

that's really a matter of opinion, most people that have responded (both here and the video help forums) seem to be of the opinion that the encodes were close and as i said, once the bit rate is upped to a decent rate, the differences disappear.

Perhaps you're not fully aware of the technicals behind. It is not that encoding per GPU would be inherently fast because GPUs are oh so fast and oh so parallel. Fact is that GPUs can do certain operations at extraordinary speeds .... most of which, alas, do not apply to video encoding.

The *major* reason why GPU encoders are as fast as they are is: usage of very simple algorithms. If you do only very few computations, then you can compute very fast. THAT is what CUDA encoder is doing. And that's what the result is looking like.

there are so many things wrong with what you just said that i could write a book explaining why you don't know what you are talking about.

concentrating on just the gt200 chip, it has 32 double pumped alu's per warp with 2 to 8 warps per chip (i'm doing this from memory, forgive me if i'm slightly off).

each alu is capable of executing one thread at a time and threads are coalesced implicitly within each warp.

any operation that can be executed by a cpu alu can be executed by a gpu alu, including boolean evaluations, such as the ones used in AI.

dx9 gpu's are fully programmable and are used to perform such complex tasks as blast dna sequencing, seti, encryption/decryption, virus scanning, fast fourier transforms, discrete and inverse discrete cosine transform, collision detection, 3d matrices, the list goes on.

the thought that gpu's are incapable of performing complex calculations is laughable to the extreme and easily disproven by looking at all the types of application accelerated by gpu's.

in fact, video encoding requires some of the least mathematically intensive calculations to perform, a fact born out by the reality that x264 uses mostly 8 bit integer operations, and few bigger than 16 bit integer operations (this comes straight from DS, in this very thread).

lastly, the claims you made concerning gpu accelerated encoders is easily disproven by the fact that gpu powered encoders exist in many forms: open cl based (there is an mpeg-2 encoder for the mac), dx9 based mpeg-2 encoders (a modified version of ffmpeg, that violates the gpl by not releasing the source code, that uses the vfw framework) and at least 4 separate cuda based encoders: main concept's cuda encoder (i'm almost positive this is the one tmpg uses), mc also has an open cl encoder, the adobe "mercury" engine (i believe developed by elemental, also licensed by cyberlink), sony's gpu avc (slow as hell, very poorly optimized).

Take a 2600K, and let x264 encode some 720p with "--superfast". You'll surely get more than 150fps. Well possible that you might see 200fps.
And it will still look better than a CUDA encode. Yes, it will. And surely it will not look worse than QuickSync. Probably still better.

at least you're admitting that you are pulling numbers out of thin air.

your claim however is one constantly used by the x264 faithful, but they conveniently ignore 2 things:

1) comparing a top of the line cpu to a midrange gpu is hardly fair, the proper comparison would be to a top of the line gpu.

2) a faster cpu would also benefit the gpu encoding times as it would be able to "push" the gpu harder, it could feed data and instructions to it faster (the cpu has a master/slave relationship with other processors in a system, same thing occurs with 2 or more cpu's, cpu 0 is master and the rest are slaves).

given how much faster cuda was in my tests, i have trouble believing that there is any fair scenario in which x264 could out speed a properly coded cuda powered encoder.

And this took roughly 4 times longer to encode than x264? Interesting!

it took that long because intel purposely disables simd optimizations if a non-intel cpu is detected, try it with an intel cpu then come talk to me about speed.

[deadrats]

Originally Posted by aegisofrime

I'm guessing that's because deadrats doesn't actually have a Sandy Bridge CPU. I wonder what happens when Intel Media SDK runs on a non-Sandy Bridge CPU, and whether the algorithm is similar to Quick Sync.

as i said, it appears that QS is little more than the IPP encoder being accelerated by dedicated hardware, i expect the algorithms used to be identical.

[weasel_]

Originally Posted by deadrats

this isn't meant as an "attack" on DS, what annoyed me is his tendency to dismiss technology as "useless" before it's even widely available,

and he is always right...
for cuda, for vp8 and now for sb....

Originally Posted by deadrats

i don't believe i ever said the cuda encoder included with tmpg express is twice as good as x264 and in fact i said that if one so desired one could find some screenshots to support the claim that one was slightly better than the other.

i alrady told you
Find topic where 10+ people did x264 vs cuda and if u dont see big differencethen u are blind
if u cant find just say i will post picture here for you

And dont complaine about low bitrate...
There is no point comparing 720p@10+mbit like u want

Originally Posted by deadrats

what annoys me more is when he dismisses technology as "useless" when the very company that is the first to license his software also

haahah
what ?
He need to lie and glorifie cuda becouse same comapny who use cuda licence x264 softwre ?...( x264 is not his (DS) like u said ) ...

[SeeManRun]

Wow, I have finally finished reading this entire thread over 2 nights, very impressive discussion.

Couple thoughts regarding deadrats arguments:
1) When compressing content, it seems to me you have to take into account the player. For example, when DVD's first came out, the hardware required to decode them was expensive and complex. As computer technology advanced, and process shrinks took place, the hardware required to decode the same DVD got cheaper and easier to make. Now a cell phone can decode a full bit-rate DVD. Blu-ray is no different, and I think most of these hardware encoders are great for on the fly trans-coding of your media as you are about to leave on a plane to a device that has a small screen and low processing power. You can quickly re-size your 1080p blu-ray to iPad specs at 150 fps and get out the door and gone. You don't care about quality much, you just want the content. Now take something like your wedding film; do you really care if it is 150 fps or 5 fps? I doubt it and you will surely let your computer run overnight if you have to so you get the absolute best quality in a format that might be able to fit on a regular CD or DVD (blu-ray burners are not that popular). You could keep it in the original format, but x264 lets you shrink it down to a smaller size while keeping almost all that quality in place, and certainly more than the 150 fps encoder would do. Back to the beginning of this random thought; the hardware required to view my encodes is much higher than the hardware required to view the trans-coded work (meant for iPhone, iPad, Android...), so it is really not suitable to use it at extreme qualities for those platforms as you have to worry about battery life as well as how long it takes to encode. But if you want to sit down on your sofa and use your HTC to watch a film, you surely don't care about how much power your HTC is using, you only care about quality, and that is what x264 lets you do, and do so in a reasonably small file size that requires much greater hardware to view than to watch that regular blu-ray rip.

2) You mentioned hard drives are cheap so why care about using low bit-rate. This makes no sense. Yes they are cheap, but you will just fill them up that much quicker. I upgraded my machine (more on that below, stay tuned), and it had 8 sata ports available, but my case only has 4 drive bays. Yes I could fill it up with 2 TiB drives for a total of 8, then if i run out because my encodes are so big i can just start daisy chaining devices to the USB and so on... Or, take longer for my encodes and keep the size down so I can fit more movies on my drives. Just because something is cheap doesn't mean it should be wasted.

3) Lastly, what DS and his believers seem to be saying, is that hardware is limited by the hardware used. If quicksync is a hardware encoder that is fixed function, it can do somethings and it can do them very well, but if a new technique is found, or a new algorithm that does something better, that can't be programmed into the hardware. Cuda and openCL have a better shot since they are programmable, but it seems that if you want the quality, the speedup is not worth it, and not to mention what happens when the next batch of cards come out and you have to go ahead and optimize again. The beauty of software is it will run on a pentium 4, AMD Turion, or a Core i7 2600k; same quality same options, just different speeds. And when the next Intel chip comes out, x264 will benefit from whatever architecture improvements they put in to handle more threads, increase IPC or access memory more quickly. If he started from scratch and made a brand new encoder on Cuda to take advantage of a new video card, he will be closing the door on all the people that don't have that brand of card and won't get the benefits of his work. You say software encoding is dead, but software encoding is really the only one that will last and survive. Hardware changes and encoders will have to be re-written each time, but x264 will still continue to run, and improve as people work on it. QS is what it is, and what it does not is all it will do in 2 years on the same chip. x264 in 2 years running on my new CPU will still be better than it is today, but QS will be the same).

My upgraded machine is a Core i7 2600k with lots of RAM on an ASUS P8P67, all at stock but this thing is turboing up to 3.8 ghz all the time. I got this specifically for my encoding with x264. I also have a GeForce 460 and am waiting for Badaboom to enable support for my card with 2.0 so I can try it out. Anyway, I just grabbed a file from the special features of Alice in Wonderland and ran it through with these settings via StaxRip (all 1080p@23.976):

The source file was 194,446,939 (after removing audio and whatever else was in there) bytes

"x264.exe" --preset ultrafast --tune film --crf 22 --output "00588_Output.h264" "00588.avs"

output was 172,483,260
encoded 2085 frames, 75.27 fps, 10649.66 kb/s

Dropping --crf to 23 yielded a file of 157,477,596 bytes, and encoded 2085 frames, 79.09 fps, 9225.78 kb/s

Using these settings gave me around 48 FPS:
"x264.exe" --profile baseline --crf 22 --level 3 --vbv-bufsize 10000 --vbv-maxrate 10000 --output ..."
This is the iPod preset on medium, with size 65,827,267

Now, I will try to get seriously high speed at 720p (all times given are just the encode, not the re-size):
x264.exe" --preset superfast --tune film --crf 22 ...
a disappointing encoded 2085 frames, 57.06 fps, 4843.12 kb/s

It appears resizing is making it slow, as my CPU was only 25% in use during that one.
Something interesting now:
x264.exe" --preset superfast --tune film --crf 18 --output...
encoded 2085 frames, 57.34 fps, 7735.73 kb/s

Seem to have hit the limit on how fast this can go when resizing. To be thorough I was using:
x264>x264 --help
x264 core:105 r1724 b02df7b

Wow, long post, but after reading these posts for hours, I had a lot to say. Wanted to take notes while reading. Hope there isn't a post length limit.
Any other tests anyone would like run on this CPU (I know the above quick tests are hardly conclusive, but 150 FPS seems tough, would love some info on how to maybe achieve that)?

[kypec]

@SeeManRun: thanks for your test results. I think though that for the most optimal results you should upgrade your x264 to latest revision, r1867 that is.
You also didn't specify where the resizing occurs: Avisynth? If yes then which method was used - Spline, Bicubic, Lanczos? Last but not least: which source filter have you used? FFMS2, DGDecNV, DirectShow?
All these conditions may have big influence on overall encoding performance observed...
Please post your full AVS script and Avisynth version used and all above questions will be answered.

[CruNcher]

Originally Posted by Didée

Continuation of "Beating a Dead Horse" ....


Except for the fact that you don't really need to "search" in order to "cherry pick" a frame that makes CUDA look bad. The truth is much more simple - Cuda IS bad.

Perhaps you're not fully aware of the technicals behind. It is not that encoding per GPU would be inherently fast because GPUs are oh so fast and oh so parallel. Fact is that GPUs can do certain operations at extraordinary speeds .... most of which, alas, do not apply to video encoding.

The *major* reason why GPU encoders are as fast as they are is: usage of very simple algorithms. If you do only very few computations, then you can compute very fast. THAT is what CUDA encoder is doing. And that's what the result is looking like.

Take a 2600K, and let x264 encode some 720p with "--superfast". You'll surely get more than 150fps. Well possible that you might see 200fps.
And it will still look better than a CUDA encode. Yes, it will. And surely it will not look worse than QuickSync. Probably still better.

(Yes, the last paragraph is pure speculation. But you can call me back on this by the time when the real tests will come up. With pleasure.)




Looking from far, it doesn't fall apart in the same obvious way as the CUDA encoding. But still ...

Extension to the above flip-flop:


And this took roughly 4 times longer to encode than x264? Interesting!

Yep superfast was a evolution in terms of Speed/Quality/Power Consumption/Complexity (+ subme 2 you can give it a tad more quality on lower bitrates and create a new profile between superfast and fast) i love it and that's why i really would like to know how QuickSync holds up (especialy as it seems to be tuned in most of the Encoding applications for speed and could be more tuned for quality) to from the first looks of Anandtech you can see it heavily blurs on the frames he chose though as always we know nothing about the test and no samples are released so its bogus and we need someone from doom9 doing the visual evaluation of it vs x264
Heck we don't know if it even uses somekind of AQ (Nvidias Encoder uses that) yet with Anandtechs results i would say nope but those could be I-frames vs P-frames or whatnot

[kolak]

Originally Posted by kypec

@SeeManRun: thanks for your test results. I think though that for the most optimal results you should upgrade your x264 to latest revision, r1867 that is.
You also didn't specify where the resizing occurs: Avisynth? If yes then which method was used - Spline, Bicubic, Lanczos? Last but not least: which source filter have you used? FFMS2, DGDecNV, DirectShow?
All these conditions may have big influence on overall encoding performance observed...
Please post your full AVS script and Avisynth version used and all above questions will be answered.

It is very funny when people do speed test, by feeding encoders through big avisynth scripts- juts render script into yv12 file

Andrew

[SeeManRun]

Originally Posted by kypec

@SeeManRun: thanks for your test results. I think though that for the most optimal results you should upgrade your x264 to latest revision, r1867 that is.
You also didn't specify where the resizing occurs: Avisynth? If yes then which method was used - Spline, Bicubic, Lanczos? Last but not least: which source filter have you used? FFMS2, DGDecNV, DirectShow?
All these conditions may have big influence on overall encoding performance observed...
Please post your full AVS script and Avisynth version used and all above questions will be answered.

With an upgraded version (x264 core:112 r1867 22bfd31) I got the following log file. Please take a look and let me know thoughts. Also should note that my CPU is only around 50% used during the test. When I use my own quality settings for blu-ray movies it is 100% on the second pass (yes I use 2 pass encoding).

Let me know what else I can adjust.

[poisondeathray]

attachments have to be approved , if you copy & paste text, or use a 3rd party free hosting site (e.g. mediafire.com, sendspace.com) there is no wait

[SeeManRun]

Originally Posted by poisondeathray

attachments have to be approved , if you copy & paste text, or use a 3rd party free hosting site (e.g. mediafire.com, sendspace.com) there is no wait

Thanks for the info. Do you prefer 3rd party posting or giant messages opposed to approving attachments by chance?

[easyfab]

Hi, here is a little test ( in french) :x264 VS MediaEspresso et Quick Sync Video with the new SB i5 2300 and i7 2600k

http://www.pcinpact.com/articles/int...ideo/415-1.htm

[shon3i]

Here's my little contribution

source: park_joy_1080p.y4m
avs script:

RawSource("park_joy_1080p.y4m",1920,1080,"YV12")
AssumeFPS(24000,1001)
Spline36Resize(1280,720)

Encoders: Mainconcept H264 using CUDA, x264 r1834
Settings: Mainconcept


x264: preset: superfast, tunings: none, ABR

cabac=1 / ref=1 / deblock=1:0:0 / analyse=0x3:0x3 / me=dia / subme=1 / psy=1 / psy_rd=0.00:0.00 / mixed_ref=0 / me_range=16 / chroma_me=1 / trellis=0 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=0 / threads=9 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=1 / keyint=250 / keyint_min=23 / scenecut=40 / intra_refresh=0 / rc=abr / mbtree=0 / bitrate=6000 / ratetol=1.0 / qcomp=0.60 / qpmin=0 / qpmax=51 / qpstep=4 / ip_ratio=1.40 / pb_ratio=1.30 / aq=1:1.00

Target bitrate: 6mbps
Encoding logs: Mainconcept

Input:
---------------------
Video source: C:/Users/shon3i/Desktop/test.avs (DirectShow Import)
Output settings:
---------------------
Filename: C:/Users/shon3i/Desktop/test_out.264
VIDEO: H.264/AVC CUDA, NTSC, 1280x720pixels, 23.976pfps, VBR, 6000kbps (max N/A)
multiplexing: Elementary
---------------------

H.264 Encoding Done.
--------------------------------------------------------------------------------
Frames: 500 incoming, 500 encoded
Avg. Bitrate: 5999.40 kbits per second
Time elapsed: 12.35 seconds
--------------------------------------------------------------------------------
Transcoding results:
------------------
Done: C:/Users/shon3i/Desktop/test.avs -> C:/Users/shon3i/Desktop/test_out.264 (0h 00m 12.523s, 39.9 FPS)
-------------------

x264:

x264 [info]: frame I:2 Avg QP:25.30 size:178487
x264 [info]: frame P:254 Avg QP:31.24 size: 49476
x264 [info]: frame B:244 Avg QP:34.03 size: 9280
x264 [info]: consecutive B-frames: 2.8% 94.8% 2.4% 0.0%
x264 [info]: mb I I16..4: 6.9% 40.9% 52.2%
x264 [info]: mb P I16..4: 0.7% 1.7% 0.9% P16..4: 76.6% 0.0% 0.0% 0.0% 0.0% skip:20.0%
x264 [info]: mb B I16..4: 0.2% 0.2% 0.0% B16..8: 11.4% 0.0% 0.0% direct:21.9% skip:66.4% L0:24.9% L1:43.2% BI:31.9%
x264 [info]: final ratefactor: 26.27
x264 [info]: 8x8 transform intra:49.0% inter:29.5%
x264 [info]: coded y,uvDC,uvAC intra: 83.7% 71.6% 54.9% inter: 32.0% 12.9% 3.3%
x264 [info]: i16 v,h,dc,p: 32% 14% 42% 12%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 16% 14% 30% 5% 7% 6% 6% 6% 9%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 14% 17% 23% 5% 10% 6% 7% 7% 11%
x264 [info]: i8c dc,h,v,p: 49% 18% 24% 8%
x264 [info]: Weighted P-Frames: Y:2.4% UV:0.8%
x264 [info]: kb/s:5826.40
encoded 500 frames, 41.20 fps, 5826.40 kb/s

Output:
Mainconcept - http://www.mediafire.com/?5c9a4geg809bnyu
x264 - http://www.mediafire.com/?95zzd5338q2rnnx

Enjoy.

[poisondeathray]

Originally Posted by easyfab

Hi, here is a little test ( in french) :x264 VS MediaEspresso et Quick Sync Video with the new SB i5 2300 and i7 2600k

http://www.pcinpact.com/articles/int...ideo/415-1.htm

Thanks, Maybe something was lost in the Google translation, but it looks on their tests x264 was actually faster on the "ultra fast" preset, and about the same speed on "very fast preset ?" But it was 1/2 the filesize with better quality ?

But - the rollover screenshots aren't of the same frame. They have a zip file, I'll have a look at that

[poisondeathray]

Originally Posted by SeeManRun

With an upgraded version (x264 core:112 r1867 22bfd31) I got the following log file. Please take a look and let me know thoughts. Also should note that my CPU is only around 50% used during the test. When I use my own quality settings for blu-ray movies it is 100% on the second pass (yes I use 2 pass encoding).

Let me know what else I can adjust.

For some reason it looks like you're deinterlacing a progressive source and doing a crop & resize

FFVideoSource("movies\00588 temp files\00588.mkv", cachefile="movies\00588 temp files\00588.ffindex")
AssumeFPS(23.976)
Crop(0,0, -Width % 8,-Height % 8)
ConvertToYV12()
Yadif()
Crop(0,4,-0,-4)
BicubicResize(1920,1072,0,0.5)

[poisondeathray]

Originally Posted by shon3i

Here's my little contribution

<snip>


Mainconcept - http://www.mediafire.com/?5c9a4geg809bnyu
x264 - http://www.mediafire.com/?95zzd5338q2rnnx

Thanks,

No surprise, not looking too good for the Mainconcept CUDA encoder...

What were your system specs shon3i ? CPU and GPU ?

[mp3dom]

Originally Posted by shon3i

Here's my little contribution
source: park_joy_1080p.y4m

The settings for both encoders are near the same?
Comparing the CUDA settings vs. x264 settings seems that CUDA uses only 1 B-frames (3 on x264), that CUDA can potentially have I frame every frame (x264 at least every 23), CUDA cannot reference B frames and doesn't have pyramid Bframe (both options enabled in x264). Probably doesn't change anything at the end anyway.