Software To Encode Blu Ray Files

[Blue_MiSfit]

I have a machine free that should very closely match yours:

Dual 3GHz Quad-Core Xeons (X5450, older, hotter version of your E5450 CPUs. Same speed though)
4GB RAM
Server 2003 32 bit

My source will be a piece of a BluRay that I've remuxed to MKV. Granted not the same source, but if your I/O isn't a factor you should actually get (slightly) FASTER speeds, since you aren't spending any CPU time on decode.

I'm using the CLI suggested here http://sites.google.com/site/x264bluray/home/1080i-p target an average bitrate of 25mbps.

I'll update this post shortly with numbers for --preset slow and --preset veryslow

Finally, please don't post samples to rapidshare. Use mediafire, megaupload, or some other service that doesn't suck horribly. Thanks :)

Derek

[Blue_MiSfit]

OK so my performance numbers line up with yours, kolak. See below:


Dual X5450 CPUs - 8 physical cores
--preset slow:
First pass - ~40% CPU Usage, 13.46fps.
Second pass - 100% CPU Usage, 9.38fps

--preset veryslow:
First pass - ~40% CPU Usage, 13.45fps.
Second pass - 100% CPU Usage, 2.52fps


Also did a test on a system with dual X5570 CPUs (3 GHz Nehalem quad cores):


Dual X5570 CPUs - 8 physical cores, 16 logical cores

--preset veryslow:
First pass - ~40% CPU Usage, 14.94fps.
Second pass - 100% CPU Usage, 3.5fps


Performance sounds about right.

Anyone have ideas on why first pass doesn't scale to 100% usage? Only thing I can think of is the frame type decision (b-adapt 2), which IIRC isn't multithreaded. Am I correct here??

Derek

[kolak]

OK so my performance numbers line up with yours, kolak. See below:


Dual X5450 CPUs - 8 physical cores
--preset slow:
First pass - ~40% CPU Usage, 13.46fps.
Second pass - 100% CPU Usage, 9.38fps

--preset veryslow:
First pass - ~40% CPU Usage, 13.45fps.
Second pass - 100% CPU Usage, 2.52fps


Also did a test on a system with dual X5570 CPUs (3 GHz Nehalem quad cores):


Dual X5570 CPUs - 8 physical cores, 16 logical cores

--preset veryslow:
First pass - ~40% CPU Usage, 14.94fps.
Second pass - 100% CPU Usage, 3.5fps


Performance sounds about right.

Anyone have ideas on why first pass doesn't scale to 100% usage? Only thing I can think of is the frame type decision (b-adapt 2), which IIRC isn't multithreaded. Am I correct here??

Derek

Yep- about the same. Maybe you also have some bottleneck :)


Andrew

[Biggiesized]

Very unlikely. Besides, 1745 has been out for quite awhile now (several weeks, which is a fairly long time for x264 updates). This is usually a good sign ;)

Derek
Seriously, what's up with that?

I need to get my fix! It's been about a month!

[kolak]

Tried different source- quite clean, shot on RED.

Slow presset (tune film):

1st pass- 20fps
2nd pass- 14fps

There is quite big speed difference depending on the source nature.

Andrew

[shon3i]

You said you have it faster :)
These are not i7 Xeons, but old E5450.

CC-HDe realtime in Quality mode on 4 core machine? No way :) (maybe if you overclok it to 4GHz)

Andrew
It is faster i have ~7fps with 4 cores, you have ~9fps with 8 cores? Mine procesor is AMD Phenom I your is Intel, i think is from same era.

Btw you don't need to go with slower presets, you can just fine beat CEE with medium preset and you will definitly have more speed. CEE is faster because is not complex as x264 is, and output quality is superb because bitrate is huge. x264 on that bitrate can be safetly downgraded to lover presents eg medium, while keep quality. CCE will probably been better in fades, dark areas.

[kolak]

It is faster i have ~7fps with 4 cores, you have ~9fps with 8 cores? Mine procesor is AMD Phenom I your is Intel, i think is from same era.

Btw you don't need to go with slower presets, you can just fine beat CEE with medium preset and you will definitly have more speed. CEE is faster because is not complex as x264 is, and output quality is superb because bitrate is huge. x264 on that bitrate can be safetly downgraded to lover presents eg medium, while keep quality. CCE will probably been better in fades, dark areas.

It's not faster- my encode will be finished earlier:)

Also - look at my 2nd test :) x264 is sensitive for source nature- speed varies a lot.

With source, which I've tried x264 does not beat CC-HDe even at slow pressets. Some frames are better, but overall CC-HDe quality is more constant. x264 has very good, but also relatively bad frames.

Andrew

[shon3i]

It's not faster- my encode will be finished earlierYep, but not significant, you should get ~15fps for same commandline, also my processor is not clocked and work at default 2.2Ghz.

Difference in speeds with different way of feeding source mean bottleneck. Your decoder use significant process power to decode frames or something else. You maybe get more speed with CCE on way that x264 gives you higher speed.

[kolak]

Yep, but not significant, you should get ~15fps for same commandline, also my processor is not clocked and work at default 2.2Ghz.

Difference in speeds with different way of feeding source mean bottleneck. Your decoder use significant process power to decode frames or something else. You maybe get more speed with CCE on way that x264 gives you higher speed.

Look at me 2nd test :) I get 14fps for 2nd pass.

It depends what you feed into encoder. It also depends how encoder is sensitive for surces with different nature. Some are more some less.

I don't have any bottleneck on source stage- tried different sources- yv12, Canopus HQ and always the same.
Blue_MiSfit also getting the same speeds.
90% people test x264 with BD ripped sources- mostly nice film source alreday after encode- very easy to encode.

Try grainy source and you won't get 7fps on your machine :)

Tried turning off PSY- not good at all.

Andrew

[shon3i]

90% people test x264 with BD ripped sources- mostly nice film source alreday after encode- very easy to encode.Yes, but on my workplace i usualy encode from HD-SDI, yv12 or some lossless codec (huffyuv for example)

Tried turning off PSY- not good at all.What about --psy-rd 1.0:0 or --psy-rd 0.7:0??

[kolak]

Yes, but on my workplace i usualy encode from HD-SDI, yv12 or some lossless codec (huffyuv for example)

What about --psy-rd 1.0:0 or --psy-rd 0.7:0??

Tried --psy-rd 0.5:0.0 - also not good. Grain tends to be blocky.
Will try with 0.8.


Andrew

[Blue_MiSfit]

kolak,

Can you describe your encoding workflow in CCE-HDe? How many passes are you running? Any filtering, custom bit allocation / quantization, etc?

Also, you posted a file on rapidshare. Any chance you could post it somewhere actually usable, like mediafire etc?

Thanks!

Derek

[Biggiesized]

Or if you upload to Rapidshare, register an account so we can download the file more than 10 times.

[kolak]

kolak,

Can you describe your encoding workflow in CCE-HDe? How many passes are you running? Any filtering, custom bit allocation / quantization, etc?

Also, you posted a file on rapidshare. Any chance you could post it somewhere actually usable, like mediafire etc?

Thanks!

Derek

2 or 3. If there is a time 4.
I almost never use filtering. Just playing with Q Charasterictics, Quantize, sometimes Grain and Matrixes.
When needed use Region for tweaking bits allocation on the frame.

Next time won't use rapidshare:)

Andrew

[Blue_MiSfit]

kolak:

It's not really fair to compare x264 to manually tweaked CCE-HDe encodes, especially when it comes to region tweaking! If you do, at least make tweaks to zones in x264.

You might argue that we're comparing two encoders, but actually we're comparing two encoder CORES. CCE-HDe has a very full featured GUI wrapped around it, and x264 very well may have a similar GUI in the future. Let's keep things fair :)

Derek

[Dark Shikari]

It's completely braindead to make a comparison which consists entirely of "looking for areas that look bad" when, with one encoder, every time you see such an area, you go back and boost the quality with zones.

It's like comparing two books, except every time you find a part you don't like in one of the books, you rewrite it (but not the other one). Then, you declare the rewritten book to be the better one. This isn't just malicious, this is cheating.

[kolak]

kolak:

It's not really fair to compare x264 to manually tweaked CCE-HDe encodes, especially when it comes to region tweaking! If you do, at least make tweaks to zones in x264.

You might argue that we're comparing two encoders, but actually we're comparing two encoder CORES. CCE-HDe has a very full featured GUI wrapped around it, and x264 very well may have a similar GUI in the future. Let's keep things fair :)

Derek

Both encodes were simple 2 pass encodes- no tweaking.
This what I have said applies to my typical workflow- not for this test.
CC-HDe gives more consistent quality between frames.

DS- is not about some parts of the encode - is more about single frames or even parts on the single frame. Blu-code does the same, where CC-HDe will keep whole frame more constant in quality (also more constant quality between I,P and B frames).

I have not finished my test and waiting for suggestions.
You can quickly see quality differences between I,P,B frames with subtract option in avisynth. I frames are much better. What is the solution?

Tried playing with PSY, but it didn't help.

Andrew

[MasterNobody]

I have not finished my test and waiting for suggestions.
Suggestion. Stop posting until you provide samples and settings (so anyone can reproduce your results) to confirm that you don't lie/cheat :devil:. Because till now you only make bold statements without objective proofs.

[kolak]

Suggestion. Stop posting until you provide samples and settings (so anyone can reproduce your results) to confirm that you don't lie/cheat :devil:. Because till now you only make bold statements without objective proofs.

I've said what settings I have used. Gave a sample of a problematic frame. Also explained what are the problems with x264 encode and asked about solution.

Tried lowering PSY- not good at all.

Andrew

[MasterNobody]

I've said what settings I have used.
I never seen your full command lines.
Gave a sample of a problematic frame.You mean rapidshare crap link which return "The file of the above link no longer exists" and which 10 download limit was probably reached even before you posted the link :devil: ?
This can't be counted as sample.
Also you don't provide source sample so we can't say that it is not in the source. And again you don't want to use public available sources (without any real reason except trolling).
Also explained what are the problems with x264 encode and asked about solution.
We don't need yours explanations, we need real samples (both which you claim "good" and "bad").

[Blue_MiSfit]

Nobody can see your sample because you put it on rapidshare and that dies after only a few downloads!!

Please re-post this sample!

Also, MasterNobody is quite correct in saying that your explanations are simply not adequate for validating your claims. We need test cases that we can actually verify on our end.

Derek

[kolak]

Nobody can see your sample because you put it on rapidshare and that dies after only a few downloads!!

Please re-post this sample!

Also, MasterNobody is quite correct in saying that your explanations are simply not adequate for validating your claims. We need test cases that we can actually verify on our end.

Derek

Explanation is very clear- much better quality I frames (than P, B).
Patches of much worse quality areas on a single frame.

And please stop saying nonsense like- we don't know if it's not on the source :)

If you read my posts you will find command line:)

I can't provide this source file, because it's a "proper" film source.
I have Avatar trailer uncompressed source- do you think I can put it on the web?

Trying download this:

http://www.freehillproductions.com/RED/

Crossing The Line trailer. Shot on RED, ProRes source. This would be good source for testing.

Don't have this encode anymore- but can redo.

Andrew

[kolak]

Another test:

Source:

ftp://ftp.ldv.e-technik.tu-muenchen.de/pub/test_sequences/1080p/

station2.yuv (fps assumed to 23.976)

x264 com:

x264 --bitrate 28000 --preset slow --tune film --weightp 0 --bframes 3 --nal-hrd vbr --vbv-maxrate 38000 --vbv-bufsize 30000 --level 4.1 --keyint 24 --b-pyramid strict --slices 4 --aud --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1 --pass 1

3 random (25,26,27), consecutive frames (source, pro encoder, x264):

http://www.mediafire.com/?39n5lzrzj83zfqw

x264_V2 is the same command but with tune grain.


Andrew

[nm]

3 random (25,26,27), consecutive frames (source, pro encoder, x264):

http://www.mediafire.com/?39n5lzrzj83zfqw

The first x264 encode keeps grain/noise in the sky. The pro encoder flattens it and x264_V2 even more so.

x264_V2 is the same command but with tune grain.

Hmm. Doesn't look like that to me. Maybe the first x264 encode was with --tune grain and V2 with --tune film?

I'm downloading the source to try it out later.

[kolak]

No tweaking in pro encoder.
Almost default settings.

x264 encodes are not swapped.


Andrew

[nm]

Did you only run one pass?

[kolak]

Did you only run one pass?

2

Redoing tune grain encode- maybe it was 1 pass!

Now it's definately 2 pass:

http://www.mediafire.com/?az7dibmsu44c6sp


also tweaked pro

http://www.mediafire.com/?bzav321o6adw3z1




Andrew

[kieranrk]

Crossing The Line trailer. Shot on RED, ProRes source. This would be good source for testing.


Crossing The Line is quite an apt film considering what the date is. (though it isn't uncompressed ;-) )

There's also:

http://warosu.org/island/Island_1080p24_lag_51_%5bB76E513A%5d.avi

(someone seems to have mirrored it)

[kolak]

Crossing The Line is quite an apt film considering what the date is. (though it isn't uncompressed ;-) )

There's also:

http://warosu.org/island/Island_1080p24_lag_51_%5bB76E513A%5d.avi

(someone seems to have mirrored it)

ProRes is good enough.
It's real source, slow, high motion, gradients of sky and also moments with deatiled elements.
I'm having problems to download it.

[kolak]

In my test x264 encodes are about 2.5MB under ideal size.

35.911Kb ideal size

x264 is about 32.767Kb
pro is 35.903Kb


Andrew

[Doom9]

Also, MasterNobody is quite correct in saying that your explanations are simply not adequate for validating your claims. We need test cases that we can actually verify on our end.I can only second that (or third?) - we all know you prefer CCE-HD, but having heard that, it's either time to move on, or we restrain ourselfes to verifiable facts.

For starters, don't hid behind sources that cannot be shared. If you need examples on how a fair comparison is to be conducted, refer to my codec comparisons... they use publicly available sources, encoders, and all settings have been published - so if anybody does not agree with the results, they have all that is needed to see with their own eyes.

[kolak]

I can only second that (or third?) - we all know you prefer CCE-HD, but having heard that, it's either time to move on, or we restrain ourselfes to verifiable facts.

For starters, don't hid behind sources that cannot be shared. If you need examples on how a fair comparison is to be conducted, refer to my codec comparisons... they use publicly available sources, encoders, and all settings have been published - so if anybody does not agree with the results, they have all that is needed to see with their own eyes.

I don't prefer any encoder- I use few, because none of them is the best for every source file.

I also see place for x264 :)

Done test on publicly available source and posted some frames- whole source has the same nature and these frames are representative for the whole file.

Tomorrow will try different source.


Andrew

[kolak]

Crossing The Line is quite an apt film considering what the date is. (though it isn't uncompressed ;-) )

There's also:

http://warosu.org/island/Island_1080p24_lag_51_%5bB76E513A%5d.avi

(someone seems to have mirrored it)

What is it?

Not interested in CGI.


Andrew

[MasterNobody]

kolak
And again you ignore what other tells you. For information, screenshots are not the same as samples. You must provide samples to be useful to this thread (and forum at all). If you forgot (or may be never known :devil: ) samples are relatively short encoded video files.

Done test on publicly available source and posted some frames- whole source has the same nature and these frames are representative for the whole file.
And this is again only words without any proof (I and a lot of other members can't believe you only on words).

[Doom9]

And where are the settings for CCE-HD? After all.. somebody else might have it and would like to reproduce (so you need to give as much information as is needed.. (e.g. ecl file if they still use that).

Also, be mindful of our forum rule 12.. better, best.. (the forum software has already tagged this thread with the best attribute) it's all a slippery territory. If nobody comes along with a CCE-HD box, it's pretty much pointless to compare because nobody will be able to verify your results. So the whole CCE-HD vs x264 argument might actually fit better into a CCE-HD support forum since x264 is obtainable by everybody whereas CCE-HD is more than one year's salary for a lot of people.

@edit: MasterNobody is correct of course.. because only few have access to commercial encoders, it makes sense that you provide the encoded samples that the average person cannot reproduce on their own - not that this would allow ruling out any mischief on your end though (VAF comes to mind) but at least it's a start.

[kolak]

kolak
And again you ignore what other tells you. For information, screenshots are not the same as samples. You must provide samples to be useful to this thread (and forum at all). If you forgot (or may be never known :devil: ) samples are relatively short encoded video files.

BMP are 100% quality of the decoded file.
You can choose any frame number to post.
Do you think I seat and create fake BMPs?

Try x264 yourself- source is free and compare.

I can decode pro video and post it as an uncompressed.

Andrew

[nm]

In my test x264 encodes are about 2.5MB under ideal size.

That's probably normal for x264 with a short clip like this one.

35.911Kb ideal size

x264 is about 32.767Kb
pro is 35.903Kb

Was that with the station2.yuv source? Which framerate did you use and how do you calculate that size exactly?

Because I arrive at these numbers:

313 frames, 24000/1001 fps, encoded at 28 Mbps:
28000000 * (313/(24000/1001.0)) / (8*1024) ~= 44621 kB

313 frames, 30000/1001 fps, encoded at 28 Mbps:
28000000 * (313/(30000/1001.0)) / (8*1024) ~= 35696 kB

If you assumed the source was 30000/1001 fps, that would explain why I got slightly better results when encoding it at 24000/1001 fps.


I can decode pro video and post it as an uncompressed.

What stops you from posting compressed streams?

[ajp_anton]

Why do all the pro screenshots look like they went through fft3dfilter before the encode?
In most parts of the pictures, I actually like the x264 version better.

[kolak]

And where are the settings for CCE-HD? After all.. somebody else might have it and would like to reproduce (so you need to give as much information as is needed.. (e.g. ecl file if they still use that).

Also, be mindful of our forum rule 12.. better, best.. (the forum software has already tagged this thread with the best attribute) it's all a slippery territory. If nobody comes along with a CCE-HD box, it's pretty much pointless to compare because nobody will be able to verify your results. So the whole CCE-HD vs x264 argument might actually fit better into a CCE-HD support forum since x264 is obtainable by everybody whereas CCE-HD is more than one year's salary for a lot of people.

@edit: MasterNobody is correct of course.. because only few have access to commercial encoders, it makes sense that you provide the encoded samples that the average person cannot reproduce on their own - not that this would allow ruling out any mischief on your end though (VAF comes to mind) but at least it's a start.

True- pointless discussion. Any authoring house can have demo of CC-HDe and compare by themself.
It's maybe just an information for others, who never will be able access pro encoders, that there are at least as good as x264 AVC encoders.
90% doom9 visitors don't know it.

Andrew

[kolak]

That's probably normal for x264 with a short clip like this one.



Was that with the station2.yuv source? Which framerate did you use and how do you calculate that size exactly?

Because I arrive at these numbers:

313 frames, 24000/1001 fps, encoded at 28 Mbps:
28000000 * (313/(24000/1001.0)) / (8*1024) ~= 44621 kB

313 frames, 30000/1001 fps, encoded at 28 Mbps:
28000000 * (313/(30000/1001.0)) / (8*1024) ~= 35696 kB

If you assumed the source was 30000/1001 fps, that would explain why I got slightly better results when encoding it at 24000/1001 fps.


What stops you from posting compressed streams?

I haven't used whole file. My download has crashed, so I have used what was downloaded- 246 frames.

Andrew

[Blue_MiSfit]

Why won't you post compressed video samples?!?

That's all anyone wants!!! Screenshots only tell you a part of the story!!!

Derek

[kolak]

Why do all the pro screenshots look like they went through fft3dfilter before the encode?
In most parts of the pictures, I actually like the x264 version better.

It's your imagination :)

There was no filtering at all.

It's not about what do you like better, but which is closer to the source :)
Compare them in Photoshop to the source frames and you will see difference. Both encoders are very good, but you can see that not the same.


Andrew

[MasterNobody]

Try x264 yourself- source is free and compare.Of course I can do this. But next is not acceptable:I can decode pro video and post it as an uncompressed.
Because this way I can't check that you don't cheating.
So I need encoded sample at least by "Pro" encoder of publically available source (used station2.yuv is OK). Until than I will continue to think that you only trolling.

[kolak]

Why won't you post compressed video samples?!?

That's all anyone wants!!! Screenshots only tell you a part of the story!!!

Derek

True- that's why I can post decompressed uncompressed/lossless version of the pro encoder.

When you get your version better ask if you can post any encoded video- the answer will be that you're not allowed, until you own the software. You may end up with problems.

Andrew

[Doom9]

It's maybe just an information for others, who never will be able access pro encoders, that there are at least as good as x264 AVC encoders.
90% doom9 visitors don't know it.Seeing as most commercial Blu-rays use commercial encoders, I doubt people would think there's but one H.264 encoder out there.
And yet, why should they care too much about something that's unobtainable to them anyway?

Now if you work for an authoring place and can get the expensive toys and would like to compare to x264, perhaps you can work out something with the x264 authors that satisfies both the needs by the copyright holders and the needs of the people helping you out.

[kolak]

Seeing as most commercial Blu-rays use commercial encoders, I doubt people would think there's but one H.264 encoder out there.
And yet, why should they care too much about something that's unobtainable to them anyway?

Now if you work for an authoring place and can get the expensive toys and would like to compare to x264, perhaps you can work out something with the x264 authors that satisfies both the needs by the copyright holders and the needs of the people helping you out.

This is the work :)

It's helping to make x264 even better, by showing that pro encoders can match or outperform x264.

It's also about GUI for x264, which I'm talking about all the time. As a compressionist you have to have ability to verify you encoding and quickly mark and re encode problematic places. All pro encoders offers this- in one or another way.
Speed is also a small issue, but because x264 is free you can have many instances running on many PCs. Farm encoding would be nice, but not essential.
You can also include output through Black Magic card to external broadcast monitor. BM's SDK is available and I think easy to implement. Blu-code and CC-HDe do it.

I think x264 came to the point where money starts talking, which is not good.


Andrew

[kieranrk]

You can also include output through Black Magic card to external broadcast monitor. BM's SDK is available and I think easy to implement.

A patch has been written for ffmpeg for input but i'm not sure about output.

[kolak]

A patch has been written for ffmpeg for input but i'm not sure about output.

Good to know.

Andrew

[Blue_MiSfit]

When you get your version better ask if you can post any encoded video- the answer will be that you're not allowed, until you own the software. You may end up with problems.


OK, so maybe I can't post compressed samples if I don't own the software.

It sounds like your company does own the software, so why can't you post compressed samples?!

Derek

[kolak]

OK, so maybe I can't post compressed samples if I don't own the software.

It sounds like your company does own the software, so why can't you post compressed samples?!

Derek

It does not own it :(


Andrew