Software To Encode Blu Ray Files

[sneaker_ger]

Yes, it's on the source.
http://www.abload.de/img/1215source3iel.png

[ajp_anton]

After 20 pages, has anyone posted a kolak-approved sample encoded by CCE-HD?
No.

Only this (some decompressed section):

http://www.mediafire.com/?7zxeva4elepjqzn


Andrew
So why do people not just stop arguing until you actually prove *anything* that you've said so far?

[kolak]

Yes, it's on the source.
http://www.abload.de/img/1215source3iel.png

Ok.

Fades are still problematic in your encode.


Andrew

[kolak]

So why do people not just stop arguing until you actually prove *anything* that you've said so far?

x264 gained a lot from this disccusion.

What do you want me to prove?

Andrew

[creamyhorror]

So what problems/weaknesses in x264's output do you think are left to deal with (besides segment encoding)? Just fades? Is there still an issue with "overly bad frames"?

It's been an interesting thread and perhaps some area of improvement for x264 will emerge. Or possibly even a fair comparison with a studio-level encoder at Blu-ray bitrates, elusive as it's proven.

[nm]

x264 gained a lot from this disccusion.
The commits so far have got nothing to do with this, AFAIK.

What do you want me to prove?
We still haven't seen an actual encoded stream from any encoder but x264. Except Dark Shikari's samples, which you didn't like.

Does your company own any other "pro" encoder so that you could post some encoded streams?

[sneaker_ger]

Same binary, same settings, except weightp is set to 1 instead of 0. (5.4 fps over both passes)

http://www.abload.de/img/185sourcekfz0.png
http://www.abload.de/img/185encodeweightp1mg8u.png
http://www.abload.de/img/185encodekdtp.png (my old encode for comparison)

I don't know if you want to call that problematic, but on these the difference to the source seems most obvious. (On other frames I had to look several times to make sure I didn't accidentally copy the source frame twice.)

http://rapidshare.com/files/431454531/island_weightp1.mkv

[kolak]

Same binary, same settings, except weightp is set to 1 instead of 0. (5.4 fps over both passes)

http://www.abload.de/img/185sourcekfz0.png
http://www.abload.de/img/185encodeweightp1mg8u.png
http://www.abload.de/img/185encodekdtp.png (my old encode for comparison)

I don't know if you want to call that problematic, but on these the difference to the source seems most obvious. (On other frames I had to look several times to make sure I didn't accidentally copy the source frame twice.)

http://rapidshare.com/files/431454531/island_weightp1.mkv

It's still bad.

I will post my video, so you can check that there are no bad (much worse quality) frames.


Andrew

[kolak]

So what problems/weaknesses in x264's output do you think are left to deal with (besides segment encoding)? Just fades? Is there still an issue with "overly bad frames"?

It's been an interesting thread and perhaps some area of improvement for x264 will emerge. Or possibly even a fair comparison with a studio-level encoder at Blu-ray bitrates, elusive as it's proven.

Bad frames were due to buggy build- this has been sorted.

Fades and high motion scenes are left (sometime x264 flattens details there).
Fades are more important.


Andrew

[mp3dom]

I'm trying latest official non-patched build from x264.nl. Maybe for films it works wonderfully, but with anime I can't get decent results in 'sensible' parts (fades, grain retention etc) and I'm obtaining again what I've wrote months ago.
This is my commandline:

--pass 2 --profile high --preset veryslow --tune film --keyint 24 --b-adapt 2 --bframes 3 --ref 4 --b-pyramid strict --open-gop bluray
--slices 4 --bitrate 35000 --vbv-maxrate 40000 --vbv-bufsize 30000 --no-fast-pskip --no-dct-decimate --weightp 2 --sar 1:1 --level 4.1
--videoformat ntsc --colorprim bt709 --transfer bt709 --colormatrix bt709 --nal-hrd vbr --aud --pic-struct

As everyone can see, bitrate used is *huge* (35 Mbps - reached during final encode so the file is not undersize/oversize) and I'm using weightp 2 to push the maximum from fades (even if it can create problems on some standalones). Sequence is about 4400 frames (3 minutes long) and is a real source of a real anime movie. I cannot share the source to public because it's unreleased material but I'm posting some screenshots and parts of it. The sequence starts with a long fade from noisy dark grey.
Just for reference: encoding speed (on my core i7 920) if someone is interested:
x264 preset veryslow: ~1-1.5 fps (variable)
proencoder: ~4-6 fps (variable - plain 2 pass vbr, no segment encoding)

Note: pro-enc source have a YV12->YUY2->YV12 colorspace conversion applied (the encoder doesn't accepts YV12). There's also a chroma shift that I'm currently investigating.

All the images follows this order: source-proencoder-x264
Every image has the first part as the original image encoded and an 'enhanced' part with luma/contrast boost that shows better what are the results.

Frame 0 (first frame, I frame):
0000 (http://i56.tinypic.com/2a9uwq8.png)
As you can see, the noise changes a lot in a strange pattern, way different than the original

Frame 45:
0045 (http://i53.tinypic.com/34tb97s.png)
This frame is at the beginning of the fadein. It starts to slowy shows some part of image (which is a 'tunnel' effect). The x264 version losts completely the grain and shows some macroblocks. From frame 45 to 60 all frames shows an alternation of grain/no grain while the original all have grain.

Frame 61:
0061 (http://i53.tinypic.com/98zsxi.png)
As you can see here, the grain is messed up and the image have too much distortion. The pro-encoder have chroma shift as said above (I'm watching into it) anyway the grain is retained in a better way.

Frame 1884:
1884 (http://i52.tinypic.com/14ka0p3.png)
Again the grain changes its pattern. You can also see some minor macroblocks. This image comes immediately after a scenechange so, presumably (but not sure) is an I frame. It needs 3-4 frames before retaining the original grain again.

Frame 2286:
2286 (http://i52.tinypic.com/2445zeo.png)
Same as above.

Frame 2291:
2291 (http://img263.imageshack.us/img263/3407/2291comparison.png)
As above, you can see that the grain is less homogeneous.


These are just examples.
To be honest and objective as possible, actually I cannot find a good/valid reason to switch to use x264 in favor of my pro-encoder of choice (the lack of segment encoding is a problem, because in this example of 35Mbps avg I need to use it but simply I cannot encode the whole 2 hrs movie at 1 fps! This will take days!)

If Dark Shikari is really interested for giving suggestions or has requests, he can contact me via PM. If he want to attack/accuse me or thinks I'm trolling, it's better if he simply ignore this post.

[Biggiesized]

mp3dom, thanks for your contribution. This discussion is very important if x264 is to be taken seriously as a professional Blu-ray encoder.

Were you using Blu-code as your professional encoder?

[Blue_MiSfit]

I'd very much like to see the real screenshots, not specific channels etc.

Also, can you post the full streams from both encoders please? If you have legal issues with posting the full streams from your "pro encoder", can you transcode to a lossless, or at least zipped YUV?

[shon3i]

I think he cannot post streams (even x264) due source legal issues.

I cannot share the source to public because it's unreleased material but I'm posting some screenshots and parts of it.

[sneaker_ger]

Also, can you post the full streams from both encoders please? If you have legal issues with posting the full streams from your "pro encoder", can you transcode to a lossless, or at least zipped YUV?

I cannot share the source to public because it's unreleased material

Maybe it would be possible to do a comparison using a publicly available source? (If you find anything that shows the same problems.)

[Puncakes]

Don't want x264 to use its compression magic? Turn it off. --trellis 0 --deadzone-intra 2 --deadzone-inter 2 --no-mbtree --qcomp 0.8 --aq-strength 0.5 --fade-compensate 1.0 --ipratio 1.0 --pbratio 1.0

Shazam, transparent encodes at Blu-ray bitrates, but crap encodes at reasonable bitrates.

[kolak]

I'd very much like to see the real screenshots, not specific channels etc.

Also, can you post the full streams from both encoders please? If you have legal issues with posting the full streams from your "pro encoder", can you transcode to a lossless, or at least zipped YUV?

Just check fades on Island trailer- the same. Big (easily visible) problems on fades. This is main x264 weakness.
Tried different things and nothing helps.


Andrew

[jpsdr]

I'm trying latest official non-patched build from x264.nl.

Maybe you should try last Jeeb's version (1772_v2), with fade compensation and Chroma weightp improvement, to check if at least on fade, there is improvements.

Edit : New commit apparently, so, i would advice to wait Jeeb's 1788 with fade compensation, to try if it improves things.

[shon3i]

Don't want x264 to use its compression magic? Turn it off. --trellis 0 --deadzone-intra 2 --deadzone-inter 2 --no-mbtree --qcomp 0.8 --aq-strength 0.5 --fade-compensate 1.0 --ipratio 1.0 --pbratio 1.0

Shazam, transparent encodes at Blu-ray bitrates, but crap encodes at reasonable bitrates.
I am aslo thinked same, anyway in near past when x264 has no AQ and PSY-RD, transparency be easily achieved with higher bitrate and weaker settings, for example on some video i use

Warning: this is old x264 cmd
--bframe 3 --b-pyramid --b-rdo --bime --weightb --ref 5 --mixed-refs --direct auto --deblock -2:-2 --crf 18 --qcomp 0.75 --ipratio 1.25 --pbratio 1.33 --partitions "i8x8,p8x8,b8x8" --8x8dct --me "hex" --subme 7 --no-fast-pskip --no-dct-decimate --trellis 1 which give me total transparency, while some other heavy settings like partition all, trellis 2 and other totaly destroy picture.

Aslo from mine test x264 deblocker produce smother image than some pro encoders, -2:-2 is something like MC -1:-1, Atemes adaptive -1 and etc.

[mp3dom]

I'd very much like to see the real screenshots, not specific channels etc.

The screenshots are real but due to the source I cannot share screenshots that can be used to easily detect the source so I've posted some part of the images. The problems anyway cover the entire image. I only have boosted the luma/contrast just to show better the artefacts. No other 'manipulation' was made.


Also, can you post the full streams from both encoders please? If you have legal issues with posting the full streams from your "pro encoder", can you transcode to a lossless, or at least zipped YUV?
No I can't, the problem is not the encoded file, but the source. The source is unreleased and not yet available so I cannot share it.


Maybe it would be possible to do a comparison using a publicly available source? (If you find anything that shows the same problems.)

No problem if I can find something similar. I mainly work with animation, not films so excluding my sources it's quite difficult to find something else. Anyway I'm watching into it.


Just check fades on Island trailer- the same. Big (easily visible) problems on fades. This is main x264 weakness.

Yes, fades are problematic at least on anime (i would say). Anyway only the first 3 screenshots posted by me refers to a 'fadein' problem. All the others screenshots came from scenechange or with moderate/high complexity scenes.


Maybe you should try last Jeeb's version (1772_v2), with fade compensation and Chroma weightp improvement, to check if at least on fade, there is improvements.

I've tried it previously but the results were exactly the same (at least on the first frames, with the long fadein). Due to this problem I thought that it was a build miscompilation and switched to official build but results were again the same, as you can see.

[Blue_MiSfit]

Frankly, if you can't share anything real, this is a completely useless test. There's absolutely no way any of it can be verified.

For all we know you could be an employee of any company that develops "pro encoders" with a vested interest in making x264 look bad.

If this was my forum, I'd remove your posts. Technically they're against Doom9's request that codec comparisons be completely transparent (i.e access to source, encodes, and settings across the board so they can all be replicated).

Derek

[Biggiesized]

I've been given permission to post a 2+ minute source that has lots of gradients and fades. The author would like you guys to test x264 with it in order to improve its fading quality and gradient retention. You can also compare other encoders to it. This footage is meant to be tested using applicable Blu-ray settings.

Should I start another thread to post it so it doesn't get lost in this kerfuffle?

[kieranrk]

Should I start another thread to post it so it doesn't get lost in this kerfuffle?

Yes you should.

[kolak]

Frankly, if you can't share anything real, this is a completely useless test. There's absolutely no way any of it can be verified.

For all we know you could be an employee of any company that develops "pro encoders" with a vested interest in making x264 look bad.

If this was my forum, I'd remove your posts. Technically they're against Doom9's request that codec comparisons be completely transparent (i.e access to source, encodes, and settings across the board so they can all be replicated).

Derek

Try any source yourself (even Island trailer)- no need to go any further. Show that there is no problem- simple.

There were many x264 encodes of this source (done by different people and with different builds) and all of them have the same problem- fades (also a smaller problems on high motion scenes).


Andrew

[Biggiesized]

I'm going to reupload the Leo test footage too. I know realize that it can be greater than 100MB in size. This will remove the hassle of multiple downloads.

[Biggiesized]

Okay, Leo is now one 7-zip archive to download:

http://rapidshare.com/files/432122371/LeoMX_1080p_i420.7z

Should I leave the others up?

[mp3dom]

Frankly, if you can't share anything real, this is a completely useless test. There's absolutely no way any of it can be verified.

I'm not interested that other users will verify my unreleased sources or other users tests my unreleased sources. It's enough that some developer/contributor, if really is interested in suggesting better parameters/improving/fixing x264, contact me (as already happened, editor's note).


For all we know you could be an employee of any company that develops "pro encoders" with a vested interest in making x264 look bad.

Bah... I'm quite astonished by your comment as, if I understand correctly, you're working in a company that made post-production or other similar tasks so you should understand our position when working with unreleased material. Also if you want to extends x264 encoder capabilities to bluray you should put in account that 99% of the times the authoring houses works with unreleased material so it's not easy to ask for sources above all in an internet forum clearly visible on all the world. What do you expects? That we choose an encoder using bluray rips as a source? That we share videos when we've signed NDA?Anyway I'm not obliged to use x264 and if you think I'm not trustworthy you can simply ignore me. In this current developement state, with my footages (animation) probably I'll continue to use other encoders at high bitrates. All others can continue to use x264 and encode HD data at 10 Mbps and says 'Wow!' (just as a note: I thought to do a good thing for the community spending my spare time (that I could invest in something better) making a comparison with my footage... evidently I was wrong). I will continue to watch the x264 progression state and I'll start to use it when it will start to completely satisfy me.


If this was my forum, I'd remove your posts. Technically they're against Doom9's request that codec comparisons be completely transparent (i.e access to source, encodes, and settings across the board so they can all be replicated).

Derek
I'm not here to persuade you (or all other users) that proencoders are better. I haven't even said what proencoder I've used for the 'comparison' because I don't want to focus the post in a "x264 vs. proencoder <name>" fight.
Anyway... no problem, if the moderators thinks that my previous post is against doom9 policy, they're free to remove it. Evidently x264 is already perfect as is.

[kieranrk]

Evidently x264 is already perfect as is.

Nobody ever said that.

[kolak]

Okay, Leo is now one 7-zip archive to download:

http://rapidshare.com/files/432122371/LeoMX_1080p_i420.7z

Should I leave the others up?

Thanks.

I have seen this footage- bit boring, but can test fades and gradients retention.

Andrew

[aegisofrime]

I'm not interested that other users will verify my unreleased sources or other users tests my unreleased sources. It's enough that some developer/contributor, if really is interested in suggesting better parameters/improving/fixing x264, contact me (as already happened, editor's note).

What did the developer tell you? If it's possible I think it would be great if you could share that with the rest of us so that we know what to do if we encounter the same problems you are having, and how to fix it.

[shon3i]

Thanks.

I have seen this footage- bit boring, but can test fades and gradients retention.

Andrew
Yep, but is good example for x264, even on veryslow x264 fail on this source, and produce banding and unacceptable fades

[Sagittaire]

I'm trying latest official non-patched build from x264.nl. Maybe for films it works wonderfully, but with anime I can't get decent results in 'sensible' parts (fades, grain retention etc) and I'm obtaining again what I've wrote months ago.
This is my commandline:

--pass 2 --profile high --preset veryslow --tune film --keyint 24 --b-adapt 2 --bframes 3 --ref 4 --b-pyramid strict --open-gop bluray
--slices 4 --bitrate 35000 --vbv-maxrate 40000 --vbv-bufsize 30000 --no-fast-pskip --no-dct-decimate --weightp 2 --sar 1:1 --level 4.1
--videoformat ntsc --colorprim bt709 --transfer bt709 --colormatrix bt709 --nal-hrd vbr --aud --pic-struct

As everyone can see, bitrate used is *huge* (35 Mbps - reached during final encode so the file is not undersize/oversize) and I'm using weightp 2 to push the maximum from fades (even if it can create problems on some standalones). Sequence is about 4400 frames (3 minutes long) and is a real source of a real anime movie. I cannot share the source to public because it's unreleased material but I'm posting some screenshots and parts of it. The sequence starts with a long fade from noisy dark grey.
Just for reference: encoding speed (on my core i7 920) if someone is interested:
x264 preset veryslow: ~1-1.5 fps (variable)
proencoder: ~4-6 fps (variable - plain 2 pass vbr, no segment encoding)

Note: pro-enc source have a YV12->YUY2->YV12 colorspace conversion applied (the encoder doesn't accepts YV12). There's also a chroma shift that I'm currently investigating.

All the images follows this order: source-proencoder-x264
Every image has the first part as the original image encoded and an 'enhanced' part with luma/contrast boost that shows better what are the results.

Frame 0 (first frame, I frame):
0000 (http://i56.tinypic.com/2a9uwq8.png)
As you can see, the noise changes a lot in a strange pattern, way different than the original

Frame 45:
0045 (http://i53.tinypic.com/34tb97s.png)
This frame is at the beginning of the fadein. It starts to slowy shows some part of image (which is a 'tunnel' effect). The x264 version losts completely the grain and shows some macroblocks. From frame 45 to 60 all frames shows an alternation of grain/no grain while the original all have grain.

Frame 61:
0061 (http://i53.tinypic.com/98zsxi.png)
As you can see here, the grain is messed up and the image have too much distortion. The pro-encoder have chroma shift as said above (I'm watching into it) anyway the grain is retained in a better way.

Frame 1884:
1884 (http://i52.tinypic.com/14ka0p3.png)
Again the grain changes its pattern. You can also see some minor macroblocks. This image comes immediately after a scenechange so, presumably (but not sure) is an I frame. It needs 3-4 frames before retaining the original grain again.

Frame 2286:
2286 (http://i52.tinypic.com/2445zeo.png)
Same as above.

Frame 2291:
2291 (http://img263.imageshack.us/img263/3407/2291comparison.png)
As above, you can see that the grain is less homogeneous.


These are just examples.
To be honest and objective as possible, actually I cannot find a good/valid reason to switch to use x264 in favor of my pro-encoder of choice (the lack of segment encoding is a problem, because in this example of 35Mbps avg I need to use it but simply I cannot encode the whole 2 hrs movie at 1 fps! This will take days!)

If Dark Shikari is really interested for giving suggestions or has requests, he can contact me via PM. If he want to attack/accuse me or thinks I'm trolling, it's better if he simply ignore this post.


well it's non realistic demonstation because why make good encoding for invisible image part?

Moreover most professional encoder use black level normalisation pre-filter for cut all PSY invisible black part. It's possible to make better encoding in low luma level but it's a volontary PSY optimisation for x264.

[Sagittaire]

Possible to have other source for Island trailer because download stop at 1 Go. I will make good professional encoding with x264 and other codec too ...

[shon3i]

well it's non realistic demonstation because why make good encoding for invisible image part?Well it's not invisible if you have well calibrated monitor and good eye. mp3dom make this enhanced versions to point people where to look, not everyone can notice, especialy when we have not full screenshots or part of stream.

[mp3dom]

well it's non realistic demonstation because why make good encoding for invisible image part?
I can assure you that it's indeed visible. The enhancement was made to show the artefacts to everybody (for those who have too dark monitor or uncalibrated). Also, as shon3i as already said, without fullscreen screenshots is hard to see it well.

[Biggiesized]

Professional encoding and reviewing should be done on monitors with higher black levels so these compression artifacts in the dark parts are easier to spot.

[Sharc]

Interesting dispute.
It seems that tremendous focus is put onto finding frames where x264 is inferior to 'pro' encoders. Any frames discovered where x264 is superior?
How about watching the movie as opposed to analyzing individual frames? How does x264 compare?

[sneaker_ger]

Possible to have other source for Island trailer because download stop at 1 Go. I will make good professional encoding with x264 and other codec too ...

ftp://island@94.242.206.22/

[nm]

Interesting dispute.
It seems that tremendous focus is put onto finding frames where x264 is inferior to 'pro' encoders. Any frames discovered where x264 is superior?

Not really because nobody has yet posted a complete CC-HDe encode of the trailer. Kolak made a 42-frame segment available as a raw AVI.

How about watching the movie as opposed to analyzing individual frames? How does x264 compare?

Fades are probably the only place where one might notice a difference on playback.

@kolak & mp3dom: which fade did you find most problematic in The Island -trailer?

[mp3dom]

If you watch it in realtime the fadein produced by x264 is *visible* inferior because the fadein is quite long (about 5 secs). For all the rest there's no visible difference in realtime... both streams are near the same. If you watch closely you can spot the difference (the proencoded stream is a bit less sharp with grain, a bit 'soften' up but some shoots have less micro blocks in complex parts, while x264 retain the original sharpness in grain but some parts have too 'different' grain retained and some minor micro blocks in complex parts). The microblocks are anyway difficult to spot, almost not-visible even in a frame by frame comparison so it's not a real problem to me.

Edit:
nm: I'm not working on the island trailer as it's not my typical 'source'. I'm working on my source (which you can see some screenshots in one of my previous post). Even this comment ^^^ refers to my source

[nm]

nm: I'm not working on the island trailer as it's not my typical 'source'. I'm working on my source (which you can see some screenshots in one of my previous post). Even this comment ^^^ refers to my source

Well, I agree with Blue_MiSfit that this is not useful or even interesting for the rest of us. We want to test and replicate things on a publicly available source.

[Sharc]

Has anyone tried revision 1788 of x264? It should improve fades.

[shon3i]

Has anyone tried revision 1788 of x264? It should improve fades.
Not usefull for authorting BD's, can cost of compatibility issues

[Sharc]

Not usefull for authorting BD's, can cost of compatibility issues
Why? Does the latest revision produce non-compliant streams?
Is 1788 different from previous revisions in this respect?

[simonhorlick]

Why? Does the latest revision produce non-compliant streams?
Is 1788 different from previous revisions in this respect?

commit 9488de4f74637360301e3cb1e6c7f25a41a93a37
Author: Jason Garrett-Glaser <darkshikari@gmail.com>
Date: Sun Nov 14 03:34:26 2010 -0800

Chroma weighted prediction
Like luma weighted prediction, dramatically improves compression in fades.
Up to 4-8db chroma PSNR gain in extreme cases (short, perfect fade-outs).
On actual videos, helps up to ~1% overall.
One example video with a decent number of fades (ef OP): 0.8% bitrate reduct
Fixes a lot of artifacts in fades at lower bitrates.

Original patch by Dylan Yudaken <dyudaken@gmail.com>.

[sneaker_ger]

It's not different from earlier versions, it's just that some players are buggy (http://x264dev.multimedia.cx/archives/212) and produce artifacts when playing encodes with weighted p prediction (at least the "smart" one), although it is allowed by the BD standard. That's why many stay away from it when encoding for Blu-Ray.

[shon3i]

Why? Does the latest revision produce non-compliant streams?
Is 1788 different from previous revisions in this respect?
No, --weightp from start are incompatible with some chipsets.

[shon3i]

It's not different from earlier versions, it's just that some players are buggy (http://x264dev.multimedia.cx/archives/212) and produce artifacts when playing encodes with weighted p prediction (at least the "smart" one), although it is allowed by the BD standard. That's why many stay away from it when encoding for Blu-Ray.
Yep, Blu-Code aslo use weightp/b.

[kolak]

Possible to have other source for Island trailer because download stop at 1 Go. I will make good professional encoding with x264 and other codec too ...

Use what you have.
Use VirtualDub to recover avi.


Andrew

[Sharc]

It's not different from earlier versions, it's just that some players are buggy (http://x264dev.multimedia.cx/archives/212) and produce artifacts when playing encodes with weighted p prediction (at least the "smart" one), although it is allowed by the BD standard. That's why many stay away from it when encoding for Blu-Ray.
Ah, the issue with the weighted p. .....:rolleyes:

[jpsdr]

Okay, Leo is now one 7-zip archive to download:
http://rapidshare.com/files/432122371/LeoMX_1080p_i420.7z
Should I leave the others up?

With what can i open this file :confused:
Is it possible to open it with VirtualDub or with an avisynth script ?
If yes, how ?

Thanks.