View Full Version : Has anyone used X264 to produce a replicated BD title?


Lyris
10th August 2009, 20:15
First of all: I apologize if this question has already been asked. However, I don't think there has ever been a central thread asking this question.

Have any Blu-ray Disc authors out there produced titles encoded using X264 (perhaps one of the various patched versions), and had its output verified and passed by a replication facility? The reason I ask is because in the coming months, I will be working on a retail BD title and if it's possible to use X264, I would absolutely love to do it due to the quality of its output.

My other option is the upcoming Netblender DoStudio Workflow Edition, comes with an integrated AVC encoder. Even although at BD bitrates, the differences can be less obvious, I still want to use the absolute best quality option within the allocated budget. The big draw of this is that its output is guaranteed to be compatible.

Lastly, to the developers of x264: thank you for this wonderful encoder and everything that you do for free. I realise that as a highly scalable system, BD compliance is not at the top of you priorities in producing an h.264 encoder. But if anyone can pitch in, I'd be very grateful!

LoRd_MuldeR
10th August 2009, 20:27
x264 can't produce streams that are 100% conform to the BD specs, because as far as we know the BD specs say that at least 4 slices are required. But x264 doesn't use slices currently.

Dark Shikari
10th August 2009, 20:29
x264 can't produce streams that are 100% conform to the BD specs, because as far as we know the BD specs say that at least 4 slices are required. But x264 doesn't use slices currently.There's a patch that adds slicing support, though the author still hasn't sent to me the version he promised...

Lyris
10th August 2009, 20:53
Could it be the versions found here? I've used it to produce streams that certainly play on the players I have here, but not with consistent success...

http://skystrife.com/x264/?C=M;O=A

Chengbin
10th August 2009, 21:12
Are you telling me that I can't encode a video with x264 in the BD's allowed maximum H.264 settings and it won't play on a Blu-ray player?

Lyris, THANK YOU for choosing x264 to encode your next blu-ray title. After you're done with it, could you tell us the title of the movie you produced? I might buy that Blu-ray just to see what "real" 1080p looks like without low passing.

LoRd_MuldeR
10th August 2009, 21:14
Could it be the versions found here? I've used it to produce streams that certainly play on the players I have here, but not with consistent success...

http://skystrife.com/x264/?C=M;O=A

I don't think so. I guess the patch mention by Dark Shikari was never released to the public -or- the (early) version released was not working properly yet.

Are you telling me that I can't encode a video with x264 in the BD's allowed maximum H.264 settings and it won't play on a Blu-ray drive?

The drive shouldn't be the problem :p

However a H.264 decoder that is strictly limited to the BD specs may fail to decode streams produced by current x264.

Consequently a BD authoring software, which checks the H.264 streams against the BD specs (and not against "real" implementations), would have to reject x264's streams.

Anyway, as far as I know most (all?) the BD players that exist in reality would handle those streams just fine. It only can't be guaranteed...

nixo
10th August 2009, 21:53
You could always use level 4.0. I've been told that doesn't require slices to be BD compliant.

--
Nikolaj

JEEB
10th August 2009, 22:12
The patch for 1195 can be found on the mailing list here (http://mailman.videolan.org/pipermail/x264-devel/2009-August/006106.html), but since D_S says "...though the author still hasn't sent to me the version he promised..." I'd think it'd be ok for you to be somewhat wary of it albeit he says he has been using it for a while...

Anyways, nice to hear that people are interested in using x264 with blu-ray manufacturing.

Lyris
10th August 2009, 22:17
Chengbin: I'll gladly share details and the production process once it's out there, yes. However, there are thankfully a lot of unfiltered AVC titles out there - all of the Disney titles I've seen seem to have got through unmolested. Low-passed BD titles are thankfully the exception rather than the norm.

Are you telling me that I can't encode a video with x264 in the BD's allowed maximum H.264 settings and it won't play on a Blu-ray drive?
Professional authoring software scrutinizes every part of the input file. By default, the output of X264 is rejected. Lord_Mulder has explained the whole situation.

When you're stamping out thousands of discs for a title that is probably going to get some attention, it "maybe" working fine on "most" BD players isn't enough for me to sleep easy!

Nixo: is 4.0 allowed on BD? What are the limitations of it when compared to 4.1?

Guest
10th August 2009, 22:21
You could always use level 4.0. I've been told that doesn't require slices to be BD compliant.
Correct. From the spec:

9.5.1.3.1 Parameter limits

[...]
If level_idc in SPS indicates level 4.1, each picture shall be encoded as multi-slice picture with 4 or
more slices per picture. The number of macroblocks in any slice shall not exceed 1/2 of the total
number of macroblocks. The number of macroblock rows in every slice in the picture should be as
equal as possible for the current picture height and interlace coding mode.

* In case of 1920x1080 video format with frame_mbs_only_flag=1, it is recommended that
each slice has 17 macroblock rows (17/17/17/17 configuration).

* In case of 1920x1080 video format with frame_mbs_only_flag=0 and
mb_adaptive_frame_field_flag=0, it is recommended that in each field, odd numbered slices
have 8 macroblock rows each and even numbered slices have 9 macroblock rows each
(8/9/8/9 configuration) or some similar configuration.

* In case of 1920x1080 video format with frame_mbs_only_flag=0 and
mb_adaptive_frame_field_flag=1, it is recommended that odd numbered slices have 16
macroblock rows each and even numbered slices have 18 macroblock rows each
(16/18/16/18 configuration) or some similar configuration.

* In case of 1280x720 video format, it is recommended that four slices have 11/11/11/12 or
some similar configuration.

nixo
10th August 2009, 22:31
Nixo: is 4.0 allowed on BD? What are the limitations of it when compared to 4.1?
As far as I know the limitations are really just a bit lower max bitrate (--vbv-maxrate 25000).

--
Nikolaj

Lyris
10th August 2009, 22:35
Very interesting - but according to x264, Level 4.0 has an upper bitrate limit of 25mbps, which I'd like to avoid.

Edit: you got there before me - 4.1 is something I'd really like to use, but your suggestion is one possibility.

Lyris
11th August 2009, 15:57
Well, here's hoping that the quality of the AVC encoder in DoStudio Workflow will be comparable to x264 at BD bitrates. The coded material is going to be razor-sharp (4K DI sourced from a RED One) and probably handheld and free-moving so hopefully things can hold up. I'll post some comparisons of the two encoders' output when I can.

Chengbin
11th August 2009, 16:36
In case if you don't know, weight-p will be available soon (hopefully), and according to Dark Shikari on the SoC page, it has the "potential to significantly improve encoding quality"

I want to ask a question. BD specs say the video needs 4 slices. When they refer to slices, do they mean the multithreading technique? If it is, I thought at 4 slices, PSNR is already -1dB?

BTW, lyris, you're the man. Thanks for doing this for us. I'm curious of how a $3000 (!!!) software compare with x264. If you don't mind asking, does the company provide you with software, or do you have to pay for the software yourself?

Lyris
11th August 2009, 16:38
Will that do anything to help compatibility, though?

Chengbin
11th August 2009, 17:19
Will that do anything to help compatibility, though?

No, just picture quality (especially with mb-tree during fades)

LoRd_MuldeR
11th August 2009, 17:40
I want to ask a question. BD specs say the video needs 4 slices. When they refer to slices, do they mean the multithreading technique? If it is, I thought at 4 slices, PSNR is already -1dB?

With "Slices" they mean that each frame should consist of at least 4 slices (parts), that can be decoded/processed independently.

If a stream consists of several slices, this allows (but doesn't enforce) slice-level multi-threading.

So if an encoder relies on slice-level multi-threading, but the stream doesn't consist of several slices, that encoder will fail to decode the stream (at an acceptable speed).

I guess they included the "4 slices" requirement in the BD specs, because they wanted to make sure that even such decoders will be able to decode the BD streams.

Anyway, all the state-of-the-art H.264 decoders (ffmpeg-MT, CoreAVC, DivX H.264, etc.) use frame-level multi-treading and hence work with single-slice streams just fine.

It appears that also the hardware decoders used in "stand-alone" BD players are capable of decoding Level 4.1 H.264 streams without multiple slices.

Also it was found that slices reduce compressibility. In case of x264 the loss was around -0.1 PSNR for 4 slices (link (http://git.videolan.org/gitweb.cgi?p=x264.git;a=blob;f=doc/threads.txt;h=3777b516117c6a7dc8e4943dcee0386449dc405c;hb=9a54c483f5f12aa5614ba0a4a4cfaee19377047e)). And: The more slices/threads, the bigger the loss!

Frame-based multi-threading not only gives more speed-up, also the loss in quality/compressibility is very small - even at high number of threads.

Lyris
11th August 2009, 18:09
If you don't mind asking, does the company provide you with software, or do you have to pay for the software yourself?
There's the possibility of both (or a combination). I'm a freelance author right now so there's nothing set in stone.

Regarding how a $3000 encoder is going to compare to x264: one of the most respected names in the video world said that none of the main studio's encoders matched the quality of x264. Certainly, on most BD titles, I can see small compression artefacts that I don't see on (quite grainy) test encodes I've done. I imagine this is the "speed over quality" argument again.

Oh, if anyone wants to buy me a copy of Cinema Craft HD, I will gladly try that out ;)

Chengbin
11th August 2009, 18:43
Could the quality difference be the fact that x264 does not use slices, where the commercial encoders must?

Sagekilla
11th August 2009, 18:47
No, slices can make a difference but they shouldn't be that huge of a difference. If it is, they're doing their slicing horribly wrong.

Most of the difference comes from the various rate control and RDO algorithms x264 uses: MB Tree, trellis, psy-rd, aq, etc.

LoRd_MuldeR
11th August 2009, 18:49
Could the quality difference be the fact that x264 does not use slices, where the commercial encoders must?

I wouldn't expect slices to make such a difference. Not at BD bitrates (36 Mbit/s).

Manao
11th August 2009, 20:51
The link given by LoRd_MuldeR applies to SD (480p). 1080p with 4 slices will suffer a slightly lower PSNR loss than 480p with 2 slices, thus 0.038dB, which roughly amounts to a 0.8% bitrate loss at same quality.

shon3i
11th August 2009, 22:26
@neuron2, where you find that blu-ray specs?

Guest
11th August 2009, 22:56
I have it through my employer as we implement decoders and players. Sorry, but I cannot re-distribute it. I can answer questions on the spec though.

shon3i
11th August 2009, 23:19
I have it through my employer as we implement decoders and players. Sorry, but I cannot re-distribute it. I can answer questions on the spec though.
Can you just list the lack's of x264 that need to became full Blu-ray compatible, and point developers what is primary?

Dark Shikari
11th August 2009, 23:21
Can you just list the lack's of x264 that need to became full Blu-ray compatible, and point developers what is primary?I have the spec as well.

Guest
11th August 2009, 23:25
If a developer wants to contact me privately about it, the appropriate information can be provided.

EDIT: Dark already has it, so that's the main man.

LoRd_MuldeR
11th August 2009, 23:34
I have the spec as well.

So you can already tell whether there are any other problems with BD compliance in x264, in addition to the known slices issue?

shon3i
11th August 2009, 23:36
I have the spec as well.
then why, not make x264 to be blu-ray compatible finaly? Last time when i asked you, you say that you do not have to specification because they are hide specs from public.

Dark Shikari
11th August 2009, 23:47
then why, not make x264 to be blu-ray compatible finaly? Last time when i asked you, you say that you do not have to specification because they are hide specs from public.I don't have the whole spec, only part of it, and you pretty much already know what that part contains (VBV limits, 3 B-frames, no P-frames referencing B-frames, GOP size, slicing, etc).

benwaggoner
12th August 2009, 04:26
Very interesting - but according to x264, Level 4.0 has an upper bitrate limit of 25mbps, which I'd like to avoid.
25 Mbps is really a very high bitrate for 1080p24 H.264! using single-slice x264 with good settings. I'd expect Level 4.0 x264 with 25 Mbps peaks to be transparent for almost all real-world film content. 1080i30 without MBAFF, could be an issue, but 24p should be just fine. HD DVD titles looked great with older encoders, shorter GOPs, and similar or even lower peaks (one highly rated title was VC-1 with 13 Mbps ABR and 19 Mbps PBR).

Remember Blu-ray's peak bitrate was defined for MPEG-2 applications. It's rather overkill for modern codecs.

kypec
12th August 2009, 06:05
Remember Blu-ray's peak bitrate was defined for MPEG-2 applications. It's rather overkill for modern codecs.My thinking exactly. I'm just a noob in encoding myself but seeing as SD content has 720x576=414 720 pixels to be delivered by MPEG-2 on standard DVD at peak video bitrates ~9 Mbps I would assume that HD material consisting of 1920x1080=2 073 600 pixels cannot require more than 5 times higher bitrate ~45 Mbps with same encoder used.
Now take into account we may use H.264 instead of older MPEG-2 and necessary video bitrate can be halved for the same visual quality I suppose. :cool:

shon3i
12th August 2009, 12:00
I have it through my employer as we implement decoders and players. Sorry, but I cannot re-distribute it. I can answer questions on the spec though.
Ok, i have several questions:

1. What is max allowed vbv-buffersize and vbv-maxrate for Level 4.0 from blu-ray specification (i.e for 4.1 are 30000/40000 buffer/maxrate), not H264.
2. So definitly L 4.0 dosen't require slices? can be then L 4.0 video primary video?
3. What is max allowed vbv-buffersize and vbv-maxrate for Level 3.2 from blu-ray specification for SD content, and what resoultion and fps allowed for SD, and is L 3.2 maximum for SD?
4. Is there anything about BD5/9 and special restrictions such as VBV or other?


That is for now.

Thanks.

Chengbin
12th August 2009, 13:16
25 Mbps is really a very high bitrate for 1080p24 H.264! using single-slice x264 with good settings. I'd expect Level 4.0 x264 with 25 Mbps peaks to be transparent for almost all real-world film content. 1080i30 without MBAFF, could be an issue, but 24p should be just fine. HD DVD titles looked great with older encoders, shorter GOPs, and similar or even lower peaks (one highly rated title was VC-1 with 13 Mbps ABR and 19 Mbps PBR).

Remember Blu-ray's peak bitrate was defined for MPEG-2 applications. It's rather overkill for modern codecs.

I thought many Blu-ray's even at mid to high 20mbps look "not that good", as in lack of fine detail

Remember this thread? (http://forum.doom9.org/showthread.php?t=147300)

LoRd_MuldeR
12th August 2009, 13:23
I thought many Blu-ray's even at mid to high 20mbps look "not that good", as in lack of fine detail

That's probably not the result of the bitrate, but a result of bad encoders and/or bad authoring ;)

Chengbin
12th August 2009, 14:34
That's probably not the result of the bitrate, but a result of bad encoders and/or bad authoring ;)

Therefore because of these bad encoders, you're lacking bitrate even though you have 25-30mbps. Therefore you're not getting "transparency"

Didn't Dark Shikari say even with x264, you need a lot more than 25mbps to reach transparency on 1080p videos (from a master)?

LoRd_MuldeR
12th August 2009, 15:35
Didn't Dark Shikari say even with x264, you need a lot more than 25mbps to reach transparency on 1080p videos (from a master)?

This is highly source dependent. I'm pretty sure for most sources you will be fine with x264 and much lower bitrates...

Chengbin
12th August 2009, 16:20
Dark Shikari, do you mind clarifying this quote in terms of what kind of source are you referring to?

x264 would require significantly more bitrate for transparency if encoding from a lossless source than from a Blu-ray. Blu-rays generally seem to have very little high-frequency content, almost as if they were lowpassed. I wouldn't be surprised if they were--and even if they aren't, the quantizatoin matrices they use often do so for them anyways.

LoRd_MuldeR
12th August 2009, 16:26
He mans that if you were encoding from an original lossless source ("Master"), then you would need significant more bitrate to retain all detail, because there is so much detail/grain in that source.

However we usually don't have access to such sources. What we have are BluRay's, which have already been filtered and compressed in a lossy way. Many detail is already gone for good when we get the data!

Re-encoding such lossy sources requires less bitrate to retain all details (the details that are still there) for obvious reasons...

Chengbin
12th August 2009, 16:40
LoRd_MuldeR, I see you didn't finish reading my post, I said "from a master". :)

Anyway, I'm very excited to see what Lyris can do with his project using x264. BTW, p_l_e_a_s_e use the full 50GB of the disk. I've seen so many Blu-rays with low 20mbps bitrate and the disk still has 10-15GB free. But then truthfully this 40mbps cap is kind of a bottleneck. If you use high average bitrates (like Jumper's 33mbps), if you use bitrate viewer on it, it will probably look like CBR.

Lyris
12th August 2009, 17:33
Sadly, if compliance isn't at least "likely", then I really doubt I'll use x264 (unless some sort of solution appears). Even if it involves going with a slightly less high quality but compliant encoder, the quality gains wouldn't be worth the amount of sleep I'd lose (or the consequences of putting a dodgy disc out there).

The best-looking disc in the world done with x264 would be pointless - not to mention a liability! - if it creates playback problems.

Chengbin: on past DVD projects, I've normally just filled the disc up - I just don't see any reason not to when you're guaranteed a certain amount of space. A little headroom is good since the sizes can be a little unpredictable, but re-encoding a bonus feature to be 3% smaller to get the whole thing to fit is really no big deal with the speed of today's SD MPEG-2 encoding.

Chengbin
12th August 2009, 18:23
If a solution for x264's compatibility doesn't come up, and you're forced to use a commercial encoder, could you encode a tiny part of the film with x264 "just for fun"?

LoRd_MuldeR
12th August 2009, 19:22
Well, it seems the outcome of this thread is that x264 should be able to produce a BD compliant stream at Level 4.0, just not at Level 4.1 yet.

Also: If somebody is willing to spend thousands of dollars for a proprietary encoder, he could also give that money to the x264 devs to complete the required slices patch ;)

Lyris
12th August 2009, 19:34
Also: If somebody is willing to spend thousands of dollars for a proprietary encoder, he could also give that money to the x264 devs to complete the required slices patch
If compliance could be guaranteed and access to a stream verifier was possible, this is an option I would certainly consider running past the money people (assuming that the off-the-shelf encoders were producing visibly poorer results). Real-world issues could get in the way though.

I know there are a lot of people who would love to see a free, high quality BD compliant AVC encoder. And also people who would probably be quite disturbed by the appearance of one...

turbojet
12th August 2009, 20:40
One thing that's been reported is some Panasonic and Samsung players don't display video with 24p enabled from x264 encoded streams. It may be an x264 issue or it may be that directshowsource changes the framerate of the input which also breaks eac3to. Either way it's something that would affect millions.

Forteen88
12th August 2009, 21:53
Well, it seems the outcome of this thread is that x264 should be able to produce a BD compliant stream at Level 4.0, just not at Level 4.1 yet.Doesn't enabling p4x4, b-pyramid (increases ref sometimes) and other resolutions than 1920x1080 or 1280x720 still create compatibility-problem?!
EDIT: Although it's no point using b-pyramid now that mb-tree doesn't allow it, which is positive in that way :) BTW, I forgot to mention that maximum 3 bframes is allowed for Blu-ray.
EDIT2: Some more resolutions are allowed, http://en.wikipedia.org/wiki/Blu-ray#Technical_specifications
Thanks for answering DS, I didn't knew that b-pyramid created that problem (beside adding a ref sometimes).

Dark Shikari
12th August 2009, 21:56
Doesn't enabling p4x4, b-pyramid (increases ref sometimes)B-pyramid violates Blu-ray rules anyways because Blu-ray rules prohibit P-frames referencing B-frames. But it's not on by default.

p4x4 is fine, but not useful for Blu-ray resolutions anyways.and other resolutions than 1920x1080 or 1280x720 still create compatibility-problem?!Well obviously, Blu-ray prohibits other resolutions.

Astrophizz
12th August 2009, 21:57
Doesn't enabling p4x4, b-pyramid (increases ref sometimes) and other resolutions than 1920x1080 or 1280x720 still create compatibility-problem?!

Then just don't turn on those functions and resize/crop the film properly.

Edit: Doh, beaten.

moviefan
12th August 2009, 22:05
B-pyramid violates Blu-ray rules anyways because Blu-ray rules prohibit P-frames referencing B-frames. But it's not on by default.

Oh, b-pyramid is not Blu-ray compliant? In that case... damn, encode again :mad:.

Does this slicing patch add the needed slice-feature to be ultimately Blu-ray compliant or are there any problems with it?

http://mailman.videolan.org/pipermail/x264-devel/2009-August/006106.html

How about a Blu-ray preset in x264 with max. settings being compliant to the specs? (except for AQ and psy-rd since it's personal taste and source dependent I guess)

cacepi
13th August 2009, 01:45
http://mailman.videolan.org/pipermail/x264-devel/2009-August/006106.html
Please stop mentioning this patch, as:

1) I didn't write it and am getting a little tired of the PMs asking me questions about it, and:

2) This commit to x264 (http://git.videolan.org/?p=x264.git;a=commit;h=7139b088129707b0daf3fd3bae9647f649deb1ba) breaks the patch and I don't have the time to fix it (although it's probably just a one-line realloc() fix), and finally:

3) Even if the patch worked with the latest revision, I still don't see how it would help as it doesn't allow you to directly specify an arbitrary number of macroblocks, just a threshold. In other words, the patch is not ready for blu-ray prime time.

benwaggoner
13th August 2009, 03:23
Ok, i have several questions:

1. What is max allowed vbv-buffersize and vbv-maxrate for Level 4.0 from blu-ray specification (i.e for 4.1 are 30000/40000 buffer/maxrate), not H264.
Not sure beyond "25 Mbps max." I don't have the Blu-ray spec handy.

2. So definitly L 4.0 dosen't require slices? can be then L 4.0 video primary video?
Yep. You could even use AVCHD spec for primary video.

3. What is max allowed vbv-buffersize and vbv-maxrate for Level 3.2 from blu-ray specification for SD content, and what resoultion and fps allowed for SD, and is L 3.2 maximum for SD?
Sorry, no idea.

4. Is there anything about BD5/9 and special restrictions such as VBV or other?
It's been recommended to use lower peaks for red-laser media for better accuracy. 3x has been bandied about (like the HD DVD "3x" mode), for which 25 Mbps video would be a fine constraint.

That said, BD-on-red-laser isn't something that a lot has been done with, and I expect we'll find significant theory/practice divergence, particularly on early hardware BD players.

benwaggoner
13th August 2009, 03:26
Chengbin: on past DVD projects, I've normally just filled the disc up - I just don't see any reason not to when you're guaranteed a certain amount of space. A little headroom is good since the sizes can be a little unpredictable, but re-encoding a bonus feature to be 3% smaller to get the whole thing to fit is really no big deal with the speed of today's SD MPEG-2 encoding.
That's find for replicated discs (well, I like to have a max 9.6 Mbps for a little headroom since very early DVD players sometimes didn't did the full 10 well).

But for DVD-R discs, early players can work better with a lower peak. So if I'm making DVD-R optimized for general consumers (so grandma with a 10 year old hand-me-down player) I'll use

Peak video + audio of 6.5 Mbps
Closed GOP


Won't matter for < 5 year old players or PC playback, of course, but raises compatibility for the old stuff.

6.5 Mbps peak isn't bad for 24p, but can be low for 30i. Carbon Mastering Mode FTW...

benwaggoner
13th August 2009, 03:38
LoRd_MuldeR, I see you didn't finish reading my post, I said "from a master". :)

Anyway, I'm very excited to see what Lyris can do with his project using x264. BTW, p_l_e_a_s_e use the full 50GB of the disk. I've seen so many Blu-rays with low 20mbps bitrate and the disk still has 10-15GB free. But then truthfully this 40mbps cap is kind of a bottleneck. If you use high average bitrates (like Jumper's 33mbps), if you use bitrate viewer on it, it will probably look like CBR.
Using up 50 GB for a movie is really specsmanship.

Having 40 Mbps peaks can make things a little easier on the compressionist, but they are NOT required for a high quality encode for VC-1 or H.264. I doubt those "low 20 Mbps" titles would have looked any better at 10 Mbps higher ABR. Bear in mind lots of BD encodes have in-loop deblocking off to avoid any potential detail loss, but I don't think that should be needed with a well-tuned adaptive quantization encoder.

It'd be interesting to mess around with BD-compliant streams to see how low CRF would have to go in order to generate ABR at 25+ Mbps.

x264's one weakness there would be Open GOP support, I suppose. However, with Level 4.0, you're allowed a 2 second GOP as opposed to the 1 second with 4.1, which can offer another efficiency boost. 25 Mbps Level 4.0 with 2 sec max GOP and single slice likely would outperform 26 Mbps Level 4.1 with a 1 sec max GOP and 4-way slicing.

Most HD film source really doesn't have THAT much detail. I did a 2.39:1 HD DVD spec 1080p MPEG-2 at 8.5 Mbps ABR disc about four years ago, and it looked really quite good most of the time. The only really bad shot was a bunch of soldiers running through a forest at dawn. Panning shots through moving foliage and lots of shadow detail was not pretty. But it was definitely HD quality for 98% of the content.

benwaggoner
13th August 2009, 03:46
He mans that if you were encoding from an original lossless source ("Master"), then you would need significant more bitrate to retain all detail, because there is so much detail/grain in that source.

However we usually don't have access to such sources. What we have are BluRay's, which have already been filtered and compressed in a lossy way. Many detail is already gone for good when we get the data!
Honestly, I've looked at plenty of real 10-bit 4:2:2 HD masters, and the kind of detail you're looking for is rarely there as well.

For CGI or VFX-heavy Digital Intermediates, sure, but I'd say the average recent BD disc is very close to what's in the master. In the cases where I've been able to compare a title someone had a complaint about with the source, most of the time the defect was present in the source (at least the 8-bit 4:2:0 compression source).

There's probably only a few dozen features that really have that intense HD "pop" that many seem to crave. But we're still mainly talking 35mm film with a 1/48th of a second shutter speed. There's a reason they have an on-set photographer instead of don't just blowing up frames for marketing materials :).

moviefan
13th August 2009, 08:38
Concerning the Blu-ray compatibility discussion with Level 4.0, is the following list a summary of key options/limitations? (media I would like to use is DVD9)


VBV-maxrate: 25000 (with L4.1 I used 30000)
VBV-bufsize: VBV-maxrate - audio-bitrate - some headroom for subtitles (???)
no b-pyramid
no multiple slices required
ref: 4 (formula gives 4 for 1920x1080 for L4.1, how about L4.0)?
bframes: 3
min-keyint: 1 (or 2?)
keyint 48 (2x fps)
nal-hrd + aud option


Could someone please confirm that these settings are maxed out within the limits of the Blu-ray specs and (probably only concerns VBV) should not be less compliant for BD9s? As many others already said, I also hope to achieve max. BD compliance with x264 and L4.0 with BD restrictions seems to be covered entirely by x264 by now.

kolak
13th August 2009, 10:27
There's the possibility of both (or a combination). I'm a freelance author right now so there's nothing set in stone.

Regarding how a $3000 encoder is going to compare to x264: one of the most respected names in the video world said that none of the main studio's encoders matched the quality of x264. Certainly, on most BD titles, I can see small compression artefacts that I don't see on (quite grainy) test encodes I've done. I imagine this is the "speed over quality" argument again.

Oh, if anyone wants to buy me a copy of Cinema Craft HD, I will gladly try that out ;)

Don't worry - it's not so so great. It's quite fast and has nice features, but quality with grainy sources is not perfect as you would expect.

Andrew

kolak
13th August 2009, 10:31
25 Mbps is really a very high bitrate for 1080p24 H.264! using single-slice x264 with good settings. I'd expect Level 4.0 x264 with 25 Mbps peaks to be transparent for almost all real-world film content. 1080i30 without MBAFF, could be an issue, but 24p should be just fine. HD DVD titles looked great with older encoders, shorter GOPs, and similar or even lower peaks (one highly rated title was VC-1 with 13 Mbps ABR and 19 Mbps PBR).

Remember Blu-ray's peak bitrate was defined for MPEG-2 applications. It's rather overkill for modern codecs.

Ben, have you seen this (check mouseover):

http://www.avsforum.com/avs-vb/showthread.php?t=1169645


It's quite ineteresting how VC-1 puts relatively more bits into B frames than AVC encode (even if AVC encode has much highier average bitrate many B frames form VC-1 are bigger).


Andrew

kolak
13th August 2009, 10:41
Ok, i have several questions:

1. What is max allowed vbv-buffersize and vbv-maxrate for Level 4.0 from blu-ray specification (i.e for 4.1 are 30000/40000 buffer/maxrate), not H264.
2. So definitly L 4.0 dosen't require slices? can be then L 4.0 video primary video?
3. What is max allowed vbv-buffersize and vbv-maxrate for Level 3.2 from blu-ray specification for SD content, and what resoultion and fps allowed for SD, and is L 3.2 maximum for SD?
4. Is there anything about BD5/9 and special restrictions such as VBV or other?


That is for now.

Thanks.


1. 30Mbit buffer and 24Mbit for maxrate.
2. Double yes.
3. 24Mbit buffer, 24Mbit maxrate, quite many resolutions, up to 60p, no- you can use level 4.1 if you need to.
4. There are special restricion in case of BD5/9. Disc read rate is limited to about 30Mbit, so there are bigger restrictions.


Andrew

kolak
13th August 2009, 10:45
Sadly, if compliance isn't at least "likely", then I really doubt I'll use x264 (unless some sort of solution appears). Even if it involves going with a slightly less high quality but compliant encoder, the quality gains wouldn't be worth the amount of sleep I'd lose (or the consequences of putting a dodgy disc out there).

The best-looking disc in the world done with x264 would be pointless - not to mention a liability! - if it creates playback problems.

Chengbin: on past DVD projects, I've normally just filled the disc up - I just don't see any reason not to when you're guaranteed a certain amount of space. A little headroom is good since the sizes can be a little unpredictable, but re-encoding a bonus feature to be 3% smaller to get the whole thing to fit is really no big deal with the speed of today's SD MPEG-2 encoding.

I don't have problems to fill up BD50 up to 99% :)
Usually they're about 90-95% full. Hollywood studios seem to have problems with this, even if calculator is a part of Windows :)

Andrew

kolak
13th August 2009, 10:47
If compliance could be guaranteed and access to a stream verifier was possible, this is an option I would certainly consider running past the money people (assuming that the off-the-shelf encoders were producing visibly poorer results). Real-world issues could get in the way though.

I know there are a lot of people who would love to see a free, high quality BD compliant AVC encoder. And also people who would probably be quite disturbed by the appearance of one...

I think, it shouldn't be free. There should be some charge and part of it could go to charity :)


Andrew

kolak
13th August 2009, 10:52
x264's one weakness there would be Open GOP support, I suppose. However, with Level 4.0, you're allowed a 2 second GOP as opposed to the 1 second with 4.1, which can offer another efficiency boost. 25 Mbps Level 4.0 with 2 sec max GOP and single slice likely would outperform 26 Mbps Level 4.1 with a 1 sec max GOP and 4-way slicing.



You can have 2 seconds GOPs only if bitarte (max bitrate) is lower than 15Mbit and this does work with 4.1 level also.


Andrew

benwaggoner
13th August 2009, 14:38
You can have 2 seconds GOPs only if bitarte (max bitrate) is smaller than 15Mbit and this does work with 4.1 level also.
Right, sorry. Conflating two different parameters there.

However, 15 Mbps peak + 2 sec GOP could be a compelling option for trying to get long-form content on DVD-5/6 media where the ABR is low enough that PBR isn't going to make it that high anyway, and you're more worried about overall efficiency.

benwaggoner
13th August 2009, 14:40
I don't have problems to fill up BD50 up to 99% :)
Usually they're about 90-95% full. Hollywood studios seem to have problems with this, even if calculator is a part of Windows :)

There's always last-minute changes in extras that can use up extra capacity. So the compressionist might target only 90% disc capacity in the disc budget to allow for some extra movie trailers or that Icelandic audio track without having to reencode everything.

benwaggoner
13th August 2009, 14:45
Ben, have you seen this (check mouseover):

http://www.avsforum.com/avs-vb/showthread.php?t=1169645


It's quite ineteresting how VC-1 puts relatively more bits into B frames than AVC encode (even if AVC encode has much highier average bitrate many B frames form VC-1 are bigger).
The default behavior from EEv3 and other tools is to have B-frames use a QP 0-2 higher than its reference frames, based on the amount of motion in the video. But CineVision PSE allows this to be overridden per-segment.

shon3i
13th August 2009, 15:29
1. 30Mbit buffer and 24Mbit for maxrate.
2. Double yes.
3. 24Mbit buffer, 24Mbit maxrate, quite many resolutions, up to 60p, no- you can use level 4.1 if you need to.
4. There are special restricion in case of BD5/9. Disc read rate is limited to about 30Mbit, so there are bigger restrictions.


Andrew
Thanks

So here is 100% Blu-Ray compatible x264 settings

* Level 4.0
* VBV-maxrate: 24000
* VBV-bufsize: 30000
* no b-pyramid
* no multiple slices required
* ref: 4
* bframes: 3
* min-keyint: 1 or 2 (2 is recommend by authoring apps)
* keyint 24 or 18
* nal-hrd + aud option

And this is for all media BD 5/9/25/50 no exceptions.

moviefan
13th August 2009, 15:45
@shon3i: And you are sure about min-keyint 2? I thought it's only valid for max bitrate of 15 Mbps...?

shon3i
13th August 2009, 16:20
@shon3i: And you are sure about min-keyint 2? I thought it's only valid for max bitrate of 15 Mbps...?
Absolutley, aslo H264 encoders such as Sonic Cinevision, Elecard use min keyint 2 as default for bluray profiles. Aslo MeGUI BD profles aslo use 2 as min.

nm
13th August 2009, 16:35
@shon3i: And you are sure about min-keyint 2? I thought it's only valid for max bitrate of 15 Mbps...?
You are confusing min-keyint with keyint (which is restricted to 1 or 2 in seconds).

Lyris
13th August 2009, 17:09
The default behavior from EEv3 and other tools is to have B-frames use a QP 0-2 higher than its reference frames, based on the amount of motion in the video. But CineVision PSE allows this to be overridden per-segment.
Ben, you're the guy to ask for this: do you have any idea if the output of Expression Encoder has been used successfully in BD authoring?

Apologies if it's a silly question, but I know very little about this tool. I seem to remember trying it many years ago without luck.

kolak
13th August 2009, 18:31
There's always last-minute changes in extras that can use up extra capacity. So the compressionist might target only 90% disc capacity in the disc budget to allow for some extra movie trailers or that Icelandic audio track without having to reencode everything.

I can understand it, but also not :)
90% is fine, but we have some disc from Warner with <20Mbit average bitrate and 40% empty BD50 :)

Andrew

Lyris
13th August 2009, 21:50
Guys (and gals): does anyone have any high quality video sources that you want me to test with these encoders.
By high quality I mean studio-grade, not recompressed from an existing compressed source.

So far, I'm encoding some clips from a RED ONE, as well as some scenes from "Big Buck Bunny" with a grain layer added on top (I will do clips of the original noise-free version too).

kieranrk
13th August 2009, 21:55
@Lyris

You could try

http://www.vuze.com/details/IK3JKDZY3JMHS7U7IMADZ36TE4K2HXCT/Army+Nurse.html

It's a DVCPROHD US military recruitment video (who knows why they decided to upload it raw?) - You'll need torrent access though but it is a legal video.
There are a few more raw versions like that on vuze too.

Also there's this raw - http://www.archive.org/details/Sita_Sings_the_Blues - 192GB in total (and it seems like you can't just download a part of it)

rack04
13th August 2009, 22:15
Thanks

So here is 100% Blu-Ray compatible x264 settings

* Level 4.0
* VBV-maxrate: 24000
* VBV-bufsize: 30000
* no b-pyramid
* no multiple slices required
* ref: 4
* bframes: 3
* min-keyint: 1 or 2 (2 is recommend by authoring apps)
* keyint 24 or 18
* nal-hrd + aud option

And this is for all media BD 5/9/25/50 no exceptions.

Correct me if I'm wrong but doesn't Level 4.0 and Level 4.1 allow 5 ref frames for 1920x1080 and 9 ref frames for 1280x720?

One thing that I've noticed on my standalone Panasonic BD-35 while playing Blu-ray discs encoded with x264 is that it freezes with ref frames greater than 7 for 1280x720.

benwaggoner
14th August 2009, 02:57
Ben, you're the guy to ask for this: do you have any idea if the output of Expression Encoder has been used successfully in BD authoring?
I doubt very much. The VC-1 library used in EEv3 is certainly capable of BD compliant output, but EEv3 doesn't support elementary stream output, and doesn't attempt to offer BD-related control.

Carbon Coder uses the same SDK, and does correctly support BD-compliant .vc1 files.

Apologies if it's a silly question, but I know very little about this tool. I seem to remember trying it many years ago without luck.
Maybe you were using something else? Expression Encoder is only a couple of years old now.

benwaggoner
14th August 2009, 07:36
Guys (and gals): does anyone have any high quality video sources that you want me to test with these encoders.
By high quality I mean studio-grade, not recompressed from an existing compressed source.
How long do you need.

I have the trailers for "The Island" and "Match Point" from Dreamworks with permission to redistribute for compression testing. I've got Lagarith 1920x1080p23.769 with 5.1 audio AVI files I could probably post somewhere.

...and that 45 minutes of 35mm "Lady Washington" source I keep thinking I'll finally be able to finish up an edit of in a couple of weeks...

shon3i
14th August 2009, 12:02
Correct me if I'm wrong but doesn't Level 4.0 and Level 4.1 allow 5 ref frames for 1920x1080 and 9 ref frames for 1280x720?

One thing that I've noticed on my standalone Panasonic BD-35 while playing Blu-ray discs encoded with x264 is that it freezes with ref frames greater than 7 for 1280x720.
From H264 specification by formula Int(MaxDPB * 1024 / 1.5 / (w * h)) where w=width, h=height and MaxDPB from table A-1 (H264 specs), give max 4 reference frames for 1080p with level 4.1 or 4.0, and 9 for 720p. But according Blu-Ray specs for 720p is alowed max 6 reference frames for level 4.1 or 4.0 and 4 ref frames for 1080p. Encoders such Sonic Cinevision and Elecard in their BD profiles always lock for more than 4 refs. So 4 ref, are safe value for all resolutions including SD material

Lyris
14th August 2009, 17:37
How long do you need.

I have the trailers for "The Island" and "Match Point" from Dreamworks with permission to redistribute for compression testing. I've got Lagarith 1920x1080p23.769 with 5.1 audio AVI files I could probably post somewhere.

...and that 45 minutes of 35mm "Lady Washington" source I keep thinking I'll finally be able to finish up an edit of in a couple of weeks...
That's a really great offer - if I set up an FTP account somewhere, would you be able to upload "The Island" trailer? I actually own the UK Warner HD DVD so there's a potential for comparison.

kolak
14th August 2009, 19:17
From H264 specification by formula Int(MaxDPB * 1024 / 1.5 / (w * h)) where w=width, h=height and MaxDPB from table A-1 (H264 specs), give max 4 reference frames for 1080p with level 4.1 or 4.0, and 9 for 720p. But according Blu-Ray specs for 720p is alowed max 6 reference frames for level 4.1 or 4.0 and 4 ref frames for 1080p. Encoders such Sonic Cinevision and Elecard in their BD profiles always lock for more than 4 refs. So 4 ref, are safe value for all resolutions including SD material

H264 specification is one thing and Blu-ray one is another. It's almost obvious that BD one is going to have more restrictions.
Another thing is that quite often pro encoders have even bigger limitation than BD spec.

Andrew

Lyris
14th August 2009, 19:39
OK, a small update.

I've tried out the Dostudio Encoder - not happy with the output, especially after having been used to x264. Visible compression artefacts in just about all cases when using 35-37mbps with the test videos I used (Red One clips, and a grained version of Big Buck Bunny). I imagine it'll serve it's purpose and make an affordable pathway into BD for many smaller producers, but as an anal-retentive videophile, it gets a no from me. X264's Level 4 ~20mbps encodes are blatantly better, a great reminder (if one was needed) that bitrate isn't everything.

There does seem to be light at the end of the tunnel, though: I've been trying Mainconcept Reference and while not quite at the level of X264's quality, what it's producing is at least slightly better than the encodes I see on retail BDs. Most importantly of all, it appears to be compliant (control is given over all necessary parameters, and the BD preset seems to set them all correctly). The next step will be working with a replicator to make sure that the output is actually compliant.

nm
14th August 2009, 20:05
There does seem to be light at the end of the tunnel, though: I've been trying Mainconcept Reference and while not quite at the level of X264's quality, what it's producing is at least slightly better than the encodes I see on retail BDs.
Do you get better results with Mainconcept at L4.1 than with x264 at L4.0?

shon3i
14th August 2009, 20:14
H264 specification is one thing and Blu-ray one is another. It's almost obvious that BD one is going to have more restrictions.
Agree, but one thing is for sure, that for 1080p is max 4 ref (according both H264 or Blu-Ray specs) and for 720p is 6 refs (according Blu-Ray specs).

Another thing is that quite often pro encoders have even bigger limitation than BD spec.

Andrew Better leave some room for errors :)



I've been trying Mainconcept Reference and while not quite at the level of X264's quality, I don't know why but Mainconcept Reference use very restricted version of their SDK, which have much more tweakable options. Only Sonic Cinevison use most advanced options from Mainconcept and can be compared with the x264, and may be approximate, even in some situations better. Mainconcept CLI use all features but :)

what it's producing is at least slightly better than the encodes I see on retail BDsMaybe hardware encoders?

Lyris
14th August 2009, 20:59
Do you get better results with Mainconcept at L4.1 than with x264 at L4.0?
No. x264 still looks better despite having around half the bit rate.

Chengbin
14th August 2009, 21:02
No. x264 still looks better despite having around half the bit rate.

I'm amazed.

This begs the question, how hard can adding slice support be? x264 used to support slices, but was removed because of a better multithreading technique. Is the old slice algorithm is bad?

shon3i
14th August 2009, 22:47
Old method can hurt more quality, but i aslo think that no one use more than 4 slices, without some special reason. Anyway if some one use more slices, quality shouldn't drop so drastically.

New metod is indeed better for decoding on modern (multi core) procesors and better for encoding because drop in quality are very, very small. But is also have only effect when is decoded with decoder which support that metod (ffdshow-mt, coreavc). The only thing which i can't understand is why we need this because we have DXVA, CUDA, which can decode Blu-Ray video normaly, and slices which are almost suported by every decoder (HW/SW), and we have enough powerfull processors to decode in single core 1080p video.

Chengbin
14th August 2009, 22:56
Old method can hurt more quality, but i aslo think that no one use more than 4 slices, without some special reason. Anyway if some one use more slices, quality shouldn't drop so drastically.

We all know slices hurt quality. But even with slices, x264 is still a lot better than commercial encoders since it doesn't hurt too badly. Unless there is compatibility issues with current features, why not just put the old code back and call it a day for Blu-ray compatibility? You can always improve it later.

shon3i
14th August 2009, 23:09
why not just put the old code back and call it a day for Blu-ray compatibility? You can always improve it later.
Exactly, I tried a few times to convince the same, in the past, but no one was interested.

Dark Shikari
14th August 2009, 23:10
Exactly, I tried a few times to convince the same, in the past, but no one was interested.You can't "put the old code back"; that would be far more difficult than just writing support anew.

plonk420
14th August 2009, 23:21
will this be on x264's Summer of Code 2010 if it's not added by then? :)

Dark Shikari
14th August 2009, 23:27
Well, at least the work is done already. You just have to "trace" it, which I believe it is much easier than writing something new."Trace" the complete overhaul of most of the frame handling code, which will be further rewritten by threaded slicetype? Are you insane?

shon3i
14th August 2009, 23:28
You can't "put the old code back"; that would be far more difficult than just writing support anew.
True, but for now I am satisfied with the fact that the 4.0 level does not require slices, because i backup my blu-rays to BD-5/9 media for now.

Lyris
15th August 2009, 00:30
OK, so now I've tested a bunch of encoders. The bitrates used for all of the tests were 35mbps average, 38mbps peak, except for X264 Level 4, which was tested at 20mbps due to the AVC Level 4 constraints. In brief:

1. X264: best quality, even at Level 4, but I can't get it to play (perhaps I have the wrong settings).

2. Mainconcept Reference: very, very, very good quality at these bitrates. It doesn't retain the high frequency grain as well as X264, but it still produces what is, in my opinion, an excellent result at the bitrates I'll be using it at. Certainly, it looks better compression-wise than just about every BD title that I've bought. I have burned discs using the trial version of Netblender DoStudio and they play back in all of the BD players I have here (Sony PS3, Panasonic BD35, Oppo BDP83, and a ridiculously expensive Denon A1). Slowest encoder of the bunch (with everything set to highest quality on 2pass): 0.7fps on a 2.8ghz Quad core.

Looking at MCR's output and judging it on its own merits, it looks fantastic. Only when you compare it to the source (and indeed to x264) do I start seeing minor flaws.

3. Netblender Dostudio Encoder: quality is not acceptable by videophile standards, but plays back fine. EDIT: turns out I'm experiencing a bug with the encoder, the poor quality is NOT the intended operation. Netblender are investigating. Fastest of the software-only encoders. Only accepts QuickTime input (d'oh... I'm very much a Windows guy. I can't live without AviSynth).

4. CodecSys CE-10. This is the one that uses a LAN-connected Playstation3 to accelerate encoding. I was getting around 6-8fps with the "Fine" mode. However, the quality is not good enough. Some frames in each GOP (presumably I frames) look excellent, sometimes even slightly better than x264, but things fall apart afterwards into compression artefacts (this is using the company's recommended BD-compliant settings). Most troublesome of all, the 4 slices in the frame are actually often VISIBLE in my test encodings! That is, you can actually see 4 clearly marked sections of the picture. No dice.

In short, I have a very good feeling about the Mainconcept Reference Encoder and hope to see its output pass verification. It's not quite up to X264's standards - but what is? - and it hopefully makes up for that in terms of compatibility with the BD spec.

Dark Shikari
15th August 2009, 00:34
OK, so now I've tested a bunch of encoders. The bitrates used for all of the tests were 35mbps average, 38mbps peak, except for X264 Level 4, which was tested at 20mbps due to the AVC Level 4 constraints. In brief:

1. X264: best quality, even at Level 4, but I can't get it to play (perhaps I have the wrong settings).Are you using the NAL-HRD patch? That's necessary for compliance.

Lyris
15th August 2009, 00:36
I am not. I'll take a look at the patches thread and try to figure that out.

Some comparisons. Images are PNGs (OK, overkill I know), and are about 5mb each.

http://lyris-lite.net/avc/squirrel.png

http://lyris-lite.net/avc/trap.png

http://lyris-lite.net/avc/overhead.png

http://lyris-lite.net/avc/fur1.png (I frame? The Codecsys encoder looks great here)...

http://lyris-lite.net/avc/fur2.png ...(but turns to mush in the next frames).

http://lyris-lite.net/avc/slice.jpg - JPEG image showing visible slices seen on the CodecSys output.

kolak
15th August 2009, 00:37
In short, I have a very good feeling about the Mainconcept Reference Encoder and hope to see its output pass verification. It's not quite up to X264's standards - but what is? - and it hopefully makes up for that in terms of compatibility with the BD spec.

Cinevision can at least match x264 or even produce better results for grainy sources, but I like quality from the latest version of x264.
Some features of Mainconcept's SDK can be found only in Cinevision and they work quite well.

Andrew

shon3i
15th August 2009, 00:40
In short, I have a very good feeling about the Mainconcept Reference EncoderToo bad you don't have access to sonic cinevision, which have advamced features of Mainconcept such as Grain optimization, several levels of AQ (including Dark AQ).

which was tested at 20mbps due to the AVC Level 4 constraints. In brief:You know, you can go up to 24mbps, and highest peak up to 30mbps. I think x264 at level 4.0 can beat everything else with 4.1 :)

Cinevision can at least match x264 or even produce better result for grainy sourcesAgree, but only with nominal blu-ray bitrates, with lover bitrates x264 still produce better looking video.

kolak
15th August 2009, 01:25
Agree, but only with nominal blu-ray bitrates, with lover bitrates x264 still produce better looking video.

Yes- I've forgotten to add this.
I'm more BD bitrates guy, so I'm not realy interested in 5Mbit HD encodes (some time ago I was).

I've done some test recently and Cinevision was slightly better, but I know it quite well, so I can tweak it. x264 has more settings and I don't have experence with it. Difference was very small- but on 2x zoom Cinevision looked more clean. This was fairly noisy 30p source, so Cinevision shined with its grain optimization.

AQ stuff in Cinevison is very powerful, but also makes problems with some sources.


Andrew

Lyris
15th August 2009, 01:34
You know, you can go up to 24mbps, and highest peak up to 30mbps. I think x264 at level 4.0 can beat everything else with 4.1
I actually tried this, but x264 gave me warnings. The mistake is probably on my end - I've been experimenting with Mainconcept more lately. I will try a patched x264.

nixo
15th August 2009, 11:58
These warnings?

x264 [warning]: VBV bitrate (24000) > level limit (20000)
x264 [warning]: VBV buffer (30000) > level limit (25000)

Main profile is used during fast first pass. I think they can be safely ignored.

--
Nikolaj

Revgen
15th August 2009, 14:40
A) Here's my understanding of blu-ray compatibility. Please correct me if I'm wrong.

1) Levels can either be 4.0 or 4.1 for HD.
2) Level 4.1 requires at least 4 slices for Blu-Ray.
3) Level 4.0 doesn't require slices.
4) Limited to 3 b-frames or less.
5) Limited to 4 ref for 1080p. Limited to 9 ref for 720p.
6) Limited to 30000kb VBV Buffer size.
7) 4.0 is limited to a max-bitrate of 24000kbps for Blu-Ray.
8) Blu-Ray doesn't handle b-pyramid well.
9) "--nal-hrd" and "--aud" options must be enabled for compatibility.
10) Max Key Intervals must be set to 24. Minimum Key Intervals must be set to either 1 or 2.


B) Here's my list of questions about blu-ray compatibility.

1) Are there limits to subme? Megui's blu-ray profile has "--subme 2" toggled.
2) Are there limits to me? Megui's blu-ray profile has "--me dia" toggled.
3) Can trellis be used? It's disabled in MeGUI's blu-ray profile. If so, then how high can the setting be?
4) Are mixed-refs allowed? Megui's blu-ray profile has them disabled.
5) Is "--mvrange 511" necessary? It's included in the MeGUI blu-ray profile.
6) Megui's blu-ray profile has "--ipratio 1.1" "--pbratio 1.1". Can these be different? If so, is there a limit?
7) MeGUI's blu-ray profile has qcomp set to "--qcomp 0.5". Is this necessary? Or can it be x264's default qcomp setting?
8) Can I disable the inloop filter?

I don't expect anybody to answer all my questions, but if some of them can be answered, then I'd appreciate it.

Chengbin
15th August 2009, 14:59
B) Here's my list of questions about blu-ray compatibility.

1) Are there limits to subme? Megui's blu-ray profile has "--subme 2" toggled.
2) Are there limits to me? Megui's blu-ray profile has "--me dia" toggled.
3) Can trellis be used? It's disabled in MeGUI's blu-ray profile. If so, then how high can the setting be?
4) Are mixed-refs allowed? Megui's blu-ray profile has them disabled.
5) Is "--mvrange 511" necessary? It's included in the MeGUI blu-ray profile.
6) Megui's blu-ray profile has "--ipratio 1.1" "--pbratio 1.1". Can these be different? If so, is there a limit?
7) MeGUI's blu-ray profile has qcomp set to "--qcomp 0.5". Is this necessary? Or can it be x264's default qcomp setting?
8) Can I disable the inloop filter?

I don't expect anybody to answer all my questions, but if some of them can be answered, then I'd appreciate it.

Subme, merange, trellis has nothing to do with decoding. Use whatever you want. MeGUI has subme 2 and me dia toggles for first pass, where you can get away with crap settings.

I read that don't touch mvrange, leave it at 511.

Mixed-refs should be allowed, but I'm not 100% sure.

ipratio and pbratio has nothing to do with decoding. Those parameters just tell x264 the bitrate allocation for each type of frame. Use whatever you like.

I believe qcomp is lowered to 0.5 because Blu-ray has a maximum bitrate of 40mbps. At Blu-ray bitrates, qcomp of 0.5 can also save some bitrate because your bitrate is already so high. You can use any qcomp you like, just the higher the qcomp, the more chances of going over the maximum bitrate, and wasting bitrate on high motion scene when they have enough bitrates.

Of course you can disable inloop deblocker. Many Blu-rays disable that because they blur too much.

Hope this helps.

Guest
15th August 2009, 15:28
Many Blu-rays disable that because they blur too much. Please provide evidence for this claim.

poisondeathray
15th August 2009, 15:30
1) Are there limits to subme? Megui's blu-ray profile has "--subme 2" toggled.
2) Are there limits to me? Megui's blu-ray profile has "--me dia" toggled.



Can you clarify: Do you mean for 1st pass settings? Because they are set differently in the Megui blu-ray profile

Revgen
15th August 2009, 15:46
Can you clarify: Do you mean for 1st pass settings? Because they are set differently in the Megui blu-ray profile

They are 2nd pass.

When I open the MEGui Blu-Ray preset, the options are shown in the box.

program --profile high --level 4.1 --preset fast --pass 2 --bitrate 8000 --stats ".stats" --thread-input --deblock -1:-1 --keyint 24 --min-keyint 2 --b-adapt 2 --direct auto --ref 1 --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 30000 --vbv-maxrate 40000 --qcomp 0.5 --no-mbtree --me dia --subme 2 --partitions none --trellis 0 --no-mixed-refs --mvrange 511 --nal-hrd --sar 1:1--output "output" "input"

Clearly 4.1 cannot be compatible without slices, so it needs to be modified. I'm just trying to figure out exactly what needs to be changed.

poisondeathray
15th August 2009, 16:17
They are 2nd pass.

When I open the MEGui Blu-Ray preset, the options are shown in the box.


We must be using different builds/presets because your commandline looks completely different (fast preset, partitions none !? )


Clearly 4.1 cannot be compatible without slices, so it needs to be modified. I'm just trying to figure out exactly what needs to be changed.


I don't think the profile needs to be modified, because it works on the majority of SAPs just fine. This thread is about replication and that's a different story...

Revgen
15th August 2009, 16:20
We must be using different builds/presets because your commandline looks completely different (fast preset, partitions none !? )



I don't think the profile needs to be modified, because it works on the majority of SAPs just fine. This thread is about replication and that's a different story...

I'm just trying to find out which settings are compatible and what the limits are. MeGUI's settings just raised my curiosity.

shon3i
15th August 2009, 16:33
obvious that people do not read posts carefully :) so i make the table to make things be easier.

http://tom.niko.users.sbb.rs/bdtable.jpg

This setings are based on Blu-Ray specification and documentation not H264.

btw.
Max bframes = 3

if you use x264 as encoder then don't use B-pyramid because can broke DBP, but B-Pyramid is generaly alowed thing (Sonic Scenarist use it but automaticly decrase ref by one)

mixed-refs is probably allowed, but some H264 developer better to answer because maybe can break DBP aslo? Mainconcept developer (Sergey) found that b-pyramid broke DBP in x264, maybe him know something about?

EDIT: About GOP thing. Blu-Ray allow both Open and Closed GOP structure, for H264.

Recommend GOP is min-max is 2-24 but everything from 1-30 can be used from both mix and max. Max value aslo depend how many b-frames used. If Bframes = 0 you can use anything from 1 to 30 for both min and max value
if bframes = 1 min also can be 1-30 but max value can be 2,4,6,8,10,12,14,16,18,20,22,24,26,28,30
if bframes = 2 min 1-30, max 3,6,9,12,15,18,21,24,27,30
if bframes = 3 min 1-30, max 4,8,12,16,20,24,28,30

minimum 2 is recommended by authorung apps, 1 is fine too

Chengbin
15th August 2009, 16:51
Please provide evidence for this claim.

I believe benwaggoner mentioned that from this thread (http://forum.doom9.org/showthread.php?t=147300&page=2)

Why would we assume 30-70% lower? Not arguing, but that seems a pretty specific assumption.

A lot of Blu-ray titles wind up having In-loop deblocking turned off for most of the title in order to eliminate the softening effect. That seems a little...aggressive to me, but that certainly would have an impact.

But anyway, I suspect you'd find that actual discs's defects are found in the IYUV masters more often than not.

But even given 20 Mbps and 70%, and using power of 0.75 as a rule of thumb, that'd be still be 14 Mbps 1080p, equivalent to 7.6 Mbps ABR 720p with x264, which should be well more than ample.

Quote:
That's part of the problem as well. They need to start using real HD digital cameras
They are, but we're still going to be watching movies shot with all kinds of cameras for decades to come.

Even I find excellent compresibility poor consolation for a bad story or poor acting .

Revgen
16th August 2009, 02:28
^I've never had problems with X264's inloop filter overblurring details at Blu-Ray or HD-DVD resolutions and bitrates as long as it's at -1:-1 or less. Even at 0:0, it's hardly noticeable. Perhaps the AVC encoders they are using have substandard inloop filter implementation.

Manao
16th August 2009, 09:05
Perhaps the AVC encoders they are using have substandard inloop filter implementation. That doesn't exist. Everybody must use the same deblocking algorithm. However, since deblocking's strength is inherently adaptive with the quantizer (the higher the quantizer the stronger), and since x264 tends to be a lot more efficient than other encoders (translate : most of the time, at the same bitrate, it'll use a quantizer lower by 1 or 2), at the same deblocking settings and same bitrates, x264 may seem to deblock less.

Revgen
16th August 2009, 09:14
That doesn't exist. Everybody must use the same deblocking algorithm. However, since deblocking's strength is inherently adaptive with the quantizer (the higher the quantizer the stronger), and since x264 tends to be a lot more efficient than other encoders (translate : most of the time, at the same bitrate, it'll use a quantizer lower by 1 or 2), at the same deblocking settings and same bitrates, x264 may seem to deblock less.

That seems like a fancy way of saying that the other AVC encoders suck. ;)

That seems to explain it then.

Dark Shikari
16th August 2009, 10:02
That doesn't exist. Everybody must use the same deblocking algorithm. However, since deblocking's strength is inherently adaptive with the quantizer (the higher the quantizer the stronger), and since x264 tends to be a lot more efficient than other encoders (translate : most of the time, at the same bitrate, it'll use a quantizer lower by 1 or 2), at the same deblocking settings and same bitrates, x264 may seem to deblock less.That seems like an odd statement; with psy, especially psy-trellis, x264 uses significantly higher quantizers than normal.

benwaggoner
16th August 2009, 21:58
Please provide evidence for this claim.
I'm not sure if there are any public references to this fact, but I can confrim that this has long been standard practice in BD production for A-list titles. At NAB, a couple different BD encoder vendors were touting their ability to turn it off as a feature.

I don't know if it's default for all the tools, or as used by all compressionists. But many studio BD titles will have big sections where in-loop is off entirely.

I also agree that this shouldn't be needed in theory, and am not sure how really needed it is in practice. Most professional optical disc compressionists don't have that much of a compression science background, and often keep using "best practices" established long ago, even if the reason for them has long since been addressed. There's plenty of exceptions to that rule. Most great Photoshop artists couldn't do a whiteboard calculation of the effect of cumulative 8-bit rounding errors. But they still end up with photos that look a lot better than mine :).

About three years ago I saw a then 10 year old Toshiba MPEG-2 encoder still being used for A-list titles!

Lyris
16th August 2009, 22:44
and often keep using "best practices" established long ago, even if the reason for them has long since been addressed.
(Cough) lowpass filter (cough) ;)

benwaggoner
16th August 2009, 23:04
(Cough) lowpass filter (cough) ;)
...which did make some sense in the early DVD era, since a progressive encode viewed on an interlaced display could have frequencies about Nyquist in 240 lines per field, resulting in some aliasing issues.

For which a vertical lowpass filter would make sense. However it seems the practice that emerged was to use a 0.3 pixel vertical blur.

Which may have been tempting because it'd make compression a decent bit easier as well.

There was quite a kerfuffle in the pre-launch days of HD DVD where we kept having to assure AVS Forum that we weren't encouraging that practice, and in fact did not include a vertical blur filter as part of PEP :).

Not that it would have been needed much even when playing back on a 1080i display, given that film sources are very rarely going to have any frequencies that high in a 1080 frame.

My bias has long been to optimize for those who have good, well-calibrated displays, since by definition they're the ones who cared about quality, and have earned it. We certainly need to make sure things don't stink on consumer displays, but the most accurate experience should be achieved by the most accureate calibration.

Lyris
17th August 2009, 00:23
I totally agree with your last paragraph - aiming high instead of the lowest common denominator is the way to go, because like you say, the people who have the equipment that creates such issues probably won't even notice or care about them anyway.

Biggiesized
17th August 2009, 02:00
I have the trailers for "The Island" and "Match Point" from Dreamworks with permission to redistribute for compression testing. I've got Lagarith 1920x1080p23.769 with 5.1 audio AVI files I could probably post somewhere.
Both would be great, but if you had to choose between the two, go with The Island.

How soon do you think you can get these up, Ben?

Chengbin
17th August 2009, 04:25
Lyris, have you tried encoding a 30-35Mbps encode using x264 to see how good it looks just for kicks?

Lyris
17th August 2009, 04:49
Yes. Many times. I wouldn't get funny looks if I called them "essentially lossless".

benwaggoner
17th August 2009, 08:11
Both would be great, but if you had to choose between the two, go with The Island.

How soon do you think you can get these up, Ben?
I'm uploading Island to my SkyDrive right now (in 50 MB chunks), and it should be done sometime tomorrow.

benwaggoner
17th August 2009, 08:21
Yes. Many times. I wouldn't get funny looks if I called them "essentially lossless".
Did you compare those to 25 Mbps peak single-slice?

My expectation is that Level 4.0 would reach quality saturation with x264 for large classes of content, particularly 1080p24. Full-frame 1080i30 would be the most likely exception, particuarly highly detailed and fast-motion stuff like sports shot with fast shutter speeds.

Before we ask the x264 devs to tackle the apparently challenging and otherwise not broadly useful work to get slicing added back in, there probably should be some clear samples of where Level 4.1 would offer a percetible quality advantage over Level 4.0 as currently supported.

Chengbin
17th August 2009, 13:37
And if x264, at level 4.0, is beating out the best commercial encoder at 35Mbps, maybe you can just use the x264 encode, since at level 4.0 it is completely compliant to the Blu-ray specs.

Revgen
17th August 2009, 14:41
Even 1920x1080i broadcast quality streams (ones actually sent from the tower before they are sent to the cable tv provider) tend to be no more than 18mbps to 22mbps. And that's for 60i material broadcast with MPEG-2. 1920x1080p at 24p with AVC or VC1 should be fine at Level 4.0.

kolak
17th August 2009, 15:02
Lyris, have you tried encoding a 30-35Mbps encode using x264 to see how good it looks just for kicks?

I've done it with uncompressed 30p source and compared to 2 pro encoders. x264 is good, very good :)


Andrew

Guest
18th August 2009, 02:42
Discussion of adaptive deblocking was moved here:

http://forum.doom9.org/showthread.php?t=148968

benwaggoner
18th August 2009, 03:27
Both would be great, but if you had to choose between the two, go with The Island.

How soon do you think you can get these up, Ben?
Give this a shot:

http://cid-bee3c9ac9541c85b.skydrive.live.com/browse.aspx/.Public/The%20Island%20Trailer

plonk420
18th August 2009, 06:12
thank you, sir! was going to offer to share it on FTP and give out the URL to those who requested it, but this works well enough!

Biggiesized
18th August 2009, 06:19
Give this a shot:

http://cid-bee3c9ac9541c85b.skydrive.live.com/browse.aspx/.Public/The%20Island%20Trailer
Thank you, Ben! You are a gentleman and a scholar.

Could you please provide a checksum as well?

I didn't know SkyDrive offered free storage. That's a handy little tool from Microsoft.

How much bandwidth are you allotted per month?

Lyris
18th August 2009, 06:27
That's some gorgeous footage, but wow, I'm surprised at what happens during the fades. I think I speak for many people in saying "thank you for giving us some studio quality footage that isn't Big Buck Bunny" :)

dvdboy
18th August 2009, 09:52
I'm trying to use x264 on some test encodes in ScenaristBD 5.1.2 using this command line:

program --profile high --pass 2 --bitrate 20000 --stats ".stats" --level 4 --keyint 24 --min-keyint 2 --ref 4 --b-adapt 2 --direct auto --deblock -1:-1 --trellis 2 --partitions p8x8,b8x8,i4x4,i8x8 --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 30000 --vbv-maxrate 24000 --qcomp 0.5 --me umh --thread-input --interlaced --output "output" "input" --mvrange 511 --nal-hrd --sar 1:1

MUI creation seems to go fine, but when I go to multiplex the project I get the following error:

2009-08-18T08:52:17Z|ERROR|MUX_SN_E_TS_UNDERFLOW_ERR|F:\x264 Test\01.00.0000\Output\MUX\BDROM\DB\BDMV\STREAM/00002.m2ts|0|Buffer underflows|TSWrapper.dll::CTSWrapper::ProcThreadMain::Video buffer underflows. - |

Apologies for such a stupid question, but is this a problem with my commandline, or does Scenarist just not like the encode for some reason?

shon3i
18th August 2009, 10:35
It is obvious that encoded stream have buffer underflows. Sinece x264 VBV sistem isn't stable and NAL HRD aslo isn't stable, you can try with new lookahead VBV or if you use maybe to try to downgrade to older x264 to see is there difference.

Aslo you miss --aud in your comand line, maybe is that catch?

dvdboy
18th August 2009, 11:05
I'll try the --aud command. THe above was created by MeGUI, which I believe reccently update it's x264 'engine'.

laserfan
18th August 2009, 14:38
Give this a shot:

http://cid-bee3c9ac9541c85b.skydrive.live.com/browse.aspx/.Public/The%20Island%20TrailerCan someone please post what the total download size (approx)? Is it a couple Gb? I need to know before I attempt to DL; my ISP limits my downloads... :(

Chengbin
18th August 2009, 14:42
Can someone please post what the total download size (approx)? Is it a couple Gb? I need to know before I attempt to DL; my ISP limits my downloads... :(

45x50mb=2.25GB

J_Darnley
18th August 2009, 14:43
I guess it is 2217MB, based on the fact the first piece is 50MB, and the last being 17MB and there being 45 in total.

laserfan
18th August 2009, 14:47
Thanks. I remembered that bw had said "50Mb chunks" but wanted to confirm.

kieranrk
18th August 2009, 15:16
Is there any way of downloading that programmatically?

Biggiesized
18th August 2009, 20:13
Is there any way of downloading that programmatically?
I don't think so. The URLs look to be randomly server generated and not batch compatible since they don't follow a sequence.

kieranrk
18th August 2009, 20:33
Never mind. I downloaded them one by one.

nm
18th August 2009, 20:34
I don't think so. The URLs look to be randomly server generated and not batch compatible since they don't follow a sequence.
wget can handle such situations but it probably requires a bit of trial-and-error to do it without mirroring the whole site :)

Biggiesized
18th August 2009, 23:12
Wow I thought I had trouble opening the split archive until I calculated a checksum on all the files only to learn that I forgot to download one part! D'oh!

EDIT: For anybody's reference, the checksum (I now have all 45 parts) that was calculated for me was:

http://img8.imageshack.us/img8/8049/checksum.png

Lyris
19th August 2009, 01:04
Wow I thought I had trouble opening the split archive until I calculated a checksum on all the files only to learn that I forgot to download one part! D'oh!
I did the exact same thing!

Interesting that there's so much banding during fades. Also there's a great shot where Ewan McGregor's eye has clearly had VFX work done to the side of it, but they've forgotten to add grain to conceal the compositing. All the things you begin to notice when you stare at video for too long :)

BTW, I've also been trying out the Elecard AVC encoder with this trailer. It has options for writing HRD info into the stream, and also support for multiple slices, so in theory it should be BDMV compliant. Results are good, but not at the level of Mainconcept Reference - or indeed, x264.

Chengbin
19th August 2009, 01:07
Interesting that there's so much banding during fades.

Are you using mbtree? mbtree is known to mess up on fades. weight-p will correct that, and it is coming in a few weeks.

woah!
19th August 2009, 01:11
Are you using mbtree? mbtree is known to mess up on fades. weight-p will correct that, and it is coming in a few weeks.

he means the source has terrible fades in it already....

tyee
19th August 2009, 01:38
I can't seem to download the trailer anymore. It was working this morning but not now. Same thing for everyone else?

CruNcher
19th August 2009, 02:03
Thx to Ben and DreamWorks for sharing this Source nice task :)

@tyee
Still works here

Audionut
19th August 2009, 03:13
I can't get the archives open. I've tried winrar, 7-zip, renaming the extensions.

I can select all the files and extract with 7-zip, but it only creats a 2.16GB .7z file that itself can't be extracted.

Any ideas what i'm doing wrong?

Lyris
19th August 2009, 03:15
I had the same problem at first - be absolutely sure that you have all 45 parts.

Audionut
19th August 2009, 03:16
you have all 45 parts.

No problem there.

Chengbin
19th August 2009, 03:57
Just to keep this thread on topic:

Lyris, so what do you plan to use? x264 at level 4.0, or Mainconcept Reference?

woah!
19th August 2009, 05:22
i am finding 15mbps with L4.0 bluray settings is near on perfect with that trailer source.... i forget how good x264 is at times...

heres a few compare screens if anyones interested...


http://tinyurl.com/njy77a

tyee
19th August 2009, 06:06
Still can't download, all I get is this url

http://yyhpsq.blu.livefilestore.com/y1po0jl8826qLY4azx-WbrsbDeznY-Yw3PAuSsBAKsMNt2dz4_g6bbr5cRAgqxGlGT3zy50y465uoNzalB6BbgiGQ/Island_1080p24_lag_51.7z.021?download

with a blank page looking at me. Both in FF and IE. Strange. Even creating a windows live ID and sign in, still the same!

imcold
19th August 2009, 07:58
I can't get the archives open. I've tried winrar, 7-zip, renaming the extensions.

You have to combine the files into one first, then decode.

Lyris
19th August 2009, 08:21
Just to keep this thread on topic:

Lyris, so what do you plan to use? x264 at level 4.0, or Mainconcept Reference?Whichever is proved to verify successfully. Hopefully x264.

multimediaman
19th August 2009, 08:28
Extracted using CLI
7z e Island_1080p24_lag_51.7z.001
No problem at all

Quite bad banding in the source
http://img149.imageshack.us/i/shotl.png/

It was a bit tricky to get lagarith working on Linux but wine wine, avisynth, av2yuv.exe solved the problem.

CruNcher
19th August 2009, 10:54
Thats the little drawback as Ben said its 8 bit not a 10 bit DPX Source :(

yau
19th August 2009, 12:06
It was a bit tricky to get lagarith working on Linux but wine wine, avisynth, av2yuv.exe solved the problem.

It seems that Lagarith decoding is coming to FFmpeg, and therefore also to MPlayer. It already works if you apply these patches (http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2009-August/074177.html) (minus the FFmpeg stuff) and add this to your MPlayer codec.conf:


videocodec fflagarith
info "FFmpeg Lagarith"
status working
fourcc LAGS
driver ffmpeg
dll lagarith
out YV12


MPlayer however seems to have problems with the audio; I extracted it using -dumpaudio and converted it using Audacity (the format seems to be PCM 24 bit, signed, little endian, 48000 Hz, 6 channels).

Biggiesized
20th August 2009, 00:40
No problem there.
Do a checksum then. I already posted the correct checksum.

Highlight all 45 parts and then select the checksum option in 7-zip.

Audionut
20th August 2009, 01:13
Yeah my checksum doesn't match. Next problem is that I am getting the same problem as tyee.

Just a blank page after trying to download again.

benwaggoner
20th August 2009, 06:31
The page is opening fine for me.

Someone who has been able to download it is welcome to reencode and post in a better way than I did. I was on my way to the airport, and did it the quickest way I thought of.

Anyway, I'm looking foward to some BD-compliant samples with it! I'll try to get the Match Point source available soon.

I also have 4:2:2 10-bit source from 35mm telecine for that "Lady Washington" shoot which I've been meaning to finish for a couple of years now. That'd be a good 6 minutes long. Hard to imagine when I'd have a free week to do a good job on it though; certainly not before October with IBC coming up.

dvdboy
20th August 2009, 09:26
It is obvious that encoded stream have buffer underflows. Sinece x264 VBV sistem isn't stable and NAL HRD aslo isn't stable, you can try with new lookahead VBV or if you use maybe to try to downgrade to older x264 to see is there difference.

Aslo you miss --aud in your comand line, maybe is that catch?

Still having issues. My command line looks like this:

program --profile high --pass 2 --bitrate 20000 --stats ".stats" --level 4 --keyint 24 --min-keyint 2 --ref 4 --b-adapt 2 --direct auto --deblock -1:-1 --trellis 2 --partitions p8x8,b8x8,i4x4,i8x8 --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 30000 --vbv-maxrate 24000 --qcomp 0.5 --me umh --thread-input --interlaced --output "output" "input" --mvrange 511 --nal-hrd --aud --sar 1:1

I'm using MeGUI as a front end.

2009-08-20T08:14:34Z|ERROR|MUX_SN_E_TS_UNDERFLOW_ERR|F:\x264 Test\01.00.0000\Output\MUX\BDROM\DB\BDMV\STREAM/00002.m2ts|0|Buffer underflows|TSWrapper.dll::CTSWrapper::ProcThreadMain::Video buffer underflows. - |

I'm running build 1183, which I believe is the most recent from this topic:

http://forum.doom9.org/showthread.php?t=89979

What is everyone else running to get files that mux?

I'll try a CBR encode and see if that works.

:thanks:

nm
20th August 2009, 11:09
program --profile high --pass 2 --bitrate 20000 --stats ".stats" --level 4 --keyint 24 --min-keyint 2 --ref 4 --b-adapt 2 --direct auto --deblock -1:-1 --trellis 2 --partitions p8x8,b8x8,i4x4,i8x8 --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 30000 --vbv-maxrate 24000 --qcomp 0.5 --me umh --thread-input --interlaced --output "output" "input" --mvrange 511 --nal-hrd --aud --sar 1:1
Perhaps the same parameters without --interlaced would work better, so try on a progressive video?

I'm running build 1183, which I believe is the most recent from this topic:
r1217 is the latest. Patched builds are announced in this thread: http://forum.doom9.org/showthread.php?t=130364

Lyris
20th August 2009, 19:43
Hey everyone,

Turns out that my dissatisfaction with the Netblender Dostudio Encoder is due to an error. Support contacted me directly and are looking at it. In other words, the results I was getting definitely weren't deemed "good enough" by Netblender. I look forward to reevaluating it.

Audionut
20th August 2009, 22:59
The page is opening fine for me.

Same here. It was only when clicking the download link that the problem arose. Its working for me now though and it turns out I must have had a corrupt part. I got it extracted.

Thanks Ben.

bautschi
21st August 2009, 01:11
Concerning the underflows with ScenaristBD 5.1.2, I don't have that software. My question therefore is, are there possibilities to check encodings for such errors? Does x264 tell me while encoding? mkvmerge?
For me it also raises the problem of what can I do to check if my encodings have some kind of errors that I have never even heard of.

I was thinking of using x264 for my regular dvb-s recordings (PAL 576i) and with 720x576 I need interlaced accoring to wikipedia, as 576p is not supported.

After researching vbv underflows ( I found some good explanation for mpeg2, but the principle hopefully apllies ) I tried some settings:

all 2 pass:
--vbv-maxrate 15000 --vbv-init 0.999 --vbv-bufsize 1000 --bitrate 15000
x264 [warning]: VBV buffer size too small, using 1800 kbit and
encoder/set.c:614: x264_sei_buffering_period_write: Assertion `( buffer_fill_final * 90000. / ( buffer_rate ) ) < pow( 2,
sps->vui.nal_hrd_parameters.i_initial_cpb_removal_delay_length )' failed.
Aborted

--vbv-maxrate 15000 --vbv-init 0.999 --vbv-bufsize 1000 --bitrate 5000 --- same

--vbv-maxrate 15000 --vbv-init 0.999 --vbv-bufsize 1500 --bitrate 5000 --- x264 [warning]: VBV buffer size too small, using 1800 kbit
encoded 368 frames, 7.09 fps, 4985.78 kb/s

As I had --min-keyint 24 --keyint 24, how can almost a second with average of 4985.78 kb/s fit with vbv-bufsize 1800? Does the decoder not need at least information from one I-frame to the next ( and even more, when a frame uses a reference from outside of the gop) ? That should be be more than in the buffer... .

As I most obviously don't understand something here, my main concern is how to check for problems. Knowing that there is a problem is better than being blind.

Mkvmerge gave no errors when merging with ac3 sound and and vlc played everything all right (Linux).

I was using x264-snapshot-20090819-2245 with x264_hrd_pd_interlace.16.diff.

complete cl:

ffmpeg -top -1 -i $infile -f rawvideo - 2>x264-ffmpeg.log | x264 $options --pass 1 --bitrate $bitrate -o out.mkv - 720x576
ffmpeg -top -1 -i $infile -f rawvideo - 2>x264-ffmpeg.log | x264 $options --pass 2 --bitrate $bitrate -o out.mkv - 720x576

$options: --profile high --tune film --slow-firstpass --level 4 --aud --min-keyint 24 --keyint 24 --bframes 3 --b-adapt 2 --ref 9 --deblock -1:-1 --subme 10 --interlaced --vbv-maxrate 15000 --vbv-init 0.999 --vbv-bufsize 1500

Hope this was not too much info ... my first action in a forum in years... .

Greetings,
bautschi

Dark Shikari
21st August 2009, 01:14
x264 will never violate the VBV restrictions you set without telling you. If it doesn't tell you, it didn't violate them.

bautschi
21st August 2009, 01:19
thank you.

Guest
21st August 2009, 01:22
x264 will never violate the VBV restrictions you set without telling you. If it doesn't tell you, it didn't violate them. Unless you care about little things like AUDs, etc., that you don't allow for.

Dark Shikari
21st August 2009, 01:26
Unless you care about little things like AUDs, etc., that you don't allow for.AUDs aren't part of the video coding layer (VCL) as far as I recall, which is what x264's VBV covers. It is not NAL VBV.

(Speaking of which, I still don't understand why Trahald is trying to patch x264 with a NAL-HRD patch despite the VBV being VCL...)

Guest
21st August 2009, 02:07
In real decoders the NALU stream is buffered, so while what you say is technically correct, there is the possibility to underflow on a real system even if the VCL VBV is fine. Am I wrong about that?

Dark Shikari
21st August 2009, 02:15
In real decoders the NALU stream is buffered, so while what you say is technically correct, there is the possibility to underflow on a real system even if the VCL VBV is fine. Am I wrong about that?Of course--any system that requires NAL HRD clearly requires NAL VBV for exactly the same reason, so only one isn't going to cut it.

Trahald
21st August 2009, 02:34
the nal hrd patch adds aud sei etc to the buffer calculation (well if i coded it right it does, and thats not a certainty.)

Trahald
21st August 2009, 02:37
btw.. id be more than happy to shift-del the hrd patch if someone in dev codes up something acceptable. until then, broken or not, its all we have.

Dark Shikari
21st August 2009, 02:39
the nal hrd patch adds aud sei etc to the buffer calculation (well if i coded it right it does, and thats not a certainty.)btw.. id be more than happy to shift-del the hrd patch if someone in dev codes up something acceptable. until then, broken or not, its all we have.Delete it? Are you crazy? How about commit it?

If you can show that it properly converts x264's VBV model to NAL-based and writes correct HRD information, there's no reason not to commit it.

Also, get back on IRC, you have stuff to do ;)

Jumpyshoes
22nd August 2009, 02:55
http://shimapan.users.sourceforge.net/island/

Here's a mirror.

Revgen
22nd August 2009, 08:28
http://shimapan.users.sourceforge.net/island/

Here's a mirror.

The avi file stopped downloading at 1.0gb. Does sourceforge have a download limit for single files?

Revgen
22nd August 2009, 23:08
Ben,

Are you absolutely sure this is original footage?

Around the 1210 to 1218 frames, Ewan MacGregor has a "blur block" on the left side of his face.

http://img43.imageshack.us/img43/8996/islandblotchpng001217.th.png (http://img43.imageshack.us/i/islandblotchpng001217.png/)

The above image is frame 1217.

DeeGee
22nd August 2009, 23:36
That doesn't look like compression artifact. Looks like they blurred something intentionally from his face?

Lyris
23rd August 2009, 03:19
I speculated on this a few pages back, it looks like compositing work (perhaps adding the freckle?) that hasn't been grain-matched. In any case, there's no reason to doubt the authenticity of the source.

Bigmango
23rd August 2009, 03:36
if you use x264 as encoder then don't use B-pyramid because can broke DBP, but B-Pyramid is generaly alowed thing (Sonic Scenarist use it but automaticly decrase ref by one)

I have read in several places and wikis around the web that while this was a problem with older x264 versions, it is now fixed in the newer versions - it is now constant based on ref.

So, what is the truth behind this? Does B-pyramid still break some player compatibility or not?

Chengbin
23rd August 2009, 03:53
I have read in several places and wikis around the web that while this was a problem with older x264 versions, it is now fixed in the newer versions - it is now constant based on ref.

So, what is the truth behind this? Does B-pyramid still break some player compatibility or not?

4 ref frames=good
3 ref frames + b-pyramid=good
4 ref frames + b-pyramid=not good.

Bigmango
23rd August 2009, 03:58
4 ref frames=good
3 ref frames + b-pyramid=good
4 ref frames + b-pyramid=not good.

Ok, thanks.

What is the better suggestion quality wise: "4 ref frames" or "3 ref frames + b-pyramid" ?

popper
23rd August 2009, 05:31
what is the point of using more than ref=3 today anyway ?, is there really any quality to be gained in your generic SD (PIP) or HD live action footage.

shon3i
23rd August 2009, 11:06
4 ref frames=good
3 ref frames + b-pyramid=good
4 ref frames + b-pyramid=not good.
Sorry but b-pyramid in x264 are not good at all, because almost always break DBP. 4 ref frames + b-pyramid is impossible combination, while 3 ref frames + b-pyramid should be fine but with normal b-pyramid which x264 dosen't have. 4ref's can be aslo unsafe in x264 only because can aslo break DBP (from earlier testing) So 3refs are only safe now with current x264 builds, until devs fix things.

So, what is the truth behind this? Does B-pyramid still break some player compatibility or not? Yea, still is broken even in new version.

CruNcher
23rd August 2009, 13:02
what is the point of using more than ref=3 today anyway ?, is there really any quality to be gained in your generic SD (PIP) or HD live action footage.

mostly not but people generally believe that higher numbers in OPSNR or SSIM have a big major effect on their viewing experience and the more the better ;)
Creating mass of content that not all of the current Device Generation is able to playback, though essentially it's not a bad thing it brings manufactures under pressure to deliver the performance from their Hardware user expect to play their "Home Made" Content on :)
In the Standalone area we already hit that mostly next will be the low power Mobile area which still struggles with what the consumers expect (and imho not all the complexity user put in their encodes is really always necessary but they are used to it already and the industry needs to comply). It will be really interesting to see how this might could push again lower complexity (Old Generation Codecs) once more as their complexity is pretty much playable on every Device that's gonna come out so the need of trans-coding is virtually gone for those, H.264 didn't reach that yet and most probably will take another Year end of 2010 till that goal will be reached.

So you can say we reached a Transcoding free consumer world for ASP and VC-1 now (but not everywhere used like in Sat Broadcast), though H.264 will take another year in that :)

dvdboy
24th August 2009, 22:32
Unfortunately, I'm still not having much luck - now using a 23.976fps file to rule out possible interlace compatibilities.

My MeGui configuration line looks like this:

program --profile high --level 4 --preset fast --pass 2 --bitrate 18000 --stats ".stats" --thread-input --deblock -1:-1 --keyint 24 --min-keyint 2 --b-adapt 2 --direct auto --ref 1 --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 30000 --vbv-maxrate 23000 --qcomp 0.5 --me dia --subme 2 --partitions none --trellis 0 --no-mixed-refs --mvrange 511 --nal-hrd --aud --sar 1:1 --output "output" "input"

I downloaded jeeb's build 1222 and placed the x264.exe file into the tools directory of MeGUI, overwriting the existing x264.exe.

I'm assuming I'm missing something fundamentally basic here - I even knocked 1Mbps off the Max Bitrate hoping to give an extra room for error, but I'm still getting the same Buffer Underflow error in Scenarist.

Any suggestions?

:thanks:

shon3i
24th August 2009, 23:32
You can try to reduce vbv-buffer to 29000 or even to 24000 or less. Aslo can you cut 50 mb sample and post here to see where is problem. Aslo you can use tools like Elecard Buffer Analyser to see where underflow occur.

kemuri-_9
25th August 2009, 00:58
since mb-tree and vbv-lookahead are used in that command line,
the problem could likely be related to this (http://forum.doom9.org/showthread.php?p=1317951#post1317951)

Dark Shikari
25th August 2009, 01:00
since mb-tree and vbv-lookahead are used in that command line,
the problem could likely be related to this (http://forum.doom9.org/showthread.php?p=1317951#post1317951)It's not related unless x264 prints a "VBV buffer underflow" message.

benwaggoner
25th August 2009, 07:52
...and as promised for testing this scenario, here's 7zip Lagarith YV12 segments of Match Point.

A rather more typcial trailer than the insanely fast cutting, high grain, and constantly moving camera of "The Island." But it's chock full of fades.

http://cid-bee3c9ac9541c85b.skydrive.live.com/browse.aspx/.Public/Match%20Point

Audionut
25th August 2009, 11:26
Can someone please provide a checksum.
I'm having problems extracting again.

LoRd_MuldeR
25th August 2009, 13:30
...and as promised for testing this scenario, here's 7zip Lagarith YV12 segments of Match Point.

A rather more typcial trailer than the insanely fast cutting, high grain, and constantly moving camera of "The Island." But it's chock full of fades.

http://cid-bee3c9ac9541c85b.skydrive.live.com/browse.aspx/.Public/Match%20Point

Is there an automated way to download all those files? Downloading 38 files by hand isn't fun ;)

Unfortunately the overview page that shows all files doesn't exhibit proper download URL's, so DTA and friends can't help here...

Can't you ask Xiph to put it on their http://media.xiph.org/ (http://media.xiph.org/video/derf/) server or at least put it on a service like Rapidshare ???

Lyris
25th August 2009, 17:15
Can anyone here share the secrets for encoding compliant Secondary Video streams?

All I have so far is that the frame rate has to match the primary video, and since we're using AVC, the secondary video must be AVC also. And the max bit rate can't go above 8mbps, but this is talking heads stuff in SD resolution, so 2mbps should do it fine...

Can anyone fill me in on the VBV buffer size? Number of B frames? Keyframe interval?

dvdboy
25th August 2009, 19:32
OK, I'm in the process of uploading two clips to my skydrive account.

1 is The Island trailer that Ben posted, encoded with x264 build 1222. This muxes well in Scenarist 5.1.3.
Download x264 encode her --> http://cid-4574e83941533037.skydrive.live.com/browse.aspx/.Public/The%20Island%20-%20x264%20-%20BD

2 is a short clip from a German H.264 broadcast that I have parsed into AVISynth using DGAVCIndex, created a Lagarith YV12 avi file with VirtuaDub and the parsed that into MeGUI using an AVISynth script. This does not mux in Scenarist 5.1.3, giving me a buffer underflow error. This has been the file I have been trying to encode with at all manner of settings.

At this time I can't understand why one AVS source file should work, and another should not - both encode without errors.

The MeGUI config line for The Island is:

program --profile high --level 4 --preset fast --pass 2 --bitrate 18000 --stats ".stats" --slow-firstpass --thread-input --deblock -1:-1 --keyint 24 --min-keyint 2 --b-adapt 2 --direct auto --ref 1 --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 30000 --vbv-maxrate 24000 --qcomp 0.5 --subme 2 --partitions all --trellis 0 --no-mixed-refs --mvrange 511 --nal-hrd --aud --sar 1:1 --output "output" "input"

The MeGUI config line for AVC_TEST is:

program --profile high --level 4 --preset fast --pass 2 --bitrate 18000 --stats ".stats" --slow-firstpass --thread-input --deblock -1:-1 --keyint 24 --min-keyint 2 --b-adapt 2 --direct auto --ref 1 --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 30000 --vbv-maxrate 24000 --qcomp 0.5 --subme 2 --partitions all --trellis 0 --no-mixed-refs --mvrange 511 --nal-hrd --aud --sar 1:1 --output "output" "input"

In the end, it didn't matter if I dropped the max buffer size or max bitrates as the above encode for the Island worked fine, but nothing worked for the other file.

I'm assuming that for an x264 setting, the above is pretty 'basic' as I'm seeing compression artifacts over the green opening slate - I'll try automated 3 pass to see if that makes any difference, otherwise what do people suggest? Stoopid n00b question, but whenever I've done a 3 pass with MeGui before it always asks me to overwrite the 2nd pass with the 3rd pass - is this correct? Is there a way to out the 2 passes seperately? Would I ever need to do that?

Many Thanks

DVD-BOY

benwaggoner
25th August 2009, 19:58
Is there an automated way to download all those files? Downloading 38 files by hand isn't fun ;)
Sorry. I did it the way that was easiest for me, not for you :).

Can't you ask Xiph to put it on their http://media.xiph.org/ (http://media.xiph.org/video/derf/) server or at least put it on a service like Rapidshare ???
I think it's fine to post it where you like. Dreamworks provided it under terms where any encoded version of it can be freely used for compression tests and other demos. I just can't share the actual DPX files out.

benwaggoner
25th August 2009, 20:00
I'm assuming that for an x264 setting, the above is pretty 'basic' as I'm seeing compression artifacts over the green opening slate - I'll try automated 3 pass to see if that makes any difference, otherwise what do people suggest?
Oh, and as for that, the slate is pretty noisy, and in a very codec-annoying way. I thought about rendering out a clean new one, but heck, this is for codec testing :).

LoRd_MuldeR
25th August 2009, 20:19
I think it's fine to post it where you like. Dreamworks provided it under terms where any encoded version of it can be freely used for compression tests and other demos. I just can't share the actual DPX files out.

I was hoping that you could upload it to a better place. Preferably a place where it can be downloaded in one piece, so I could run the download over night :p

With my 128 kbit/s connection (upstream) it would take far too long to upload a file of that size :(

BTW: I downloaded the first three parts - just for test. And both, 7-Zip and WinRAR, refuse to open the archive with "unspecified error".

At least with RAR you can open multi-volume archives, even if you don't have all volumes (yet). So is anything wrong with my download or is that normal?

multimediaman
25th August 2009, 20:51
With ffv1+flac in mkv that trailer size shrinks to 1.7 GB ;)

benwaggoner
25th August 2009, 22:37
With ffv1+flac in mkv that trailer size shrinks to 1.7 GB ;)
Feel free :).

shon3i
25th August 2009, 22:52
@dvdboy, did x264 show VBV underflow during encode? since you use MeGUI for encoding check the log

dvdboy
25th August 2009, 23:52
@dvdboy, did x264 show VBV underflow during encode? since you use MeGUI for encoding check the log

I was ready to report back that it hadn't, when I noticed this:

[Information] Log
-[Information] Versions
--[NoImage] MeGUI Version : 0.3.1.1053
--[NoImage] OS : Windows XP Professional x86 SP2 (5.1.131072.2600)
--[NoImage] Latest .Net Framework installed
-[Information] Hardware
--[NoImage] CPU : Intel(R) Core(TM)2 CPU 6600 @ 2.40GHz
-[Information] Log for job18 (video, SOURCE_lag_YV12.avs -> )
--[Information] [25/08/2009 23:28:57] Started handling job
--[Information] [25/08/2009 23:28:57] Preprocessing
--[NoImage] Job commandline: "C:\Program Files\megui\tools\x264\x264.exe" --profile high --level 4 --preset fast --pass 1 --bitrate 18000 --stats "F:\WORKING_FILES\LAGARITH_SOURCE\small_test.stats" --thread-input --deblock -1:-1 --keyint 24 --min-keyint 2 --b-adapt 2 --direct auto --ref 1 --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 30000 --vbv-maxrate 24000 --qcomp 0.5 --me dia --subme 2 --partitions none --trellis 0 --no-mixed-refs --mvrange 511 --nal-hrd --aud --sar 1:1 --output NUL "F:\WORKING_FILES\LAGARITH_SOURCE\SOURCE_lag_YV12.avs"
--[Information] [25/08/2009 23:28:58] Encoding started
--[NoImage] Standard output stream
--[NoImage] Standard error stream
---[NoImage] avis [info]: 1920x1080 @ 23.98 fps (193 frames)
---[NoImage] x264 [info]: using SAR=1/1
---[NoImage] x264 [warning]: VBV bitrate (24000) > level limit (20000)
---[NoImage] x264 [warning]: VBV buffer (30000) > level limit (25000)
---[NoImage] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
---[NoImage] x264 [info]: profile Main, level 4.0
---[NoImage]
---[NoImage] x264 [info]: slice I:11 Avg QP:13.67 size:138864
---[NoImage] x264 [info]: slice P:58 Avg QP:15.22 size:113580
---[NoImage] x264 [info]: slice B:124 Avg QP:16.75 size: 73611
---[NoImage] x264 [info]: consecutive B-frames: 2.7% 6.6% 37.9% 52.7%
---[NoImage] x264 [info]: mb I I16..4: 65.8% 0.0% 34.2%
---[NoImage] x264 [info]: mb P I16..4: 53.1% 0.0% 0.0% P16..4: 29.0% 0.0% 0.0% 0.0% 0.0% skip:17.9%
---[NoImage] x264 [info]: mb B I16..4: 19.6% 0.0% 0.0% B16..8: 28.0% 0.0% 0.0% direct:26.4% skip:26.0% L0:32.8% L1:22.1% BI:45.1%
---[NoImage] x264 [info]: direct mvs spatial:97.6% temporal:2.4%
---[NoImage] x264 [info]: coded y,uvDC,uvAC intra:75.4% 91.1% 79.5% inter:36.9% 42.8% 9.4%
---[NoImage] x264 [info]: kb/s:17136.4
---[NoImage] encoded 193 frames, 5.90 fps, 17136.50 kb/s
--[Information] [25/08/2009 23:29:31] Postprocessing
---[Information] Deleting intermediate files
--[Information] [25/08/2009 23:29:31] Job completed
-[Information] Log for job19 (video, SOURCE_lag_YV12.avs -> small_test.264)
--[Information] [25/08/2009 23:29:31] Started handling job
--[Information] [25/08/2009 23:29:31] Preprocessing
--[NoImage] Job commandline: "C:\Program Files\megui\tools\x264\x264.exe" --profile high --level 4 --preset fast --pass 3 --bitrate 18000 --stats "F:\WORKING_FILES\LAGARITH_SOURCE\small_test.stats" --thread-input --deblock -1:-1 --keyint 24 --min-keyint 2 --b-adapt 2 --direct auto --ref 1 --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 30000 --vbv-maxrate 24000 --qcomp 0.5 --subme 2 --partitions all --trellis 0 --no-mixed-refs --mvrange 511 --nal-hrd --aud --sar 1:1 --aud --output "F:\WORKING_FILES\LAGARITH_SOURCE\small_test.264" "F:\WORKING_FILES\LAGARITH_SOURCE\SOURCE_lag_YV12.avs"
--[Information] [25/08/2009 23:29:31] Encoding started
--[NoImage] Standard output stream
--[NoImage] Standard error stream
---[NoImage] avis [info]: 1920x1080 @ 23.98 fps (193 frames)
---[NoImage] x264 [info]: using SAR=1/1
---[NoImage] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
---[NoImage] x264 [info]: profile High, level 4.0
---[NoImage]
---[NoImage] x264 [info]: slice I:11 Avg QP:14.06 size:132101
---[NoImage] x264 [info]: slice P:58 Avg QP:15.03 size:126728
---[NoImage] x264 [info]: slice B:124 Avg QP:16.61 size: 76492
---[NoImage] x264 [info]: consecutive B-frames: 2.7% 6.6% 37.9% 52.7%
---[NoImage] x264 [info]: mb I I16..4: 56.3% 19.6% 24.1%
---[NoImage] x264 [info]: mb P I16..4: 29.3% 24.7% 7.7% P16..4: 12.2% 4.3% 2.9% 0.6% 0.5% skip:17.6%
---[NoImage] x264 [info]: mb B I16..4: 19.3% 0.0% 0.0% B16..8: 27.1% 2.6% 2.1% direct:23.4% skip:25.6% L0:31.7% L1:21.6% BI:46.7%
---[NoImage] x264 [info]: 8x8 transform intra:23.3% inter:14.4%
---[NoImage] x264 [info]: direct mvs spatial:85.5% temporal:14.5%
---[NoImage] x264 [info]: coded y,uvDC,uvAC intra:76.5% 91.5% 79.3% inter:35.5% 42.8% 9.8%
---[NoImage] x264 [info]: kb/s:18175.5
---[NoImage] encoded 193 frames, 8.41 fps, 18175.54 kb/s
--[Information] Final statistics
---[NoImage] Video Bitrate Desired: 18000 kbit/s
---[NoImage] Video Bitrate Obtained (approximate): 18175 kbit/s
--[Information] [25/08/2009 23:29:54] Postprocessing
---[Information] Deleting intermediate files
--[Information] [25/08/2009 23:29:54] Job completed
-[Information] Log for job20 (video, SOURCE_lag_YV12.avs -> small_test.264)
--[Information] [25/08/2009 23:29:54] Started handling job
--[Information] [25/08/2009 23:29:58] Preprocessing
--[NoImage] Job commandline: "C:\Program Files\megui\tools\x264\x264.exe" --profile high --level 4 --preset fast --pass 3 --bitrate 18000 --stats "F:\WORKING_FILES\LAGARITH_SOURCE\small_test.stats" --thread-input --deblock -1:-1 --keyint 24 --min-keyint 2 --b-adapt 2 --direct auto --ref 1 --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 30000 --vbv-maxrate 24000 --qcomp 0.5 --subme 2 --partitions all --trellis 0 --no-mixed-refs --mvrange 511 --nal-hrd --aud --sar 1:1 --aud --output "F:\WORKING_FILES\LAGARITH_SOURCE\small_test.264" "F:\WORKING_FILES\LAGARITH_SOURCE\SOURCE_lag_YV12.avs"
--[Information] [25/08/2009 23:29:58] Encoding started
--[NoImage] Standard output stream
--[NoImage] Standard error stream
---[NoImage] avis [info]: 1920x1080 @ 23.98 fps (193 frames)
---[NoImage] x264 [info]: using SAR=1/1
---[NoImage] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
---[NoImage] x264 [info]: profile High, level 4.0
---[NoImage]
---[NoImage] x264 [info]: slice I:11 Avg QP:14.07 size:131778
---[NoImage] x264 [info]: slice P:58 Avg QP:15.03 size:126953
---[NoImage] x264 [info]: slice B:124 Avg QP:16.58 size: 76980
---[NoImage] x264 [info]: consecutive B-frames: 2.7% 6.6% 37.9% 52.7%
---[NoImage] x264 [info]: mb I I16..4: 56.5% 20.0% 23.5%
---[NoImage] x264 [info]: mb P I16..4: 29.4% 24.9% 7.4% P16..4: 12.2% 4.3% 3.0% 0.6% 0.5% skip:17.7%
---[NoImage] x264 [info]: mb B I16..4: 19.4% 0.0% 0.0% B16..8: 27.2% 2.5% 2.0% direct:23.4% skip:25.5% L0:32.0% L1:21.3% BI:46.7%
---[NoImage] x264 [info]: 8x8 transform intra:23.5% inter:14.4%
---[NoImage] x264 [info]: direct mvs spatial:84.7% temporal:15.3%
---[NoImage] x264 [info]: coded y,uvDC,uvAC intra:76.7% 91.6% 79.5% inter:35.7% 43.0% 9.9%
---[NoImage] x264 [info]: kb/s:18245.0
---[NoImage] encoded 193 frames, 8.25 fps, 18245.07 kb/s
--[Information] Final statistics
---[NoImage] Video Bitrate Desired: 18000 kbit/s
---[NoImage] Video Bitrate Obtained (approximate): 18245 kbit/s
--[Information] [25/08/2009 23:30:21] Postprocessing
---[Information] Deleting intermediate files
----[Information] [25/08/2009 23:30:21] Successfully deleted F:\WORKING_FILES\LAGARITH_SOURCE\small_test.stats
--[Information] [25/08/2009 23:30:21] Job completed


So does x264 believe the buffer / bitrate limits to be different to what was posted earlier in this thread, or did I misread your earlier post:

http://forum.doom9.org/showpost.php?p=1315154&postcount=106

...tbc

dvdboy
25th August 2009, 23:53
Re-ran the test with lower buffer / bitrate limits:

[Information] Log
-[Information] Versions
--[NoImage] MeGUI Version : 0.3.1.1053
--[NoImage] OS : Windows XP Professional x86 SP2 (5.1.131072.2600)
--[NoImage] Latest .Net Framework installed
-[Information] Hardware
--[NoImage] CPU : Intel(R) Core(TM)2 CPU 6600 @ 2.40GHz
-[Information] Log for job21 (avs, LAGARITH_YV12.avs -> )
--[Information] [25/08/2009 23:40:37] Started handling job
--[Information] [25/08/2009 23:40:38] Preprocessing
--[Information] [25/08/2009 23:40:38] Encoding started
--[Information] [25/08/2009 23:40:52] Postprocessing
---[Information] Deleting intermediate files
--[Information] [25/08/2009 23:40:52] Job completed
-[Information] Log for job22 (video, LAGARITH_YV12.avs -> )
--[Information] [25/08/2009 23:40:52] Started handling job
--[Information] [25/08/2009 23:40:52] Preprocessing
--[NoImage] Job commandline: "C:\Program Files\megui\tools\x264\x264.exe" --profile high --level 4 --preset fast --pass 1 --bitrate 18000 --stats "F:\WORKING_FILES\LAGARITH_SOURCE\01_small_test.stats" --thread-input --deblock -1:-1 --keyint 24 --min-keyint 2 --b-adapt 2 --direct auto --ref 1 --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 25000 --vbv-maxrate 20000 --qcomp 0.5 --me dia --subme 2 --partitions none --trellis 0 --no-mixed-refs --mvrange 511 --nal-hrd --aud --sar 1:1 --output NUL "F:\WORKING_FILES\LAGARITH_SOURCE\LAGARITH_YV12.avs"
--[Information] [25/08/2009 23:40:52] Encoding started
--[NoImage] Standard output stream
--[NoImage] Standard error stream
---[NoImage] avis [info]: 1920x1080 @ 23.98 fps (193 frames)
---[NoImage] x264 [info]: using SAR=1/1
---[NoImage] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
---[NoImage] x264 [info]: profile Main, level 4.0
---[NoImage]
---[NoImage] x264 [info]: slice I:11 Avg QP:13.82 size:134121
---[NoImage] x264 [info]: slice P:58 Avg QP:15.17 size:112927
---[NoImage] x264 [info]: slice B:124 Avg QP:16.77 size: 71664
---[NoImage] x264 [info]: consecutive B-frames: 2.7% 6.6% 37.9% 52.7%
---[NoImage] x264 [info]: mb I I16..4: 66.1% 0.0% 33.9%
---[NoImage] x264 [info]: mb P I16..4: 53.1% 0.0% 0.0% P16..4: 29.2% 0.0% 0.0% 0.0% 0.0% skip:17.7%
---[NoImage] x264 [info]: mb B I16..4: 19.5% 0.0% 0.0% B16..8: 28.1% 0.0% 0.0% direct:26.5% skip:25.8% L0:32.8% L1:22.0% BI:45.2%
---[NoImage] x264 [info]: direct mvs spatial:97.6% temporal:2.4%
---[NoImage] x264 [info]: coded y,uvDC,uvAC intra:75.2% 91.2% 79.3% inter:37.0% 43.2% 9.4%
---[NoImage] x264 [info]: kb/s:16807.0
---[NoImage] encoded 193 frames, 5.87 fps, 16807.07 kb/s
--[Information] [25/08/2009 23:41:25] Postprocessing
---[Information] Deleting intermediate files
--[Information] [25/08/2009 23:41:25] Job completed
-[Information] Log for job23 (video, LAGARITH_YV12.avs -> 01_small_test.264)
--[Information] [25/08/2009 23:41:25] Started handling job
--[Information] [25/08/2009 23:41:25] Preprocessing
--[NoImage] Job commandline: "C:\Program Files\megui\tools\x264\x264.exe" --profile high --level 4 --preset fast --pass 3 --bitrate 18000 --stats "F:\WORKING_FILES\LAGARITH_SOURCE\01_small_test.stats" --thread-input --deblock -1:-1 --keyint 24 --min-keyint 2 --b-adapt 2 --direct auto --ref 1 --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 25000 --vbv-maxrate 20000 --qcomp 0.5 --subme 2 --partitions all --trellis 0 --no-mixed-refs --mvrange 511 --nal-hrd --aud --sar 1:1 --aud --output "F:\WORKING_FILES\LAGARITH_SOURCE\01_small_test.264" "F:\WORKING_FILES\LAGARITH_SOURCE\LAGARITH_YV12.avs"
--[Information] [25/08/2009 23:41:25] Encoding started
--[NoImage] Standard output stream
--[NoImage] Standard error stream
---[NoImage] avis [info]: 1920x1080 @ 23.98 fps (193 frames)
---[NoImage] x264 [info]: using SAR=1/1
---[NoImage] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
---[NoImage] x264 [info]: profile High, level 4.0
---[NoImage]
---[NoImage] x264 [info]: slice I:11 Avg QP:14.06 size:131970
---[NoImage] x264 [info]: slice P:58 Avg QP:15.02 size:127117
---[NoImage] x264 [info]: slice B:124 Avg QP:16.62 size: 76332
---[NoImage] x264 [info]: consecutive B-frames: 2.7% 6.6% 37.9% 52.7%
---[NoImage] x264 [info]: mb I I16..4: 56.3% 19.7% 24.1%
---[NoImage] x264 [info]: mb P I16..4: 29.3% 24.8% 7.6% P16..4: 12.2% 4.4% 3.0% 0.6% 0.5% skip:17.6%
---[NoImage] x264 [info]: mb B I16..4: 19.3% 0.0% 0.0% B16..8: 27.1% 2.5% 2.1% direct:23.4% skip:25.6% L0:31.9% L1:21.4% BI:46.7%
---[NoImage] x264 [info]: 8x8 transform intra:23.4% inter:14.4%
---[NoImage] x264 [info]: direct mvs spatial:83.1% temporal:16.9%
---[NoImage] x264 [info]: coded y,uvDC,uvAC intra:76.6% 91.5% 79.3% inter:35.5% 42.8% 9.8%
---[NoImage] x264 [info]: kb/s:18176.6
---[NoImage] encoded 193 frames, 7.81 fps, 18176.70 kb/s
--[Information] Final statistics
---[NoImage] Video Bitrate Desired: 18000 kbit/s
---[NoImage] Video Bitrate Obtained (approximate): 18176 kbit/s
--[Information] [25/08/2009 23:41:50] Postprocessing
---[Information] Deleting intermediate files
--[Information] [25/08/2009 23:41:50] Job completed
-[Information] Log for job24 (video, LAGARITH_YV12.avs -> 01_small_test.264)
--[Information] [25/08/2009 23:41:50] Started handling job
--[Information] [25/08/2009 23:44:24] Preprocessing
--[NoImage] Job commandline: "C:\Program Files\megui\tools\x264\x264.exe" --profile high --level 4 --preset fast --pass 3 --bitrate 18000 --stats "F:\WORKING_FILES\LAGARITH_SOURCE\01_small_test.stats" --thread-input --deblock -1:-1 --keyint 24 --min-keyint 2 --b-adapt 2 --direct auto --ref 1 --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 25000 --vbv-maxrate 20000 --qcomp 0.5 --subme 2 --partitions all --trellis 0 --no-mixed-refs --mvrange 511 --nal-hrd --aud --sar 1:1 --aud --output "F:\WORKING_FILES\LAGARITH_SOURCE\01_small_test.264" "F:\WORKING_FILES\LAGARITH_SOURCE\LAGARITH_YV12.avs"
--[Information] [25/08/2009 23:44:24] Encoding started
--[NoImage] Standard output stream
--[NoImage] Standard error stream
---[NoImage] avis [info]: 1920x1080 @ 23.98 fps (193 frames)
---[NoImage] x264 [info]: using SAR=1/1
---[NoImage] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
---[NoImage] x264 [info]: profile High, level 4.0
---[NoImage]
---[NoImage] x264 [info]: slice I:11 Avg QP:14.00 size:132840
---[NoImage] x264 [info]: slice P:58 Avg QP:15.03 size:126955
---[NoImage] x264 [info]: slice B:124 Avg QP:16.61 size: 76602
---[NoImage] x264 [info]: consecutive B-frames: 2.7% 6.6% 37.9% 52.7%
---[NoImage] x264 [info]: mb I I16..4: 56.2% 19.8% 24.0%
---[NoImage] x264 [info]: mb P I16..4: 29.4% 25.0% 7.3% P16..4: 12.2% 4.3% 3.0% 0.6% 0.5% skip:17.7%
---[NoImage] x264 [info]: mb B I16..4: 19.4% 0.0% 0.0% B16..8: 27.2% 2.5% 2.0% direct:23.3% skip:25.5% L0:31.9% L1:21.4% BI:46.6%
---[NoImage] x264 [info]: 8x8 transform intra:23.5% inter:14.3%
---[NoImage] x264 [info]: direct mvs spatial:84.7% temporal:15.3%
---[NoImage] x264 [info]: coded y,uvDC,uvAC intra:76.6% 91.5% 79.4% inter:35.5% 42.8% 9.8%
---[NoImage] x264 [info]: kb/s:18210.2
---[NoImage] encoded 193 frames, 8.15 fps, 18210.22 kb/s
--[Information] Final statistics
---[NoImage] Video Bitrate Desired: 18000 kbit/s
---[NoImage] Video Bitrate Obtained (approximate): 18210 kbit/s
--[Information] [25/08/2009 23:44:48] Postprocessing
---[Information] Deleting intermediate files
----[Information] [25/08/2009 23:44:48] Successfully deleted F:\WORKING_FILES\LAGARITH_SOURCE\01_small_test.stats
--[Information] [25/08/2009 23:44:48] Job completed


Still fails to mux in Scenarist.

What I can't understand is that Ben's file encodes and muxes fine, it's just this test file I'm using - but both are Lagarith. I thknk I need to sort out another test file and see what happens with that...

Dark Shikari
25th August 2009, 23:53
I was ready to report back that it hadn't, when I noticed this:



So does x264 believe the buffer / bitrate limits to be different to what was posted earlier in this thread, or did I misread your earlier post:The first pass is Main Profile, which has different limits than High. But this doesn't matter; it's only the first pass.

Biggiesized
26th August 2009, 05:36
My apologies if this comes off a bit rude, Ben, but if you have access to the source DPX files, why is there such bad banding in the fades in the source you made available to us?

shon3i
26th August 2009, 07:37
@dvdboy, maybe you can try encode without --nal-hrd command, but after encode you must use h264info tool to rewrite HRD information. Maybe somewere problem in HRD. Also you can try to decrase/incrase bitrate.

multimediaman
26th August 2009, 12:40
Feel free :).

OK

Match_Point_trailer___ffv1_flac-001.mkv - 981.52MB (http://www.zshare.net/download/64675929883271dc/)
md5sum d9697eca0fb548233adff21a7277716e

Match_Point_trailer___ffv1_flac-002.mkv - 773.27MB (http://www.zshare.net/download/64680730873f8bf6/)
md5sum af683b1799e348ebaec5abbfe948edb3

popper
26th August 2009, 16:09
Is there an automated way to download all those files? Downloading 38 files by hand isn't fun ;)
Unfortunately the overview page that shows all files doesn't exhibit proper download URL's, so DTA and friends can't help here...

Can't you ask Xiph to put it on their http://media.xiph.org/ (http://media.xiph.org/video/derf/) server or at least put it on a service like Rapidshare ???

LoRd_MuldeR, you should be able to make and post a small, single self contained working cross platform 'rebol view' script to do this using REBOL PARSE given your background in programing.... for use on any current and future skydrive.live.com files uploaded there.

or potentially any other site URL too for that matter, or even stick it in a local or remote rebol GUI or CGI rebol web server that takes any URL given it, and process it as you wish, if you want to go that far for fun OC.

im not good at scripting but it seems you could open a 'rebol view' shell and type a simple
page: read http://cid-bee3c9ac9541c85b.skydrive.live.com/browse.aspx/.Public/Match%20Point

typing
print page
will show it in the shell

to get a current updated page locally in the page: (or whatever you want to use) variable
then rebol parse the page for all the
http:// hash .skydrive.live.com/self.aspx/.Public/Match%20Point/Match%20Point%20LAGS%20YV12.7z. Nos URL links into another found: variable

then do another rebol read and parse on each found URL there thats got the
http:// something .blu.livefilestore.com. ..... whatever

and do a binary write save to disk of each 7z in there, as per that short http://musiclessonz.com freehand GUI painting program in section 4. below
see also 6.5 Running Command Line Applications

--------cut and paste inside a rebol view shell to see it work
view center-face layout [
h1 "Paint:"
s: area black 650x350 feel [engage: func [f a e] [
if a = 'over [append s/effect/draw e/offset show s]
if a = 'up [append s/effect/draw 'line]]] effect [draw [line]
]
b: btn "Save Image" [save/png %a.png to-image s alert "Saved 'a.png'"]
btn "Clear" [s/effect/draw: copy [line] show s]
]

----- cut

http://www.rebol.com/view-platforms.html
http://www.nabble.com/How-to-properly-parse-HTML-and-XHTML-Meta-Tags-td19448593.html
http://compsci.ca/v3/viewtopic.php?t=17706
http://musiclessonz.com/rebol.html#section-8.9

LoRd_MuldeR
26th August 2009, 16:22
LoRd_MuldeR, you should be able to make and post a cross platform 'rebol view' script to do this using REBOL PARSE given your background in programing.... for use on any current and future skydrive.live.com files uploaded there.

Sure I could write a download script for that purpose. But it's far too much trouble for downloading one sample video. I'm not willing to do that :p

Especially if the problem can be solved easily by using a better hosting platform.

Thanks to "multimediaman" I'm downloading the clip from http://zshare.net/ in only two pieces and in a much more space-efficient format :)

popper
26th August 2009, 17:17
Sure I could write a download script for that purpose. But it's far too much trouble for downloading one sample video. I'm not willing to do that :p
Especially if the problem can be solved easily by using a better hosting platform.

Thanks to "multimediaman" I'm downloading the clip from http://zshare.net/ in only two pieces and in a much more space-efficient format :)

sure, but i thought people might like to at least try it for fun....
as already, theres two clips there on skydrive.live.com that we all want to get and try at some point, there will be several more in time i hope too, thanks to benwaggoner etc, a generic rebol script might be very handy later.

LoRd_MuldeR
26th August 2009, 17:19
It's very kind of Benwaggoner to share his footage. But I really hope he will use a different hosting platform next time.

I know he likes to stick with MS solutions, but it's really pain, especially in this case :D

Lyris
26th August 2009, 18:31
Ben actually tried many times to upload to the FTP account I set up, but the transfer kept failing because of a problem on this end. The SkyDrive solution is the one which worked properly, so take from that what you will.

Just so it doesn't get lost in amongst all this The Island trailer discussion, I'll ask again:

Can anyone here share the secrets for encoding compliant Secondary Video streams?

All I have so far is that the frame rate has to match the primary video, and since we're using AVC, the secondary video must be AVC also. And the max bit rate can't go above 8mbps, but this is talking heads stuff in SD resolution, so 2mbps should do it fine...

Can anyone fill me in on the VBV buffer size? Number of B frames? Keyframe interval?

benwaggoner
26th August 2009, 18:35
My apologies if this comes off a bit rude, Ben, but if you have access to the source DPX files, why is there such bad banding in the fades in the source you made available to us?
Alas, there was banding in the DPX sequence.

DI only sets an upper bound for quality, not a lower bound :).

I could probably do a XScaler dithered version as well.

kolak
26th August 2009, 20:15
Ben actually tried many times to upload to the FTP account I set up, but the transfer kept failing because of a problem on this end. The SkyDrive solution is the one which worked properly, so take from that what you will.

Just so it doesn't get lost in amongst all this The Island trailer discussion, I'll ask again:

Can anyone here share the secrets for encoding compliant Secondary Video streams?

All I have so far is that the frame rate has to match the primary video, and since we're using AVC, the secondary video must be AVC also. And the max bit rate can't go above 8mbps, but this is talking heads stuff in SD resolution, so 2mbps should do it fine...

Can anyone fill me in on the VBV buffer size? Number of B frames? Keyframe interval?

BD spec book cost 5K$:)


Andrew

kolak
26th August 2009, 20:18
Alas, there was banding in the DPX sequence.

DI only sets an upper bound for quality, not a lower bound :).

I could probably do a XScaler dithered version as well.

Ben, is XScaler available to buy? I know that it comes with Cinevision PSE, but is there a way to get it seperately?
Did MS ever done multithreaded version?

Andrew

benwaggoner
26th August 2009, 21:31
Ben, is XScaler available to buy? I know that it comes with Cinevision PSE, but is there a way to get it seperately?
Did MS ever done multithreaded version?
I'll look into it.

The various post houses did their own segmented multithreaded mplementations (so an 8-core machine would simultaneously XScale the source in 8 chunks at once).

Biggiesized
27th August 2009, 07:19
Stacey Spears said that XScaler was "made available freely to just about anyone who wanted it" or something along those lines on the AVS Forum. I couldn't tell if this was in reference to studios only however.

It's a pity that the DPX sequence had banding as well. Perhaps they were a bit too aggressive in moving those bits around (12-bit linear should be the minimum DI IMO).

EDIT: If there is one benefit to being a Microsoft employee, it's that you get to play around with sweet tools.

Biggiesized
27th August 2009, 07:22
Ben actually tried many times to upload to the FTP account I set up, but the transfer kept failing because of a problem on this end. The SkyDrive solution is the one which worked properly, so take from that what you will.

Just so it doesn't get lost in amongst all this The Island trailer discussion, I'll ask again:

Can anyone here share the secrets for encoding compliant Secondary Video streams?

All I have so far is that the frame rate has to match the primary video, and since we're using AVC, the secondary video must be AVC also. And the max bit rate can't go above 8mbps, but this is talking heads stuff in SD resolution, so 2mbps should do it fine...

Can anyone fill me in on the VBV buffer size? Number of B frames? Keyframe interval?

Don't quote me, but I think VBV can match the Blu-ray spec for primary streams (30000 kbs), but you may want to go to something much lower like 12000 to be safe since your max video data rate is only 8 mbps.

I think the GOP length can be 2 seconds (minimum IDR is 1 frame). I think B frames are limited to 2 and reference frames to 4.

BTW, Lyris I've been unable to register to comment on your website. What gives? Bad CAPTCHA script?

Lyris
27th August 2009, 08:08
BTW, Lyris I've been unable to register to comment on your website. What gives? Bad CAPTCHA script?
D'oh - will look into that!

Thanks for the suggestions, I'll try those out and see if I can get Secondary video to mux.

kolak
27th August 2009, 18:53
Stacey Spears said that XScaler was "made available freely to just about anyone who wanted it" or something along those lines on the AVS Forum. I couldn't tell if this was in reference to studios only however.

It's a pity that the DPX sequence had banding as well. Perhaps they were a bit too aggressive in moving those bits around (12-bit linear should be the minimum DI IMO).

EDIT: If there is one benefit to being a Microsoft employee, it's that you get to play around with sweet tools.

It's rather not available for commercial use. Microsoft was giving for free PEP encoder at NAB, but later licens has expired.

Andew

shon3i
27th August 2009, 21:10
Can anyone here share the secrets for encoding compliant Secondary Video streams?

All I have so far is that the frame rate has to match the primary video, and since we're using AVC, the secondary video must be AVC also. And the max bit rate can't go above 8mbps, but this is talking heads stuff in SD resolution, so 2mbps should do it fine...

Can anyone fill me in on the VBV buffer size? Number of B frames? Keyframe interval?There is no secrets, situation is very clear :)

Only restrtictions from default are:

for SD (720x576 or 720x480)
Level must be 3.2
vbv maxrate=8000
vbv buffersize=12000 (must be equal to level 3 because max DBP is 3037.5)
max ref frames=6 for 720x480 and 5 for 720x576

for HD (720 or 1080)
level 4.0 or 4.1 (slices are required for 4.1)
vbv maxrate=40000 for Level 4.1, 24000 for level 4.0
vbv buffersize=30000
max ref frames=6 for 720 and 4 for 1080


other things are same as primary video (bframes=3, keyframe interval and etc..)

Lyris
27th August 2009, 22:13
Thank you :) I will see if I can get this to mux successfully when I get home (just got back from a Sony SXRD projector event - the wonders of optics in masking compression artefacts never ceases to amaze me). The idea of secondary video is pretty exciting.

rack04
27th August 2009, 22:21
(bframes=3, keyframe interval and etc..)

One thing that threw me for a loop was --tune animation changing the default bframes 3 to bframes 5.

Dark Shikari
27th August 2009, 22:24
One thing that threw me for a loop was --tune animation changing the default bframes 3 to bframes 5.In general, tunings and presets can change things; if you absolutely need setting X to be set to value A, explicitly set it. Don't make assumptions of the defaults.

benwaggoner
30th August 2009, 17:01
Thank you :) I will see if I can get this to mux successfully when I get home (just got back from a Sony SXRD projector event - the wonders of optics in masking compression artefacts never ceases to amaze me). The idea of secondary video is pretty exciting.
This job would be a lot easier if everyone just stuck to CRT!

Given all the overdriven LCD displays with brighness set for daylight viewing but watched in the dark, pretty much all codecs need to have some low-luma range DQuant and even de-noise to reducing blocking-in-blacks. The video gamma curve the perceptual uniformity of a CRT, but that's out the window with LCD and plasma in the low luma range.

popper
30th August 2009, 21:53
This job would be a lot easier if everyone just stuck to CRT!

Given all the overdriven LCD displays with brighness set for daylight viewing but watched in the dark, pretty much all codecs need to have some low-luma range DQuant and even de-noise to reducing blocking-in-blacks. The video gamma curve the perceptual uniformity of a CRT, but that's out the window with LCD and plasma in the low luma range.

you havent gone and tryed to buy a reasonably priced HDTV recently (and by 'recently' i mean the last 2 or 3 Years)
then Ben?

you Cant buy anything competitive or even higher mid range and HD capable in the main UK retail outlets these days thats Not an LCD, even if you wanted to... perhaps its different in the US but not the EU.

benwaggoner
30th August 2009, 22:41
you havent gone and tryed to buy a reasonably priced HDTV recently (and by 'recently' i mean the last 2 or 3 Years)
then Ben?

you Cant buy anything competitive or even higher mid range and HD capable in the main UK retail outlets these days thats Not an LCD, even if you wanted to... perhaps its different in the US but not the EU.
I'm well aware that the CRT is dead.

The problem is that the entire digital video delivery system still assumes CRT as a baseline. And we got lucky in that CRT and the human eye had a very similar gamma curve in terms of input level to perceptual value, and thus perceptual uniformity. But since LCD doesn't have the same characteristics, we get weird stuff in the low lumas.

And that's stuff that PSNR and SSIM aren't going to catch, since they also assume perceptual uniformity of the luma range.

Dark Shikari
31st August 2009, 11:57
So what do you guys think about a slicing patch?

'cause I have one working, and it even works with one-MB-per slice at 1080p (8100 slices!). Fun fact: both my stream analyzers fail with that many slices; only ffh264 (compiled with MAX_SLICES=8192) and CoreAVC decode it correctly. JM works, but takes about 92 seconds per frame.

Speaking of which, is there any reason to have slices which consist only of horizontal bands and don't end in the middle of a row?

Chengbin
31st August 2009, 12:39
So what do you guys think about a slicing patch?

I think the commercial encoders should be very, very, VERY afraid :D

moviefan
31st August 2009, 13:07
So what do you guys think about a slicing patch?

'cause I have one working, and it even works with one-MB-per slice at 1080p (8100 slices!).

Sounds like x264 is going to be another step closer to a fully Blu-ray compliant H264 encoder. Great work! I'm looking forward to this patch and I am totally excited about the weightp one that is hopefully coming soon ;).

Selur
31st August 2009, 13:16
So what do you guys think about a slicing patch?
HAR, HAR, HAR ! (but I would keep the slice number a bit down ;))

Cu Selur

Lyris
31st August 2009, 13:22
you Cant buy anything competitive or even higher mid range and HD capable in the main UK retail outlets these days thats Not an LCD, even if you wanted to... perhaps its different in the US but not the EU.
Popper, check out the high end Panasonic NeoPDP Plasma displays. They have a THX preset which is pretty accurate to D65/Rec.709 out of the box (colour is still a bit off, though). We've been harassing them for some time to add more calibration control though.

So what do you guys think about a slicing patch?
I think the existence of this thread and the number of pages it's reached should answer that question :D

Emulgator
31st August 2009, 13:23
Just gimme four ;-)

LoRd_MuldeR
31st August 2009, 13:31
I personally don't care about BD and thus I don't need slices. So I hope this won't effect Non-Sliced encoding.

But yes, it seems there is great demand for this in the "community" ;)

wyti
31st August 2009, 13:37
It would be very nice to be more compliant with blueray spec, and if it's an optional switch, it will not hurt anything ;)

LoRd_MuldeR
31st August 2009, 13:40
...unless some efficiency and/or maintainability must be sacrificed to integrate slices support, which I hope isn't the case :D

benwaggoner
31st August 2009, 18:48
I personally don't care about BD and thus I don't need slices. So I hope this won't effect Non-Sliced encoding.
Well, there can be a latency advantage with slicing for both encode and decode. So there are cases beyond Blu-ray Level 3.1 where slicing could be a useful feature for live broadcasting and particularly conferencing.

I'm still doubtful that Level 3.1 will offer any perceptible quality advantage with 1080p24 content over Level 3.0.

Blue_MiSfit
31st August 2009, 19:10
Sounds good to me. As I understand it, most players are perfectly happy without slices - but I'm always okay with better compatibility. "Most players" is a no-go if you're going to replicate discs :)

As long as it doesn't kill quality horribly! BluRay does have plenty of head room for bitrate ;)

~MiSfit

creamyhorror
31st August 2009, 19:23
I'm still doubtful that Level 3.1 will offer any perceptible quality advantage with 1080p24 content over Level 3.0.
Level 4, you mean. And surely it would matter with some H.264 encoders on some very noisy, high-motion material...?

shon3i
31st August 2009, 20:09
Well that is good news indeed, this should be done long time ago ;)

benwaggoner
31st August 2009, 21:51
Level 4, you mean.
Yes, sorry.

And surely it would matter with some H.264 encoders on some very noisy, high-motion material...?
At 1080p24? 25 Mbps is a lot of bits per pixel at 1080p24. 1080i30 I could see cases where it would matter, particularly without MBAFF in x264. But I'd want to see some real-world examples where there's a noticible visual improvement with 1080p24.

Dark Shikari
31st August 2009, 22:28
Multi-slice encoding support (http://git.videolan.org/?p=x264.git;a=commit;h=4d553edf178bf0ae01547731a48e1fb08c5cc1f4)

shon3i
31st August 2009, 23:10
Multi-slice encoding support (http://git.videolan.org/?p=x264.git;a=commit;h=4d553edf178bf0ae01547731a48e1fb08c5cc1f4)
I can not believe what I just saw, this is amazing. Can you explane switches?

For example i want to use 4 slices for BD compilancy what to need to set?

Thanks, Thanks,Thanks, Thanks,Thanks, Thanks,Thanks, Thanks :)

btw what gonna be for NAL HRD patch?

Dark Shikari
31st August 2009, 23:14
OMFG I can not believe what I just saw, this is amazing. Can you explane switches?

For example i want to use 4 slices for BD compilancy what to need to set?--slices 4? :sly:

shon3i
31st August 2009, 23:19
--slices 4? :sly:
Ok :scared:

is there any reason for other two switches?? Is there some rule about?

Dark Shikari
31st August 2009, 23:21
Ok :scared:

is there any reason for other two switches?? Is there some rule about?--slice-max-size is useful for RTP streaming on clients which insist on slices smaller than MTU. --slice-max-mbs is useful if you need to specify exact slice sizes instead of a general number per frame.

shon3i
31st August 2009, 23:39
OK ;) one more info

how much quality can drop? what about mt method

Lyris
31st August 2009, 23:47
Multi-slice encoding support (http://git.videolan.org/?p=x264.git;a=commit;h=4d553edf178bf0ae01547731a48e1fb08c5cc1f4)
I just choked on my lunch!
Time to give this a whirl!

Blue_MiSfit
1st September 2009, 00:13
/cheer!

Lyris
1st September 2009, 00:42
Anyone compiled yet? I'm used to getting my software spoon-fed to me and am getting failed hunks following this guide (http://forum.doom9.org/showthread.php?p=723782#post723782).

Dark Shikari
1st September 2009, 00:42
The NAL-HRD patch likely needs updating.

Lyris
1st September 2009, 00:47
Yeah, that's the one I was trying to apply. D'oh.

Chengbin
1st September 2009, 01:17
Wow, Dark Shikari, you're an amazing guy! You said it was a lot of work, and 2 weeks later we get it! It's the summer of features for x264!

Anything else missing for x264?

Dark Shikari
1st September 2009, 01:19
Wow, Dark Shikari, you're an amazing guy! You said it was a lot of workI don't think I ever said that slicing was actually that much work... ;)

crypto
3rd September 2009, 23:09
This is amazing. Thanks Dark Shikari, for the slicing patch.

moviefan
23rd September 2009, 19:37
One question that has popped up when I tried to find a workaround for the unfinished nal-hrd patch: Can I encode without the nal-hrd patch and then run the stream through h264info checking only the "Add Picture Structure" option? Or are there any problems with that method?

Trahald
25th September 2009, 13:10
As a way to get some software to accept your file maybe. Anything h264info spits out is guaranteed not to be perfect buffer wise because the original stream didnt consider the data being added is buffer calculation. having said that, as long as buffer usage wasnt pinned to max, that shouldnt be an issue for most streams. Best bet will be to wait, but since you are probably just making backups for yourself, you will have to judge if imperfection will be acceptable for you.

moviefan
25th September 2009, 13:16
Okay, so the patch does not simply add some information to the stream but also influences calculations in the encoding process. I do seek BD compliance rather than the mere acceptance of my stream by some software, so waiting seems the only option. Waiting for a final/corrected version of the patch implies what kind of time range? Days, weeks, months...? I of course don't expect a date, but just a vague estimation... Actually, is there any discussion forum where developers exchange their ideas and where progress of things can be looked at?

Trahald
25th September 2009, 15:52
IRC is probably the best place right now. #x264dev on freenode