Encoding Video for Blu-Ray using H264/AVC

[shon3i]

This site and this thread is how to encode for Blu-Ray (BD-R or DVD) with pure Blu-Ray structure, and if your target is not BD structure, or it's mkv or something else, you should not use any of these options.

[kieranrk]

This site and this thread is how to encode for Blu-Ray (BD-R or DVD) with pure Blu-Ray structure, and if your target is not BD structure, or it's mkv or something else, you should not use any of these options.

Yes, you will have timecode problems if you try and use pulldown with mkv, mp4 etc.

[dvdboy]

Despite having successfully encoded Blu-ray streams with x264 before, all of my attempts since Friday have produced "buffer underrun" errors in both Scenarist BD 5.1.3 and 5.2.

I've tried using both MeGUI and the commandline directly:

Open GOP

"C:\Software\AVC Encoding\MeGUI_1911_x86_WithoutInstaller\tools\x264\x264.exe" --bitrate 28000 --preset medium --tune film --weightp 1 --bframes 3 --nal-hrd vbr --vbv-maxrate 38000 --vbv-bufsize 30000 --profile high --level 4.1 --keyint 30 --b-pyramid strict --open-gop bluray --slices 4 --tff --aud --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1 --pass 1 -o "E:\DVD Department\output.264" "E:\DVD Department\input.avs"

"C:\Software\AVC Encoding\MeGUI_1911_x86_WithoutInstaller\tools\x264\x264.exe" --bitrate 28000 --preset medium --tune film --weightp 1 --bframes 3 --nal-hrd vbr --vbv-maxrate 40000 --vbv-bufsize 30000 --profile high --level 4.1 --keyint 30 --b-pyramid strict --open-gop bluray --slices 4 --tff --aud --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1 --pass 2 -o "E:\DVD Department\output.264" "E:\DVD Department\input.avs"

Closed GOP

"C:\Software\AVC Encoding\MeGUI_1911_x86_WithoutInstaller\tools\x264\x264.exe" --bitrate 28000 --preset medium --profile high --tune film --weightp 1 --bframes 3 --nal-hrd vbr --vbv-maxrate 38000 --vbv-bufsize 30000 --level 4.1 --keyint 30 --b-pyramid strict --open-gop none --slices 4 --tff --aud --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1 --pass 1 -o "E:\DVD Department\output.264" "E:\DVD Department\input.avs"

"C:\Software\AVC Encoding\MeGUI_1911_x86_WithoutInstaller\tools\x264\x264.exe" --bitrate 28000 --preset medium --profile high --tune film --weightp 1 --bframes 3 --nal-hrd vbr --vbv-maxrate 40000 --vbv-bufsize 30000 --level 4.1 --keyint 30 --b-pyramid strict --open-gop none --slices 4 --tff --aud --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1 --pass 2 -o "E:\DVD Department\output.264" "E:\DVD Department\input.avs"

The reason I (tried) to add the --profile high lines, was I was concerned that the streams were being incorrectly flagged as Main Profile. :stupid:

The input file is a Quicktime via QTInput.

x264 core:114 r1913 5fd3dce

Just noticed a mismatch in the vbv maxrate on each pass, so I will run another test.

[shon3i]

what is resolution and fps of source video ?

[dvdboy]

what is resolution and fps of source video ?

1080i30 (29.97). Audio is exported WAV via Quicktime.

Format : MPEG-4
Format profile : QuickTime
Codec ID : qt
File size : 17.2 GiB
Duration : 11mn 31s
Overall bit rate : 214 Mbps
Encoded date : UTC 2011-03-10 19:02:28
Tagged date : UTC 2011-03-10 19:18:26
Writing library : Apple QuickTime
Media/UUID : 5AE9027B-9FFD-47E6-92A7-FD752340BA3D

Video
ID : 1
Format : ProRes
Format profile : High
Codec ID : apch
Duration : 11mn 31s
Bit rate mode : Variable
Bit rate : 212 Mbps
Width : 1 888 pixels
Original width : 1 920 pixels
Height : 1 062 pixels
Original height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 29.970 fps
Color space : YUV
Chroma subsampling : 4:2:2
Bits/(Pixel*Frame) : 3.525
Stream size : 17.1 GiB (99%)
Language : English
Encoded date : UTC 2011-03-10 19:02:28
Tagged date : UTC 2011-03-10 19:18:26

Audio
ID : 2
Format : PCM
Format settings, Endianness : Big
Format settings, Sign : Unsigned
Codec ID : in24
Duration : 11mn 31s
Bit rate mode : Constant
Bit rate : 2 304 Kbps
Channel(s) : 2 channels
Sampling rate : 48.0 KHz
Bit depth : 24 bits
Stream size : 190 MiB (1%)
Language : English
Encoded date : UTC 2011-03-10 19:02:28
Tagged date : UTC 2011-03-10 19:18:26

[JK1974]

This site and this thread is how to encode for Blu-Ray (BD-R or DVD) with pure Blu-Ray structure, and if your target is not BD structure, or it's mkv or something else, you should not use any of these options.

Sorry, maybe I got something wrong, but I don´t understand this. I encoded 1080p24 as well as 1080i50 with the settings mentioned and made MKVs out of it for the following reasons:
1) I believe(d) that if I use a BD like encoding, I get higher compatibility as decoder chips are "optimized" for playing BD and AVCHD compliant video material.
2) I am open to demux and reauthor it as BD in case I want advanced features like menus, PiP etc. in case that those features are available in affordable BD authoring tools in the next few years and Blu-ray media become more widespread.

I was able to play back at least the 1080i50 encodings as AVCHD discs (tsmuxer) with my PS3, the 1080p24 encodings I did so far also made no problems with network media players (Sigma Designs/Realtek chipsets).

But maybe my situation is different as I don´t use any pulldown features...

[shon3i]

@dvdboy, did you try mux without audio ? can be audio problem.

@JK1974,

1) yes
2) no, because if you mux BD compatible stream to MKV, you will destroy some data which is irrevocable.

btw fake intelaced is less important as pulldown, i mean it will not effect on decoding process.

[latet]

2) no, because if you mux BD compatible stream to MKV, you will destroy some data which is irrevocable.

What exactly? Can you please write more about it?
Thank you.

[shon3i]

What exactly? Can you please write more about it?
Thank you.
There is significant difference between containers. Not all containers require same, and encoding process is different depending on container.

[JK1974]

There is significant difference between containers. Not all containers require same, and encoding process is different depending on container.

Still more details?
I am sure that I demuxed an MKV, made an AVCHD disc with TSMuxer and played it back flawlessly on the PS3. However, it might work as TSMuxer as well as the PS3 don´t check more details in the stream.
However, I am still a little concerned about my encodings as I just started editing and encoding my AVCHD camcorder recordings and wanted to use an open format like MKV instead of the uncomfortable and feature-poor AVCHD disc format.

[shon3i]

Still more details?Calculations for AUD and NAL data is different for specified container, therefore nal-hrd command will be different for different container, but aslo muxing raw stream to container (MKV, MP4) will be aslo strip some of this data without backing it on demux.

In real world, only few BD chips currently care of this (it will not play that stream). And there is BD Verification tools that will not approve stream, so if you don't care about 100% compliant stream then this should not bothering you.

[latet]

And there is BD Verification tools that will not approve stream

Could please write the name of such tool?

Thanks,

latet

[shon3i]

Could please write the name of such tool?

Thanks,

latet
Sony Verifier, Panasonic Verifier and many more BD disc verifiers, used in studios.

[Sagittaire]

Despite having successfully encoded Blu-ray streams with x264 before, all of my attempts since Friday have produced "buffer underrun" errors in both Scenarist BD 5.1.3 and 5.2.

I've tried using both MeGUI and the commandline directly:

Open GOP



Closed GOP



The reason I (tried) to add the --profile high lines, was I was concerned that the streams were being incorrectly flagged as Main Profile. :stupid:

The input file is a Quicktime via QTInput.

x264 core:114 r1913 5fd3dce

Just noticed a mismatch in the vbv maxrate on each pass, so I will run another test.

During encoding you change maxrate to 38000 for 40000 Kbps between first and second in your profil ... ???

IMO change maxrate at 35000 for mux compatibility (main stream PCM 5.1 and secondary stream AC3 5.1 ~ 5 Mbps)

With average near max it's perhaps better to make 3 pass for better HRD compatibility and peak bitrate quality.

[kieranrk]

1080i30 (29.97). Audio is exported WAV via Quicktime.


Make sure the framerate x264 reports is 30000/1001 and not anything else.

PM if you want to post any non-public information.

[Rudde]

Hey, I'm building a FreeBSD server and was wondering if there is any x264 build for FreeBSD that actively get updated?

[dvdboy]

@kieranrk @sagittaire

Thanks for the responses, sorry for not replying. I've been doing a number of tests with small pieces of footage that I felt happy posting samples from, but they all muxed in Scenarist fine! I then substituted the input in my 'good' commandline with the source material I was having issues with above, and this failed to mux.

Now this footage is side-by-side 3D footage, and I assume x264 doesn't 'care' about the contents of the footage, but to confirm I stretched the left eye back out to 1920, but it still failed. I kicked off a full program, which I haven't had a chance to drop into Scenarist yet.

@Saggitaire - yeah I noticed the mismatched max values and corrected that in my subsequent tests. I'll try both 3-pass and a lower max bitrate, although I though I was being quite conservative with 28Mbps Average, 38Mbps Max...
@kieranrk - I will give that a go in case the footage is funky - I assume a AssumeFPS line in the AVISynth will do the trick?

Many thanks for your help.

[pepelugil]

I have some of these "1080i25" blu-rays, that have frames encoded progressively but to be standard compliant are flagged as interlaced and I want to backup them on BD-25.

I've added/changed these settings from my usual 100% 23,976 blu-ray compliant backups: --no-interlaced --fake-interlaced --keyint 25 --fps 25000/1000

Scenarist correctly accepts and muxes the resulting x264 stream without any error, but MediaInfo says that the resulting m2ts has 25 fps progressive while the original is correctly detected as 25fps MBAFF.

Do I have to worry about my 100% blu-ray compliant 1080i25 backups using x264 with the settings I'm using?

Greetings

[nm]

MediaInfo says that the resulting m2ts has 25 fps progressive while the original is correctly detected as 25fps MBAFF.

Then the original is MBAFF and not fake-interlaced. With x264 the latter is significantly more efficient, at least until we get adaptive MBAFF.

Do I have to worry about my 100% blu-ray compliant 1080i25 backups using x264 with the settings I'm using?

Nope. AFAIK, there shouldn't be problems with --fake-interlaced. The guide (http://sites.google.com/site/x264bluray/home/1080i-p) suggests --pic-struct with it.

[mp3dom]

pic-struct is mandatory for interlaced, fake-interlaced or pulldowned streams and optional for progressive stream.
Your file is accepted by Scenarist but this is not safe. Scenarist made only some minor checks so you should use a BD Verifier if you want to be 100% sure.

[pepelugil]

I always use --pic-struct (even for progressive stream) so one less "problem".

As for BD verifiers, I have Elecard StreamEye Studio, but if I not wrong it doesn't check BD specification, so it seems that I'll have to trust Scenarist.

[shon3i]

I always use --pic-struct (even for progressive stream) so one less "problem".

As for BD verifiers, I have Elecard StreamEye Studio, but if I not wrong it doesn't check BD specification, so it seems that I'll have to trust Scenarist.
You should not worry, becuase stream is progressive and it's marked as interlaced while frames are still frames not fields. Mediainfo and other tools just use header to examine data. Your stream must be interlaced or marked as interlaced (fake interlace) or will not be BD compilant.

[mp3dom]

Shon3i, just out of curiosity, have you checked a fake-interlaced 25p with Sony Verifier? Because I've made a 25p with fake-interlaced just now to test. Scenarist accepts and mux (and identify the stream as interlaced) but my verifier (BDQuest) shows one error. It identify the stream as progressive while streams from pro-encoders (that allows the same thing, input a 25/30 progressive stream, encode as progressive and only flag as interlaced) are OK.
The error from BDQuest is (just for reference):

"invalid combination of horizontal size of frame (1920), vertical size of frame (1088), pic_width_in_mbs_minus1(119), pic_height_in_maps_units_minus1(33), frame_mbs_only_flag(0), framerate (25), and scanning type (progressive) for 1920x1080"
"frame_mbs_only_flag" is 0 and "mb_adaptive_field_frame_flag" is 0. 1920x1080 video (field) is expected to contain 8/9/8/9 rows in slices. Found 17/17/17/17


We know that BDQuest is not fully reliable, but who knows this time. Anyway CineVision (which supports the same thing as x264, set progressive source and flag as interlaced) flag the stream as MBAFF.

[A.Fenderson]

I've added/changed these settings from my usual 100% 23,976 blu-ray compliant backups: --no-interlaced --fake-interlaced --keyint 25 --fps 25000/1000

I'm not finding any info on the --no-interlaced option on mewiki (http://mewiki.project357.com/wiki/X264_Settings)or x264 --fullhelp. Mewiki has one reference to --interlaced, but it tries to link back to a non-existent entry on the same page. What does this switch do?

[kemuri-_9]

I'm not finding any info on the --no-interlaced option on mewiki (http://mewiki.project357.com/wiki/X264_Settings)or x264 --fullhelp. Mewiki has one reference to --interlaced, but it tries to link back to a non-existent entry on the same page. What does this switch do?

--no-interlaced causes the input to be treated as progressive, even if it were detected as interlaced.

[Phil_L]

Hi

After a lot of trial and errors and reading tips here about x264 I finally settled on using MeGUI, and can successfully encode to 1080i/50 for Blu-ray. That is the good.

The bad is finding authoring software that will accept the output so looking for pointers. This is what I have found:

Sony DVD Architect, get tswrapper errors as it doesn't like MBAFF, so that's out.

Corel MovieFactory 7 - test clips I had success, clips accepted without re-rendering, perfect. But then when I tried some longer footage I got DVD engine errors, it would get so far then fail during mux'ing. I found the part mux'd mts file and so could see where it was failing. After some checking I found x264 was encoding with some high peak bit-rates of 50+, when MovieFactory got to something above ~45 it bombed. After some more trial and error I found reducing the buffer size the high peaks were reduced, with a buffer of 15000 I managed to get a clip almost mux'd in MovieFactory before it bombed out, it still managed to peak at one point.

I'm encoding with target bitrate of 36,000, and the max is the preset 40,000. The same bitrate settings via MainConcept's H264 I have no problems with, and when I check bit-rates it doesn't peak hardly above 40,000.

Now I gather with the buffer then peaks above 50,000 are valid for Blu-ray.

So my issue is how to author a Blu-ray disc with the x264 output?

I've also tried Cyberlink software, that wouldn't even recognise the files and I tried TMPGEnc Authoring Works 4, but it only wants MPEG2 for Blu-ray.

I have had success with MultiAVCHD, but I'm not entirely happy with the menu creation options, it's great for a free tool, but doesn't lend itself to more fancy graphical menus, unless I just haven't discovered that functionality.

I've also seen mention of Blu-ray verifiers, where would you get those so I can test the output is okay?

Any help or pointers appreciated.

Regards

Phil

[shon3i]

Shon3i, just out of curiosity, have you checked a fake-interlaced 25p with Sony Verifier? Because I've made a 25p with fake-interlaced just now to test. Scenarist accepts and mux (and identify the stream as interlaced) but my verifier (BDQuest) shows one error. It identify the stream as progressive while streams from pro-encoders (that allows the same thing, input a 25/30 progressive stream, encode as progressive and only flag as interlaced) are OK.
The error from BDQuest is (just for reference):

"invalid combination of horizontal size of frame (1920), vertical size of frame (1088), pic_width_in_mbs_minus1(119), pic_height_in_maps_units_minus1(33), frame_mbs_only_flag(0), framerate (25), and scanning type (progressive) for 1920x1080"
"frame_mbs_only_flag" is 0 and "mb_adaptive_field_frame_flag" is 0. 1920x1080 video (field) is expected to contain 8/9/8/9 rows in slices. Found 17/17/17/17


We know that BDQuest is not fully reliable, but who knows this time. Anyway CineVision (which supports the same thing as x264, set progressive source and flag as interlaced) flag the stream as MBAFF.
Yep i checked it several times. Btw update your bdquest to lastest version they had bug with decoder.

[dvdboy]

Make sure the framerate x264 reports is 30000/1001 and not anything else.

PM if you want to post any non-public information.

A belated update but this in the end appears to be the problem - I added AssumeFPS(30000,1001) to the AVISynth script and Scenarist muxed the file fine.

I ran into a similar problem with an After Effects rendered Top Menu Background - even though everything said 23.976fps, I was getting the Buffer Underrun error. When I watched the x264 log go through, I noticed the framerate was being interpreted as 2997/125 rather than 24000/1001. Again AssumeFPS did the trick.

Many thanks for your help.

[kolak]

I always use --pic-struct (even for progressive stream) so one less "problem".

As for BD verifiers, I have Elecard StreamEye Studio, but if I not wrong it doesn't check BD specification, so it seems that I'll have to trust Scenarist.

Scenarist check is quite basic- has not much to do with strict BD compliancy.


Andrew

[lutinor]

that's sad that there isn't an accessible real bd check :(

[iSeries]

Could the first post be updated with the new --bluray-compat option for those of us (mainly me lol) who are a bit confused as to how to use it please?

[shon3i]

yes, it will be updated asap ;)

[sneaker_ger]

Could you also make a remark at the top that says that these settings are for creating 100% compliant Blu-Ray structures, not for encoding from Blu-Ray sources or to matroska/mp4? I see a lot of people crippling their settings for no need.

[shon3i]

Could you also make a remark at the top that says that these settings are for creating 100% compliant Blu-Ray structures, not for encoding from Blu-Ray sources or to matroska/mp4? I see a lot of people crippling their settings for no need.
Done :P

[sneaker_ger]

Great work. Thx.

[laserfan]

This is discussion about settings that are for creating 100% compliant Blu-Ray structures, not for encoding from Blu-Ray sources to matroska/mp4 or other media, if you encode to those skip this nightmare :)
And I thought it was just me who felt this way! ;)

Thanks for updating.

[kieranrk]

EDIT never mind

[rapscallion]

@shon3i

First page, 1080i example command line doesn't include --pic-struc ? Is that because it's only "required" for fake interlaced?

Also, your example assumes tff. Doesn't x264 encode @ MBAFF ?

What if the source is MBAFF, do you assume --tff command or something else?

[mp3dom]

@shon3i
First page, 1080i example command line doesn't include --pic-struc ? Is that because it's only "required" for fake interlaced?

It's required for interlaced and pulldown too, but in that case the pic-struct parameter is on by default, so no need to explicitly set.


Also, your example assumes tff. Doesn't x264 encode @ MBAFF ?
MBAFF is a way to encode interlaced footage (the other is PAFF). This refers to encoding, but the source footage still is tff or bff interlaced. MBAFF is not yet 100% implemented in x264. Currently it lacks the 'adaptive' part.


What if the source is MBAFF, do you assume --tff command or something else?
With AviSynth you can see which is the fieldorder and set x264 accordingly (with --tff or --bff)

[rapscallion]

@mp3dom

Thank you ! Very concise answers and just what I was looking for.

I opened w/avisynth via VirtualDub and I don't see the fieldorder in the file info.
Please advise how I do that. Thanks

[mp3dom]

AssumeTFF()
SeparateFields()

If the preview have jerky movements (watch in dynamic zones) then your video is BFF, otherwise is TFF. (You can do the opposite, AssumeBFF().SeparateFields() and if you have jerky motion your video is TFF, otherwise is BFF)

[shon3i]

BIG UPDATE: After kieranrk point me to part of source, --bluray-compat do some magic aslo beside only hacks.

Basically will reduce some of options to BD compliant ones. Here what will do:

automatically will down b-pyramid to strict, weightp to 1, bframes to 3, reference frames to 4 or 6 for 720/480/576 res, enable aud and nalhrd vbr.

so command line is pretty short now :)

[rapscallion]

AssumeTFF()
SeparateFields()

If the preview have jerky movements (watch in dynamic zones) then your video is BFF, otherwise is TFF. (You can do the opposite, AssumeBFF().SeparateFields() and if you have jerky motion your video is TFF, otherwise is BFF)

Thanks again mp3dom.

That worked, the vid is tff.

[mp3dom]

Isn't better to include in the bluray-compat also pic-struct? Considering that it's mandatory for fake-interlaced, pulldown and interlaced and is optional (but doesn't create problem and resolve some warnings coming from some BD verifiers) for progressive... It will never create problems on BD side.

[shon3i]

Now is mandatory for (interlaced, fakeinterlaced and pulldown) using bluray compat, since r1938 or r1939. Is not for progressive and not will be because is not in specs (yes i know)

[jpsdr]

weightp to 1

Yek ??? Oh no !! I must redo my encode and add weightp 2 in the commande line (it was in the preset before)... :(

[laserfan]

BIG UPDATE: After kieranrk point me to part of source, --bluray-compat do some magic aslo beside only hacks.

Basically will reduce some of options to BD compliant ones. Here what will do:

automatically will down b-pyramid to strict, weightp to 1, bframes to 3, reference frames to 4 or 6 for 720/480/576 res, enable aud and nalhrd vbr.

so command line is pretty short now :)
Thanks for this--you shon3i are a beacon of light in the x264 sea of Dark-ness! ;)

[jpsdr]

Now is mandatory for (interlaced, fakeinterlaced and pulldown) using bluray compat, since r1938 or r1939. Is not for progressive and not will be because is not in specs (yes i know)
r1938/r1939...???
According x264.nl, we are still on 1937...

[shon3i]

r1938/r1939...???
According x264.nl, we are still on 1937...
Just be patient, there is more commits so probably will be much higher revision than 1938 is coming.

[Stereodude]

Why does x264 sometimes whiff the target bitrate on a 2pass encode so badly?

Most of the time I've found it gets within a few kbit/sec (usually on the low side) when I'm encoding at a target bitrate to fill a Blu-Ray disc. However, I've found that ocassionally x264 will miss the bitrate by a lot more, generally on the low side but not always.

Here's the command line I used:x264.exe --bitrate 19110 --preset veryslow --tune film --weightp 1 --bframes 3 --ref 4 --bluray-compat --b-adapt 2 --vbv-maxrate 40000 --vbv-bufsize 30000 --level 4.1 --keyint 24 --open-gop --slices 4 --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1 --pass 1 --qpfile ___.chp -o NUL ___.AVS
x264.exe --bitrate 19110 --preset veryslow --tune film --weightp 1 --bframes 3 --ref 4 --bluray-compat --vbv-maxrate 40000 --vbv-bufsize 30000 --level 4.1 --keyint 24 --open-gop --slices 4 --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1 --pass 2 --qpfile ___.chp -o ___.264 ___.AVSI asked for a bitrate of 19110kbit/sec and I got a file with a bitrate of 19067.15kbit/sec. :confused: It's not the end of the world, but I'm curious.