AVC for BluRay

[toboda]

we tried X264 for h264 encoding of a 100 min. anime feature film. Everything looks fine and compliant (through Scenarist mui generator).
The problem is that some scenes are pixeled. We are looking to make some scenes with other settings using a script with the ranges and the corresponding settings. The next step could be to merge all the outcoming files into a big final stream. with the DOS command copy /b in the m2v world this was possible but we don't get proper results with the h264 files because Scenarist refuses to accept.
My question is if anybody knows about how to manage this merging or (more advanced) how it would be possible to encode choosen scenes with other settings in other words: making segmented (re-)encoding.

Thanks for advice,

Tobias.

[Dark Shikari]

First of all, the most obvious thing would be to try to deal with this issue of pixelation from an encoder level: can you post some screenshots or stream clips of the problem?

For segmented re-encoding, the problem could be that some of the settings are stream-wide, so you can't change them on a per-segment basis, or at least Scenarist won't let you.

[toboda]

Hi Shikari,

i'll try to sum up my probs with x264. If you pm me, I can give you access to our ftp, where you can download sample images and the encoded files.
You'll find the settings I used for encoding at the end of this post. The source used is 1920x1080x32bit DPX converted to uncompressed avi
1920x1080x24bit@24fps using Carbone Coder.

1) Pixalation: As already mentioned, pixalation and blocking appeared on all encodes. The source has a lot of artifical "grain" which is used to simulate
the view through a DV-camera. This - combined with the fact, that it is anime material - seems to be very challenging for the encoder.

2) Bitrate control: I need to encode within strict bitrate bounderies to enable bit-budgeting for the final BR-Disc. Analysing my encodes with Elecard
streameye showed, that bitrate varies heavily (peaks to 50mbit - lows to 1mbit). Even changing the encoding mode didnt help much.
Is there any way to control the bitrate more strictly in x264? What I need is a 2 pass encode, that adapets to the complexity of the
source within strictly defined bitrate borders.
As my encodes played fine on bdp-s300 and ps3: How "serious" are those peaks. Can they lead to stuttering playback on standalone
players or can I just ignore them as long as the average bitrate + audio doesnt exceed the max. bitrate specified for BR?

3) Source: megui says that the source is not 16x16 compliant at a resolution of 1920x1080. Could this cause problems due to some weird internal resizing?
Could the colorspace conversion be a problem?

That just sums up my basic problems. if you need more specific info on something just let me know. Big thanks in advance for your help and time.

Finally here are my encoding settings:

program --pass 2 --bitrate 20000 --stats ".stats" --level 4.1
--keyint 12 --min-keyint 1 --ref 4 --mixed-refs --no-fast-pskip
--bframes 4 --b-adapt 2 --b-rdo --bime --weightb --direct auto
--trellis 2 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --ipratio 1.0
--pbratio 1.0 --vbv-bufsize 40000 --vbv-maxrate 25000 --ratetol 0.1
--qcomp 0.1 --me umh --threads auto --thread-input --aq-mode 0 --progress
--no-psnr --no-ssim --output "output" "input" --mvrange 511
--aud --nal-hrd --sar 1:1

Greetings Toboda

[Dark Shikari]

Hi Shikari,

i'll try to sum up my probs with x264. If you pm me, I can give you access to our ftp, where you can download sample images and the encoded files.Just PM it to me. I don't think I need to PM you first, given you have the link ;)
1) Pixalation: As already mentioned, pixalation and blocking appeared on all encodes. The source has a lot of artifical "grain" which is used to simulate
the view through a DV-camera. This - combined with the fact, that it is anime material - seems to be very challenging for the encoder.Psy-RD is your weapon of choice for grain. Use a recent x264 version.
2) Bitrate control: I need to encode within strict bitrate bounderies to enable bit-budgeting for the final BR-Disc. Analysing my encodes with Elecard
streameye showed, that bitrate varies heavily (peaks to 50mbit - lows to 1mbit). Even changing the encoding mode didnt help much.
Is there any way to control the bitrate more strictly in x264? What I need is a 2 pass encode, that adapets to the complexity of the
source within strictly defined bitrate borders.
As my encodes played fine on bdp-s300 and ps3: How "serious" are those peaks. Can they lead to stuttering playback on standalone
players or can I just ignore them as long as the average bitrate + audio doesnt exceed the max. bitrate specified for BR?This is what VBV maxrate and bufsize are for. x264 will never silently violate the maxrate you set; a violation will result in warning messages. Elecard doesn't necessarily measure max bitrate relative to the buffer you set, so its not a reliable tool for measuring VBV compliance.3) Source: megui says that the source is not 16x16 compliant at a resolution of 1920x1080. Could this cause problems due to some weird internal resizing?
Could the colorspace conversion be a problem?There is no internal resizing. 1080p isn't mod16, but that's not a big problem.

program --pass 2 --bitrate 20000 --stats ".stats" --level 4.1
--keyint 12 --min-keyint 1 --ref 4 --mixed-refs --no-fast-pskip
--bframes 4 --b-adapt 2 --b-rdo --bime --weightb --direct auto
--trellis 2 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --ipratio 1.0
--pbratio 1.0 --vbv-bufsize 40000 --vbv-maxrate 25000 --ratetol 0.1
--qcomp 0.1 --me umh --threads auto --thread-input --aq-mode 0 --progress
--no-psnr --no-ssim --output "output" "input" --mvrange 511
--aud --nal-hrd --sar 1:1

Greetings Toboda
1. Ipratio/pbratio 1.0 are stupid. I agree lower values for these might be better at Blu-ray bitrates for maintaining grain, but 1.0 is way too low.
2. Keyint 12? I thought you could get away with at least 14.
3. Qcomp 0.1? Again, dumb. The default is fine.
4. AQ mode 0? Well, no wonder. Adaptive quantization is very important to maintaining grain and detail in flat areas. It might be advisable to have it a bit lower than default, but 0? No way.
5. Isn't b-frames 4 above the limit? I thought Blu-ray required 3 or less.
6. Subme 7, no good reason not to.
7. Don't touch ratetol; it doesn't do what you think it does.

[Sagittaire]

Your setting are not good for quality and for compliance:

"--keyint 12 --min-keyint 1"
the GOP max for BD is 1 sec. --keyint 24 --min-keyint 1 for 24 fps source is the best setting.


"--ref 4 --bframes 4 --b-adapt 2"
t's not compliant with BD profil. --ref 4 --bframes 2 --b-adapt 2 is a good setting. With your bitrate b-adapt is useless (too slow).


"--vbv-bufsize 40000 --vbv-maxrate 25000"
it's not compliant with BD profil. --vbv-bufsize 30000 --vbv-maxrate 40000 is the max for video primary stream. 30 Kbits is a really high buffer and stream can have really higher local bitrate than maxrate (40 Mbps for BD). It's not a problem and stream will be perfectly compliant. VBV work like that to have a max local rate higher than the max optical disk rate.


"--pbratio 1.0 --ratetol 0.1 --qcomp 0.1"
Don't touch the rate control setting if you don't know how RC work. --ipratio 1.10 --pbratio 1.10 produce really good work with short GOP at 1 sec and really constant local quality at high bitrate.

Try this profil ...

@REM -----------------------------------------------------------
@REM
@REM Profil BluRay 1080p23.976 extra high quality
@REM
@REM -----------------------------------------------------------


@REM Source file name (suffit de mettre la source ici)
set E_SRC=Lossless.avs

@REM Set of quality (ici la qualité 1-50)
set E_BR=21.3

@REM Set of max bitrate (ici le bitrate max)
set MAX_BR=40000

@REM Set of Buffer (ici le buffer)
set BUF_BR=30000

@REM Set credit (frame de début du générique)
set CRE_FR=201560

@REM Set end credit (frame de fin du générique)
set END_FR=207442



@REM Profil

x264.exe --threads auto --thread-input --keyint 24 --min-keyint 1 --crf %E_BR% --vbv-maxrate %MAX_BR% --vbv-bufsize %BUF_BR%
--mvrange 511 --level 4.1 --bframe 2 --b-rdo --bime --weightb --ref 4 --mixed-refs --direct auto --deblock -2:-2 --ipratio 1.10 --pbratio 1.10
--partitions "all" --8x8dct --me "umh" --subme 7 --trellis 2 --no-fast-pskip --no-dct-decimate --aud --nal-hrd --sar 1:1 --zone %CRE_FR%,%END_FR%,b=0.33
--progress --pass 1 --stats "stat.log" -o 1080p_Q1.264 %E_SRC%

pause


Psy-rd is good for grain retention but produce ringing.
AQ is perhaps not ideal with really clean anime.
With really clean anime higher inloop value produce gerenraly better visual result. Anyway it's not really good for grain retention.

[Dark Shikari]

With your bitrate b-adapt is useless (too slow).Biprediction actually gives more benefit at higher bitrates than lower (PSNR-wise, at least)...

[toboda]

Thx for your help and sry for my late reply. I think I got to the bottom of the problem. It seems, that the source file
is the problem.
I downloaded Ripbot and fed it the uncompressed file and the same material ripped from a bluray as mpeg-2 and encoded to bluray.
The uncompressed source came out faulty all over again, while the mpeg-2 source was okay.

I'm just guessing here, but could there be a kind of "buffer-underrun" while encoding? (at least the datarate of uncompressed full-HD avi is pretty high)
Or could the problem come from the windows software raid,
the source files reside on?

Concerning your settings: At least in megui, I was not able
to enable "all partitions" @ high lvl 4.1. There is always a warning, that this setting is not supported at that lvl
and it is switched to "unrestricted".
MediaCoder even states, you should only enable that setting on SD Material. What am I not getting here?

Another prob with your settings still persists:
Bitrate peaks. This may not be a prob if you only multiplex
a single ac3 stream in your bluray project, but as soon
as you'll try to integrate more bandwith consuming audio,
the multiplex fails.

Concerning the settings of the video buffer:
In your settings you choose 30000/40000.
What exactly is the problem, when lowering the max_bitrate to 25000? What negativ effects on playability or quality can this have?

UPDATE: I converted the mpeg2-hd source to avi and then fed it to ripbot. The resulting encode was okay!
Thus it seems that the dpx to avi conversion is the prob, although it was done in two diffrent apps
(After Effects 7 and Carbone Coder 3).
Greetz

[shon3i]

x264.exe --threads auto --thread-input --keyint 24 --min-keyint 1 --crf %E_BR% --vbv-maxrate %MAX_BR% --vbv-bufsize %BUF_BR%
--mvrange 511 --level 4.1 --bframe 2 --b-rdo --bime --weightb --ref 4 --mixed-refs --direct auto --deblock -2:-2 --ipratio 1.10 --pbratio 1.10
--partitions "all" --8x8dct --me "umh" --subme 7 --trellis 2 --no-fast-pskip --no-dct-decimate --aud --nal-hrd --sar 1:1 --zone %CRE_FR%,%END_FR%,b=0.33
--progress --pass 1 --stats "stat.log" -o 1080p_Q1.264 %E_SRC%
Isn't --partition "all" unsuported by BluRay profile, generaly i mean for P4x4

[Sagittaire]

Isn't --partition "all" unsuported by BluRay profile, generaly i mean for P4x4

As far I know BD support all the partition ...

[toboda]

I'm now encoding from PNG-files and everything seems ok so far.
While playing around with the settings, I noticed the following:

The more I lower the value for quant max in 2-pass encoding,
the more buffer underrun warnings I get in the encoding log.
Beyond a certain value the second pass starts but outputs nothing. There is no error warning and megui will just sit there doing nothing.
Now the relating questions:

How serious are these buffer underuns? I burned some of the
encodes that just had a few of them to bluray and they played flawlessly.

Which value is "underrun" when this warning appears? Is it actual size of the videobuffer (it was set to 40000 for sa bluray) or a problem with max datarate specified (25000)?

Thx for ur help!

[Sagittaire]

well 40 Kbits is not the max for BD ... MeGui profil are simply wrong. 40 Mbps is the max for Bitrate. Use 30 Kbits for Buffer and 40 Mbps for Max bitrate.

[shon3i]

well 40 Kbits is not the max for BD ... MeGui profil are simply wrong. 40 Mbps is the max for Bitrate. Use 30 Kbits for Buffer and 40 Mbps for Max bitrate.
Correct, i checked few BluRay titles with Elecard Buffer Analyser, and all shows buffer @ 30000, and max rate @ 40000.

[poisondeathray]

Correct, i checked few BluRay titles with Elecard Buffer Analyser, and all shows buffer @ 30000, and max rate @ 40000.

If this is true, then maybe someone should notify Sharktooth? (RE: MeGUI blu-ray profiles)

[shon3i]

Here is screenshot from Sony SPHE 2008 International Blu-Ray Demo Version 4 1080p

http://img300.imageshack.us/img300/6141/19296263fa9.th.jpg (http://img300.imageshack.us/img300/6141/19296263fa9.jpg)

Have buffer @ 30000 and max rate @ 33000

[toboda]

Some more interesting things showed during my tests:

After doing some intensive testing with the png source, I was able to pin down the problem of extrem blockiness that occured.
It seems that when ever the source is based on the DPX files,
the problem with the blockiness appears. After encoding longer sequences of the movie it also happend with the png source.
This doesn't happen with the mpeg-2 (ripped from BR) as source. After trying a lot of settings, I ended up reducing
the GOP size to 4. This finally fixed the broken frames when
encoding the png source.
I assume, that the png source is a lot more dynamic in terms of color and brightness. Could this be the reason why x264
has difficulties to encode the video at larger gops?

Concering the buffer issue: I changed my vbv setting to 30000
as u suggested. The encodes just play fine on both ps3 and
BDP-S300. No obvious difference to the 40000 setting.

Greetz

[toboda]

Perhaps someone can help me with this. The encode from
png now is fine so far. The last problem that remains is:
In (3sec+) still shots some parts of the picture change slightly, although there shouldn't be any change.
Any idea, which setting is causing this problem?
:thanks: for your help!