x264 "Macroblock Tree Ratecontrol" testing (committed)

[J_Darnley]

x264 cannot run if it cannot allocate all the memory it needs.

[TheImperial2004]

Then that means , Mb-tree will produce better results with bigger RAM ?

[LoRd_MuldeR]

I'm not sure becuase I use AviDemux , but if encoding a file with Mb-tree needs 3GB for example , and x264 limited at 2GB . Will that affect encoding quality or compression capability ?

Avidemux is only provided as a 32-Bit binary at the moment. So there is a 2 GB limit of the entire Avidemux process, including x264 (which is running inside the Avidemux process).

If x264 cannot allocate the required memory, it will simply fail. However this is unlikely to happen, because even with 1080p content there's enough memory available for reasonable RC Lookahead values.

And since Avidemux doesn't offer options the adjust the RC Lookahead yet, you are protected against insane RC Lookahead values :D

Then that means , Mb-tree will produce better results with bigger RAM ?

No, it doesn't! It means that MB-Tree allocates more memory than the "classical" ratecontrol.

Furthermore the memory required by MB-Tree ratecontrol depends highly on the RC Lookahead value. The higher the RC Lookahead is raised, the more memory will be required by x264!

But as said before, Avidemux uses the default RC Lookahead value, which is 40. So memory consumption remains reasonable ;)

A higher RC Lookahead probably gives better results (while consuming more memory), but even the "placebo" preset only uses a RC Lookahead of 60 (the current maximum is 250).

[TheImperial2004]

So , An RC of 250 will still work with 2GB while encoding 1080p ?

Can you provide a regular Libx264 build with RC of 250 so we can test this ?

:thanks:

[J_Darnley]

So , An RC of 250 will still work with 2GB while encoding 1080p ?

No! Loren has said that the buffered frames for 1920x1080 take 12 MB so with a lookahead of 250 x264 requires 12*250 = 3000 MB. This does not account for other memory that x264 will require.

[LoRd_MuldeR]

So , An RC of 250 will still work with 2GB while encoding 1080p ?

No, it probably wouldn't! Such insane RC Lookahead values would try to allocate an extraordinary amount of RAM and probably run into the 2 GB limit.

I already told you that even x264's "veryslow" and "placebo" presets only use a RC Lookahaed of 60. Why do you want to go even higher ???

Can you provide a regular Libx264 build with RC of 250 so we can test this ?

Nope, because I won't encourage people to use insane settings. My libx264 builds are already patches to increase RC Lookahead slightly over the default ;)

(With SubME=8 it will be raised to 50; with SubME=9 it will be raised to 60. And yes, I know this is a hackish workaround ^^)

[TheImperial2004]

So unless you use x64 version of x264 , you can't use 250 with 1080p (which can't be done with AviDemux AFAIK) ... I guess we'll have to upgrade to 64bit sooner or later :)

[LoRd_MuldeR]

So unless you use x64 version of x264 , you can't use 250 with 1080p

Probably not. But why do want to use such insane values? Do you think higher is always better ???

Anything above RC Lookahead 60 will probably give only a very subtle improvement (if at all), but require more memory for sure.

So don't blindly use overkill settings. Instead try to find the sweet spot!

[TheImperial2004]

You got me there Mulder :)

I like your way of explaining things ;)

[MfA]

Since it's tied to qcomp I assume this only changes quantizers after motion estimation right? If so, any plans to use the knowledge of future blocks in the motion refinement step too?

[Dark Shikari]

Since it's tied to qcomp I assume this only changes quantizers after motion estimation right?Huh? Qcomp didn't change quantizers after motion estimation either.

[TheImperial2004]

Probably not. But why do want to use such insane values? Do you think higher is always better ???

Anything above RC Lookahead 60 will probably give only a very subtle improvement (if at all), but require more memory for sure.

So don't blindly use overkill settings. Instead try to find the sweet spot!
I also noticed that when I use 2-Pass mode , I got a .MBTREE file and my free RAM stays at a rasonable value . But with CRF I don't get such a file and RAM reaches Zero in an instant ... Why can't we make Libx264 generate this file regardless of the encoding mode ?

[MfA]

Let me rephrase that, when you do RDO during subpixel refinement does it take into account the errors caused in dependent blocks for different motion vectors? (I guess you'd have to re-encode dependent blocks with the settings from the previous pass but with a changed MC residual.)

[kemuri-_9]

I also noticed that when I use 2-Pass mode , I got a .MBTREE file and my free RAM stays at a rasonable value . But with CRF I don't get such a file and RAM reaches Zero in an instant ... Why can't we make Libx264 generate this file regardless of the encoding mode ?

the file is generated during the 1st pass which is then used in the 2nd pass.

you can also generate it with crf by specifying --pass 1, but unless you do a 2nd pass, you won't see the benefit of this file

[Dark Shikari]

Let me rephrase that, when you do RDO during subpixel refinement does it take into account the errors caused in dependent blocks for different motion vectors? (I guess you'd have to re-encode dependent blocks with the settings from the previous pass but with a changed MC residual.)No, x264 doesn't have a macroblock-level lookahead.

[ajp_anton]

I also noticed that when I use 2-Pass mode , I got a .MBTREE file and my free RAM stays at a rasonable value . But with CRF I don't get such a file and RAM reaches Zero in an instant ... Why can't we make Libx264 generate this file regardless of the encoding mode ?Have you tried --slow-firstpass ?

[TheImperial2004]

the file is generated during the 1st pass which is then used in the 2nd pass.

you can also generate it with crf by specifying --pass 1, but unless you do a 2nd pass, you won't see the benefit of this file
You can't specify any CLI commands in AviDemux ... :)

[J_Darnley]

Have you tried --slow-firstpass ?
This is is not a library option anyway.

[Snowknight26]

It is.
http://git.videolan.org/?p=x264.git;a=blob;f=x264.c

178 H0( " --slow-firstpass Don't use faster settings with --pass 1\n" );

[kemuri-_9]

It is.
http://git.videolan.org/?p=x264.git;a=blob;f=x264.c

no it is not: x264.c is not a part of libx264

[J_Darnley]

It is.
http://git.videolan.org/?p=x264.git;a=blob;f=x264.c
From x264.c

378 static struct option long_options[] =
379 {
...
386 { "slow-firstpass", no_argument, NULL, OPT_SLOWFIRSTPASS },
...
817 case OPT_SLOWFIRSTPASS:
818 b_turbo = 0;
819 break;
...
857 /* Set faster options in case of turbo firstpass. */
858 if( b_turbo && b_pass1 )
859 {
860 param->i_frame_reference = 1;
861 param->analyse.b_transform_8x8 = 0;
862 param->analyse.inter = 0;
863 param->analyse.i_me_method = X264_ME_DIA;
864 param->analyse.i_subpel_refine = X264_MIN( 2, param->analyse.i_subpel_refine );
865 param->analyse.i_trellis = 0;
866 }


[EDIT] Thanks kemuri

[EDIT2]Also:

diff --git a/x264.h b/x264.h
index 26ac421..e61040e 100644 (file)
--- a/x264.h
+++ b/x264.h
@@ -35,7 +35,7 @@

#include <stdarg.h>

-#define X264_BUILD 67
+#define X264_BUILD 68

/* x264_t:
* opaque handler for encoder */
@@ -83,7 +83,6 @@ typedef struct x264_t x264_t;
#define X264_CQM_FLAT 0
#define X264_CQM_JVT 1
#define X264_CQM_CUSTOM 2
-#define X264_RC_NONE -1
#define X264_RC_CQP 0
#define X264_RC_CRF 1
#define X264_RC_ABR 2
@@ -103,8 +102,7 @@ static const char * const x264_transfer_names[] = { "", "bt709", "undef", "", "b
static const char * const x264_colmatrix_names[] = { "GBR", "bt709", "undef", "", "fcc", "bt470bg", "smpte170m", "smpte240m", "YCgCo", 0 };

/* Colorspace type
- * legacy only; nothing other than I420 is really supported.
- */
+ * legacy only; nothing other than I420 is really supported. */
#define X264_CSP_MASK 0x00ff /* */
#define X264_CSP_NONE 0x0000 /* Invalid mode */
#define X264_CSP_I420 0x0001 /* yuv 4:2:0 planar */
@@ -118,8 +116,7 @@ static const char * const x264_colmatrix_names[] = { "GBR", "bt709", "undef", ""
#define X264_CSP_MAX 0x0009 /* end of list */
#define X264_CSP_VFLIP 0x1000 /* */

-/* Slice type
- */
+/* Slice type */
#define X264_TYPE_AUTO 0x0000 /* Let x264 choose the right type */
#define X264_TYPE_IDR 0x0001
#define X264_TYPE_I 0x0002
@@ -129,14 +126,16 @@ static const char * const x264_colmatrix_names[] = { "GBR", "bt709", "undef", ""
#define IS_X264_TYPE_I(x) ((x)==X264_TYPE_I || (x)==X264_TYPE_IDR)
#define IS_X264_TYPE_B(x) ((x)==X264_TYPE_B || (x)==X264_TYPE_BREF)

-/* Log level
- */
+/* Log level */
#define X264_LOG_NONE (-1)
#define X264_LOG_ERROR 0
#define X264_LOG_WARNING 1
#define X264_LOG_INFO 2
#define X264_LOG_DEBUG 3

+/* Threading */
+#define X264_THREADS_AUTO 0 /* Automatically select optimal number of threads */
+
/* Zones: override ratecontrol or other options for specific sections of the video.
* See x264_encoder_reconfig() for which options can be changed.
* If zones overlap, whichever comes later in the list takes precedence. */

No mention of any related change

[Snowknight26]

Whoops, libx264. Shows you how well my selective reading works. :o

[LoRd_MuldeR]

Why can't we make Libx264 generate this file regardless of the encoding mode ?

Because the stats file is for 2-Pass mode and there's no point in having a stats file for Single-Pass modes (CQP, CRF, 1-Pass ABR).

[kemuri-_9]

Because the stats file is for 2-Pass mode and there's no point in having a stats file for Single-Pass modes (CQP, CRF, 1-Pass ABR).

reasons can be left to the person and their whims,
that is why every mode can actually create 2pass .stat files... even lossless mode

it's up to the application using libx264 to allow this in the end or not, so.... idk if avidemux prevents this or not.

[LoRd_MuldeR]

reasons can be left to the person and their whims,
that is why every mode can actually create 2pass .stat files... even lossless mode

it's up to the application using libx264 to allow this in the end or not, so.... idk if avidemux prevents this or not.

I'm aware that not only ABR mode can create a stats file. For example CRF can be used the create a stats file and then that stats file can be used for the second pass.

Still my point stands: If you aren't going re-use the stats file for a second pass, it would be useless to create a stats file (except for debugging purposes, maybe).

[detmek]

Hi. This is my first post here, but I have been frequently visiting this forum for over a year.
About stats file in one pass mode. Leiming's GUI has an option conditional double pass that will run first pass CRF or ABR and create stats file. If output file size is in expected margins, it's OK. If it's not, it will run another pass and will use stats from the first pass.
P.S. Sorry, english is not my primary language.

[J_Darnley]

Yes, you can do that. I believe Dark Shikari has said that is what Facebook does. If you have a question, please end with a question mark.

[Dark Shikari]

Yes, you can do that. I believe Dark Shikari has said that is what Facebook does.Correct.

[TheImperial2004]

D.S. can you tell us what new features are coming to x264 ? I'm following your diary (http://x264dev.multimedia.cx/) but nothing mentioned there ...

[detmek]

Yes, my post is confusing. It's not a question. I just wanted to comment conversation between kemuri-_9 and LoRd_Mulder about mbtree stats in one pass mode. It can be usefull feature and can be easily implemented in MeGUI but I don't know about Avidemux, especially through libx264.
Btw, is mbtree less efficient in 2-pass mode? I tried it today and I got SSIM 0.9885540 without mbtree and 0.9888014 with mbtree. Test clip is Diablo 3 trailer 720p.

[Disabled]

I'm encoding videos in CRF mode but append the --pass 1 option to get a stats file. I do it, because later I delete the encode and always keep the source. But if I need an encode again, I just do a second pass based on the stats file I saved. Now with the mbtree file being quite large, I wonder if it would be possible to delete the mbtree file and only keep the stats file? So that on a second pass with mbtree enabled but no file found, it re-does the mb-tree algorithm to save the space.
I tried it, but it doesn't work. Would it be complicated to implement it, or are there some technical limitations to what I want?

[benwaggoner]

I'm encoding videos in CRF mode but append the --pass 1 option to get a stats file. I do it, because later I delete the encode and always keep the source. But if I need an encode again, I just do a second pass based on the stats file I saved. Now with the mbtree file being quite large, I wonder if it would be possible to delete the mbtree file and only keep the stats file? So that on a second pass with mbtree enabled but no file found, it re-does the mb-tree algorithm to save the space.
I tried it, but it doesn't work. Would it be complicated to implement it, or are there some technical limitations to what I want?
Have you tried applying lossless compression to the mbtree file? Perhaps 7zip et al could shrink it down to something more easily storable. I'd expect (without actually haven't looked at it in the slightest :)) that it'd have a fair amount of redundency and hence compressibility.

[LoRd_MuldeR]

I'd expect (without actually haven't looked at it in the slightest :)) that it'd have a fair amount of redundency and hence compressibility.

Not as much as I would have expected...

[Dark Shikari]

Have you tried applying lossless compression to the mbtree file? Perhaps 7zip et al could shrink it down to something more easily storable. I'd expect (without actually haven't looked at it in the slightest :)) that it'd have a fair amount of redundency and hence compressibility.I explicitly designed it so that it wouldn't. It shouldn't be that compressible.

[juGGaKNot]

1232 stall about 2-3 minutes between passes, it this because it does the first 60 frames fast ? ( lookahead ) ?

also mbtree works in the first pass right ? normal fast 1pass can cause quality problems ?

just asking, i have not seen any.

[gizzin]

If they can beat x264 in visual quality on ordinary test clips without postprocessing, I'll eat my hat.


lol...

[moviefan]

If I experience some very very rare banding in dark scences (most probably because the source is a little weak there), Would lowering qcomp from now 0.75 help? As far as I know, qcomp balances between CBR and CQ and mbtree takes more bits from complex scences for redistribution at lower qcomp values. Except for debanding a little, could this help? Is the default 0.6 for qcomp generally recommended for clean 1080p sources?

[Dark Shikari]

If I experience some very very rare banding in dark scences (most probably because the source is a little weak there), Would lowering qcomp from now 0.75 help? As far as I know, qcomp balances between CBR and CQ and mbtree takes more bits from complex scences for redistribution at lower qcomp values. Except for debanding a little, could this help? Is the default 0.6 for qcomp generally recommended for clean 1080p sources?You can raise adaptive quantization; lowering qcomp probably won't help (at least with MB-tree on).

[xyloy]

Awesome news!

In fact, I did already a test of mb-tree with my DVD of Terminator 2 Extreme Edition, but I did not know how to turn it off (nor what it was) before reading this thread.

So I did a little mb-tree/no mb-tree versus test (x86 x264 rev.1251 from x264.nl) with the intro of Day of the Tentacle (captured with DOSBox ZMBV codec and with PCM for audio), just for the fun of it. :P

Source :
Day of the Tentacle
DOSBox ZMBV (8 bits per pixel) capture
320x200 pixels, 70 FPS

AviSynth script:

DirectShowSource(".\tentacl_000.avi")
ConverttoYV12()


It was a two-pass 55 kbps encode (I blindly tried first with 10 as a nominal bitrate, but x264 said something like "failed 2nd pass curve, minimal bitrate should be around 52 kbps").

first, here are the command line results :
1st pass without mb-tree

D:\>"c:\Program Files (x86)\megui\tools\x264\x264.exe" --pass 1 --bitrate 55 --
stats "D:\DOTT(no-mbtree).stats" --keyint 700 --min-keyint 70 --ref 16 --no-fast
-pskip --b-adapt 2 --b-pyramid --direct auto --subme 9 --trellis 2 --partitions
all --me umh --merange 32 --thread-input --no-dct-decimate --output NUL "D:\DSS.
avs" --fullrange on --no-mbtree
avis [info]: 320x200 @ 70.09 fps (52217 frames)
x264 [warning]: width or height not divisible by 16 (320x200), compression will
suffer.
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile Main, level 2.1
x264 [info]: frame I:96 Avg QP:37.00 size: 2888
x264 [info]: frame P:50094 Avg QP:39.95 size: 100
x264 [info]: frame B:2027 Avg QP:47.36 size: 85
x264 [info]: consecutive B-frames: 93.9% 2.1% 1.7% 2.3%
x264 [info]: mb I I16..4: 71.9% 0.0% 28.1%
x264 [info]: mb P I16..4: 19.7% 0.0% 0.0% P16..4: 2.3% 0.0% 0.0% 0.0% 0
.0% skip:78.0%
x264 [info]: mb B I16..4: 0.2% 0.0% 0.0% B16..8: 4.5% 0.0% 0.0% direct:
7.3% skip:88.0% L0:35.3% L1:64.4% BI: 0.3%
x264 [info]: final ratefactor: 51.90
x264 [info]: direct mvs spatial:99.7% temporal:0.3%
x264 [info]: coded y,uvDC,uvAC intra:1.3% 14.1% 6.6% inter:0.7% 1.5% 0.8%
x264 [info]: kb/s:58.8

encoded 52217 frames, 208.49 fps, 58.89 kb/s

2nd pass without MB-tree

D:\>"c:\Program Files (x86)\megui\tools\x264\x264.exe" --pass 2 --bitrate 55 --
stats "D:\DOTT(no-mbtree).stats" --keyint 700 --min-keyint 70 --ref 16 --no-fast
-pskip --b-adapt 2 --b-pyramid --direct auto --subme 9 --trellis 2 --partitions
all --me umh --merange 32 --thread-input --no-dct-decimate --output "D:\DOTT(no-
mbtree).mp4" "D:\DSS.avs" --fullrange on --no-mbtree
avis [info]: 320x200 @ 70.09 fps (52217 frames)
x264 [warning]: width or height not divisible by 16 (320x200), compression will
suffer.
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 2.1
mp4 [info]: initial delay 285362 (scale 10000000)
x264 [info]: frame I:96 Avg QP:29.66 size: 5023
x264 [info]: frame P:50094 Avg QP:32.06 size: 78
x264 [info]: frame B:2027 Avg QP:36.53 size: 69
x264 [info]: consecutive B-frames: 93.9% 2.1% 1.7% 2.3%
x264 [info]: mb I I16..4: 31.9% 30.3% 37.8%
x264 [info]: mb P I16..4: 0.6% 0.3% 0.0% P16..4: 2.1% 0.4% 2.2% 0.0% 0
.0% skip:94.3%
x264 [info]: mb B I16..4: 0.1% 0.0% 0.0% B16..8: 9.0% 0.6% 0.2% direct:
0.3% skip:89.7% L0:42.2% L1:57.6% BI: 0.2%
x264 [info]: 8x8 transform intra:34.8% inter:41.4%
x264 [info]: direct mvs spatial:96.5% temporal:3.5%
x264 [info]: coded y,uvDC,uvAC intra:13.0% 27.2% 18.1% inter:0.4% 1.1% 0.7%
x264 [info]: ref P L0 77.0% 3.9% 2.3% 0.8% 0.6% 1.2% 1.9% 2.8% 1.7% 2.
0% 1.1% 0.6% 0.7% 0.8% 1.6% 1.0%
x264 [info]: ref B L0 90.0% 2.0% 0.8% 0.4% 0.2% 1.5% 1.7% 1.2% 0.7% 0.
4% 0.2% 0.2% 0.2% 0.2% 0.2%
x264 [info]: ref B L1 97.3% 2.7%
x264 [info]: kb/s:48.7

encoded 52217 frames, 46.57 fps, 48.74 kb/s

1st pass with MB-tree

D:\>"c:\Program Files (x86)\megui\tools\x264\x264.exe" --pass 1 --bitrate 55 --
stats "D:\DOTT(mbtree).stats" --keyint 700 --min-keyint 70 --ref 16 --no-fast-ps
kip --b-adapt 2 --b-pyramid --direct auto --subme 9 --trellis 2 --partitions all
--me umh --merange 32 --thread-input --no-dct-decimate --output NUL "D:\DSS.avs
" --fullrange on --rc-lookahead 250
avis [info]: 320x200 @ 70.09 fps (52217 frames)
x264 [warning]: width or height not divisible by 16 (320x200), compression will
suffer.
x264 [warning]: b-pyramid + mb-tree is not supported
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile Main, level 2.1
x264 [info]: frame I:96 Avg QP:30.97 size: 4086
x264 [info]: frame P:50087 Avg QP:37.59 size: 96
x264 [info]: frame B:2034 Avg QP:48.12 size: 72
x264 [info]: consecutive B-frames: 93.9% 2.1% 1.7% 2.3%
x264 [info]: mb I I16..4: 54.0% 0.0% 46.0%
x264 [info]: mb P I16..4: 17.4% 0.0% 0.0% P16..4: 2.1% 0.0% 0.0% 0.0% 0
.0% skip:80.5%
x264 [info]: mb B I16..4: 0.2% 0.0% 0.0% B16..8: 5.2% 0.0% 0.0% direct:
5.6% skip:88.9% L0:34.7% L1:65.1% BI: 0.2%
x264 [info]: final ratefactor: 41.96
x264 [info]: direct mvs spatial:98.0% temporal:2.0%
x264 [info]: coded y,uvDC,uvAC intra:1.9% 10.1% 5.3% inter:0.7% 1.3% 0.8%
x264 [info]: kb/s:57.3

encoded 52217 frames, 93.35 fps, 57.44 kb/s

2nd pass with MB-tree

D:\>"c:\Program Files (x86)\megui\tools\x264\x264.exe" --pass 2 --bitrate 55 --
stats "D:\DOTT(mbtree).stats" --keyint 700 --min-keyint 70 --ref 16 --no-fast-ps
kip --b-adapt 2 --b-pyramid --direct auto --subme 9 --trellis 2 --partitions all
--me umh --merange 32 --thread-input --no-dct-decimate --output "D:\DOTT(mbtree
).mp4" "D:\DSS.avs" --fullrange on --rc-lookahead 250
avis [info]: 320x200 @ 70.09 fps (52217 frames)
x264 [warning]: width or height not divisible by 16 (320x200), compression will
suffer.
x264 [warning]: b-pyramid + mb-tree is not supported
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 2.1
mp4 [info]: initial delay 285362 (scale 10000000)
x264 [info]: frame I:96 Avg QP:26.35 size: 6781
x264 [info]: frame P:50087 Avg QP:34.74 size: 86
x264 [info]: frame B:2034 Avg QP:40.48 size: 53
x264 [info]: consecutive B-frames: 93.9% 2.1% 1.7% 2.3%
x264 [info]: mb I I16..4: 29.3% 24.4% 46.3%
x264 [info]: mb P I16..4: 0.7% 0.5% 0.0% P16..4: 2.8% 0.4% 1.3% 0.0% 0
.0% skip:94.2%
x264 [info]: mb B I16..4: 0.1% 0.0% 0.0% B16..8: 8.9% 0.5% 0.1% direct:
0.1% skip:90.2% L0:46.0% L1:53.9% BI: 0.1%
x264 [info]: 8x8 transform intra:40.6% inter:43.9%
x264 [info]: direct mvs spatial:94.1% temporal:5.9%
x264 [info]: coded y,uvDC,uvAC intra:11.3% 20.5% 15.1% inter:0.4% 1.4% 0.7%
x264 [info]: ref P L0 63.8% 6.3% 4.3% 1.6% 1.5% 2.2% 2.6% 3.5% 2.2% 2.
7% 1.6% 1.2% 1.3% 1.3% 2.4% 1.4%
x264 [info]: ref B L0 90.3% 1.2% 0.5% 1.3% 1.5% 1.6% 0.2% 1.4% 0.8% 0.
5% 0.1% 0.4% 0.1% 0.1% 0.2%
x264 [info]: kb/s:54.3

encoded 52217 frames, 46.03 fps, 54.31 kb/s

Screenshots :

Source :
http://img411.imageshack.us/img411/6307/dottsource.png

two passes encode without mb-tree :
http://img21.imageshack.us/img21/2165/dottnombtree.png

two passes encode with mb-tree and --rc-lookahead 250 :
http://img411.imageshack.us/img411/9574/dottmbtree.png
(I could have posted other screenshots, but you can see the difference right away : there's only need to look at the actions verbs' text, almost entirely unreadable without mb-tree. There's also a huge difference on the action phrase, on the old clock, and on Bernard's arms)

DOSBox' video capture used as source (http://www.megaupload.com/?d=1I92DMSY) (13.92 MB)
No mb-tree video (http://www.megaupload.com/?d=IZR6THTA) (8.83 MB)
mb-tree video (http://www.megaupload.com/?d=72H92WHO) (9.33 MB)
(all with sound)

In short : mb-tree is very impressive! :)

[MatLz]

Hi xyloy,
Your encode with mbtree has more bitrate than the without mbtree encode. Can you do a try for an approximately equal bitrate?

[poisondeathray]

It is impressive... but the mb-tree version has 11.5% more bitrate.
EDIT: MatLz beat me ... :)

[LoRd_MuldeR]

And why is the video encoded at 70fps? The motion in that video isn't anywhere near 70fps, it's more like 15fps ;)

[xyloy]

DOSBox encodes it at 70 FPS. I dunno why, but every capture I do with DOSBox' own capture capabilities is at 70 FPS. That's probably why I can't capture 3D games without terrible lags (and DOSBox can use only one CPU..).

Anyway, in short : I don't know why, and it seems to be hardcoded that way DOSBox' code. :confused:



Edit:
Hi xyloy,
Your encode with mbtree has more bitrate than the without mbtree encode. Can you do a try for an approximately equal bitrate?
I'll try with 200 kbps. May be I'm too close to the "minimum bitrate" that x264 guessed for this source (x264's 2nd pass bitrate curve warning message I reported).

[LoRd_MuldeR]

DOSBox encodes it at 70 FPS. I dunno why, but every capture I do with DOSBox' own capture capabilities is at 70 FPS. That's probably why I can't capture 3D games without terrible lags (and DOSBox can use only one CPU..).

Anyway, in short : I don't know why, and it seems to be hardcoded that way DOSBox' code. :confused:



It's probably related to the screen refresh-rate that DOSBox simulates. However you should down-sample the video before encoding it!
http://avisynth.org/mediawiki/ChangeFPS

I guess that 10 fps should be sufficient to retain all motion in that clip. Which means that currently 6/7 of all frames are useless.
If there's no motion between two consecutive frames, these "dummy" frames usually shouldn't eat too many bits.
But on the other hand at those ultra-low bitrates coding those frames, even if they contain no data, may cause a significant overhead...

[MatLz]

[QUOTE=xyloy;1325025]I'll try with 200 kbps.[QUOTE]

But maybe we will not see the difference with a 'high' bitrate like 200kbps.
Just increase the bitrate to 60 for the nombtree encode, maybe the result will be near as 54kbps

[xyloy]

Ok, here it is.

AviSynth script:

DirectShowSource(".\tentacle_000.avi")
ConvertToYV12()


command line results :

D:\>"c:\Program Files (x86)\megui\tools\x264\x264.exe" --pass 1 --bitrate 60 --
stats "D:\DOTT(60kbps-no-mbtree).stats" --keyint 700 --min-keyint 70 --ref 16 --
no-fast-pskip --b-adapt 2 --b-pyramid --direct auto --subme 9 --trellis 2 --part
itions all --me umh --merange 32 --thread-input --no-dct-decimate --output NUL "
D:\DSS.avs" --fullrange on --no-mbtree
avis [info]: 320x200 @ 70.09 fps (52218 frames)
x264 [warning]: width or height not divisible by 16 (320x200), compression will
suffer.
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile Main, level 2.1
x264 [info]: frame I:96 Avg QP:36.04 size: 3332
x264 [info]: frame P:50095 Avg QP:38.86 size: 106
x264 [info]: frame B:2027 Avg QP:46.39 size: 87
x264 [info]: consecutive B-frames: 93.9% 2.1% 1.7% 2.3%
x264 [info]: mb I I16..4: 69.9% 0.0% 30.1%
x264 [info]: mb P I16..4: 18.5% 0.0% 0.0% P16..4: 2.6% 0.0% 0.0% 0.0% 0
.0% skip:78.9%
x264 [info]: mb B I16..4: 0.2% 0.0% 0.0% B16..8: 4.7% 0.0% 0.0% direct:
6.9% skip:88.2% L0:35.2% L1:64.3% BI: 0.5%
x264 [info]: final ratefactor: 50.43
x264 [info]: direct mvs spatial:99.7% temporal:0.3%
x264 [info]: coded y,uvDC,uvAC intra:1.4% 14.1% 6.7% inter:0.9% 1.6% 0.9%
x264 [info]: kb/s:62.1

encoded 52218 frames, 208.90 fps, 62.25 kb/s

D:\>"c:\Program Files (x86)\megui\tools\x264\x264.exe" --pass 2 --bitrate 60 --
stats "D:\DOTT(60kbps-no-mbtree).stats" --keyint 700 --min-keyint 70 --ref 16 --
no-fast-pskip --b-adapt 2 --b-pyramid --direct auto --subme 9 --trellis 2 --part
itions all --me umh --merange 32 --thread-input --no-dct-decimate --output "D:\D
OTT(60kbps-no-mbtree).mp4" "D:\DSS.avs" --fullrange on --no-mbtree
avis [info]: 320x200 @ 70.09 fps (52218 frames)
x264 [warning]: width or height not divisible by 16 (320x200), compression will
suffer.
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 2.1
mp4 [info]: initial delay 285362 (scale 10000000)
x264 [info]: frame I:96 Avg QP:27.30 size: 6234
x264 [info]: frame P:50095 Avg QP:29.80 size: 86
x264 [info]: frame B:2027 Avg QP:35.05 size: 75
x264 [info]: consecutive B-frames: 93.9% 2.1% 1.7% 2.3%
x264 [info]: mb I I16..4: 33.9% 21.9% 44.2%
x264 [info]: mb P I16..4: 0.6% 0.3% 0.1% P16..4: 2.5% 0.4% 1.2% 0.0% 0
.0% skip:94.9%
x264 [info]: mb B I16..4: 0.1% 0.1% 0.0% B16..8: 9.1% 0.6% 0.2% direct:
0.3% skip:89.6% L0:40.8% L1:59.0% BI: 0.3%
x264 [info]: 8x8 transform intra:29.3% inter:35.6%
x264 [info]: direct mvs spatial:96.5% temporal:3.5%
x264 [info]: coded y,uvDC,uvAC intra:13.8% 26.8% 19.0% inter:0.4% 1.5% 0.8%
x264 [info]: ref P L0 72.4% 4.8% 2.9% 1.1% 0.7% 1.4% 2.2% 3.4% 2.0% 2.
3% 1.2% 0.7% 0.8% 0.9% 1.9% 1.2%
x264 [info]: ref B L0 89.2% 2.3% 0.7% 0.5% 0.2% 1.6% 2.0% 1.2% 0.8% 0.
4% 0.2% 0.2% 0.3% 0.3% 0.3%
x264 [info]: ref B L1 97.3% 2.7%
x264 [info]: kb/s:54.5

encoded 52218 frames, 45.05 fps, 54.52 kb/s

D:\>"c:\Program Files (x86)\megui\tools\x264\x264.exe" --pass 1 --bitrate 55 --
stats "D:\DOTT(55kbps-mbtree).stats" --keyint 700 --min-keyint 70 --ref 16 --no-
fast-pskip --b-adapt 2 --b-pyramid --direct auto --subme 9 --trellis 2 --partiti
ons all --me umh --merange 32 --thread-input --no-dct-decimate --output NUL "D:\
DSS.avs" --fullrange on --rc-lookahead 250
avis [info]: 320x200 @ 70.09 fps (52218 frames)
x264 [warning]: width or height not divisible by 16 (320x200), compression will
suffer.
x264 [warning]: b-pyramid + mb-tree is not supported
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile Main, level 2.1
x264 [info]: frame I:96 Avg QP:30.97 size: 4086
x264 [info]: frame P:50088 Avg QP:37.59 size: 96
x264 [info]: frame B:2034 Avg QP:48.12 size: 72
x264 [info]: consecutive B-frames: 93.9% 2.1% 1.7% 2.3%
x264 [info]: mb I I16..4: 54.0% 0.0% 46.0%
x264 [info]: mb P I16..4: 17.4% 0.0% 0.0% P16..4: 2.1% 0.0% 0.0% 0.0% 0
.0% skip:80.5%
x264 [info]: mb B I16..4: 0.2% 0.0% 0.0% B16..8: 5.2% 0.0% 0.0% direct:
5.6% skip:88.9% L0:34.7% L1:65.1% BI: 0.2%
x264 [info]: final ratefactor: 41.96
x264 [info]: direct mvs spatial:98.0% temporal:2.0%
x264 [info]: coded y,uvDC,uvAC intra:1.9% 10.1% 5.3% inter:0.7% 1.3% 0.8%
x264 [info]: kb/s:57.3

encoded 52218 frames, 94.71 fps, 57.44 kb/s

D:\>"c:\Program Files (x86)\megui\tools\x264\x264.exe" --pass 2 --bitrate 55 --
stats "D:\DOTT(55kbps-mbtree).stats" --keyint 700 --min-keyint 70 --ref 16 --no-
fast-pskip --b-adapt 2 --b-pyramid --direct auto --subme 9 --trellis 2 --partiti
ons all --me umh --merange 32 --thread-input --no-dct-decimate --output "D:\DOTT
(55kbps-mbtree).mp4" "D:\DSS.avs" --fullrange on --rc-lookahead 250
avis [info]: 320x200 @ 70.09 fps (52218 frames)
x264 [warning]: width or height not divisible by 16 (320x200), compression will
suffer.
x264 [warning]: b-pyramid + mb-tree is not supported
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 2.1
mp4 [info]: initial delay 285362 (scale 10000000)
x264 [info]: frame I:96 Avg QP:27.02 size: 6432
x264 [info]: frame P:50088 Avg QP:35.69 size: 86
x264 [info]: frame B:2034 Avg QP:40.84 size: 53
x264 [info]: consecutive B-frames: 93.9% 2.1% 1.7% 2.3%
x264 [info]: mb I I16..4: 29.3% 25.5% 45.2%
x264 [info]: mb P I16..4: 0.8% 0.5% 0.0% P16..4: 2.8% 0.4% 1.8% 0.0% 0
.0% skip:93.6%
x264 [info]: mb B I16..4: 0.1% 0.0% 0.0% B16..8: 9.2% 0.5% 0.1% direct:
0.1% skip:90.0% L0:46.5% L1:53.4% BI: 0.1%
x264 [info]: 8x8 transform intra:36.6% inter:44.6%
x264 [info]: direct mvs spatial:94.1% temporal:5.9%
x264 [info]: coded y,uvDC,uvAC intra:10.3% 19.4% 13.7% inter:0.4% 1.4% 0.7%
x264 [info]: ref P L0 66.6% 6.2% 4.3% 1.5% 1.4% 1.9% 2.3% 3.1% 2.0% 2.
5% 1.4% 1.1% 1.2% 1.2% 2.1% 1.2%
x264 [info]: ref B L0 89.5% 1.3% 0.6% 1.1% 1.9% 2.0% 0.2% 1.4% 0.7% 0.
4% 0.1% 0.4% 0.1% 0.1% 0.1%
x264 [info]: kb/s:54.2

encoded 52218 frames, 43.81 fps, 54.25 kb/s

Screenshots :

Source :
http://img36.imageshack.us/img36/7026/dottsource2.png
two passes at 60 kbps without mb-tree :
http://img179.imageshack.us/img179/2165/dottnombtree.png
two passes at 55 kbps with mb-tree and --rc-look-ahead 250 :
http://img245.imageshack.us/img245/9120/dott55kbpsmbtree.png

Files :
No-mbtree version at 60 kbps (http://rapidshare.de/files/48336024/DOTT_60kbps-no-mbtree_.mkv.html)
mb-tree version at 55 kbps (http://rapidshare.de/files/48335993/DOTT_55kbps-mbtree_.mkv.html)

[MatLz]

Well, it's not for annoying you, but when I said you to change the bitrate to 60, it was for exactly the same source (number of frames, fps)
And now the difference is not really noticeable cause divide fps by 7 is near multiply bitrate by 7 in term of quality

[LoRd_MuldeR]

That clip is really small. Maybe you could apply something like NNEDI2_rpow2(rfactor=2) (http://web.missouri.edu/~kes25c/nnedi2.zip) before encoding it.

At the upscaled version differences may become more easy to spot...

[xyloy]

Well, it's not for annoying you, but when I said you to change the bitrate to 60, it was for exactly the same source (number of frames, fps)
:o

I'll edit my previous post, then.

And now the difference is not really noticeable cause divide fps by 7 is near multiply bitrate by 7 in term of quality
Yep, that's what I figured.

That clip is really small. Maybe you could apply something like NNEDI2_rpow2(rfactor=2) before encoding it.
Sounds interesting, I'll give it a try.

[MatLz]

(Out of topic)
Why the encoders add lot lot lot lot of colors if the source is only count 256 colors?