x264 "Macroblock Tree Ratecontrol" testing (committed)

Dark Shikari · 3rd August 2009, 01:17

Warning: (not really) Highly Experimental

It's done and committed! Grab the latest x264 from x264.nl!

Patch RC1
Download RC1 (Windows, old)
Download RC1 (Windows debug build, old)
Patch v0.13
Download 0.13 (Windows)
Download 0.13 (Windows debug build)
Patch v0.11
Download 0.11 (Windows, old)
Download 0.11 (Windows debug build, old)
Patch v0.09 (old)
Download 0.09 (Windows, old)
Download 0.09 (Windows debug build, old)

Updates:

Changes in RC1:

1. Statsfile format changed; now takes only 2 bytes per macroblock. Varying-endianness now supported. Error handling improved.

2. More commandline validation fixes.

3. Clip propagation factors just in case they overflow.

4. Change default lookahead to 40 to save some memory.

5. It's now called --rc-lookahead, not --lookahead, at Akupenguin's request.

6. Various other things, just read the damn commit message in the patch.

Changes in 0.13:

1. Bugs fixed in the VBV patch. It's still separate, as per below.

2. B-adapt 1 now supported.

3. All B-adapt code is now merged into MB-tree code instead of duplicated. B-adapt 1 should be slightly better than it was before (even without MB-tree). In general, all three B-adapt modes now use the same basic framework.

4. MB-tree strength is now adjustable through qcomp. Raising qcomp lowers MB-tree strength; it works the same way that qcomp did before MB-tree, with qcomp 1 meaning constant frame quantizer. This lets you experiment a bit more; there's some possibility that the default qcomp 0.6 is not the psy-optimal value--it might be higher.

5. New --no-psy option, which disables all psy optimizations in x264 that don't improve PSNR or SSIM. This includes psy-RD/trellis, but also the recently-added chroma lambda offset and the magical "num_frames = j" line that causes MB-tree to pretend that non-scenecut I-frames don't exist.

6. "Fast" preset is now slower, "faster" preset added to cover the gap.

7. MB-tree is now default on presets "fast" and higher. Lookahead values are smaller for faster presets.

Changes in 0.11:

1. VBV now supported. Includes a separate VBV patch which fixes most VBV issues that MB-tree has; this will be committed separately, as it doesn't require MB-tree. It should also improve 1-pass VBV (especially CBR) performance on clips with lots of static content, e.g. anime, CGI.

Do note that CBR doesn't benefit much from MB-tree though, especially with small buffer sizes.

2. Lots of trivial parameter-handling fixes, e.g. lookahead automatically disabled on second pass.

FAQ:

How do I use it?

It's on by default. --no-mbtree turns it off.

What doesn't it work with yet?

--b-pyramid

How can I tweak it?

--lookahead; default is 50, higher takes more memory and is slower, but should give better results. It's probably overkill for most sources. If you get crashes immediately after the lookahead fills up, you probably ran out of memory (max is 2GB for x264 on 32-bit Windows).

--qcomp; works the same as it did before with actual qcomp, higher numbers lower MB-tree strength.

How much does it cost, speed-wise?

A bit. Not too much though. Speed cost is linearly proportional to --lookahead.

How can I help test it?

Use it.

Make sure you don't get any odd errors or warnings, such as asserts failing, crashes, etc. Try various combinations of the relevant settings (b-adapt, bframes, mbtree, lookahead, pass 1/2, crf, keyint, scenecut) that I might not have thought of.

Try it on various sources; see if you can find places where, with --tune ssim, at the same bitrate as without mbtree, it gives worse SSIM (or, equally, places where Global PSNR goes down with tune psnr).

Make sure it doesn't fall apart visually either.

Also, note that CRF is redefined again by this patch--I set it so that it kept the same bitrate on Blackpearl as before, but it may vary for different videos.

What does it actually do?

It tracks the propagation of information from future blocks to past blocks across motion vectors. It could be described as localizing qcomp to act on individual blocks instead of whole scenes. Thus instead of lowering quality in high-complexity scenes (like x264 currently does), it'll only lower quality on the complex part of the scene, while for example a static background will remain high-quality. It also has many other more subtle effects, some potentially negative, most probably not.

This helps at all bitrates, but can even help at phenomenally low bitrates where the video would otherwise fall apart completely, like in this 67kbps anime clip.

Should I go around distributing this patch and getting everyone I know to use it for their everyday encodes?

No. The commandline options here may change in the future, as may the statsfile format and various other aspects of the patch, so it's not a good idea to make this patch something that gets put in ordinary x264 builds. Furthermore, I might have broken something unrelated to mbtree.

Why does it give different output even without mbtree?

B-adapt (all modes) was slightly modified to work better with mbtree, so the frametype decisions may change somewhat.

What improvement does it give?

I've gotten up to a 70% SSIM increase.

You're joking, right?

No.

Seriously? How the hell can you get a 70% quality increase?

Magic.

OK, you're just making this u--wait a minute, what's this new statsfile, and why is it using up all my hard disk space?

Oh, that? Kyahahahahahaha.

Snowknight26 · 3rd August 2009, 01:56

You know you want to provide a 64-bit Windows binary.

Any reason why the exe is so tiny?

I have to say, clever picture..

Dark Shikari · 3rd August 2009, 01:57

Quote:

Originally Posted by Snowknight26

You know you want to provide a 64-bit Windows binary.

try this

Quote:

Originally Posted by Snowknight26

Any reason why the exe is so tiny?

kkrunchy

Quote:

Originally Posted by Snowknight26

I have to say, clever picture..

Kuukunen made that.

Audionut · 3rd August 2009, 01:59

Quote:

Originally Posted by Dark Shikari

Magic.

Black?

Snowknight26 · 3rd August 2009, 02:10

Quote:

Originally Posted by Dark Shikari

kkrunchy

Farbrausch, should have known. I dare say I'm more impressed by that than the mbtree.

Funny how the QP for the mbtree version is twofold that of a normal build:

Code:

C:\temp\Encoding\test>cd ..

C:\temp\Encoding>avs2yuv -raw test/test.avs -   | x264_mbtree_x64 --crf 19 --mbt
ree --bframes 4 --b-adapt 2 --ref 4 --b-pyramid  --direct auto --deblock -3:-3 -
-subme 10 --trellis 2 --analyse all --no-dct-decimate --me umh --ssim --fps 2400
0/1001 --output "test/test.mbtree.mkv" - 480x320
x264 [info]: 480x320 @ 23.98 fps
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 2.1
test/test.avs: 480x320, 10000000/417083 fps, 481 frames
x264 [info]: slice I:240   Avg QP:20.82  size:    79
x264 [info]: slice P:241   Avg QP:24.08  size:    79
x264 [info]: consecutive B-frames: 100.0%  0.0%  0.0%  0.0%  0.0%
x264 [info]: mb I  I16..4: 100.0%  0.0%  0.0%
x264 [info]: mb P  I16..4: 90.4%  0.0%  0.0%  P16..4:  9.2%  0.0%  0.0%  0.0%  0
.0%    skip: 0.4%
x264 [info]: 8x8 transform  intra:0.0%  inter:-1.$%
x264 [info]: coded y,uvDC,uvAC intra:0.0% 0.1% 0.0% inter:0.0% 0.0% 0.0%
x264 [info]: ref P L0   0.0% 100.0%
x264 [info]: SSIM Mean Y:1.0000000
x264 [info]: kb/s:15.2

encoded 481 frames, 151.83 fps, 16.87 kb/s

C:\temp\Encoding>avs2yuv -raw test/test.avs -   | x264.x64 --crf 19 --bframes 4
--b-adapt 2 --ref 4 --b-pyramid  --direct auto --deblock -3:-3 --subme 10 --trel
lis 2 --analyse all --no-dct-decimate --me umh --ssim --fps 24000/1001 --output
"test/test.mkv" - 480x320
x264 [info]: 480x320 @ 23.98 fps
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 2.1
test/test.avs: 480x320, 10000000/417083 fps, 481 frames
x264 [info]: slice I:240   Avg QP:10.04  size:    83
x264 [info]: slice P:241   Avg QP:10.01  size:    91
x264 [info]: consecutive B-frames: 100.0%  0.0%  0.0%  0.0%  0.0%
x264 [info]: mb I  I16..4: 100.0%  0.0%  0.0%
x264 [info]: mb P  I16..4: 90.4%  0.0%  0.0%  P16..4:  9.2%  0.0%  0.0%  0.0%  0
.0%    skip: 0.4%
x264 [info]: 8x8 transform  intra:0.0%  inter:-1.$%
x264 [info]: coded y,uvDC,uvAC intra:0.0% 0.1% 0.0% inter:0.0% 0.0% 0.0%
x264 [info]: ref P L0   0.0% 100.0%
x264 [info]: SSIM Mean Y:1.0000000
x264 [info]: kb/s:16.7

encoded 481 frames, 162.39 fps, 18.08 kb/s

C:\temp\Encoding>pause
Press any key to continue . . .

And what did you mean by doesn't work with --b-pyramid? Different outputs seems to show that it does something differently.

Dark Shikari · 3rd August 2009, 02:11

Quote:

Originally Posted by Audionut

Black?

Of course.

IgorC · 3rd August 2009, 02:11

Quote:

--lookahead; default is 50

Is lookahead measured in frames? 50 frames?

Great patch. Untill now I can say I get +4% of bitrate improvement for the same OPSNR and +20% for the same SSIM (foreman test sequence).

Dark Shikari · 3rd August 2009, 02:12

Quote:

Originally Posted by Snowknight26

Farbrausch, should have known. I dare say I'm more impressed by that than the mbtree.

Funny how the QP for the mbtree version is twofold that of a normal build:

Code:

C:\temp\Encoding\test>cd ..

C:\temp\Encoding>avs2yuv -raw test/test.avs -   | x264_mbtree_x64 --crf 19 --mbt
ree --bframes 4 --b-adapt 2 --ref 4 --b-pyramid  --direct auto --deblock -3:-3 -
-subme 10 --trellis 2 --analyse all --no-dct-decimate --me umh --ssim --fps 2400
0/1001 --output "test/test.mbtree.mkv" - 480x320
x264 [info]: 480x320 @ 23.98 fps
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 2.1
test/test.avs: 480x320, 10000000/417083 fps, 481 frames
x264 [info]: slice I:240   Avg QP:20.82  size:    79
x264 [info]: slice P:241   Avg QP:24.08  size:    79
x264 [info]: consecutive B-frames: 100.0%  0.0%  0.0%  0.0%  0.0%
x264 [info]: mb I  I16..4: 100.0%  0.0%  0.0%
x264 [info]: mb P  I16..4: 90.4%  0.0%  0.0%  P16..4:  9.2%  0.0%  0.0%  0.0%  0
.0%    skip: 0.4%
x264 [info]: 8x8 transform  intra:0.0%  inter:-1.$%
x264 [info]: coded y,uvDC,uvAC intra:0.0% 0.1% 0.0% inter:0.0% 0.0% 0.0%
x264 [info]: ref P L0   0.0% 100.0%
x264 [info]: SSIM Mean Y:1.0000000
x264 [info]: kb/s:15.2

encoded 481 frames, 151.83 fps, 16.87 kb/s

C:\temp\Encoding>avs2yuv -raw test/test.avs -   | x264.x64 --crf 19 --bframes 4
--b-adapt 2 --ref 4 --b-pyramid  --direct auto --deblock -3:-3 --subme 10 --trel
lis 2 --analyse all --no-dct-decimate --me umh --ssim --fps 24000/1001 --output
"test/test.mkv" - 480x320
x264 [info]: 480x320 @ 23.98 fps
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 2.1
test/test.avs: 480x320, 10000000/417083 fps, 481 frames
x264 [info]: slice I:240   Avg QP:10.04  size:    83
x264 [info]: slice P:241   Avg QP:10.01  size:    91
x264 [info]: consecutive B-frames: 100.0%  0.0%  0.0%  0.0%  0.0%
x264 [info]: mb I  I16..4: 100.0%  0.0%  0.0%
x264 [info]: mb P  I16..4: 90.4%  0.0%  0.0%  P16..4:  9.2%  0.0%  0.0%  0.0%  0
.0%    skip: 0.4%
x264 [info]: 8x8 transform  intra:0.0%  inter:-1.$%
x264 [info]: coded y,uvDC,uvAC intra:0.0% 0.1% 0.0% inter:0.0% 0.0% 0.0%
x264 [info]: ref P L0   0.0% 100.0%
x264 [info]: SSIM Mean Y:1.0000000
x264 [info]: kb/s:16.7

encoded 481 frames, 162.39 fps, 18.08 kb/s

C:\temp\Encoding>pause
Press any key to continue . . .

It looks like your source failed, because your input was all black

Quote:

Originally Posted by IgorC

Is lookahead measured in frames? 50 frames?

Yes.

Snowknight26 · 3rd August 2009, 02:14

Quote:

Originally Posted by Dark Shikari

It looks like your source failed, because your input was all black

It's not actually.

Crash after 100 frames:

Code:

C:\temp\Encoding\test>cd ..

C:\temp\Encoding>avs2yuv -raw test/test.avs -   | x264_mbtree_x64 --crf 19 --mbt
ree --lookahead 500 --bframes 16 --b-adapt 2 --ref 4 --b-pyramid  --direct auto
--deblock -3:-3 --subme 10 --trellis 2 --analyse all --no-dct-decimate --me umh
--ssim --fps 24000/1001 --output "test/test.mbtree.mkv" - 1920x816
x264 [info]: 1920x816 @ 23.98 fps
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 4.0
test/test.avs: 1920x816, 10000000/417083 fps, 1378 frames
Output error: wrote only 1785423 of 2350080 bytes

C:\temp\Encoding>avs2yuv -raw test/test.avs -   | x264.x64 --crf 19 --bframes 16
 --b-adapt 2 --ref 4 --b-pyramid  --direct auto --deblock -3:-3 --subme 10 --tre
llis 2 --analyse all --no-dct-decimate --me umh --ssim --fps 24000/1001 --output
 "test/test.mkv" - 1920x816
x264 [info]: 1920x816 @ 23.98 fps
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 4.0
test/test.avs: 1920x816, 10000000/417083 fps, 1378 frames
250 frames: 2.03 fps, 7743.86 kb/s

Other one is still going.. samples a bit big so hopefully someone can provide a debug build before I resort to uploading it.

Dark Shikari · 3rd August 2009, 02:33

All crashes that I've seen so far that occurred with very large --lookahead values traced back perfectly to x264_malloc, so you should check with gdb before reporting such issues as bugs.

Snowknight26 · 3rd August 2009, 02:50

I have to say, the bitrate difference can be enormous sometimes:

Code:

C:\temp\Encoding>avs2yuv -raw test/test.avs -   | x264_mbtree_x64 --crf 19 --mbt
ree --bframes 0 --b-adapt 0 --ref 0 --direct spatial --deblock -3:-3 --subme 1 -
-trellis 0 --analyse all --me umh --ssim --fps 24000/1001 --output "test/test.mb
tree.mkv" - 1920x816
x264 [info]: 1920x816 @ 23.98 fps
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 4.0
test/test.avs: 1920x816, 10000000/417083 fps, 1378 frames
x264 [info]: slice I:20    Avg QP:14.02  size:131728
x264 [info]: slice P:1358  Avg QP:15.01  size: 37773
x264 [info]: mb I  I16..4: 24.8% 47.2% 28.0%
x264 [info]: mb P  I16..4: 22.4% 18.9%  1.6%  P16..4: 35.4%  8.6%  1.8%  0.1%  0
.0%    skip:11.2%
x264 [info]: 8x8 transform  intra:44.2%  inter:41.4%
x264 [info]: coded y,uvDC,uvAC intra:35.8% 87.5% 74.4% inter:13.7% 58.9% 19.6%
x264 [info]: SSIM Mean Y:0.9929551
x264 [info]: kb/s:7506.7

encoded 1378 frames, 30.29 fps, 7506.94 kb/s

C:\temp\Encoding>avs2yuv -raw test/test.avs -   | x264.x64 --crf 19 --bframes 0
--b-adapt 0 --ref 0 --direct spatial --deblock -3:-3 --subme 1 --trellis 0 --ana
lyse all --me umh --ssim --fps 24000/1001 --output "test/test.mkv" - 1920x816
x264 [info]: 1920x816 @ 23.98 fps
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 4.0
test/test.avs: 1920x816, 10000000/417083 fps, 1378 frames
x264 [info]: slice I:21    Avg QP:10.66  size:187241
x264 [info]: slice P:1357  Avg QP:11.45  size: 73091
x264 [info]: mb I  I16..4: 23.5% 37.2% 39.3%
x264 [info]: mb P  I16..4: 20.8% 17.8%  5.0%  P16..4: 32.7% 10.9%  4.0%  0.4%  0
.1%    skip: 8.4%
x264 [info]: 8x8 transform  intra:40.7%  inter:31.8%
x264 [info]: coded y,uvDC,uvAC intra:51.6% 88.9% 81.5% inter:36.5% 73.8% 41.9%
x264 [info]: SSIM Mean Y:0.9953622
x264 [info]: kb/s:14353.1

encoded 1378 frames, 26.12 fps, 14353.34 kb/s

Audionut · 3rd August 2009, 02:58

Well you're shooting for crf 19 and getting AVG QP's of 10.

Dark Shikari · 3rd August 2009, 03:00

Quote:

Originally Posted by Snowknight26

I have to say, the bitrate difference can be enormous sometimes:

Or... maybe it's a bug?

Upload a sample of the output.

Snowknight26 · 3rd August 2009, 03:18

http://stfcc.org/misc/test.mbtree.10mb.mkv
http://stfcc.org/misc/test.10mb.mkv

Dark Shikari · 3rd August 2009, 03:39

Quote:

Originally Posted by Snowknight26

http://stfcc.org/misc/test.mbtree.10mb.mkv
http://stfcc.org/misc/test.10mb.mkv

Nope, it seems like MB-tree is in the right here; x264 was lowering the quantizers way too much before for its own good.

Snowknight26 · 3rd August 2009, 04:15

Quote:

Originally Posted by Dark Shikari

All crashes that I've seen so far that occurred with very large --lookahead values traced back perfectly to x264_malloc, so you should check with gdb before reporting such issues as bugs.

Couldn't get 64-bit x264+avs2yuv working with gdb but nevertheless, seems you were right:

Code:

C:\temp\Encoding\test>cd ..

C:\temp\Encoding>gdb --args x264_mbtree_debug --crf 19 --mbtree --lookahead 500
--bframes 16 --b-adapt 2 --ref 4 --b-pyramid  --direct auto --deblock -3:-3 --su
bme 10 --trellis 2 --analyse all --no-dct-decimate --me umh --ssim --fps 24000/1
001 --output "test/test.mbtree.mkv" "test/test.avs"
GNU gdb 6.8
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-pc-mingw32"...
(gdb) run
Starting program: C:\temp\Encoding/x264_mbtree_debug.exe --crf 19 --mbtree --loo
kahead 500 --bframes 16 --b-adapt 2 --ref 4 --b-pyramid --direct auto --deblock
-3:-3 --subme 10 --trellis 2 --analyse all --no-dct-decimate --me umh --ssim --f
ps 24000/1001 --output test/test.mbtree.mkv test/test.avs
[New thread 4380.0x10c0]
Error: dll starting at 0x76f10000 not found.
Error: dll starting at 0x768d0000 not found.
Error: dll starting at 0x76f10000 not found.
Error: dll starting at 0x76e10000 not found.
warning: DllMain: hModule=0x10000000, ulReason=1, lpReserved=0x00000000, gRefCnt
 = 0

warning: DllGetClassObject() CLSID: CAVIFileSynth

warning: 00943450->CAVIFileSynth::CAVIFileSynth()

warning: 00943450->CAVIFileSynth::AddRef() gRefCnt=1, m_refs=1

warning: 00943450->CAVIFileSynth::QueryInterface() {00000001-0000-0000-c000-0000
00000046} (IClassFactory)

warning: 00943450->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2

warning: 00943450->CAVIFileSynth::Release() gRefCnt=1, m_refs=1

warning: DllGetClassObject() result=0x0, object=00943458

warning: 00943450->CAVIFileSynth::CreateInstance()

warning: 009434A8->CAVIFileSynth::CAVIFileSynth()

warning: 009434A8->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=1

warning: 009434A8->CAVIFileSynth::QueryInterface() {00000000-0000-0000-c000-0000
00000046} (IUnknown)

warning: 009434A8->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2

warning: 009434A8->CAVIFileSynth::Release() gRefCnt=2, m_refs=1

warning: 00943450->CAVIFileSynth::CreateInstance() result=0x0, object=009434A8

warning: 009434A8->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=2

warning: 009434A8->CAVIFileSynth::Release() gRefCnt=2, m_refs=1

warning: 00943450->CAVIFileSynth::Release() gRefCnt=1, m_refs=0

warning: 00943450->CAVIFileSynth::~CAVIFileSynth(), gRefCnt = 1

warning: 009434A8->CAVIFileSynth::QueryInterface() {00000000-0000-0000-c000-0000
00000046} (IUnknown)

warning: 009434A8->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2

warning: 009434A8->CAVIFileSynth::Release() gRefCnt=1, m_refs=1

warning: 009434A8->CAVIFileSynth::QueryInterface() {00020025-0000-0000-c000-0000
00000046} (unsupported!)

warning: 009434A8->CAVIFileSynth::QueryInterface() {0000010b-0000-0000-c000-0000
00000046} (IPersistFile)

warning: 009434A8->CAVIFileSynth::AddRef() gRefCnt=2, m_refs=2

warning: 009434A8->CAVIFileSynth::QueryInterface() {00020020-0000-0000-c000-0000
00000046} (IAVIFile)

warning: 009434A8->CAVIFileSynth::AddRef() gRefCnt=3, m_refs=3

warning: 009434A8->CAVIFileSynth::Load("test/test.avs", 0x0)

warning: 009434A8->CAVIFileSynth::Release() gRefCnt=2, m_refs=2

warning: 009434A8->CAVIFileSynth::Release() gRefCnt=1, m_refs=1

warning: 009434A8->CAVIFileSynth::GetStream(*, 73646976(vids), 0)

[New thread 4380.0xd1c]
warning: DllMain: hModule=0x10000000, ulReason=2, lpReserved=0x00000000, gRefCnt
 = 1

[New thread 4380.0x13c8]
[New thread 4380.0xb7c]
warning: DllMain: hModule=0x10000000, ulReason=2, lpReserved=0x00000000, gRefCnt
 = 1

warning: DllMain: hModule=0x10000000, ulReason=2, lpReserved=0x00000000, gRefCnt
 = 1

[New thread 4380.0x13f8]
[New thread 4380.0x10bc]
[New thread 4380.0x8b4]
warning: DllMain: hModule=0x10000000, ulReason=2, lpReserved=0x00000000, gRefCnt
 = 1

[New thread 4380.0xb24]
[New thread 4380.0x80]
warning: DllMain: hModule=0x10000000, ulReason=2, lpReserved=0x00000000, gRefCnt
 = 1

warning: DllMain: hModule=0x10000000, ulReason=2, lpReserved=0x00000000, gRefCnt
 = 1

[New thread 4380.0x644]
warning: DllMain: hModule=0x10000000, ulReason=2, lpReserved=0x00000000, gRefCnt
 = 1

warning: DllMain: hModule=0x10000000, ulReason=2, lpReserved=0x00000000, gRefCnt
 = 1

[...]


warning: DllMain: hModule=0x10000000, ulReason=3, lpReserved=0x00000000, gRefCnt
 = 2

[New thread 4380.0xfb8]35.18 fps, 0.00 kb/s, eta 0:00:36
warning: DllMain: hModule=0x10000000, ulReason=2, lpReserved=0x00000000, gRefCnt
 = 2

warning: DllMain: hModule=0x10000000, ulReason=3, lpReserved=0x00000000, gRefCnt
 = 2

[New thread 4380.0xac8]35.08 fps, 0.00 kb/s, eta 0:00:36
warning: DllMain: hModule=0x10000000, ulReason=2, lpReserved=0x00000000, gRefCnt
 = 2

warning: DllMain: hModule=0x10000000, ulReason=3, lpReserved=0x00000000, gRefCnt
 = 2

[New thread 4380.0xed0]34.99 fps, 0.00 kb/s, eta 0:00:36

Program received signal SIGSEGV, Segmentation fault.
x264_malloc (i_size=6983680) at common/common.c:729
729     common/common.c: No such file or directory.
        in common/common.c
(gdb) bt
#0  x264_malloc (i_size=6983680) at common/common.c:729
#1  0x00424229 in x264_frame_new (h=<incomplete type>) at common/frame.c:69
#2  0x00427bdb in x264_frame_pop_unused (h=<incomplete type>)
    at common/frame.c:959
#3  0x0040de2c in x264_encoder_encode (h=<incomplete type>, pp_nal=0x28fa78,
    pi_nal=0x28fa7c, pic_in=0x28fc60, pic_out=0x28fa80)
    at encoder/encoder.c:1454
#4  0x00402fdd in Encode_frame (h=0x70cb2a0, hout=0x942e28, pic=0x28fc60)
    at x264.c:1043
#5  0x00403335 in Encode (param=0x28fd00, opt=0x28fce0) at x264.c:1127
#6  0x004013c0 in main (argc=32, argv=0x9431a8) at x264.c:113
(gdb)

imk · 3rd August 2009, 04:59

Here's my quick tests:

http://pastie.org/569295
http://imk.cx/temp/dead_space_mbtree0.mkv (13,94 MB)
http://imk.cx/temp/dead_space_mbtree1.mkv (13,95 MB)

SSIM
0.9728932 -> 0.9740763

Here's two comparison shots:

Dark_Shikari let me know that single frame comparisons are useless for ratecontrol comparisons, but with the above shot, you can see that it brings out some more detail in the darker areas.

thewebchat · 3rd August 2009, 05:48

Honestly, the only difference I see in that screenshot is that the position of the blocking artifacts shifted.

Audionut · 3rd August 2009, 07:27

Pass 2 is slower with --b-adapt 2

benwaggoner · 3rd August 2009, 07:30

Quote:

Originally Posted by Dark Shikari;1310980[b

What does it actually do?[/b]

It tracks the propagation of information from future blocks to past blocks across motion vectors. It could be described as localizing qcomp to act on individual blocks instead of whole scenes. Thus instead of lowering quality in high-complexity scenes (like x264 currently does), it'll only lower quality on the complex part of the scene, while for example a static background will remain high-quality. It also has many other more subtle effects, some potentially negative, most probably not.

Cool. When you had me check out that Touhou clip a while ago, I figured something like this would be what was required. Have you tried it with this build?

The only decent looking encode of that I ever got was using PEP and ROI encoding, specifying a max QP for the static parts of the image so that the crazy motion parts wouldn't starve the background of bits and drive up their QP so high. Once encoded at a decent QP on the I-frame, they would be pretty much skip blocks the rest of the GOP and so didn't take up material bandwidth anyway.

Cool to see something that could achieve similar results without manual ROI and with moving images.

3rd August 2009, 01:17	#1 \| Link
Dark Shikari x264 developer Join Date: Sep 2005 Posts: 8,667	x264 "Macroblock Tree Ratecontrol" testing (committed) Warning: (not really) Highly Experimental It's done and committed! Grab the latest x264 from x264.nl! Patch RC1 Download RC1 (Windows, old) Download RC1 (Windows debug build, old) Patch v0.13 Download 0.13 (Windows) Download 0.13 (Windows debug build) Patch v0.11 Download 0.11 (Windows, old) Download 0.11 (Windows debug build, old) Patch v0.09 (old) Download 0.09 (Windows, old) Download 0.09 (Windows debug build, old) Updates: Changes in RC1: 1. Statsfile format changed; now takes only 2 bytes per macroblock. Varying-endianness now supported. Error handling improved. 2. More commandline validation fixes. 3. Clip propagation factors just in case they overflow. 4. Change default lookahead to 40 to save some memory. 5. It's now called --rc-lookahead, not --lookahead, at Akupenguin's request. 6. Various other things, just read the damn commit message in the patch. Changes in 0.13: 1. Bugs fixed in the VBV patch. It's still separate, as per below. 2. B-adapt 1 now supported. 3. All B-adapt code is now merged into MB-tree code instead of duplicated. B-adapt 1 should be slightly better than it was before (even without MB-tree). In general, all three B-adapt modes now use the same basic framework. 4. MB-tree strength is now adjustable through qcomp. Raising qcomp lowers MB-tree strength; it works the same way that qcomp did before MB-tree, with qcomp 1 meaning constant frame quantizer. This lets you experiment a bit more; there's some possibility that the default qcomp 0.6 is not the psy-optimal value--it might be higher. 5. New --no-psy option, which disables all psy optimizations in x264 that don't improve PSNR or SSIM. This includes psy-RD/trellis, but also the recently-added chroma lambda offset and the magical "num_frames = j" line that causes MB-tree to pretend that non-scenecut I-frames don't exist. 6. "Fast" preset is now slower, "faster" preset added to cover the gap. 7. MB-tree is now default on presets "fast" and higher. Lookahead values are smaller for faster presets. Changes in 0.11: 1. VBV now supported. Includes a separate VBV patch which fixes most VBV issues that MB-tree has; this will be committed separately, as it doesn't require MB-tree. It should also improve 1-pass VBV (especially CBR) performance on clips with lots of static content, e.g. anime, CGI. Do note that CBR doesn't benefit much from MB-tree though, especially with small buffer sizes. 2. Lots of trivial parameter-handling fixes, e.g. lookahead automatically disabled on second pass. FAQ: How do I use it? It's on by default. --no-mbtree turns it off. What doesn't it work with yet? --b-pyramid How can I tweak it? --lookahead; default is 50, higher takes more memory and is slower, but should give better results. It's probably overkill for most sources. If you get crashes immediately after the lookahead fills up, you probably ran out of memory (max is 2GB for x264 on 32-bit Windows). --qcomp; works the same as it did before with actual qcomp, higher numbers lower MB-tree strength. How much does it cost, speed-wise? A bit. Not too much though. Speed cost is linearly proportional to --lookahead. How can I help test it? Use it. Make sure you don't get any odd errors or warnings, such as asserts failing, crashes, etc. Try various combinations of the relevant settings (b-adapt, bframes, mbtree, lookahead, pass 1/2, crf, keyint, scenecut) that I might not have thought of. Try it on various sources; see if you can find places where, with --tune ssim, at the same bitrate as without mbtree, it gives worse SSIM (or, equally, places where Global PSNR goes down with tune psnr). Make sure it doesn't fall apart visually either. Also, note that CRF is redefined again by this patch--I set it so that it kept the same bitrate on Blackpearl as before, but it may vary for different videos. What does it actually do? It tracks the propagation of information from future blocks to past blocks across motion vectors. It could be described as localizing qcomp to act on individual blocks instead of whole scenes. Thus instead of lowering quality in high-complexity scenes (like x264 currently does), it'll only lower quality on the complex part of the scene, while for example a static background will remain high-quality. It also has many other more subtle effects, some potentially negative, most probably not. This helps at all bitrates, but can even help at phenomenally low bitrates where the video would otherwise fall apart completely, like in this 67kbps anime clip. Should I go around distributing this patch and getting everyone I know to use it for their everyday encodes? No. The commandline options here may change in the future, as may the statsfile format and various other aspects of the patch, so it's not a good idea to make this patch something that gets put in ordinary x264 builds. Furthermore, I might have broken something unrelated to mbtree. Why does it give different output even without mbtree? B-adapt (all modes) was slightly modified to work better with mbtree, so the frametype decisions may change somewhat. What improvement does it give? I've gotten up to a 70% SSIM increase. You're joking, right? No. Seriously? How the hell can you get a 70% quality increase? Magic. OK, you're just making this u--wait a minute, what's this new statsfile, and why is it using up all my hard disk space? Oh, that? Kyahahahahahaha. __________________ Follow x264 development progress \| akupenguin quotes \| x264 git status ffmpeg and x264-related consulting/coding contracts \| Doom10 Last edited by Dark Shikari; 10th August 2009 at 22:53.

3rd August 2009, 02:33	#10 \| Link
Dark Shikari x264 developer Join Date: Sep 2005 Posts: 8,667	All crashes that I've seen so far that occurred with very large --lookahead values traced back perfectly to x264_malloc, so you should check with gdb before reporting such issues as bugs. __________________ Follow x264 development progress \| akupenguin quotes \| x264 git status ffmpeg and x264-related consulting/coding contracts \| Doom10

3rd August 2009, 02:58	#12 \| Link
Audionut Registered User Join Date: Nov 2003 Posts: 1,280	Well you're shooting for crf 19 and getting AVG QP's of 10. __________________ http://www.7-zip.org/

3rd August 2009, 04:59	#17 \| Link
imk \|ﾝ､) Join Date: Feb 2008 Posts: 77	Here's my quick tests: http://pastie.org/569295 http://imk.cx/temp/dead_space_mbtree0.mkv (13,94 MB) http://imk.cx/temp/dead_space_mbtree1.mkv (13,95 MB) SSIM 0.9728932 -> 0.9740763 Here's two comparison shots: Dark_Shikari let me know that single frame comparisons are useless for ratecontrol comparisons, but with the above shot, you can see that it brings out some more detail in the darker areas. Last edited by imk; 3rd August 2009 at 05:26.

3rd August 2009, 07:27	#19 \| Link
Audionut Registered User Join Date: Nov 2003 Posts: 1,280	Pass 2 is slower with --b-adapt 2 __________________ http://www.7-zip.org/

3rd August 2009, 01:56	#2 \| Link
Snowknight26 Registered User Join Date: Aug 2007 Posts: 1,426	You know you want to provide a 64-bit Windows binary. Any reason why the exe is so tiny? I have to say, clever picture..

3rd August 2009, 03:18	#14 \| Link
Snowknight26 Registered User Join Date: Aug 2007 Posts: 1,426	http://stfcc.org/misc/test.mbtree.10mb.mkv http://stfcc.org/misc/test.10mb.mkv

3rd August 2009, 05:48	#18 \| Link
thewebchat Advanced Blogging Join Date: May 2009 Posts: 483	Honestly, the only difference I see in that screenshot is that the position of the blocking artifacts shifted.

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode