x264 "Macroblock Tree Ratecontrol" testing (committed)

[juGGaKNot]

x264_x86_r1195_juGGaKNot (http://www.mediafire.com/download.php?mfy1vjjqaym)
GCC 4.4.0 generic, fprofiled, patched
Source: GIT

Applied patches :

x264_win_zone_parse_fix_05.diff (http://www.mediafire.com/download.php?kqymujywhtw)
x264_hrd_pulldown.15_interlace.diff (http://www.mediafire.com/download.php?ejhnmitmjt3)
x264_Macroblock_Tree_Ratecontrol_011.diff (http://www.mediafire.com/download.php?nlmj3d4t3yd)

Please check Doom9.org patches thread (http://forum.doom9.org/showthread.php?t=130364), Doom9.org Macroblock tree Ratecontrol thread (http://forum.doom9.org/showthread.php?t=148686) and GIT shortlog (http://git.videolan.org/gitweb.cgi?p=x264.git;a=shortlog) for more info.

Compiled by juGGaKNot (http://forum.doom9.org/member.php?u=144865) on August 04-2009, 12:20:00 GMT with GCC 4.4.0 on Windows XP SP-2 32-bit.

Platform: X86
System: MINGW
asm: yes
avis input: yes
mp4 output: yes
pthread: yes
debug: no
gprof: no
PIC: no
shared: no
visualize: no

[imk]

http://imk.cx/junk/x264_mbtree.7z
Updated with v0.11 of patch.

[G_M_C]

http://imk.cx/junk/x264_mbtree.7z
Updated with v0.11 of patch.

Wich patches did u use ? HRD by any chance ?

[IgorC]

I remember there was a fixed version of SSIM avisynth plugin 0.24a. (with lumimask true)

dunno about behavior x264's internal SSIM.

[imk]

Wich patches did u use ? HRD by any chance ?

No. Since this is for testing mbtree, the only patches I used were my ICC patch and the mbtree patch.

[Chengbin]

This patch significantly changed CRF (unless compressibility has significantly gone up).

I encoded both videos with CRF 18, same settings.

With mbtree:1366Kbps
Without mbtree:878Kbps

This was done with anime.

So the new CRF 18 should be 15-16?

[Dark Shikari]

This patch significantly changed CRF (unless compressibility has significantly gone up).

I encoded both videos with CRF 18, same settings.

With mbtree:1366Kbps
Without mbtree:878Kbps

This was done with anime.

So the new CRF 18 should be 15-16?Again, as I've said at least a dozen times now, the patch redefines CRF, so different sources will get higher or lower bitrates!

Your source is not every source, and I am not going to customize x264 to match your source at the expense of everything else.

[G_M_C]

Wich patches did u use ? HRD by any chance ?

x264_x86_r1195_juGGaKNot (http://www.mediafire.com/download.php?mfy1vjjqaym)
GCC 4.4.0 generic, fprofiled, patched
Source: GIT

Applied patches :

x264_win_zone_parse_fix_05.diff (http://www.mediafire.com/download.php?kqymujywhtw)
x264_hrd_pulldown.15_interlace.diff (http://www.mediafire.com/download.php?ejhnmitmjt3)
x264_Macroblock_Tree_Ratecontrol_011.diff (http://www.mediafire.com/download.php?nlmj3d4t3yd)

Please check Doom9.org patches thread (http://forum.doom9.org/showthread.php?t=130364), Doom9.org Macroblock tree Ratecontrol thread (http://forum.doom9.org/showthread.php?t=148686) and GIT shortlog (http://git.videolan.org/gitweb.cgi?p=x264.git;a=shortlog) for more info.

Compiled by juGGaKNot (http://forum.doom9.org/member.php?u=144865) on August 04-2009, 12:20:00 GMT with GCC 4.4.0 on Windows XP SP-2 32-bit.

Platform: X86
System: MINGW
asm: yes
avis input: yes
mp4 output: yes
pthread: yes
debug: no
gprof: no
PIC: no
shared: no
visualize: no

Again, as I've said at least a dozen times now, the patch redefines CRF, so different sources will get higher or lower bitrates!

[...]

Well my first test is allmost done. Made a 20 minutes reencode of a docu i capped from BBCHD. Will put it on BD9, so i've used juGGaKNot build with HRD.

For testing purposes i've enabled 2-pass mode with both --ssim and --psnr. Will paste results tomorrow, after i've done test no2 (the one without --mbtree (which will be the only difference in the commandline/source etc. used).

This test will also test if --mbtree is accepted by STD's like my Panasonic BD30 (and incidentily if the mbtree patch coincides with the other patches juGGaKNot used, like the hrd-patch).

[Chengbin]

Again, as I've said at least a dozen times now, the patch redefines CRF, so different sources will get higher or lower bitrates!

Your source is not every source, and I am not going to customize x264 to match your source at the expense of everything else.

That's not what I meant. What I was trying to say is, if this patch reduces bitrate by 35% with the same CRF value, it should say something like lower CRF value by (say 2) to compensate for this patch, not just vaguely stating it redefines CRF.

[Dark Shikari]

That's not what I meant. What I was trying to say is, if this patch reduces bitrate by 50% with the same CRF value, it should say something like lower CRF value by (say 2) to compensate for this patch, not just vaguely stating it redefines CRF.Please stop responding to my posts without reading them.

[moviefan]

@Chengbin: What DS (as I understood) is saying is that CRF most probably hasn't shifted by a constant value but it has entirely changed and this change cannot easily be described on basis of one encoding sample. You simply need to find a new CRF that meets your demands, but I think you always have to if you don't want to waste bitrate or lack quality in the end.

[Firethe1]

Hi,
its sad that this is my first post, but x264_mbtree keeps crashing with one input file ...
The crash occurs around frame 2-20 ...
Already tried to reduce lookahead to 0.

commandline:
C:\Dokumente und Einstellungen\Fire>"D:\$Test\temp\x264_mbtree11.exe" --lookahea
d 0 --pass 1 --mbtree --ref 2 --bitrate 500 --qcomp 0.95 --bframes 3 --b-adapt 2
--mixed-refs --no-fast-pskip --weightb --direct auto --subme 10 --trellis 2 --p
artitions all --8x8dct --me umh --merange 32 --nr 50 --stats "D:\$Test\temp\test
2.avs.mbtree.stats" --threads 1 --thread-input -o "D:\$Test\temp\test2.avs.mbtre
e.mkv" "D:\$Test\temp\test2.avs"
avis [info]: 1280x720 @ 23.98 fps (1750 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile Main, level 3.1
[0.2%] 4/1750 frames, 43.01 fps, 0.00 kb/s, eta 0:00:40

even this cmd crashes: (works with latest x264.exe from x264.nl 32 Bit)
C:\Dokumente und Einstellungen\Fire>"D:\$Test\temp\x264_mbtree11.exe" -o D:\$Tes
t\temp\test2.avs.mkv D:\$Test\temp\test2.avs
avis [info]: 1280x720 @ 23.98 fps (1750 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 3.1
[0.1%] 2/1750 frames, 32.26 fps, 0.00 kb/s, eta 0:00:54

gdb: (hopefully i did it right ... but i dont think so xD)
D:\$Test\temp>gdb.exe D:\$Test\temp\x264_mbtree11_debug.exe
GNU gdb (GDB) 6.8.50.20090804
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "i386-pc-mingw32".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
This binary was built by Equation Solution <http://www.Equation.com>...
(gdb) run -o a.mkv test2.avs
Starting program: D:\$Test\temp\x264_mbtree11_debug.exe -o a.mkv test2.avs
[New Thread 4472.0x12e4]
warning: Can not parse XML library list; XML support was disabled at compile tim
e
[New Thread 4472.0x1248]
[New Thread 4472.0x1208]
[New Thread 4472.0x264]
[New Thread 4472.0x858]
[New Thread 4472.0xf90]
warning: HEAP[x264_mbtree11_debug.exe]:
warning: Invalid Address specified to RtlFreeHeap( 00240000, 00FAF174 )


Program received signal SIGTRAP, Trace/breakpoint trap.
0x7c91120f in ?? ()
(gdb) bt
#0 0x7c91120f in ?? ()
#1 0x7c97c201 in ?? ()
#2 0x0022ebe8 in ?? ()
#3 0x7c97c63e in ?? ()
#4 0x00faf16c in ?? ()
#5 0x00240000 in ?? ()
#6 0x00faf174 in ?? ()
#7 0x0022ec5c in ?? ()
#8 0x7c97d826 in ?? ()
#9 0x00240000 in ?? ()
#10 0x00faf16c in ?? ()
#11 0x7c97d9dc in ?? ()
#12 0x00240000 in ?? ()
#13 0x00faf174 in ?? ()
#14 0x40000060 in ?? ()
#15 0x00000040 in ?? ()
#16 0x00f37e48 in ?? ()
#17 0x003e0000 in ?? ()
#18 0x003e0000 in ?? ()
#19 0x007d0037 in ?? ()
#20 0x0101ecec in ?? ()
#21 0x0022ebd8 in ?? ()
#22 0x0022ebdc in ?? ()
#23 0x0022ed10 in ?? ()
#24 0x00014320 in ?? ()
#25 0x00faf16c in ?? ()
#26 0x00240000 in ?? ()
#27 0x7c97d994 in ?? ()
#28 0x7c959e1c in ?? ()
#29 0x00010000 in ?? ()
#30 0x0022ebfc in ?? ()
#31 0x7c937764 in ?? ()
#32 0x0022ed34 in ?? ()
#33 0x7c91e900 in ?? ()
#34 0x7c97d9b8 in ?? ()
#35 0x00000001 in ?? ()
#36 0x0022ed44 in ?? ()
#37 0x7c959e1c in ?? ()
#38 0x00240000 in ?? ()
#39 0x50000061 in ?? ()
#40 0x00faf174 in ?? ()
#41 0x00240000 in ?? ()
#42 0x00faf174 in ?? ()
#43 0x40000060 in ?? ()
#44 0x40000060 in ?? ()
#45 0x7c92003d in ?? ()
#46 0x00efa744 in ?? ()
#47 0x00efa740 in ?? ()
#48 0x00240000 in ?? ()
#49 0x0022ed60 in ?? ()
#50 0x7c91e900 in ?? ()
#51 0x7c937768 in ?? ()
#52 0xffffffff in ?? ()
#53 0x7c937764 in ?? ()
#54 0x7c007553 in ?? ()
#55 0x003e0000 in ?? ()
#56 0x0022ebf0 in ?? ()
#57 0x7c92003d in ?? ()
#58 0x0022ed84 in ?? ()
#59 0x7c91e900 in ?? ()
#60 0x7c937768 in ?? ()
#61 0xffffffff in ?? ()
#62 0x7c937764 in ?? ()
#63 0x7c937553 in ?? ()
#64 0x00240000 in ?? ()
#65 0x40000060 in ?? ()
#66 0x7c92003d in ?? ()
#67 0x00e98f4c in ?? ()
#68 0x0022edd8 in ?? ()
#69 0x00e95038 in ?? ()
#70 0x0022ec40 in ?? ()
#71 0x003e0000 in ?? ()
#72 0x003e0000 in ?? ()
#73 0x7c91e900 in ?? ()
#74 0x7c920040 in ?? ()
#75 0xffffffff in ?? ()
#76 0x7c92003d in ?? ()
#77 0x7c001432 in ?? ()
#78 0x7c001463 in ?? ()
#79 0x0022ec4c in ?? ()
#80 0x00240000 in ?? ()
#81 0x0022ede0 in ?? ()
#82 0x7c91e900 in ?? ()
#83 0x7c937768 in ?? ()
#84 0xffffffff in ?? ()
#85 0x7c937764 in ?? ()
#86 0x7c007553 in ?? ()
#87 0x003e0000 in ?? ()
#88 0x0022ec70 in ?? ()
#89 0x7c92003d in ?? ()
#90 0x0022ee04 in ?? ()
#91 0x7c91e900 in ?? ()
#92 0x7c937768 in ?? ()
#93 0xffffffff in ?? ()
#94 0x0022ee14 in ?? ()
#95 0x7c937553 in ?? ()
#96 0x00240000 in ?? ()
#97 0x40000060 in ?? ()
#98 0x00faf174 in ?? ()
#99 0x0022eea8 in ?? ()
#100 0x00f37e90 in ?? ()
#101 0x00f08ca8 in ?? ()
#102 0x0022ed44 in ?? ()
#103 0x0022f218 in ?? ()
#104 0x0022f218 in ?? ()
#105 0x0022ed54 in ?? ()
#106 0x000113d8 in ?? ()
#107 0x0022f218 in ?? ()
#108 0x0022ecd8 in ?? ()
#109 0x77be2070 in ?? ()
#110 0x0022f218 in ?? ()
#111 0x7c91e900 in ?? ()
#112 0x7c920040 in ?? ()
#113 0xffffffff in ?? ()
#114 0x7c92003d in ?? ()
#115 0x774cd01c in ?? ()
#116 0x00240000 in ?? ()
#117 0x00000000 in ?? ()

AVS:
DirectShowSource("X:\test.mp4",audio=false)
LoadPlugin("D:\***\UnDot.dll")
UnDot()
ConvertToYV12()
SelectRangeEvery(5000,250)

AVS plays fine in mpc ... and this is the first file which crashes ...

If you need more information/i should try something then tell me ^^

[Dark Shikari]

Don't use Avisynth input during debugging; use raw YUV input only.

[Firethe1]

thx for the quick response ... could you also tell me how i get my avs input to a raw yuv file? thx ;)

found avs2yuv.exe .. tried like this: avs2yuv.exe test2.avs -raw -o test2.yuv
put also crashes ... maybe its avisynth ...
AppName: avs2yuv.exe AppVer: 0.0.0.0 ModName: avisynth.dll
ModVer: 2.5.7.0 Offset: 000168f0

Updated to Avisynth 2.5.8 with directshowsource plugin patch and it works ... sry ;)

[Dark Shikari]

thx for the quick response ... could you also tell me how i get my avs input to a raw yuv file? thx ;)

found avs2yuv.exe .. tried like this: avs2yuv.exe test2.avs -raw -o test2.yuv
put also crashes ... maybe its avisynth ...It probably is Avisynth ;)

[Yoshiyuki Blade]

Tests on some anime I have are looking good so far!

Even at lower bitrates, an aq and psy-rd value of 1.0 doesn't create as much horrible artifacts as before.

Testing with --tune animation now.

[Chengbin]

Please stop responding to my posts without reading them.

@Chengbin: What DS (as I understood) is saying is that CRF most probably hasn't shifted by a constant value but it has entirely changed and this change cannot easily be described on basis of one encoding sample. You simply need to find a new CRF that meets your demands, but I think you always have to if you don't want to waste bitrate or lack quality in the end.

I understood him well.

If one encoding says with the same CRF value an encoding dropped 35% in bitrate than the unpatched, that means the same CRF value gives lower bitrate encodes. Of course with different sources the percentage will differ, but I seriously doubt it will vary much, let alone 35%.

How about as a suggestion, when CRF changes, can you tell us what is the new "CRF 18"?

[Sagekilla]

Why don't you do some testing? Go pick out a bunch of clips, test them with and without mbtree.

Find what the average change in crf is, then you'll know. I know you think that it might not vary much, but the only way you can really prove that is by testing more than one source.

Otherwise, I could pull out a source and say "Hey, I think this changes the crf upwards by 5" which could be completely wrong.

I'd do testing myself, but I don't have the time for it at the moment (Going away on vacation soon). If anyone is willing to do the testing, you or otherwise, it would be useful for everyone.

[nm]

If one encoding says with the same CRF value an encoding dropped 35% in bitrate than the unpatched, that means the same CRF value gives lower bitrate encodes. Of course with different sources the percentage will differ, but I seriously doubt it will vary much, let alone 35%.
From the first post (emphasis mine):
Also, note that CRF is redefined again by this patch--I set it so that it kept the same bitrate on Blackpearl as before, but it may vary for different videos.

[Chengbin]

Why don't you do some testing? Go pick out a bunch of clips, test them with and without mbtree.

Find what the average change in crf is, then you'll know. I know you think that it might not vary much, but the only way you can really prove that is by testing more than one source.

Otherwise, I could pull out a source and say "Hey, I think this changes the crf upwards by 5" which could be completely wrong.

I'd do testing myself, but I don't have the time for it at the moment (Going away on vacation soon). If anyone is willing to do the testing, you or otherwise, it would be useful for everyone.

CRF has been changed twice in I think 2 months (AutoVAQ (from the time it officially been adopted in x264, I didn't take it seriously before), and mbtree).

If I did testing (which I did for AutoVAQ with all kinds of sources), by the time I finished with AutoVAQ testing (which is a few weeks ago), my data is obsolete because of mbtree. Who knows when another change could happen. Maybe an update to mbtree will change CRF again. I think you can understand why I'm asking.

Just to keep the thread on topic, are there any compression improvements with this patch? Or is it visuals only?

[wyti]

plz learn to read...
if metrics increase it means that's a compression improvement.
metrics almost always go down when it's visually (aq PSNR goes down, psyrd / trellis both go down...)

And the change at the crf value may happend until the commit of this patch, this is a TEST patch, and everything that is true today may be wrong tomorow. It's not for real encoding, but to see what it does, where it fails, where it does great and improve this path before he got commited

[Sagekilla]

Yes, mbtree should improve quality, visually and metrically speaking.

It might not be as easy to judge in crf mode given the wide fluctuations we've seen, but it should be optimal when it comes out of testing.

[Dark Shikari]

Original post updated with v0.13.

[Zachs]

@DS

In your OP, you said that mb-tree is on by default. And --preset fast and faster also turns it on. That means, other presets have it turned off?

[Dark Shikari]

@DS

In your OP, you said that mb-tree is on by default. And --preset fast and faster also turns it on. That means, other presets have it turned off?Yes, "faster", "veryfast", and "ultrafast" have it off.

Fast and above (medium, slow, etc) have it on.

[Zachs]

@DS

Thanks.

BTW, is mb-tree with b-pyramid any closer for testing?
How much does b-pyramid add to encodes anyway?

[Dark Shikari]

@DS

Thanks.

BTW, is mb-tree with b-pyramid any closer for testing?
How much does b-pyramid add to encodes anyway?I probably will commit without pyramid support, then add pyramid support when Trahald finishes MMCO-related stuff.

Currently I don't think MB-tree with pyramid is a good idea as long as we allow referenced B-frames to end up in the reference list.

[G_M_C]

Well my first test is allmost done. Made a 20 minutes reencode of a docu i capped from BBCHD. Will put it on BD9, so i've used juGGaKNot build with HRD.

For testing purposes i've enabled 2-pass mode with both --ssim and --psnr. Will paste results tomorrow, after i've done test no2 (the one without --mbtree (which will be the only difference in the commandline/source etc. used).

This test will also test if --mbtree is accepted by STD's like my Panasonic BD30 (and incidentily if the mbtree patch coincides with the other patches juGGaKNot used, like the hrd-patch).

As promised. I give the log of pass 2 of both encodes. Used same commandline on both encodes, only removed/added --mbtree.

Commandline used:
<<x264>> --threads auto --thread-input --psnr --ssim "%IN_TITLE%.avs" --stats "%IN_TITLE%.stats" --output "%OUT_TITLE%.264" --pass 2 --bitrate 4450 --vbv-bufsize 30000 --vbv-maxrate 24000 --level 4.1 --deblock -1:-1 --keyint 25 --min-keyint 1 --nal-hrd --fps 25 --aud --sar 1:1 --bframes 3 --b-adapt 2 --ref 6 --no-fast-pskip --ipratio 1.3 --pbratio 1.2 --direct auto --subme 10 --trellis 2 --psy-rd 1.0:0.2 --partitions all --me umh --merange 24 --mvrange 511 --aq-strength 1.0 --aq-mode 1 --mbtree

Encode without --mbtree
avis [info]: 1280x720 @ 25.00 fps (35001 frames)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.1 Cach
e64
x264 [info]: profile High, level 4.1
x264 [info]: slice I:2212 Avg QP:19.00 size: 73712 PSNR Mean Y:46.71 U:50.33
V:50.92 Avg:47.62 Global:46.51
x264 [info]: slice P:12247 Avg QP:21.08 size: 32098 PSNR Mean Y:43.76 U:47.87
V:48.92 Avg:44.78 Global:43.59
x264 [info]: slice B:20542 Avg QP:22.86 size: 10825 PSNR Mean Y:43.02 U:47.50
V:48.58 Avg:44.09 Global:43.19
x264 [info]: consecutive B-frames: 4.1% 18.0% 57.1% 20.8%
x264 [info]: mb I I16..4: 8.2% 84.8% 7.0%
x264 [info]: mb P I16..4: 1.8% 14.8% 0.6% P16..4: 44.0% 20.5% 13.5% 0.2% 0
.2% skip: 4.5%
x264 [info]: mb B I16..4: 0.5% 1.9% 0.1% B16..8: 42.1% 1.6% 2.4% direct:
9.2% skip:42.3% L0:35.6% L1:48.1% BI:16.3%
x264 [info]: 8x8 transform intra:84.6% inter:74.8%
x264 [info]: direct mvs spatial:76.3% temporal:23.7%
x264 [info]: coded y,uvDC,uvAC intra:80.9% 80.7% 46.4% inter:30.4% 32.1% 3.9%
x264 [info]: ref P L0 78.9% 11.5% 5.4% 2.2% 1.3% 0.7%
x264 [info]: ref B L0 85.2% 8.9% 3.6% 1.5% 0.9%
x264 [info]: SSIM Mean Y:0.9784036
x264 [info]: PSNR Mean Y:43.516 U:47.805 V:48.844 Avg:44.553 Global:43.481 kb/s:
4448.60

encoded 35001 frames, 4.39 fps, 4448.63 kb/s
]

Encode with --mbtree
avis [info]: 1280x720 @ 25.00 fps (35001 frames)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.1 Cach
e64
x264 [info]: profile High, level 4.1
x264 [info]: slice I:2211 Avg QP:18.28 size: 89332 PSNR Mean Y:47.28 U:50.78
V:51.30 Avg:48.17 Global:47.36
x264 [info]: slice P:12255 Avg QP:21.25 size: 32438 PSNR Mean Y:43.83 U:47.96
V:48.98 Avg:44.85 Global:43.92
x264 [info]: slice B:20535 Avg QP:24.79 size: 8944 PSNR Mean Y:43.01 U:47.53
V:48.62 Avg:44.07 Global:43.39
x264 [info]: consecutive B-frames: 4.2% 17.4% 59.0% 19.5%
x264 [info]: mb I I16..4: 10.8% 79.8% 9.4%
x264 [info]: mb P I16..4: 1.9% 15.0% 0.6% P16..4: 43.2% 19.4% 13.4% 0.2% 0
.3% skip: 6.0%
x264 [info]: mb B I16..4: 0.4% 1.8% 0.1% B16..8: 40.3% 1.4% 1.8% direct:
6.0% skip:48.2% L0:37.7% L1:47.7% BI:14.6%
x264 [info]: 8x8 transform intra:82.3% inter:74.7%
x264 [info]: direct mvs spatial:72.8% temporal:27.2%
x264 [info]: coded y,uvDC,uvAC intra:80.6% 80.1% 48.2% inter:26.1% 26.6% 3.5%
x264 [info]: ref P L0 78.7% 11.7% 5.5% 2.2% 1.3% 0.7%
x264 [info]: ref B L0 86.5% 8.1% 3.3% 1.3% 0.8%
x264 [info]: SSIM Mean Y:0.9790297
x264 [info]: PSNR Mean Y:43.564 U:47.884 V:48.917 Avg:44.603 Global:43.742 kb/s:
4449.59

encoded 35001 frames, 4.41 fps, 4449.62 kb/s

I saw a rise in both SSIM and PSNR, but not shockingly high percentages. But i did notice a difference of 2 points in the avg QP used for b-frames. At the same time I-frames got a slightly lower QP, and the QP for P stayed about the same.

But other than that, all went as if nothing changed. My STD ran these files as usual (no change noticed), and visual quality was about the same (with a slight plus on the --mbtree side).

Will run another encode with some different parameters while i'm at work (with/without VAQ). I'll post about those when i get back from work.

[tetsuo55]

Currently I don't think MB-tree with pyramid is a good idea as long as we allow referenced B-frames to end up in the reference list.Do you mean the that with that other patch b-frames will be freed from reference frames?

This might raise b-frame and b-pyramid compatibility with standalone devices! (or break them)

[juGGaKNot]

x264_x86_r1195_juGGaKNot (http://www.mediafire.com/download.php?5izgtetmjqz)
GCC 4.4.0, modified, fprofiled, patched
Source: GIT

Applied patches :

x264_win_zone_parse_fix_05.diff (http://www.mediafire.com/download.php?kqymujywhtw)
x264_hrd_pulldown.15_interlace.diff (http://www.mediafire.com/download.php?ejhnmitmjt3)
x264_Macroblock_Tree_Ratecontrol_013.diff (http://www.mediafire.com/download.php?jyhmmrnzotn)
x264_juGGaKNot_preset_01.diff (http://www.mediafire.com/download.php?5zwqzy0ymn2) ( This just adds a preset, juggaknot, download it to see the settings )

It is fprofiled on a uncompressed 300 frames source with very high motion

Please check Doom9.org patches thread (http://forum.doom9.org/showthread.php?t=130364), Doom9.org Macroblock tree Ratecontrol thread (http://forum.doom9.org/showthread.php?t=148686) and GIT shortlog (http://git.videolan.org/gitweb.cgi?p=x264.git;a=shortlog) for more info.

Compiled by juGGaKNot (http://forum.doom9.org/member.php?u=144865) with GCC 4.4.0 on Windows XP SP-2 32-bit.

Platform: X86
System: MINGW
asm: yes
avis input: yes
mp4 output: yes
pthread: yes
debug: no
gprof: no
PIC: no
shared: no
visualize: no

[imk]

CRF has been changed twice in I think 2 months (AutoVAQ (from the time it officially been adopted in x264, I didn't take it seriously before), and mbtree).

This is what it means to use bleeding edge.

Changes happen, they happen often, and they break compatibility.

Using the latest revisions from git will guarantee changes are happening, let alone a brand new patch.

If you want everything to stay consistent for longer periods of time, stick to a specific revision or version.

[juGGaKNot]

x264_x86_r1195_juGGaKNot_CCG_4.4.1 (http://www.mediafire.com/download.php?3wtyqmzzj1m)
GCC 4.4.1, modified, fprofiled, patched
Source: GIT

Applied patches :

x264_win_zone_parse_fix_05.diff (http://www.mediafire.com/download.php?kqymujywhtw)
x264_hrd_pulldown.15_interlace.diff (http://www.mediafire.com/download.php?ejhnmitmjt3)
x264_Macroblock_Tree_Ratecontrol_013.diff (http://www.mediafire.com/download.php?jyhmmrnzotn)
x264_juGGaKNot_preset_01.diff (http://www.mediafire.com/download.php?5zwqzy0ymn2) ( This just adds a preset, juggaknot, download it to see the settings )

It is fprofiled on a uncompressed 300 frames source with very high motion

Please check Doom9.org patches thread (http://forum.doom9.org/showthread.php?t=130364), Doom9.org Macroblock tree Ratecontrol thread (http://forum.doom9.org/showthread.php?t=148686) and GIT shortlog (http://git.videolan.org/gitweb.cgi?p=x264.git;a=shortlog) for more info.

Compiled by juGGaKNot (http://forum.doom9.org/member.php?u=144865) with GCC 4.4.1 on Windows XP SP-2 32-bit.

Platform: X86
System: MINGW
asm: yes
avis input: yes
mp4 output: yes
pthread: yes
debug: no
gprof: no
PIC: no
shared: no
visualize: no

[rack04]

Here are my results using v 0.13 of MB Tree patch.

Pass 1:
--preset fast --tune animation --pass 1 --slow-firstpass --bitrate 1000 --level 4.1 --keyint 24 --min-keyint 1
--vbv-bufsize 30000 --vbv-maxrate 24000 --thread-input --direct auto --b-adapt 2 --nal-hrd --ssim --psnr

SSIM/PNSR:
avis [info]: 1920x1080 @ 24.00 fps (400 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 4.1

x264 [info]: slice P:155 Avg QP:33.65 size: 5986 PSNR Mean Y:41.42 U:45.49 V:46.99 Avg:42.49 Global:41.27
x264 [info]: slice B:226 Avg QP:36.38 size: 1917 PSNR Mean Y:41.33 U:45.80 V:47.55 Avg:42.45 Global:41.26
x264 [info]: consecutive B-frames: 7.6% 35.2% 22.8% 24.1% 3.9% 6.3%
x264 [info]: mb I I16..4: 28.8% 68.2% 3.0%
x264 [info]: mb P I16..4: 3.1% 5.0% 0.2% P16..4: 12.2% 0.9% 0.5% 0.0% 0.0% skip:78.2%
x264 [info]: mb B I16..4: 1.1% 0.0% 0.0% B16..8: 4.7% 0.2% 0.1% direct: 0.5% skip:93.4% L0:51.8% L1:44.8% BI: 3.4%
x264 [info]: final ratefactor: 31.95
x264 [info]: 8x8 transform intra:60.4% inter:82.6%
x264 [info]: direct mvs spatial:98.7% temporal:1.3%
x264 [info]: coded y,uvDC,uvAC intra:38.3% 54.2% 18.5% inter:1.7% 1.8% 0.1%
x264 [info]: ref P L0 67.1% 16.0% 9.3% 7.5%
x264 [info]: ref B L0 78.8% 13.1% 8.1%
x264 [info]: SSIM Mean Y:0.9695521
x264 [info]: PSNR Mean Y:41.434 U:45.704 V:47.348 Avg:42.529 Global:41.318 kb/s:1238.14

encoded 400 frames, 2.59 fps, 1238.63 kb/s

Pass 2:
--preset slow --tune animation --pass 2 --bitrate 1000 --level 4.1 --keyint 24 --min-keyint 1
--vbv-bufsize 30000 --vbv-maxrate 24000 --thread-input --trellis 2 --sar 1:1 --aud --nal-hrd --ssim --psnr

SSIM/PNSR:
avis [info]: 1920x1080 @ 24.00 fps (400 frames)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 4.1

x264 [info]: slice P:155 Avg QP:35.74 size: 4759 PSNR Mean Y:40.53 U:44.54 V:46.20 Avg:41.60 Global:40.84
x264 [info]: slice B:226 Avg QP:38.61 size: 1726 PSNR Mean Y:40.80 U:44.88 V:46.79 Avg:41.89 Global:41.09
x264 [info]: consecutive B-frames: 7.6% 35.2% 22.8% 24.1% 3.9% 6.3%
x264 [info]: mb I I16..4: 33.7% 64.6% 1.7%
x264 [info]: mb P I16..4: 3.4% 3.8% 0.1% P16..4: 10.9% 0.8% 1.0% 0.0% 0.0% skip:79.9%
x264 [info]: mb B I16..4: 0.5% 0.8% 0.0% B16..8: 6.9% 0.1% 0.1% direct: 0.2% skip:91.5% L0:45.6% L1:53.5% BI: 0.9%
x264 [info]: 8x8 transform intra:60.2% inter:93.7%
x264 [info]: direct mvs spatial:93.4% temporal:6.6%
x264 [info]: coded y,uvDC,uvAC intra:33.4% 48.8% 14.5% inter:1.1% 1.2% 0.1%
x264 [info]: ref P L0 62.0% 18.8% 10.8% 8.3%
x264 [info]: ref B L0 84.5% 9.7% 5.8%
x264 [info]: SSIM Mean Y:0.9666423
x264 [info]: PSNR Mean Y:40.736 U:44.759 V:46.566 Avg:41.811 Global:41.023 kb/s:984.06

encoded 400 frames, 3.75 fps, 984.44 kb/s

Pass 1:
--preset fast --tune animation --pass 1 --slow-firstpass --bitrate 1000 --level 4.1 --keyint 24 --min-keyint 1
--vbv-bufsize 30000 --vbv-maxrate 24000 --thread-input --direct auto --b-adapt 2 --nal-hrd --ssim --psnr --no-mbtree

SSIM/PNSR:
avis [info]: 1920x1080 @ 24.00 fps (400 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 4.1

x264 [info]: slice P:167 Avg QP:31.99 size: 6805 PSNR Mean Y:41.16 U:45.00 V:46.80 Avg:42.20 Global:40.61
x264 [info]: slice B:214 Avg QP:34.39 size: 2249 PSNR Mean Y:41.01 U:45.17 V:47.09 Avg:42.10 Global:40.51
x264 [info]: consecutive B-frames: 13.4% 29.9% 23.6% 23.1% 5.2% 4.7%
x264 [info]: mb I I16..4: 33.0% 64.4% 2.6%
x264 [info]: mb P I16..4: 2.9% 5.4% 0.2% P16..4: 12.2% 1.0% 0.5% 0.0% 0.0% skip:77.8%
x264 [info]: mb B I16..4: 1.2% 0.0% 0.0% B16..8: 5.3% 0.2% 0.1% direct: 0.6% skip:92.6% L0:51.2% L1:44.6% BI: 4.2%
x264 [info]: final ratefactor: 33.01
x264 [info]: 8x8 transform intra:59.5% inter:81.6%
x264 [info]: direct mvs spatial:98.6% temporal:1.4%
x264 [info]: coded y,uvDC,uvAC intra:38.6% 53.2% 15.0% inter:2.0% 1.9% 0.1%
x264 [info]: ref P L0 69.0% 15.3% 8.8% 6.9%
x264 [info]: ref B L0 79.1% 12.7% 8.2%
x264 [info]: SSIM Mean Y:0.9642562
x264 [info]: PSNR Mean Y:41.117 U:45.110 V:46.981 Avg:42.183 Global:40.593 kb/s:1211.68

encoded 400 frames, 2.58 fps, 1212.12 kb/s

Pass 2:
--preset slow --tune animation --pass 2 --bitrate 1000 --level 4.1 --keyint 24 --min-keyint 1
--vbv-bufsize 30000 --vbv-maxrate 24000 --thread-input --trellis 2 --sar 1:1 --aud --nal-hrd --ssim --psnr --no-mbtree

SSIM/PNSR:
avis [info]: 1920x1080 @ 24.00 fps (400 frames)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 4.1

x264 [info]: slice P:167 Avg QP:33.94 size: 5149 PSNR Mean Y:40.49 U:44.43 V:46.34 Avg:41.56 Global:40.13
x264 [info]: slice B:214 Avg QP:35.98 size: 1948 PSNR Mean Y:40.87 U:44.95 V:46.98 Avg:41.96 Global:40.44
x264 [info]: consecutive B-frames: 13.4% 29.9% 23.6% 23.1% 5.2% 4.7%
x264 [info]: mb I I16..4: 34.4% 64.1% 1.5%
x264 [info]: mb P I16..4: 2.9% 4.0% 0.1% P16..4: 10.7% 1.0% 1.5% 0.0% 0.0% skip:79.9%
x264 [info]: mb B I16..4: 0.5% 0.8% 0.0% B16..8: 7.9% 0.1% 0.1% direct: 0.2% skip:90.4% L0:44.1% L1:54.6% BI: 1.3%
x264 [info]: 8x8 transform intra:61.7% inter:94.0%
x264 [info]: direct mvs spatial:95.8% temporal:4.2%
x264 [info]: coded y,uvDC,uvAC intra:35.1% 49.0% 12.5% inter:1.2% 1.2% 0.1%
x264 [info]: ref P L0 64.3% 17.8% 10.1% 7.7%
x264 [info]: ref B L0 84.9% 9.4% 5.8%
x264 [info]: SSIM Mean Y:0.9614927
x264 [info]: PSNR Mean Y:40.741 U:44.744 V:46.714 Avg:41.820 Global:40.333 kb/s:983.32

encoded 400 frames, 3.49 fps, 983.57 kb/s

[moviefan]

@rack04: Are you sure you encoded the mbtree one with bitrate 1000? Average bitrate is almost 6000 in the end...

[rack04]

@rack04: Are you sure you encoded the mbtree one with bitrate 1000? Average bitrate is almost 6000 in the end...

Looks like you're right. I'll re-encode and edit my post. Results have been updated. Sorry for the mistake.

[moviefan]

Looks like you're right. I'll re-encode and edit my post. If that is still an option.

Sure, I'd love to see some further benchmarks about the mbtree patch. Thanks in advance!

[imk]

http://imk.cx/junk/x264_mbtree.7z -- Updated with v0.13 of patch.

Patches used:
x264_hrd_pulldown.15_interlace.diff
x264_icc_03_win.diff
x264_win_zone_parse_fix_05.diff
mbtree patch v0.13

[IgorC]

Is it normal that avinaptic informs (lookahead default)?:

0.9 patch version
1st pass - lookahead=50
2d pass - lookahead=50

0.13 patch version
1st pass - lookahead=50
2d pass - lookahead=0

OPSNR is identical and SSIM is almost: 0.9v - .9926303, 0.13v - .9926298.

[wyti]

Changes in 0.11:

1. VBV now supported. Includes a separate VBV patch which fixes most VBV issues that MB-tree has; this will be committed separately, as it doesn't require MB-tree. It should also improve 1-pass VBV (especially CBR) performance on clips with lots of static content, e.g. anime, CGI.

Do note that CBR doesn't benefit much from MB-tree though, especially with small buffer sizes.

2. Lots of trivial parameter-handling fixes, e.g. lookahead automatically disabled on second pass.

So completely normal.

[nurbs]

Yes; see first post, changes in 0.11, point 2.

edit: beaten again :(

[Dark Shikari]

I will fix it later to read "lookahead" from the first pass statsfile rather than print zero.

[juGGaKNot]

OT : dark

new download link for version kkrunchya2 ?

a1 works fine, any updates on a2 ? link is down on the site and only other google result is an asian gui 2.3 but it needs registration ..

[Daiz]

I guess I might post these images as well. Done with patch 0.11. Click the images to open them in new tabs. Also, I totally forgot to include --ssim and --psnr in the command line options, but the images speak for themselves...

Source is a 5000-frame clip from a lossless H.264 stream that was encoded from an MPEG-2 Transport Stream.

Without mbtree:
x264 --pass 2 --stats "mbtree0.log" --bitrate 300 --psy-rd 1.0:0.2 --b-adapt 2 --bframes 3 --ref 15 --mixed-refs
--b-pyramid --me umh --subme 10 --merange 32 --partitions all --direct auto --trellis 2 -o "mbtree0.mkv" "18l.avs"

pass 1 - encoded 5000 frames, 68.38 fps, 290.88 kb/s
pass 2 - encoded 5000 frames, 16.38 fps, 299.97 kb/s

http://img40.imageshack.us/img40/3988/x264300kbpsmbtree0.png (http://img40.imageshack.us/img40/3988/x264300kbpsmbtree0.png)
Frame type B, size 646

With mbtree:
x264t --pass 2 --stats "mbtree1.log" --bitrate 300 --psy-rd 1.0:0.2 --b-adapt 2 --bframes 3 --ref 15 --mixed-refs
--mbtree --me umh --subme 10 --merange 32 --partitions all --direct auto --trellis 2 -o "mbtree1.mkv" "18l.avs"

pass 1 - encoded 5000 frames, 54.05 fps, 289.55 kb/s
pass 2 - encoded 5000 frames, 17.30 fps, 300.30 kb/s

http://img29.imageshack.us/img29/6396/x264300kbpsmbtree1.png (http://img29.imageshack.us/img29/6396/x264300kbpsmbtree1.png)
Frame type B, size 494

I have to say the difference is quite incredible.

[Chengbin]

That's amazing.

I want to do some testing with real life videos. How do I get a log from x264?

Also, is there a page explaining what SSIM and PSNR are and stuff?

[Fishman0919]

That's amazing.

I want to do some testing with real life videos. How do I get a log from x264?

Also, is there a page explaining what SSIM and PSNR are and stuff?


LINK (http://mewiki.project357.com/wiki/X264_Settings#psnr) to MeWiki for x264. Click on SSIM and PSNR to see details about them

[Snowknight26]

ffmpeg -i "test/LosslessTouhou.mkv" -vcodec rawvideo -f rawvideo -an -pix_fmt yuv420p - | x264.x64 --pass 1 --bitrate 50 --stats "test/test.stats" --bframes 5 --b-adapt 2 --direct auto --deblock -3:-3 --subme 4 --analyse none --me hex --fps 60 --output NUL - 640x480
ffmpeg -i "test/LosslessTouhou.mkv" -vcodec rawvideo -f rawvideo -an -pix_fmt yuv420p - | x264.x64 --pass 2 --bitrate 50 --stats "test/test.stats" --bframes 5 --b-adapt 2 --ref 5 --no-fast-pskip --b-pyramid --direct auto --deblock -3:-3 --subme 10 --trellis 2 --analyse all --8x8dct --me umh --ssim --fps 60 --output "test/test.mkv" - 640x480

ffmpeg -i "test/LosslessTouhou.mkv" -vcodec rawvideo -f rawvideo -an -pix_fmt yuv420p - | x264_mbtree13 --pass 1 --bitrate 50 --stats "test/test.stats" --bframes 5 --b-adapt 2 --direct auto --deblock -3:-3 --subme 4 --analyse none --me hex --fps 60 --output NUL - 640x480
ffmpeg -i "test/LosslessTouhou.mkv" -vcodec rawvideo -f rawvideo -an -pix_fmt yuv420p - | x264_mbtree13 --pass 2 --bitrate 50 --stats "test/test.stats" --bframes 5 --b-adapt 2 --ref 5 --no-fast-pskip --b-pyramid --direct auto --deblock -3:-3 --subme 10 --trellis 2 --analyse all --8x8dct --me umh --ssim --fps 60 --output "test/test.mbtree.mkv" - 640x480

http://i26.tinypic.com/vq15kg.png

http://i27.tinypic.com/2cz7sxj.png

Guess which is which.

Edit: Out of curiosity, does mbtree work with the lossless mode?

[Dark Shikari]

--bitrate 50ｗｗｗｗｗｗｗ

[Snowknight26]

Unfortunately the QP @ 50kbps was 51 for the first pass so the 2nd pass had to converge to ~400kbps. Even @ --bitrate 500 the average QP was ~49.

[Chengbin]

While doing tests with the crowdrun footage, I came across this warning on the second pass. This warning came many many times.

"MB-tree frametype (xxx) doesn't match actual frametype 0."
"x264_encoder_encode failed"

BTW, I wrote " --stats "c:\mbtree.log" to try to get a log file from x264. It gave me a stats file from the first pass. What do I write to get a log of the output? I don't have quick enough fingers to print screen the output the second it finishes.

[Dark Shikari]

While doing tests with the crowdrun footage, I came across this warning on the second pass. This warning came many many times.

"MB-tree frametype (xxx) doesn't match actual frametype 0."
"x264_encoder_encode failed"

BTW, I wrote " --stats "c:\mbtree.log" to try to get a log file from x264. It gave me a stats file from the first pass. What do I write to get a log of the output? I don't have quick enough fingers to print screen the output the second it finishes. 2> output.txt

It sounds like you specified the wrong input passfile (for example).@Snowknight26

Yes, mbtree does work with lossless.No, it doesn't, stop answering questions that you don't know about!