Log in

View Full Version : x264 CAVLC RD

DeathTheSheep
4th January 2006, 20:14
With both MeGUI and even the VFW, I am unable to do the following with CABAC disabled:
1. Enable Trellis
2. Enable any sort of rate distortion mechanisms (ME6/7 or B-RDO don't work).

I use constant quantizer mode (28) with UMH 16-type ME.

The problem has persisted for at least 10 revisions including the current one (395 Sharktooth), regardless of reference frame settings, B-frame settings, CQM settings, 8x8 settings, ME-type settings, etc.

I havn't directly tried the commandline yet, but since MeGUI is merely a GUI for it, I didn't deem it necessary.
Here is a sample (failed) commandline from MeGUI:
"x264.exe" --qp 36 --keyint 240 --ref 3 --filter 1,1 --no-cabac --subme 6 --analyse p8x8,b8x8,i4x4,p4x4 --direct none --progress --output "C:\Documents and Settings\HP_Owner\Desktop\bleachBoysToFreeMySoul\dd.264" "C:\Documents and Settings\HP_Owner\Desktop\bleachBoysToFreeMySoul\script.avs"
which produces the exact same results as with --subme 5 or Partition descision 5 in the VFW.
With the above commandline via raw output and muxed into AVI via AVC2AVI, the AVI header displays this:
http://www.myfilehut.com/userfiles/5120/CollectedBuilds/header.PNG

This does indeed appear to me as a rather nasty bug. Any suggestions?
akupenguin
4th January 2006, 21:19
Not a bug.
The p8rd patch (i.e. --subme 7) removes all support for cavlc rd, since we didn't feel like updating it for all the new features in cabac rd. (And no, just leaving the old cavlc code would not have worked)
Trellis has always required cabac, for similar reasons: cabac trellis is a completely different algorithm from cavlc trellis, and I didn't bother implementing both.
DeathTheSheep
4th January 2006, 22:28
x264's trellis, as I now understand, never supported CAVLC in the first place.
But what about --subme 6 (not the patch)? I sort of suspected that --subme 7 wouldn't work without CABAC, but should --subme 6 (which I did the above tests on) behave in same way?
slavickas
4th January 2006, 23:05
removes all support for cavlc rd
if I correctly understood subme 6 for CAVLC removed too...
DeathTheSheep
4th January 2006, 23:48
Hmm, so by this reasoning, if I get a revision without the patch (x264.nl?) it should be supported.
I'll try it...
DeathTheSheep
5th January 2006, 20:13
Update:
The builds on bob0r's site work like a charm when it comes to CAVLC rate distortion. This is a must-have for iPod/QT/non-cabac decoding!

Take care, y'all!