[New patch] Improved multi-hexagon search

[Dark Shikari]

http://i14.tinypic.com/4vrbc6w.png
(Note the FPS values are singlethreaded on 720p footage on a Core 2 2Ghz)

IMH: its a new motion search algorithm aimed to sit between ESA and UMH in terms of speed and quality. Its basically what I came up with after trashing simulated annealing for not really working that well.

Its main advantage other than its speed and quality fitting that niche between UMH and ESA is that unlike ESA it can be multithreaded, which means it is much more practical for everyday encoding.

The algorithm is somewhat complex, but not overly so:

1. Run the regular UMH search, but instead of ending with HEX, go to Improved Multi-Hex instead.

2. If the SAD cost of the current MV is below 100, terminate and jump to ordinary HEX instead. This increases speed at little quality loss.

3. Do an ordinary hex search.

4. Create a radius-8 hexagon around the current MV. Run short-range hex searches on all those. Note that throughout this we never move the current MV; the best that's found is saved once we're done, so all the searches are centered around the same point.

Also note that when the modified HEX search finds a new MV better than all previous ones and terminates due to maxing its range rather than hitting a dead end (i.e. better MVs could be around the corner), it will continue for another full range. This allows shorter ranges to be specified in the IMH algorithm while ensuring that if a good candidate is found, it isn't ignored.

5. For every 8 MERange starting with 16, do another hexagon around the current MV. The range of these gets longer the farther out you go, as the hexagons get farther and farther apart. However, unlike the earlier hexagons, these are adaptively chosen--if the SAD cost of the center of the starting hexagon is over twice that of the best SAD cost so far, the search is skipped. This loses some quality, but vastly increases speed, especially at high MEranges.

6. Finally, save the best result found so far and end the motion search.

Patch link (http://tjhsst.edu/~jgarrett/IMH.diff) (Works with Hadamard patch only)

Patch link 2 (http://tjhsst.edu/~jgarrett/IHM noHadamard.diff) (Works without Hadamard patch only)

The difference between the two patches is only 7 characters; its a single line that conflicted, so no big deal.

(Edit: Minor typo fixed in both patches. It would have kept the code from compiling though.)

[akupenguin]

Also note that when the modified HEX search finds a new MV better than all previous ones and terminates due to maxing its range rather than hitting a dead end (i.e. better MVs could be around the corner), it will continue for another full range. This allows shorter ranges to be specified in the IMH algorithm while ensuring that if a good candidate is found, it isn't ignored.

Might it be better to just remove the range limitation on HEX?

If the SAD cost of the current MV is below 100, terminate and jump to ordinary HEX instead.
Is 100 supposed to be independent of block size? 16x16 and 4x4 blocks each with SAD=100 are very different match qualities.

[Dark Shikari]

Might it be better to just remove the range limitation on HEX?
No, that makes it far too slow for use in IMH.

When running a plain HEX search, --merange 16 is plenty to reach the optimal result 99% of the time.

In IMH, I use ranges like 2, 4, and 6--much lower numbers. I can't afford to run the full HEX search as it takes too long, since I'm running so many HEX searches. Its better to do a bunch of short ones, and run it longer on the best ones.

Also, I found that if I made it so that when a HEX found a new best MV and ran out of range, the range was eliminated, the benefit was basically zero over my current method.


Is 100 supposed to be independent of block size? 16x16 and 4x4 blocks each with SAD=100 are very different match qualities.That's a good idea; I overlooked that, since --me imh is indeed doing the PIXEL_4x4 motion searches.

[RaynQuist]

How are you computing the percentages?

Are you still planning on running exhaustive RDO?

[Dark Shikari]

How are you computing the percentages?
Its % SSIM increase over --me hex --merange 16.

To be exact, its 1 - (1/(1-NewSSIM)) / (1/(1-OldSSIM)).
Are you still planning on running exhaustive RDO?An exhaustive RDO search would require me understanding how to call Qpel RDO in an arbitrary manner on any block type, and I don't know the code well enough to do that.

[Dark Shikari]

Is 100 supposed to be independent of block size? 16x16 and 4x4 blocks each with SAD=100 are very different match qualities.
Changing the code here to SAD_THRESH gave a small quality boost per bitrate. Measurable, at least. Higher or lower doesn't seem to help though, so I had the number nearly right it seems.

[Sharktooth]

http://tjhsst.edu/~jgarrett/IMH%20noHadamard.diff ---> 404

[Dark Shikari]

http://tjhsst.edu/~jgarrett/IMH%20noHadamard.diff ---> 404
I mispelled the filename, but their SSH server is down so I can't fix it. So I've fixed the link here instead.

[Sagittaire]

I mispelled the filename, but their SSH server is down so I can't fix it. So I've fixed the link here instead.

Where is the correct hadamard patch. x264_satd_fpel.07.diff from aku seem not work with IMH.diff

[Dark Shikari]

Where is the correct hadamard patch. x264_satd_fpel.07.diff from aku seem not work with IMH.diffI was basing it on my own --hadamard patch--it should take very little modification to make it work with Aku's though; no algorithms have to be changed, just the fact that they probably modify one or two lines of the same code in non-conflicting ways.

[Romario]

When we can expect build, based on these patches? Sharktooth?

[Sharktooth]

:search:
there's already one...

[Dark Shikari]

Also, this is still in the pipeline (I'll post it later once its more finalized, like IMH is now), but I have an experimental --subme 8 algorithm that gives 0.6% better quality per bitrate than --subme 7 but slows encoding down by a mere 2.2%. Not surprisingly, it consists of even more hexagons, replacing the diamond RD searches with a set of much more accurate HEX algorithms.

[Sharktooth]

i think there's no reason to add subme 8. you can replace subme 7 with the new algo.

[Dark Shikari]

i think there's no reason to add subme 8. you can replace subme 7 with the new algo.
True--the difference between 7 and 8 in terms of speed is minute (now down to 1.8% with a small optimization).

#define COST_MV_RD2( mx, my, satd) \
{ \
if( satd <= bsatd * SATD_THRESH)\
{ \
int cost; \
cache_mv[0] = cache_mv2[0] = mx; \
cache_mv[1] = cache_mv2[1] = my; \
cost = x264_rd_cost_part( h, i_lambda2, i8, m->i_pixel ); \
COPY3_IF_LT( bcost, cost, bmx, mx, bmy, my ); \
} \
}

#define COST_MV_RD_X3_DIR( m0x, m0y, m1x, m1y, m2x, m2y, costs) \
{ \
COST_MV_SATD(m0x+omx,m0y+omy, satds[0]);\
COST_MV_SATD(m1x+omx,m1y+omy, satds[1]);\
COST_MV_SATD(m2x+omx,m2y+omy, satds[2]);\
if( satds[0] <= bsatd * SATD_THRESH) \
{ \
cache_mv[0] = cache_mv2[0] = m0x+omx; \
cache_mv[1] = cache_mv2[1] = m0y+omy; \
int cost = x264_rd_cost_part( h, i_lambda2, i8, m->i_pixel ); \
(costs)[0] = cost;\
COPY3_IF_LT( bcost, cost, bmx, m0x+omx, bmy, m0y+omy ); \
} \
else (costs)[0] = COST_MAX;\
if( satds[1] <= bsatd * SATD_THRESH) \
{ \
cache_mv[0] = cache_mv2[0] = m1x+omx; \
cache_mv[1] = cache_mv2[1] = m1y+omy; \
int cost = x264_rd_cost_part( h, i_lambda2, i8, m->i_pixel ); \
(costs)[1] = cost;\
COPY3_IF_LT( bcost, cost, bmx, m1x+omx, bmy, m1y+omy ); \
} \
else (costs)[1] = COST_MAX;\
if( satds[2] <= bsatd * SATD_THRESH) \
{ \
cache_mv[0] = cache_mv2[0] = m2x+omx; \
cache_mv[1] = cache_mv2[1] = m2y+omy; \
int cost = x264_rd_cost_part( h, i_lambda2, i8, m->i_pixel ); \
(costs)[2] = cost;\
COPY3_IF_LT( bcost, cost, bmx, m2x+omx, bmy, m2y+omy ); \
} \
else (costs)[2] = COST_MAX;\
}

void x264_me_refine_qpel_rd2( x264_t *h, x264_me_t *m, int i_lambda2, int i8 )
{
// don't have to fill the whole mv cache rectangle
static const int pixel_mv_offs[] = { 0, 4, 4*8, 0 };
int16_t *cache_mv = h->mb.cache.mv[0][x264_scan8[i8*4]];
int16_t *cache_mv2 = cache_mv + pixel_mv_offs[m->i_pixel];
const int16_t *p_cost_mvx, *p_cost_mvy;
const int bw = x264_pixel_size[m->i_pixel].w>>2;
const int bh = x264_pixel_size[m->i_pixel].h>>2;
const int i_pixel = m->i_pixel;
DECLARE_ALIGNED( uint8_t, pix[16*16], 16 );
int bcost = m->i_pixel == PIXEL_16x16 ? m->cost : COST_MAX;
int bmx = m->mv[0];
int bmy = m->mv[1];
int pmx, pmy, omx, omy, i;
unsigned bsatd, satds[6];

if( m->i_pixel != PIXEL_16x16 && i8 != 0 )
x264_mb_predict_mv( h, 0, i8*4, bw, m->mvp );
pmx = m->mvp[0];
pmy = m->mvp[1];
p_cost_mvx = m->p_cost_mv - pmx;
p_cost_mvy = m->p_cost_mv - pmy;
COST_MV_SATD( bmx, bmy, bsatd );
COST_MV_RD2( bmx, bmy, 0);

/* check the predicted mv */
if( (bmx != pmx || bmy != pmy)
&& pmx >= h->mb.mv_min_spel[0] && pmx <= h->mb.mv_max_spel[0]
&& pmy >= h->mb.mv_min_spel[1] && pmy <= h->mb.mv_max_spel[1] )
{
int satd;
COST_MV_SATD( pmx, pmy, satd );
COST_MV_RD2( pmx, pmy, satd);
}
omx = bmx;
omy = bmy;
/*wide subpel hex search */
for( i = 2; i < 10; i+=2 )
{
if( bmy > h->mb.mv_max_spel[1] - 2 )
break;
omx = bmx;
omy = bmy;
COST_MV_SATD( omx, omy - 2*i, satds[0] );
COST_MV_SATD( omx, omy + 2*i, satds[1] );
COST_MV_SATD( omx - 2*i, omy + 1*i, satds[2] );
COST_MV_SATD( omx - 2*i, omy - 1*i, satds[3] );
COST_MV_SATD( omx + 2*i, omy + 1*i, satds[4] );
COST_MV_SATD( omx + 2*i, omy - 1*i, satds[5] );
COST_MV_RD2( omx, omy - 2*i, satds[0]);
COST_MV_RD2( omx, omy + 2*i, satds[1]);
COST_MV_RD2( omx - 2*i, omy + 1*i, satds[2]);
COST_MV_RD2( omx - 2*i, omy - 1*i, satds[3]);
COST_MV_RD2( omx + 2*i, omy + 1*i, satds[4]);
COST_MV_RD2( omx + 2*i, omy - 1*i, satds[5]);
}

/* subpel hex search */
int dir = -2;
int costs[6];
COST_MV_RD_X3_DIR( -2,0, -1, 2, 1, 2, costs );
COST_MV_RD_X3_DIR( 2,0, 1,-2, -1,-2, costs+3 );
if( dir != -2 )
{
static const int hex2[8][2] = {{-1,-2}, {-2,0}, {-1,2}, {1,2}, {2,0}, {1,-2}, {-1,-2}, {-2,0}};
bmx += hex2[dir+1][0];
bmy += hex2[dir+1][1];
/* half hexagon, not overlapping the previous iteration */
for( i = 1; i < 10; i++ )
{
if( bmy > h->mb.mv_max_spel[1] - 1 )
break;
static const int mod6[8] = {5,0,1,2,3,4,5,0};
const int odir = mod6[dir+1];
COST_MV_RD_X3_DIR( hex2[odir+0][0], hex2[odir+0][1],
hex2[odir+1][0], hex2[odir+1][1],
hex2[odir+2][0], hex2[odir+2][1],
costs );
dir = -2;
COPY2_IF_LT( bcost, costs[0], dir, odir-1 );
COPY2_IF_LT( bcost, costs[1], dir, odir );
COPY2_IF_LT( bcost, costs[2], dir, odir+1 );
if( dir == -2 )
break;
bmx += hex2[dir+1][0];
bmy += hex2[dir+1][1];
}
}
//square refine
omx = bmx;
omy = bmy;
COST_MV_SATD( omx, omy - 1, satds[0] );
COST_MV_SATD( omx, omy + 1, satds[1] );
COST_MV_SATD( omx - 1, omy, satds[2] );
COST_MV_SATD( omx + 1, omy, satds[3] );
COST_MV_RD2( omx, omy - 1, satds[0]);
COST_MV_RD2( omx, omy + 1, satds[1]);
COST_MV_RD2( omx - 1, omy, satds[2]);
COST_MV_RD2( omx + 1, omy, satds[3]);
COST_MV_SATD( omx - 1, omy - 1, satds[0] );
COST_MV_SATD( omx + 1, omy + 1, satds[1] );
COST_MV_SATD( omx - 1, omy + 1, satds[2] );
COST_MV_SATD( omx + 1, omy - 1, satds[3] );
COST_MV_RD2( omx - 1, omy - 1, satds[0]);
COST_MV_RD2( omx + 1, omy + 1, satds[1]);
COST_MV_RD2( omx - 1, omy + 1, satds[2]);
COST_MV_RD2( omx + 1, omy - 1, satds[3]);

if( bmy > h->mb.mv_max_spel[1] )
bmy = h->mb.mv_max_spel[1];

m->cost = bcost;
m->mv[0] = bmx;
m->mv[1] = bmy;

x264_macroblock_cache_mv ( h, 2*(i8&1), i8&2, bw, bh, 0, bmx, bmy );
x264_macroblock_cache_mvd( h, 2*(i8&1), i8&2, bw, bh, 0, bmx - pmx, bmy - pmy );
}

Algorithm:

1. Do the normal initialization and other such stuff done in the regular algorithm. Check the predicted MV, etc.

2. Do a wide hexagon search starting at radius 4 and going to radius 16 (quarterpel numbers, so that's actually 2 to 4). Note that in each pass the center of the hexagon resets to the best vector so far. This might not seem optimal, but testing shows it is considerably better than not doing so, and that the hexagon search here actually improves quality by 0.2% and increases speed because it reduces the number of iterations in step 3.

3. Do an ordinary HEX search starting at the best vector so far, just like in ME HEX.

4. Do a square refine.

5. Finish up.

If there's anything potentially unstable about the algorithm (going out of MVrange, array out of bounds, etc), say so.

Also note that another small boost comes from a change to this matrix:

/* presets selected from good points on the speed-vs-quality curve of several test videos
* subpel_iters[i_subpel_refine] = { refine_hpel, refine_qpel, me_hpel, me_qpel }
* where me_* are the number of EPZS iterations run on all candidate block types,
* and refine_* are run only on the winner. */
static const int subpel_iterations[][4] =
{{1,0,0,0},
{1,1,0,0},
{0,1,1,0},
{0,2,1,0},
{0,2,1,1},
{0,2,1,2},
{0,0,2,2},
{0,0,2,2},
{0,0,4,10}};

Note the extremely high Hpel/Qpel values for --subme 8 (which would become --subme 7). Compared to the cost of the RD search, these take very little time. Also note that the searches called for here are only run until they stop giving any benefit, just like ME HEX, so high values merely serve to help in the rare cases when they would create benefit, and create no speed penalty in any other situation. Also note that these iterations may actually be speeding up the RD algorithm I'm using because they get us closer to the ideal value, meaning fewer HEX iterations are required.

[Terranigma]

I so can't wait to try this. :)

[Dark Shikari]

I so can't wait to try this. :)I might even be able to hex-optimize the ordinary Qpel search :p.

The reason this works is because since the qpel resizing filter does modify the image, the best qpel location is not necessarily right next to the best fullpel location. This means that a plain old diamond search to find the best nearby qpel location might not be the best idea, since the best qpel location might be 1 or 2 pixels away.

Whoever coded the radius-2 hexagon algorithm used in ME_HEX (aku?) did a very good job; its an extremely efficient algorithm and though its not the best quality for a wide search since its only a gradient descent algorithm, it makes up for that with its speed and as such can be used, well, all over the place. I might even create a macro that allows me to specify the comparison function for the hex algorithm to use, so I can call it arbitrarily without even copy-pasting the code.

I've found that in some cases HEX is actually faster than DIA search, because Hex jumps two spaces in each iteration with 3 block comparisons per iteration, while DIA jumps one space with 2 block comparisons per iteration, so HEX actually moves 33% faster. Most of the reason HEX can be slower is because it finds better results than DIA, and so has to continue for a longer time in order to find the best result.

[Sagekilla]

Wow Shikari, have you got enough hexagon searches yet? :p Oh well, if works, keep doing it!

[Dark Shikari]

Wow Shikari, have you got enough hexagon searches yet? :p Oh well, if works, keep doing it!
Well after adding --b-rdo to my commandline x264 crashes about 200 frames in with the above code.

So I'm guessing there's some corner case that causes a segfault or such, probably due to my complete laziness in (not) checking MVranges. It'll probably take me a while to find it :p

And no, there is no such thing as too many hexagons. Its pretty much the optimal search shape for speed and quality in my (limited) experience.

[Dark Shikari]

Bug fixed, it was a variable initialization bug :rolleyes:

Code updated.

[Dark Shikari]

New subme code updated with a new fix that gave another 0.15% quality for about 0.6% speed loss.

Current gain over --subme 7 is about 0.75% quality for the cost of 2.5% speed.

[DeathTheSheep]

That is quite random. And unexpected. And interesting... :cool:

What, is there suddenly a subme 8?

You realize you are truly pleasing (and to no small degree) those folks interested in what are termed "insane" settings. Nice. ;)

[Dark Shikari]

That is quite random. And unexpected. And interesting... :cool:

What, is there suddenly a subme 8?

You realize you are truly pleasing (and to no small degree) those folks interested in what are termed "insane" settings. Nice. ;)On sharktooth's advice I'm just going to have it replace --subme 7 since its only marginally slower and offers 50% more quality boost over --subme 6 than normal --subme 7 does.

And yes, I know I'm making people with OCD (Obsessive Compressive Disorder) happy ;)

[Sharktooth]

well i think the new algo can replace subme 7... there's no reason to use a new subme mode since the speed difference is really low.

[Terranigma]

wizboy11, interested in compiling a build with this patch and hadamard? ;)

[RaynQuist]

That is quite random. And unexpected. And interesting... :cool:

What, is there suddenly a subme 8?

You realize you are truly pleasing (and to no small degree) those folks interested in what are termed "insane" settings. Nice. ;)

Actually it's not pleasing me at all since I can no longer blindly use the slowest options and must find a good trade-off point.

[Dark Shikari]

Actually it's not pleasing me at all since I can no longer blindly use the slowest options and must find a good trade-off point.
Well --me imh --hadamard --threads 2 will give you better speed on a dual core proc than --me esa with somewhat better results, since --me esa doesn't allow threads.

All this means is that --me esa --hadamard --ref 16 --merange 64 is somehow, somehow, not recommended ;)

[akupenguin]

Actually it's not pleasing me at all since I can no longer blindly use the slowest options and must find a good trade-off point.
That was always the case. Or did you blindly use --me esa --merange 512 ?

[TheRyuu]

wizboy11, interested in compiling a build with this patch and hadamard? ;)

Calling me out eh?

I'm getting a "malformed patch" error when I use the hadamard compatible one (and hadamard patch was already installed)

Is that ok?

And it won't compile either. :p

gcc -O4 -ffast-math -Wall -I. -DHAVE_MMX -DARCH_X86 -DSYS_MINGW -DHAVE_PTHREAD -s -fomit-frame-pointer -march=pentium2 -m32 -mfpmath=387 -c -o encoder/me.o encoder/me.c
encoder/me.c: In function 'x264_me_search_ref':
encoder/me.c:366: error: 'X264_ME_IMH' undeclared (first use in this function)
encoder/me.c:366: error: (Each undeclared identifier is reported only once
encoder/me.c:366: error: for each function it appears in.)
make: *** [encoder/me.o] Error 1

This is with the 667b source.

[AGDenton]

Here (http://agerschenfeld.free.fr/x264/x264_satdcmp_imh_aq.tbz)'s a SVN-675 version with IMH, akupenguin's satd_fpel.07, the IMH patch and the AQ patch thrown in for good measure.

It compiles fine, at least on Mac OS X.

I'd also like to add the new --subme 7, but I don't know where to inject the code...

AG

[Dark Shikari]

Here (http://agerschenfeld.free.fr/x264/x264_satdcmp_imh_aq.tbz)'s a SVN-675 version with IMH, akupenguin's satd_fpel.07, the IMH patch and the AQ patch thrown in for good measure.

It compiles fine, at least on Mac OS X.

I'd also like to add the new --subme 7, but I don't know where to inject the code...

AGI'll modify the patch in a bit to make it a --subme 7 instead of --subme 8, then you should be able to combine it easily.

[DeathTheSheep]

Great, please do. :) Your old patches didn't work as well, though (as wizboy and I can both attest to) with MingW's patch system. So we end up doing the whole thing by hand. :p

Akupenguin's patches work like a dream though...how does he do it?!

[Sagekilla]

That is quite random. And unexpected. And interesting... :cool:

What, is there suddenly a subme 8?

You realize you are truly pleasing (and to no small degree) those folks interested in what are termed "insane" settings. Nice. ;)

Oh yes he is, especially come christmas when I may upgrade to a quad core to get practically a 3x boost in raw power.

Edit: At this point, I've done nearly everything short of increasing the reference frames from 4 (which really makes -no- difference from my tests) and enabling ESA. Plus, with all the ME optimizations that Shikari has been up to lately, I'm starting to think using the (new) ESA and leaving refs at 4 would probably give me more quality and less speed loss then actually using the full 16 reference frames!

[akupenguin]

Akupenguin's patches work like a dream though...how does he do it?!
svn diff

[DeathTheSheep]

That's it??! Then why do the other patches behave differently? :eek:

[akupenguin]

Shikari ran instead
diff -r -u5 x264/ x264.edit/
which should also be valid, but isn't identical. I can apply his patches...

[Unearthly]

I managed to almost build it with VS, but unfortunately the SATD esa algorithm uses some C stuff that MS's compiler does not support (array lengths not defined at compile). I'll try to get around to installing Cygwin this weekend.

I tried using IMH with SATD with the ESA commented out for now, and I must say I like it so far. Although for sure you can say the benefit is not worth the speed, it still runs fast enough to run my encode overnight. :p

[Dark Shikari]

Oh yes he is, especially come christmas when I may upgrade to a quad core to get practically a 3x boost in raw power.

Edit: At this point, I've done nearly everything short of increasing the reference frames from 4 (which really makes -no- difference from my tests) and enabling ESA. Plus, with all the ME optimizations that Shikari has been up to lately, I'm starting to think using the (new) ESA and leaving refs at 4 would probably give me more quality and less speed loss then actually using the full 16 reference frames!Yup, that's the point of many of my low-speed-but-higher-quality patches; currently the only thing you can do to sacrifice speed for quality beyond a certain point is to enable more reference frames, which is usually just completely useless.

--me imh is mildly slower than --me umh for a much better gain than you'd get from going from --ref 4 to --ref 7, for example (probably a similar speed loss). --hadamard slows things down by about 50% for UMH and 60-70% for IMH, but is still better than the benefit from --ref 4 to --ref 16.

--me esa --hadamard is still crazy slow though :p

[Dark Shikari]

Here's your --subme 7 diff... it only affects one C file and it doesn't conflict at all with --hadamard or other patches as far as I can tell.

--- x264/encoder/me.c 2007-07-17 04:13:25.937500000 -0700
+++ x264/encoder/me.c 2007-09-13 17:24:40.390625000 -0700
@@ -27,19 +27,22 @@

/* presets selected from good points on the speed-vs-quality curve of several test videos
* subpel_iters[i_subpel_refine] = { refine_hpel, refine_qpel, me_hpel, me_qpel }
* where me_* are the number of EPZS iterations run on all candidate block types,
* and refine_* are run only on the winner. */
+ //The --subme 7 values are much higher because since they get the motion search
+ //closer to the optimal value, they actually tend to save time in the more intensive
+ //RD search that follows.
static const int subpel_iterations[][4] =
{{1,0,0,0},
{1,1,0,0},
{0,1,1,0},
{0,2,1,0},
{0,2,1,1},
{0,2,1,2},
{0,0,2,2},
- {0,0,2,2}};
+ {0,0,4,10}};

static void refine_subpel( x264_t *h, x264_me_t *m, int hpel_iters, int qpel_iters, int *p_halfpel_thresh, int b_refine_qpel );

#define BITS_MVD( mx, my )\
(p_cost_mvx[(mx)<<2] + p_cost_mvy[(my)<<2])
@@ -535,11 +538,11 @@
int hpel = subpel_iterations[h->mb.i_subpel_refine][0];
int qpel = subpel_iterations[h->mb.i_subpel_refine][1];

if( m->i_pixel <= PIXEL_8x8 && h->sh.i_type == SLICE_TYPE_P )
m->cost -= m->i_ref_cost;
-
+
refine_subpel( h, m, hpel, qpel, NULL, 1 );
}

#define COST_MV_SAD( mx, my ) \
{ \
@@ -799,25 +802,56 @@
dst = h->pixf.mbcmp[i_pixel]( m->p_fenc[0], FENC_STRIDE, src, stride ) \
+ p_cost_mvx[mx] + p_cost_mvy[my]; \
COPY1_IF_LT( bsatd, dst ); \
}

-#define COST_MV_RD( mx, my, satd, dir ) \
+#define COST_MV_RD( mx, my, satd) \
{ \
- if( satd <= bsatd * SATD_THRESH \
- && (dir^1) != odir \
- && (dir<0 || !p_visited[(mx)+(my)*16]) ) \
+ if( satd <= bsatd * SATD_THRESH)\
{ \
int cost; \
cache_mv[0] = cache_mv2[0] = mx; \
cache_mv[1] = cache_mv2[1] = my; \
cost = x264_rd_cost_part( h, i_lambda2, i8, m->i_pixel ); \
COPY3_IF_LT( bcost, cost, bmx, mx, bmy, my ); \
- if(dir>=0) p_visited[(mx)+(my)*16] = 1; \
} \
}

+#define COST_MV_RD_X3_DIR( m0x, m0y, m1x, m1y, m2x, m2y, costs) \
+{ \
+ COST_MV_SATD(m0x+omx,m0y+omy, satds[0]);\
+ COST_MV_SATD(m1x+omx,m1y+omy, satds[1]);\
+ COST_MV_SATD(m2x+omx,m2y+omy, satds[2]);\
+ if( satds[0] <= bsatd * SATD_THRESH) \
+ { \
+ cache_mv[0] = cache_mv2[0] = m0x+omx; \
+ cache_mv[1] = cache_mv2[1] = m0y+omy; \
+ int cost = x264_rd_cost_part( h, i_lambda2, i8, m->i_pixel ); \
+ (costs)[0] = cost;\
+ COPY3_IF_LT( bcost, cost, bmx, m0x+omx, bmy, m0y+omy ); \
+ } \
+ else (costs)[0] = COST_MAX;\
+ if( satds[1] <= bsatd * SATD_THRESH) \
+ { \
+ cache_mv[0] = cache_mv2[0] = m1x+omx; \
+ cache_mv[1] = cache_mv2[1] = m1y+omy; \
+ int cost = x264_rd_cost_part( h, i_lambda2, i8, m->i_pixel ); \
+ (costs)[1] = cost;\
+ COPY3_IF_LT( bcost, cost, bmx, m1x+omx, bmy, m1y+omy ); \
+ } \
+ else (costs)[1] = COST_MAX;\
+ if( satds[2] <= bsatd * SATD_THRESH) \
+ { \
+ cache_mv[0] = cache_mv2[0] = m2x+omx; \
+ cache_mv[1] = cache_mv2[1] = m2y+omy; \
+ int cost = x264_rd_cost_part( h, i_lambda2, i8, m->i_pixel ); \
+ (costs)[2] = cost;\
+ COPY3_IF_LT( bcost, cost, bmx, m2x+omx, bmy, m2y+omy ); \
+ } \
+ else (costs)[2] = COST_MAX;\
+}
+
#define SATD_THRESH 17/16

void x264_me_refine_qpel_rd( x264_t *h, x264_me_t *m, int i_lambda2, int i8 )
{
// don't have to fill the whole mv cache rectangle
@@ -826,100 +860,114 @@
int16_t *cache_mv2 = cache_mv + pixel_mv_offs[m->i_pixel];
const int16_t *p_cost_mvx, *p_cost_mvy;
const int bw = x264_pixel_size[m->i_pixel].w>>2;
const int bh = x264_pixel_size[m->i_pixel].h>>2;
const int i_pixel = m->i_pixel;
-
DECLARE_ALIGNED( uint8_t, pix[16*16], 16 );
int bcost = m->i_pixel == PIXEL_16x16 ? m->cost : COST_MAX;
int bmx = m->mv[0];
int bmy = m->mv[1];
int pmx, pmy, omx, omy, i;
- int odir = -1, bdir;
- unsigned bsatd, satds[4];
-
- int visited[16*13] = {0}; // only need 13x13, but 16 is more convenient
- int *p_visited = &visited[6+6*16];
-
+ unsigned bsatd, satds[6];
+ omx = bmx;
+ omy = bmy;
if( m->i_pixel != PIXEL_16x16 && i8 != 0 )
x264_mb_predict_mv( h, 0, i8*4, bw, m->mvp );
pmx = m->mvp[0];
pmy = m->mvp[1];
p_cost_mvx = m->p_cost_mv - pmx;
p_cost_mvy = m->p_cost_mv - pmy;
COST_MV_SATD( bmx, bmy, bsatd );
- if( m->i_pixel != PIXEL_16x16 )
- COST_MV_RD( bmx, bmy, 0, -1 );
-
+ COST_MV_RD( bmx, bmy, 0);
+
/* check the predicted mv */
if( (bmx != pmx || bmy != pmy)
&& pmx >= h->mb.mv_min_spel[0] && pmx <= h->mb.mv_max_spel[0]
&& pmy >= h->mb.mv_min_spel[1] && pmy <= h->mb.mv_max_spel[1] )
{
int satd;
COST_MV_SATD( pmx, pmy, satd );
- COST_MV_RD( pmx, pmy, satd, -1 );
+ COST_MV_RD( pmx, pmy, satd);
}
-
- /* mark mv and mvp as visited */
- p_visited[0] = 1;
- p_visited -= bmx + bmy*16;
- {
- int mx = bmx ^ m->mv[0] ^ pmx;
- int my = bmy ^ m->mv[1] ^ pmy;
- if( abs(mx-bmx) < 7 && abs(my-bmy) < 7 )
- p_visited[mx + my*16] = 1;
- }
-
- /* hpel diamond */
- bdir = -1;
- for( i = 0; i < 2; i++ )
+
+ // Wide subpel hexagon search. The time it takes is basically negated
+ // by the time it saves in the hex search that follows, so its basically a free
+ // quality boost.
+ for( i = 2; i < 10; i+=2 )
{
if( bmy > h->mb.mv_max_spel[1] - 2 )
break;
omx = bmx;
omy = bmy;
- odir = bdir;
- COST_MV_SATD( omx, omy - 2, satds[0] );
- COST_MV_SATD( omx, omy + 2, satds[1] );
- COST_MV_SATD( omx - 2, omy, satds[2] );
- COST_MV_SATD( omx + 2, omy, satds[3] );
- COST_MV_RD( omx, omy - 2, satds[0], 0 );
- COST_MV_RD( omx, omy + 2, satds[1], 1 );
- COST_MV_RD( omx - 2, omy, satds[2], 2 );
- COST_MV_RD( omx + 2, omy, satds[3], 3 );
- if( bmx == omx && bmy == omy )
- break;
+ COST_MV_SATD( omx, omy - 2*i, satds[0] );
+ COST_MV_SATD( omx, omy + 2*i, satds[1] );
+ COST_MV_SATD( omx - 2*i, omy + 1*i, satds[2] );
+ COST_MV_SATD( omx - 2*i, omy - 1*i, satds[3] );
+ COST_MV_SATD( omx + 2*i, omy + 1*i, satds[4] );
+ COST_MV_SATD( omx + 2*i, omy - 1*i, satds[5] );
+ COST_MV_RD( omx, omy - 2*i, satds[0]);
+ COST_MV_RD( omx, omy + 2*i, satds[1]);
+ COST_MV_RD( omx - 2*i, omy + 1*i, satds[2]);
+ COST_MV_RD( omx - 2*i, omy - 1*i, satds[3]);
+ COST_MV_RD( omx + 2*i, omy + 1*i, satds[4]);
+ COST_MV_RD( omx + 2*i, omy - 1*i, satds[5]);
}
-
- /* qpel diamond */
- bdir = -1;
- for( i = 0; i < 2; i++ )
- {
- if( bmy > h->mb.mv_max_spel[1] - 1 )
- break;
- omx = bmx;
- omy = bmy;
- odir = bdir;
- COST_MV_SATD( omx, omy - 1, satds[0] );
- COST_MV_SATD( omx, omy + 1, satds[1] );
- COST_MV_SATD( omx - 1, omy, satds[2] );
- COST_MV_SATD( omx + 1, omy, satds[3] );
- COST_MV_RD( omx, omy - 1, satds[0], 0 );
- COST_MV_RD( omx, omy + 1, satds[1], 1 );
- COST_MV_RD( omx - 1, omy, satds[2], 2 );
- COST_MV_RD( omx + 1, omy, satds[3], 3 );
- if( bmx == omx && bmy == omy )
- break;
+ //Subpel hex search. Same as ME HEX except with RDO.
+ int dir = -2;
+ int costs[6];
+ COST_MV_RD_X3_DIR( -2,0, -1, 2, 1, 2, costs );
+ COST_MV_RD_X3_DIR( 2,0, 1,-2, -1,-2, costs+3 );
+ if( dir != -2 )
+ {
+ static const int hex2[8][2] = {{-1,-2}, {-2,0}, {-1,2}, {1,2}, {2,0}, {1,-2}, {-1,-2}, {-2,0}};
+ bmx += hex2[dir+1][0];
+ bmy += hex2[dir+1][1];
+ /* half hexagon, not overlapping the previous iteration */
+ for( i = 1; i < 10; i++ )
+ {
+ if( bmy > h->mb.mv_max_spel[1] - 1 )
+ break;
+ static const int mod6[8] = {5,0,1,2,3,4,5,0};
+ const int odir = mod6[dir+1];
+ COST_MV_RD_X3_DIR( hex2[odir+0][0], hex2[odir+0][1],
+ hex2[odir+1][0], hex2[odir+1][1],
+ hex2[odir+2][0], hex2[odir+2][1],
+ costs );
+ dir = -2;
+ COPY2_IF_LT( bcost, costs[0], dir, odir-1 );
+ COPY2_IF_LT( bcost, costs[1], dir, odir );
+ COPY2_IF_LT( bcost, costs[2], dir, odir+1 );
+ if( dir == -2 )
+ break;
+ bmx += hex2[dir+1][0];
+ bmy += hex2[dir+1][1];
+ }
}
-
+ //Square refine. Same as in ME HEX.
+ omx = bmx;
+ omy = bmy;
+ COST_MV_SATD( omx, omy - 1, satds[0] );
+ COST_MV_SATD( omx, omy + 1, satds[1] );
+ COST_MV_SATD( omx - 1, omy, satds[2] );
+ COST_MV_SATD( omx + 1, omy, satds[3] );
+ COST_MV_RD( omx, omy - 1, satds[0]);
+ COST_MV_RD( omx, omy + 1, satds[1]);
+ COST_MV_RD( omx - 1, omy, satds[2]);
+ COST_MV_RD( omx + 1, omy, satds[3]);
+ COST_MV_SATD( omx - 1, omy - 1, satds[0] );
+ COST_MV_SATD( omx + 1, omy + 1, satds[1] );
+ COST_MV_SATD( omx - 1, omy + 1, satds[2] );
+ COST_MV_SATD( omx + 1, omy - 1, satds[3] );
+ COST_MV_RD( omx - 1, omy - 1, satds[0]);
+ COST_MV_RD( omx + 1, omy + 1, satds[1]);
+ COST_MV_RD( omx - 1, omy + 1, satds[2]);
+ COST_MV_RD( omx + 1, omy - 1, satds[3]);
if( bmy > h->mb.mv_max_spel[1] )
bmy = h->mb.mv_max_spel[1];

m->cost = bcost;
m->mv[0] = bmx;
m->mv[1] = bmy;

x264_macroblock_cache_mv ( h, 2*(i8&1), i8&2, bw, bh, 0, bmx, bmy );
x264_macroblock_cache_mvd( h, 2*(i8&1), i8&2, bw, bh, 0, bmx - pmx, bmy - pmy );
}
-

[foxyshadis]

I managed to almost build it with VS, but unfortunately the SATD esa algorithm uses some C stuff that MS's compiler does not support (array lengths not defined at compile). I'll try to get around to installing Cygwin this weekend.

I tried using IMH with SATD with the ESA commented out for now, and I must say I like it so far. Although for sure you can say the benefit is not worth the speed, it still runs fast enough to run my encode overnight. :p

Shikari's C99isms are all of the random declaration variety, here's a patch to fix that in VC8: http://foxyshadis.slightlydark.com/random/subme7_vc8-2.diff The IMH patch has a couple instances of this, but all you need to do is move several declarations before "if(bcost < 100) goto me_hex2;" and pull "static const int mod6[8] = {5,0,1,2,3,4,5,0};\" to the opening of its scope.

It's the hadamard that has the var-arrays, for that one: http://foxyshadis.slightlydark.com/random/x264_satd_fpel.07-vc8.diff

I'll likely take them down once this is over.

[DeathTheSheep]

@AGDenton: maybe you didn't notice, but that's pretty broken. Not even one mention of IMH in x264.c, and that's as far as I looked. Mac OS X... :rolleyes:

@Dark Shikari: Every time I tried to stick that in a .diff, I get patch unexpectedly ends in middle of line
patch: **** malformed patch at line 274:
But I patched it by hand and all is well (but it took a while).

[Dark Shikari]

@AGDenton: maybe you didn't notice, but that's pretty broken. Not even one mention of IMH in x264.c, and that's as far as I looked. Mac OS X... :rolleyes:

@Dark Shikari: Every time I tried to stick that in a .diff, I get patch unexpectedly ends in middle of line
patch: **** malformed patch at line 274:
But I patched it by hand and all is well (but it took a while).I fixed it, I think its because I truncated a - off the end :p

Also, by the way, I'm not done yet--my next area of the code I'm looking to improve is the analysis of predictors before the main motion estimation function--the algorithm used here is relatively naive, so after all these patches go through, or perhaps even before, I'll have something fancier out :)

[AGDenton]

@DeathTheSheep : that would be because it's in x264.h... Since I don't use the CLI (only libx264.a), I didn't notice. The rest works fine, though.

Here (http://agerschenfeld.free.fr/x264/x264_satdcmp_aq_subme_imh.tbz)'s the corrected source, updated with the new subme -7.

[Dark Shikari]

A new patch for you folks... this is quite beta and I'm not sure if it'll even patch properly, but as long as it can eventually get integrated into the "Big bunch of Dark Shikari patches" things should be fine.

diff -u5 -r x264/common/common.c x264/common/common.c
--- x264/common/common.c 2007-07-19 03:14:52.387375000 -0700
+++ x264/common/common.c 2007-09-13 23:47:49.296875000 -0700
@@ -449,10 +449,12 @@
p->analyse.i_mv_range = atoi(value);
OPT2("mvrange-thread", "mv-range-thread")
p->analyse.i_mv_range_thread = atoi(value);
OPT2("subme", "subq")
p->analyse.i_subpel_refine = atoi(value);
+ OPT2("me-prepass", "meprepass")
+ p->analyse.i_me_prepass = atobool(value);
OPT("bime")
p->analyse.b_bidir_me = atobool(value);
OPT("chroma-me")
p->analyse.b_chroma_me = atobool(value);
OPT2("b-rdo", "brdo")
@@ -900,10 +902,11 @@
s += sprintf( s, " deblock=%d:%d:%d", p->b_deblocking_filter,
p->i_deblocking_filter_alphac0, p->i_deblocking_filter_beta );
s += sprintf( s, " analyse=%#x:%#x", p->analyse.intra, p->analyse.inter );
s += sprintf( s, " me=%s", x264_motion_est_names[ p->analyse.i_me_method ] );
s += sprintf( s, " subme=%d", p->analyse.i_subpel_refine );
+ s += sprintf( s, " me-prepass=%d", p->analyse.i_me_prepass );
s += sprintf( s, " brdo=%d", p->analyse.b_bframe_rdo );
s += sprintf( s, " mixed_ref=%d", p->analyse.b_mixed_references );
s += sprintf( s, " me_range=%d", p->analyse.i_me_range );
s += sprintf( s, " chroma_me=%d", p->analyse.b_chroma_me );
s += sprintf( s, " trellis=%d", p->analyse.i_trellis );
diff -u5 -r x264/encoder/me.c x264/encoder/me.c
--- x264/encoder/me.c 2007-07-17 04:13:25.937500000 -0700
+++ x264/encoder/me.c 2007-09-14 00:01:47.140625000 -0700
@@ -59,10 +59,27 @@
int cost = h->pixf.fpelcmp[i_pixel]( m->p_fenc[0], FENC_STRIDE, src, stride ) \
+ p_cost_mvx[ mx ] + p_cost_mvy[ my ]; \
COPY3_IF_LT( bpred_cost, cost, bpred_mx, mx, bpred_my, my ); \
}

+#define COST_MV_HPEL2( mx, my, cost ) \
+{ \
+ int stride = 16; \
+ uint8_t *src = h->mc.get_ref( m->p_fref, m->i_stride[0], pix, &stride, mx, my, bw, bh ); \
+ cost = h->pixf.fpelcmp[i_pixel]( m->p_fenc[0], FENC_STRIDE, src, stride ) \
+ + p_cost_mvx[ mx ] + p_cost_mvy[ my ]; \
+}
+
+#define COST_MV_HPEL3( mx, my) \
+{ \
+ int stride = 16; \
+ uint8_t *src = h->mc.get_ref( m->p_fref, m->i_stride[0], pix, &stride, mx, my, bw, bh ); \
+ int cost = h->pixf.fpelcmp[i_pixel]( m->p_fenc[0], FENC_STRIDE, src, stride ) \
+ + p_cost_mvx[ mx ] + p_cost_mvy[ my ]; \
+ COPY3_IF_LT( bestcost, cost, bestx, mx, besty, my ); \
+}
+
#define COST_MV_X3_DIR( m0x, m0y, m1x, m1y, m2x, m2y, costs )\
{\
uint8_t *pix_base = p_fref + bmx + bmy*m->i_stride[0];\
h->pixf.fpelcmp_x3[i_pixel]( m->p_fenc[0],\
pix_base + (m0x) + (m0y)*m->i_stride[0],\
@@ -175,25 +192,92 @@
bmx = x264_clip3( m->mvp[0], mv_x_min*4, mv_x_max*4 );
bmy = x264_clip3( m->mvp[1], mv_y_min*4, mv_y_max*4 );
pmx = ( bmx + 2 ) >> 2;
pmy = ( bmy + 2 ) >> 2;
bcost = COST_MAX;
-
+
/* try extra predictors if provided */
if( h->mb.i_subpel_refine >= 3 )
{
- COST_MV_HPEL( bmx, bmy );
- for( i = 0; i < i_mvc; i++ )
- {
- const int mx = x264_clip3( mvc[i][0], mv_x_min*4, mv_x_max*4 );
- const int my = x264_clip3( mvc[i][1], mv_y_min*4, mv_y_max*4 );
- if( mx != bpred_mx || my != bpred_my )
- COST_MV_HPEL( mx, my );
- }
- bmx = ( bpred_mx + 2 ) >> 2;
- bmy = ( bpred_my + 2 ) >> 2;
- COST_MV( bmx, bmy );
+ COST_MV_HPEL( bmx, bmy );
+ if(!h->param.analyse.i_me_prepass)
+ {
+ for( i = 0; i < i_mvc; i++ )
+ {
+ const int mx = x264_clip3( mvc[i][0], mv_x_min*4, mv_x_max*4 );
+ const int my = x264_clip3( mvc[i][1], mv_y_min*4, mv_y_max*4 );
+ if( mx != bpred_mx || my != bpred_my )
+ COST_MV_HPEL( mx, my );
+ }
+ }
+ else
+ {
+ for( i = 0; i < i_mvc; i++ )
+ {
+ const int mx = x264_clip3( mvc[i][0], mv_x_min*4, mv_x_max*4 );
+ const int my = x264_clip3( mvc[i][1], mv_y_min*4, mv_y_max*4 );
+ if( mx != bpred_mx || my != bpred_my )
+ {
+ int bestcost;
+ int bestx = mx;
+ int besty = my;
+ COST_MV_HPEL2( mx, my, bestcost );
+ COPY3_IF_LT( bpred_cost, bestcost, bpred_mx, bestx, bpred_my, besty );
+ if(bestcost < 2*bpred_cost)
+ {
+ int n;
+ int dir = -2;
+ COST_MV_HPEL2(bestx-2,besty,costs[0]);
+ COST_MV_HPEL2(bestx-1,besty+2,costs[1]);
+ COST_MV_HPEL2(bestx+1,besty+2,costs[2]);
+ COST_MV_HPEL2(bestx+2,besty,costs[3]);
+ COST_MV_HPEL2(bestx+1,besty-2,costs[4]);
+ COST_MV_HPEL2(bestx-1,besty-2,costs[5]);
+ COPY2_IF_LT( bestcost, costs[0], dir, 0 );
+ COPY2_IF_LT( bestcost, costs[1], dir, 1 );
+ COPY2_IF_LT( bestcost, costs[2], dir, 2 );
+ COPY2_IF_LT( bestcost, costs[3], dir, 3 );
+ COPY2_IF_LT( bestcost, costs[4], dir, 4 );
+ COPY2_IF_LT( bestcost, costs[5], dir, 5 );
+ if( dir != -2 )
+ {
+ static const int hex2[8][2] = {{-1,-2}, {-2,0}, {-1,2}, {1,2}, {2,0}, {1,-2}, {-1,-2}, {-2,0}};
+ bestx += hex2[dir+1][0];
+ besty += hex2[dir+1][1];
+ for( n = 1; n < i_me_range && CHECK_MVRANGE(bestx, besty); n++ )
+ {
+ static const int mod6[8] = {5,0,1,2,3,4,5,0};
+ const int odir = mod6[dir+1];
+ COST_MV_HPEL2(hex2[odir+0][0]+bestx,hex2[odir+0][1]+besty,costs[0]);
+ COST_MV_HPEL2(hex2[odir+1][0]+bestx,hex2[odir+1][1]+besty,costs[1]);
+ COST_MV_HPEL2(hex2[odir+2][0]+bestx,hex2[odir+2][1]+besty,costs[2]);
+ dir = -2;
+ COPY2_IF_LT( bestcost, costs[0], dir, odir-1 );
+ COPY2_IF_LT( bestcost, costs[1], dir, odir );
+ COPY2_IF_LT( bestcost, costs[2], dir, odir+1 );
+ if( dir == -2 )
+ break;
+ bestx += hex2[dir+1][0];
+ besty += hex2[dir+1][1];
+ }
+ }
+ COST_MV_HPEL3(bestx+1,besty-1);
+ COST_MV_HPEL3(bestx+1,besty);
+ COST_MV_HPEL3(bestx+1,besty+1);
+ COST_MV_HPEL3(bestx,besty-1);
+ COST_MV_HPEL3(bestx,besty+1);
+ COST_MV_HPEL3(bestx-1,besty-1);
+ COST_MV_HPEL3(bestx-1,besty);
+ COST_MV_HPEL3(bestx-1,besty+1);
+ COPY3_IF_LT(bpred_cost,bestcost,bpred_mx,bestx,bpred_my,besty);
+ }
+ }
+ }
+ }
+ bmx = ( bpred_mx + 2 ) >> 2;
+ bmy = ( bpred_my + 2 ) >> 2;
+ COST_MV( bmx, bmy );
}
else
{
/* check the MVP */
COST_MV( pmx, pmy );
diff -u5 -r x264/x264.c x264/x264.c
--- x264/x264.c 2007-07-17 04:13:26.437500000 -0700
+++ x264/x264.c 2007-09-13 23:46:01.937500000 -0700
@@ -230,11 +230,12 @@
H0( " --merange <integer> Maximum motion vector search range [%d]\n", defaults->analyse.i_me_range );
H1( " --mvrange <integer> Maximum motion vector length [-1 (auto)]\n" );
H1( " --mvrange-thread <int> Minimum buffer between threads [-1 (auto)]\n" );
H0( " -m, --subme <integer> Subpixel motion estimation and partition\n"
" decision quality: 1=fast, 7=best. [%d]\n", defaults->analyse.i_subpel_refine );
- H0( " --b-rdo RD based mode decision for B-frames. Requires subme 6.\n" );
+ H0( " --me-prepass Run an ME prepass on predictors. Requires subme 3 or higher.\n");
+ H0( " --b-rdo RD based mode decision for B-frames. Requires subme 6 or higher.\n" );
H0( " --mixed-refs Decide references on a per partition basis\n" );
H1( " --no-chroma-me Ignore chroma in motion estimation\n" );
H1( " --bime Jointly optimize both MVs in B-frames\n" );
H0( " -8, --8x8dct Adaptive spatial transform size\n" );
H0( " -t, --trellis <integer> Trellis RD quantization. Requires CABAC. [%d]\n"
@@ -403,10 +404,11 @@
{ "me", required_argument, NULL, 0 },
{ "merange", required_argument, NULL, 0 },
{ "mvrange", required_argument, NULL, 0 },
{ "mvrange-thread", required_argument, NULL, 0 },
{ "subme", required_argument, NULL, 'm' },
+ { "me-prepass", no_argument, NULL, 0 },
{ "b-rdo", no_argument, NULL, 0 },
{ "mixed-refs", no_argument, NULL, 0 },
{ "no-chroma-me", no_argument, NULL, 0 },
{ "bime", no_argument, NULL, 0 },
{ "8x8dct", no_argument, NULL, '8' },
diff -u5 -r x264/x264.h x264/x264.h
--- x264/x264.h 2007-07-12 22:46:06.718500000 -0700
+++ x264/x264.h 2007-09-13 23:46:32.562500000 -0700
@@ -219,10 +219,11 @@
int i_me_method; /* motion estimation algorithm to use (X264_ME_*) */
int i_me_range; /* integer pixel motion estimation search range (from predicted mv) */
int i_mv_range; /* maximum length of a mv (in pixels). -1 = auto, based on level */
int i_mv_range_thread; /* minimum space between threads. -1 = auto, based on number of threads. */
int i_subpel_refine; /* subpixel motion estimation quality */
+ int i_me_prepass; /* run an ME prepass on predictors */
int b_bidir_me; /* jointly optimize both MVs in B-frames */
int b_chroma_me; /* chroma ME for subpel and mode decision in P-frames */
int b_bframe_rdo; /* RD based mode decision for B-frames */
int b_mixed_references; /* allow each mb partition in P-frames to have it's own reference number */
int i_trellis; /* trellis RD quantization */

What it does: Runs an ME prepass on all predictors. The prepass is in the form of a hex search.

Effect: at pretty high settings (--me umh, --subme 7, --no-fast-pskip, etc), a 16% drop in speed and a 1% increase in quality. This means its more effective at boosting quality than --me imh relative to the speed cost. Note also the speed cost is independent of the ME algorithm chosen; so if you're doing say --merange 24 --me imh, you'll hardly notice the speed loss from this algorithm. That also means, of course, that if you use --me-prepass on very low encoding settings you'll notice a huge speed drop.

Commandline: --me-prepass

Combining my improved --subme 7 (+0.75% quality), --me imh (+1% quality) and --me-prepass (+1% quality) will get you roughly the same boost you'd get going from --subme 5 to --subme 7 all over again; probably a much better choice than going from --ref 8 to --ref 16.

[Terranigma]

Can anyone compile a working build with the patches for x86 & x64, or link all that's required so that I can do this myself?

[Terranigma]

Sorry to double post, But I wanted to say that Cef (http://mirror05.x264.nl/Cef/) has an experimental build up with imh and the improved subme7. The only thing that's missing is --me-prepass command.

[Cef]

Can anyone compile a working build with the patches for x86 & x64, or link all that's required so that I can do this myself?

He (http://mirror05.x264.nl/Cef/force.php?file=./x264_x86_dark.7z)re (http://mirror05.x264.nl/Cef/force.php?file=./x264_x64_dark.7z) you go.

x264_xxx_exp corresponds to this (http://forum.doom9.org/showthread.php?p=1044198#post1044198) request. It has hrd_pulldown669h.diff, some Bframe changes, and some patches from Dark_Shikari, and some other stuff. Check the .rar in Sagittaire's post to see exactly.

[Terranigma]

Thanks a lot. Everything is running smoothly. :)

[Terranigma]

cef, no hadamard?

[vortex_hl]

cef, no hadamard?
imho it's now --fpel-cmp

[Terranigma]

I'm sorry, but when I input that, it gives me an error.
http://img62.imageshack.us/img62/4698/hadamarderrorbp3.png

[Dark Shikari]

imho it's now --fpel-cmpThat's a really bad commandline IMO (if its a boolean); it says nothing about what it does. If it was a string-based commandline like --fpel-cmp sad vs --fpel-cmp satd, that would be fine though.

[Terranigma]

Thanks Dark Shikari, it now works. The user before me did'nt specify that i'd have to enter in sad or satd after the command. :)

[Dark Shikari]

Thanks Dark Shikari, it now works. The user before me did'nt specify that i'd have to enter in sad or satd after the command. :)Hah, so apparently we were thinking the same thing about the commandlines :p:)

[DeathTheSheep]

Cef, how exactly did you get these patches to work?
In Msys, trying: patch -u -p1 < imh.diff (or some others) throws errors and doesn't patch x264.c or x264.h! (and akupenguin's patch works but doesn't find the files to patch, so I have to specify the directories of each file individually).

What program do you use? What commandline?

[Unearthly]

Ok, so I gave up compiling with VS as it just wouldn't do SATD ESA right (crashed every time). I installed cygwin and got a build with the:
-SATD (hadamard)
-IMH
-New subme7
-me-prepass
patches.

Compiled with pthreads, SSSE3, but no mp4, gpac was not working and I don't feel like fighting it at the moment.

http://rapidshare.com/files/55808006/x264.zip.html

Giant merged patch (for SVN 675) included in the zip.

Going to test some of these new options overnight now.

[foxyshadis]

Did you guys replace if(bcost < 100) with if(bcost < SAD_THRESH(100))? I forgot to do that in the patch I put up. You'll have to double-define x264_pixel_size_shift or raise its scope though.

[Dark Shikari]

Did you guys replace if(bcost < 100) with if(bcost < SAD_THRESH(100))? I forgot to do that in the patch I put up.
Yeah, that should be fixed. I'm not sure if I updated my patch with that. It gives a small quality boost, and more importantly, its more correct.

[Unearthly]

It's not in the patch in the first post. Also, SAD_THRESH requires 'x264_pixel_size_shift'. Is this:

static const int x264_pixel_size_shift[7] = { 0, 1, 1, 2, 3, 3, 4 };

the right one? (It's the only one I saw.)

[Dark Shikari]

It's not in the patch in the first post. Also, SAD_THRESH requires 'x264_pixel_size_shift'. Is this:

static const int x264_pixel_size_shift[7] = { 0, 1, 1, 2, 3, 3, 4 };

the right one? (It's the only one I saw.)Yup, that's it.

Basically what that matrix does is shift 100 by the correct value depending on what size partition the motion estimation function is dealing with.

If you use (bcost < 100), that applies far more strongly to 4x4 blocks, for example, than 16x16 blocks. SAD_THRESH(100) divides by 16, for example, when dealing with 4x4 blocks, since they're 16 times smaller.

[Unearthly]

Updated the link in my post with the new build and patch.

[Dark Shikari]

One slight problem it seems... IMH is listed above UMH on x264 --longhelp. Not sure if this was my mistake or someone else's.

Right now it is

DIA
HEX
IMH
UMH
ESA

which implies that IMH is faster than UMH, which its not. It should be flipped.

[Unearthly]

My mistake, due to some manual merging (start using svn diff's and it won't happen again ;) ). Though anyone downloading this experimental build should know the difference.

[Dark Shikari]

My mistake, due to some manual merging (start using svn diff's and it won't happen again ;) ). Though anyone downloading this experimental build should know the difference.What's the best way to use svn diff? I'm not really familiar with svn except in terms of the usual downloading source from svn, so what commandline would I use?

[Unearthly]

I use TortoiseSVN on Windows which has a 'Create Patch' option, it seems the CLI equivalent would be:

svn diff >mypatch.diff

That will create a patch for all of your changed files in the directory. I'm not sure if there is an easy way to pick and choose which files to patch using CLI though, maybe someone else can help with that.

[Dark Shikari]

I use TortoiseSVN on Windows which has a 'Create Patch' option, it seems the CLI equivalent would be:

svn diff >mypatch.diff

That will create a patch for all of your changed files in the directory. I'm not sure if there is an easy way to pick and choose which files to patch using CLI though, maybe someone else can help with that.One reason why I like regular diff is because I can very easily remove sections from it... because often the diff has far, far more things in it than I actually want to be in the patch :p

My programming organization is a complete mess.

[DeathTheSheep]

due to some manual merging (start using svn diff's and it won't happen again ).

This. ;)

[foxyshadis]

It's the same unified diff, you can still remove chunks, the headers are just a bit different. I can mostly get away with just swapping out the headers, in fact, though sometimes line numbers need fudging.

[DeathTheSheep]

Header editing, I see. Having no clue what to swap with what, manual editing is the way to go for me. Which takes some time. :)

Dark Shikari, what's the best way to apply your patches (i.e. what works for you without manually editing in some of the code)?

[Unearthly]

One reason why I like regular diff is because I can very easily remove sections from it... because often the diff has far, far more things in it than I actually want to be in the patch :p

My programming organization is a complete mess.

The only difference I know for sure is the header:

Index: common/common.c
===================================================================
--- common/common.c (revision 675)
+++ common/common.c (working copy)

The actual contents of the diff should be the same, though your diffs often didn't match the SVN source so I thought it might have to do with your different patch creator.

So you can edit them just the same. Just be careful manually editing in case the start line/number of lines change, because then if they aren't right the patch won't work. Removing whole sections should always work though.

[Sharktooth]

that's called unified diff... the header tells what files are changed so the patch can be applied to all the files.
add -u switch to enable the unified diff creation

http://en.wikipedia.org/wiki/Diff#Unified_format

[DeathTheSheep]

In the mean time, could someone apply all the patches to the source code and upload that patched code? I'd like to take a look at things when they're all laid out in one source. :)

[Dark Shikari]

Anyways, here's some statistics on the new version.

Source: The very difficult wire scene of Elephant's Dream, 1001 frames, 1280x720.

Commandline:
x264_new --bframe 16 --b-pyramid --me imh --threads 2 --thread-input --me-prepass --ref 4 --mixed-refs --merange 16 --subme 7 --b-rdo --trellis 2 --bime --weightb --partitions all --direct auto --8x8dct --no-fast-pskip --crf 20

SSIM: 0.9761527
OPSNR: 41.918
Bitrate: 5764.82
FPS: 1.16
Improvement: 3.4%

Commandline:
x264_new --bframe 16 --b-pyramid --me imh --threads 2 --thread-input --ref 4 --mixed-refs --merange 16 --subme 7 --b-rdo --trellis 2 --bime --weightb --partitions all --direct auto --8x8dct --no-fast-pskip --crf 20

SSIM: 0.9753538
OPSNR: 41.782
Bitrate: 5697.25
FPS: 1.39
Improvement: 1.24%

Commandline:
x264_new --bframe 16 --b-pyramid --me umh --me-prepass --threads 2 --thread-input --ref 4 --mixed-refs --merange 16 --subme 7 --b-rdo --trellis 2 --bime --weightb --partitions all --direct auto --8x8dct --no-fast-pskip --crf 20

SSIM: 0.9761320
OPSNR: 41.919
Bitrate: 5788.20
FPS: 1.32
Improvement: 2.9%

Commandline:
x264_new --bframe 16 --b-pyramid --me umh --threads 2 --thread-input --ref 4 --mixed-refs --merange 16 --subme 7 --b-rdo --trellis 2 --bime --weightb --partitions all --direct auto --8x8dct --no-fast-pskip --crf 20

SSIM: 0.9753368
OPSNR: 41.784
Bitrate: 5727.27
FPS: 1.61
Improvement: 0.64%

Commandline:
x264_old --bframe 16 --b-pyramid --me imh --threads 2 --thread-input --me-prepass --ref 4 --mixed-refs --merange 16 --subme 7 --b-rdo --trellis 2 --bime --weightb --partitions all --direct auto --8x8dct --no-fast-pskip --crf 20

SSIM: 0.9752701
OPSNR: 41.774
Bitrate: 5748.29
FPS: 1.6

Interesting: I'm surprised how effective --me-prepass was. The completely free --subme 7 optimization is nice too. :)

--me-prepass should be the most useful in scenes with complex motion: if the various predictors are very far apart, it will vastly improve x264's choice of which predictor to use.

[vortex_hl]

I'm sorry, but when I input that, it gives me an error.
http://img62.imageshack.us/img62/4698/hadamarderrorbp3.png
That's a really bad commandline IMO (if its a boolean); it says nothing about what it does. If it was a string-based commandline like --fpel-cmp sad vs --fpel-cmp satd, that would be fine though.
Thanks Dark Shikari, it now works. The user before me did'nt specify that i'd have to enter in sad or satd after the command. :)

sorry. i thought you will look x264 --help page before use --fpel-cmp function.

[Cef]

new (http://mirror05.x264.nl/Cef/force.php?file=./x264_x86_dark.7z) build (http://mirror05.x264.nl/Cef/force.php?file=./x264_x64_dark.7z) with akupenguin's new satd fpel patch (I used .11 version), see this post (http://forum.doom9.org/showthread.php?p=1045280#post1045280).
previous build was crashing with --fpel-cmp satd anyway :p so please update if you have dl'd it.
rev676 is also up.

[Terranigma]

new (http://mirror05.x264.nl/Cef/force.php?file=./x264_x86_dark.7z) build (http://mirror05.x264.nl/Cef/force.php?file=./x264_x64_dark.7z) with akupenguin's new satd fpel patch (I used .11 version), see this post (http://forum.doom9.org/showthread.php?p=1045280#post1045280).
previous build was crashing with --fpel-cmp satd anyway :p so please update if you have dl'd it.
rev676 is also up.
Thanks Cef, you're the best :)
I liked how you used the word "new" for one version, and "build" for the other one. =P

[DeathTheSheep]

Say, Cef, you still wouldn't mind 7zipping a source package for me (since the patches are giving me qualms), right? :)

[Dark Shikari]

Oh, also, --me-prepass not surprisingly is not beneficial with --me esa :p

It raises bitrate and quality, but for --me esa it can't get enough quality to make up for its bitrate increase. For all other motion search modes, the quality it gains is far more worthwhile.

[Dark Shikari]

I see --me esa now works with --threads in rev 676... can this be added to the patch here, to work with --fpel-cmp sad and satd?

[DeathTheSheep]

Yes. I tried it. :)

[DeathTheSheep]

Oh, also, --me-prepass not surprisingly is not beneficial with --me esa :p

It raises bitrate and quality, but for --me esa it can't get enough quality to make up for its bitrate increase. For all other motion search modes, the quality it gains is far more worthwhile.

I found the opposite results to be true for me: SSIM decreased a negligible amount, while filesize decreased over 1% for my high-motion animated source... And this is with very high merange satd sea.

Cheers!

[Dark Shikari]

I found the opposite results to be true for me: SSIM decreased a negligible amount, while filesize decreased over 1% for my high-motion animated source... And this is with very high merange satd sea.

Cheers!
Now that's very interesting.

It could be the high motion; in one sense --me-prepass considerably increases the effective MERange of the motion estimation algorithm, so that would boost the effectiveness of SATD SEA in cases where high --merange would increase its effectiveness a lot.

[Cef]

Say, Cef, you still wouldn't mind 7zipping a source package for me (since the patches are giving me qualms), right? :)

Here you go (http://mirror05.x264.nl/Cef/force.php?file=./x264_src_dark.7z)

[St Devious]

Wow man , some nice work by u here . especially for me since i always use subme 7 coz time is not as big a factor as quality. Keep improving the algo's .

Try other things which can improve quality too .

when is the build including this update for subme 7 coming ?

[Sagekilla]

Hey, is there a patched and compiled build with all the goodies we have so far yet? I'd love to try it out after my little hiatus on encoding the past week.

[martino]

Here is are some test encodes that I run this afternoon. Hopefully they'll be of some help. This was done on a Core 2 Duo E6750 @ 2.67GHz (stock frequency).

Kenkou Zenrakei Suieibu Umisho (EP11) Anime 704x400 23.976fps - first 4000 frames

--qp 18 --ref 8 --mixed-refs --no-fast-pskip --bframes 5 --b-pyramid --b-rdo --bime --weightb --direct auto --subme 7 --trellis 2 --analyse all --8x8dct --me umh --merange 24 --threads auto
avis [info]: 704x400 @ 23.98 fps (4000 frames)
x264 [info]: using cpu capabilities: MMX MMXEXT SSE SSE2 SSSE3
x264 [info]: slice I:48 Avg QP:15.00 size: 25415 PSNR Mean Y:53.75 U:54.82 V:54.87 Avg:54.02 Global:52.68
x264 [info]: slice P:2053 Avg QP:18.00 size: 8066 PSNR Mean Y:48.81 U:49.73 V:50.03 Avg:49.07 Global:48.74
x264 [info]: slice B:1899 Avg QP:19.74 size: 2692 PSNR Mean Y:47.55 U:49.28 V:49.49 Avg:48.01 Global:47.76
x264 [info]: mb I I16..4: 30.5% 41.6% 27.9%
x264 [info]: mb P I16..4: 7.2% 7.4% 2.3% P16..4: 31.0% 9.5% 3.7% 0.4% 0.3% skip:38.2%
x264 [info]: mb B I16..4: 0.1% 0.2% 0.1% B16..8: 14.9% 1.4% 3.4% direct: 4.5% skip:75.3%
x264 [info]: 8x8 transform intra:43.7% inter:42.6%
x264 [info]: direct mvs spatial:99.2% temporal:0.8%
x264 [info]: ref P 68.7% 13.5% 6.7% 3.5% 2.4% 2.1% 1.8% 1.4%
x264 [info]: ref B 77.6% 11.3% 3.9% 2.2% 1.5% 1.5% 1.2% 0.9%
x264 [info]: SSIM Mean Y:0.9947880
x264 [info]: PSNR Mean Y:48.270 U:49.581 V:49.835 Avg:48.624 Global:48.270 kb/s:1097.76
11.90 fps
1097.87 kb/s


--qp 18 --ref 8 --mixed-refs --no-fast-pskip --bframes 5 --b-pyramid --b-rdo --bime --weightb --direct auto --subme 7 --trellis 2 --analyse all --8x8dct --me imh --merange 24 --threads auto
avis [info]: 704x400 @ 23.98 fps (4000 frames)
x264 [info]: using cpu capabilities: MMX MMXEXT SSE SSE2 SSSE3
x264 [info]: slice I:48 Avg QP:15.00 size: 25415 PSNR Mean Y:53.75 U:54.82 V:54.87 Avg:54.02 Global:52.68
x264 [info]: slice P:2053 Avg QP:18.00 size: 8055 PSNR Mean Y:48.80 U:49.72 V:50.02 Avg:49.06 Global:48.73
x264 [info]: slice B:1899 Avg QP:19.74 size: 2691 PSNR Mean Y:47.55 U:49.27 V:49.48 Avg:48.00 Global:47.75
x264 [info]: mb I I16..4: 30.5% 41.6% 27.9%
x264 [info]: mb P I16..4: 7.2% 7.3% 2.2% P16..4: 31.1% 9.5% 3.7% 0.4% 0.3% skip:38.2%
x264 [info]: mb B I16..4: 0.1% 0.2% 0.1% B16..8: 14.9% 1.4% 3.4% direct: 4.5% skip:75.3%
x264 [info]: 8x8 transform intra:43.6% inter:42.5%
x264 [info]: direct mvs spatial:98.3% temporal:1.7%
x264 [info]: ref P 68.7% 13.5% 6.7% 3.5% 2.4% 2.1% 1.8% 1.4%
x264 [info]: ref B 77.3% 11.5% 4.0% 2.1% 1.5% 1.5% 1.2% 0.9%
x264 [info]: SSIM Mean Y:0.9947911
x264 [info]: PSNR Mean Y:48.266 U:49.568 V:49.819 Avg:48.618 Global:48.265 kb/s:1096.54
10.47 fps
1096.65 kb/s


--qp 18 --ref 8 --mixed-refs --no-fast-pskip --bframes 5 --b-pyramid --b-rdo --bime --weightb --direct auto --subme 7 --trellis 2 --analyse all --8x8dct --me imh --merange 24 --me-prepass --threads auto
avis [info]: 704x400 @ 23.98 fps (4000 frames)
x264 [info]: using cpu capabilities: MMX MMXEXT SSE SSE2 SSSE3
x264 [info]: slice I:48 Avg QP:15.00 size: 25192 PSNR Mean Y:53.76 U:54.88 V:54.95 Avg:54.05 Global:52.71
x264 [info]: slice P:2036 Avg QP:18.00 size: 7996 PSNR Mean Y:48.82 U:49.69 V:50.00 Avg:49.07 Global:48.74
x264 [info]: slice B:1916 Avg QP:19.74 size: 2728 PSNR Mean Y:47.53 U:49.28 V:49.48 Avg:47.99 Global:47.73
x264 [info]: mb I I16..4: 30.3% 42.1% 27.7%
x264 [info]: mb P I16..4: 7.2% 7.4% 2.2% P16..4: 30.9% 9.5% 3.8% 0.4% 0.3% skip:38.2%
x264 [info]: mb B I16..4: 0.1% 0.3% 0.1% B16..8: 15.1% 1.4% 3.5% direct: 4.5% skip:75.0%
x264 [info]: 8x8 transform intra:43.8% inter:42.4%
x264 [info]: direct mvs spatial:99.6% temporal:0.4%
x264 [info]: ref P 68.5% 13.5% 6.7% 3.4% 2.4% 2.1% 1.8% 1.4%
x264 [info]: ref B 76.9% 11.4% 4.0% 2.2% 1.7% 1.6% 1.2% 1.0%
x264 [info]: SSIM Mean Y:0.9947875
x264 [info]: PSNR Mean Y:48.261 U:49.557 V:49.807 Avg:48.611 Global:48.257 kb/s:1089.30
8.42 fps
1089.42 kb/s


--qp 18 --ref 8 --mixed-refs --no-fast-pskip --bframes 5 --b-pyramid --b-rdo --bime --weightb --direct auto --subme 7 --trellis 2 --analyse all --8x8dct --me imh --merange 24 --fpel-cmp satd --threads auto
avis [info]: 704x400 @ 23.98 fps (4000 frames)
x264 [info]: using cpu capabilities: MMX MMXEXT SSE SSE2 SSSE3
x264 [info]: slice I:48 Avg QP:15.00 size: 25713 PSNR Mean Y:53.71 U:54.66 V:54.64 Avg:53.93 Global:52.64
x264 [info]: slice P:2082 Avg QP:18.00 size: 7958 PSNR Mean Y:48.80 U:49.71 V:50.02 Avg:49.06 Global:48.74
x264 [info]: slice B:1870 Avg QP:19.74 size: 2682 PSNR Mean Y:47.51 U:49.27 V:49.45 Avg:47.97 Global:47.73
x264 [info]: mb I I16..4: 30.5% 41.2% 28.2%
x264 [info]: mb P I16..4: 7.3% 7.1% 2.1% P16..4: 30.5% 9.9% 3.5% 0.5% 0.3% skip:38.8%
x264 [info]: mb B I16..4: 0.1% 0.2% 0.1% B16..8: 14.9% 1.4% 3.4% direct: 4.6% skip:75.3%
x264 [info]: 8x8 transform intra:42.9% inter:44.1%
x264 [info]: direct mvs spatial:99.0% temporal:1.0%
x264 [info]: ref P 69.5% 13.3% 6.4% 3.3% 2.3% 2.0% 1.7% 1.4%
x264 [info]: ref B 77.5% 11.4% 4.0% 2.1% 1.6% 1.4% 1.1% 0.9%
x264 [info]: SSIM Mean Y:0.9947945
x264 [info]: PSNR Mean Y:48.256 U:49.566 V:49.811 Avg:48.607 Global:48.265 kb/s:1094.17
4.72 fps
1094.29 kb/s


--qp 18 --ref 8 --mixed-refs --no-fast-pskip --bframes 5 --b-pyramid --b-rdo --bime --weightb --direct auto --subme 7 --trellis 2 --analyse all --8x8dct --me imh --merange 24 --me-prepass --fpel-cmp satd --threads auto
avis [info]: 704x400 @ 23.98 fps (4000 frames)
x264 [info]: using cpu capabilities: MMX MMXEXT SSE SSE2 SSSE3
x264 [info]: slice I:46 Avg QP:15.00 size: 26108 PSNR Mean Y:53.70 U:54.72 V:54.71 Avg:53.95 Global:52.60
x264 [info]: slice P:2018 Avg QP:18.00 size: 8046 PSNR Mean Y:48.79 U:49.70 V:50.00 Avg:49.05 Global:48.73
x264 [info]: slice B:1936 Avg QP:19.74 size: 2694 PSNR Mean Y:47.55 U:49.29 V:49.50 Avg:48.00 Global:47.76
x264 [info]: mb I I16..4: 30.3% 41.2% 28.5%
x264 [info]: mb P I16..4: 7.6% 7.3% 2.1% P16..4: 30.4% 10.2% 3.6% 0.5% 0.3% skip:37.9%
x264 [info]: mb B I16..4: 0.1% 0.3% 0.1% B16..8: 14.9% 1.5% 3.4% direct: 4.5% skip:75.3%
x264 [info]: 8x8 transform intra:42.8% inter:43.9%
x264 [info]: direct mvs spatial:99.3% temporal:0.7%
x264 [info]: ref P 69.5% 13.2% 6.5% 3.3% 2.3% 2.1% 1.8% 1.4%
x264 [info]: ref B 77.6% 11.1% 3.9% 2.3% 1.6% 1.5% 1.1% 1.0%
x264 [info]: SSIM Mean Y:0.9947976
x264 [info]: PSNR Mean Y:48.246 U:49.560 V:49.812 Avg:48.599 Global:48.260 kb/s:1086.30
3.82 fps
1086.42 kb/s


--qp 18 --ref 8 --mixed-refs --no-fast-pskip --bframes 5 --b-pyramid --b-rdo --bime --weightb --direct auto --subme 7 --trellis 2 --analyse all --8x8dct --me esa --merange 24 --threads auto
avis [info]: 704x400 @ 23.98 fps (4000 frames)
x264 [info]: using cpu capabilities: MMX MMXEXT SSE SSE2 SSSE3
x264 [info]: slice I:48 Avg QP:15.00 size: 25415 PSNR Mean Y:53.75 U:54.82 V:54.87 Avg:54.02 Global:52.68
x264 [info]: slice P:2053 Avg QP:18.00 size: 8053 PSNR Mean Y:48.83 U:49.71 V:50.00 Avg:49.08 Global:48.74
x264 [info]: slice B:1899 Avg QP:19.74 size: 2686 PSNR Mean Y:47.57 U:49.25 V:49.46 Avg:48.01 Global:47.76
x264 [info]: mb I I16..4: 30.5% 41.6% 27.9%
x264 [info]: mb P I16..4: 7.3% 7.3% 2.1% P16..4: 31.1% 9.7% 3.8% 0.5% 0.3% skip:37.9%
x264 [info]: mb B I16..4: 0.1% 0.2% 0.1% B16..8: 14.8% 1.4% 3.4% direct: 4.5% skip:75.4%
x264 [info]: 8x8 transform intra:43.4% inter:41.9%
x264 [info]: direct mvs spatial:99.4% temporal:0.6%
x264 [info]: ref P 69.0% 13.3% 6.6% 3.4% 2.4% 2.1% 1.8% 1.4%
x264 [info]: ref B 77.0% 11.6% 4.1% 2.2% 1.6% 1.5% 1.2% 0.9%
x264 [info]: SSIM Mean Y:0.9948316
x264 [info]: PSNR Mean Y:48.292 U:49.551 V:49.802 Avg:48.631 Global:48.272 kb/s:1095.86
5.04 fps
1095.97 kb/s


--qp 18 --ref 8 --mixed-refs --no-fast-pskip --bframes 5 --b-pyramid --b-rdo --bime --weightb --direct auto --subme 7 --trellis 2 --analyse all --8x8dct --me esa --merange 24 --me-prepass --fpel-cmp satd --threads auto
avis [info]: 704x400 @ 23.98 fps (4000 frames)
x264 [info]: using cpu capabilities: MMX MMXEXT SSE SSE2 SSSE3
mp4 [info]: initial delay 250 (scale 2997)
x264 [info]: slice I:46 Avg QP:15.00 size: 26108 PSNR Mean Y:53.70 U:54.72 V:54.71 Avg:53.95 Global:52.60
x264 [info]: slice P:2018 Avg QP:18.00 size: 8044 PSNR Mean Y:48.85 U:49.68 V:49.98 Avg:49.09 Global:48.75
x264 [info]: slice B:1936 Avg QP:19.74 size: 2688 PSNR Mean Y:47.59 U:49.27 V:49.48 Avg:48.03 Global:47.77
x264 [info]: mb I I16..4: 30.3% 41.2% 28.5%
x264 [info]: mb P I16..4: 7.5% 7.1% 2.0% P16..4: 30.9% 10.4% 3.7% 0.5% 0.3% skip:37.5%
x264 [info]: mb B I16..4: 0.1% 0.2% 0.1% B16..8: 14.9% 1.4% 3.5% direct: 4.5% skip:75.3%
x264 [info]: 8x8 transform intra:42.9% inter:43.1%
x264 [info]: direct mvs spatial:99.6% temporal:0.4%
x264 [info]: ref P 69.8% 13.0% 6.5% 3.3% 2.2% 2.0% 1.8% 1.4%
x264 [info]: ref B 77.6% 11.1% 3.9% 2.3% 1.6% 1.5% 1.1% 1.0%
x264 [info]: SSIM Mean Y:0.9948508
x264 [info]: PSNR Mean Y:48.299 U:49.540 V:49.794 Avg:48.633 Global:48.275 kb/s:1085.60
2.17 fps
1085.72 kb/s


Anything else that I should try out, or any corrections to the settings that I used...?

[buzzqw]

what's looking better ?

BHH

[martino]

Take a look at the SSIM? It's rather impossible by looking at the videos side to side, as one would have guessed. But myself I see --me imh with --me-prepass as a manageable combo (ie don't slow the process down by too much).

--fpel-cmp satd looks like a speed killer (maybe I used it wrongly?), and doesn't seem to provide a marginal improvement. So I don't see myself using this one, maybe only if I have the whole night for an encode to complete (which nowadays I don't since I shut down my PC overnight)...


And yeah; Thank you Dark Shikari for these patches and everyone else who contributed to them! :)

[Terranigma]

--fpel-cmp satd looks like a speed killer (maybe I used it wrongly?)
No, you're using it properly. I've noticed it drops encoding speed significantly as well with a marginal increase in quality. It's for the insane of the insane; those who wants to squeeze out every bit of quality they can (well imho).

[Dark Shikari]

No, you're using it properly. I've noticed it drops encoding speed significantly as well with a marginal increase in quality. It's for the insane of the insane; those who wants to squeeze out every bit of quality they can (well imho).Yup, its less effective on a per-quality basis than any other option, I think.

It also seems to be more effective on live-action footage than on anime.

With the new multithreaded optimized ESA (thanks aku!) it could be very good; my preliminary testing shows that --me esa --fpel-cmp satd may actually be faster than --me imh --me-prepass --fpel-cmp satd.

Another interesting thing is that --me imh --me-prepass gives results almost equivalent to that of --me esa in all cases I've done so far :)

[akupenguin]

Elephants dream 720p (all of it, not just your wire scene)
looks like SATD ESA dominates all the other SATD options.

edit: graph removed, see later post for update

[Dark Shikari]

Wow, that's interesting--with the optimizations made to it, SATD ESA makes all the other SATD options completely unnecessary.

And its interesting how --me-prepass --me imh wipes the floor with --me esa, for SAD.

Obviously it must have taken an ungodly amount of encoding to make that, but I wonder how much --me-prepass helps --me esa?

[akupenguin]

Not so ungodly. Note the 720p SATD ESA merange=32 ran at 8fps, as compared to your 0.05 fps. 8core Xeon ftw.

[Dark Shikari]

Not so ungodly. Note the 720p SATD ESA merange=32 ran at 8fps, as compared to your 0.05 fps. 8core Xeon ftw.:scared:

The 0.05 FPS was the completely unoptimized version too, using a combination of SEA SAD and thresholded SEA SATD on that, and then adding in your tweak to allow multithreaded ESA... suddenly ESA actual became somewhat practical. :)

[Dark Shikari]

Also, in your image, are you measuring quality/bitrate or just bitrate? I assume its quality/bitrate because Prepass in my experience raises bitrate to gain quality, but its labelled as just bitrate, so I'm not exactly sure.

[akupenguin]

quality/bitrate. Though most of the change was in bitrate, none of the samples differed by more than 1% quality.
Running ESA prepass now, estimated 6 hours.

[akupenguin]

http://akuvian.org/images/ed720_me_cmp.png

[Dark Shikari]

I assume that's a typo, and the top one is SATD ESA prepass ;)

But interesting results, though not surprising--very low MERanges will give worse results with Prepass, because Prepass may sometimes move the exhaustive search radius away from the ideal value.

[akupenguin]

Out of all of these data points, prepass never reduces quality. But prepass is slower than incrementing merange, so it's only useful after you hit diminishing returns on merange.

[fields_g]

on the previous graph, I noticed ESA had 9 points tested MERange values tested, while others had 8. What are the meranges that were tested for these?

[akupenguin]

4..32 in steps of 4, and also tested 6 for ESA.

[Atak_Snajpera]

Does r680 contain IMH and PREPASS?

[akupenguin]

See for yourself. The changelog is public...

[Atak_Snajpera]

Any particular reason why IMH is not included?

[Dark Shikari]

Any particular reason why IMH is not included?I will guess because IMH was only particularly useful because, unlike ESA, it could be multithreaded. Now that ESA has multithreading as an option, IMH isn't nearly as useful.

[Sagittaire]

Any particular reason why IMH is not included?

Certainely because there are not exhaustive test for these patchs. In my test (HPII trailer) the improvement is not really high (0.1 dB) with dramatic speed loss. I think that now it will be really difficult to obtain metric improvement for x264. The futur for x264 is certainely HVS improvement:
- Better grain/noise preservation
- HVS Adaptative Quantisation (complexity and dark masking)
- Perhaps usefull internal filter like dark noise filtering

[Terranigma]

I liked imh. I don't care if the quality gain is minimal, the speed and quality ratio is what I preferred. I'll be sticking to the previous builds with imh then. esa, for those with slower pc's is a no no, even if multi-threaded.

How about changing umh's algorithm with imh then?

[akupenguin]

imh won't replace umh, because imh is an extension to umh. It doesn't cost any code to support both. They might, however, be merged into one commandline option which selects between umh and imh depending on merange.

[Terranigma]

They might, however, be merged into one commandline option which selects between umh and imh depending on merange.
Please, i'd really appreciate it. :)
I personally use merange 24+ depending on my source's resolution (so maybe imh can be activated starting from merange 20 or so). :D

[foxyshadis]

Cef's builds will be supporting IMH until it's merged into svn in some form, so there's no need to hold out on updating.

[Terranigma]

Cef's builds will be supporting IMH until it's merged into svn in some form, so there's no need to hold out on updating.
Actually, before I posted my response, I tested out Cef's latest build and got the following:
http://img411.imageshack.us/img411/7536/imherrorhn3.png
:p

[Sharktooth]

there are several builds on cef's site... get the correct one.
however IMHO IMH is quite useless too...

[Cef]

The regular builds I keep up to date just have AQ and thread pool patches.
Should I add anything else ?

[Terranigma]

there are several builds on cef's site... get the correct one.
however IMHO IMH is quite useless too...

Those are older builds, i'd like to be able to use imh with the latest svn. According to the graphs made by aku, you're correct, but it's still an improvment over the standard umh (even if it's by a small margin. Some of us would like to be able to use it :D)


The regular builds I keep up to date just have AQ and thread pool patches.
Should I add anything else ?

Well for now, please add the imh search algorithm until aku does his magic with umh. :D

[Dark Shikari]

Those are older builds, i'd like to be able to use imh with the latest svn. According to the graphs made by aku, you're correct, but it's still an improvment over the standard umh (even if it's by a small margin. Some of us would like to be able to use it :D)
Why not just use ESA then if you don't care that much about speed? Its multithreaded now, too.

[Terranigma]

Why not just use ESA then if you don't care that much about speed? Its multithreaded now, too.

I never said I did'nt care about speed. I usually use the exact same settings for both passes, so i'm curious as to how much the quality will differ if I chosed to use umh for the first pass and esa for the 2nd/final pass vs. esa for both passes. Any idea? :D

[Dark Shikari]

I never said I did'nt care about speed. I usually use the exact same settings for both passes, so i'm curious as to how much the quality will differ if I chosed to use umh for the first pass and esa for the 2nd/final pass vs. esa for both passes. Any idea? :DProbably meaningless--I've found that using higher settings for the first pass sometimes actually hurts quality by a miniscule amount, pointing out that beyond a certain point it goes past the margin of error.

I wouldn't use beyond --me hex and --subme 5 for the first pass, personally.

[Terranigma]

Thanks for the sage advice. So I guess the total eta for 2 passes should be equaled to or possibly less than 2 passes imh (subme7) if I used those settings for the first pass and esa for the second (with subme 7)?

[le_canz]

I did some tests with 2 pass encoding, with various sources (lossless with Elephant's Dream, 3 samples ripped from DVDs and 4 others from DV camcorder)

Using --partitions none --me dia and --subme 7 for the first pass produced the best results I could achieve :)

[Terranigma]

Using --partitions none --me dia and --subme 7 for the first pass produced the best results I could achieve :)
That's just insane. I always thought using the exact same settings will yield the optimum result. Thanks so much guys :D

[fields_g]

Yea, IMH has 2 small windows where it is better than the other methods, but I think they are small enough that you might as well just use something a little more or less cpu intensive.

[le_canz]

That's just insane. I always thought using the exact same settings will yield the optimum result. Thanks so much guys :D

That's what I thought too... However the quality gain is very small.

But it's worth to know this, as it helps saving encoding time.

[Terranigma]

Cef, Could you add the me-prepass then? I just noticed that command doesn't work either. :D

[Dark Shikari]

Cef, Could you add the me-prepass then? I just noticed that command doesn't work either. :DYes, that is much more useful as unlike --me imh it helps with all ME search modes, even with ESA, and so has nothing that can be used in its place.

[fields_g]

Yes, that is much more useful as unlike --me imh it helps with all ME search modes, even with ESA, and so has nothing that can be used in its place.

Hey, I like what I hear. I was about to make the same suggestion!

[Terranigma]

Just got finished examining my x264 esa encode with fpel satd and prepass (and a high merange). Looks amazing! Thanks Dark Shikari and le_canz for tips on using a sorta fast first pass followed by an almost 2x slower second pass. Hopefully Cef will add in prepass, as like you said Shikari, is very useful. :D

[burfadel]

He had prepass in his x264_x86_exp.7z build from the 15th, maybe a patch was missed for this build? it is a great feature!

[Terranigma]

He had prepass in his x264_x86_exp.7z build from the 15th, maybe a patch was missed for this build? it is a great feature!

He said earlier that all he includes in his normal builds was the pthreads and aq-patch, and then went on to ask us what else should be added. Prepass is the only useful patch I can think of, I mean, just look at the graph made by aku. :eek:

[burfadel]

He said earlier that all he includes in his normal builds was the pthreads and aq-patch, and then went on to ask us what else should be added. Prepass is the only useful patch I can think of, I mean, just look at the graph made by aku. :eek:

ah ok! I missed that :eek: lol, well I guess he has his answer now :D, it definately makes an impressive difference, and is a worthwhile inclusion!

[akupenguin]

Ideas for prepass:
Eliminate duplicate predicted mvs. I didn't do it before because it's not worthwhile when the cost of a duplicate is just 1 SAD, but if you run a whole subpel search then even saving an occasional duplicate should be worth the check. Also try eliminating near-duplicates within a fraction of a pixel of eachother.
Does it need to be qpel? hpel would be faster not only because it's fewer mv checks, but also because generating hpel samples is just pointer manipulation while generating qpel samples requires a dsp function.
CHECK_MVRANGE takes fullpel values, so your use of it in prepass limits the range to 1/4 of what it should be.

[Dark Shikari]

Ideas for prepass:
Eliminate duplicate predicted mvs. I didn't do it before because it's not worthwhile when the cost of a duplicate is just 1 SAD, but if you runa whole subpel search then even saving an occasional duplicate should be worth the check. Also try eliminating near-duplicates within a fraction of a pixel of eachother.
Does it need to be qpel? hpel would be faster not only because it's fewer mv checks, but also because generating hpel samples is just pointer manipulation while generating qpel samples requires a dsp function.
CHECK_MVRANGE takes fullpel values, so your use of it in prepass limits the range to 1/4 of what it should be.Ah, interesting.

The search could definitely be converted to fullpel; I doubt that would affect the results much; it might even improve the results.

I didn't realize that the subpel samples had to be generated in real-time.

[DeathTheSheep]

Any progress on the fullpel conversion? :p

[tetsuo55]

Dark Shikari, any updates on this? is it still even relevant?

[Dark Shikari]

Dark Shikari, any updates on this? is it still even relevant?Not relevent. It was only useful because ESA wasn't threaded, and now it is obselete.

With the recent ESA optimizations, it has become even more useless :p

[DeathTheSheep]

Maybe he was talking about me-prepass (also a topic of this thread). That, IMHO, will always be relevant, but whether or not there is interest in updating it is another story entirely. :p

[Sagekilla]

Maybe he was talking about me-prepass (also a topic of this thread). That, IMHO, will always be relevant, but whether or not there is interest in updating it is another story entirely. :p

I'd be interested in seeing me prepass and the fpel-cmp being merged to svn, with a little more tweaking to make it faster I suppose.

[DeathTheSheep]

Perhaps fastref, too. Or at least keep all the [quality-boost] patches in one place and up to date. I'd be willing to collect, host, and update frequently once I get a decent web space established.

[Sagekilla]

Perhaps fastref, too. Or at least keep all the patches in one place and up to date. I'd be willing to collect, host, and update frequently once I get a decent web space established.


I'm sure you could set up a rudimentary webpage on one of those free hosting sites and use mediafire to provide the hosting for all the files.

[DeathTheSheep]

Haha, I already have more than that, at least. But if I'm going to take on a repository of patches into which a lot of hard work and labor are pumping, I'd like a little more professionalism than that, not to mention decent control, backup, and data loss prevention. But soon. Soon...

[Zanejin]

With the recent ESA optimizations, it has become even more useless :p

Do these recent optimizations make ESA faster or pretty much equivalent in speed to IMH on a single-core CPU? I still run one of those old gadgets (Didée does as well, if I remember correctly).

[Dark Shikari]

Do these recent optimizations make ESA faster or pretty much equivalent in speed to IMH on a single-core CPU? I still run one of those old gadgets (Didée does as well, if I remember correctly).No idea, probably close at least.