About ChronoCross fprofiled x264 Builds

[Deinorius]

Hi

I know that fprofiled means, that the builds are double compiled with some test runs between the first and the second.

Isn't it so, that these Builds run fastest with the settings he used with the test runs?
If so, which settings did you used ChronoCross?
Are the performance improvements only with specific CPU architectures measurable, or can they be faster with Athlon XP CPUs too? I will test it later, but it would be interesting how it gets faster with Athlon 64 and P4 CPUs too. :)

[hellfred]

Hi

I know that fprofiled means, that the builds are double compiled with some test runs between the first and the second.

Isn't it so, that these Builds run fastest with the settings he used with the test runs?
If so, which settings did you used ChronoCross?
Are the performance improvements only with specific CPU architectures measurable, or can they be faster with Athlon XP CPUs too? I will test it later, but it would be interesting how it gets faster with Athlon 64 and P4 CPUs too. :)
I did one fprofiled buids by adding the path to a avisynth script. Then I saw that the file was encoded several times with different settings. I did not take any notes to what settings where used for each run, just that there was one -crf, too.
So the used parameters are always the same and coded into the building process, most probably the Makefile. I do not think one can force special settings, but I might be wrong here.

Hellfred

EDIT:
I had a look at the Makefile. All info needed are there:
SRC2 = $(SRCS) $(SRCCLI)
# These should cover most of the important codepaths
OPT0 = --crf 30 -b1 -m1 -r1 --me dia --no-cabac
OPT1 = --crf 18 -b2 -m3 -r3 --me hex -8 --cqm jvt --direct spatial
OPT2 = --crf 24 -b3 -m6 -r6 --me umh -8 -w -t1 -A all --b-pyramid --b-rdo --mixed-refs

ifeq (,$(VIDS))
fprofiled:
@echo 'usage: make fprofiled VIDS="infile1 infile2 ..."'
@echo 'where infiles are anything that x264 understands,'
@echo 'i.e. YUV with resolution in the filename, or avisynth.'
else
fprofiled:
make clean
mv config.mak config.mak2
sed -e 's/CFLAGS.*/& -fprofile-generate/; s/LDFLAGS.*/& -fprofile-generate/' config.mak2 > config.mak
make x264$(EXE)
$(foreach V, $(VIDS), $(foreach I, 0 1 2, ./x264$(EXE) $(OPT$I) $(V) --progress -o $(DEVNULL) ;))
rm -f $(SRC2:%.c=%.o)
sed -e 's/CFLAGS.*/& -fprofile-use/; s/LDFLAGS.*/& -fprofile-use/' config.mak2 > config.mak
make
rm -f $(SRC2:%.c=%.gcda) $(SRC2:%.c=%.gcno)
mv config.mak2 config.mak
endif

From the usage comments one can deduce, that only clips can be provided by the compiler. And I think that OPT0 to OPT2 got used while building a -fprofiled binary.

[ChronoCross]

If people can provide more tests that it's slower then I'll revert back. According to akupenguin and Jarod(IRC) it should run at least a little better as it's able to better detect which calculations are most frequently used.

As always I'm very open for feedback.

[Deinorius]

I did a test on my Athlon XP-M 2000+ with an Anime DVD (of course no filters).
AE_Standard: M.E. Algorithm: Hexagonal, --crf 24

I got a difference of 1,1 % for the fprofiled build. It's faster, but not much.
May be, an Athlon 64 (X2) can run faster (in differences of course).

[ChronoCross]

yeah fprofiled definitely isn't worth the cost in compiling time for the speedup but I guess every little bit counts.

[Deinorius]

That for sure. It's still your work. :D

[akupenguin]

Not worth what?
At 1.1%, if it takes 1 extra minute to compile with fprofiled and you encode for 90 minutes, you break even. If you encode a full movie, or if several people use the same binary, it's a win.
Yes, fprofiled can take lots of time if you make it encode lots of stuff... so don't do that. For all the evidence I've seen, a hew hundred frames is plenty of test material for it.

[Audionut]

Just a little observasion.

--crf 24 --ref 5 --mixed-refs --bframes 3 --b-pyramid --b-rdo --bime --weightb --filter -1,-1 --subme 6 --trellis 2 --analyse all --8x8dct --ipratio 1.2 --direct auto --me umh --progress --no-psnr

Bob0r build 487 --crf mode time to encode 164secs. Bitrate 650.16903

Bob0r build 492 --crf mode time to encode 155secs. Bitrate 650.16903

ChronoCross build 492 --crf mode time to encode 169secs. Bitrate 661.28991

AMD 3500 2gb ram.

[ChronoCross]

additionally it may or may not take shorter or longer depending on the source.

[ChronoCross]

I'd be interested in testing the same build using fprofiled and without.

http://www.chronocrossdev.com/x264.exe 492 no-fprofiled.

[Audionut]

ChronoCross Build 492 no-fprofiled time to encode 168secs Bitrate 661.28991

Both builds take longer to encode than bob0r's and result in larger files.

[Audionut]

But here's some interesting result's.

--pass 2 --bitrate 1000 --stats "C:\DVD_VIDEO\VIDEO_TS\test.stats" --ref 5 --mixed-refs --bframes 3 --b-pyramid --bime --weightb

--filter -1,-1 --analyse all --8x8dct --ipratio 1.2 --direct auto --progress --no-psnr

Turbo first pass.

bob0r's build.

First pass time. 36secs. Second pass time. 79secs.
QP's. I: 22.44 P: 24.73 B: 25.25
Bitrate: 1002.8339


ChronoCross Build.

First pass time. 37secs. Second pass time. 78secs.
QP's. I: 22.67 P: 24.87 B: 25.39
Bitrate: 1003.3284

ChronoCross build = In 2 pass mode, same speed. But you can notice that it result's in not only larger final bitrate, but also uses larger QP's.

A quick update before I hit the reply button.

--crf 24 --ref 5 --mixed-refs --bframes 3 --b-pyramid --bime --weightb --filter -1,-1 --analyse all --8x8dct --ipratio 1.2 --direct auto --progress --no-psnr

bob0r's build:
80secs encoding time.
Bitrate: 689.2117

ChronoCross build:
80secs encoding time.
Bitrate: 712.1506

The slowness of your builds "seems" to be related to,
RDO mode
--me umh
trellis 2.

Whether just 1 or all of these. Still testing.:)
edit: No idea what's causing your builds to need more bits.

[celtic_druid]

My builds, if you want to compare them: http://ffdshow.faireal.net/mirror/x264/

rev492 won't be up until later though.

[akupenguin]

edit: No idea what's causing your builds to need more bits.
Because ChronoCross added some patches, and bob0r didn't. Thus the relative speeds tell you nothing about compilation methods.

[Audionut]

Because ChronoCross added some patches


Perhaps the need to test the patch/s and ascertain if the effect of the patch itself outways the effect on bitrate and speed.

Standard command line.
--crf 24 --ref 5 --mixed-refs --bframes 3 --b-pyramid --bime --weightb --filter -1,-1 --analyse all --8x8dct --ipratio 1.2 --direct auto --progress --no-psnr

bob0r's build:
--------------------77secs, 13.08fps, 689.2117kbps
with trellis 2--------82secs, 12.21fps, 697.3224kbps
me umh-------------96secs, 10.36fps, 679.0373kbps
rdo-----------------89secs, 11.17fps, 662.4685kbps
b-rdo---------------95secs, 10.54fps, 651.3324kbps
all options enabled--153secs, 6.55fps, 650.1690kbps

ChronoCross build:
---------80secs, 12.50fps, 712.1506kbps
trellis 2--85secs, 11.72fps, 716.1358kbps
me umh--98secs, 10.22fps, 701.9114kbps
rdo------93secs, 10.84fps, 677.5118kbps
b-rdo----101secs, 9.95fps, 669.2685kbps

celtic_druid build:
all options enabled--159secs, 6.33fps, 650.1690kbps


x264.exe --pass 2 --bitrate 1000 --stats ".stats" --ref 5 --mixed-refs --bframes 3 --b-pyramid --b-rdo --bime --weightb --filter -1,-1 --subme 6 --trellis 2 --analyse all --8x8dct --ipratio 1.2 --direct auto --me umh --progress --no-psnr

bob0r's build:
pass1 36secs, 28.22fps
pass2 197secs, 5.08fps

Chronocross build:
pass1 37secs, 27.55fps
pass2 214secs, 4.66fps

[Pomyk]

You should also compare with metric (SSIM, OPSNR).

[ChronoCross]

I was gonna say that speed and size might also be indicitive of quality.

[Audionut]

Both celtic_druid's and bob0r's builds produce identical oputput's (bitrate and ssim/psnr).

--crf 24 --ref 5 --mixed-refs --bframes 3 --b-pyramid --b-rdo --bime --weightb --filter -1,-1 --subme 6 --analyse all --8x8dct --ipratio 1.2 --direct auto --me umh --progress

bob0r's/celtic_druid's build.
bitrate 639.68
psnr Y:39.030 U:43.584 V:44.395 Avg:40.133 Global:39.760
ssim 75.1440

ChronoCross build.
bitrate 658.99
psnr Y:39.339 U:43.778 V:44.556 Avg:40.423 Global:40.034
ssim 76.3476


--pass 2 --bitrate 1000 --stats ".stats" --ref 5 --mixed-refs --bframes 3 --b-pyramid --b-rdo --bime --weightb --filter -1,-1 --subme 6 --analyse all --8x8dct --ipratio 1.2 --direct auto --me umh --progress
With fast first pass.

bob0r's/celtic_druid's build.
bitrate 999.82
psnr Y:41.070 U:44.835 V:45.668 Avg:42.061 Global:41.654
ssim 81.7244

ChronoCross build.
bitrate 999.99
psnr Y:41.191 U:45.067 V:45.874 Avg:42.196 Global:41.779
ssim 82.3037

[ChronoCross]

tada......higher quality lol.

[Yama4050242]

i have trouble in understanding fprofiled, can anyone kindly explain it more

[lcx]

i have trouble in understanding fprofiled, can anyone kindly explain it more


Read this article: http://msdn.microsoft.com/library/en-us/dv_vstechart/html/profileguidedoptimization.asp
to understand how profiling helps the compiler perform better code optimizations.

[Hans Ohlo]

Builds Information

Rev. Number: 498 svn + patches + fprofiled

Applied Patches:

Adaptive Quantization v.5 by Haali Updated by Loren Merrit (akupenguin)


New Decimation v2 by Alex_W


Cli Signature by Sharktooth Modified by ChronoCross


Removed Patches:

Check subpel predictors by Alex_W.
Rev 492 Made the patch inoperable.

can someone please explain what effects these patches have?
thx

[Sharktooth]

they're self explanatory.
the first does a sort of adaptive quantization, the second is a new algo for decimation (should give more quality) and the third is just a signature on the CLI (command line interface) output (in the CC build is just "built by ChronoCross").

[bob0r]

Quote Alex_W:

ok here's a patch that makes coefficient decimation on P-frames optional: x264_optional_pframe_decimation.diff (http://files.x264.nl/Alex_W/patches/x264_optional_pframe_decimation.diff)
the option is --no-coef-thresh


Also:

ChronoCross: x264_check_subpel_predictors_r498.diff (http://files.x264.nl/Alex_W/patches/x264_check_subpel_predictors_r498.diff)

Happy applying!

[bob0r]

@ChronoCross

Can you please put all applied (and none working) patches online aswell?

[shon3i]

How to use that "New Decimation v2 by Alex_W" is exist some switch or it is automated

[ChronoCross]

New decimation is automatic. I'll be putting all patches online shortly.

[Hans Ohlo]

if these patches enable higher quality, why aren't they in the x264 source? i mean, do they break some kind of spec or compatibility?

[Kostarum Rex Persia]

Well, no, these patches are experimental, at the moment. But, using these patches give us higher encoding quality.

When all patches be finalized, bob0r will implement them in the official x264 build.

[sasam]

I think it is up to pengvado to decide if they make it in the SVN. bob0r just has a script that automatically compiles them from SVN and that's it...

[bob0r]

I think it is up to pengvado to decide if they make it in the SVN. bob0r just has a script that automatically compiles them from SVN and that's it...

You my friend........... are correct.

[sasam]

Ha ha thank you my friend ^_^
I don't know about decimation patch but I don't think AQ will ever make it to the official build unless aku one day decides to make psychovisual enhancement for x264 and include modified AQ in it...or something like that...

[Sharktooth]

ehrr... what official builds? x264 has NO official builds...
Getting x264

The latest x264 source code can always be found by anonymous SVN repository:

# svn co svn://svn.videolan.org/x264/trunk x264

Or grab a daily tarball.

Unofficial builds for Windows are available from Jarod, Celtic Druid, or Sharktooth.
FYI Jarod=bob0r=x264.nl

[sasam]

By official builds I meant builds made directly from SVN without any additional changes...

[Kostarum Rex Persia]

Sharktooth, I am sorry for asking you this, but can you ever consider to start making x264 builds once again, when your condition allow.

I really miss your builds, because I think that your builds and patches are simply the best.

[ChronoCross]

Sharktooth, I am sorry for asking you this, but can you ever consider to start making x264 builds once again, when your condition allow.

I really miss your builds, because I think that your builds and patches are simply the best.

Considering all his patches are in my builds.......I think he can take a break. no need to put too much pressure on him.

[IgorC]

The most important thing is that all patchs are here. And build of ChronoCross has it.
MeGui, profiles and matrices are on the forum.
Cry for nothing important.

[ChronoCross]

Okay for those of you using AQ. Does it still work? what settings do you use? I haven't heard any complaints so I was assuming it works. Could you let me know? Thanks.

[Deinorius]

About some performance gains form my side: Using a Fansub as Source I had a gain of 10,63 % on a P-M 1,7 GHz. Using a DVD-Source only 0,7 %.

[shon3i]

OK ChronoCross what about new AQ what is max value.

[foxyshadis]

I'm guessing it's a raw quant # boost. So if the block was normally 20 and you had a value of 4 there, it'd be given a value of 24. Buuuut, maybe not, I'll give you my results later.

I like the strategy of using aq on any blocks that won't be needed later, rather than trying to guess what needs it or doesn't. =o

[sade]

if i understood the patch correctly then no
c->mb_aq_offs[mb_xy] = x264_clip3( rc->mb_aq_offs[mb_xy] * h->param.rc.i_aq_tcplx >> 12, 0, 6 );
rc->mb_aq_offs[mb_xy] is the SAD of the best mv found for the mb in an 16x16 dia search.
h->param.rc.i_aq_tcplx is of course the new aq option and >> 12 is the same as / 4096. I would guess that a good range may be 10 - 100 but i am not sure.
But maybe I simply read it wrong.

[ChronoCross]

OK ChronoCross what about new AQ what is max value.

aku says a range of 0 to 10 is good. the max value is infinity.

[HOLiC]

So if I want to use aq option, I just need to add:

--aq tcplx 8

to the command line, right?

[Sharktooth]

in rev503B you can find both haali's and pengvado's AQ options

[Alizar]

Um, ChronoCross, this is potentially a stupid question, but in the daily builds page you list your 503B but I don't see it listed in your directory. Is it up and I'm looking in the wrong place or...?

[ChronoCross]

503B is sharktooths. my 503 already has all the patches listed. Including both AQ's

[Alizar]

503B is sharktooths. my 503 already has all the patches listed. Including both AQ's

My point being that looking here http://www.chronocrossdev.com/apps/x264/

I don't see any v503. I see up to v502.

[jethro]

My point being that looking here http://www.chronocrossdev.com/apps/x264/I don't see any v503. I see up to v502.

Me too.
BTW, I think Chronocross' builds are the quality benchmark now.

[ChronoCross]

My point being that looking here http://www.chronocrossdev.com/apps/x264/

I don't see any v503. I see up to v502.

sorry dude I must have been sleeping when I uploaded it. I put it in the wrong folder. it's in there now. Sorry for any problems.