x264 profiling

[Egladil]

Is it possible to profile x264 using mingw? it first look i found no gprof mingw packages.

[FingCmo]

Hi,

Could someone please explain what the "fprofiled:" section of the x264 Makefile does?

I am used to the "gprof" profiling tool to create a ".dot" file which I then read
via Graphviz to produce beautiful call graphs and some runtime statistics.

e.g. like this:

$ gcc -pg test.c
$ ./a.out
$ gprof a.out gmon.out | awk -f gprof2dot.awk > test1.dot

Does the "fprofiled:" section of the Makefile do something similar as above?
If so how (input, outputs, dirs)?
OR does it just automatically optimize the CLI and/or library for future use?

Does Valgrind and/oprofile produce .dot output I can use for drawing
call graphs and other memory usage statistics?

Thanks again for your time,
Ad.

[Dark Shikari]

fprofiled is to create an fprofiled version of x264. If you don't know what fprofiling is, read the gcc manpage.

gprof is crap. Don't use it for any purpose; it can't profile asm functions. Use oprofile.

[FingCmo]

Dark,

Thanks again for your reply.

I looked up OProfile and there doesn't seem to be one for OS X (Leopard). Any close derivative for Mac?

If not, I might try profiling on Fedora which is running on my Mac
via VirtualBox virtualization. Will this be an issue for Oprofile?

Could Valgrind be an adequate substitue for Oprofile?
Or do they address different issues?

Thanks,
Ad.

[yuvi]

Shark works well for profiling on OS X. Dunno about formatting it for Graphviz, but Instruments has pretty graphs.

[Mr VacBob]

oprofile doesn't work under virtualization (at least, not under VMWare).

Luckily, Shark works perfectly anyway, aside from having a very buggy debuginfo/line-number reader.

[akupenguin]

Valgrind is for debugging, not profiling. The most it can give as a profile is a count of instructions executed, not cycles.

[FingCmo]

All,

Any 2nd opinion on whether oprofile will work on a program running on Linux that is virtualized on VirtualBox? i.e.

OSX[ VirtualBox( Linux[ oprofile( test.c ) ] ) ]

I will test it out and report back soon.

Thanks,
Ad.

[FingCmo]

All,

I somehow can not produce a "gmon.out" file using x264 Makefile.

I know "gprof" is not the grand enchilada of profilers but I am trying to use it just once as a baseline before I try Oprofile, Shark and others.

This is what I did on my OS X with gcc 4.0.1:
Within my GIT x264 directory I did:

$ make clean
$ make
$ ./x264 --profile baseline -o out_stef.264 stefan_cif.yuv 352x288

To make sure it encoded OK I played the file as below:

$ ffplay out_stef.264

My problem is, even though I added "-g -p -pg" to CFLAGS and LDFLAGS as well removing "-s", I do not get the "gmon.out"
file.

I have posted my "config.mak" below.

#-------------------------------------------
prefix=/Users/me/Documents/install
exec_prefix=${prefix}
bindir=${exec_prefix}/bin
libdir=${exec_prefix}/lib
includedir=${prefix}/include
ARCH=X86
SYS=MACOSX
CC=gcc
CFLAGS=-O4 -ffast-math -Wall -I. -falign-loops=16 -mdynamic-no-pic -march=i686 -mfpmath=sse -msse -DHAVE_MMX -DARCH_X86 -DSYS_MACOSX -DHAVE_PTHREAD -g -p -pg
LDFLAGS=-lm -lpthread -g -p -pg
AR=ar
RANLIB=ranlib
STRIP=strip
AS=yasm
ASFLAGS= -O2 -f macho -DPREFIX
EXE=
VIS=no
HAVE_GETOPT_LONG=1
DEVNULL=/dev/null
#-------------------------------------------

What am I doing wrong to not get the "gmon.out"?

Thanks for your time,
Ad.

[Mr VacBob]

Use "make fprofiled".

[FingCmo]

Thanks,

I did use the fprofiled as below:

$ make fprofiled VIDS=stefan_cif.yuv

I also modified my profiled section to keep the
profile .264 videos and not to recompile as below.
ALL 8 of the OPT# encoded videos play OK with ffplay.

Here is the modified fprofiled section:

#-------------------------------------------
fprofiled:
$(MAKE) clean
mv config.mak config.mak2
sed -e 's/CFLAGS.*/& -fprofile-generate/; s/LDFLAGS.*/& -fprofile-generate/' config.mak2 > config.mak
$(MAKE) x264$(EXE)
$(foreach V, $(VIDS), $(foreach I, 0 1 2 3 4 5 6 7, ./x264$(EXE) $(OPT$I) --threads 1 $(V) 352x288 -o $(V)_$I.264 ;))
# rm -f $(SRC2:%.c=%.o)
# sed -e 's/CFLAGS.*/& -fprofile-use/; s/LDFLAGS.*/& -fprofile-use/' config.mak2 > config.mak
# $(MAKE)
# rm -f $(SRC2:%.c=%.gcda) $(SRC2:%.c=%.gcno)
mv config.mak2 config.mak
endif

#-------------------------------------------

Still no gmon.out in my local directory :(

Any way to use shark similarly without the XCode GUI?

Thanks again,
Ad.

[FingCmo]

Shark works well for profiling on OS X. Dunno about formatting it for Graphviz, but Instruments has pretty graphs.

All,

Has any one ever tried generating dot file / call graph
using the output from Shark?
I m trying to use "gprof2dot" to make call graphs using
the profiling info from Shark. I can't figure that out yet.

I tried this below with no luck:

$ ./gprof2dot.py -f shark shark_report1 | dot -Tpng -o output.png

I get the errors:

#-------------------------------------

Traceback (most recent call last):
File "./gprof2dot.py", line 2192, in <module>
Main().main()
File "./gprof2dot.py", line 2117, in main
self.profile = parser.parse()
File "./gprof2dot.py", line 1348, in parse
raise ParseError('failed to parse', line)
__main__.ParseError: failed to parse: '+ 79.0%, thread_start, libSystem.B.dylib'

#-------------------------------------
Thanks,
Ad.

[FingCmo]

All,

I found the solution to my questions above.

1--- The Makefile modification I made works using "make
fprofiled". gprof is not expected to work OK on
Leopard (OS X 10.5) so I just generated the
symbols using "make fprofiled" as above and then
ran the executable on Apple's Shark, to generate
a call-graph.

2--- The command line for parsing Shark's output above
is correct. The Python script had a bug and the
gprof2dot (Shark Parser) author was nice to me and
fixed it :)
Also make sure the shark output contents uses
"samples" not percentages before saving the shark
output (e.g. shark_report1 above).

3--- There is also another great Eclipse based call-graph
generator I am testing on Fedora called: Eclipse/CallGraph.
Installing it on Fedora is easy: 'yum install eclipse-callgraph'

http://www.fedoraproject.org/wiki/Eclipse/CallGraph

4--- I will be testing Valgrind and OProfile on Fedora that
I am virtualizing on my Mac.

Hopefully I will be a productive contributor to this great x264
library soon. :)

Thank you all for your time,
Ad.