x264 development

[Dark Shikari]

I'm getting the feeling that I should "borrow" from the Encode() / Encode_frame() example since it shows how to open the encoder then process pictures and then dump the *pic to whatever you want. I want mp4 container so I'm going to use the implementation that's already there.You should look at the changes in the latest lavf/ffms patch (see the relevant thread), as the easiest way to use the API with mp4 is to:

1) set b_repeat_headers to 0
2) set b_annexb to 0
3) call x264_encoder_headers to get the headers
4) send these off to the muxer
5) call x264_encoder_encode for each frame, sending them off to the muxer

This method is used in x264 with said patch applied, so you can use it as an example.

[rallymax]

A stupid question to be sure but I'm going to ask it anyway.
Are the bit rates & buffer size in the params.rc byte or kbytes and is it per frame or per second?
I ask because the FFmpeg avcodec shim (libx264.c) is dividing the number by 1000.
x4->params.rc.i_bitrate = avctx->bit_rate / 1000;
x4->params.rc.i_vbv_buffer_size = avctx->rc_buffer_size / 1000;
x4->params.rc.i_vbv_max_bitrate = avctx->rc_max_rate / 1000;
I wonder because I used the x264_param_default and bypassed pretty much everything in the above file and my 1440x1080 5 second machbeth chart video with a burnt in time code is only 45k! (don't get me wrong, I'm thrilled that my Premiere plugin is working, but there is something definitely amiss with my datarate! :D )
'pointers appreciated.

[rallymax]

kinda offtopic: RE: libavcodec & libavformat
What container (mpeg-2, other) is it ok to mux a H.264 and a AC3 stream?
Using the FFmpeg output_example.c as a reference I've not been able to get avformat/avcodec to open a legal audio stream yet. It's fine opening a mpeg-2 container and putting a video stream of H.264 in it.
thx.

[Blue_MiSfit]

If by MPEG-2 you mean MPEG Program Stream or Transport Stream, then yes both can hold H.264 just fine, though the latter is more common IME.

H.264 + AC3 is compatible with almost every container, including TS, PS/MPG, MKV, MP4, M2TS, etc... It's not compatible with ASF/WMV last I checked, and AC3 doesn't play nice with MOV last I checked either. Hell, you can probably bastardize AVI into doing it as well ;)

~MiSfit

[creamyhorror]

By the way, rallymax, DebugMode Frameserver apparently now supports Premiere CS4: http://www.debugmode.com/frameserver/newin27.php and it's been open-sourced.

So Premiere CS4 users have a serviceable x264 option now.

[Lam3rD]

If by MPEG-2 you mean MPEG Program Stream or Transport Stream, then yes both can hold H.264 just fine, though the latter is more common IME.

MPEG-PS can hold h.264? Can you share your source, as this isn't mentioned in wikipedia or anywhere I could find.
http://en.wikipedia.org/wiki/Comparison_of_container_formats
(I'm not saying that wikipedia is the most correct source of information that exist)

Why do they need to combine new technology with obsolete technology, when there are better alternatives? Where is it used and how could one multiplex them?

[kieranrk]

kinda offtopic: RE: libavcodec & libavformat
What container (mpeg-2, other) is it ok to mux a H.264 and a AC3 stream?
Using the FFmpeg output_example.c as a reference I've not been able to get avformat/avcodec to open a legal audio stream yet. It's fine opening a mpeg-2 container and putting a video stream of H.264 in it.
thx.

You might well have some problems with libavformat's ts muxer because it has a lot of deficiencies.

[sneaker_ger]

MPEG-PS can hold h.264? Can you share your source, as this isn't mentioned in wikipedia or anywhere I could find.

mkv2vob (http://www.mkv2vob.com) and ffmpeg can mux H.264 into PS AFAIK(which can be played on the PS3 for example), but I don't know if it's allowed by any standard or some kind of "hack".

[CruNcher]

last time i tested it was far away from ISO interoperable, also it is nice to have H.264 in PS seeking is very fast and complexity isn't really high :D
Personally i find it perfect for H.264 with AC3 as audio, but that's only my personal view :)

[Blue_MiSfit]

I do wish for better seeking performance with H.264 in MKV or MP4 :(

[nurbs]

Try shorter keyframe intervals.

[Blue_MiSfit]

Well sure, that's always an option, but how come other containers seek better, even with default keyints?

[nurbs]

What other conatainers? The only container apart from mp4 and mkv people normally use for AVC is m2ts and then the streams are mostly limited to a 1 or 2 second keyint to be blu-ray/avchd compliant.

edit:
I'm genuinely interested. Reading my own post it sounds a bit provocative although it's not meant to be.

[Elbart_]

Personally, I only had seeking issues with h264 and MP4 when I set MPC to use its internal MP4-splitter. I disabled it, e voila, with using Haali's splitter, seeking works instantly.

[LoRd_MuldeR]

When I try to compile the latest x264 GIT version (http://git.videolan.org/gitweb.cgi?p=x264.git;a=commit;h=9899cd539f738ad12750683e4079f08ee32ccc0d), I get this error:

ar rc libx264.a common/mc.o common/predict.o common/pixel.o common/macroblock.o
common/frame.o common/dct.o common/cpu.o common/cabac.o common/common.o common/m
date.o common/set.o common/quant.o common/vlc.o encoder/analyse.o encoder/me.o e
ncoder/ratecontrol.o encoder/set.o encoder/macroblock.o encoder/cabac.o encoder/
cavlc.o encoder/encoder.o encoder/lookahead.o common/x86/mc-c.o common/x86/predi
ct-c.o common/x86/cabac-a.o common/x86/dct-a.o common/x86/deblock-a.o common/x86
/mc-a.o common/x86/mc-a2.o common/x86/pixel-a.o common/x86/predict-a.o common/x8
6/quant-a.o common/x86/sad-a.o common/x86/cpu-a.o common/x86/dct-32.o common/x86
/pixel-32.o
ranlib libx264.a
gcc -o x264.exe x264.o input/yuv.o input/y4m.o input/thread.o output/raw.o outpu
t/matroska.o output/matroska_ebml.o output/flv.o output/flv_bytestream.o input/a
vs.o input/thread.o libx264.a -lpthreadGC2 -lwsock32 -s
input/thread.o:thread.c:(.data+0x0): multiple definition of `thread_input'
input/thread.o:thread.c:(.data+0x0): first defined here
collect2: ld returned 1 exit status
make: *** [x264.exe] Error 1

:confused:

[kemuri-_9]

gcc -o x264.exe x264.o input/yuv.o input/y4m.o input/thread.o output/raw.o outpu
t/matroska.o output/matroska_ebml.o output/flv.o output/flv_bytestream.o input/a
vs.o input/thread.o libx264.a -lpthreadGC2 -lwsock32 -s
input/thread.o:thread.c:(.data+0x0): multiple definition of `thread_input'
input/thread.o:thread.c:(.data+0x0): first defined here
collect2: ld returned 1 exit status
make: *** [x264.exe] Error 1[/code]

:confused:

r1415 was a wrong commit as it added input/thread.o to the x264.exe link again (so it's on there twice and causes that multiple definition error), revert back to r1414 which works fine.
the GIT has already reset back to r1414.

[LoRd_MuldeR]

r1415 was a wrong commit as it added input/thread.o to the x264.exe link again (so it's on there twice), revert back to r1414 which works fine

Okay. Thanks for clarification. Seems it was removed from GIT in the meantime :)

[skottish]

I'm getting a segfault trying to encode from png frames:

x264 --version
x264 0.84.1414 63b74d8
built on Jan 30 2010, gcc: 4.4.3

Platform: X86_64
System: LINUX
asm: yes
avs input: no
lavf input: yes
ffms input: no
mp4 output: no
pthread: yes
debug: yes
gprof: no
PIC: yes
shared: yes
visualize: no

ffmpeg -version
FFmpeg version SVN-r21544, Copyright (c) 2000-2010 Fabrice Bellard, et al.
built on Jan 30 2010 10:35:01 with gcc 4.4.3
configuration: --prefix=/usr --enable-gpl --enable-libx264 --enable-shared --disable-static --enable-pthreads --enable-network --enable-x11grab --enable-postproc --enable-avfilter --enable-avfilter-lavf
libavutil 50. 7. 0 / 50. 7. 0
libavcodec 52.49. 0 / 52.49. 0
libavformat 52.48. 0 / 52.48. 0
libavdevice 52. 2. 0 / 52. 2. 0
libavfilter 1.17. 0 / 1.17. 0
libswscale 0. 9. 0 / 0. 9. 0
libpostproc 51. 2. 0 / 51. 2. 0


gdb x264
GNU gdb (GDB) 7.0.1
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/bin/x264...done.
(gdb) run ./kali_%06d.png --preset veryslow --threads 0 --fps 9.9 -o 4.mkv
Starting program: /usr/bin/x264 ./kali_%06d.png --preset veryslow --threads 0 --fps 9.9 -o 4.mkv
[Thread debugging using libthread_db enabled]
[image2 @ 0x6a33b0]MAX_READ_SIZE:5000000 reached
lavf [warning]: converting from rgb24 to YV12
lavf [info]: 720x576p 0:1 @ 25/1 fps (cfr)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast FastShuffle SSEMisalign LZCNT
[New Thread 0x7ffff574a710 (LWP 22789)]
x264 [info]: profile High, level 4.0

Program received signal SIGSEGV, Segmentation fault.
0x000000000040cfd6 in x264_nal_encode (dst=0xfffffffff574b015 <Address 0xfffffffff574b015 out of bounds>, b_annexb=0,
nal=<value optimized out>) at common/common.c:735
735 common/common.c: No such file or directory.
in common/common.c
(gdb) where
#0 0x000000000040cfd6 in x264_nal_encode (dst=0xfffffffff574b015 <Address 0xfffffffff574b015 out of bounds>,
b_annexb=0, nal=<value optimized out>) at common/common.c:735
#1 0x0000000000415f1b in x264_encoder_encapsulate_nals (h=0xa1d7b0) at encoder/encoder.c:1183
#2 0x00000000004160c6 in x264_encoder_headers (h=0xa1d7b0, pp_nal=0x7fffffffe0d0, pi_nal=0x7fffffffe0ec)
at encoder/encoder.c:1221
#3 0x00000000004074d6 in Encode (param=<value optimized out>, opt=0x7fffffffe7e0) at x264.c:1514
#4 0x0000000000408e78 in main (argc=<value optimized out>, argv=<value optimized out>) at x264.c:127

[kemuri-_9]

I'm getting a segfault trying to encode from png frames:

you'll get a seg fault encoding anything on any system that formerly had -DHAVE_MALLOC_H in the gcc CFLAGS, which includes linux.
r1413 broke a number of things,
we're working getting a fix committed.
for the time being revert to r1412 if you don't want to wait for it.

[skottish]

you'll get a seg fault encoding anything on any system that formerly had -DHAVE_MALLOC_H in the gcc CFLAGS, which includes linux.
r1413 broke a number of things,
we're working getting a fix committed.
for the time being revert to r1412 if you don't want to wait for it.

Yikes!

Thanks for the response. I don't need x264 right now; I was just testing to see if I could lose mjpegtools once and for all (png2yuv).

[Dark Shikari]

Fixed.

[skottish]

Fixed.

Yes it is. Very, very nice!

[Snowknight26]

Fixed.

Now how about that other one (http://forum.doom9.org/showthread.php?p=1366915#post1366915)? :)

[LoRd_MuldeR]

Is it intensional that b_intra_refresh isn't initialized in x264_param_default() (http://git.videolan.org/gitweb.cgi?p=x264.git;a=blob;f=common/common.c;h=6d1d7f06392b6bd2d8d7abd64d94c1b389e9b531;hb=fcf70c2f9853d74ac077a7af016e28eef05e2f5a#l39) ?

[Dark Shikari]

Is it intensional that b_intra_refresh isn't initialized in x264_param_default() (http://git.videolan.org/gitweb.cgi?p=x264.git;a=blob;f=common/common.c;h=6d1d7f06392b6bd2d8d7abd64d94c1b389e9b531;hb=fcf70c2f9853d74ac077a7af016e28eef05e2f5a#l39) ?Should be in theory, but it's memsetted to zero to begin with.

[b66pak]

@Dark Shikari it is possible to implement a cutting process for raw output when encoding with a qpfile?

example: qpfile.txt


100 I -1
200 I -1 new
300 I -1
400 I -1 new
500 I -1


an the output will be:

output.264 (with frames 000 to 199)
output(2).264 (with frames 200 to 399)
output(3).264 (with frames 400 to end)
_

[me7]

I grabbed x264 r1419 from x264.nl a few hours ago, now the site has r1416 only. Was 1419 pulled for being broken?

[Dark Shikari]

I grabbed x264 r1419 from x264.nl a few hours ago, now the site has r1416 only. Was 1419 pulled for being broken?There was never an r1419. It was an x264.nl build error due to a broken merge. It didn't differ from r1416.

[b66pak]

@Dark Shikari it is possible to implement a cutting process for raw output when encoding with a qpfile?

example: qpfile.txt


100 I -1
200 I -1 new
300 I -1
400 I -1 new
500 I -1


an the output will be:

output.264 (with frames 000 to 199)
output(2).264 (with frames 200 to 399)
output(3).264 (with frames 400 to end)
_


or may be a "--cutfile" option (reading from a cutfile.txt)?
_

[rallymax]

Hi Dark',

I noticed that you were the last person to checkin changes to libavcodec/libx264.c in the FFmpeg project.

Update libx264.c to use new libx264 features
With b_keyframe instead of IDR for detecting keyframes, ffmpeg should now
support periodic encoding with periodic intra refresh (although there is no
interface option for it yet).
Set the new timebase values for full VFR input support.
Bump configure to check for API version 83.

What do these changes do to the default values?
My goal here is AVCHD and Blu-ray legal output and want to work out what I need to tweak from the default init of the oc-> struct.

[Dark Shikari]

Blu-ray legal output is not possible through libavcodec currently.

The default values did not change.

[rallymax]

can you elaborate on that. ie - can I make changes that will make it legal?

[Dark Shikari]

can you elaborate on that. ie - can I make changes that will make it legal?Not easily. You need a patched x264, and you'll find it much easier to use (lib)x264 directly.

[Snowknight26]

Any chance of differing the FFMS2 DLL name between 32-bit and 64-bit builds? Can't seem to have both x264 versions in the same directory otherwise.

[kemuri-_9]

Any chance of differing the FFMS2 DLL name between 32-bit and 64-bit builds? Can't seem to have both x264 versions in the same directory otherwise.

put the DLLs elsewhere?
windows does have syswow64 and system32 for this...
or you could make your own directory <somedir> underneath each and add system32\<somedir> to the PATH
to have window's automatic syswow64 -> system32 mapping (for x86) do the right thing (x64 would already be correct in this situation).

[LoRd_MuldeR]

Can't it be linked statically to remove the DLL dependency and avoid possible confusion?

With the "put the DLL in your system folder" method you will run into the "DLL Hell" problem, as soon as you want to test different x264 builds that expect different FFMS2 versions...

[rallymax]

Not easily. You need a patched x264, and you'll find it much easier to use (lib)x264 directly.

Well, I'm hacking on libavformat anyhow because the m2tsenc doesn't have BDAV/BDMV support. (not that it's a hard thing to do to put a 4 byte TP_extra_header on front of the 188byte packet).

I'm also not a huge fan of how ffmpeg passes params to x264 so I'm modifying the struct directly.
Consequently - I can therefore treat my use of x264 as a direct libx264 usage.

So, if you can point me to the exact patch I need and a comprehensive list of what needs to change in the x264_param struct I'll be able to get it going.

much appreciated.

btw, can someone send me a message as to how I get a username on the SVN repos for FFmpeg so that I can check in the feature add of the BDAV/BDMV support?

[Dark Shikari]

Well, I'm hacking on libavformat anyhow because the m2tsenc doesn't have BDAV/BDMV support. (not that it's a hard thing to do to put a 4 byte TP_extra_header on front of the 188byte packet).

I'm also not a huge fan of how ffmpeg passes params to x264 so I'm modifying the struct directly.
Consequently - I can therefore treat my use of x264 as a direct libx264 usage.

So, if you can point me to the exact patch I need and a comprehensive list of what needs to change in the x264_param struct I'll be able to get it going.

much appreciated.The NAL HRD patch. Use search!btw, can someone send me a message as to how I get a username on the SVN repos for FFmpeg so that I can check in the feature add of the BDAV/BDMV support?You don't "get a username". You submit your patch to the mailing list and follow the guidelines for patch submission.

[kemuri-_9]

Can't it be linked statically to remove the DLL dependency and avoid possible confusion?

A) MinGW: yes
B) ICL:
- I) x86: with a lot of annoying workarounds
- II) x86_64: absolutely not


With the "put the DLL in your system folder" method you will run into the "DLL Hell" problem, as soon as you want to test different x264 builds that expect different FFMS2 versions...
sure, but seeing at how little the FFMS2 API changes, I don't really care.

[Brother John]

I recently stumbled across a minor inconsistency between the --psy-rd settings given at the command line and the user data x264 writes to the stream. Both these command lines
x264 test.avs -o test15.264 --psy-rd 1.0:0.15
and
x264 test.avs -o test20.264 --psy-rd 1.0:0.20
according to MediaInfo produce identical user data containing
psy=1 / psy_rd=1.0:0.2
(tested with x264 r1416 win32 from x264.nl)
The file sizes are slightly different though, so I guess the encoding itself is fine and the second decimal gets lost on its way to the user data string.

It happens consistently for me. Can anyone confirm this behaviour?

[Underground78]

It happens consistently for me. Can anyone confirm this behaviour?

Yes but it's the normal behaviour (cf this (http://forum.doom9.org/showthread.php?p=1359071#post1359071)) ...

[Underground78]

Hello,

Is it possible that some recent changes in mp4 output have made mp4 files created by x264 incorrectly decoded by VLC (screenshot (http://img10.hostingpics.net/pics/2097bug_mp4.png) - video.mp4 (http://www.mediafire.com/?w0jlmzlzb5g)) ? The same file is correctly played by MPC-HC and is correctly played by VLC after remuxing it to MKV.

Thanks

[Dark Shikari]

Hello,

Is it possible that some recent changes in mp4 output have made mp4 files created by x264 incorrectly decoded by VLC (screenshot (http://img10.hostingpics.net/pics/2097bug_mp4.png) - video.mp4 (http://www.mediafire.com/?w0jlmzlzb5g)) ? The same file is correctly played by MPC-HC and is correctly played by VLC after remuxing it to MKV.

ThanksVLC has always sucked with MP4 files. I've seen that problem for years.

[Underground78]

Ok, thank you.

[Sharc]

I recently stumbled across a minor inconsistency between the --psy-rd settings given at the command line and the user data x264 writes to the stream. Both these command lines
x264 test.avs -o test15.264 --psy-rd 1.0:0.15
and
x264 test.avs -o test20.264 --psy-rd 1.0:0.20
according to MediaInfo produce identical user data containing
psy=1 / psy_rd=1.0:0.2
(tested with x264 r1416 win32 from x264.nl)
The file sizes are slightly different though, so I guess the encoding itself is fine and the second decimal gets lost on its way to the user data string.

It happens consistently for me. Can anyone confirm this behaviour?
MediaInfo displays only 1 digit after the decimal, i.e. it rounds 0.15 to 0.2.

[LoRd_MuldeR]

MediaInfo displays only 1 digit after the decimal, i.e. it rounds 0.15 to 0.2.

Nope. MediaInfo shows the x264 settings string as stored in the file. x264 will round Psy RDO/Trellis to one digit when writing that info, so it cannot be read out at a higher precision later:

/****************************************************************************
* x264_param2string:
****************************************************************************/
char *x264_param2string( x264_param_t *p, int b_res )
{
int len = 1000;
char *buf, *s;
if( p->rc.psz_zones )
len += strlen(p->rc.psz_zones);
buf = s = x264_malloc( len );
if( !buf )
return NULL;

[....]

s += sprintf( s, " psy=%d", p->analyse.b_psy );
if( p->analyse.b_psy )
s += sprintf( s, " psy_rd=%.1f:%.1f", p->analyse.f_psy_rd, p->analyse.f_psy_trellis );

If you want x264 to store those values at a higher precision, you need to change exactly two characters ;)

[Sharc]

Ah, good to know, thanks.

[hajj_3]

x264.nl site has been down for a few days, is there server problems or something?

[mariush]

Yeah... if you guys need some temporary host let me know, I'm willing to help you out

[sneaker_ger]

It's probably just a temporary issue. Use http://mirror01.x264.nl for now.