x264 development

[Doom9]

This is a partial continuation of the former x264 development thread (http://forum.doom9.org/showthread.php?t=80910). This thread is meant purely for development purposes, which does not include making any feature requests, especially not for features to be added to the VfW interface!. Any VfW relates issues have to go here (http://forum.doom9.org/showthread.php?t=108569) instead. Any bugreports also do not belong to this thread but either in the VfW thread, or the x264 cli bugreport thread (http://forum.doom9.org/showthread.php?p=798509#post798509).

[macher]

I am new to x264 code
The decoder testing part in x264.c was removed some months ago and it has not returned since then. The exact release was a snapshot of 24-08-2005.
I want to work on the decoder for a project of my own. In that old snapshot the only thing in the coments was something like "The decoder test part is not working and we would appreciate some help in this direction". I would like to know what eactly were the problems in this regard and how much work i would have to put in to get it working again ?

[akupenguin]

The decoder supposedly supports baseline profile only. I have no idea how close to working it was in 2004, since I have never compiled it. In addition to just making it work, much will need to be updated to match changes in the functions shared with the encoder, which have been modified since then.

[macher]

from the compilation that i have i cannot get a clue as to which profiles it is supporting because i didnt see any comments about it(may b i ddint look hard enough). but i think it should be a bit more advanced.

[akupenguin]

No B-frames, no cabac, no wpred, no interlacing, and it was written before high profile existed. That makes it baseline.

[macher]

and nobody ever mad eany changes to it ?

[akupenguin]

Fenrir converted from cvs to svn in 2004-06-03, at which point earlier revision history was lost. The decoder has not been modified since then.

[macher]

is there any other open source decoder which supports the main profile or at least compatible with common output formats?

[akupenguin]

http://forum.doom9.org/showthread.php?t=108570

[macher]

thanks

[lspbeyond]

I have implemented the x264 decoder supported baseline profile several weeks ago. Unexpected, the current x264 program structure is not suitable for decoder. Especially of the interpolation module and the macroblock context cache load/save module. The experimental results show that it's half lower than ffmpeg libavcodec.
However, I won't stop improving my decoder. For I think the structure of h.264 decoder of ffmpeg libavcodec is so bad. And it's decoding speed is not fast enough.

[Leo 69]

I think the structure of h.264 decoder of ffmpeg libavcodec is so bad. And it's decoding speed is not fast enough.

That's right. Kick some CoreAVC's asses out there too :D

And now my question to the developers. What is to be done to the X264 encoder in order to improve visual quality and compressibility (i.e. what features are yet to be implemented) ?
Is there a roadmap for improving or anything else ?
It is always interesting to know in advance about what we may expect in the future, isn't it ?

Thanks

[akupenguin]

There is no plan, do not ask about future features, I will not discuss anything I'm writing until it's ready to test.

[Kostarum Rex Persia]

Well, Leo 69, try to read TODO list from http://students.washington.edu/lorenm/src/x264/todo.txt

[Tommy Carrot]

The RD subpel motion estimation (--subme 7) added in rev.476 is the old patch finally committed to the svn, or is it a completely new approach?

[Romario]

I don't know for sure, but I think that RD subpel motion estimation(--subme 7) is a old patch, with necessary changes.

[ChronoCross]

it's the old patch updated to work with the new svn. I think he may have made some updates to it in other ways as well.

[CEC]

Is the in-loop filter adaptive??? I made a high bitrate encode (720x576 2500 kbps) once with in-loop and once without! I can't see a difference between them!!!!!!

[Sharktooth]

yes it is.

[CEC]

Thanks for making it clear to me!!!:thanks:

In the Sticky: "MPEG-4 AVC/H.264 Information" bond doesn't say anything about adaptive loop in x264!!!

[IgorC]

I hope this link will be considirated as corelative (not like crossposting) http://forum.doom9.org/showthread.php?p=810697#post810697

[akupenguin]

In the Sticky: "MPEG-4 AVC/H.264 Information" bond doesn't say anything about adaptive loop in x264
Becuase all h264 codecs have adaptive loopfilter. It's impossible to make one that isn't adaptive, and still comply with the standard.

[7zeal]

I am trying to compress (2 pass, sharktooth's x264 530 - on the HQ-Insane profile, and 2000 kbps) a HD 720p movie (1280x528 @ 23.976 - a little cropped, it's widescreen) from wmv content.
I am running the first pass, as so:

H:\Movies>x264.exe --f
ps 23.976 --pass 1 --bitrate 2000 --stats "H:\Movies\[......] 16x9 72
0p WM9 HD-DVD.stats" --bframes 3 --b-pyramid --filter -2,-1 --subme 1 --analyse
none --direct auto --me dia --cqmfile "C:\Codecs\x264\Matrices\eqm_avc_hr.cfg" --progress --no-psnr --output NUL "H:\Movies\[.........] 16x9 720p WM9 HD-DVD.avs"
x264 [info]: file name gives 16x9
avis [info]: 1280x528 @ 23.98 fps (181032 frames)
x264 [info]: using cpu capabilities MMX MMXEXT SSE SSE2 3DNow!

and it finally ends up writing this (you can see, not ALL the frames were written to the .stats file)

encoded frames: 180851/181032 (99.9%), 10.24 fps, eta 0:00:17
H:\Movies>

Now, as I call pass 2 with all the other options enabled, it throws out this error:
x264 [error]: 2nd pass has more frames than 1st pass (181032 vs 180999)
x264_encoder_open failed

I am using the same x264 commandline as the latest MeGui or StaxRip 0.9.9.2, in fact this first occured while trying to compress through MeGui, x264 HQ-Insane, then I wrote the commands on my own into the console, trying to figure out what I should change - no success. StaxRip 0.9.9.0 had the same difficulty (with Sharktooth's x264 build 467 if I remember well)

I tried to trim the movie from the avs script, cutting until that last frame encoded into the .stats file , so pass 2 worked.
However, though x264 said "23.98 fps", as I took a part of the encoded movie and muxed it into mp4 (latest mp4box), BSPlayer recognized it as 25 fps. (?)
The aac encoded is perfectly synched with the source wav from the wmv, BUT the encoded x264 movie (even from minute 3) seems to be earlier I think than the wmv. (???) why..
This happened before I ran x264 with --fps 23.976 - but I don't think it made a difference since it recognized it anyway as 23.98.

[daverc]

encoded frames: 180851/181032 (99.9%), 10.24 fps, eta 0:00:17

It happens to me when the source is directshowsource in the avs script and ffdshow does audio filtering, especially channel mixing.

Try to disable every filtering before launching your avs script.

[Sharktooth]

this is the dev thread and it has nothing to do with those kind of questions...

[7zeal]

I thought it was an x264 bug, doesn't that qualify?

[Sharktooth]

yes, sorry, i was a bit too drastic.

[7zeal]

It happens to me when the source is directshowsource in the avs script and ffdshow does audio filtering, especially channel mixing.

Try to disable every filtering before launching your avs script.

Thanks, seems like I may have a lead here..:P
(note: have lastest ffdshow-2546-gcc4.0.3-sse2-x264)
I had "Dolby Surround Downmix" on at audio processing plugins to be loaded automatically, but don't know if it's to blame, unchecked it anyway. (oh, and "subtitles" was also ticked for automatic loading in ffdshow at video decoder config).
I also have 'Avisynth' ticked at the video decoder config... hmm
It's suppose to bring ffdshow filters into avisynth. I left it on.
Is there anything else I should be doing ? I'll get back to you after running the first pass.

Thanks

[7zeal]

Same behaviour .. All ffdshow filters out, and the first pass still stops at 99% , 24 sec remaining.
What's wrong?

[ChronoCross]

Same behaviour .. All ffdshow filters out, and the first pass still stops at 99% , 24 sec remaining.
What's wrong?

This is the development thread. Bug-reports do not go here. Open a new thread or find the bug-thread. Thanks

[Xesdeeni]

Please forgive me (and be kind) if these are braindead questions (consider the source):

I'm trying to determine what the output file format is for the various apps wrapping the codec, specifically under Windows. Obviously the DirectShow AVI filter wraps the data in an AVI file format. But does the command line application use another file format? Which one(s)? And is it true that the h.264 standard file format is Quicktime (MOV)?

Xesdeeni

[GodofaGap]

CLI can output to raw h264, Matroska and mp4 (if compiled with GPAC library). There is not one standard container for h264.

[bond]

There is not one standard container for h264.indeed, there are three:
1) .mp4
2) .mpg
3) .ts

[Sergey A. Sablin]

indeed, there are three:
1) .mp4
2) .mpg
3) .ts

what is .mpg here? PS? or AVC Annex B format?

[RBF]

Sergey
.mpg -> PS
AVC Annex B is the byte stream format, not the container format, неправда ли?

[Sergey A. Sablin]

Sergey
.mpg -> PS
AVC Annex B is the byte stream format, not the container format, неправда ли?

right, just want to clarify things - I saw many .mpg files with TS/PS/VES/AnnexB inside. Sometimes even just audio.

[bond]

yep, i meant PS with .mpg

[Romario]

Look at this, can anyone confirm that devs read this.

http://www3.intel.com/cd/software/products/asmo-na/eng/219766.htm

[KoD]

Intel has those libraries on sale for quite some time. But it's not like they're free for use.

[Romario]

I don't know why Intel want money for this.

If this package become freeware, then devs can optimize their programs and codecs for Intel Core 2 architecture MUCH EASIER.

[Koti]

I don't know why Intel want money for this.

If this package become freeware, then devs can optimize their programs and codecs for Intel Core 2 architecture MUCH EASIER.

you could always buy the libraries for x264 dev's ;)

[Romario]

Yes, but I can't.

[KoD]

Because Intel is not making money only from selling Core2 Duo processors. They are actually producing more hardware and software than just cpus for personal computers.

[Romario]

Certainly, but...

If someone decide to make pressure on Intel software division, who knows what can happen? :)

[Sharktooth]

It's like asking M$ to make Windows OpenSource...
and however x264 has ASM optimizations that maybe are even faster than intel libs...

[Adub]

faster, really? is that because they are done by hand? But I thought that x264 hadn't been all the way optimized yet.

[Sharktooth]

Hand optimizations are always better than third party libs (that work well only on certain scenarios), also qualitywise x264 is as fast if not faster than commercial h.264 encoders... so how can you say it's not optimized?
Maybe there could be further speed optimizations but it's already quite fast.

[Adub]

I thought that Ateme blew it out of the water, maybe not in terms of quality, but speed wise. Maybe I am just hearing voices again.

Also, I ask about hand optimizations because I am interesting in learning to program and speed is a fetish. Ha! :)

[imcold]

so, learn about programming. It's fun and it doesn't hurt - allright, debugging can be very frustrating, but the bigger will be your joy after you finnaly manage to make the code behave the way you want.
Large programs are broken into smaller parts: functions. When you want to make your program run faster, you search for the functions that are most computationally intensive and then you find a better (faster) algorithm for it, you rewrite it in assembly, or very often both. Compiler may not generate the best assembly code for your function -it's just a program, not thinking being- in contrast with creator of the code, who is perfectly aware of what the function is supposed to do. Optimalizations (most of the time?) take advantage of SIMD (single instruction - multiple data) instructions: MMX, 3DNow!, SSE1/2/3 aso. They are kind of... parallel processing instructions, they provide huge speedup if properly used. X264 has plenty of optimized functions. It's kind of complicated for compiler to rewrite your function into SIMD instructions. So is for humans... but humans are intelligent, thinking beings, and creative too. You can find out an algorithm, that would work much better in SIMD. Compiler sort of... can't. That's why I believe humans will always (or for a very long time) be better for optimizing.

Also, about optimizing for some CPU - under this I understand using instructions that suit the CPU is best at: be it 3DNow for (old?) amds, SSE2 for core2, 64-bit instructions for 64bit processors on 64-bit OS etc. So in my eyes x264 is pretty much optimized for core2. Also I admire the programmers that take part in developing our beloved oss encoders and find them very skilled and inteligent (and I am most thankful for their work); I doubt that Intel's optimized code would be significantly faster - IF it would be faster at all.

sorry if this was kind of boring reading :D (also corrections welcomed). I also think that x264 is very fast taking it's complexicity (that makes me dizzy) into consideration (hey, it got faster by >30% during 2006 according to my tests). Big thankyou for the devs.

[pgb]

Hear, hear, imcold!

Anyone wanting to learn more about real programming and squeezing the most out of those clock cycles could do worse than read Don Knuth's magnum opus:

http://www-cs-faculty.stanford.edu/~knuth/taocp.html

And I'd like to add my thanks to the X264 developers too.

Happy holidays, everyone!