What's left to improve for x264?

[Chengbin]

I've been out of the loop for video encoding for 2-3 years now, but I remain interested.

Looking at the git for x264, it seems development has slowed down dramatically. Dark Shikari's x264 blog (something I enjoy reading very much) hasn't had a new post for almost 2 years. There is no x264 Google Summer of Code in 2012 or 2013.

Is x264 getting to XviD's point where there's not much more to do? If there is, what's left to add or improve for x264?

[Guest]

Every project that is developed seriously with a view to making it useful and bug-free, and it's pretty obvious x264 qualifies with flying colors in that regard, is going to see a gradual tailing off of fixes and revisions. x264 is limited by the specification, and at some point you approach the limitations of the specification. Possibly HEVC will replace it but for now x264 rules. I wouldn't expect any major advances but that may be due to a limitation of my imagination.

[Chengbin]

Every project that is developed seriously with a view to making it useful and bug-free, and it's pretty obvious x264 qualifies with flying colors in that regard, is going to see a gradual tailing off of fixes and revisions. x264 is limited by the specification, and at some point you approach the limitations of the specification. Possibly HEVC will replace it but for now x264 rules. I wouldn't expect any major advances but that may be due to a limitation of my imagination.

Thanks for the reply neuron2!

I understand x264 is limited by specification, but speed optimization is also a large part. Especially x264, which is renowned for its extreme optimization.

I remember seeing a hand drawn chart of check boxes of speed optimizations that could be done for x264 by Dark Shikari a few years ago. Can Dark Shikari, or another x264 developer comment on the amount of optimization left that can still be implemented, and approximately how much optimization that would translate into?

[Sagittaire]

Every project that is developed seriously with a view to making it useful and bug-free, and it's pretty obvious x264 qualifies with flying colors in that regard, is going to see a gradual tailing off of fixes and revisions. x264 is limited by the specification, and at some point you approach the limitations of the specification. Possibly HEVC will replace it but for now x264 rules. I wouldn't expect any major advances but that may be due to a limitation of my imagination.

well not really exactly. x264 have certainely approch the PSNR limit of H264 standard but HVS optimisation is always possible (see lame project). Anyway it's really complexe to make serious developpement in this domain for indepandant dev like for x264 project. Dev can just read research publication in this domain and try to apply them in x264 code. There are many domain where x264 can make better job in futur:
- Adaptative Quantisation (HVS): stable AQ1 is complexity masking. Luma masking and contrast masking can be really usefull.
- weighted prediction have some difficulty to keep constant quality for x264. Ateme WPred make really better job for example.

[mastrboy]

The latest revisions has seen a lot of AVX2 optimizations, but if I read them correctly most of them are for 8bit only, so the same optimizations should hopefully be added also for 10bit.
There's also AVX3 coming in 2015/2016 which x264 might benefit from.

[Asmodian]

[...] approximately how much optimization that would translate into?

This isn't known, it is really hard to quantify how much any idea would actually help. It is even hard to quantify how much an idea did help after it is implemented let alone before it is implemented.

Unless you mean speed optimizations? I am not sure but it might be easier to guess how much they would help.

[Chengbin]

This isn't known, it is really hard to quantify how much any idea would actually help. It is even hard to quantify how much an idea did help after it is implemented let alone before it is implemented.

Unless you mean speed optimizations? I am not sure but it might be easier to guess how much they would help.

Yes, sorry, I meant speed optimizations.

[benwaggoner]

Low hanging fruit in rate control: a --max-rf parameter that can be used instead of or with --bitrate for 2nd pass encodes.

That would make multi-bitrate capped VBR encoding a lot more efficient as .stats and .mbtree files could be reused from an analysis pass.

It would also be nice for a variety of scenarios to be able specify a max RF and a max ABR at the same time. Thus "CRF=18, unless it would be over 12 MBps, than do 12 MBps."

[kabelbrand]

That would make multi-bitrate capped VBR encoding a lot more efficient as .stats and .mbtree files could be reused from an analysis pass.

This should be possible already even with different output resolutions. I don't think you can mix ABR and CRF though.

http://mailman.videolan.org/pipermail/x264-devel/2012-April/009294.html
x264 supports this with adaptive GOP too; all encodes with the same lookahead settings and the same source will give the same distribution of frame types, no matter what the bitrate.

http://mailman.videolan.org/pipermail/x264-devel/2012-July/009431.html
Support changing resolutions between passes even with macroblock-tree on (at the cost of a small amount of compression).

[mandarinka]

Thus "CRF=18, unless it would be over 12 MBps, than do 12 MBps."

Am I missing something? I thought that is what x264 does when you encode in crf modde with vbv enabled?

[Groucho2004]

Am I missing something? I thought that is what x264 does when you encode in crf modde with vbv enabled?
I think you are confusing average and maximum bitrate.

[benwaggoner]

The problem is that you can't use --crf and --pass 2 at the same time.

[detmek]

No, but you can use --crf xx --pass 1 --slow-firstpass and, if obtained file is too large use --pass 2 --bitrate xxxx with previously generated stat file.
If you are good in writing scripts you can write a BAT script to automaticly do that. Or some of developers could add this into their GUIs.

[06_taro]

http://wiki.videolan.org/X264_TODO

[detmek]

The problem is that you can't use --crf and --pass 2 at the same time.

BTW, there was a GUI that could do conditional double pass, CRF first pass and, if size doesn't match, second bitrate pass - leimings x264 gui (http://www.free-codecs.com/download/Leiming_x264_GUI.htm). It worked by generating a BAT file.

[iwod]

They have a monthly Newsletter on their mailing list if you want to know what is the latest.

That Todo list seems huge. But i think anymore improvement in terms of Quality / Speed will take a lot of time to grind. x264 speed is already hand ( assembly ) optimized, and Quality difference should be minimal.

I would rather they start to grind on h.265 :D

[mandarinka]

That todo list is more like a list of things "that could be done" if somebody cared. As you can see, most of those things haven't got any adopters during the years the document exists.

I think you could get some WIP but abandoned patches from IRC for stuff like weightp improvements (I think somebody even had builds of x264 with kmeans weightp, but IIRC it crashed without mbtree or something and it wasn't clear if it really works). There was also --me trellis, which wasn't finalised and merged in the end, but the code for it should be obtainable.

x264 is really fairly mature and it probably isn't straightforward to really push the core working of it any further.

[Dark Shikari]

That todo list is more like a list of things "that could be done" if somebody cared. As you can see, most of those things haven't got any adopters during the years the document exists.Yeah, this is more the intent; the TODO list documents ideas and abandoned concepts that someone could experiment with if they wanted something to poke at.

I originally created it because I wanted to try to avoid the situation where most of these just existed in my/pengvado's head.

[benwaggoner]

No, but you can use --crf xx --pass 1 --slow-firstpass and, if obtained file is too large use --pass 2 --bitrate xxxx with previously generated stat file.
If you are good in writing scripts you can write a BAT script to automaticly do that. Or some of developers could add this into their GUIs.
That would address the case of a single file encode, and I've got the .bat file that can do that.

But it wouldn't help the multiple bitrate encode problem. Being able to reuse analysis pass .stats and .mbtree files could save a lot of CPU and memory on additional encodes from the same source. Doing a simultaneous full bitrate range encode that includes several 1080p and 720p encodes each can eat up 10+ GB of RAM.