MSU x264 improvement published

[DmitriyV2]

As I have already told, VideoGroup is making research in area of video compression. Last autumn we have started to analyze x264 sources. Now we have first results!

The first modification is start parameters of codec. We replaced "start_qp" predefined parameter with function of target bitrate. It allows codec to predict first frame size more precisely. The improvement is significant at low bitrates with relatively short sequences. In our tests we outperform original version of codec (sources 02.12.06) up to 2 Db.

Main impact is provided on small sequences (on the begin of video)
http://www.compression.ru/video/x264/images/image002_en.png

Improvement is bigger on low bitrates:
http://www.compression.ru/video/x264/images/image016.gif

More detailed information (including modified sources, binaries and examples) can be found here:
http://www.compression.ru/video/x264/x264_improvement_en.html

Enjoy! :)

[bob0r]

@DmitriyV2

Codec's web pages:

x264 web page
Another x264 web page


x264 web page <-- main website is http://developers.videolan.org/x264.html

x264.nl is an unofficial site for win32 builds.

Please use http://x264.nl (don't use www., www. pisses me off)

[akupenguin]

Why did you choose (a*bitrate^b+c) as your model? QP is inherently a log scale, so (a*log(bitrate)+b) seems more applicable.
Also, that ignores resolution, so (a*log(bitrate)+b*log(pixels)+c).

Could you include a 2pass encode on the per-frame psnr graph? That being the target that ABR would achieve if it had perfect knowledge of all frames' complexity.

[Sirber]

Any build with updated source? :D

I only do low bitrates :D :D :D

Good work!!!

[IgorC]

Many people already have "experience" with OPSNR values. Even the different revisions of x264 may have different OPSNR ( approx. +/- 0.3 maybe more) and SSIM. But it has nothing to do with quality. Firstly check visually then check SSIM and OPSNR values.
We've already seen that before MANY times. The biggest value of PSNR and smooth as hell video. :(
Cheers.

P.S. Psnr increase -> SSIM decrease and viseverse. As a result the quality is approx. the same.
If PSNR and SSIM increase that's another deal.

[Kostarum Rex Persia]

Very good work, DmitriyV2. Do you plan to continue that test, but with newest x264 source, this time, from March 9?

[DmitriyV2]

Any build with updated source? :D
I only do low bitrates :D :D :D
Good work!!!

Sure! See "Download" section on page!
MSU Patched:
http://www.compression.ru/video/x264/x264_msu_improved.zip
Original:
http://www.compression.ru/video/x264/x264.zip

Thanks! :)

[DmitriyV2]

Very good work, DmitriyV2. Do you plan to continue that test, but with newest x264 source, this time, from March 9?
Sure! This project is not so big, but we plan to continue.

[DmitriyV2]

x264 web page <-- main website is http://developers.videolan.org/x264.html
Ok! We will fix this.

Please use http://x264.nl (don't use www., www. pisses me off)
Ok! Only for you! ;)

[DmitriyV2]

Why did you choose (a*bitrate^b+c) as your model? QP is inherently a log scale, so (a*log(bitrate)+b) seems more applicable.
Also, that ignores resolution, so (a*log(bitrate)+b*log(pixels)+c).

Could you include a 2pass encode on the per-frame psnr graph? That being the target that ABR would achieve if it had perfect knowledge of all frames' complexity.
Really we test a big amount of variants and select varian, that provide the biggest performance gain for several videos.

Now we work on more complex x264 measurement, but we allready spend a lot of time for some technical problems with codec state saving. Code is good, but some hack's (like goto) bother clear returning to previous state. Looks like with solving of this problem there will be easy to make more complex experiments with x264 and provide better results in 1- and even 2-pass modes (variety of decisions in H.264 is pretty big).

[foxyshadis]

If this works for abr, can results be reasonably extended to crf, or are they too different? Very interesting results, I will try patching the latest version and see what happens.

[akupenguin]

crf never had the problem that this patch tries to address. cbr worked too, since vbv limited the 1st frames sizes.

[hpn]

I just manually patched rev.465 with the MSU source and did a short test at 450kbps (2500frames). It's not a big PSNR difference (if any) compared to the clean SVN build, but maybe I'm not encoding with the best options for MSU.

[akupenguin]

I just manually patched rev.465 with the MSU source and did a short test at 450kbps (2500frames)
Not short enough. The patch only matters on the first few frames.

[sysKin]

So basically, you invented a single-line patch that is based on a not-very-well-thought-of model and which tries to optimize one parameter known to have very little effect on anything?

I don't want to criticize your work, but, if you don't mind, I'd like to express my criticizm on the hype you created. Even if you do it right (taking resolution into account is a no-brainer), this hardly deserves a single email with a patch, in my very humble opinion.

I'm sorry if my words sound harsh...

[edit] Since your graph shows improved quality for almost all frames, it's pretty obvious that this can only be achieved with one clip having a higher bitrate. What are the bitrates of the two clips?

[sasam]

sysKin, I also think its a lot of hype for this modification alone, but they said that they will try to improve more complex x264 functions from which the 2pass mode and long encodes will benefit as well so its OK with me...
Its nice to see more people getting involved in x264...

[akupenguin]

but they said that they will try to improve more complex x264 functions from which the 2pass mode and long encodes will benefit as well so its OK with me...
So hype that instead. Don't claim "up to 2 dB improvement!" for something that only affects a few frames using certain option sets known to be suboptimal (i.e. abr).

[Audionut]

With a movie encode for example. The first few frames are usually black.
So am I correct in thinking that this patch would do nothing for a real encode?

However, if the experience learned here, could be used to improve an entire movie encode, that would be something to boast about.

[DmitriyV2]

I don't want to criticize your work, but, if you don't mind, I'd like to express my criticizm on the hype you created.
Who can help with removing "goto" from x264 code and prepare codec state accurate saving?

This work already take up serious time and we have serious problem with code.

[DmitriyV2]

Not short enough. The patch only matters on the first few frames.
Sure. We explaine on page - this is change in initial codec state.

We assume that not so good results of x264 in comparison with other codecs on standard sequences like flower, tennis and so on was due to this point what we find.

This is our first result of x264 improvement. It's code already optimized pretty good, so not so easy to find place, where it can be seriously improved.

Most future improvement of x264 will be in range 0.05-0.15 dB - and this will be result of a weeks of research. But only if big amount of such small improvement will be summarized - final result better than other codecs can be reached.

In this research now we collide with problems with accurate codec status (state) saving (for future returning to this state). Such feature will allow essential increasing of research efficiency.

Who also work with x264 code improvement here now?

[IgorC]

Who can help with removing "goto" from x264 code and prepare codec state accurate saving?


Considirating your previos erroneous experience in comparison area (when in your result Xvid was even worse than MS MPEG4-2 and the last when Xvid was very worse than WMV9. When all visual, psnr,ssim results say vice versa) etc. Many people will doubt about your help (even to remove goto from c/c++ of x264). Sorry for my hard critics but it's true.

[DmitriyV2]

@IgorC

You criticize many people (as I see in your other posts).

Maybe you did not do enough things by your own hands? :)

P.S. Thank you for idea to measure SSIM (it's also become bigger).

[IgorC]

@IgorC
Maybe you did not do enough things by your own hands? :)
[/SIZE]

I study and work. And do some little (but complex ... not single line code ;) ) electronic projects c/c++ + hardware. And the video/audio compression is only my hobby.

Just run a little test. OPSNR is lower for r.465+MSU patch -0.09

[Sagittaire]

In this research now we collide with problems with accurate codec status (state) saving (for future returning to this state). Such feature will allow essential increasing of research efficiency.

1 pass ABR will never the best mode for quality simply because 1 pass can't analyse the overall complexity and choose best Rate Control for quality. Your conclusion are certainely good but only for for 1 pass ABR mode and nobody use this mode for best target quality/size result.

If you want make "real life comparison test" choose always multi-pass mode.

[DarkZell666]

An improvement is an improvement, and an improvement can always be improved, so all stop whinning and let them have a try =)

[akupenguin]

Who can help with removing "goto" from x264 code and prepare codec state accurate saving?
Why exactly do you want to remove "goto"? I assume you mean only the one at encoder.c line 1480, because the others are all trivial error handlers. (Yes, I can easily convert the goto into some other construct if you can convice me that something else is better. So tell me what you're trying to do that doesn't work now.)
Do you encode a whole frame, then undo it and encode again with different settings? Is your state saving any more complex than a deep-copy of x264_t, or are you just trying to find an efficient way by not copying the frame buffers?

[DmitriyV2]

Dear all! I was in trip, so only now answer to you.

I study and work. And do some little (but complex ... not single line code ;) ) electronic projects c/c++ + hardware. And the video/audio compression is only my hobby.
Or! One young guy from our lab (who has the same hobby) extremly wants to test any you public projects or papers! :)

Just run a little test. OPSNR is lower for r.465+MSU patch -0.09
Fine! :) You know C++! You know how to measure! So you may suggest your own patch that will increase quality for any files (maybe based on our patch, maybe not)! It's will be really cool - to see you patch that increase quality maybe +0.01 dB, but for all files.

Your conclusion are certainely good but only for for 1 pass ABR mode and nobody use this mode for best target quality/size result. If you want make "real life comparison test" choose always multi-pass mode.
We have request for 1-pass mode from companies (I think that most devices has 1-pass mode only).

Also I remember your opinoin from previous thread :). As I wrote - we plan to remove limitation for 1-pass for next H.264 comparison, so if developers will deside to use 2-pass mode in "Best PSNR" category - that's ok.

An improvement is an improvement, and an improvement can always be improved, so all stop whinning and let them have a try =)
Sure! Thanks for post! :)

[DmitriyV2]

Why exactly do you want to remove "goto"? I assume you mean only the one at encoder.c line 1480, because the others are all trivial error handlers. (Yes, I can easily convert the goto into some other construct if you can convice me that something else is better. So tell me what you're trying to do that doesn't work now.)
Do you encode a whole frame, then undo it and encode again with different settings? Is your state saving any more complex than a deep-copy of x264_t, or are you just trying to find an efficient way by not copying the frame buffers?

Thanks for very practical post, concern our problem! :goodpost:

During this time Sergey Sablin (he appear here under this name) wrote us directly. We sent him our patched source code of x264 with ability to save codec state and he helped us to fix problem! So now we can move forward with more sofisticated measurement on long sequences.

King regards for help suggestion to you and for help to Sergey!

[IgorC]

Or! One young guy from our lab (who has the same hobby) extremly wants to test any you public projects or papers! :)

You don't beleive me. It's up only to you

You know C++! You know how to measure! So you may suggest your own patch that will increase quality for any files (maybe based on our patch, maybe not)! It's will be really cool - to see you patch that increase quality maybe +0.01 dB, but for all files.

You want to say that MSU imporved the x264 +0.01 opsnr? Well, your "improvement" for first 10 sec. of video for 700 kbit/s was only +0.15 +0.20 opsnr and only for 1st pass.

some math : 1 movie 7200 secs
(10sec/7200sec) * 0.2 opsnr = + 0.00028 opsnr with your patch and only for 1 pass.

You used suboptimal settings of x264 for 1 pass. You patch doesn't work for 2 pass.

C/C++ is used in many areas if you didn't know.
I will say directly I don't understand almost nothing on x264 code, because it's not my area.
Yes my electronic projects have nothing new ( something like LCD + temperature sensor). But I'm not saying that I'm genious and can improve the performance of componets like your statements about "+2db".

I'm not saing that you didn't know how to program UART 16xxx , parallel/serie ports etc....
The same way I would ask you to participate on my area. You even don't know how to measure resistence.

Don't try to mess up when you've received (not only for me) hard critics.

If you have something to say then send me pm. I never read them.

[Blue_MiSfit]

I think we all might want to take a step back here. This thread has degraded into pointless bickering.

This MSU patch seems to improve quality (somewhat) when using 1 pass ABR. That much is apparent, no?

Okay, so nobody on this forum uses 1 pass ABR, because x264 is mostly used for DVD backup where the source files are available for a 2 pass encode. However, sometimes you need to do a 1 pass encode - capping off of TV or some other similar goal.

While this patch doesnt really help out most x264 users, it is nevertheless valid.

I'm no genius. I've only been observing this community and doing my own encoding work for a couple of years. I'm no engineer or programmer. But it seems to me that the work that has been put into this patch is perfectly valid and could be potentially used to improve the quality of 1 pass commerical AVC encoders, which is a good thing :)

Let's all just stop taking potshots at one another eh? Lets all get along :D

Just my two cents.

~misfit

[DmitriyV2]

You don't beleive me. It's up only to you.
This is up to all guys, you critic on this forum.

Do you have ANY contribution to ANY open project (or you own free projects)? Maybe you add your cores on VHDL to http://www.opencores.org/? It's ok, just let us know! :)

[IgorC]

BE SURE I let know about it at certain time.
If it will be here it won't be like patch with +0.00000002 opsnr.
And if it would something like your "patch" I wouldn't show that.

All russian and spanish sites and forums of video compression will be informed about your impovement of x264. you got it for yourself. :)

[DmitriyV2]

Igor!
It's very simple idea!
To spend time to create or improve something for other guys.
Why not?

[DmitriyV2]

I try to understand - why you did not spend your time to create or improve something? You hate this idea? Why? (sorry for questions - sometimes I try to understand deeply other guys).

[DarkZell666]

I might be mistaken but people (in general, not only here) seem to want more, always more ... and are never satisfied if only a 'small' improvement is made, even if it actually IS an improvement (I'm not able to produce such results myself so I'm quite happy with it :)).

Some would obviously get nervous seeing some 'up to +2db PSNR improvement' statement and only observing a tiny average +0.01dB ... for sure there is some sort of distortion of reality here ;)

But, the progress made on x264 by it's devs' is much more significant than just a +0.01db, which is why posting such promising screenshots might be a bit insulting ...

I think the work that has been done is rather interesting and a good _starting_ point, and is to be _continued_ ...

[IgorC]

I try to understand - why you did not spend your time to create or improve something? You hate this idea? Why? (sorry for questions - sometimes I try to understand deeply other guys).

are you speaking about how to spend time to create?
when there will be +0.01 opsnr improvements for real 2 pass encoding and not only for first 10 sec only then person would say "there was very tiny impovement".

First of all people spend their time to learn how things are working and only then improve it. It will be unlogical to impove something when you have only remote idea about it. For example your patch.

[bond]

guys, plz be nice to each other. i think every effort made to enhance x264 is a good thing :)

all in all its the decision of the main x264 devs whether they accept a patch or not

[akupenguin]

Short answer: patch is rejected until my original questions are addressed:

Please include resolution. Anything that uses the same QP for the same bitrate in QCIF as in 1080p is just wrong.

Why did you choose (a*bitrate^b+c) as your model? Please demonstrate that it's better than the logical model (logarithmic) if you have in fact done such a comparison. This may depend on the results of the first question.

Or one might try a more theoretically sound approach to the whole problem, which is to adjust the initial satd->bitrate predictors (ratecontrol.c lines 288-295) so that an ad-hoc qp isn't needed.

[IgorC]

DmitriyV2 Please answer to questions.
Few times I saw you online. There is no excuse that you were busy or on vacations.

[DmitriyV2]

I't better to spend time to do something.

akupenguin - I guess we prepare parer with approach and results description.

[Audionut]

DmitriyV2, you stated that your patch raised psnr.

You have been critized.

And in my opinion, for good reason.

Your patch increases psnr for the first couple of frames.
Now, let's put this into a real life senario.

A. It don't work on 2 pass.
B. It's only having a major effect on the black frames that are at the start of any movie.

This patch means absolutly nothing to me, and to hazard a guess from the responces you have recieved, it doesn't mean diddly squat to many other either.

While your work to improve x264 is good.
Perhaps next time, you will wait until you have produced a patch that will benifit end users.
edit: or, say that you have produced a patch that does such and such, how do we all go about making it benifit the end user.

Not: We increased the psnr on the first 5 frames of an abr encode, We Are The Bomb.

[DmitriyV2]

Again! :)

We start project with goal to improve x264 in September 2005. There was a big amount of tests and approaches and not all of them are finalized now (we plan to use there results in future). Also, of course, big time was used for code understanding.

Published modification is modification of start parameters of codec. We analyzed several sequences and found that start parameters was not optimal frequently, especially for low bit.

New formula of such parameters was suggested. On schene from Matrix movie on first 30 frames advantage was +4-5dB, on 30-100 frames around +2 dB, on 100-230 frames +1 db and less on next frames where codec collect correct statistics and become more optimal. Please see first post for detailes http://forum.doom9.org/showthread.php?p=797167#post797167
Of course then longer sequence, then less significant start parameters changes. So this is important only for small sequences, including commonly used "standart" sequences of codecs testing.

Nobody claim that this is optimal patch or something also. Main ides: we found a place, where codec can be improved - let's do it. You can use our patch, you can improve it (good ideas was in this thread), you can suggest you own ideas. Let's move forward! I think this place will be patched anyway due to codec improvement work. (maybe akupenguin will make this contribution).

Yes! I know that nobody here use one pass encoding. But, please, be patient, sometimes it's useful!

Not long ago guys from Motorola Research Center contact us about H.264 comparison technical detailes. They test codecs for usage in new cell phones to save cell phone camera video. And they use short sequence, and they use low bitrate, and 1 pass.

For us main result - was help with codec stage saving. This was technical thing, that stops our progress for a long time, because advanced codec measurement for long movies impossible without codec compression rollback. So a we receive support (king thanks to Sergey) after publication and become happy and return to research. :)

Final idea: let's move forward for fun. We can help to anybody who will patch this place (and now we working on long sequences). And we are very appreciate for help to us, that allows us to move forward.

[foxyshadis]

Audionut, 6 people including yourself have already made that point. He doesn't need to be hounded for one mistake (some overenthuseasm) forever by you and IgorC. He also pointed out that it's meant for streaming, which invalidates your point about 2-pass, plus they've already moved on to researching other tweaks. (A glance at google video or youtube shows how many streamed videos don't start out black.) My understanding is that most of their research is going to be in making it more streaming-friendly.

Edit: Oops, Dmitriy replied first. :p

[Sirber]

Can we commit this and move along?

[DmitriyV2]

My understanding is that most of their research is going to be in making it more streaming-friendly.
Sure!

Edit: Oops, Dmitriy replied first. :p
Thanks anyway! :) Hope flame will be stopped.

[bratao]

DmitriyV2,
Thank you for your great work ! Keep on !
Im opensource developer, and i love a plenity of people saying "thank you", but i know that always have people that talk shit !

[Sirber]

Im opensource developer, and i love a plenity of people saying "thank you", but i know that always have people that talk shit !:goodpost:

Usualy, more "shit" than "thanks" :(

[ChronoCross]

some people mistake helpful critism for shit. which is why everyone thinks there is so much shit being given to open source developers.

[Sirber]

badly posted "helpful critism" is usualy interpreted as shit...

[IgorC]

I can't understand one thing :
If MSU are caring about imporvement of x264 then why they don't cowork with devs.
Why they just go for private close invistigation and only then make a big presentation?

We have already seen those "private projects" so many times and it always finished by commercial proclamations as "dvd quality at 400 kbps" or "CD quality audio at 64 kbit/s".

If they _realy_ cary about open source development of x264 then they just submit there little patchs and will discuss it here or pm.

During 5 months they impoved only first 10 secs at 700 kbit/s and only for 0.2 opsnr ( 1 pass only). Now this patch is already obsolete for last revision of x264.
So lets wait next 5 months . Maybe there will be real impovement 0.05-0.10 opsnr for 2 pass encoding. But there is no sure that 5 months later x264 will still need that patch.

MSU are working not only on x264. Each month they have some new projects - filters, lossless, big and long comparisons.
They are not going to work hard on it. They are looking for commercial structures will look at them that´s why there is "+2db improvements on x264" on MSU sites.

Many projects only in one month http://www.compression.ru/video/index.htm
When they will have time to work on x264?

[DmitriyV2]

When they will have time to work on x264?

You guess, you suggest a "helpful critism"?

Again! :)

That's why we post a place of improvement, description of suggested idea and sources. Main idea: we found a place, where codec can be improved - let's do it. You can use our patch, you can improve it (good ideas was in this thread), you can suggest you own ideas. I think this place will be patched anyway due to codec improvement work.

Now we work on main codec rate control.

Inspirational support is welcome! :)

[IgorC]

When there will be real result for 2 pass and large number of frames people will help and test it.

Now there is rejected patch. -> Destructive critics. Time to think for you.

[dragongodz]

ok enough already.

DmitriyV2 made a ridiculous claim of 2Db improvement because he used a tiny sample. he was already, rightly, called to task for that claim.

to those who only ever use 2 pass, what the hell are you even posting in this thread for ? if you dont use 1 pass abr ever then this has got nothing to do with you and you have absolutly no right to criticise someone for trying to improve it, let alone dare to tell someone they should not be trying to make any improvement to it because you dont use it.

as for questioning why he has tried to make this change on his own rather than work with the devs, again who the hell do you think you are ? if someone wants to try and make an improvement and only submit it to the devs or even here then that is up to them. again you have absolutly no right to bitch about it or try and come up with some conspiracy theory.

akupenguin has rejected the changes and asked some questions. if DmitriyV2 wants to see these changes made then he either answers them or makes changes based on what akupenguin has said. these are the only important things and this constant bickering back and forth is pointless. bond already said to be nice but that seems to have fallen on deaf ears. please continue to to take pot shots if your only purpose is to see this thread closed.

[bond]

Strike for rule 16 for IgorC

as dragongodz and foxyshadis and me said it has been enough said about this, no need to bug around anymore

akupengiun said what he wants to get from DmitriyV2, now its up to DmitriyV2 to answer his questions...

[DmitriyV2]

akupengiun said what he wants to get from DmitriyV2, now its up to DmitriyV2 to answer his questions...
We will prepare them. There was generated about 12Mb of numbers in CSV files (by frames results on several sequences - smaller amount of data is unreasonable because average advantage of patched formula will be lower). Next week we will simulate them in Mathlab and provide results.

Also last several month's we are subscribed on x264 developers maillist. This result was announced simualteniously here and there (Sergey Sablin, that helped us with codec state saving, wrote us from maillist announce).

[IgorC]

Any news from "+2dB improvement"?

[DmitriyV2]

Now we sending new measured information on x264 directly to Loren Merritt (x264 rate control developer).

[Romario]

Why not you publish new measured information on x264 in public?

[DmitriyV2]

Why not you publish new measured information on x264 in public?
As scientific guys, we did not like to publish scratch data and try to get up data. And this is additional work, so we publish less results, than we have.

But we going to prepare a report on x264 options efficiency.

[Adub]

Just thought I would say thanks for your guys' work. It is much appreciated.