x264 to JM comparable quality

[jakor]

I ran several tests comparing these two and it seemes it is impossible to choose any preset or whatever in x264 to even get close to JM. actually I use MVC version of JM (view 0, which is actually normal AVC stream).
I use preset placebo, no psy, subme 9 etc. the best quality possible. and its constQ.
JM is better by 0.5 dB in quality and by 4% in bitrate.
Any coments ?

[thewebchat]

Have you tried, perhaps, looking at the output?

[Dark Shikari]

Try --preset placebo --tune psnr, in two-pass mode of course.

You might still have left adaptive quantization on (no-psy does not turn it off), which will greatly lower PSNR.

Additionally, if you meant that you used constant quantizer in x264, constant quantizer disables macroblock-tree, one of x264's most impressive optimizations PSNR-wise.

Furthermore, note that if you set constant quantizer, non-adaptive GOP, all psy optimizations off, and so forth, x264 will almost certainly be worse than JM in terms of PSNR. This is intentional. It's also about 1000 times faster.

[LoRd_MuldeR]

JM is better by 0.5 dB in quality and by 4% in bitrate.

And how does the speed compare to x264?

[jakor]

sometimes speed matters nothing while quality matters everything.
--tune psnr and --aq-mode 0 did not help at all, preset placebo was already turned on.
the experiment motivation was to compare how x264 and JM behave themselves in equal conditions. And letting x264 doing 2-nd pass will break the experiment conditions.

Maybe it was intended for psnr to be lower, but this much lower... I don't know.

I will check subjective quality and return later.

[jakor]

yes. x264 has severe I frame fluctuation and several annoying artifacts, while Jm does not have any of those.

[Dark Shikari]

What's with the sudden explosion of people reporting quality issues with x264, but not uploading any samples whatsoever of the problem? :confused:

Is this some bizarre attempt at FUD?

Furthermore, seriously, complaining about visual quality after turning off psy optimizations? Is this a joke?yes. x264 has severe I frame fluctuationUnless you can upload a output sample, input file, and settings which generate the output sample, I call bullshit. "Severe I-frame fluctuation" doesn't even make any meaningful sense in constant-quantizer mode. That's like saying that there's "severe speed fluctuation" in a car barreling down an empty highway with the cruise control on.

[jakor]

Hey, buddy,
I'm not blaming neither you neiter x264.
All I mean - is that x264 is now a great huge field - and impossible to turn every piece of grass on it into a diamond. For several reasons I can not upload now any video sample,
all I can say - that for the same condition x264 beats in terms of psnr (0.2 dB) and bitrate (10%) JM software which really sucks.
And all this looks very strange for me...

[Theliel]

Anyway I think DS have reason here. Imagine you said x264 have a critical bug, but you cant show this to anybody.
Maybe its true, but without a sample the bug cant be fixed ;)

[jakor]

I gave a pretty straigtforward explanation of my experiment. You can choose whatever source you have - set const QP and go ahead. If you got x264 ahead of JMVC software - you can prove my words wrong, but until then - my words are not BS.

[Dark Shikari]

I gave a pretty straigtforward explanation of my experiment. You can choose whatever source you have - set const QP and go ahead. If you got x264 ahead of JMVC software - you can prove my words wrong, but until then - my words are not BS.I have tested a number of sources and haven't gotten "severe I-frame fluctuation and several annoying artifacts". Of course, since you still haven't said what quantizer you were testing at, or any other remotely useful information (like the input file), it's obviously impossible to replicate any of what you've done.

Again, you are the one making claims; you must back them up. If you cannot, a moderator will likely close this thread, since it is completely pointless at this point.

[creamyhorror]

yes. x264 has severe I frame fluctuation and several annoying artifacts, while Jm does not have any of those.
Surely you can post a screenshot of the artifacts? Not even a video sample, just a screenshot.

the experiment motivation was to compare how x264 and JM behave themselves in equal conditions. And letting x264 doing 2-nd pass will break the experiment conditions.
Why should equal conditions apply? Using a common bitrate is certainly important, but why do you need to use constant quantizer instead of 1-pass CRF or 2-pass?

[jakor]

actually, my point is not in making claims. My point is a question - what setting would allow x264 to come closer to JMVC encoder.
what I said about I frame fluctuation - was an answer to the question - what's on the output. quant for I frames 32, for B frames 37. pyramidical gop.
I need a constq condition because JMVC works only with constq. So I encoded with JMVC - gave the same quants to the x264 encoder - and the result is strange.
So, DS - artifacts and I frame fluctuations - are no interest of mine. Have you tested x264 with the same quants as JM software (any quant, I'm interested in any test result)? did you watch the same as me - psnr quality is worse with x264 and bitrate is higher ?
Again - I'm not blaming x264. I'm searching where I am wrong because the result is so suspicious.
I also did a very straigtforward test IPP - const q, gop length is 8. and again x264 was far behind JM. DS, have you done such tests with constQ ?

[Chengbin]

There are several problems with your test.

1. Why CQ mode? If JM cannot encode under CRF, then that's JM's fault. You shouldn't handicap x264 to cover JM's flaw. x264 performs far, far better in CRF or 2 pass.

2. PSNR doesn't mean much.

3. Quanitizers are possibly different in x264 than JM.

jakor, you want a little, you have to give a little. If you can't visually explain a visual problem, we're got gonna get far.

[Dark Shikari]

actually, my point is not in making claims. My point is a question - what setting would allow x264 to come closer to JMVC encoder.
what I said about I frame fluctuation - was an answer to the question - what's on the output. quant for I frames 32, for B frames 37. pyramidical gop.
I need a constq condition because JMVC works only with constq. So I encoded with JMVC - gave the same quants to the x264 encoder - and the result is strange.
So, DS - artifacts and I frame fluctuations - are no interest of mine. Have you tested x264 with the same quants as JM software (any quant, I'm interested in any test result)? did you watch the same as me - psnr quality is worse with x264 and bitrate is higher ?
Again - I'm not blaming x264. I'm searching where I am wrong because the result is so suspicious.
I also did a very straigtforward test IPP - const q, gop length is 8. and again x264 was far behind JM. DS, have you done such tests with constQ ?This was the last test done (http://akuvian.org/src/x264/jm_comparison/), back when x264 was vastly inferior to what it is today.

I don't seriously test against JM because it is far too slow for even testing purposes (over 3 years to encode one 1080p movie).

[jakor]

thanks a lot ! very useful information for me

[IgorC]

Where can we get binaries of last JM encoder?

[Sagekilla]

@IgorC: http://iphome.hhi.de/suehring/tml/

What I find particularly shocking is that they have updates that say:


- encoder: Early termination in various distortion computation functions (AT)
- encoder/decoder: Various speed optimizations/restructuring/cleanups (AT)
- encoder/decoder: removal of almost all global variables (AT)
- encoder/decoder: Severe restructuring and merging of Various encoder and decoder functions in lcommon (AT)


Yet it's still horrifically slow. Though if I ever wrote an encoder, it'd probably be just as (if not slower) slow ;)

[Manao]

JM/KTA purposes is not meant to be a practical encoder. Its aim is to be an encoder in which adding new coding tools is easy, and in which the testing of those coding tools is pertinent. Speed only matters when it gets so slow you can't test the new tools anymore. But at the present time, CPU gets faster, faster than JM gets slower, so it's not so much of an issue for them. It is also meant to be a reference, as an encoder and a decoder. And, as an encoder, it's meant to be a high quality reference, PSNR-wise, without rate control.

[jakor]

But at the present time, CPU gets faster, faster than JM gets slower, so it's not so much of an issue for them.
don't agree - in 2002 encoding D1 into AVC took about 3-5 secs on athlon 900, now on core 2 duo the same procedure takes twice more....

[G_M_C]

Hmmm, you what strikes me in this thread ?

The fact that there are claims of artikacts and irregularities, but that jakor doesnt seem to want to post examples / screenshots, or even the simples thing; His x264-commandline and/or JM settings used (does JM also use a commandline).

How can we comment on any claim if the claim is not backed up by a foundation ?

[Manao]

"At the present time". 2002 is a long time ago... I even wonder if at that time, JM wasn't the fastest AVC encoder - for lack of competition.

[skynetman]

I do not understand why in 2009 someone still uses PSNR as the only quality metric for codec comparison.

This pdf has a comparison of different metrics, check the last page to see that SAME psnr has a lot of different QUALITY to human eye:
http://foulard.ece.cornell.edu/dmc27/vsnr/vsnr_a57.pdf

SSIM instead of PSNR would be a good start.
http://www.ece.uwaterloo.ca/~z70wang/research/ssim/

[Manao]

I do not understand why in 2009 someone still uses PSNR as the only quality metric for codec comparisonFor 4 reasons :
- there are no "open" metrics that fix the short comings of PSNR. SSIM is somewhat better, but using it implies an underlying psychovisual model that is far from being universal. And blindly following SSIM will create the same pitfalls PSNR has.
- PSNR retains some very interesting mathematical properties, making it a metric we can optimize for with a theoretical certainty.
- PSNR used correctly is a valid quality metric.
- video engineers are used to handle PSNR

[skynetman]

I can't see any psyco-anything in SSIM:

http://img188.imageshack.us/img188/4140/clipboardhz.jpg

And X264 has already it implemented in options anyway, so no problems at all!
But it seems from this old mailing list post that x264 implementation of ssim is not standard:
http://mailman.videolan.org/pipermail/x264-devel/2006-November/002395.html


SSIM fixes these problems to a significant degree. It is not *the best*
video/image quality metric, but it is the best that can be computed
quickly. SSIM absolute values can be reasonably compared: as a very
rough guide, below 0.7 is barely watchable, 0.8-0.85 has some visible
distortion but ok for most people, and 0.9 and higher are
indistinguishable from the original.

Please note that SSIM in x264 is not calculated the way that the "real"
SSIM algorithm (and papers) say it should be. There is no windowing, no
luma masking, no motion masking, and only a sample of all possible
positions are calculated. However, it is within few percent of the
"real" SSIM in most cases, while being about a hundred times faster to
calculate. "Most cases" here pretty much means any natural footage; it
may be possible to construct synthetic footage so as to defeat the
approximations used, but you are extremely unlikely to ever encounter
such. If you want real SSIM, bug me to release my code for that ;)

[nurbs]

The implied psychovisual model is that a higher score means it looks better which isn't necessarily true. There are psy options in x264 that decrease SSIM, e.g. QPRD, but nonetheless people may prefer them over files encoded without them.

[skynetman]

It probably happens also because these metrics were thought for static images and not video...

[Manao]

There's an underlying psyhcovisual model because SSIM basically weights local distorsion according to local variance. Thus, it implies that high local variance equals less visible distorsion. That is a psychovisual model.

[Dark Shikari]

There are psy options in x264 that decrease SSIM, e.g. QPRDNo, just, no.

[skynetman]

@Dark: to understand if something is better or worse developing x264 do you use your eyes or some metric? It seems from logs that you already test with psnr or ssim.
Has there been some cases where both metrics failed to represent a better result?

[nurbs]

Seems like I didn't pay enough attention when I read the commit log of QPRD.
I overlooked the bold part QPRD is only available with adaptive quantization enabled and generally improves SSIM and visual quality.
and skipped right to the last lineNote that this generally hurts PSNR and SSIM, and so is only on when psy-RD is on.

[Dark Shikari]

@Dark: to understand if something is better or worse developing x264 do you use your eyes or some metric? It seems from logs that you already test with psnr or ssim.
Has there been some cases where both metrics failed to represent a better result?Yes, anything called "psy".

[florinandrei]

You might still have left adaptive quantization on (no-psy does not turn it off), which will greatly lower PSNR.

Additionally, if you meant that you used constant quantizer in x264, constant quantizer disables macroblock-tree, one of x264's most impressive optimizations PSNR-wise.

sigh

x264 still involves a lot of black magic like this. Neither the website nor this forum provide a link to a reasonable, practical, example-based tutorial about the various options and optimizations.

Let's say I want to achieve something like constant quantizer (one-pass encoding, need control over the output quality) but without turning off all those optimizations - what do I need to do?
Last time I tried that, I just took the Blu-Ray preset from MeGUI and selected CQ instead of two-pass and thought that would be pretty good. Obviously, based on what you said, it's not good at all.

[Dark Shikari]

sigh

x264 still involves a lot of black magic like this. Neither the website nor this forum provide a link to a reasonable, practical, example-based tutorial about the various options and optimizations.

Let's say I want to achieve something like constant quantizer (one-pass encoding, need control over the output quality) but without turning off all those optimizations - what do I need to do?
Last time I tried that, I just took the Blu-Ray preset from MeGUI and selected CQ instead of two-pass and thought that would be pretty good. Obviously, based on what you said, it's not good at all.It's only black magic if you're blind and can't read the documentation. The MeGUI wiki has an enormous amount of documentation on practically every option that exists, and even some that don't.

And it isn't hard to figure out that "--tune psnr" will give optimal PSNR (what the OP was going for).

[florinandrei]

It's only black magic if you're blind and can't read the documentation. The MeGUI wiki has an enormous amount of documentation on practically every option that exists, and even some that don't.

Look, I don't want to be confrontational or anything like that, but:

1. Software X's wiki is not the obvious place to learn about configuring software Y.

2. What is it with open source video projects and attitude issues? Before it was ffmpeg, now it seems like x264 is working to take its place in that regard. :mad: If you call people "blind" and what not, nobody wins. A little self-control costs nothing and can work miracles.

Anyway, thank you for the info. I really mean it. I'll go read the wiki now.

[Dark Shikari]

Look, I don't want to be confrontational or anything like that, but:

1. Software X's wiki is not the obvious place to learn about configuring software Y.But you were complaining about MeGUI ;)

(Also, re your complaints, I have a fix coming soon. Be patient.)

[nakTT]

sigh

x264 still involves a lot of black magic like this. Neither the website nor this forum provide a link to a reasonable, practical, example-based tutorial about the various options and optimizations.

Let's say I want to achieve something like constant quantizer (one-pass encoding, need control over the output quality) but without turning off all those optimizations - what do I need to do?
Last time I tried that, I just took the Blu-Ray preset from MeGUI and selected CQ instead of two-pass and thought that would be pretty good. Obviously, based on what you said, it's not good at all.
Come on man. Even a "Certified Newbie" like me know that there is no black magic involve. Even newbie can customize the advance setting provided you read the guide and know what you are doing.

If you don't understand some of the option, you can always ask in this forum and I can assure you that most of the people here are very helpful (even to newbie like me).

[ajp_anton]

sigh

x264 still involves a lot of black magic like this. Neither the website nor this forum provide a link to a reasonable, practical, example-based tutorial about the various options and optimizations.

Let's say I want to achieve something like constant quantizer (one-pass encoding, need control over the output quality) but without turning off all those optimizations - what do I need to do?
Last time I tried that, I just took the Blu-Ray preset from MeGUI and selected CQ instead of two-pass and thought that would be pretty good. Obviously, based on what you said, it's not good at all.He said CQ is bad. Have you ever seen anyone say otherwise?
CRF is what you want for 1-pass.

[creamyhorror]

Let's say I want to achieve something like constant quantizer (one-pass encoding, need control over the output quality) but without turning off all those optimizations - what do I need to do?
And why have you chosen constant quantizer as your 1-pass choice? Why not CRF (constant quality)? Have you even read any of the guides to x264?

Last time I tried that, I just took the Blu-Ray preset from MeGUI and selected CQ instead of two-pass and thought that would be pretty good. Obviously, based on what you said, it's not good at all.
Constant quantizer is no 'good', constant quality (CRF) is 'good'. If you chose the latter then there is no problem. Although you shoudn't be using the Blu-Ray preset unless you're actually creating a stream for eventual Blu-Ray use.

[thewebchat]

Originally Posted by florinandrei View Post
Let's say I want to achieve something like constant quantizer (one-pass encoding, need control over the output quality) but without turning off all those optimizations - what do I need to do?

You kind of can't have MBTree rate control in constant quantizer mode, which implies that the quantizer is, well, constant.