Comparison of H.264 (which uses only I-frames) and Motion JPEG

[Redhat_doom]

Hi,

I want to compare the coding efficiency of H.264 (which uses only I-frames) with Motion JPEG. To do so I used ffmpeg and I obtained the attached graph for a CIF (352x288) "Foreman" video sequence (300 frames, 30fps). As shown in this graph, for each curve, the average PSNR (for luma component) is saturated as bitrate increases. But what is the problem?

Of course, such high bitrates overkill a CIF video and so I decided to use higher resolution videos as well but the same phenomena was occured (of course at higher bitrates say 24K for a 704x576 video). I expect that at higher bitrates MJPEG must hit H.264. But it seems that MJPEG can never beat the performance of H.264 (with only I-frames). But am I right?

More details:
I used the following commands for converting a YUV sequence to MJPEG:

# ffmpeg -r 25 -s 352x288 -psnr -i foreman.yuv -b 6000k foreman.mjpeg
using this instruction, I get the average PSNR Y = 42.94. But if I increase the bitrate to say 7000k,8000k,10000k or even higher, the average PSNR Y remains the same
and it seems that the average PSNR is saturated at 42.94!!

I tried to set the maxrate and bufsize using the following command:
# ffmpeg -r 25 -s 352x288 -psnr -i foreman.yuv -maxrate 8000k -b 8000k -bufsize 8000k foreman.mjpeg
but the problem still remains! But what is the problem? Thanks!

[LoRd_MuldeR]

Well, JPEG has a fixed quality scale between 1 and 100. But even 100 is still not lossless. JPEG can't do lossless at all!

So if you throw enough bitrate at a MJPEG encoder, it will hit the maximum quality of 100 sooner or later. At that point (M)JPEG cannot deliver better quality, no matter what bitrate you offer theoretically.

In contrast H.264 at low quantizers can deliver very high quality (probably better than JPEG at 100). x264 can even deliver "true" lossless compression, if you choose a quantizer of "0".

Nevertheless MJPEG at a quality of 100 should be far enough to deliver a quality that you can't distinguish from the original ;)

[multimediaman]

According to mencoder manual you can encode lossless jpeg video:
vcodec=<value>
[...]
ljpeg
Lossless JPEG
mjpeg
Motion JPEG

But filesizes are huge with ljpeg.

[LoRd_MuldeR]

Also LJPEG isn't standard JPEG compression, I guess. Maybe it would be more interesting to test LJPEG against lossless H.264 then.

Side note: If you want to compare the PSNR values of x264 to another encoder, you must specify "--psy-rd 0:0" to get meaningful results.

[Redhat_doom]

Well, JPEG has a fixed quality scale between 1 and 100. But even 100 is still not lossless. JPEG can't do lossless at all!

So if you throw enough bitrate at a MJPEG encoder, it will hit the maximum quality of 100 sooner or later. At that point (M)JPEG cannot deliver better quality, no matter what bitrate you offer theoretically.

In contrast H.264 at low quantizers can deliver very high quality (probably better than JPEG at 100). x264 can even deliver "true" lossless compression, if you choose a quantizer of "0".

Nevertheless MJPEG at a quality of 100 should be far enough to deliver a quality that you can't distinguish from the original ;)

Thanks! But as you mentioned, MJPEG must hit H.264 sooner or later but based on that graph, it will never happen because of that saturation! Actually, I can't understand why those saturations were happened!

[Redhat_doom]

According to mencoder manual you can encode lossless jpeg video:


But filesizes are huge with ljpeg.

I used this command to use LJPEG:

ffmpeg -r 25 -s 352x288 -psnr -i forman.yuv -b 6000k -vcodec ljpeg foreman.avi

but it returned this error: [ljpeg @ 0x3f400] colorspace not supported in jpeg

do you know what's the problem? ;)

[LoRd_MuldeR]

Thanks! But as you mentioned, MJPEG must hit H.264 sooner or later

Why? H.264 uses the available bitrate MUCH more efficiently than MJPEG can. So at any bitrate x264 should beat MJPEG quality wise (and PSNR wise).

Furthermore MJPEG will reach the JPEG quality level 100/100 at some point. At that point PSNR can't improve, not even by adding more bitrate...

[Redhat_doom]

Why? H.264 uses the available bitrate MUCH more efficiently than MJPEG can. So at any bitrate x264 should beat MJPEG quality wise (and PSNR wise).

Furthermore MJPEG will reach the JPEG quality level 100/100 at some point. At that point PSNR can't improve, not even by adding more bitrate...

Oh, I got that ;) So, if we throw enough bitrate, MJPEG will hit JPEG with a quality factor of 100 but since JPEG is not lossless, the efficiency of it is always lower than H.264 (which can be lossless). However, I think if we use JPEG2000 then we can expect that at a specific point, it can beat H.264 because JPEG2000 can be lossless. Am I right? If so, do you know any way to convert a YUV video sequence to Motion JPEG2000? (MJPEG2000 :D)

[LoRd_MuldeR]

Oh, I got that ;) So, if we throw enough bitrate, MJPEG will hit JPEG with a quality factor of 100 but since JPEG is not lossless, the efficiency of it is always lower than H.264 (which can be lossless).

The problem is not that JPEG is inherently lossy, because H.264 is lossy too (at least in that way we use it usually).

I guess the problem is more that MJPEG simply can't produce a better quality than JPEG at quality 100. So if you are encoding to MJEPG with a quality factor of 100, then this is the highest possible quality that MJPEG can deliver and also the highest possible bitrate that MJPEG can consume. This is the point where PSNR can't become any better with (M)JPEG. But with H.264 you will be able to get better PSNR values than (M)JPEG at 100. You only need to lower the H.264 quantizer enough (provide enough bitrate). I think this is what we see in your graph...

However, I think if we use JPEG2000 then we can expect that at a specific point, it can beat H.264

This is highly questionable. H.264 compresses more efficient than JPEG2000, I think. So I assume that x264 will beat JPEG2000 quality wise at any bitrate.

Or in other words: As long as you give the same bitrate to both, x264 and the MJPEG2000 encoder, then the x264 encode will look better than the MJPEG2000 encode.

Anyway, in contrast to JPEG, JPEG2000 shouldn't have that barrier where you can't improve the quality any further by adding more bitrate...

[Audionut]

However, I think if we use JPEG2000 then we can expect that at a specific point, it can beat H.264 because JPEG2000 can be lossless.

A lossless coder can only beat another in size and speed. Not quality.

[LoRd_MuldeR]

A lossless coder can only beat another in size and speed. Not quality.

...but in "quality at a given bitrate" or "quality per bit". And I'm sure x264 will win that competition against JPEG2000 :)

[Redhat_doom]

1- I am agree with you that the compression performance of H.264 is better than JPEG2000. But I want to see this issue in practice. Actually, I want to generate a similar graph for comparing H.264 and Motion JPEG2000. But how to convert a YUV sequence to MJPEG2000?

2-
Anyway, JPEG2000 shouldn't have that barrier where you can't improve the quality any further by adding more bitrate...

So, if there is no barrier for JPEG2000, then it means that we don't have such a saturation problem with it and its curve goes up furthuremore BUT as you can see from that graph, the corresponding curve for H.264 is saturated. Therfore, it is possible that JPEG2000 can approach H.264.

[LoRd_MuldeR]

So, if there is no barrier for JPEG2000, then it means that we don't have such a saturation problem with it and its curve goes up furthuremore BUT as you can see from that graph, the corresponding curve for H.264 is saturated. Therfore, it is possible that JPEG2000 can approach H.264.

There should be no reason for H.264 to "saturate" unless you reach the point where the encode is identical to the source.

Please note that "Typical values for the PSNR in lossy image and video compression are between 30 and 50 dB, where higher is better."

So in your graph x264 reached virtually perfect quality at ~5000 kbps. At that point no further improvement is possible.

(except for "true" lossless compression, of course)

[Stevie]

Just for the understanding... H264 in I-Frame mode means, every picture is saved separately as in MJPEG?
Is the CPU consumption higher with H264? I need a codec that produces little CPU usage but still offers
I-Frame encoding.


Cheers,

Stevie


P.S. Wow, my first post in 7 years

[LoRd_MuldeR]

Just for the understanding... H264 in I-Frame mode means, every picture is saved separately as in MJPEG?

I-Frame only means that each frame in the H.264 stream is encoded without reference to other frames (that is: no inter frame compression).

Hence you take a random frame from the stream and decode it without having decode any reference frames first. This is similar to MJPEG, yes.

About speed: There are many options you can tweak in x264 to get faster compression. But the encoding efficiency will suffer, of course...

[Stevie]

Oh wait, I didn't mean the encoding speed!
I meant decoding/playback CPU usage :)
I compose music to picture and would like to have
my resources left for audio processing.

[LoRd_MuldeR]

The CPU usage for H.264 decoding highly depends on how optimized your H.264 decoder is ;)

And if you use a GPU-based H.264 decoder, such as CoreAVC with CUDA enabled, then the CPU usage will be almost zero for H.264 playback.

Last but not least, using CAVLC instead of CABAC will significantly lower the CPU usage for H.264 playback, but it will also significantly hurt the coding efficiency!

But if you care about coding efficiency, then you wouldn't use I-Frame only encoding :D

[Stevie]

Exactly, encoding efficiency isn't an issue.
You have to imagine:
I got my video in my sequencer and scrub a lot back and forth, when composing.
All the video has to be is fast and responsive! This isn't granted when using B or P-frames.
I need every single picture so that the speed of the video doesn't slow down due to decoding.
Well, CoreAVC sounds like a good idea, indeed. I will try that!
Does CUDA also work with ATI cards?

[LoRd_MuldeR]

Exactly, encoding efficiency isn't an issue.

Then why not simply use something like HuffYUV ???

Does CUDA also work with ATI cards?

Nope. CUDA is NVIDIA-only technology. NVIDIA's CUDA and ATI's FireStream are not compatible at all.

And since ATI missed to promote their own GPGPU solution properly (in contrast to NVIDIA), everybody is using CUDA right now :D

But there is a great chance that OpenCL, which is not vendor-specific, will replace both, CUDA and FireStream, soon...

[Stevie]

Now THAT (HuffYUV) would really take too much space :)

[Comatose]

Or in other words: As long as you give the same bitrate to both, x264 and the MJPEG2000 encoder, then the x264 encode will look better than the MJPEG2000 encode.
But what he's saying is that logically, as more bitrate is given to JPEG, it should begin approaching H.264, but the quality remains static.

[LoRd_MuldeR]

But what he's saying is that logically, as more bitrate is given to JPEG, it should begin approaching H.264, but the quality remains static.

For a fair and useful comparision you must give both encoders the very same baitrate. And then x264 will win against (M)JPEG quality-wise at any given bitrate.

Also there is the point where JPEG reaches a compression of 100/100 and cannot look any better, no matter how much bitrate you throw at it.

The same applies to JPEG2000, only that JPEG2000 will not be that far behind x264. Also JPEG2000 shouldn't have that bitarte limitation where it cannot look any better...

[ajp_anton]

Where are those wasted bits going? Shouldn't it be unable to use the extra bits after reaching maximum quality?

[Zanthra]

Keep in mind that h.264 is a very new codec, designed for a huge variety of uses, and JPEG is very old now, and was designed to lossily compress data by very large amounts. JPEG is old, weak, and provides a fraction the compression of an IFrame in H.264.

[10L23r]

Where are those wasted bits going? Shouldn't it be unable to use the extra bits after reaching maximum quality?

well to start off, there's chroma subsampling

[LoRd_MuldeR]

well to start off, there's chroma subsampling

That's not the point. Subsampling happens in x264 the same way as in MJPEG. The point is: (M)JPEG has a fixed "quality" scale from 1 to 100.

100 (which still is far from being lossless) is the maximum quality that JPEG can reach and also the maximum bitrate that (M)JPEG can consume.
At that point it is impossible for MJPEG to consume more bits for better quality. Hence you won't be able to raise the bitrate any further.

So x264 will not only beat MJPEG at any given bitrate, also the maximum possible quality (at unrestricted bitrate) of x264 is better!

Where are those wasted bits going? Shouldn't it be unable to use the extra bits after reaching maximum quality?

Yes. I assume that the encoder will simply miss the desired target bitrate if you specify a target bitrate that is higher than what (M)JPEG does consume at quality 100/100.

Also the MJPEG encoders I'm used to don't have an ABR or CBR mode. Instead you specify the quality level in the range 1...100 and that's it. Resulting bitrate not known in advance.

[roozhou]

The same applies to JPEG2000, only that JPEG2000 will not be that far behind x264. Also JPEG2000 shouldn't have that bitarte limitation where it cannot look any better...

Actually in my test JPEG2000 worse than SNOW's I-frame.

[*.mp4 guy]

(M)JPEG has a fixed "quality" scale from 1 to 100.

Mjpeg can use any quantization matrix that you specify, atleast if it conforms to the jpeg specifications. Though I am not aware of any jpeg coder that allows you to set an arbitrary cqm, it is allowable by the jpeg standard, the precise degree to which this extends the achievable quality scale is beyond my knowledge. That said, there is, atleast theoretically, more flexibility within the jpeg standard and therefore derivative standards such as mjpeg, then most people realise.

I have not read through this enire thread, but it apears that the debate is centered on if it is possible for mjpeg to ever achieve the same or higher quality then an equivelent bitrate I frame only AVC stream. It isn't possible. There are a lot of reasons, but they are all incredibly self evident if you have any understanding of the standards in question.

[chornobyl]

I've allways wondering, why with gradual codec improvement, nobody tries to port all those fancy stuff(aq, psy, trellis) back to jpeg, while still not leaving standarts boundaries and preserving compatibility.

[LoRd_MuldeR]

I've allways wondering, why with gradual codec improvement, nobody tries to port all those fancy stuff(aq, psy, trellis) back to jpeg, while still not leaving standarts boundaries and preserving compatibility.

Because JPEG is an old and widely adopted standard. You can't simply hack in new features into JPEG, because this would break compatibility to millions of existing applications ;)

Instead you'd need to define a completely new format/standard as a successor of JPEG. And this has happened already. Just think of PNG and JPEG2000.

Anyway, people are still using JPEG, mainly because it's such a common format (any web-browser and any image-viewer can decode it) and because it's sufficient for most tasks.

[akupenguin]

Trellis and psy are encoder-side and require no support from the standard. Though jpeg doesn't involve many decisions which would benefit from psy.

[LoRd_MuldeR]

Trellis and psy are encoder-side and require no support from the standard. Though jpeg doesn't involve many decisions which would benefit from psy.

But AQ requires support from the standard to assign a separate quantizer to each block. Don't think JPEG can do that.