CAVLC -> CABAC?

[omion]

I've just started to encode an HDTV movie, and have been playing around with the settings. After a few quick tests, I found that at 720p24 I can't play back CABAC-encoded files, but I can play CAVLC. Is there any tool available which can "recompress" between the two?

My idea is to encode with CAVLC and just accept the slightly larger file until I get my next computer. Then I'll "recompress" the file to CABAC to save space.

So is there anything available which can do this? If not, is it something simple enough to expect it to be available in the future?

[akupenguin]

There is no such tool existing, but it shouldn't be hard to write.

[Chainmax]

Forgive my complete and utter ignorance, but it can be done in a lossless way? How?

[Sharktooth]

Converting CABAC compressed data in CAVLC... something like converting a 7-zip archive into a normal zip.

[virus]

Forgive my complete and utter ingorance, but it can be done in a lossless way? How?
Entropy coding via CABAC or CAVLC is done as last step, after ME and quantization have taken place. Let's say it's a lossless process that just "writes down to the bitstream" the already-computed data (transformed&quantized coefficients, macroblock types, motion vectors, plus additional side-infos). All you need to do is to decode the CAVLC/CABAC stream and reencode it to the other format, while signalling in the slice headers that you've switched entropy coder type. Nothing will be lost in the process, since detail removal (which comes from quantization) need not be performed again. You just redo the very last step in your encoding chain (see what I've written above).

[Chainmax]

So, it would be akin to switching headers on a video stream or demuxing and remuxing into a different container?

[omion]

@Chainmax:
It would be a bit like changing containers, but Sharktooth's zip analogy is better. The data is compressed with either CABAC or CAVLC before it is written to a file.

By the way, I found I can read 720p24 CABAC movies as long as I use the overlay mixer. Since I watch a lot of subbed anime, I always have VMR9 renderless on, but it seems to be a bit too slow in this case.

[j7n]

As far as I understand CABAC considerably increases CPU load during playback. I think that after recompressing into CAVLC my older computer would be able to decode my H.264 files in real-time. Has the situation changed after 7 months and a tool to convert between CABAC & CAVLC become available?

[Limobar]

I would welcome a CABAC-to-CAVLC-converter because that would make more encodes decodable for the Xbox. :)

[codimahi]

As far as I understand CABAC considerably increases CPU load during playback. I think that after recompressing into CAVLC my older computer would be able to decode my H.264 files in real-time. Has the situation changed after 7 months and a tool to convert between CABAC & CAVLC become available?

Conversion from CABAC to CAVLC includes CABAC decoding and CAVLC encoding in which both are very computational intensive. Let's say for a 1hr h.264 CABAC encoded movie, this recompression may take more than 3hrs(depend on resoulution). I don't think so any body would like to do this before seeing a very interesting movie.

[j7n]

However computation intesive or unoptimized that program might be it would really help if purchasing a faster computer (or re-encoding the video) is not an option.

The choice is real simple:
*- spend 3hrs and watch the movie
*- do not watch the movie at all

[check]

@Chainmax:
It would be a bit like changing containers, but Sharktooth's zip analogy is better. The data is compressed with either CABAC or CAVLC before it is written to a file.

By the way, I found I can read 720p24 CABAC movies as long as I use the overlay mixer. Since I watch a lot of subbed anime, I always have VMR9 renderless on, but it seems to be a bit too slow in this case.
Slightly off topic - I'm assuming you use VMR to view subtitles. However, MPC's internal subtitle renderer uses VMR7 (which is faster), and vsfilter runs best in overlay, so why use VMR9?

[akupenguin]

Conversion from CABAC to CAVLC includes CABAC decoding and CAVLC encoding in which both are very computational intensive. Let's say for a 1hr h.264 CABAC encoded movie, this recompression may take more than 3hrs(depend on resoulution). I don't think so any body would like to do this before seeing a very interesting movie.
No. CABAC is about 20-40% of the decoding time (depending on bitrate). CAVLC is less. If you can play the CAVLC version in realtime (otherwise you wouldn't be converting), then the CABAC version can't be more than 30% slower than realtime.
But the CABAC->CAVLC converter doesn't have to do any of the other steps of decoding, it just parses the bitstream and writes it out again. So at the high end (CABAC=40% of decoding, CAVLC version takes 100% CPU), I'd expect the converter to run no slower than 1.5x realtime.

[Inventive Software]

Potentially stupid question, but I'll ask anyway.

Does the CABAC/CAVLC take place in each frame after the ME and bitstream has been written? Or is it done before the bitstream is written? Or is it done completely different? :D

EDIT: Does CAVLC happen anyway even if CABAC is enabled?

[sysKin]

Does the CABAC/CAVLC take place in each frame after the ME and bitstream has been written? Or is it done before the bitstream is written?CABAC/CAVLC is THE two ways of writing a bitstream ;)

Does CAVLC happen anyway even if CABAC is enabled? No, it's one or the other.

[Limobar]

There is no such tool existing, but it shouldn't be hard to write.

I'd expect the converter to run no slower than 1.5x realtime.

Dear Developers,

Please consider writing a tool that converts CABAC to CAVLC. People that use the Xbox (XBMC) would then be able to enjoy so much more content that is available, but not decodable because of the limited processing power of the Xbox and the processing power needed by CABAC.

The combination of a tool that appears not to be that hard to write (according to akupenguin) and the reasonable conversion time needed, in my opinion, makes it worth the effort. I hope you feel the same.

Regards,
Limobar

[pokazene_maslo]

I second this request. It would be a useful tool for users with slower machines.

[iwod]

Does this tools exist after nearly 3 years?

[Shevach]

CABAC is about 20-40% of the decoding time .

In average CABAC performance is 20-40% higher than CAVLC one. But CABAC encoding/decoding can have severe performance peaks (unlike CAVLC where peaks are significantly smaller).
The reason is that CABAC is very hard for optimization due to back-dependency between bins. Choice of context models for some bins depends on the values of previous bins (e.g. the bins of mb_type). So, CABAC can not easely pipelined.

Some encoders deliberately switch to CAVLC mode when performance peak in CABAC encoding/decoding expected.
Thus a smart CABAC->CAVLC converter should switch to CAVLC only if video content might cause performance peak.

[roozhou]

Could someone write a CABAC <-> CAVLC bitstream filter for ffmpeg?

[akupenguin]

In average CABAC performance is 20-40% higher than CAVLC one. But CABAC encoding/decoding can have severe performance peaks
Not the point. That post was in response to someone asking how much total time it would take to re-entropy-code a video, for which peaks don't matter.

[Shevach]

My replay actually concerns the following your statement:
"If you can play the CAVLC version in realtime (otherwise you wouldn't be converting), then the CABAC version can't be more than 30% slower than realtime."
So, the purpose of CABAC->CAVLC converter is to enable real-time playback on low performance decoder.
Let's suppose that the decoder is enough powerful to playback CABAC mode. As I wrote earlier CABAC encoding/decoding might cause performance peaks which in turn can cause the decoder to drop pictures (i.e. cause jitter). To avoid jitter effects resulted by CABAC decoding perfromance peaks it is required a particular tool which would convert performance-problematic pictures into CAVLC mode.
Frankly speaking some encoders use the above trick: if due to performance peaks real-time encoding is problematic then the encoder switches to CAVLC mode.

[Sagekilla]

What do you mean encoding/decoding would cause peaks that would cause the decoder to drop pictures?

All you're doing is decoding a CABAC (or CAVLC) stream, converting it to CAVLC (or CABAC) and saving it to a file. It'll go as fast as it can, and in doing so it won't "drop" a picture.

If a particular frame takes a while to decode, then it just means the encoding stage won't be run until it receives a frame. But that doesn't mean the frame will be dropped at all.

If you mean the converter selectively converts frames that tend to take more decoding time than others, then yes this would be useful. I didn't know hybrid CABAC/CAVLC streams were possible though.

You'd need to write a tool to measure the peak bitrate a given computer can handle, and then build in an extra 10 - 20% CPU usage since it's not guaranteed the decoder will get 100% CPU usage.

[Shevach]

If you mean the converter selectively converts frames that tend to take more decoding time than others, then yes this would be useful.

This is my point.
For a CABAC coded stream the converter should estimate the decoding time per picture and if the estimated value exceeds a pre-defined threshold then the converter should re-encode the picture in CAVLC mode (even if the switch to CAVLC mode might result more bit-rate).
Otherwise if a decoder can't decode a frame during
1/frame_rate seconds it might make the decoder to drop the picture.

[EmuAGR]

I wonder if anyone is working on a converter... It would be useful for old computers.

[Comatose]

It would be useful indeed, and not just for old computers.

[neuron2]

It would be useful indeed, and not just for old computers. How so?

[EmuAGR]

Maybe also hardware media players?

[benwaggoner]

Maybe also hardware media players?
Hardware players either don't support CABAC at all (Baseline and AppleTV), or have the hardware horsepower to handle CABAC for all supported profiles/levels. This is only something that would matter for software playback.

Of course, CABAC is only one part of performance on low-end machines. The bigger issue is really too high peak bitrates (which make CABAC and the rest of the decode slower).

For higher bitrate content, it's useful to think about the lowest bitrate you want to support, and constrain the peak buffer to something that works on those devices. There's a whole lot of encodes out there with effectively unconstrained peaks, which seems like overkill for me. For most film content at a reasonable bitrate, a peak 1.5x the ABR really provides most of the value of VBR.

[Dark Shikari]

For most film content at a reasonable bitrate, a peak 1.5x the ABR really provides most of the value of VBR.Someone hasn't tried to encode the HBO logo lately ;)

[benwaggoner]

Someone hasn't tried to encode the HBO logo lately ;)
That's an excellent example of what would be excluded from "most" :). That definitily gave us some angina doing the Sopranos and Band of Brothers HD DVDs with very early versions of PEP.

That said, encoding content without peak constraints that included a meaningful percentage of that kind of pathological content could lead to visible quality reductions in the show itself. It's sometimes better to let some reallly hard but nonessential parts wind up looking not so great in order to preserve enough bits for the easier to encode and more.