H.264 scaled decoding?

Sidi · 24th June 2009, 09:38

Does anybody know if is it possible to improve H.264 decoding performance in simmilar manner than JPEG? I mean scaling during decode (for example to 1/8), which speeds up at least two times on IJL. Is there any decoder that supports anything like this?
Or is there any other mean of making this process faster (other than I-Frame decoding only), even with lower quality? Lets say it is Baseline H.264.

Adub · 25th June 2009, 21:09

One way that I know of is to turn in-loop deblocking off.

LoRd_MuldeR · 25th June 2009, 21:21

It should be noted that turning off in-loop deblocking for a H.264 stream that was encoded with in-loop deblocking enabled, will not only cause more visible blocks. It will cause corrupted output! That's because deblocking is part of H.264 encoding process. P-Frames and B-Frames reference deblocked frames. Skipping the deblocking causes "wrong" references frames, so the error sums up from frame to frame...

Shevach · 30th June 2009, 15:06

More idea.

You can discard pictures which are not used for reference (provided that an input stream contains non-reference pictures).

LoRd_MuldeR · 30th June 2009, 15:46

Quote:

Originally Posted by Shevach

You can discard pictures which are not used for reference (provided that an input stream contains non-reference pictures).

Why should a stream contain pictures that are neither referenced nor displayed?

I think any picture, referenced or not, will be displayed at least. So you can't simply "discard" any pictures.

That would result in missing pictures and produce jumpy output!

What you can do: Skip the deblocking part on non-referenced pictures (all b-frames, unless b-pyramid is used).

Shevach · 1st July 2009, 07:40

Quote:

Originally Posted by LoRd_MuldeR

Why should a stream contain pictures that are neither referenced nor displayed?

Perhaps I was mis-understood.
I suggested to discard pictures which are not used for reference despite these pictures are displayed. Thus the decoding process can be accelerated.
I think this is a popular action implemented in many decoders to cope with performance peaks.

Frankly speaking a stream can contain dangling fields which some decoders do not display.

LoRd_MuldeR · 1st July 2009, 15:18

So you want to "discard" pictures that are not used for reference. But you still want to display these pictures

How do you want to display a picture that has been discarded ???

As said in my previous post: I believe the only the thing you can do is skipping the deblocking on non-referenced frames, but not discard those frames entirely!

Furthermore I think skipping the deblocking on non-referenced frames won't give that much speed-up. The only frames that aren't referenced are b-frames.

And even that is only the case with b-pyramid disabled. With b-pyramid b-frames may be referenced by other b-frames as well.

Last but not least deblocking is not the most compute-intensive task. CABAC decoding is, given that CABAC is used...

nm · 1st July 2009, 15:42

Quote:

Originally Posted by LoRd_MuldeR

So you want to "discard" pictures that are not used for reference. But you still want to display these pictures

How do you want to display a picture that has been discarded ???

No, he suggested skipping those frames completely, i.e. dropping them within the decoder. As you said, this causes jumpiness, but one could argue that it's still "improved decoding performance at lower quality".

Edit: By the way, libavcodec already does have this option. Try -lavdopts skipframe=nonref with MPlayer.

Quote:

Last but not least deblocking is not the most compute-intensive task. CABAC decoding is, given that CABAC is used...

The relative time required for these tasks depends a lot on the frame resolution and bitrate though. I'd guess that at low bitrates deblocking would be harder than CABAC with current software implementations.

LoRd_MuldeR · 1st July 2009, 15:59

Quote:

Originally Posted by nm

No, he suggested skipping those frames completely, i.e. dropping them within the decoder. As you said, this causes jumpiness, but one could argue that it's still "improved decoding performance at lower quality".

That's true. Some dropped frame here and there may still be better than a complete desync.

Quote:

Originally Posted by nm

The relative time required for these tasks depends a lot on the frame resolution and bitrate though. I'd guess that at low bitrates deblocking would be harder than CABAC with current software implementations.

Yeah, but I think decoding performance hardly becomes a problem for SD content. More for HD content (720p or 1080p).

And then the bitrate probably is high enough to make CABAC the performance bottleneck...

Shevach · 2nd July 2009, 11:03

Nm is right. I suggested to discard non-reference pictures and consequently don't display them.
Before discarding of pictures a decoder should care the very particular case when in a single complementary field pair one field is used for reference while another not. Indeed, in such case how the frame can be displayed if one of the two fields is absent.

As usual a removal of non-reference frames causes jerkiness artifacts. Although in case of significant motion or a scene cut the removal several frames is not noticeable.

24th June 2009, 09:38	#1 \| Link
Sidi Registered User Join Date: Jun 2009 Posts: 1	H.264 scaled decoding? Does anybody know if is it possible to improve H.264 decoding performance in simmilar manner than JPEG? I mean scaling during decode (for example to 1/8), which speeds up at least two times on IJL. Is there any decoder that supports anything like this? Or is there any other mean of making this process faster (other than I-Frame decoding only), even with lower quality? Lets say it is Baseline H.264.

25th June 2009, 21:09	#2 \| Link
Adub Fighting spam with a fish Join Date: Sep 2005 Posts: 2,714	One way that I know of is to turn in-loop deblocking off. __________________ FAQs:Bond's AVC/H.264 FAQ Site:Adubvideo Zsmooth - Cross-platform smoothing for Vapoursynth

25th June 2009, 21:21	#3 \| Link
LoRd_MuldeR Software Developer Join Date: Jun 2005 Location: Last House on Slunk Street Posts: 13,251	It should be noted that turning off in-loop deblocking for a H.264 stream that was encoded with in-loop deblocking enabled, will not only cause more visible blocks. It will cause corrupted output! That's because deblocking is part of H.264 encoding process. P-Frames and B-Frames reference deblocked frames. Skipping the deblocking causes "wrong" references frames, so the error sums up from frame to frame... __________________ Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊ Last edited by LoRd_MuldeR; 25th June 2009 at 21:27.

1st July 2009, 15:18	#7 \| Link
LoRd_MuldeR Software Developer Join Date: Jun 2005 Location: Last House on Slunk Street Posts: 13,251	So you want to "discard" pictures that are not used for reference. But you still want to display these pictures How do you want to display a picture that has been discarded ??? As said in my previous post: I believe the only the thing you can do is skipping the deblocking on non-referenced frames, but not discard those frames entirely! Furthermore I think skipping the deblocking on non-referenced frames won't give that much speed-up. The only frames that aren't referenced are b-frames. And even that is only the case with b-pyramid disabled. With b-pyramid b-frames may be referenced by other b-frames as well. Last but not least deblocking is not the most compute-intensive task. CABAC decoding is, given that CABAC is used... __________________ Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊ Last edited by LoRd_MuldeR; 1st July 2009 at 15:27.

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

30th June 2009, 15:06	#4 \| Link
Shevach Video compressionist Join Date: Jun 2009 Location: Israel Posts: 126	More idea. You can discard pictures which are not used for reference (provided that an input stream contains non-reference pictures).

2nd July 2009, 11:03	#10 \| Link
Shevach Video compressionist Join Date: Jun 2009 Location: Israel Posts: 126	Nm is right. I suggested to discard non-reference pictures and consequently don't display them. Before discarding of pictures a decoder should care the very particular case when in a single complementary field pair one field is used for reference while another not. Indeed, in such case how the frame can be displayed if one of the two fields is absent. As usual a removal of non-reference frames causes jerkiness artifacts. Although in case of significant motion or a scene cut the removal several frames is not noticeable.