slices in x264 for IP streaming

[jonatan]

Hi, I would like to use x264 fo streaming over IP and I will have a packet size limit of 1400 bytes. Due to resolution, framerate and bitrate restrictions, each frame will consist of around 10 packets. However, the connection is not perfect so there can be packet-losses.

The thing is, if one packet is lost (and each frame is coded in a single slice) the rest of that frame will be lost, since the last packets cannot be decoded.

What I would like to have is the functionality of setting a max size for a slice (in my case 1400) and that the encoder starts on a new slice when the limit is reached. If each packet contains one slice a packet-loss will only affect that slice and the rest o the frame can be decoded properly. Note! This is not about adding slices for the purpose of multi-threading!

My questions are:

Has anything like this been done in x264?
Do anyone have an idea of how much work is required and can anyone give me hints on where in the source code the changes should be made (and how)?
Or is it maybe something that one of the developers could implement without to much effort?
Last question, if this functionality is added, will it be adopted to the source code?

Thank you in advance, Jonatan

[Gabriel_Bouvigne]

I think that there is a patch which was sent on the mailing list to do this.

Now, regarding the error-resilience of such scheme:
Let's say you have 4 slices (abcd) per frame, and on frame 0 we are loosing slice b. What will happen for slices abc of frame 1 which are referencing b0?
You won't be able to decode them properly, so you in fact lost ability to properly decode b0, a1, b1, c1. And now, what about frame 2 which is referencing frame 1?

All this to say that if you want slices for error resilience, you'd better:
*using I frames only or
*be able to restrict mv to slice boundaries or
*have a smart decoder and use constraint_intra_pred flag

[Dark Shikari]

Originally Posted by Gabriel_Bouvigne

I think that there is a patch which was sent on the mailing list to do this.

Now, regarding the error-resilience of such scheme:
Let's say you have 4 slices (abcd) per frame, and on frame 0 we are loosing slice b. What will happen for slices abc of frame 1 which are referencing b0?
You won't be able to decode them properly, so you in fact lost ability to properly decode b0, a1, b1, c1. And now, what about frame 2 which is referencing frame 1?

All this to say that if you want slices for error resilience, you'd better:
*using I frames only or
*be able to restrict mv to slice boundaries or
*have a smart decoder and use constraint_intra_pred flag

And given the absurd cost in bits of those options, you're better off just encoding normally and using FEC.

[jonatan]

Thing is, I will have low-delay requirements so resending is not an option neither is periodic Intra frames since it will add delay and cost to much or reduce quality, and the characteristics of the network are not known from start and can change in time, so just using FEC is not reliable enough (or will cost to much). However, feedback is possible so the encoder will know (like a second later) that a packet ha been lost and can then repair it with an Intra picture or constrained Intra macroblocks. Nevertheless I do want the quality to be as good as possible in the meantime, and I do want the repair to be as cheap as possible and that's why I need slices.

/ Jonatan

[Dark Shikari]

Originally Posted by jonatan

]Nevertheless I do want the quality to be as good as possible in the meantime, and I do want the repair to be as cheap as possible and that's why I need slices.

Slices don't change the fact that if you lose a packet, every single frame until the next IDR frame will be corrupted.

[jonatan]

Slices don't change the fact that if you lose a packet, every single frame until the next IDR frame will be corrupted.

No, but there is a huge difference if each frame looks crappy allover, or if it's just 10% of the picture area that looks bad.

[Dark Shikari]

Originally Posted by jonatan

No, but there is a huge difference if each frame looks crappy allover, or if it's just 10% of the picture area that looks bad.

But again, unless you restrict motion vectors to within slice boundaries, which significantly hurts compression, the picture will progressively degrade from 10% destroyed to 100% destroyed quite quickly.

[jonatan]

But again, unless you restrict motion vectors to within slice boundaries, which significantly hurts compression, the picture will progressively degrade from 10% destroyed to 100% destroyed quite quickly.

That depends on the motion in the sequence and anyway, a progressive degradation is way much better than an instant destruction.

[Gabriel_Bouvigne]

In some cases, slices with error feedback and restricted MV make sense (ex: videoconf over low bandwidth channel).
Restricted MV will hurt compression, but likely less than in "natural" sequences.

[jonatan]

I think that there is a patch which was sent on the mailing list to do this.

How can I find this patch?

[benwaggoner]

Originally Posted by Gabriel_Bouvigne

In some cases, slices with error feedback and restricted MV make sense (ex: videoconf over low bandwidth channel).
Restricted MV will hurt compression, but likely less than in "natural" sequences.

In theory yes. But for most content, there isn't that much vertical motion. I'm always surprised by how little difference it makes in practice with typical content.

Now, a pogo-stick championship shot hand-held...