Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 AVC / H.264

Reply
 
Thread Tools Search this Thread Display Modes
Old 9th December 2004, 23:59   #1  |  Link
bond
Registered User
 
Join Date: Nov 2001
Posts: 9,770
wide range of possible reference frames in mpeg-4 avc/h.264?

blame on me if this has been discussed already, but i couldnt find it

i wonder about whether allowing a huge range of possible multiple reference frames would actually help compressibility
i mean in most movies there often is a switch between different scenes (eg if two people talk with each other), where eg first person A is shown talking (scene 1), than the picture switches to person B (scene 2), and than to person A again (scene 3, showing the same camera angle aso as in scene 1...)

couldnt multiple reference frames be very useful if they could be used between different scenes, meaning for example referencing from scene 3 to scene 1?
i realise that this would slow down encoding a lot, but couldnt this be optimised by also taking scenechanges in account (eg only looking at the frames shortly before a scenechange at the end of the scene before the last scene), realising that there are really a lot switches between/to similar scenes in movies? so to say it could be an option created which says: only look at the 3 last frames at the end of the 3 last scenechanges or so...

damn, its hard to describe what i mean, i hope its clear
__________________
Between the weak and the strong one it is the freedom which oppresses and the law that liberates (Jean Jacques Rousseau)
I know, that I know nothing (Socrates)

MPEG-4 ASP FAQ | AVC/H.264 FAQ | AAC FAQ | MP4 FAQ | MP4Menu stores DVD Menus in MP4 (guide)
Ogg Theora | Ogg Vorbis
use WM9 today and get Micro$oft controlling the A/V market tomorrow for free

Last edited by bond; 10th December 2004 at 00:16.
bond is offline   Reply With Quote
Old 10th December 2004, 00:42   #2  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,988
that's a really good idea IMHO

Although most ideas sound good to me, and quite often my own sound great and then don't when I tell other people about them...

Regardless, this would be interesting, but as you said VERY slow... Ateme's AVC really slows down above 1 or 2 of these, no?
__________________
These are all my personal statements, not those of my employer :)
Blue_MiSfit is offline   Reply With Quote
Old 10th December 2004, 04:16   #3  |  Link
PowerMacG4
Oh boy
 
PowerMacG4's Avatar
 
Join Date: Nov 2002
Posts: 126
When the I/P/B decision thinks there is enough change in a frame to warrant an I frame, it could check back to the 2 or so previous P-frames before the previous I-frames and see the deviation from those, and then use the smallest P frame instead. This would KILL seeking, though... I suppose.
PowerMacG4 is offline   Reply With Quote
Old 10th December 2004, 05:59   #4  |  Link
akupenguin
x264 developer
 
akupenguin's Avatar
 
Join Date: Sep 2004
Posts: 2,392
Yes, it's doable. There is a provision in the spec for "long-term references", and I can't imagine what else they intended it for.
It wouldn't even be too slow if you restrict it to scenechanges.
And it would only kill seeking in a 1-pass encode; the second pass would know when to keep long-term refs, and when it's ok to insert seek-points.

So it's on my todo list for x264. But I'm not Ateme, and I do this in my spare time, so no guarantees on when I finish.
akupenguin is offline   Reply With Quote
Old 10th December 2004, 10:24   #5  |  Link
Mug Funky
interlace this!
 
Mug Funky's Avatar
 
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,555
i don't know how much would be gained from this though... i mean, what percentage of a movie's total size is made up of i-frames?

though one thing it'd be great for is shorter term stuff - think of all that news footage you see where there's a relatively boring and low motion scene (some fat politician walking around or shaking hands), that is sent straight to q31 hell because of camera flashes. multiple reference frames would handle these scenes with almost no increase in size, i would think. a similar situation exists with music videos.

also, wouldn't seeking be fine if the referred frames are only i-frames?
i suppose there'd be less increase in efficiency in a lot of cases though.

[edit]

Akupenguin: isn't anybody else working on x264? i'd do it but i can't code and don't have the maths, so i wouldn't be of much use
__________________
sucking the life out of your videos since 2004
Mug Funky is offline   Reply With Quote
Old 10th December 2004, 11:13   #6  |  Link
Valeron
Registered User
 
Join Date: Feb 2004
Location: Canton
Posts: 230
Good idea, indeed.
But maybe not for AVC or any present generation codec.
When the CPU is fast enough, I think features include this complex idea will be add into the video compression tech.

btw, I think "Cell" can handle this but not P4.
Valeron is offline   Reply With Quote
Old 10th December 2004, 19:33   #7  |  Link
snacky
Registered User
 
Join Date: Aug 2004
Posts: 44
Quote:
Originally posted by Mug Funky

though one thing it'd be great for is shorter term stuff - think of all that news footage you see where there's a relatively boring and low motion scene (some fat politician walking around or shaking hands), that is sent straight to q31 hell because of camera flashes. multiple reference frames would handle these scenes with almost no increase in size, i would think.
The use of multiple reference frames DOES help here, and for a camera flash, it may only take two reference frames to drastically reduce the required bitrate.

For an admittedly unrealistic example of how multiple reference frames can help with flashing scenes, see http://ikaruga.co.uk/~snacky/rft/hq-version/

The first file was encoded with only one reference frame. The other ones all have identical video content, yet they're less than 7% the size because they use multiple references. (as you can see in this case, only 2 are needed and allowing more has no effect.) akupenguin tells me that even greater efficiency could be achieved if x264 implemented reference list reordering.
snacky is offline   Reply With Quote
Old 10th December 2004, 20:24   #8  |  Link
akupenguin
x264 developer
 
akupenguin's Avatar
 
Join Date: Sep 2004
Posts: 2,392
Quote:
Originally posted by Mug Funky
i don't know how much would be gained from this though... i mean, what percentage of a movie's total size is made up of i-frames?
I-frames are somewhere around 1% by framecount and 5% by byte size in my encodes. But that's normal movies, without too many camera flashes and the like.

Quote:
though one thing it'd be great for is shorter term stuff - think of all that news footage you see where there's a relatively boring and low motion scene (some fat politician walking around or shaking hands), that is sent straight to q31 hell because of camera flashes. multiple reference frames would handle these scenes with almost no increase in size, i would think. a similar situation exists with music videos.
Yes, multiple reference frames are great for that sort o thing. But x264 uses them suboptimally, due to the fixed IDR cycle. (Set idrint=999 to see how much compression you could potentially gain with the optimal decision algorithm. But that will kill seeking until I actually impelement such an algorithm.)

Quote:
also, wouldn't seeking be fine if the referred frames are only i-frames?
i suppose there'd be less increase in efficiency in a lot of cases though.
a) You probably want to keep the most recent frame in the previous scene, not the first frame.
b) It's not allowed. Decoding must be separate from seeking, and the decoder does not have callbacks into the demuxer. It could work if you design the whole media playback system around a new codec, but not in H.264.

Quote:
Akupenguin: isn't anybody else working on x264? i'd do it but i can't code and don't have the maths, so i wouldn't be of much use
We get the occasional patch, and other devs are sometimes around to apply them, but I think no one else is doing much related to the compression algorithms.
akupenguin is offline   Reply With Quote
Old 10th December 2004, 21:19   #9  |  Link
Gabriel_Bouvigne
L.A.M.E. developer
 
Gabriel_Bouvigne's Avatar
 
Join Date: Dec 2001
Location: Paris - France
Posts: 276
Quote:
The first file was encoded with only one reference frame. The other ones all have identical video content, yet they're less than 7% the size because they use multiple references. (as you can see in this case, only 2 are needed and allowing more has no effect.)
Even with only 1 reference frame, you can still choose your reference frame.
Gabriel_Bouvigne is offline   Reply With Quote
Old 10th December 2004, 21:28   #10  |  Link
bond
Registered User
 
Join Date: Nov 2001
Posts: 9,770
Quote:
Originally posted by akupenguin
Yes, it's doable. There is a provision in the spec for "long-term references", and I can't imagine what else they intended it for.
It wouldn't even be too slow if you restrict it to scenechanges.
And it would only kill seeking in a 1-pass encode; the second pass would know when to keep long-term refs, and when it's ok to insert seek-points.

So it's on my todo list for x264. But I'm not Ateme, and I do this in my spare time, so no guarantees on when I finish.
sounds great

is this part of the normal multiple reference specs (eg in the main profile) or is this part of some special extension (eg frext or so)?
__________________
Between the weak and the strong one it is the freedom which oppresses and the law that liberates (Jean Jacques Rousseau)
I know, that I know nothing (Socrates)

MPEG-4 ASP FAQ | AVC/H.264 FAQ | AAC FAQ | MP4 FAQ | MP4Menu stores DVD Menus in MP4 (guide)
Ogg Theora | Ogg Vorbis
use WM9 today and get Micro$oft controlling the A/V market tomorrow for free
bond is offline   Reply With Quote
Old 11th December 2004, 00:01   #11  |  Link
akupenguin
x264 developer
 
akupenguin's Avatar
 
Join Date: Sep 2004
Posts: 2,392
It's part of Main profile, and libavcodec supports it. (At least, I see code with that intended result. As I don't know of any encoder that uses it, and JM is a pain to deal with, I haven't tested the support.)
akupenguin is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 22:26.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.