H.264 status and definition

[bobololo]

Originally posted by Sagittaire
I think it's possible: for exemple MPEG4-ffvfw use inloop ...

Unfortunately, H.264 and mpeg-4 (part 2) are definitely very different and there is no mean to exploit h.264 tools in part 2 encoder.

-- bobololo

[bond]

Originally posted by virus
I have a question for those of you who know the specs (I've got the draft for H.264 lying somewhere in my HD but haven't read it yet).have a look at my mpeg-4 info sticky

Originally posted by bobololo
Unfortunately, H.264 and mpeg-4 (part 2) are definitely very different and there is no mean to exploit h.264 tools in part 2 encoder.hm afaik xvids vhq mode is a backported rate distortion from avc

[virus]

Originally posted by bond
have a look at my mpeg-4 info sticky
I did. But, besides the graph with the differences between the standards, I was unable to find a clear-cut answer to that specific question... sorry, I probably overlooked something. But I'm inclined to trust bobololo on that: you simply cannot make a "transitional" codec it seems.


hm afaik xvids vhq mode is a backported rate distortion from avc
mmhhh... I'm a bit doubtful on that :)
Not really sure that H.264 RD-Opt is _exactly_ the same as XviD VHQ.
The underlying concept may be the same, but the actual implementation may differ I think. Probably sysKin can answer much better on that.

[Manao]

hm afaik xvids vhq mode is a backported rate distortion from avcPerhaps, but nobody forbids you from doing so, because you do not change the decoding process by trying to choose the best MV / quantizer at encoding.

[bond]

Originally posted by virus
I did. But, besides the graph with the differences between the standardsi mean you asked for a draft of the h.264 standard, you will find a link to it in my sticky

[virus]

Originally posted by bond
i mean you asked for a draft of the h.264 standard, you will find a link to it in my sticky
sorry for not making that clearer before, but I said "I've got the draft (already), but haven't read it yet" (lazyness :rolleyes:, it's quite big...). Thanks for the help anyway.

[bobololo]

Originally posted by bond
have a look at my mpeg-4 info sticky

hm afaik xvids vhq mode is a backported rate distortion from avc

The Rate-Distorsion Optimisation (RDO) isn't a coding tool defined by the spec (like bframe, inloop deblocking filter, cabac, etc.). It's a process that helps the encoder to take the best decision from a coding efficiency point of view.

For instance, given a macroblock from a picture and a given quantizer step, the encoder can code it as INTRA or INTER. The RDO process evaluates that INTRA coding would require let's say 100 bits and involves a distorsion of 35 dB. Beside, when coded as INTER, the required bits is 20 and distorsion is 34.5 dB. In such case, the RDO process tells the encoder that the coding cost of INTRA is much higher than INTER's and therefore INTER coding should be choosen even its distorsion is higher.

As you can see, this process doesn't affect really the tools provided by the specs. It can be used by any encoders as soon as it need to take optimal decisions. RDO is actually is also used in high end MPEG-2 broadcast encoders. Their performance is much higher than "traditional" mpeg-2 encoders available on PC.

-- bobololo

[aketon]

What is the "Max. number of reference frames" in ffdshow that you are usually using???

[unmei]

i used 10 for some tests and 15 for others. If you enter 16 or more the box turns red so i guess you shouldn't do that.
Also i could imagine higher numbers take more processor power or memory to decode, so if you use high values and get playback problems it might be worth trying with lower numbers.

[bond]

Originally posted by aketon
What is the "Max. number of reference frames" in ffdshow that you are usually using???i assume this value defines the number of previous frames used for inter motion search, in the reference h.264 encoder its set to 5 by default (and it seems to only allow a range from 1-5)
afaik mpeg-4 asp only allows max. 1 reference, so this feature is a nice example where avc simply expands the possibilites of the tools already existing in mpeg-4 asp

[unmei]

afaik mpeg-4 asp only allows max. 1 reference

you should make that 2 in the case of B-frames i think.

[bond]

Originally posted by unmei
you should make that 2 in the case of B-frames i think. no, this has nothing to do with b-frames!

[unmei]

hmm my thought was, it looks at the past and the future neighbor P- or I- frame, but maybe i'm really completely off track ..then can you clarify what it actually is.

[virus]

Originally posted by bond
afaik mpeg-4 asp only allows max. 1 reference (not sure tough) yeah, and 2 for B-frames of course ;)
I fail to see how using 10 reference frames may help in the coding of a macroblock, though. Because 10 references = 10 motion vectors, no? That's a really big overhead, I think.
Nonetheless 2 o 3 seems (to me) reasonable, as a method to reduce the influence of noise in the prediction step.

Someone smarter than me can explain that stuff? :D

[Tommy Carrot]

I think it means that it searches for the most similar frame in the previous x frames, and chooses it as the reference frame. This ability is particularly useful at flashing, where the previous frame is highly different, but there is a frame before that which is quite similar, so the motion information can be much smaller.

[aketon]

So, nobody is sure about what "Max. number of reference frames" really is!!!:rolleyes:
Although, I believe that 10 is a very good choice!:)

[virus]

Originally posted by Tommy Carrot
I think it means that it searches for the most similar frame in the previous x frames, and chooses it as the reference frame.Well, in that case the "number of reference frames" will be 1, freely chosen and not necessarily the previous I/P frame, but still only 1. Wouldn't be better to search for your reference block in many different frames (that is, doing what you said but on a per-block basis)?

Anyway, maybe I'm mixing up the concepts of "number of frames from which you can draw reference macroblocks" and "number of macroblocks blended to form the prediction (and thus needing a MV each)" :rolleyes:

[Tommy Carrot]

Originally posted by virus
Well, in that case the "number of reference frames" will be 1, freely chosen and not necessarily the previous I/P frame, but still only 1. Wouldn't be better to search for your reference block in many different frames (that is, doing what you said but on a per-block basis)?

Anyway, maybe I'm mixing up the concepts of "number of frames from which you can draw reference macroblocks" and "number of macroblocks blended to form the prediction (and thus needing a MV each)" :rolleyes:

Nah, i think your idea would introduce a lot of overhead, so in most cases it wouldn't help at all. I think it refers to the pile of the potential frames wherefrom it can choose the closest match.

[virus]

Originally posted by Tommy Carrot
i think your idea would introduce a lot of overhead:confused:
Why? You only need to add a single number to every macroblock representation (identifying which frame contains your ref. block). Considering the amount of stuff coded for each block, like MB type, the MV (2 components, dx & dy) and the DCT coefficients (which, on a 16x16 macroblock, are 256 minus the trailing ones which are 0 in every 8x8 block), adding a single number doesn't seem a big overhead to me...

[bond]

Originally posted by aketon
So, nobody is sure about what "Max. number of reference frames" really is!!!:rolleyes: well as i said before i am 99% sure that it is the number of previous frames used for inter motion search

to answer, how this exactly works, will need a h.264 developer, like bobololo, i think :D

[shlezman]

To fix the confussion about the reference frames :

The encoder can inter-predict (motion compensate) each block (8x8) from any of the frames buffered in the frame-buffer, thus creating a very large space to predict from, compared to MPEG4 with a single frame of history.

The frame buffer consists of two parts, the short-term buffer which stores only the LAST X decoded frames and the long-term buffer can be used to buffer frames for unlimited time.

The frame buffer is defined in a very flexible manner so that many other applications can be done with it.

The larger the frame buffer is, the more memory the decoder uses. More then 4 shrt-term frames (which are the ones supported by x.264) is usually unnecessary.

[aketon]

After some tests, I am 100% sure about what "Max. number Reference Frames" really do! I made an Image Sequence of 100 frames! Then I start to blank every sixth frame! Then when I made an encode with ffdshow, the first time I used 1 Reference Frame and the second time I used 6 Reference Frames! And here are the results!

1 Reference Frame
244 kb

6 Reference Frames
146 kb

Really cool!!!! Right???????:D

[Tommy Carrot]

I made two encoding with 1 and 10 reference frames (other than that they have identical settings), and to my surprise the first encoding had smaller filesize (34,208kb vs. 35,852kb), and also the image was slightly sharper. Apparently the multiple reference frames option is not perfect in x264.

[virus]

Here's my test with "max number of ref. frames".

ffdshow-20040501 (warning, not the latest x264 revision)
500 frames, 560x304 @ 25fps, quant 13
default stuff + CABAC

max # ref. frames - bitrate:
1: 670 kbps
2: 662 kbps
3: 655 kbps
4: 654 kbps
5: 653 kbps
6: 653 kbps
7: 653 kbps
8: 653 kbps

Can you see a pattern? :D

virus

[aketon]

I've got "almost" the same strange results with virus!

I tried all the possible values for "Max. Number Of Reference Frames" in ffdsow 20040520!

CABAC ON
LOOP FILTER OFF
EVERYTHING ELSE DEFAULT

Here are the results:
100 FRAMES

REF-1
570 KB

REF-2
374 KB

REF-3
380 KB

REF-4
384 KB

REF-5
388 KB

REF-6
390 KB

REF-7
390 KB

REF-8
390 KB

REF-9
390 KB

REF-10
390 KB

Strange results! Is there anyone who can give an explanation, please???
And one more thing:

CAN ANYONE PLEASE TELL ME WHY FFDSHOW 20040520 DOESN'T ACCEPT MORE THAN 10 REFERENCE FRAMES????
EVERY TIME I TRIED TO USE VALUES LARGER THAN 10, FFDSHOW SWITCHED BACK TO 10!

[aketon]

OK! I've just done one more test:

VIDEO
----------
672x272
824 frames
25 fps

FFDSHOW SETUP
----------------------------
Quantizer 12
CABAC ON
Loop Filter OFF
All the analyzer flags ON (Inter PSUB8x8 OFF)
Maximum I frame interval 250
Minimum I frame interval 1


RESULTS:
----------------
REF-1 13.494 KB
REF-2 13.578 KB
REF-3 13.648 KB
REF-4 13.702 KB
REF-5 13.740 KB
REF-6 13.776 KB
REF-7 13.800 KB
REF-8 13.816 KB
REF-9 13.830 KB
REF-10 13.844 KB

The higher the value the bigger the filesize (this time at least)!

[aketon]

After a lot of testing, I found that 1 max. reference frame makes the encoded video not even smaller but and a little bit more detailed!!! I made too many tests just to be sure! I tried every different kind of fast motion and slow motion scenes I had and 1 max. reference frame was every time the winner! In some tests although using bigger values (from 2 to 5), I had smaller filesizes! From all the 35 tests I done with different values of reference frames, only in 3 of them, values 2 to 5 gave me better results!:confused:

[bond]

more than 1 possible references to choose from should make the codec more efficient, not worse, so it seems that x264 isnt really able atm to decide what frame to use as reference to get the best result :(

[Tommy Carrot]

Originally posted by aketon
After a lot of testing, I found that 1 max. reference frame makes the encoded video not even smaller but and a little bit more detailed!!! I made too many tests just to be sure! I tried every different kind of fast motion and slow motion scenes I had and 1 max. reference frame was every time the winner! In some tests although using bigger values (from 2 to 5), I had smaller filesizes! From all the 35 tests I done with different values of reference frames, only in 3 of them, values 2 to 5 gave me better results!:confused:

Exactly, this is what i told a few posts above yours. Something is clearly not right with x264 in this regard, so far i would recommend using 1 reference frame (with the exception of cartoons, where i believe using more reference frames is still beneficial due to the lot of repetitive image sequences).

[aketon]

Exactly, this is what i told a few post above yours. Something is clearly not right with x264 in this regard, so far i would recommend using 1 reference frame.

I totally agree with you!!!

[ToiletDuck]

After downloading a few of the samples and playing them I have no problems what-so-ever replaying them.
Duck

[bond]

according to Skal allowing more than 1 reference frames will also require more bits, as there additionally is the need for marking which frame is refered to too
the more frames allowed to choose from, the more bits required ("by using VLC it's: 1 bit if using Ref-1, 3 bits if using Ref-2,Ref-3,Ref-4, 5bits if using Ref-5 ... Ref-9, etc..."), the required bits would be smaller when using cabac i think

therefore if the bitrate savings because of using not simply the previous frame as reference is smaller than the bits needed for sending the framenumber, using this feature doesnt really make sense
this will heavily depend on the encoded content of course

[aketon]

Does anybody know why Justin's x264 builds accepts only 24 and 32 bits of color??? Shouldn't they accept YV12 color also? I am saying this because every time I choose this codec, virtualdub displays under known restrictions "Valid depths: 24 32" !!!

If anybody is going to tell me why I don't just use ffdshow, is because it hasn't got 51 quantizers to choose from!!!!:)

[ToiletDuck]

even though YV12 it isn't 12bit color is it? That would look horrible. I'd imagine it should stay at 32 bit for best results.

[RadicalEd]

Originally posted by ToiletDuck
even though YV12 it isn't 12bit color is it? That would look horrible. I'd imagine it should stay at 32 bit for best results.

You must be thinking of 12bit RGB. This is 12but YUV. Big difference :| They're actually not comparable, because all the YUV colorspaces I know of don't reduce bitdepth directly, they merely reduce chroma resolution.

[billou2k]

Yes thats right,
12bits is just an average per pixel.
Using YUV makes it possible to reduce the number of information for the colour without the eyes detecting it (normally;-) ):
Basically your picture is the sum of three layers: Y U and V.
The same goes for RGB but the difference is in the way these layers are created:
The Y is a black and white layer. This is the most visible component for the eye, that's why every pixel has a Y value (8bits=1Byte).
Then the two other layers create the colour layer. As the eye is less sensitive to spatial colour changes you can afford to have two or sometimes more pixels with the same colour value.
That's what YUV 4:2:2 or 4:1:1 or 4:2:0 stand for.

In YV12 : YUV 4:2:0, as said before each pixel as a Y value.
but then you only have 1 couple of U and V (8bits each) coding the colour component for 4 pixels (2*2) at the same time.
So if you average over these 4 pixels: you have
4*8 bits for the luminance + 2*8 bits for the colour / 4 pixels = 32+16/4=12 bits per pixel.

Working in YUV 4:2:2 (UYVY,YUY2..) is similar expect that you have a better color resolution : the U and V values are used to code the colour of 2 pixels (instead of 4 in YV12)
so in this case you use 2*8 (2 pixels with 1 value of Y for each ) + 2*8 (1 couple of U and V to code the colour of the two pixels) / 2 pixels = 16bits per pixel

Finally 32bits color means 4 times 8 bits: which means that on top of your 3 RGB /YUV layers you have an alpha layer.. and its absolutely useless for video. All video use 24bits to define a pixel at the moment.
Some new 3d video card can produce 64 or 128 bits colors for 3d rendering to improve contrast between high lighted and shadow areas but I dont think it is gonna be used for video.

(i hope it's clear:D)

[Nic]

@aketon: as you may have guessed that only refers to the RGB. Basically it means the codec won't accept 16bit RGB, etc. VDub should be more specific.

-Nic

[WaryWolf]

Nic: how are you getting along with the x264 source? i don't want to pry or rush you, i'm just wondering what you think of it and what you want to do... :)

[Nic]

Had basic 2pass code working. but off for a holiday in Vegas tomorrow morning so ya won't hear from me till around the 8th. See ya :)

[SeeMoreDigital]

Originally posted by Nic
but off for a holiday in Vegas tomorrow morning Are you off to Vegas get married, to gamble or both?

Stick some chips on the roulette table for me!


Cheers

[chenm001]

you can set fourCC to 'H264', it can be play by ffdshow
Originally posted by aketon
If anybody is going to tell me why I don't just use ffdshow, is because it hasn't got 51 quantizers to choose from!!!!:)

[billou2k]

I'm trying to get an idea of how x264 and other encoders compare to the reference encoder. Unfortunatly the only way I have to watch reference encoded videos is uncompressed yuv which takes "some" space.
Is there anyway to use an existing wrapper/decoder with the files encoded from reference encoder?

[Tommy Carrot]

Originally posted by billou2k
I'm trying to get an idea of how x264 and other encoders compare to the reference encoder. Unfortunatly the only way I have to watch reference encoded videos is uncompressed yuv which takes "some" space.
Is there anyway to use an existing wrapper/decoder with the files encoded from reference encoder?
If i remember correctly, Nic made some modification of the reference encoder, which accepts AVS as input, and outputs AVI. Unfortunately he did not release it to the public, but i suppose if you ask him nicely, he will do it. ;)

[bond]

Originally posted by Tommy Carrot
If i remember correctly, Nic made some modification of the reference encoder, which accepts AVS as input, and outputs AVI. Unfortunately he did not release it to the public, but i suppose if you ask him nicely, he will do it.guys use search!
there have been some posts about mod of the reference encoder allowing .avs input (long before nic said he will work on h.264)

Originally posted by billou2k
Unfortunatly the only way I have to watch reference encoded videos is uncompressed yuv which takes "some" space.
Is there anyway to use an existing wrapper/decoder with the files encoded from reference encoder?again, plz use search and read the mpeg-4 sticky
the reference encoder outputs plain .264 bitstreams, which can be played back in dshow with the moonlight and mainconcept filterset without a problem

[billou2k]

Sorry to look like the lazy guy who asks before searching but I did actually search before.
I've participated in the thread where the avs modified version of the reference code was discussed. That's where I said that it was crashing for me. I also replied to the person who asked me for more details by email. The reason was apparently that the AVS version crashes on interlaced content for some unknown reason.

Anyway, I have encoded several h.264 files with the JM82 but I cant find how to use the moonlight or mainconcept decoder with dshow.

I'm playing with graphedit but I dont seem to find the proper combination of filters...
I also guess that the decoders arent compatible with some option from the reference encoder so I'm trying to playback simple files but still no success yet...
Would you mind just explain shortly the way you do it?

Thanks a lot!

[bond]

Originally posted by billou2k
Anyway, I have encoded several h.264 files with the JM82 but I cant find how to use the moonlight or mainconcept decoder with dshow.dont waste time with moonlight, it plays .264 raw files, but its bloated, buggy and places a logo during playback on the picture

I'm playing with graphedit but I dont seem to find the proper combination of filters...

Would you mind just explain shortly the way you do it?hm if i remember it right moonlight always placed itself as the default used filterset after installing
what else gets used for playing .264 files if not moonlight?

[thegeby]

It looks like the EBU statement on MPEG 4 AVC licensing terms has had some effect....

New licencing terms for public service TV (http://www.mpegla.com/news/n_04-05-18_avc.pdf)

[Tommy Carrot]

Something has been borked in x264, the latest build creates bad artifacts/blocks, especially around high contrast areas. The older versions didn't have this problem, and this is not a cabac problem either, it occurs even without it. Here (http://www.fw.hu/carrotland/bugged_bridge.avi) is a short sample clip.

[Rash]

Oh yes Tommy. That sample of your looks terrible. :(

[BoNz1]

Tommy Carrot, IIRC there was a bug that was fixed with motion compensation. So, check the build date you may want to get a new build. I remember seeing artifacts like this a little while ago and I think they were fixed when I grabbed the latest sources.