Bit stream structure of Access Unit and VCL NAL unit in MPEG-4

[Rouhi]

Hi
Do you know how can i detect any I-Frame in a MPEG-4 AVC video by parsing the bit stream of the video?
In another language i am looking for some bit symbols which shows an I-Frame start and end.
I am trying to extract the coded information in the coded picture inside the i-frame.
Any Idea?:confused:

[Selur]

here's the comment I added in the FrameCounter sources:
/**
* NAL Units start code: 00 00 00 01 X Y
* X = IDR Picture NAL Units (25, 45, 65)
* X = Non IDR Picture NAL Units (01, 21, 41, 61) ; 01 = b-frames, 41 = p-frames
* since frames can be splitted over multiple NAL Units only count the NAL Units with Y != 00
**/ don't remember where I got that info when I wrote the code but it shouldn't be wrong,.. (if it is I guess someone here will correct me ;))

Cu Selur

[Rouhi]

Many thanks Selur. You are right and i can find the bite stream 00 00 00 01 65 in my video which contain only an i-frame. But what i need is that how can i conform this with the standard ITU-T H.264 2010? Please have a look on below schematic design of a VCL-NAL unit in head of an Access Unit.

http://i52.tinypic.com/2ibd9q8.jpg

As can be seen the three byte start code prefix of each VCL-NAL unit is: 00 00 01 in hex. but before these three byte, there is a byte as VCL NAL header. In some texts i read that it is leading zero byte. in this case the total would be 00 00 00 01 and it is same as you said.
but in the standard, chapter 7 page 42 says that the first header byte contains 3 section: 1 bit zero. two bit and NAL reference id. these two bit would be zero too if and only if the picture is not refrencing to other pictures. and as the i-frame is a stand alone picture so these two bits would be zero too. ok?
But the rest of 5 bits. the standard says these bits represent NAL unit type. According to the standard on page 64, these 5 bits would be zero if the NAL unit is a Non-VCL NAL unit!!!!
This is the problem in conforming your suggested byte stream and standard content. i-frame should be in a VCL-NAL units and with respect to this fact the rest five bits should be an unsigned integer between 1 to 5.

By the way, do you have any idea about the Access unit delimiter bytes syntax? and how can i access your source code of frame count if it is an open source free software.

Thanks for your cooperation in this discussion.

[Selur]

how can i conform this with the standard ITU-T H.264 2010?
No clue, I never looked at the 2010 version. :)

FrameCounter Source: http://stashbox.org/985676/FrameCounter.zip
With the rest I really can't help since it's ages since I last looked at the specs&co and touched the source code (while using my brain),... ;) I just modify it if I find a source that makes problems by using a HexEditor and looking at the video stream that's problematic.
I just thought it might be useful for those trying to do something similar,.. :P

Sorry, that I can't be more of a help.

Cu Selur

Ps.: if you find additional infos that could tune my FrameCounter please share.

[kieranrk]

Read Annex B of the spec.

[Rouhi]

Read Annex B of the spec.
I have read the annex B. it is in brief and just two pages in byte-stream of NAL unit. Complete information can be found in chapter 7 and 9. Thanks anyway.

Frame Counter Source: http://stashbox.org/985676/FrameCounter.zip
Thanks Selur. I have checked your frame count software and noticed that it detect the 1 frame videos which i had created them, correctly. But in a 17 MB video clip in MPEG-4 , with about 5 min length, the software just analysed 17 Frames and counted totally 19 frames. Please explain what's the difference between analysed frames counted and total counted frames. BTW do you think this number is reasonable for such a video clip?

[Selur]

just analysed 17 Frames and counted totally 19 frames.
The progress indication 'Frame count analyse X of Y' doesn't count frames but MB. So it analysed 17MB and found 19 frames if this is a reasonable number is hard to say,... I would have anticipated that one would find around 25*60*5 = 7500 frames in a 5min clip so it's at least unexpected,..

[Rouhi]

as can be seen in below image, for a 17 MB, 5 Minute video in MPEG-4 AVC format, only 19 frames has been detected. Please look at my parameter passing syntax to the framecounter.exe, is that correct?

http://i54.tinypic.com/2vcz2xi.jpg

Is it possible that the Framecounter software is not designed according to the latest version of MPEG-4 which is MPEG-4 AVC with H.264 standard?

[Rouhi]

Please have a look on below screen shot. It belongs to a video in MPEG-4 AVC format, containing just one frame only (which naturally should be IDR I-Frame).

http://i53.tinypic.com/35hg1p0.jpg

As can be seen in red box, the word 00 00 00 01 65 can be seen and it is unique. but before this word, there are 3 other words which start with 00 00 00 01 (in green boxes). Whats your idea?

BTW, when i open the 17 MB 5 Min video in hex editor, Huge amount of bits can be seen before first occurrence of the word 00 00 00 01 65 (or 25, or 45). You can see some part of the start bit stream of this video on below picture. The first occurrence of 00 00 00 01 has been shown in red box.

http://i51.tinypic.com/16lclqc.jpg

[Selur]

but before this word, there are 3 other words which start with 00 00 00 01 (in green boxes). Whats your idea?You think these are frames? they seem too small ;)

BTW, when i open the 17 MB 5 Min video in hex editor, Huge amount of bits can be seen before first occurrence of the word 00 00 00 01 65
are you certain this is even a raw stream,.. stuff like "FLV... MetaData' shouldn't be in a raw stream,...

what do ffprobe and mediainfo say to your file?

[Rouhi]

/**
* NAL Units start code: 00 00 00 01 X Y
* X = IDR Picture NAL Units (25, 45, 65)
* X = Non IDR Picture NAL Units (01, 21, 41, 61) ; 01 = b-frames, 41 = p-frames
* since frames can be splitted over multiple NAL Units only count the NAL Units with Y != 00
**/
I find out the first two 00 in above bitstream are not needed and actually not conform with standard H.264 2010.
Help me alot, thanks. I would be pleased if you explain about x=21 or 61 in non IDR pictures. what type of picture they are?

And all IDR pictures (x=25,45,65) are I-Frames? According to standard the nal_unit_type (as can be seen in right the digit at above) equal to 5 means it is an IDR picture and if equal to 1 it is non IDR picture.

[Selur]

I find out the first two 00 in above bitstream are not needed and actually not conform with standard H.264 2010.
Are you sure? (iirc there needed to be a zero byte (00) before the start code prefix, see also: http://lists.mpegif.org/pipermail/mp4-tech/2006-October/006944.html)

I would be pleased if you explain about x=21 or 61 in non IDR pictures. what type of picture they are?
should be sp- and sb- slices but I don't know which is which,...

I uploaded some of the raw samples I use to test the frameCounter here: http://www.multiupload.com/KK1SHU9A6O

current source code with 00 00 01: http://forum.selur.de/post42.html#p42
seems to work fine unless source is encoded with --sliced-threads,... (then the count is far too high, I guess each slice is counted as a single frame :/)
-> fixed by not checking for 'Y != 0' but '(Y & 0x80) > 0'

Cu Selur

Ps.: current source code is over at: http://forum.selur.de/post42.html#p42
00 00 00 01 65 X
00 00 00 01 41 X
1) Byte X contains the first value for "first_mb_in_slice" ,
2) I only want to count the first Slice of each Frame
3) the "first_mb_in_slice" should always be 0
4) "first_mb_in_slice" is Exp-Golomb coded
5) so in the first slice X / MSB needs to be set 1000 0000
-> HEX value neeed to be >= 0x80 which should be equal to a '(X & 0x80h) > 0' check.

(thanks to bigotti5&LigH for this)

[Rouhi]

Dear Selur
it is possible that one of the reasons for counting many frames with this header is that you are counting 00 00 01 65 X and 00 00 01 41 X. Is that right? if you count both, so you are counting all referenced pictures which P-Frames are among them too (00 00 01 41 X).

Regarding the first zero byte , in the standard , appendix B says:
"When it is the first byte stream NAL unit in the bitstream, it may also contain one or more additional leading_zero_8bits syntax elements.".
with respect to this it seems that if we count the first start code prefix with 8 leading zero bits (00 00 00 01) and for the rest count only the start code prefix (00 00 01), then we can get most correct result. What you reckon?

btw, I have downloaded the 72 MB of videos from your suggested link. and your altered source code in C++ for counting I-Frames start with word: 00 00 01 X Y frames. It seems i am not lucky today because your code did not compiled in my C++ compiler(Bloodshed Dev) and even the videos can not be played. Is it possible to inform me how can i run your source code? which compiler do i need?

[Selur]

I attached my files to http://forum.selur.de/post42.html#p42 it's fairly easy to compile if you keep in mind that I use the Qt Framework.
Here's a small how to compile the source on Ubunut:
1. install Compile Environment (linux-build essential + qt sdk)
2. extract the source in a folder
3. generate the make file: in the source folder run qmake FrameCounter.pro
4. build the code: run make in the source folder
On Windows systems easiest way is probably to:
1. install Qt SDK (http://qt.nokia.com/downloads), this should install all the QT libraries, MinGW and QT Creator as IDE
2. extract the source in a folder
3. start the Creator, open the project File (.pro)
4. tell the Creator to build the project
(alternatively if you use eclipse as IDE, install the Qt eclipse integration, run qmake and then compile it with eclipse)

If you want to use other compilers or need static builds it's a bit more complicated,.. ;)
I for example on Windows I use the VS C++ 10 compiler and build static builds from inside eclipse (I little pain to setup), on Linux I build via command line and on Mac OS X I use Qt Creator.

Cu Selur

Ps.: Frame count seams to be fine with latest version on the samples I use,..

[Rouhi]

OOOps, I update my post before see your reply.. very strange. would you read my last post again

[Selur]

I think it's okay to skip the zero byte, the problem with to many frame was the criteria when to count slices an when not, but the fix from bigotti5 helped there. ;)

[Rouhi]

Dear Selur
My platform is windows and when i compile and build your project code, the final exe file cant be run and i encounter the Mingwm10.dll missing error. I think i need a static one. can you guide me how can i convert it from QT dynamic to static?
I tried to use "configure -static" but the platform can not be detected. using -platform command also did not help. Please advice.

[Selur]

If it complains about Mingwm10.dll it's not a static build, that means you need the mingw and the qt dlls. If you want to make static builds you need to:
1. recompile the qt sdk with the configure flag
2. make sure your build tools use the new compiled static libraries
see: http://www.qtcentre.org/wiki/index.php?title=Building_static_Qt_on_Windows

Wrote you a small guide how to compile a static version of Qt, see: http://forum.selur.de/post211.html#p211 (with gcc form MinGW or the VS C++ compiler)

Cu Selur

ps.: uploaded a static mingw32 build of FrameCounter to: http://www.multiupload.com/OU50J2TICG

[Rouhi]

Dear Selur
Would you please compile and save your code in form of exe file and upload it or simply email me ?
I am writing a couple of code lined and i want to compare the results.
ahrouhi a t hotmail dot com
regards

[Rouhi]

BTW in computing the file size i can see in your code:

double size = file.size() / 1048576.0;

would you explain why you change the file size to a coefficient of 2 pow 20? does it because of the size of Double and bits or you have any other reason?

[Selur]

It's because of the 'status'-line that comes later:
std::cerr << "Frame count analyse at " << sizecount << " of " << size << std::endl;
and that I each time read:
data.append(file.read(1024 * 1024)); //read 1MB
to clean it up a bit one could rewrite:
int sizecount = 1;

double size = file.size() / 1048576.0;
int basePatternSize = basePattern.size();
int patternSize = basePatternSize + 2;
QString prefix;
bool validY = false;
QHash<char, int> counted, unknown, spspps;
while (!file.atEnd()) {
data.append(file.read(1024 * 1024)); //read 1MB
datasize = data.size();
if (datasize > 1024*1024*20) {
std::cerr << "Read 20MB and found no frame!" << std::endl;
return 0;
}
if (datasize < patternSize) {
break; //no complete pattern -> finished
}
//output position
std::cerr << "Frame count analyse at " << sizecount << " of " << size << std::endl;
sizecount++;
to:
int sizecount = 1;
int readSize = 2 << 20;
double size = file.size() / (double(readSize));
int basePatternSize = basePattern.size();
int patternSize = basePatternSize + 2;
QString prefix;
bool validY = false;
QHash<char, int> counted, unknown, spspps;
while (!file.atEnd()) {
data.append(file.read(readSize)); //read 1MB
datasize = data.size();
if (datasize > readSize*20) {
std::cerr << "Read 20MB and found no frame!" << std::endl;
return 0;
}
if (datasize < patternSize) {
break; //no complete pattern -> finished
}
//output position
std::cerr << "Frame count analyse at " << sizecount << " of " << size << std::endl;
sizecount++;

so no real reason. :)

Cu Selur

[Rouhi]

I have written a little program in C++ which try to count the frames according to what i have find out from H.264 standard and our discussion with Selur too and his code. In most of the cases there is a conformity between our results but some differences can be seen in the sliced videos and also counting 0x67 (SPS) frames. it can be seen at below image of the spread sheet. The videos selected among those video which you had shared already.

http://i52.tinypic.com/35bzcaw.jpg

Whats your idea? It seems it is due to the role of Y byte. I did not consider Y in the stream 00 00 01 X Y because i could not find about the role of Y in the standard.

[Selur]

you count all the slices,.. (if a frame is sliced into 5 slices you would count the frame 5 times)

here's why and how I look at Y:

problem: I only want to count the first slice of each frame
- looking at: 00 00 01 X Y
Y holds the value of "first_mb_in_slice"
- the first "first_mb_in_slice" value of the first slice needs to be 0
So far so good, this is why I first checked Y against 0 which was a mistake (thanks to bigotti5 for pointing it out!)
- "first_mb_in_slice" is exp-golomb (http://en.wikipedia.org/wiki/Exponential-Golomb_coding) coded!!
- this means in the first slice the first byte is always set
1000 0000
so the hex value of is always >= 0x80
=> one needs to check Y >= 0x80 which I now do with the (Y & 0x80h) > 0 check

hope this helps

Cu Selur

[Rouhi]

my understanding from your text is that the first byte (Macro Block byte) in the first slice (in any sliced frame) should be equal to zero , which its representation in golomb code is 1000 000 (0x80). these first bytes in other slices of frame can be any value grater than 0 because they are not the first byte of first macro block. am i right?

so i should find out firstly that the value of Y (00 00 01 X Y) of each 5-byte-word which i read from the video file, is equal to zero or not, if it is zero then it can be considered as a frame. if this byte is not equal to zero but the prior 4 bytes are in format 00 00 01 (25 or 45 or 65or 41 or...) it means that it is one of the slices of a frame.the frame slices.

I change my code according to this concept and when i check the value of Y (00 00 01 X Y), the fifth byte of any word that i read from the file with zero, my results would be same as yours but when i check the Y value with 0x80 (Exponential-Golomb Codes equal to zero) i cant get correct answer.
I use my variable as unsigned char in c.
It seems that if Y=00 the value store in ordinary format and not in golomb code format.

[Selur]

am i right?
yup, that's how bigotti5 explained it and how I understood it :)
not sure if it's faster to first check Y in 00 00 1 X Y and then X or if it's faster to check first X in 00 00 1 X Y and then Y, but in the end the result should be the same. :)

Cu Selur

[Rouhi]

But as i wrote before my question is:
I change my code according to this concept and when i check the value of Y (00 00 01 X Y), the fifth byte of any word that i read from the file with zero, my results would be same as yours but when i check the Y value with 0x80 (Exponential-Golomb Codes equal to zero) i cant get correct answer.
I use my variable as unsigned char in c.
It seems that if Y=00 the value store in ordinary format and not in golomb code format.

you check Y with value 0x00 or 0x80 for counting frames(not slices of frames)?

[Rouhi]

I find out something new and I think the bitstream format in sliced frames in H.264 is a little different.
When I look at the bit stream of a non sliced video frames I can see the format 00 00 01 65 88 as an I_Frame header. in this format the Y=0x88 and it is the Golomb presentation for 0x00 if you look at it like this: hex: 0000 0000 and the Golomb code would be : 1000 1000 which is 0x88.

But the same video with 4 slice per each frame has the following sequence of I-Frame header values:

0x 00 00 01 65 88
0x 00 00 01 65 03
0x 00 00 01 65 01
0x 00 00 01 65 00
0x 00 00 01 65 88
0x 00 00 01 65 03
0x 00 00 01 65 01
0x 00 00 01 65 00
.
.
This presentation shows that the Golomb value of Y=00 representing the last slice of a sliced frame. This sequence for B and P frame is a little different as below: (For P_frames as an example: )

0x 00 00 01 41 9E (or 9F or 9A …)
0x 00 00 01 41 03
0x 00 00 01 41 01
0x 00 00 01 41 00
0x 00 00 01 41 9F (or 9F or 9A …)
0x 00 00 01 41 03
0x 00 00 01 41 01
0x 00 00 01 41 00
.
.
As can be seen, it seems that for counting the B and P frames we should focus on first 4 bits of Y (MSB) and these four MSB bits should contain the value of 0x9.

Would you please let me know your idea about this fact? would you consult with bigotti5?

[Selur]

you check Y with value 0x00 or 0x80 for counting frames(not slices of frames)?
Yes I always use validY = (data[yIndex] & 0x80) > 0;, don't know beforehand if the stream is sliced or not. ;)

(0x88 & 0x80) > 0: true
(0x03 & 0x80) > 0: false
(0x01 & 0x80) > 0: false
(0x00 & 0x80) > 0: false
(0x9A & 0x80) > 0: true
(0x9B & 0x80) > 0: true
(0x9C & 0x80) > 0: true
(0x9D & 0x80) > 0: true
(0x9E & 0x80) > 0: true
(0x9F & 0x80) > 0: true
-> true for all values >= 0x80

Not sure what I should consult with bigotti5 about, as far as I can tell the check seems to do what it is ment to do,..

[Rouhi]

Ok, seems the & operator do the same thing with what i understand.
for SPS and PPS farmes do you consider the value of Y too? if yes what is the value of Y should be for them.
BTW in standard we have SP slice and SI slice. how should i conform SPS and PPS on the slices mentioned in standard(SPS for SP slice and PPS for SI slice?)

[Selur]

I use the same check for SPS/PPS (http://forum.selur.de/post42.html#p42) but I never thought about whether it made sense or not since I'm not counting them for the frame count. :)
Atm. I don't see a reason to count the Sequence Parameter Sets (SPS) or the Picture Parameter Sets (PPS), atm I only count them in FrameCounter just for the fun of it and to be alert if I should encounter a yet unknown Y value. :)

Cu Selur

[Rouhi]

OMG, I suppose they are SP slice and SI slice. Ok then. so they should not be counted among frames.
Tnx again.
Now i want to go one step deeper, inside the frames.do you have any information about the intra frames bitstream? as you know,in I_Frames, the frame content consists of , the ICT coefficients and intra frame prediction of MB and blocks. They should be stored in Golomb format. I want to find them. do you have any idea in this new area?

[Selur]

Nope, sorry never looked into it. :)

[Rouhi]

I am not sure that this field has been finished or not but i would like to say:
Thanks alot dear Selur and all other experts that helped me in this field.
I will open another topic for discussing about deep inside the frames codes.
Again thank you very much for your contribution

[Rouhi]

Hi Selur
when i was testing my code and your code for frame counting on the video which u shared with me, the results was same and reasonable for number of frames. but when i test both codes on different other *.264 videos, i cant get reasonable results. your code for a 22MB video just count 38 frames and mine count only 4 I frames and 10 P and 10 B frames.
It seems may be the format is not standard.
For being confident about the correct process of algorithm, i need to convert to H.264 videos myself.
Using JM software (the open source software of ITU-T for H.264) need raw format as input. Can you recommend any free software that convert the video to raw format or standard MPEG-4 AVC format?
Do u think the below ffmpeg command is correct to convert to mpeg-4 AVC?

ffmpeg -i h264.mp4 -vcodec copy -vbsf h264_mp4toannexb -an out.h264

[Selur]

I normally use MP4box (which is a part of the gpac project (http://gpac.wp.institut-telecom.fr/)) to extract raw streams from .mp4 and .mov files.
If you want to create H.264 content I would recommend x264 as encoder. (the JM software is way to slow for any practical purpose)

Cu Selur

[Rouhi]

I have downloaded MP4BOX but the software can not produce any raw format in form of *.RGB or *.YUV
It accept only AVI as input and even i couldn't run it on any AVI because the software cant detect them. I have downloaded the final version.

[Selur]

Oh, misunderstanding you want to produce raw RGB/YUV files,..
you wrote ffmpeg -i h264.mp4 -vcodec copy -vbsf h264_mp4toannexb -an out.h264 so I thought you wanted to create raw H.264 streams, since that line just extracts the h.264 stream. :)

Here's what you can use to create IYUV (<>Yv12):
ffmpeg -v -10 -i "path to input" -threads 8 -vsync 0 -an -pix_fmt yuv420p -f rawvideo test.yuv
or
mencoder "path to input" -ovc raw -noskip -vid 0 -vf scale,format=i420 -forcedsubsonly -noautosub -nosound -mc 0 -lavdopts threads=8 -really-quiet -of rawvideo -o test.yuv

Here's what you can use to create NV12:
ffmpeg -v -10 -i "path to input" -threads 8 -vsync 0 -an -r 25 -pix_fmt nv12 -f rawvideo test.yuv

so using ffmpeg you can simply change the pix_fmt to specify what sort of raw video you want,...
Call
ffmpeg -pix_fmt list
to see:
Pixel formats:
I.... = Supported Input format for conversi
.O... = Supported Output format for conversi
..H.. = Hardware accelerated format
...P. = Paletted format
....B = Bitstream format
FLAGS NAME NB_COMPONENTS BITS_PER
-----
IO... yuv420p 3 12
IO... yuyv422 3 16
IO... rgb24 3 24
IO... bgr24 3 24
IO... yuv422p 3 16
IO... yuv444p 3 24
IO... yuv410p 3 9
IO... yuv411p 3 12
IO.P. gray 1 8
IO..B monow 1 1
IO..B monob 1 1
I..P. pal8 1 8
IO... yuvj420p 3 12
IO... yuvj422p 3 16
IO... yuvj444p 3 24
..H.. xvmcmc 0 0
..H.. xvmcidct 0 0
IO... uyvy422 3 16
..... uyyvyy411 3 12
IO.P. bgr8 3 8
.O..B bgr4 3 4
IO.P. bgr4_byte 3 4
IO.P. rgb8 3 8
.O..B rgb4 3 4
IO.P. rgb4_byte 3 4
IO... nv12 3 12
IO... nv21 3 12
IO... argb 4 32
IO... rgba 4 32
IO... abgr 4 32
IO... bgra 4 32
IO... gray16be 1 16
IO... gray16le 1 16
IO... yuv440p 3 16
IO... yuvj440p 3 16
IO... yuva420p 4 20
..H.. vdpau_h264 0 0
..H.. vdpau_mpeg1 0 0
..H.. vdpau_mpeg2 0 0
..H.. vdpau_wmv3 0 0
..H.. vdpau_vc1 0 0
IO... rgb48be 3 48
IO... rgb48le 3 48
.O... rgb565be 3 16
IO... rgb565le 3 16
.O... rgb555be 3 15
IO... rgb555le 3 15
.O... bgr565be 3 16
IO... bgr565le 3 16
.O... bgr555be 3 15
IO... bgr555le 3 15
..H.. vaapi_moco 0 0
..H.. vaapi_idct 0 0
..H.. vaapi_vld 0 0
IO... yuv420p16le 3 24
IO... yuv420p16be 3 24
IO... yuv422p16le 3 32
IO... yuv422p16be 3 32
IO... yuv444p16le 3 48
IO... yuv444p16be 3 48
..H.. vdpau_mpeg4 0 0
..H.. dxva2_vld 0 0
.O... rgb444be 3 12
.O... rgb444le 3 12
.O... bgr444be 3 12
.O... bgr444le 3 12
I.... y400a 2 16
..... bgr48le 3 48
..... bgr48be 3 48
a list of the pixel formats ffmpeg supports,...

Cu Selur

[Rouhi]

It seems that the problem still there when i want to count the frames in a big file. I tried to input the raw video as input and then convert it to raw h.264 format by JM and FFMPEG but when i use the frame count algorithm on them i get very very unexpected results. for example only 10 I frame in a 20 min video.... How it is possible? would you check yourself?

[Selur]

I had no problem with H.264 material created with Quicktime, x264, NVIDA encoder, never thought about using JM to encode any large files, since it's far to slow. :)
What command lines do you use? (no motivation here to read up on ffmpeg&JM switches ;))

Cu Selur

[Rouhi]

For JM there is a huge config file *.cfg, which thousands of parameter should be set but mostly set to default fortunately. Some strange information that JM ask is
1- number of frame to be converted( no default for whole the video stream, you should enter an integer)
2-size of frame. no default size is there. so you should get the size of source file in some way and enter this parameter.
3-allocating reference buffer and
4-RDO quantisation
I get tired of JM, is terribly slow software.
Can you guide me how can i convert YUV to raw h.264? By FFMPEG and x264 or NVIDA or Quicktime. If they dont add container header and some extra information to the pure h.264 file.

i use the previous ffmpeg command for convert any video format to h.264. have a look at it and say am i wrong?
ffmpeg -i "input file(yuv or ...." -vcodec copy -vbsf h264_mp4toannexb -an out.h264

[Selur]

That command line is ment to use ffmpeg to open an input file (-i "input file(yuv or ...."), copy the video stream (-vcodec copy) using the h264 stream write routine (-vbsf h264_mp4toannexb) and write the output to a file named out.h264 (-an out.h264).
So like I wrote before it just extracts a h.264 stream,.. and does not what you want. :)

Can you guide me how can i convert YUV to raw h.264?
assuming you created a raw Yv12 stream, i.e. with something like: ffmpeg -v -10 -i "g:\Hybrid\test - clips\test.avi" -threads 8 -vsync 0 -an -pix_fmt yuv420p -f rawvideo test.yuv (remember width, height and framerate of the stream since you will need it later)
you can convert it to raw h.264 using:

x264:
x264 ALL_THE_OPTIONS_YOU_WANT --input-res THE_RESOLUTION --fps FRAME_RATE --output NAME_OF_THE_OUTPUT THE_INPUT
i.e.
x264 --crf 18 --input-res 640x352 --fps 25 --output test1.264 test.yuvWhere 640 is the width, 352 is the height and 25 is the frame rate of the input. '--crf 18' tells x264 to use 1pass crf mode with a target rate factor of 18; call x264 --fullhelp to see all the x264 command line switches.

ffmpeg:
ffmpeg -f rawvideo -pix_fmt yuv420p -s THE_RESOLUTION -r FRAME_RATE -i THE_INPUT -an -vcodec libx264 ALL_THE_OPTIONS_YOU_WANT -threads 0 -r FRAME_RATE -f h264 NAME_OF_THE_OUTPUT
i.e.
ffmpeg -f rawvideo -pix_fmt yuv420p -s 640x352 -r 25 -i "g:\Hybrid\test.yuv" -an -vcodec libx264 -crf 18 -threads 0 -r 25 -f h264 test2.264Where 640 is the width, 352 is the height, 25 is the frame rate and yuv420p the pixel format of the input. '-crf 18' tells libx264 to use 1pass crf mode with a target rate factor of 18.

the nvidia based encoder I wrote: (requires a newer nvida card)
cuda --input THE_INPUT --resolution THE_RESOLUTION --fps FRAME_RATE ALL_THE_OPTIONS_YOU_WANT --output NAME_OF_THE_OUTPUT
cuda --input test.yuv --resolution 640x352 --fps 25 --output test3.264 --showFrameStats 100
Where 640 is the width, 352 is the height and 25 is the frame rate of the input. (didn't add any additional options, call cuda to see all the available options)

If they dont add container header and some extra information to the pure h.264 file.
x264 will add some additional infos, but no container header,...

Some strange information that JM ask is ...
Not really strange, it's a simple show case - default implementation - that is ment to show that it is possible to make an encoder, it's not ment to be ready for normal use or to be user-friendly. ;)

Cu Selur

[Shevach]

here's the comment I added in the FrameCounter sources:
/**
* NAL Units start code: 00 00 00 01 X Y
* X = IDR Picture NAL Units (25, 45, 65)
* X = Non IDR Picture NAL Units (01, 21, 41, 61) ; 01 = b-frames, 41 = p-frames
* since frames can be splitted over multiple NAL Units only count the NAL Units with Y != 00
**/ don't remember where I got that info when I wrote the code but it shouldn't be wrong,.. (if it is I guess someone here will correct me ;))

Cu Selur

The condition Y!=0 is incorrect. Indeed, Y contains first_mb_in_slice synstax element. If first_syntax_element=0 then Y shall be non-zero.
On the other hand if first_mb_in_slice is 1, 2 or 3 then Y is also non-zero.
I think the correct condition should be if ( Y&0x80!=0)

[Selur]

Yes, that was cleared later in this thread,.. -> http://forum.doom9.org/showpost.php?p=1498138&postcount=12 ;)

[kypec]

I'd go with the most simple process:x264.exe -o outputfilename.264 inputfilename

[Selur]

@kypec: afaik this will not work since the input is raw and so x264 needs at least the resolution (unless you use a raw format like yuv4mpeg which contains the resolution infos)

[Rouhi]

In response to Shevach i should remind that the first zero byte in 0x 00 00 00 01 X Y is not necessary too. we had a long discussion with Selur and finally noticed if we consider three byte start code prefix ( 0x 00 00 01 X Y ) the result of frame counter is accurate.

BTW do you have any experience in extracting intra-prediction-mode coefficient from the content of file?

[Rouhi]

Two question:

I convert a 5 min 720x480 raw video to 264 video by ffmpeg by below command:

ffmpeg -f rawvideo -pix_fmt yuv420p -s 720x480 -r 30 -i c:\*.yuv -an -vcodec libx264 -threads 0 -r 30 -f h264 c:\*.264

and then i use both (you and mine)frame counters on the generated video separately, the result was same....(number of I-Frames =1411)
but when i convert the same video from avi format to 264 video by the below command:

ffmpeg -i *.avi -vcodec copy -vbsf h264_mp4toannexb -an *.h264

both frame counters give wrong results (just 10 I-Frames counted in this situation which for 5 min video clip is absolutely wrong). what do you think?


BTW when i used again the same raw video format (yuv) as input but reduced the size of output video to half by below command:

ffmpeg -f rawvideo -pix_fmt yuv420p -s 360x240 -r 30 -i c:\video\dariush2.yuv -an -vcodec libx264 -threads 0 -r 30 -f h264 c:\dariush2.264

This time the number of I-Frames in generated video, counted by both frame-counters, are same but has been increased to 1907.
What is your analysis for this behaviour of encoder? Does the size of frames, affect the decision criteria of encoder for generation I-Frames? If yes why?

[nm]

ffmpeg -i *.avi -vcodec copy -vbsf h264_mp4toannexb -an *.h264

Does that AVI contain H.264 video? If not, the output is not H.264. Post the full ffmpeg log from that conversion.

[Selur]

I second nm's question.
ffmpeg -i *.avi -vcodec copy -vbsf h264_mp4toannexb -an *.h264
Tells ffmpeg to extract the video stream from all avi files and use the h264_mp4toannexb write routine to do so, if the content of the .avi files is not H.264 to begin with the output is probably just junk. :)
(it's not a conversion but an extraction that's done but the ffmpeg call)

Does the size of frames, affect the decision criteria of encoder for generation I-Frames? If yes why?
If you resize a video you drop/add information to the video which can influence the scene change decision and the decision when to insert an I-Frame,...
Intuitively I would have suspected that the I frame count would go down with lower frame size, which isn't the case here,...

Cu Selur

[Rouhi]

i know that the some video formats such as .avi, .3gp, .flv are just container for other genuine formats such as MPEG4AVC/H.264 or MPEG2 but the question is that does it affect the ffmpeg output if the basic format of the input video is not MPEG4AVC/H.264? as we know the ffmpeg can convert raw format to MPEG4AVC/H.264 so the basic format of input video should not affect its functionality in converting the video to output.

If you resize a video you drop/add information to the video which can influence the scene change decision and the decision when to insert an I-Frame,...

i reckon if we change the scale of a video proportionally in height and width, so the whole scene remain intact and naturally the decision criteria for I-Frame in encoder should not be affected, because Just pixels number reduced and all content are same.