Bit stream structure of Access Unit and VCL NAL unit in MPEG-4

[Rouhi]

A unrelated to this topic question : You guys have any experience with SIFT (Scale Invariant Feature Transform) and SURF (advanced SIFT) tools and algorithms? Are they open source application that their input is an image and their output are feature vectors? How can i access to them?

[Selur]

Sift&Co: OpenCV (http://opencv.willowgarage.com/wiki/) includes methods&co (http://opencv.willowgarage.com/documentation/cpp/feature_detection.html) that allow you to extract the feature vectors.

How can i access to them?
Read the api and doc, write you own code that uses the libraries. :)

the whole scene remain intact and naturally the decision criteria for I-Frame in encoder should not be affected, because Just pixels number reduced and all content are same.
When resizing/filtering you normally add/remove noise/details which influences the similarity between images which can influence the scene-change detection (which should insert an I-frame depending of the similarity of frames).
The amount of influence this has in real life might be minimal, but it is there and depending on the resizer/filter used might not be soo small,...

Other than that without a detailed step-by-step guide what you did an an example file to reproduce the effect this is all wild guessing. The effect of adding/removing noise/details through resizing should not account for such a 35% increase (1411 vs. 1907) in I-frames. So you probably used other settings or changed something else too.

Cu Selur

[nm]

i know that the some video formats such as .avi, .3gp, .flv are just container for other genuine formats such as MPEG4AVC/H.264 or MPEG2 but the question is that does it affect the ffmpeg output if the basic format of the input video is not MPEG4AVC/H.264? as we know the ffmpeg can convert raw format to MPEG4AVC/H.264 so the basic format of input video should not affect its functionality in converting the video to output.

That's correct when you are re-encoding the video, as you did in the first command with -vcodec libx264. But in the second command you were only copying the video track with -vcodec copy. That doesn't change the video format and it remains the same as in the source, which in your case wasn't H.264.

[Rouhi]

Thanks Selur for OpenCV and about my mistake in frame counting after resizing the video i should say Sorry.
there was a mistake in my command line and when i corrected it, the number of frames (i p and b) did not change after resizing the video dimensions to half vertically and horizontally.

[Rouhi]

we know that the frames in H.264 standard has three colour planes: Y, Cb and Cr. Do you know how can we differentiate them when we are dealing with uncompressed data?

Second question is is it possible to extract the each frame planes in 3 different images by help of ffmpeg?
The below command extract one video frame per second (-r 1) and output image format is jpeg.

ffmpeg -i input.avi -r 1 -s WxH -f image2 Img-%03d.jpeg

what I want is the output images be separated by each colour plane(Y,Cb,Cr) Naturally the results would be 3 times number of pictures.

[Selur]

afaik ffmpeg's cli can't separate the color planes and output them each as an RGB image.
Should be possible with Avisynth, but you need someone who is more familiar with Avisynth than me for this. :) (Didée&Co should now a way to do this)

[Rouhi]

I dont emphasize on input video format to be AVI, it can be pure h.264 or any other formats.
BTW how can i play the video which are in raw h.264 format generated by previous commands in ffmpeg? The VLC can not play them.

[Selur]

mplayer and ffplay both can play raw h.264 (I would prefer mplayer and don't like vlc ;))

[Rouhi]

I have installed the mplayer and i checked it with some h.264 videos which u shared before, the software play them nicely. but i converted a video to a raw video in format of yuv and then convert the yuv to 264, the problem is that this 264 can not be played by mplayer, it shows many segments of main video and in green colour ... seems like wrong frame rate....

[Rouhi]

I find out that by the below command i can extract Y and U and V plane of every second image of any video frames:

ffmpeg -i input.avi -r 1 -s WxH -f image2 Img-%03d.y

or for just y plane:

ffmpeg -i input.avi -r 1 -s WxH -f image2 -pix_fmt gray Img-%03d.y

do you know how can i view these raw picture formats with extensions *.y, *.u and *.v?

[Selur]

Sorry, last time I looked at such files I used matlab with some self-written routines iirc. and that is far to long ago to remember. :)
(Out of curiosity which calls to you use to extract all the planes?)

Cu Selur

[nm]

i converted a video to a raw video in format of yuv and then convert the yuv to 264, the problem is that this 264 can not be played by mplayer, it shows many segments of main video and in green colour ... seems like wrong frame rate....

Wrong framerate wouldn't mess up frame content like that. There's something else going on. Post the full command-lines used to convert the video to raw YUV and then encode it.

[Selur]

Agree sounds more like a colorspace missmatch,.. Yv12 vs YUV2 vs ....

Cu Selur

[Rouhi]

I dont know what is the problem with my ffmpeg conversion of an input video to YUV. Because when i want to play the generated yuv file (with Mplayer . or YUV player software), i cant see the video correctly. result is like below pic:
http://i53.tinypic.com/2whftzc.jpg

the commands which i used are :
1- ffmpeg -i my.flv out.yuv
2- ffmpeg -v -10 -i my.flv -threads 8 -vsync 0 -an -pix_fmt yuv420p -f rawvideo out.yuv

What is the problem?

[Selur]

What is the problem?
1. ffmpeg -i my.flv out.yuv
you are not specifying the pix_fmt,... so depending on the input colorspace ffmpeg might choose different yuv sub-formats,...

2. ffmpeg -v -10 -i my.flv -threads 8 -vsync 0 -an -pix_fmt yuv420p -f rawvideo out.yuv
looks fine,..

3. How does you mplayer call look like?
do you specify the demuxer, resolution and fps?
it should look like:
mplayer -demuxer rawvideo -rawvideo w=640:h=352:fps=25 out.yuv

[Rouhi]

I finally found the reason of the problem, i made a mistake in frame dimension when i converted the source video to yuv and also in converting to 264 raw video. now every things looks correct.
Thanks anyway for your nice advices...

[Rouhi]

Hi
I need your experience regarding FFMPEG . I want to use FFMPEG functionality in my c codes under windows platform. Would you help me in below questions?
1-How can i use FFMPEG in c to get the dimensions of an input video?
2-How can i use the below ffmpeg command in c language?
ffmpeg -i input.avi -r 1 –f image2 –vcodec rawvideo -pix_fmt yuvj420p Img-%03d.yuv
BTW if i want to run my code on linux machines, can i simply run the exe file generated in windows platform? if not please advice how can i compile my code under linux?

[Selur]

2-How can i use the below ffmpeg command in c language?
For Hybrid (cross-plattform, Linux/Windows/Mac tool I write&use) I use the Qt Framework. It's really simple there to call an external program and capture it's std::out/std::err output.
(you need to write different routines to do this on each system unless you use a framework that does this part for you)

1-How can i use FFMPEG in c to get the dimensions of an input video?
you call ffmpeg -i "Path to input" and capture the output and parse it for a line like
Stream #0.0: Video: mpeg4, yuv420p, 640x352 [PAR 1:1 DAR 20:11], 25 tbr, 25 tbn, 25 tbc
personally I would check if the line contains (Steam, tbr, tbn and tbc) and then grab the resolution out of that line

if i want to run my code on linux machines, can i simply run the exe file generated in windows platform? if not please advice how can i compile my code under linux?
a. you need to create a make file that can be handled by gcc to compile you code on a linux machine (recommend)
b. you could try if you can call your windows exe through WINE (might work)

Cu Selur

Ps.: I can recommend Qt (http://qt.nokia.com/) as an easy to use cross-plattform framework.

[Rouhi]

I can call ffmpeg by :

int analyse(string FilePath)
{

int result;
string com_path;
com_path="c:\\test\\ffmpeg -i " + FilePath;
result = system(com_path.c_str());
return result;
}

This function run the ffmpeg with the video file path as an input parameter.
But the result is on screen. I want to analyse the result to find the dimensions. can i analyse it in my code?

[nm]

I'd call libavformat and libavcodec directly from the C code. There are tutorials around.

[Selur]

You need to check std::out iirc,..

[Rouhi]

I could not get any information by stdout. I can get some information from a running executable file by help of popen(). But the problem is that the command:
ffmpeg -i "path of a video"
is not a correct command of ffmpeg although its result shows the frame size. I am looking for a command of ffmpeg which is only for retrieving frame size and any other general info of an input video.

I'd call libavformat and libavcodec directly from the C code. There are tutorials around.
You are right, there are hundreds of functions in ffmpeg, but the problem is that i cant find my desired functions among them. Can u introduce me a tutorial which shows the functions and their detail informations like input and output and...

[Selur]

I am looking for a command of ffmpeg which is only for retrieving frame size and any other general info of an input video.
ffmpeg doesn't have some sort of info parameter
ffprobe is ment for analysis

[nm]

You are right, there are hundreds of functions in ffmpeg, but the problem is that i cant find my desired functions among them. Can u introduce me a tutorial which shows the functions and their detail informations like input and output and...

Tutorial: http://dranger.com/ffmpeg/
You probably only need to read the first part.

Older tutorial: http://www.inb.uni-luebeck.de/~boehme/using_libavcodec.html

libavcodec example program in the source tree: http://git.libav.org/?p=libav.git;a=blob;f=libavcodec/api-example.c;hb=HEAD
(it doesn't demonstrate libavformat usage though)

Libav API reference is here (http://libav.org/doxygen/master/index.html) (or if you are using the ffmpeg branch instead of libav, here (http://www.ffmpeg.org/doxygen/trunk/index.html)). For example, frame dimensions are in AVCodecContext (http://libav.org/doxygen/master/structAVCodecContext.html).