x264 can't work with video buffer?!

[bartnovsky]

I am trying capture and encode in real time with x264 video from a webcam. Just for reasons of curiosity i feed x264 (through avisynth) with a really big video stream of ~500 mbps (and want an output of ~1 mbps). I stack few same pictures from the webcam to get an almost full hd resolution 1920x960.

Here is what i see:

during encoding (and capturing) x264 says that it works at ~13fps and that video stream is 1920x960@25fps. Say i want to capture 250 frames which is 10 sec. And it turns out that the capture (and encoding as well) itself lasts for ~20 sec (!!!). The output file is exactly 10 sec in length and the motion there is twice as fast as it should be... That was really suprizing for me)

Perhaps i am unduly ambitious or my pc is too slow, but once x264 is incapable of encoding in real time, why it doesn't keep unencoded frames in some buffer? Each frame of raw video is ~3mb, the whole video is ~625mb and it can entirely be placed in my RAM, though while encoding i haven't seen for x264 to take more than 400mb of RAM.

I don't have illusions that someone will change code of x264 due to my caprices. And x264 works really great and fast (as compared to others) with files, but not with streams. Maybe i should take care of that myself. In linux it's simple - "mkfifo" and the buffer is ready for work, but how to make it under win32?

[Dark Shikari]

All x264 does is take frames in, and give frames out. It cannot drop frames, and for every frame that goes in, a frame comes out.

If it appears frames are being dropped and the video is "faster," that's not being done by x264, it's being done the frames reach x264.

[ajp_anton]

What happens if frames come in faster than x264 can process?

[LoRd_MuldeR]

What happens if frames come in faster than x264 can process?

This can't happen:

In order to feed in the next frame the host application calls the corresponding function from libx264. x264 won't return from that call, until it's ready to accept yet another frame.

And in the case of the x264 CLI encoder, we use Avisynth input. So x264.exe is in complete control of when it requests which frames from the source AVS script.

If we use a pipe, x264.exe again is in control of when it reads a frame from the pipe buffer. The call to read the next frame from the buffer will block, if there's no next frame yet.

The same way the call to put another frame into the pipe buffer (from the source application) will block, if the buffer is currently completely full...

[ajp_anton]

So if I have
a) Live video -> avisynth -> x264 CLI
b) Graph: Live video -> x264vfw
Are there always buffers before bottlenecks (x264), how big are they and how can I control them?

[LoRd_MuldeR]

a) How do you get "Live Video" into Avisynth? Each AVS script starts with some XYZSource() function. Usually these source functions are required to be seek-able, in a frame-accurate way.

Note that Avisynth source functions don't "push" frames into the script. Instead the script requests frames from the function as needed. If at all, the LiveSource() would have to take care of buffering itself.

I even think LiveSource() would need to buffer ALL frames internally (e.g. using a Temp file), because any source frame might be requested again by the script at a later time...


b) The VFW interface uses a strict "one source frame in, one encoded frame out" method. You call the function to encode exactly one frame and that function blocks until it returns the encoded frame.

So whatever application calls the x264 VFW Codec must deal with buffering itself. I think the same applies (more or less) when working with the libx264 natively...

[roozhou]

How do you capture from a webcam with avisynth, directshowsource?
Avisynth requires fixed fps and fixed length of video before processing.
If x264 can only encode at 13fps, directshowsource will drop frames but avisynth still assumes it is 25fps. This can explain why your encoding lasted 20 sec and your video played twice as fast.

Try my dshow based x264 tools. No matter how fast your encoding goes, you will always get a video in normal speed.
http://forum.doom9.org/showthread.php?t=141441

[ajp_anton]

a) Actually this also uses a graph, and it ain't pretty... directshowsource(".grf",framecount=[large number]). I don't use this method.
So I'm guessing that x264 is just requesting frames when it needs them, and it will get whatever the current frame is in the livestream?

b) Here I'm using Graphstudio to split the stream into preview and x264vfw+avimux. I have no idea how this works in detail, I only see the video following this path I've set up. Can there for example be multiple frames in the same graph, only at different "steps"?
And what is calling x264vfw here, Graphstudio or the next/previous step in the graph?

[LoRd_MuldeR]

a) Actually this also uses a graph, and it ain't pretty... directshowsource(".grf",framecount=[large number]). I don't use this method.
So I'm guessing that x264 is just requesting frames when it needs them, and it will get whatever the current frame is in the livestream?

x264 requests frame X from Avisynth when it needs that frame. At that moment Avisynth will request frame X from DirectShowSource().

And DirectShowSource() will deliver whatever the DirectShow graph returns for that frame. Any buffering or skipping would need to happen on the DirectShow side.

It's not under control of x264 what happens. It's not even under the control of Avisynth, I think.


b) Here I'm using Graphstudio to split the stream into preview and x264vfw+avimux. I have no idea how this works in detail, I only see the video following this path I've set up. Can there for example be multiple frames in the same graph, only at different "steps"?

It seems you are mixing up DirectShow and VFW. GraphStudio is for DirectShow filters, not for VFW Codecs, isn't it?


And what is calling x264vfw here, Graphstudio or the next/previous step in the graph?

There is always a host application that calls x264 VFW. For example VirtualDub (most likely VirtualDub in fact ^^).

The host applications calls the VFW Codec many times. It provides exactly one source frame per call. And each calls blocks, until the encoded frame is returned. This is how VFW works.

If there is any buffering or frame skipping going on for "Live" input, this would need to happen inside the host application, I think...

[roozhou]

It's not under control of x264 what happens. It's not even under the control of Avisynth, I think.


This is the limitation of avisynth. Avisynth cannot handle "push" mode source filters well until it becomes time based(perhaps never).

@ajp_anton
I suggest you first store your captured frames in uncompressed format(or lossless codecs like lagarith), then encode them with x264. In this way you never gets dropped frame.

[ajp_anton]

It seems you are mixing up DirectShow and VFW. GraphStudio is for DirectShow filters, not for VFW Codecs, isn't it?Well x264 appears in the encoders-list in Graphstudio after I install the vfw.
@ajp_anton
I suggest you first store your captured frames in uncompressed format(or lossless codecs like lagarith), then encode them with x264. In this way you never gets dropped frame.I've been using lagarith for lossless compression, but going to try x264 (lossless) for smaller filesizes -> longer captures.

[roozhou]

I've been using lagarith for lossless compression, but going to try x264 (lossless) for smaller filesizes -> longer captures.

You should switch to faster x264 settings and use a bit higher bitrate so that your encode can catch up with captures without quality loss.

[ajp_anton]

You should switch to faster x264 settings and use a bit higher bitrate so that your encode can catch up with captures without quality loss.The problem isn't x264 being slow, it's that if I browse folders etc, I'm afraid I might do something that causes a spike in CPU usage.

[roozhou]

The problem isn't x264 being slow, it's that if I browse folders etc, I'm afraid I might do something that causes a spike in CPU usage.

Set the priority of your encoding process to High.

[Chengbin]

Set the priority of your encoding process to High.

I seriously don't recommend that. Since x264 will always max out your CPU (unless you're doing first pass with b-adapt 2), even dragging the mouse will become laggy.

[roozhou]

I seriously don't recommend that. Since x264 will always max out your CPU (unless you're doing first pass with b-adapt 2), even dragging the mouse will become laggy.

It's different because he's doing real-time encoding.
That's why I also suggest he use faster settings.

[bartnovsky]

I gained that during real-time encoding using x264 real fps slightly deviates from capture fps (25). This leads to lag audio from video stream, because it plays back with normal framerate.
I would like to solve this problem by adjusting timecodes in *.mkv output file. Tuning of playback framerate is acheved by inserting the timer in corresponding function responsible for frame timecodes. Attached you will find some details of corrections to source code (snapshot from July 10th was used). Could anyone help to make a multithreaded build for win32 with this patches??

[roozhou]

I would like to solve this problem by adjusting timecodes in *.mkv output file. Tuning of playback framerate is acheved by inserting the timer in corresponding function responsible for frame timecodes. Attached you will find some details of corrections to source code (snapshot from July 10th was used). Could anyone help to make a multithreaded build for win32 with this patches??
So you get the timestamp from system time?

DirectShow gives correct timestamp for each frame but avisynth just drops it. Again direct264 (http://forum.doom9.org/showthread.php?t=141441) has already solved this problem. If you need to do it yourself, please take a look at my source code.

[bartnovsky]

Yes, i use timestamps from system time.
I have tried your direct264 build with my capture avs-script. Timecodes obtained via mkvextract are the same as for 25 fps (0, 40, 80....) without any corrections for realtime (2>error.txt outputs variable framerate). Audio stream will lag from video again. When will the point from your wishlist concerning .grf-files be realized?

[roozhou]

When will the point from your wishlist concerning .grf-files be realized?

It's not in the wishlist. It's already there:)

Don't encode from an avs. Plz directly encode from an .grf file.
And remember to leave the video renderer in the .grf.

[bartnovsky]

Thanks a lot, but the problem is not solved in my case. I continue to obtain cfr timecodes in .mkv-file. May be I should use special command-line options for it?

[roozhou]

What's your command-line options?
I tested encoding from a simple graph with my webcam (USB Video Device -> ffdshow video decoder -> video renderer). It works fine.

[bartnovsky]

x264.exe -B 1500 -f 0:0 -A "none" -b 3 --no-chroma-me
--me "dia" -m 1 --merange 16 -I 250 --min-keyint 20 --ipratio
1.40 --qcomp 0.6 --qpmin 10 --qpmax 51 --qpstep 4 --ref 1
--mixed-refs --direct "temporal" --trellis 0 --non-deterministic
--threads 0 --frames 2000000 -p 1 --stats "x264_2pass.log" -o
test.mkv video.grf
I am using BlackMagic Intensity Pro capture card.
As i saw in your sources, implementation of set_eop_mkv() there was the same as in "official" snapshots from ftp://ftp.videolan.org/pub/videolan/x264/snapshots/. With this builds I have delay 3 seconds per hour for audio (because video is played back faster than captured)
And I have rebuilt your sources with amendments attached above. But nothing was changed. It seems like neither SMPlayer nor VLC don't "feel" this timecodes, but it is correctly extracted by mkvextract. Also multiplicating this timecodes on any scaling factor don't influence on the rate of video playback.

[roozhou]

As i saw in your sources, implementation of set_eop_mkv() there was the same as in "official" snapshots from ftp://ftp.videolan.org/pub/videolan/x264/snapshots/.

Yes, but set_eop_mkv gets p_picture->i_pts from the encoder, and i_pts = dshow_timestamp * fps_num / 10000000.