x264 realtime output parsing

[lucassp]

Hi guys,

I'm trying to run a bash script, which triggers an x264 encode, and I want to parse the text output in realtime and return the text to the an internet browser in some way.

Is this possible? I mean can I run a PHP script (or anything else) from bash only when the output of the x264 console gets updated. So I can write the output to a text file, parse it with php, and return it to the browser using AJAX.

Thank you,
Lucian

[LoRd_MuldeR]

See this thread:
http://forum.doom9.org/showthread.php?t=156292

Especially the replies here:
* http://forum.doom9.org/showpost.php?p=1427899&postcount=5
* http://forum.doom9.org/showpost.php?p=1428001&postcount=9

About the "web" specific stuff: I guess you'll have to update the content (progress indicator) of the web page with JavaScript+XMLHttpRequest in regular intervals.
But this will require some application/service running on your server that can answer the current encoding progress on request.
So probably you should first write the server part ("backend") that will invoke/monitor the x264 process. Then move forward to design the web-interface ("frontend").
I'd imagine two threads running in parallel in the backend process: One that continuously reads/parses the x264 output and one that answers the HTTP requests.

[lucassp]

Thank you!

I'm comfortable with a PHP backend. I was thinking of running a bash script which starts my encode and using a named pipe to pass the output to another bash script which parses the text and calls a php cli script which updates a database. Using another PHP script and an ajax request I will update the web based browser interface.

I know this is not the perfect solution but any other suggestions and recommandations are welcome.

Thanks again!

[LoRd_MuldeR]

I'd write a program, probably in C++ or Pascal (or C# if you like), which invokes the x264 process with stderr redirection and which also does the parsing in "real time".

You will find a lot of sample code for that, even when not familiar with C++/Pascal. See the links in my above post.

Actually it would be sufficient to write the current status to a simple text file in regular intervals. Then the PHP script to generate the answers for your AJAX request will simple read out the text file.

Maybe there are better (more sophisticated) methods to pass the status to the PHP script, but I assume this method is very straight forward and easy to implement...

[lucassp]

I've started coding something :)

But I'm having some trouble getting something from the x264 stderr output:

./x264 --bitrate 300 --preset veryfast -o video.mp4 video.f4v 2>&1 | grep frames

...and I get a blank screen. All I want to get are the lines containing "frames", including the last line where x264 reports its progress.

./x264 --bitrate 300 --preset veryfast -o video.mp4 video.f4v 2>&1 | grep fps

...this gets only gets me the ffms info line:

ffms [info]: 640x474p 1:1 @ 30000/1001 fps (vfr)

Thank you!

[LoRd_MuldeR]

But I'm having some trouble getting something from the x264 stderr output:

./x264 --bitrate 300 --preset veryfast -o video.mp4 video.f4v 2>&1 | grep frames

...and I get a blank screen. All I want to get are the lines containing "frames", including the last line where x264 reports its progress.

That looks correct to me. However I'm not sure if 'grep' is suitable to parse the x264 status output in realtime.

I assume grep waits for the line to end (linebreak character), before it parses/processes that line. However while x264 prints progress there won't be any linebreak.

You'd need something that works like my tool here:
http://forum.doom9.org/showthread.php?p=1417496#post1417496

[lucassp]

Good point! Thanks!

Are there other any *nix commands which can grab it?

[lucassp]

I've tried something:
./x264 --bitrate 300 --preset veryfast -o video.mp4 video.f4v 2>log.txt | ./test.sh

and test.sh is:
#!/bin/bash
while sleep 1; do
line="$(awk '/frames/' log.txt)";
echo $line;
echo ${line:1:5};
done

and the output is:
[1.2%] 108/9105 frames, 118.63 fps, 350.51 kb/s, eta 0:01:15
0.1%]
[4.3%] 396/9105 frames, 202.98 fps, 312.68 kb/s, eta 0:00:42
0.1%]
[7.5%] 684/9105 frames, 231.92 fps, 316.98 kb/s, eta 0:00:36
0.1%]
[10.8%] 981/9105 frames, 245.92 fps, 313.49 kb/s, eta 0:00:33
0.1%]
[14.0%] 1278/9105 frames, 254.81 fps, 308.53 kb/s, eta 0:00:30
0.1%]
[16.9%] 1539/9105 frames, 256.31 fps, 308.50 kb/s, eta 0:00:29
0.1%]
[19.6%] 1782/9105 frames, 251.07 fps, 312.16 kb/s, eta 0:00:29
0.1%]


I'm just wondering: why does the substring always output 0.1%?