An approach to make x264 work faster

[bartnovsky]

I am into a project about encoding a very peculiar video content: a person in a classroom gives a talk on a blackboard (that might be on a scientific conference, at school etc.). We record it using a Full HD camera and encode using h.264.

As a rule the moving person takes a small fraction (ab. 1/6) of the whole area (1920x1080). So that effectively the picture is almost statical. My simple observation is that if i encode a 10sec sample (1920x1080) from 90mbps to 1mbps and it takes say 20 sec, than encoding the cropped sample (640x500) would take just 7 sec. Amazing!) Hence, using a simple motion detect i could encode such a video 2-3 times faster. So my question is:

is there a way to attach some "motion detect" on a part of video (GOP say of 10 - 250 frames) in order to say to my x264 (or whatever) that a shadow on the blackboard is not of primary concern and is below some "threshold" of motion, but the moving person is above this threshold (i just need a rectangular area with him, not a shape of his haircut or something).

http://forum.doom9.org/attachment.php?attachmentid=9568&stc=1&d=1236358396

Somehow (if i understand it correctly) x264 seeks for motion (where it is irrelevant for me) and expects a warms to appear from blackboard suddenly))) And it takes a lot of pc's resources. That's a fact. Here (http://www.codeproject.com/KB/audio-video/Motion_Detection.aspx) is an open code on C# for motion detect that seems to me quite useful. But i am not sure if the problem could be (or already was) solved by somebody. Is there any plugin for mencoder/ffmeg/x264 etc. to do it?

Thanks in advance for any help!

[Dust Signs]

If the person only occupies 1/6th of the screen why not focus this person instead of recording 5/6th of wall/floor etc. which is irrelevant anyways? I don't see the point in that. You record, save and process a massive amount of data which will not be used in the first place. As the picture you attached has not yet been aproved, I may be missing the point, but at the moment I really don't see any reason for an approch which is that complex to encode content which could be recorded differently :confused:

Dust Signs

[nm]

The (mostly) static blackboard contents are very important in science/math lectures. I think bartnovsky's idea is quite interesting.

[Dark Shikari]

Fast P-skip already does this. Odds are, however, that the background isn't actually static, just close to static, so it doesn't actually get entirely skipped.

[bartnovsky]

If the person only occupies 1/6th of the screen why not focus this person instead of recording 5/6th of wall/floor etc.


That's the point: these 5/6th part of area is irrelevant in a level of fast motion only. But indeed i want to watch and see formulas etc. that appear on the blackboard during the lecture. An observation is that a usual professor (within say 10 sec) walks in a very small part of image. And it'd be natural to get advantage of this.

Fast P-skip already does this. Odds are, however, that the background isn't actually static, just close to static, so it doesn't actually get entirely skipped.

If i am not mistaken, Dark Shikari, fast P-skip is used by default in x264. But it's boolen. I can only set it to "--no-fast-pskip". Obviously there are non-idealities in almost static image (which is not actually static) in any video due to a noise of camera, of capture device...

I find my idea to use external "motion detect" not very wise. Because x264 anyway absorbs huge amount of sophisticated algorithms and it is so strange and pity that it does not have some threshold within "fast P-skip" option...

Is there a way to improve on x264 to utilize its own motion detect???

[Dark Shikari]

it does not have some threshold within "fast P-skip" option... Yes it does. It skips in the case of a motion vector within 1 quarter-pixel position of the predicted vector, a residual SATD of under 300, and no remaining coefficients after quantization/decimation. If this decision is made, the analysis instantly terminates.
Is there a way to improve on x264 to utilize its own motion detect???x264's motion detection is already quite good, probably much better than anything you would write externally. If you want to let x264 skip things more, you might want to try an external denoiser like hqdn3d. If you want to raise the skip threshold, you can practically do this by raising the deadzone values.

[Chengbin]

I would like to say something as well about the static scenes.

In many animes like Family Guy and American Dad, there are often static (like true static scenes) that occurs for a significant amount of time.

Dark Shikari, you said something about raising deadzone values to skip more. Would this help with compression in encoding very static (and simple) videos? Is it a high profile feature? If I change the value from false, what value do I put in there?

[smok3]

if there really is no motion x264 should do a terrific job (i did some screen recordings with crf, that came our like 118 kbps, this one for example http://somestuff.org/video_guides/all/flvplayer.php?moviename=movies/photoshopBasicSeparatedMask-x992y784.mp4 )

now, staring at your screenshots, i would;

a. top half of the screen is completely useless, so crop that away...
b. use some temporal filter (ff3dgpu for example)
c. remove the chroma if it is not needed?

[Sagekilla]

Like smok3 said, your best bet would be to crop away sections that are not needed. Most people crop away the black bars on movies with ARs > 16:9, so why not do the same here?

[cogman]

Here's a dumb question, X264 has pretty good motion compression, but would it be possible to identify the part of the film that isn't changing at all, create a sort of picture overlay, and move the moving part into position. (So, kindof like adding boarders that look exactly like the surrounding area) That should remove any grain problems and allow x264's motion detection to very quickly determine the rest.

Of course, if the camera moves at all then that wont work very well, It probably is best to just crop off what you don't need.

[LoRd_MuldeR]

Here's a dumb question, X264 has pretty good motion compression, but would it be possible to identify the part of the film that isn't changing at all, create a sort of picture overlay, and move the moving part into position. (So, kindof like adding boarders that look exactly like the surrounding area) That should remove any grain problems and allow x264's motion detection to very quickly determine the rest.

In fact that's what x264 does already, but in a more sophisticated way. If an area doesn't change, it will be predicted perfectly from the previous frames and no data needs to be stored for these areas. However noise is a problem, as it causes static areas to not be 100% static. If you raise the threshold enough so that "noisy" static areas are treated as 100% static areas, that probably would lead to an ugly border between the static and none-static areas. Solution for this, as suggested before: Instead of raising the threshold, remove the noise before encoding, so static areas become truly static...

[ajp_anton]

Encoding the noise as a static picture will make the "edges" to that area to look ugly. Plus you will probably see the text better with the noise moving than not.
Removing the noise may cause damage to the text.

To speed up motion search, why not just use a very low merange? Or do you expect anything faster than this guy's hand on the screen?

[Dark Shikari]

Encoding the noise as a static picture will make the "edges" to that area to look ugly. Plus you will probably see the text better with the noise moving than not.
Removing the noise may cause damage to the text.

To speed up motion search, why not just use a very low merange? Or do you expect anything faster than this guy's hand on the screen?The vast majority of calls to hex/dia motion search (not applicable to umh and above) terminate after only a couple iterations, regardless of merange.