X264 encoding is going really slow on H.264 source - any ideas to fix?

[lchiu7]

First of all I aill admit at not being a x264 expert at all. I find a decent set of parameters and just use them. Up till now my requirements have been very modest - take some divx content and encode it in AVC so I can play it on my PSP. On my modest AMD X2 3600 under Vista I can get up to 50fps encode speeds.

But lately I have been trying to encode DVB-T HD broadcasts into files taking a bit less space. Original broadcasts are taking 4G/hour and I would like to get them down to about 1.2G. They are in H.264 format and are 720p or 1080i.

I use the following (one pass) set of parameters - the utmost quality isn't important but I am aiming for a certain average bit rate

x264.exe --threads 0 --bitrate 4000 --level 4.0 --bframes 1 --b-rdo --weightb --subme 5 --trellis 1 --analyse p8x8,b8x8,i4x4,p4x4 --me dia --thread-input --progress --no-psnr --no-ssim file.avs

The avs script only has a couple of statements in it

DirectShowSource("file.s")
converttoyv12()

While this works okay it's really slow. In the above hardware I am lucky to get over 2.5fps. Given the encode of the divx material runs so fast is there something in the above parameter string that is causing the speed to be so slow and/or is decoding native H.264 content and re-encoding it just really CPU intensive?

As some additional data, when the encodes are running I am running the Haali Media Splitter to split the ts streams into video and audio elementary streams but that's all.

Short of upgrading to a really grunt machine (but I can't see getting more than 20% increases) are there other strategies I could look at?

Thanks

Larry

[Sagekilla]

Something must be, severely wrong to be getting only 2.5 fps. Using CoreAVC as my decoder, and with a simple DSS().ConvertToYV12() (on a 1080p blu-ray source, too) I can get over 2 fps with extremely slow settings on an Operton 170 (~ Athlon X2 4000+)

There should be no reason your encodes are going that absurdly slow on 720p or 1080i with that fast of settings.

[kemuri-_9]

x264.exe --threads 0 --bitrate 4000 --level 4.0 --bframes 1 --b-rdo --weightb --subme 5 --trellis 1 --analyse p8x8,b8x8,i4x4,p4x4 --me dia --thread-input --progress --no-psnr --no-ssim file.avs

--b-rdo has been removed since r996, so you apparently aren't using an up-to-date version if that's still there

[Dark Shikari]

--b-rdo has been removed since r996, so you apparently aren't using an up-to-date version if that's still thereAnd furthermore even then, it requires subme>=6 to work ;)

[lchiu7]

--b-rdo has been removed since r996, so you apparently aren't using an up-to-date version if that's still there

That might explain it. I also have been using MeGui and always agree to updates. Grabbed the x264.exe from the MeGui tools directory and it has a compile date of 1 Nov 2008. ALso turning off the --b-rdo option and running it again. I am now seeing

avis [info]: 1920x1088 @ 25.00 fps (7740 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Slow
x264 [info]: profile Main, level 4.0
[5.9%] 455/7740 frames, 3.00 fps, 4126.04 kb/s, eta 0:40:26

which seems better.

I guess the only thing now I can do is get a faster CPU(s)

Thanks

[kemuri-_9]

That might explain it. I also have been using MeGui and always agree to updates. Grabbed the x264.exe from the MeGui tools directory and it has a compile date of 1 Nov 2008.


according to sharktooth's regular updates in the daily builds thread,
megui should have r1029 right now, which was released more around Nov 20th/21st (depending on your timezone)

can check the version by doing x264 --version (checking the version is more reliable than going by creation/mod dates)

[Dark Shikari]

That might explain it. I also have been using MeGui and always agree to updates. Grabbed the x264.exe from the MeGui tools directory and it has a compile date of 1 Nov 2008. ALso turning off the --b-rdo option and running it again. I am now seeing

avis [info]: 1920x1088 @ 25.00 fps (7740 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Slow
x264 [info]: profile Main, level 4.0
[5.9%] 455/7740 frames, 3.00 fps, 4126.04 kb/s, eta 0:40:26

which seems better.

I guess the only thing now I can do is get a faster CPU(s)

ThanksYou could also use faster settings ;)

But yes, an Athlon 64 is pretty slow. And HD is hard.

[lchiu7]

You could also use faster settings ;)

But yes, an Athlon 64 is pretty slow. And HD is hard.

What would be some faster settings if I didn't care so much about quality - just want to compress these files but maintain something better than SD.

I guess quad core Intel might be next :)

doing --version I get

x264 0.65.1016M dbc5ef0

[Dark Shikari]

What would be some faster settings if I didn't care so much about quality - just want to compress these files but maintain something better than SD.

I guess quad core Intel might be next :)

doing --version I get

x264 0.65.1016M dbc5ef0Here's the fastest x264 settings you can possibly use, if I didn't forget anything:

--me dia --subme 0 --scenecut=-1 --threads auto --partitions none --no-cabac --aq-mode 0

This will be really fast. Really fast: 111fps on one core of a 2Ghz Core 2 for a DVD source, for example. You will be very input-bottlenecked.

Here's slightly more reasonable fast settings:

--me dia --subme 1 --no-cabac --partitions none --threads auto --scenecut=-1

And here's more balanced fast settings:

--subme 3 --8x8dct --threads auto --bframes 3

[lchiu7]

Here's the fastest x264 settings you can possibly use, if I didn't forget anything:

--me dia --subme 0 --scenecut=-1 --threads auto --partitions none --no-cabac --aq-mode 0

This will be really fast. Really fast: 111fps on one core of a 2Ghz Core 2 for a DVD source, for example. You will be very input-bottlenecked.

Here's slightly more reasonable fast settings:

--me dia --subme 1 --no-cabac --partitions none --threads auto --scenecut=-1

And here's more balanced fast settings:

--subme 3 --8x8dct --threads auto --bframes 3

Cool, thanks. Got up to 7fps on the first one and you did say HD is hard and my CPU is not that fast. The other constraint that I didn't mention that might be a factor (though given the fps rate not sure it would), the HD source is coming off a network drive. Guess I could move it locally but not sure that would make such a difference as the parameters you supplied would

[refulgentis]

Cool, thanks. Got up to 7fps on the first one and you did say HD is hard and my CPU is not that fast. The other constraint that I didn't mention that might be a factor (though given the fps rate not sure it would), the HD source is coming off a network drive. Guess I could move it locally but not sure that would make such a difference as the parameters you supplied would

it sounds like it would make a huge difference
EDIT: never mind, for some reason I was thinking you had a core 2 duo

[Audionut]

I would copy the file to a local drive. Vista would be using way to many resources with the file coming from the network.

[pcordes]

I use the following (one pass) set of parameters - the utmost quality isn't important but I am aiming for a certain average bit rate

x264.exe --threads 0 --bitrate 4000 --level 4.0 --bframes 1 --b-rdo --weightb --subme 5 --trellis 1 --analyse p8x8,b8x8,i4x4,p4x4 --me dia --thread-input --progress --no-psnr --no-ssim file.avs


x264's CRF mode won't give you a reliable bitrate, but it's in the same ball park most of the time. If you're going to put your vids onto 4.4GB DVDs or something, use crf and then fill up the remaining space on the DVD with error-recovery data from dvdisaster or par2.

Also, as other's have said, you didn't exactly pick the best x264 settings. You used some slow ones that improve quality, but some low quality options, too. You could have got the same quality at much faster encoding speed if you'd picked more balanced options. I'm not sure if there is an up-to-date table of best quality-speed tradeoffs in order of increasing quality. DS posted a table a year ago
http://forum.doom9.org/showthread.php?t=130786
It's probably a little stale: p4x4 seems to have fallen from favour, and most people say it's not usually helpful except at low rez and high bitrate (in high rez vids, important details aren't usually that small, so p8x8 blocks can handle them ok, as I understand it.) Since p4x4 slows down the encode, so probably just leave --partitions at the default, instead of using -A all.

There are guides to x264 settings, though.
http://mewiki.project357.com/wiki/X264_Settings is the officially recommended source for more than most people ever want to know about each x264 option. It's fairly up-to-date, too, but it doesn't tell you things like: --me umh is probably a more cpu-efficient way to gain quality than --trellis 1. (I think that's true: DS turned on trellis with hex. But not dia.) Probably --me hex (the default) --trellis 0 would be faster and higher quality than --me dia --trellis 1. Definitely don't turn on --me tesa unless you already have everything else cranked up, since on most videos it barely helps any beyond --me umh. Also, you probably want to turn up --refs to 2 or 3 before increasing motion-search too much. DS turned up the refs pretty early compare to --subme and --me, which is interesting. I've always been curious about exactly what kinds of vids benefit most from multiple refs. I assume ones with strobe effects or flickery lighting will benefit a lot. And ones where the background looks the same after something is done obscuring it for several frames. I guess I'm a big fan of multiple refs. :p I was also interested in seeing --me umh with reduced merange. Interesting to see how that compares to --me hex. (DS probably didn't intend that table for this much scrutiny, or as anything definitive, so I'll stop now. A table like this should go into a wiki somewhere and be kept up to date, since it would help a lot of people see what order they should turn on options in. Conversely, a lot of people don't seem to know what combo of options to use, and end up making bad choices that are slow and low quality. :( )

Note that --subme numbers have been re-mapped to include bime (on at subme >= 5) and b-rdo (on at subme >= 7). See the mewiki guide for how to map the old numbers (from DS's table) to new numbers.

BTW, someone please correct me if I'm wrong about anything. While I was writing this, I realized that I haven't actually tested most of my assumptions about which options help more.