If I don't care about encoding time, can I improve this setting?

[Chengbin]

Here is my setting to encode anime for my Archos 5. I have some limits to settings due to the decoder, I can't use high profile, maximum b frame is 4, maximum ref frame is 6. If I don't care about encoding time, is there any other setting I can improve?

cabac=1 / ref=6 / deblock=1:-1:-1 / analyse=0x1:0x110 / me=umh / subme=9 / psy_rd=1.0:0.0 / mixed_ref=1 / me_range=32 / chroma_me=1 / trellis=2 / 8x8dct=0 / cqm=0 / deadzone=21,11 / chroma_qp_offset=-2 / threads=6 / nr=0 / decimate=1 / mbaff=0 / bframes=4 / b_pyramid=1 / b_adapt=2 / b_bias=0 / direct=3 / wpredb=1 / keyint=1000 / keyint_min=25 / scenecut=40(pre) / rc=2pass / bitrate=449 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / vbv_maxrate=25000 / vbv_bufsize=14000 / ip_ratio=1.40 / pb_ratio=1.30 / aq=1:0.10

[Atak_Snajpera]

me=tesa

[LoRd_MuldeR]

Use more ref frames. Use 8x8dct. Use more b-frames, although more than 4 will rarely be helpful. Maybe play with Psy Trellis.

[Dark Shikari]

partitions=all, and IMO AQ a bit higher than 0.1 would help.

[Chengbin]

me=tesa

Does that improve compression a lot? I hope it is not like p4x4 where it only improve compression by 0.5% while doubling encoding time. I still care somewhat about encoding time, but I won't double the encoding time to improve compression by 1%.

Dark Shikari: Can you point out where "partitions" is located in the setting, I'm a bit blind, sorry. Also, can you give me a rough estimate of improvement of compression and how much more time used using me=tesa and partitions=all? Thanks a lot.

[Atak_Snajpera]

Does that improve compression a lot? I hope it is not like p4x4 where it only improve compression by 0.5% while doubling encoding time. I still care somewhat about encoding time, but I won't double the encoding time to improve compression by 1%.

Also, can you give me a rough estimate of improvement of compression and how much more time used using me=tesa and partitions=all?

If I don't care about encoding time, can I improve this setting?
??????

[LoRd_MuldeR]

Does that improve compression a lot? I hope it is not like p4x4 where it only improve compression by 0.5% while doubling encoding time.

I highly doubt that p4x4 will double the encoding time!

It doesn't help much though, except for low resolutions and very high bitrates. So usually you can turn it off safely and safe a few CPU cycles.

But you asked for SLOW options, so I'd enable p4x4. Options like "--me esa" or even "--me tesa" will cause a much bigger slowdown...

Can you point out where "partitions" is located in the setting, I'm a bit blind, sorry.

The "--partitions" parameter is an alias for "--analyze". Also note that "all" includes "p4x4".

BTW: Is there any particular reason why you lowered the VAQ strength to 1/10 of the default value ???

[Chengbin]

I highly doubt that p4x4 will double the encoding time!

It doesn't help much though, except for low resolutions and very high bitrates. So usually you can turn it off safely and safe a few CPU cycles.

But you asked for SLOW options, so I'd enable p4x4. Options like "--me esa" or even "--me tesa" will cause a much bigger slowdown...



The "--partitions" parameter is an alias for "--analyze". Also note that "all" includes "p4x4".

I think I said I can't use high profile.

OK, I'm just making an example. I just knew that p4x4's improvement is not worth the extra time.

Does "me tesa" double (or come close to) the encoding time?

[Atak_Snajpera]

Does "me tesa" double (or come close to) the encoding time?
more or less

[LoRd_MuldeR]

Using "--me tesa" together with "--merange 32" will be REALLY slow. Probably not worth the improvement.

But initially you said that you didn't care about encoding time, so ...

[Chengbin]

Do they affect decoding? Does it take more CPU power to decode a video encoded with merange 32 and me tesa?

If they don't, I'll go ahead with this. I got lots of time. I hope these two options can make good use of my quad core.

BTW, is "me tesa" useful with movie content?

[LoRd_MuldeR]

Do they affect decoding? Does it take more CPU power to decode a video encoded with merange 32 and me tesa?

No, I don't think so.

BTW, is "me tesa" useful with movie content?

Depends on what you consider "useful".

As said before: The speed loss for "--me tesa" probably isn't worth the gain in quality for the average user.

But you explicitly stated that speed doesn't matter, so it may be useful for you...

[cogman]

No, I don't think so.



Depends on what you consider "useful".

As said before: The speed loss for "--me tesa" probably isn't worth the gain in quality for the average user.

But you explicitly stated that speed doesn't matter, so it may be useful for you...

I thought I heard somewhere that less threads = more quality. So if encoding time doesn't matter, slap that thing down to one thread instead of 6.

As others have said, tesa doesn't effect decoding time/speed. (well, I guess it does cause some bits to shift around in some manner which would technically change the speed, but it wouldn't be noticeable)

Doesn't trellis 1 yield better quality then trellis 2?

[Ranguvar]

Less threads _may_ affect quality - if you get very insane with it, meaning above 12 or so I would guess. And even then, minor enough that even stuff like --me tesa would be smarter than wasting cores with low threads.

Trellis 1 _used_ to be better, but not any more.

[LoRd_MuldeR]

I thought I heard somewhere that less threads = more quality. So if encoding time doesn't matter, slap that thing down to one thread instead of 6.

There are cases where a huge number of threads hurts the quality. But the effect should be so small that is negligible!


As others have said, tesa doesn't effect decoding time/speed. (well, I guess it does cause some bits to shift around in some manner which would technically change the speed, but it wouldn't be noticeable)

TESA is simply a slower (more accurate) motion estimation algorithm.

I think the output will differ between TESA and faster ME methods (if it didn't there would be no point for TESA), but not in a way that effects decoding speed.

The decoder will only "see" the resulting MV's, but it doesn't care which algorithm had been used by the encoder to find these MV's.


Doesn't trellis 1 yield better quality then trellis 2?

Back in the times when there was no Psy RDO/Trellis, people reported that Trellis 2 tends to kill fine detail. But this should no longer be the case with recent versions!

BTW: Psy RDO is on by default, Psy Trellis is not. One may want to play with Psy Trellis for retaining even more detail.

[Dark Shikari]

Back in the times when there was no Psy RDO/Trellis, people reported that Trellis 2 tends to kill fine detail. But this should no longer be the case with recent versions!AQ probably had a lot to do with it also, as the problem with trellis 2 was it tended to decimate more details in flat areas--a problem that AQ resolved.

[cogman]

There are cases where a huge number of threads hurts the quality. But the effect should be so small that is negligible!




TESA is simply a slower (more accurate) motion estimation algorithm.

I think the output will differ between TESA and faster ME methods (if it didn't there would be no point for TESA), but not in a way that effects decoding speed.

The decoder will only "see" the resulting MV's, but it doesn't care which algorithm had been used by the encoder to find these MV's.




Back in the times when there was no Psy RDO/Trellis, people reported that Trellis 2 tends to kill fine detail. But this should no longer be the case with recent versions!

BTW: Psy RDO is on by default, Psy Trellis is not. One may want to play with Psy Trellis for retaining even more detail.

Well, the difference between TESA and UMH is about the same as going from 6 to 1 core. If the guy wants all out quality, then that is one of the speed risks he could take (all be it, with pretty low yields in quality increases, much like going from 6 reference frames to 16 in live action video)

Dang it though, now I going to have to play with trellis and psy-RD to see what looks the best.

[Chengbin]

Turns out me=tesa and merange=32 wasn't as slow as I imagined. I'm getting about 43 fps for first pass and 18 fps for the second pass.

Is it normal for x264 to only use 40% of my CPU in the first pass and 95% of my CPU in the second pass?

[LoRd_MuldeR]

Well, the difference between TESA and UMH is about the same as going from 6 to 1 core.

No so sure about that. Going from UMH to TESA will give a small improvement at least.

While going from 6 to 1 core reduces encoding speed to ~1/4 on a Quadcore for usually no benefit, except for extremely rare cases maybe...

[IgorC]

I'm not sure but how about 3d pass. It can improve bitrate distribution. It should be on par with --tesa imrpovement (if not better).
Quality wise settings for first pass also good for final quality.

[Soichiro]

Is it normal for x264 to only use 40% of my CPU in the first pass and 95% of my CPU in the second pass?

Yes, because b-adapt 2 (which only runs in the first pass) isn't fully multithreaded, so it won't use all of your cores in the first pass.

I'm not sure but how about 3d pass. It can improve bitrate distribution. It should be on par with --tesa imrpovement (if not better).

I've only heard of 3rd pass giving improvements on extremely short videos, and have actually heard of it slightly hurting quality in some cases, so overall I think it would be a waste of time unless these videos are around 20 seconds long or something, and wasting time on --me tesa would be better.

[Dark Shikari]

Yes, because b-adapt 2Stop spreading this misinformation. B-adapt isn't threaded, period. Nothing to do with the "2".

[LoRd_MuldeR]

Stop spreading this misinformation. B-adapt isn't threaded, period. Nothing to do with the "2".

But the fact that "B-adapt isn't threaded" may become a bottleneck with "2" as it's much slower (and better) than "1". At least when you use a big number of b-frames.

It's not a big problem when using a reasonable number of b-frames, like 3-4. Anyway Chengbin stated that encoding time doesn't matter...

[Chengbin]

OK, 1080p to 480p using those settings are not fun. Took 7 hours to encode a 100min movie.

[Sagekilla]

That's... what 6 fps? I wish I could encode that fast. Most of mine take around 30 hours to complete. (1080p though)

[Atak_Snajpera]

480p not 1080p!

[Chengbin]

That's... what 6 fps? I wish I could encode that fast. Most of mine take around 30 hours to complete. (1080p though)

It is 1080 to 480. 7.6 fps for the second pass. BTW, I'm encoding in main profile, not high.

I've never successfully encoded 1080p to 1080p, because x264 keep crashing on me for some unknown reason. I only tried in AutoMKV's slug profile (basically every setting maxed out) and first pass takes 15 hours (it fails at like 70-80% point). I quit after a few tries. CRF works for some reason.

[Sagekilla]

Hm, do you still get those results with recent revision of x264? There should be no issues with encoding regardless of what resolution you use. x264 handled up to 4096x2304 flawlessly for me.

[Chengbin]

Hm, do you still get those results with recent revision of x264? There should be no issues with encoding regardless of what resolution you use. x264 handled up to 4096x2304 flawlessly for me.

I used r1016 I believe. I think it's probably that my computer is not good enough. I only have 3GB of RAM

After I encode all of my videos for my Archos 5, I will try again.

[LoRd_MuldeR]

I used r1016 I believe. I think it's probably that my computer is not good enough. I only have 3GB of RAM

r1016 is pretty old. You should update to the latest build. There were various optimizations committed in the past few weeks. Also 3GB of RAM is more than enough for x264. Unless you use excessive pre-processing filters that eat a lot of memory, RAM isn't really a limitation for video encoding. The encoding speed is limited by the CPU.

[Chengbin]

r1016 is pretty old. You should update to the latest build. There were various optimizations committed in the past few weeks. Also 3GB of RAM is more than enough for x264. Unless you use excessive pre-processing filters that eat a lot of memory, RAM isn't really a limitation for video encoding. The encoding speed is limited by the CPU.

I meant when I did the 1080 to 1080 I used r1016. I use r1071 now. x264 is using over 1.5GB of RAM encoding 1080 to 1080.

[LoRd_MuldeR]

I meant when I did the 1080 to 1080 I used r1016. I use r1071 now. x264 is using over 1.5GB of RAM encoding 1080 to 1080.

That sounds like far too much to me :confused:

http://img184.imageshack.us/img184/64/x264ramub8.th.png (http://img184.imageshack.us/img184/64/x264ramub8.png)

[Snowknight26]

What really worries me is the non-parallelized deblocking. :P

[LoRd_MuldeR]

What really worries me is the non-parallelized deblocking. :P

That message originates from Avisynth. From FFVideoSource() to be precise :p

[Sagekilla]

The only reason why I could imagine why it'd eat up 1.5 GB of RAM is if you used some rather... ridiculous settings. i.e. --ref 16 and --me tesa.

[Chengbin]

This is my x264 RAM usage encoding a video from a DVD. The settings are maxed out.

http://img379.imageshack.us/img379/3169/x264lj3.png

Yes, It takes 1.5GB for 1080p encoding probably because x264 settings are maxed out completely.

[LoRd_MuldeR]

Yes, It takes 1.5GB for 1080p encoding probably because x264 settings are maxed out completely.

That's still only ~50% of your physical memory, so no reason to worry.

[Chengbin]

That's still only ~50% of your physical memory, so no reason to worry.

Vista takes 1GB alone, and x264's RAM usage grows continuously. So I fear that I'm short on RAM because after 10-15 hours of first pass, x264 just quits, the command box is still there, but the encoded frame number doesn't move (x264 not using any CPU), but RAM usage is still there, at around 1.7GB.

[Dark Shikari]

Vista takes 1GB alone, and x264's RAM usage grows continuously.No it doesn't, since x264 doesn't malloc anywhere except initialization.

If RAM usage is growing over time, it's Avisynth or your decoder, not x264.

[Chengbin]

No it doesn't, since x264 doesn't malloc anywhere except initialization.

If RAM usage is growing over time, it's Avisynth or your decoder, not x264.

Huh, that's strange. I believe I recall that x264 started out using 1.2GB or something, then slowly crawl up.

In fact, in 20 minutes, my current encodind using x264 just used 10MB more RAM (look at post 36).

http://img508.imageshack.us/img508/4618/x264id1.png

[Dark Shikari]

Huh, that's strange. I believe I recall that x264 started out using 1.2GB or something, then slowly crawl up.

In fact, in 20 minutes, my current encodind using x264 just used 10MB more RAM.

http://img508.imageshack.us/img508/4618/x264id1.pngThe listing for the "x264.exe" process on your process manager includes the thread spawned for Avisynth and the decoder.

[Chengbin]

The listing for the "x264.exe" process on your process manager includes the thread spawned for Avisynth and the decoder.

OK thanks, that clears it up.

[Snowknight26]

The listing for the "x264.exe" process on your process manager includes the thread spawned for Avisynth and the decoder.

Unless you're doing it this way:

http://www.stfcc.org/misc/memorybeast.png

[video_magic]

Chengbin:
Post your Avisynth avs script.

Encoding quality and speed is obviously affected by both the input to x264 and its' settings.

[imk]

You can use Process Explorer (http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx) to better see what's going on with memory usage.

[LoRd_MuldeR]

Vista takes 1GB alone, and x264's RAM usage grows continuously.

Note that Vista fills free/unsed RAM with "prefetch" data. This RAM appears to be in use (not available), but in fact Vista will free up that RAM immediately when an application requires more RAM than available. I read the performance overhead for freeing RAM used by "prefetch" is negligible, so it's almost like that RAM was free. At least that's the theory ^^

[cpelliott]

These are the settings I use that take a day to encode a 1080p movie. My target is a NMT/Popcorn Hour A-100, which supports up to HP@L4.1. Even though the thread title is about ignoring time, I would like to speed things up a bit. I'm using svn 1063. Any recommendations on where to start to speed things up without sacrificing too much quality?

crf=22:level=41:ref=3:mixed-refs=1:bframes=5:weightb=1:b-adapt=2:direct=auto:b-pyramid=1:me=umh:subme=9:
analyse=all:8x8dct=1:trellis=1:no-fast-pskip=1:psy-rd=1,1:vbv-bufsize=30000:vbv-maxrate=40000

[nm]

- bframes=3 is probably enough. With b-adapt=2, more bframes may slow encoding down significantly.
- remove no-fast-pskip. Fast pskip is fine when using AQ.
- If you have a multi-core CPU, tell x264 to select a suitable number of encoding threads (threads=auto or threads=0).

If that's still too slow, try a lower subme and maybe me=hex.

[J_Darnley]

rev 1072
one less B-frame
use fast-pskip (the default)

Dark Shikari has said that he would rather use a quicker me method before lowing subme:
<Dark_Shikari> I'd rather encode with me hex subme 9 than me umh subme 6

[me7]

Dark Shikari has said that he would rather use a quicker me method before lowing subme:
<Dark_Shikari> I'd rather encode with me hex subme 9 than me umh subme 6

Interesting, since umh + subme 6 is exactly what the MeGui CRF HighQuality profile says :confused: