Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

Domains: forum.doom9.org / forum.doom9.net / forum.doom9.se

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 AVC / H.264
Register FAQ Today's Posts Search

Reply
 
Thread Tools Search this Thread
Old 28th June 2007, 16:12   #1  |  Link
pcat
Registered User
 
Join Date: Nov 2006
Posts: 30
How to optimize for speed rather than compression?

Hello,

I'm need to compress a lot of dvb mpeg2 (SD) material (usually at ~4 Mbs), and currently I'm using x264 (crf mode). I'm very satisfied with the result, however, the speed is still too slow and I could accomodate bigger files. I read a lot of intersting discussions here to tweak x264 in order to approach transparency, or to reach good compression level with high image quality. However, in my case I would like to trade a bit the compression level for the speed, at the expense of file size and not (much) at the expense of quality.

I don't know if it is possible, or if ASP would be better in that use case.
In this thread,
http://forum.doom9.org/showthread.php?p=1014786
I have read suggested settings for transparent encoding where people are using subme at level 1 and not analyzing 4x4 partions, which speed up things. Are there other way to improves speed at the expense of compression, but not hurting too much the quality?

The sessings I'm currently using are
crf 22, b-frames 2, ref 3, deblock -1,-1, qcomp 0.75 and the rest are defaults.

Thanks

Last edited by pcat; 28th June 2007 at 16:15.
pcat is offline   Reply With Quote
Old 28th June 2007, 16:15   #2  |  Link
Sirber
retired developer
 
Sirber's Avatar
 
Join Date: Oct 2002
Location: Canada
Posts: 8,979
Quote:
and the rest are defaults.
Which defaults?
Sirber is offline   Reply With Quote
Old 28th June 2007, 16:38   #3  |  Link
buzzqw
HDConvertToX author
 
Join Date: Nov 2003
Location: Cesena,Italy
Posts: 6,550
i suppose x264 defaults..

try setting --ref 1 and subme 4 , you should gain speed and not loose to much quality

BHH
__________________
HDConvertToX: your tool for BD backup
MultiX264: The quick gui for x264
AutoMen: The Mencoder GUI
AutoWebM: supporting WebM/VP8
buzzqw is offline   Reply With Quote
Old 28th June 2007, 16:50   #4  |  Link
pcat
Registered User
 
Join Date: Nov 2006
Posts: 30
Quote:
Originally Posted by Sirber View Post
Which defaults?
Mencoder defaults, which I assume are the x264 defaults (like 0.6 for qcomp, cabac enabled, adaptive b frame allocation enabled, motion estimation algorithm is hexagon radius 2, supel refinement mode 5, etc.)

The command line I use is :

mencoder $1 -oac copy -ovc x264 \
-x264encopts crf=22:8x8dct:bframes=2:frameref=3:deblock=-1,-1 \
-o temp.avi
... next is MP4Box stuff for muxing to .mp4

Pierre

Last edited by pcat; 29th June 2007 at 09:26.
pcat is offline   Reply With Quote
Old 28th June 2007, 17:02   #5  |  Link
pcat
Registered User
 
Join Date: Nov 2006
Posts: 30
Quote:
Originally Posted by buzzqw View Post
i suppose x264 defaults..

try setting --ref 1 and subme 4 , you should gain speed and not loose to much quality

BHH
Thanks. ToS_Maverick suggested for "transparent" encoding (so targeting perfect quality) to go as low as subme 1 (although in my understanding reducing subme means lower quality). How can this be explained?
pcat is offline   Reply With Quote
Old 28th June 2007, 17:22   #6  |  Link
buzzqw
HDConvertToX author
 
Join Date: Nov 2003
Location: Cesena,Italy
Posts: 6,550
one os simpliest guide for x264 are in man page of mencoder (imho)

http://www.mplayerhq.hu/DOCS/HTML-si...n/MPlayer.html
and this link in particular
http://www.mplayerhq.hu/DOCS/HTML-si...menc-feat-x264

you can resolve all (near) your doubt

BHH
__________________
HDConvertToX: your tool for BD backup
MultiX264: The quick gui for x264
AutoMen: The Mencoder GUI
AutoWebM: supporting WebM/VP8
buzzqw is offline   Reply With Quote
Old 28th June 2007, 20:28   #7  |  Link
pcat
Registered User
 
Join Date: Nov 2006
Posts: 30
Quote:
Originally Posted by buzzqw View Post
one os simpliest guide for x264 are in man page of mencoder (imho)

http://www.mplayerhq.hu/DOCS/HTML-si...n/MPlayer.html
and this link in particular
http://www.mplayerhq.hu/DOCS/HTML-si...menc-feat-x264

you can resolve all (near) your doubt

BHH
Sorry, but I of course know the mplayer/mencoder doc and man, and I obviously wouldn't have asked my question here if clarifying the issue was simply a matter of reading the man

More precisely: the doc, as most other infos about x264 settings, says that increasing subq leads to better quality at the expense of encoding speed.

What seems strange to me is that knowledgeable people here at doom9 have experimented settings focusing on detail preservation (including noise or film grain) and, to my surprise, subme is at 1.

These settings also use other features such as tweaked deadzone parameters and aq-strength / sensivity, but speaking of subme, the best setting in that case seems to be 1, which means to me that a high quality can be obtain without subme >= 6 (as usually suggested).

Pierre
pcat is offline   Reply With Quote
Old 28th June 2007, 21:32   #8  |  Link
buzzqw
HDConvertToX author
 
Join Date: Nov 2003
Location: Cesena,Italy
Posts: 6,550
subme 1? i would like to see some test...
could be ok with some source... but with others ?

i stay with subme 5

Also.. sorry i haven't read your reply.. i left the tab (of broswer) open.. so i havent a refresh

BHH
__________________
HDConvertToX: your tool for BD backup
MultiX264: The quick gui for x264
AutoMen: The Mencoder GUI
AutoWebM: supporting WebM/VP8
buzzqw is offline   Reply With Quote
Old 29th June 2007, 03:38   #9  |  Link
R3Z
Silver Über Alles
 
R3Z's Avatar
 
Join Date: Aug 2006
Location: Australia
Posts: 269
Quote:
Originally Posted by pcat View Post
Sorry, but I of course know the mplayer/mencoder doc and man, and I obviously wouldn't have asked my question here if clarifying the issue was simply a matter of reading the man

More precisely: the doc, as most other infos about x264 settings, says that increasing subq leads to better quality at the expense of encoding speed.

What seems strange to me is that knowledgeable people here at doom9 have experimented settings focusing on detail preservation (including noise or film grain) and, to my surprise, subme is at 1.

These settings also use other features such as tweaked deadzone parameters and aq-strength / sensivity, but speaking of subme, the best setting in that case seems to be 1, which means to me that a high quality can be obtain without subme >= 6 (as usually suggested).

Pierre
Although the documentation refers to higher subme values indicating higher quality, this in practice doesnt seem to be the case. It appears that lower subme values do not affect quality, but do affect compression size.

I wouldnt recommend going below subme5 because the size of the file just gets too big. Its better to use the deadzone settings and a decent matrix like the prestige one to keep grain.
R3Z is offline   Reply With Quote
Old 29th June 2007, 05:07   #10  |  Link
check
phjbdpcrjlj2sb3h
 
check's Avatar
 
Join Date: Sep 2005
Location: Western Australia
Posts: 1,691
err, "quality" and "compression" are one and the same. If you are using target bitrate mode, you have quality. If you're talking in terms of target quantizer/ratefactor mode, you have compression.
It's better to talk only in terms of encoder efficiency. Lower submes are less efficient, and I don't believe lower submes help preserve grain.
check is offline   Reply With Quote
Old 29th June 2007, 05:37   #11  |  Link
R3Z
Silver Über Alles
 
R3Z's Avatar
 
Join Date: Aug 2006
Location: Australia
Posts: 269
Quote:
Originally Posted by check View Post
err, "quality" and "compression" are one and the same. If you are using target bitrate mode, you have quality. If you're talking in terms of target quantizer/ratefactor mode, you have compression.
It's better to talk only in terms of encoder efficiency. Lower submes are less efficient, and I don't believe lower submes help preserve grain.
Not disputing they are related, but to say something is higher "quality" at subme 7 than at subme 1 isnt correct. Subme 6 or 7 with rdo will not look as good as subme 1 and hence you cant call it higher quality.
R3Z is offline   Reply With Quote
Old 29th June 2007, 06:37   #12  |  Link
check
phjbdpcrjlj2sb3h
 
check's Avatar
 
Join Date: Sep 2005
Location: Western Australia
Posts: 1,691
Quote:
Originally Posted by R3Z View Post
Subme 6 or 7 with rdo will not look as good as subme 1 and hence you cant call it higher quality.
Could you please upload a sample and two commandlines that show this? I tried a few sources back last time this was brought up and was truly unable to find any source that this help true for. It just ended up with either more blocking for higher deblock or more mosquito noise for lower deblock.
check is offline   Reply With Quote
Old 29th June 2007, 09:17   #13  |  Link
pcat
Registered User
 
Join Date: Nov 2006
Posts: 30
Quote:
Originally Posted by check View Post
Could you please upload a sample and two commandlines that show this? I tried a few sources back last time this was brought up and was truly unable to find any source that this help true for. It just ended up with either more blocking for higher deblock or more mosquito noise for lower deblock.
Here is the posts where it is discussed, and where you can find the exact command line :
http://forum.doom9.org/showthread.ph...106#post996106
http://forum.doom9.org/showthread.ph...829#post996829

@R3Z: Thanks for the clarification about subme. So if understand well, lowering subme is a right way to reach my goal : increasing speed without compromizing quality. Compression efficiency will drop, and the file size will be bigger, but I have margin for that. I will experiment to find the right level. Are there other parameters that would influence in the same way (faster as the expense of compression but not visual quality) ?

Thanks again!

Last edited by pcat; 29th June 2007 at 09:25.
pcat is offline   Reply With Quote
Old 29th June 2007, 09:44   #14  |  Link
audyovydeo
Registered User
 
audyovydeo's Avatar
 
Join Date: Apr 2007
Posts: 464
@pcat

have you read this document :

http://www.yuvsoft.com/pdf/x264_para...comparison.pdf

??

Very exhaustive. Even though it dates back to august 2006 (about 100 versions have gone by since then), it identifies the optimal options down to a fault.


Personally I find --ref greatly speed & quality, so I am loath to go below 3. You could also set --merange lower than the 16 default.
Also, I'd rather set a lower --subme than leave trellis out.


cheers
audyovydeo
audyovydeo is offline   Reply With Quote
Old 29th June 2007, 09:58   #15  |  Link
pcat
Registered User
 
Join Date: Nov 2006
Posts: 30
@audyovydeo:

Thanks a lot. I didn't know about the referenced PDF, it is great info !

Pierre

Last edited by pcat; 29th June 2007 at 10:47.
pcat is offline   Reply With Quote
Old 29th June 2007, 10:55   #16  |  Link
R3Z
Silver Über Alles
 
R3Z's Avatar
 
Join Date: Aug 2006
Location: Australia
Posts: 269
Quote:
Originally Posted by pcat View Post
Here is the posts where it is discussed, and where you can find the exact command line :
http://forum.doom9.org/showthread.ph...106#post996106
http://forum.doom9.org/showthread.ph...829#post996829

@R3Z: Thanks for the clarification about subme. So if understand well, lowering subme is a right way to reach my goal : increasing speed without compromizing quality. Compression efficiency will drop, and the file size will be bigger, but I have margin for that. I will experiment to find the right level. Are there other parameters that would influence in the same way (faster as the expense of compression but not visual quality) ?

Thanks again!
As was said above, ref frames do slow encoding down alot. I would never go under subme5 personally, it defeats the purpose of getting some sort of decent compression.

Be careful with trellis though, it wont help keep grain. That document is great btw.
R3Z is offline   Reply With Quote
Old 29th June 2007, 11:35   #17  |  Link
pcat
Registered User
 
Join Date: Nov 2006
Posts: 30
Quote:
Originally Posted by R3Z View Post
As was said above, ref frames do slow encoding down alot. I would never go under subme5 personally, it defeats the purpose of getting some sort of decent compression.
.
decent compression is relative. In my use case, I need to squeeze DVB materials from 2.5-4 GB into about 1-1.5 GB.

However going too slow is a problem because of the rate I'm getting new files : in practice, I have to throw away the recordings which I haven't time to compress, which in a sense means 100% compression at 0% quality !

My encodes usually end up between 0.8 GB and 1.2 GB with my current settings, at a quality level I'm happy with. So I have a good margin, and for me "decent compression" just means not going much beyond 1.5 GB.

Last edited by pcat; 29th June 2007 at 11:41.
pcat is offline   Reply With Quote
Old 29th June 2007, 12:34   #18  |  Link
R3Z
Silver Über Alles
 
R3Z's Avatar
 
Join Date: Aug 2006
Location: Australia
Posts: 269
Quote:
Originally Posted by pcat View Post
decent compression is relative. In my use case, I need to squeeze DVB materials from 2.5-4 GB into about 1-1.5 GB.

However going too slow is a problem because of the rate I'm getting new files : in practice, I have to throw away the recordings which I haven't time to compress, which in a sense means 100% compression at 0% quality !

My encodes usually end up between 0.8 GB and 1.2 GB with my current settings, at a quality level I'm happy with. So I have a good margin, and for me "decent compression" just means not going much beyond 1.5 GB.
Its always great to find a sweet spot Problem with me is that my recording are of all different resolutions, quality and importance. The sweet spot tends to move around !
R3Z is offline   Reply With Quote
Old 29th June 2007, 12:36   #19  |  Link
audyovydeo
Registered User
 
audyovydeo's Avatar
 
Join Date: Apr 2007
Posts: 464
Quote:
Originally Posted by pcat View Post
decent compression is relative.
Quick ! Someone hammer that line in a slab of marble !!


Quote:
Originally Posted by pcat View Post
My encodes usually end up between 0.8 GB and 1.2 GB with my current settings, at a quality level I'm happy with. So I have a good margin, and for me "decent compression" just means not going much beyond 1.5 GB.
How do you evaluate quality : you use your eyes, or one of the two metrics provided ?


Maybe your best bet is to use --bitrate single-pass.
From here : http://en.wikipedia.org/wiki/H264

I see that the "max video bitrate for High Profile" for SD content (which I take to be 720x576@25 as that's my case)
is 12.5Mb/s.
You could try aiming at 1/3 of that bitrate with a 1pass ABR cmdline, and see what speed / metrics / filesize you get.


cheers
audyovydeo
audyovydeo is offline   Reply With Quote
Old 29th June 2007, 13:21   #20  |  Link
pcat
Registered User
 
Join Date: Nov 2006
Posts: 30
Quote:
Originally Posted by audyovydeo View Post
Quick ! Someone hammer that line in a slab of marble !!



Quote:
How do you evaluate quality : you use your eyes, or one of the two metrics provided ?
I refer to my subjective perception of quality: i tried at several crf level, 24 wasn't good enough, 22 not perfect but very good to my eyes and file size at this level are already smaller than my needs so I settled for that. In fact, as I have some room, if I don't find a way to use this bit space margin for significantly better speed, I will use it to go to crf 21 (although it may hurts speed a little more). But it seems that subme 3 or 4 could be the solution, I will try that.

Quote:
Maybe your best bet is to use --bitrate single-pass.
From here : http://en.wikipedia.org/wiki/H264

I see that the "max video bitrate for High Profile" for SD content (which I take to be 720x576@25 as that's my case)
is 12.5Mb/s.
You could try aiming at 1/3 of that bitrate with a 1pass ABR cmdline, and see what speed / metrics / filesize you get.
Thanks for the suggestion! However, my main problem is speed. I'm already satisfied with the visual quality and I'm just trying to optimize the speed issue. The crf mode I'm using is one pass, and I'm not sure that ABR mode would be faster, am I wrong?

Pierre

Last edited by pcat; 29th June 2007 at 13:26.
pcat is offline   Reply With Quote
Old 29th June 2007, 13:46   #21  |  Link
check
phjbdpcrjlj2sb3h
 
check's Avatar
 
Join Date: Sep 2005
Location: Western Australia
Posts: 1,691
1pass bitrate would be a bad move from crf. It would take the same time, but quality would be far less. The one advantage would be you could predict final filesize, however the quality loss would be significant.
Anyway, the ffmpeg defaults aren't actually the same as the x264 ones last I checked, they were slightly weird in some areas. How much faster do you want the encode to be? My three general suggestions would be: use few bframes, use fewer refs, use faster m.e. algo, but if you can tell us what sort of speed increase you want this will help a lot.

I can certainly see a difference between the images, I can certainly not see a large improvement. The comparison which most clearly shows a lack of improvement is the second set. Look at the fuzzy out of focus objects in the background and it is clear that no more grain is being retained from loss of precision in motion estimation.
check is offline   Reply With Quote
Old 29th June 2007, 14:18   #22  |  Link
audyovydeo
Registered User
 
audyovydeo's Avatar
 
Join Date: Apr 2007
Posts: 464
Quote:
Originally Posted by check View Post
1pass bitrate would be a bad move from crf. It would take the same time, but quality would be far less. The one advantage would be you could predict final filesize, however the quality loss would be significant.
I contest that !
Last week I set about figuring out - the hard way - which ratecontrol method is most efficient. I am still at it, but preliminary results tell me that whatever the ratecontrol method chosen, the SSIM/bitrate pair is mostly the same.

Here is a graph for the QP vs CRF test. I ran encodes varying crf from 1 to 39, and QP from 20 to 30.



For each encode I gather the SSIM + kb/s pair.
x-axis are the --crf or --qp values. left y-axis is the bitrate in 1000kb/s, right y-axis is ssim.
The curves for QP and CRF are literally identical. If you shift the SSIM-QP curve to align it to the other ssim curve, the bitrates also overlap.

Had I known, I would also have jotted down the fps values, but time's precious these days ...

audyovydeo
audyovydeo is offline   Reply With Quote
Old 29th June 2007, 14:35   #23  |  Link
check
phjbdpcrjlj2sb3h
 
check's Avatar
 
Join Date: Sep 2005
Location: Western Australia
Posts: 1,691
Neither of those modes is 1 pass ABR . --qp/--crf and --bitrate are not equivalent at all, because they are constrained by different variables.

Also, if you do some quick perceptual comparisons between --qp # and --crf # you should see that they look visually very similar. --crf works by increasing the quantizer where high visual quality is not needed, leading to a significantly smaller file at low to zero quality loss.

Last edited by check; 29th June 2007 at 17:53.
check is offline   Reply With Quote
Old 29th June 2007, 16:31   #24  |  Link
audyovydeo
Registered User
 
audyovydeo's Avatar
 
Join Date: Apr 2007
Posts: 464
Quote:
Originally Posted by check View Post
Neither of those modes is 1 pass ABR
True: I'm at work and only have a partial dataset.
I will post my complete results later on, in a more coherent manner.

cheers
audyovydeo
audyovydeo is offline   Reply With Quote
Old 30th June 2007, 10:50   #25  |  Link
PuzZLeR
WiLD CaRD
 
PuzZLeR's Avatar
 
Join Date: Mar 2007
Location: Toronto Canada
Posts: 258
At the risk of sounding rude, you are defeating the purpose entirely with extensive testing and comparisons. This is how you're going to make things go fast? You want great speed, not the best quality so it should be very easy when you're willing to make that sacrifice.

A quick solution to your problem, among several others, of optimizing speed for a bit less quality/compression is to simply use one of the iPod profiles and encode that way with CRF (NOT 1-pass bitrate which sucks). It's relatively good quality but many of the heavier features such as MRFs, B frames, CABAC, etc are disabled which should give you that speed you need as well in your encoding (and fast decoding too as a bonus).

This is what I use when I only want a quick encode of something with decent quality instead of testing a million things. It's actually better than decent quality in my opinion with this profile.

I believe you should be testing a million things when it's optimal quality you're trying to acheive, not optimal speed.

Last edited by PuzZLeR; 30th June 2007 at 11:07.
PuzZLeR is offline   Reply With Quote
Old 30th June 2007, 11:03   #26  |  Link
ToS_Maverick
x264 Tester
 
Join Date: Dec 2005
Location: Austria, near Vienna
Posts: 223
ah, dammit, once i don't read the whole forum, there's a thread about my settings

@pcat, about subme 1:
as i searched the forum, for a more transparent setting, i found out that x264 uses SAD instead of SATD with subme 1. don't ask me what it is, or what it does, i just tested it
at this time, i hadn't tested the adaptive quantization patch, so i worked with subme, trellis and the deadzone switches.

but after AQ, my whole life changed
forget about subme, or the deadzones, all you need is AQ.

if you want to get a bit more speed, don't use subme 6, or trellis, use max 2 refs, hex search... i got it all wrapped up in this commandline:

Code:
 --crf 20.0 --level 3 --keyint 100 --min-keyint 1 --ref 2 --no-fast-pskip --bframes 2 --weightb --filter -2,-2 --analyse p8x8,b8x8,i8x8 --8x8dct --vbv-bufsize 1835 --vbv-maxrate 10000 --threads auto --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output "" "" --aq-strength 0.9
if somebody wonders about the keyints or the level-switches:
keyint 100 increases the bitrate by a minimum, but guarantees a faster seeking. about the levels: i like it if my encodes are level-compliant
ToS_Maverick is offline   Reply With Quote
Old 30th June 2007, 15:35   #27  |  Link
pcat
Registered User
 
Join Date: Nov 2006
Posts: 30
@ToS_Maverick:

Thanks for the updated line !
I'm going to experiment with it, do you know the equivalent of --aq-strength for MEncoder? It looks like 'ratetol', but I'm not sure...

Pierre
pcat is offline   Reply With Quote
Old 30th June 2007, 16:41   #28  |  Link
ToS_Maverick
x264 Tester
 
Join Date: Dec 2005
Location: Austria, near Vienna
Posts: 223
no, ratetol is a different switch.

i don't know if AQ is implemented in MEncoder... guys do you know more?

you could try it without AQ... depends on the source. some movies don't need it at all, some look horrible without it.
ToS_Maverick is offline   Reply With Quote
Old 30th June 2007, 21:29   #29  |  Link
akupenguin
x264 developer
 
akupenguin's Avatar
 
Join Date: Sep 2004
Posts: 2,392
MEncoder doesn't implement any x264 options. It passes all -x264encopts arguments on to x264's own option parser. So if you patch x264 to add some option, you can automatically use that option in mencoder.
akupenguin is offline   Reply With Quote
Old 30th June 2007, 21:58   #30  |  Link
audyovydeo
Registered User
 
audyovydeo's Avatar
 
Join Date: Apr 2007
Posts: 464
Quote:
Originally Posted by PuzZLeR View Post
At the risk of sounding rude, you are defeating the purpose entirely with extensive testing and comparisons. This is how you're going to make things go fast? You want great speed, not the best quality so it should be very easy when you're willing to make that sacrifice.

A quick solution to your problem, among several others, of optimizing speed for a bit less quality/compression is to simply use one of the iPod profiles and encode that way with CRF (NOT 1-pass bitrate which sucks). It's relatively good quality but many of the heavier features such as MRFs, B frames, CABAC, etc are disabled which should give you that speed you need as well in your encoding (and fast decoding too as a bonus).

This is what I use when I only want a quick encode of something with decent quality instead of testing a million things. It's actually better than decent quality in my opinion with this profile.

Mhh, I dont know if you're addressing yourself to pcat or to me.
Your comments aren't rude, they're welcome.

My previous posting was too quick : I actually tested CRF/vs/QP/vs/2pABR, agree that 1pABR is a different story.


Quote:
Originally Posted by PuzZLeR View Post
I believe you should be testing a million things when it's optimal quality you're trying to acheive, not optimal speed.
Exactly what I'm doing, sir. I didn't even look at encoding speeds in my tests, let alone collect them.
audyovydeo is offline   Reply With Quote
Old 30th June 2007, 23:14   #31  |  Link
pcat
Registered User
 
Join Date: Nov 2006
Posts: 30
Quote:
Originally Posted by PuzZLeR View Post
At the risk of sounding rude, you are defeating the purpose entirely with extensive testing and comparisons. This is how you're going to make things go fast? You want great speed, not the best quality so it should be very easy when you're willing to make that sacrifice.
I'm not sure you're adressing my point. To sum up, I want great speed, very good quality (similar to the level achieved for example by ToS_Maverick), and average compression (this is on what I compromize). I can for instance accept +50% of target file size at the same quality as very efficient settings, if is much faster.

I don't want to trade the speed for both the encoding time AND the visual quality.

Quote:
A quick solution to your problem, among several others, of optimizing speed for a bit less quality/compression is to simply use one of the iPod profiles and encode that way with CRF (NOT 1-pass bitrate which sucks). It's relatively good quality but many of the heavier features such as MRFs, B frames, CABAC, etc are disabled which should give you that speed you need as well in your encoding (and fast decoding too as a bonus).
As you can see from my previous posts, I'm already using CRF mode and few bframes. I'm not sure that disabling completely B frames etc. would still worth the use of x264. Lavc MPEG4 would likely be more effective for my purpose, using a low compression/high bitrate ASP profile optimized for speed. I have not looked at profiles for iPod, but they are likely designed for smaller resolution than what I'm considering.

Quote:
I believe you should be testing a million things when it's optimal quality you're trying to acheive, not optimal speed.
I don't see why... I'm trying to set up the best automatic procedure to batch encode bunch of materials. Spending some time finding the right parameters clearly worth it for me, and I think I can decide for myself what I need to optimize or not

Pierre
pcat is offline   Reply With Quote
Old 1st July 2007, 00:07   #32  |  Link
PuzZLeR
WiLD CaRD
 
PuzZLeR's Avatar
 
Join Date: Mar 2007
Location: Toronto Canada
Posts: 258
Perhaps I was a bit hasty in determining a sacrifice of quality. It's compression you're willing to sacrifice. Although there is a relationship between the two, I do see your point.

However, my point still stands.

Theoretically speaking, if you want high speed, high quality and are willing to sacrifice compression, then just disable many key quality/compression features and compensate with higher bitrate. It's that simple really. You get the quality and speed but less compression is the cost.

For example, all you really need to do is turn off stuff like MRF, CABAC, B frames/pyramids to 0, or even subme from 5 to 4 and add bitrate.

And you don't even need to go through the hassle of calculating this extra bitrate either. This is why I also mentioned CRF because it does exactly that - compensates for you with the exact added bitrate to the quality you want.

Just to give you an example of CRF's brilliance in compensation:

It is well known that CABAC has about a 10%+ increase in compression over CAVLC (10%+ better quality per file size). However, CABAC slows down encoding speed (and decoding speed too) but to many, myself included, the compression is worth it. But if you're willing to accept less compression - turn CABAC off. As well, use CRF. What will happen, paretibus ceribus, is that CRF will actually give about 10%+ more bitrate for CAVLC encoding in compensation. You get the quality, and speed, but less compression. This will happen for all features you disable for added speed - automatically and accurately adjusted for you.

Anyhow, it's fun to play around with options, and this thread's ideas are informative, but from my point of view, this problem is really more simpler than it sounds. Just trying to help.
PuzZLeR is offline   Reply With Quote
Old 1st July 2007, 05:20   #33  |  Link
akupenguin
x264 developer
 
akupenguin's Avatar
 
Join Date: Sep 2004
Posts: 2,392
The encoder optimizes between 3 parameters: quality, bitrate, and speed.
There are some complex interactions between options, but lets ignore them for now. To a first approximation, if you use CRF or CQP mode then the only option that affects quality is --crf or --qp, and all other options tradeoff speed vs bitrate (note: this does not mean that you can measure just the bitrate to determine the effectiveness of some option. I said approximation.). If you use ABR, CBR, or 2pass mode, then the only option that affects bitrate is --bitrate, and all other options tradeoff quality vs speed. I can merge those two statements to become: Ratecontrol options trade quality vs bitrate, and all other options trade speed vs bitrate.

So there's two independent decisions to make: What quality you want (determines the value of --crf), and how much cpu-time you're willing to spend (determines everything else).
There's more complex interactions here, but I'll ignore them too. To a first approximation, each speed-vs-bitrate option costs a certain amount of time and reduces bitrate by a certain %. You pick some ratio of how much time you're willing to spend to save any given amount of bitrate, and enable all the options whose ROI is better than that.

Quote:
Originally Posted by PuzZLeR View Post
Theoretically speaking, if you want high speed, high quality and are willing to sacrifice compression, then just disable many key quality/compression features and compensate with higher bitrate. It's that simple really. You get the quality and speed but less compression is the cost.

For example, all you really need to do is turn off stuff like MRF, CABAC, B frames/pyramids to 0, or even subme from 5 to 4 and add bitrate.
The problem then becomes, which options to turn off? If you only care about quality-per-bitrate and ignore speed, then you can just enable most options, and you only have to tweak the ones with psy effects. If you only care about speed and ignore quality-per-bitrate, you should use xvid instead. But most people fall between those extremes, and there's no more or less optimization possible just because your ROI threshold is higher.

CABAC is the option with the best ROI. It has a significant cost in decoding speed, but insignificant cost in encoding speed. (Ok, so there's a 4th parameter to optimize: decoding speed) To be precise, the absolute amount of cpu time used to encode CABAC is the same as to decode CABAC, but the encoder does so much other stuff that it's proportionally less.

Some numbers (of course they depend on content, this is just one example):
CABAC: 10% bitrate / 2.8% speed = 3.6 roi
8x8dct: 7.4% bitrate / 9% speed = 0.80 roi
-t1: 4.8% bitrate / 10.5% speed = 0.45 roi
-Anone vs default: 12.5% bitrate / 31% speed = 0.40 roi
-m1 vs -m3: 9.1% bitrate / 25% speed = 0.36 roi
...

Quote:
Originally Posted by pcat View Post
I don't want to trade the speed for both the encoding time AND the visual quality.
Speed measures the same thing as encoding time, so you can't trade one for the other.

Last edited by akupenguin; 15th November 2007 at 10:05. Reason: typo
akupenguin is offline   Reply With Quote
Old 2nd July 2007, 00:53   #34  |  Link
PuzZLeR
WiLD CaRD
 
PuzZLeR's Avatar
 
Join Date: Mar 2007
Location: Toronto Canada
Posts: 258
Hello Akupenguin,

My point is more general and theoretical. My CABAC example was more hypothetical.

In fact, let’s just ignore encoding speed for a moment. My REAL point is that it doesn’t matter which options you disable. If you lose quality/file size as a result of these dropped features, you can “get the quality back” with more bitrate in your encoding.

Result: Same quality, less compression.

Now let’s bring back the speed parameter. Applying this thinking, just disable features that are known to slow down speed. Yes, you may lose quality/file size, but you can always “get the quality back” once more with added bitrate in your encoding.

Result: Same quality, faster encoding speed, less compression.

Now, to determine this exact extra bitrate necessary is the easiest thing of all. CRF calculates that extra bitrate to great accuracy automatically. In fact, in most cases, one needs only to do a test CRF encode with small clips, with and without the features enabled, and calculate the percentage difference and apply that to the macro picture later, even with ABR encoding. CRF is quite clever in fact, and I thank you for that feature. Well done.

Result: Same exact quality, faster encoding speed, less exact compression.

So, with my being on the lazier side of testing many different speed increasing options, and only use those proven, and sacrificing compression, as far as I’m concerned, problem solved, and QED.

However:
Quote:
Originally Posted by akupenguin View Post
The encoder optimizes between 3 parameters: quality, bitrate, and speed.
...and...
Quote:
Originally Posted by akupenguin View Post
To a first approximation, each speed-vs-bitrate option costs a certain amount of time and reduces bitrate by a certain %. You pick some ratio of how much time you're willing to spend to save any given amount of bitrate, and enable all the options whose ROI is better than that.
You're right on. I personally think there’s another confusion going on in this thread. The three parameters: quality, bitrate and speed (and sometimes the 4th decoding speed) are compared on inaccurate scales. The rate of increase for encoding time using these quality enhancement features is much sharper than the rate of increase in the quality curve it yields. However, the curve is much smoother when comparing rates of compression with these features to the quality level of the output. Determining this ROI is not linear is what I'm trying to say here, but considering these trade-offs, I would stick with the smoother curves...
PuzZLeR is offline   Reply With Quote
Old 2nd July 2007, 06:02   #35  |  Link
SealTooGreat
Registered User
 
Join Date: Dec 2006
Posts: 302
What is ROI ?
SealTooGreat is offline   Reply With Quote
Old 2nd July 2007, 06:22   #36  |  Link
mitsubishi
Registered User
 
Join Date: Sep 2006
Location: UK
Posts: 416
Quote:
Originally Posted by SealTooGreat View Post
What is ROI ?
Republic of Ireland

Or in this context, "return on investment", ie what you get back in quality for what you put in, in terms of encoding time.
mitsubishi is offline   Reply With Quote
Old 2nd July 2007, 06:35   #37  |  Link
PuzZLeR
WiLD CaRD
 
PuzZLeR's Avatar
 
Join Date: Mar 2007
Location: Toronto Canada
Posts: 258
Quote:
Originally Posted by SealTooGreat View Post
What is ROI ?
It's a financial expression by nature but used in many contexts. A very simplified formula regarding investing in a venture is:

ROI = revenue made/money invested. An ROI of 10 would suggest that you got back $10 for every $1 of your investment. Anytime ROI > 1, you made a profit.

The idea is to get the maximum ROI for your investments – to get the most money back for every dollar/euro/whatever that you input/invest.

What Akupenguin was referring to was getting the most of your input parameters – choosing the right combination of x264 features that will bring back the best output whether it's best quality/file size, best quality/fastest encoding speed, or whatever necessary to serve your objective in the most optimal way.
PuzZLeR is offline   Reply With Quote
Old 2nd July 2007, 06:37   #38  |  Link
PuzZLeR
WiLD CaRD
 
PuzZLeR's Avatar
 
Join Date: Mar 2007
Location: Toronto Canada
Posts: 258
@mitsubishi,

Either you have a faster trigger than me, or you must of hit "Submit Reply" when I started typing....
PuzZLeR is offline   Reply With Quote
Old 2nd July 2007, 08:54   #39  |  Link
akupenguin
x264 developer
 
akupenguin's Avatar
 
Join Date: Sep 2004
Posts: 2,392
Quote:
Originally Posted by PuzZLeR View Post
You're right on. I personally think there’s another confusion going on in this thread. The three parameters: quality, bitrate and speed (and sometimes the 4th decoding speed) are compared on inaccurate scales. The rate of increase for encoding time using these quality enhancement features is much sharper than the rate of increase in the quality curve it yields. However, the curve is much smoother when comparing rates of compression with these features to the quality level of the output. Determining this ROI is not linear is what I'm trying to say here, but considering these trade-offs, I would stick with the smoother curves...
What does a speed-vs-quality curve have to do with a bitrate-vs-quality curve? You can't avoid picking a point on both curves (by setting both speed options and ratecontrol options), so what does it mean to stick with one of them?
Right, ROI is not linear, it's exponential: %speed per %bitrate. So I always visualize it with log-log graphs.
akupenguin is offline   Reply With Quote
Old 2nd July 2007, 09:46   #40  |  Link
pcat
Registered User
 
Join Date: Nov 2006
Posts: 30
@Akupenguin and PuZzler:

Thanks a lot for the clarification. If I understand well, basically if I'm happy with the quality achieved with CRF 22, I can safely turn off any other feature until I reach good speed with still acceptable file size ?

- in particular: the number of reference frames, the number of B-frames, subme, and CABAC which seems to have the most significant impact on encoding speed.


@Akupenguin, regarding MEncoder: it does not implement x264 as it just "use" the x264 engine or library, but it does not "exec" a standalone x264 binary and thus doesn't forward the command line arguments "verbatim" to x264 but has to parse them. And I don't know if the AQ patch are included in the x264 used by MEncoder...

Thanks again!

Pierre
pcat is offline   Reply With Quote
Old 2nd July 2007, 10:03   #41  |  Link
akupenguin
x264 developer
 
akupenguin's Avatar
 
Join Date: Sep 2004
Posts: 2,392
Didn't I just say that CABAC has the least significant impact on encoding speed?

OK, so mencoder parses the options. The parsing is simply: options are separated by ':', and an option name is separate from its value by '=' (or if there isn't an '=' then the option is treated as boolean). None of the option names appear in mencoder's source code. Mencoder just passes the strings to libx264 without caring what they mean.
The last time I updated mencoder for new x264 options was 2006-07-31 (r542). Several options have been added to x264 since then, but no change in mencoder was required. Likewise, no change in mencoder is required to support aq.

Last edited by akupenguin; 2nd July 2007 at 17:50.
akupenguin is offline   Reply With Quote
Reply


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 23:13.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2026, vBulletin Solutions Inc.