Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
24th January 2010, 22:10 | #1 | Link |
Registered User
Join Date: Oct 2008
Posts: 25
|
Can I make these options more efficient?
I'm running a personal server for recording and encoding HD TV. Depending on network, the source can be kind of bad where I get major blocking in fade to/from blacks and cross fades. I've set this up to be a fully autonomous project with predictable results in speed and quality. I have a few target devices and other misc resolutions and bitrates I encode for. I would use x264's presets but I cannot when using mencoder. Since even with 98% reliability of signal quality with OTA broadcasts I still get the occasional split second drop, mencoder does a better job and slewing A/V sync back to what it should be. As such, I set everything manually and through mencoder. I've done my best to convert the syntax but I might have made a mistake somewhere.
I have 4 basic presets in two groups. Basically "SD" and "HD" types of resolutions. Bitrate varies between resolutions but in a close to linear ratio of # macroblocks per frame. 1080p is the exception due to storage constraints. Low-Res preset #1 --subme 8 --no-mbtree --no-fast-pskip --dct-decimate --keyint 240 or 300 --keyint-min 12 or 15 --mixed-refs --deblock 0:0 --bframes 0 --b-adapt 0 --no-cabac --qpmin 1 --qpmax 51 --qpstep 25 --ratetol 25 --qcomp 0.50 --direct auto --partitions i4x4,p4x4,p8x8 --no-8x8dct --psy-rd 0.4,0.2 --trellis 1 Low-Res preset #2 --subme 8 --no-mbtree --no-fast-pskip --dct-decimate --keyint=$max-key --keyint-min=$min-key --mixed-refs --deblock 1:1 --bframes 0 --b-adapt 0 --no-cabac --qpmin 1 --qpmax 51 --qpstep 25 --ratetol 35 --qcomp 0.75 --direct auto --partitions i4x4,p4x4,p8x8 --no-8x8dct --psy-rd 0.4,0.2 --trellis 1 Hi-Res preset #1 --no-mbtree --fast-pskip --no-dct-decimate --keyint=$max-key --keyint-min=$min-key --mixed-refs --deblock 2:2 --bframes 1 --b-adapt 0 --b-pyramid strict --cabac --qpmin 1 --qpmax 51 --qpstep 25 --ratetol 35 --qcomp 0.75 --direct auto --partitions i8x8,p8x8,b8x8 --8x8dct --psy-rd 0.4,0 --trellis 0 Hi-Res preset #2 --no-mbtree --fast-pskip --no-dct-decimate --keyint=$max-key --keyint-min=$min-key --mixed-refs --deblock 3:3 --bframes 2 --b-adapt 0 --b-pyramid strict --cabac --qpmin 1 --qpmax 51 --qpstep 15 --ratetol 15 --qcomp 0.75 --direct auto --partitions i8x8,p8x8,b8x8 --8x8dct --psy-rd 0.4,0 --trellis 0 Resolution specific options (n = native = no super-sampling/scaling) 160x90 --bitrate 80 --ref 6 --merange 4 --me umh --level 20 --threads 3 # Low Res #1 228x128 --bitrate 160 --ref 6 --merange 8 --me umh --level 20 --threads 3 # Low Res #1 320x180 --bitrate 320 --ref 6 --merange 8 --me umh --level 20 --threads 4 # Low Res #2 480x270 --bitrate 690 --ref 6 --merange 16 --me umh --level 21 --threads 4 # Low Res #2 640x360 --bitrate 1260 --ref 6 --merange 16 --me umh --level 30 --threads 5 # Low Res #2 854x480 --bitrate 2160 --ref 6 --merange 16 --me hex --level 31 --subme 9 --threads 5 # Hi Res #1 1024x576 --bitrate 3150 --ref 6 --merange 16 --me hex --level 31 --subme 7 --threads 5 # Hi Res #1 1280x720 --bitrate 4950 --ref 5 --merange 16 --me dia --level 31 --subme 6 --threads 5 # Hi Res #2 Down sampled from inverse telecine 1080i content 1280x720 --bitrate 4950 --ref 5 --merange 16 --me dia --level 31 --subme 6 --threads 5 # Hi Res #2 1920x1080 --bitrate 9000 --ref 4 --merange 16 --me dia --level 41 --subme 7 --threads 4 # Hi Res #2 Inverse telecine 1080i content (I specifically checked each merange value in multiples of 4. Increaseing them further resulted in no increase of quality based on psnr or ssim.) Naturally the lower resolutions naturally cannot be threaded as well as the higher resolutions, or so I've learned. I would still like to learn what I can/should tweak in the first two in order to increase threading efficiency if possible and increase their quality a bit if I can. I added a fixed number of b-frames in my higher res encodings since they actually sped up my encodings while their increasing quality. Now about speed, I have found it to be considerably faster to encode and inverse telecine a 1080i @ 30 fps stream to 720p @ 24 fps than it is to encode a 720p @ 60 fps stream to 1024x576 @ 30fps despite having more macroblocks per second. I would much like to solve this conundrum if possible. However, I can encode at real time or faster speeds when I drop resolutions "one notch" like the above examples and I do not want to loose this speed. and I actually would like to gain more speed in this area if possible without a drop in quality. I've done my best to understand what each option does to the encoding and these settings are my 3rd revision from when I started this project more than a year ago. By no means am I an expert and in some respects I'm a complete n00b. I would like to ask if you guys don't mind helping me refine these settings to make encoding more efficient for compression and speed. One last note, I use ABR and not CRF since I much prefer to have some control over final file size output. A 5-10% deviation is acceptable for my needs. Unless CFR provides avery significant difference in quality/speed, I don't see it as a viable alternative. Thank you all greatly in advance Last edited by Digital Corpus; 25th January 2010 at 09:22. Reason: syntax for options |
25th January 2010, 01:38 | #2 | Link | |
Registered User
Join Date: Mar 2005
Location: Finland
Posts: 2,641
|
Quote:
Last edited by nm; 25th January 2010 at 10:54. Reason: Fixed a typo. |
|
25th January 2010, 09:20 | #3 | Link |
Registered User
Join Date: Oct 2008
Posts: 25
|
That is a thought worth considering. Now I have a lot of tweaks for rate control so I'm not sure how those would be needed or not if I used CRF. Do you think there would be likely drops in quality with the use of VBV? I'm still trying to understand it so I can possibly use it for encoding high[er] quality streaming video.
|
25th January 2010, 11:44 | #4 | Link | ||
Registered User
Join Date: Mar 2005
Location: Finland
Posts: 2,641
|
Quote:
Also, you seem to be tweaking lots of things back and forth between the presets and the resolution-specific options. Personally I'd just copy x264's medium or slow preset and only change things that are required by the playback devices. Quote:
|
||
25th January 2010, 12:22 | #5 | Link | |||
x264 developer
Join Date: Sep 2005
Posts: 8,666
|
Quote:
Quote:
Quote:
It is posts like this that make me want to pull a Theora and remove almost all user-facing options completely because if you give someone a button, they will press it. And even after it shoots them in the leg, they'll go limping around for more buttons to press!
__________________
Follow x264 development progress | akupenguin quotes | x264 git status ffmpeg and x264-related consulting/coding contracts | Doom10 Last edited by Dark Shikari; 25th January 2010 at 12:38. |
|||
25th January 2010, 20:48 | #6 | Link |
Derek Prestegard IRL
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,996
|
LOL!!!!
Oh man.. He's right, you know - all drama set aside Just use something very simple, like the defaults!!!! If it's too slow, speed it up with --preset fast or something. If you can tolerate slower speeds, then use --preset slow, slower, or veryslow. Add --tune film or --tune animation if you want. And of course, do specify the rate control mode you're interested in. If you must have control over file size, 2 pass is the way to do it. If you can afford some unpredictibility, then CRF with some VBV isn't a terrible idea. ~MiSfit
__________________
These are all my personal statements, not those of my employer :) |
26th January 2010, 07:54 | #7 | Link | ||
Registered User
Join Date: Oct 2008
Posts: 25
|
Quote:
Not using MB-tree was a decision I had to make since it increase encoding time by about 25% when I first started using it. I have a copy of x264 from 11-24-09. I've tried manually compiling mplayer and x264 separately instead of through portage, but mplayer isn't reading the x264 install. Until that is fixed, I cannot easily tell how much the recent fixes have sped up mbtree so I have it disabled. As for mixed-refs, I didn't see enough of a reason to unset it. I have it listed so I know what is default so I know it is redundant. The combination of bframe settings was picked due to seeing an increase in psnr and ssim while still maintaining faster than real time encoding speed. I would use my own eye to judge this, but they become glazed over when watching the same clip more than about 5 times. If you want to knock me for this logic, explain why. Now for the various qp settings, I set those ages ago and see through the wiki that the options I have are fairly stupid. This kind of detail should be listed under the logical "--fullhelp" since it's kinda fairly useful. If poor documentation makes for poor decisions, then don't get mad at the user for uninformed decisions. Partition selections and 8x8dct are chosen for speed reasons and negligible differences in qualitly. The most understandable explination as to how these related to each other I found in mplayer's man page: Code:
partitions=<list> Enable some optional macroblock types (default: p8x8,b8x8,i8x8,i4x4). p8x8 Enable types p16x8, p8x16, p8x8. p4x4 Enable types p8x4, p4x8, p4x4. p4x4 is recommended only with subq >= 5, and only at low res- olutions. b8x8 Enable types b16x8, b8x16, b8x8. i8x8 Enable type i8x8. i8x8 has no effect unless 8x8dct is enabled. i4x4 Enable type i4x4. all Enable all of the above types. none Disable all of the above types. Regardless of this option, macroblock types p16x16, b16x16, and i16x16 are always enabled. The idea is to find the type and size that best describe a certain area of the picture. For example, a global pan is better represented by 16x16 blocks, while small moving objects are better represented by smaller blocks. Quote:
|
||
26th January 2010, 08:00 | #8 | Link | ||
x264 developer
Join Date: Sep 2005
Posts: 8,666
|
Quote:
If you don't want to learn how the internals of x264 work, use the presets. x264 --fullhelp lists them, so you can easily copy them over to mencoder. Quote:
__________________
Follow x264 development progress | akupenguin quotes | x264 git status ffmpeg and x264-related consulting/coding contracts | Doom10 Last edited by Dark Shikari; 26th January 2010 at 08:09. |
||
26th January 2010, 08:04 | #9 | Link | |
Registered User
Join Date: Oct 2008
Posts: 25
|
Quote:
|
|
26th January 2010, 08:07 | #10 | Link | ||
x264 developer
Join Date: Sep 2005
Posts: 8,666
|
Quote:
Quote:
|
||
26th January 2010, 08:08 | #11 | Link | |
Registered User
Join Date: Oct 2008
Posts: 25
|
Quote:
I thought my first post was concise enough to explain why I had to choose the route I did so I would not be mocked. I was wrong. I didn't want to write an essay explaining the countless hours I've spent staring at dozens of video encodings, looking at metrics, and comparing encoding times and the decisions I made based on all of that. Half of what I read on these boards states that the quality is subjective. If that is true, then why mock me for finding something that works for me? Last edited by Digital Corpus; 26th January 2010 at 08:15. Reason: finishing the end thought... |
|
26th January 2010, 08:10 | #12 | Link | |
Registered User
Join Date: Oct 2008
Posts: 25
|
Quote:
What i have pasted above, looks like this in mencoder: Code:
nombtree:fast_pskip:nodct_decimate:keyint=$max_key:keyint_min=$min_key:mixed_refs:deblock=2,2:bframes=1:nob_adapt:b_pyramid=strict:cabac:qp_min=1:qp_max=51:qp_step=25:ratetol=35:qcomp=0.75:direct_pred=auto:partitions=i8x8,p8x8,b8x8:8x8dct:psy-rd=0.4,0:trellis=0 |
|
26th January 2010, 08:14 | #13 | Link | |
x264 developer
Join Date: Sep 2005
Posts: 8,666
|
Quote:
I would advise you next time to not attack the people who are trying to help you. Good luck finding someone to spend hours for free trying to meet your unreasonable expectations.
__________________
Follow x264 development progress | akupenguin quotes | x264 git status ffmpeg and x264-related consulting/coding contracts | Doom10 Last edited by Dark Shikari; 26th January 2010 at 08:17. |
|
26th January 2010, 08:19 | #14 | Link | ||
Registered User
Join Date: Oct 2008
Posts: 25
|
Quote:
Quote:
|
||
26th January 2010, 08:26 | #15 | Link |
Registered User
Join Date: Oct 2008
Posts: 25
|
I'll probably give this a shot. Will the size output be constrained so that it will not go beyond a certain size due to the VBV algorithms? I have no problem with going smaller and saving bits. I'm mostly worried about over allocation and going beyond what I can reasonably expect. Would I notice any speed difference between CRF and ABR?
|
26th January 2010, 08:44 | #17 | Link |
Registered User
Join Date: Oct 2008
Posts: 25
|
That is the wiki I came across earlier today. Thank you though. I am changing my qp settings to something sane as a result. I have a background in mathematics though through multivariable Calculus and I'm a professional photographer. Already stated I'm by no means perfect, but the only way to make a good decision if I think I want to tweak something to see how it comes out, the more I need to know. cplxblur is the only other option I've considered changing.
|
26th January 2010, 16:31 | #18 | Link | ||
Registered User
Join Date: Aug 2009
Location: 61.45° , 23.86°
Posts: 120
|
Quote:
|
||
26th January 2010, 17:08 | #19 | Link |
Registered User
Join Date: Mar 2005
Location: Finland
Posts: 2,641
|
Do try the presets too. Developers have selected them carefully to achieve good results at different speed levels. There should be no need to make additional resolution-specific tunings unless you are targeting certain playback devices. Just use a faster preset for higher resolutions to speed the encoding up.
|
26th January 2010, 21:35 | #20 | Link | |
Registered User
Join Date: Feb 2002
Location: San Jose, California
Posts: 4,428
|
Quote:
Dark may not phrase everything as politely as one might want but he is offering sincere advice with strong imagery to get the point across that you do not need to mess with weird settings, just use the presets. From the beginning there have been a lot users in the forums posting command lines that have a lot of problems in them which is why the presets were created in the first place. Seeing users not using them and choosing very odd options which results in video that takes longer to compress and/or looks worse then it should for the size would be hard to face after spending all that thought setting up the presets (and improving x264 so much! Dark has a lot invested in the quality of x264). Ok, my more then "use the presets": Please DO NOT turn off mb-tree to increase encode speed! You have no-fast-pskip on and mb-tree off, the first for quality and the second for speed I assume? mb-tree is better than not doing fast-pskip by a long shot. If you are just trying to trade quality for speed at the higher resolutions using the speed presets, or the settings as they set if you cannot use them for some reason, will provide a better trade off; more speed for less loss in quality per byte, eg. which --me and --subme to pick for speed vs. quality. Opps, I did a "use the presets" after all. Just delete everything after --keyint unless you are targeting a device that doesn't support cabac? Why are you messing with de-block? That looks like like very blurry settings at the higher reses. If b-frames speed up your encode and improve quality at higher resolutions why turn them off at the lower ones? Try some crf encodes, it does provides a very significant difference in quality/speed and I have never had a run-away bitrate using crf if set near the default of 23 (though crf <17 can get very large sometimes, noisy sources etc.). crf is the most efficient mode for quality per size per encode time and the videos that are larger will have needed the size to get the same quality. It is sort of like doing a two-pass encode across all the videos in your collection (taking bits from easy to encode videos and giving them to the hard to encode ones), not that it works like that at all but it sort of results in it. x264 has been changing pretty fast so do not keep too much of what you know of the different options from older versions. I hope this helps and you can get past the "those options are crazy" sentiment and see why we think that. Last edited by Asmodian; 26th January 2010 at 21:44. |
|
Tags |
crf, ota, real time, realtime, x264 |
Thread Tools | Search this Thread |
Display Modes | |
|
|