Can I make these options more efficient?

[Digital Corpus]

I'm running a personal server for recording and encoding HD TV. Depending on network, the source can be kind of bad where I get major blocking in fade to/from blacks and cross fades. I've set this up to be a fully autonomous project with predictable results in speed and quality. I have a few target devices and other misc resolutions and bitrates I encode for. I would use x264's presets but I cannot when using mencoder. Since even with 98% reliability of signal quality with OTA broadcasts I still get the occasional split second drop, mencoder does a better job and slewing A/V sync back to what it should be. As such, I set everything manually and through mencoder. I've done my best to convert the syntax but I might have made a mistake somewhere.

I have 4 basic presets in two groups. Basically "SD" and "HD" types of resolutions. Bitrate varies between resolutions but in a close to linear ratio of # macroblocks per frame. 1080p is the exception due to storage constraints.

Low-Res preset #1
--subme 8 --no-mbtree --no-fast-pskip --dct-decimate --keyint 240 or 300 --keyint-min 12 or 15 --mixed-refs --deblock 0:0 --bframes 0 --b-adapt 0 --no-cabac --qpmin 1 --qpmax 51 --qpstep 25 --ratetol 25 --qcomp 0.50 --direct auto --partitions i4x4,p4x4,p8x8 --no-8x8dct --psy-rd 0.4,0.2 --trellis 1

Low-Res preset #2
--subme 8 --no-mbtree --no-fast-pskip --dct-decimate --keyint=$max-key --keyint-min=$min-key --mixed-refs --deblock 1:1 --bframes 0 --b-adapt 0 --no-cabac --qpmin 1 --qpmax 51 --qpstep 25 --ratetol 35 --qcomp 0.75 --direct auto --partitions i4x4,p4x4,p8x8 --no-8x8dct --psy-rd 0.4,0.2 --trellis 1

Hi-Res preset #1
--no-mbtree --fast-pskip --no-dct-decimate --keyint=$max-key --keyint-min=$min-key --mixed-refs --deblock 2:2 --bframes 1 --b-adapt 0 --b-pyramid strict --cabac --qpmin 1 --qpmax 51 --qpstep 25 --ratetol 35 --qcomp 0.75 --direct auto --partitions i8x8,p8x8,b8x8 --8x8dct --psy-rd 0.4,0 --trellis 0

Hi-Res preset #2
--no-mbtree --fast-pskip --no-dct-decimate --keyint=$max-key --keyint-min=$min-key --mixed-refs --deblock 3:3 --bframes 2 --b-adapt 0 --b-pyramid strict --cabac --qpmin 1 --qpmax 51 --qpstep 15 --ratetol 15 --qcomp 0.75 --direct auto --partitions i8x8,p8x8,b8x8 --8x8dct --psy-rd 0.4,0 --trellis 0

Resolution specific options (n = native = no super-sampling/scaling)
160x90 --bitrate 80 --ref 6 --merange 4 --me umh --level 20 --threads 3 # Low Res #1
228x128 --bitrate 160 --ref 6 --merange 8 --me umh --level 20 --threads 3 # Low Res #1
320x180 --bitrate 320 --ref 6 --merange 8 --me umh --level 20 --threads 4 # Low Res #2
480x270 --bitrate 690 --ref 6 --merange 16 --me umh --level 21 --threads 4 # Low Res #2
640x360 --bitrate 1260 --ref 6 --merange 16 --me umh --level 30 --threads 5 # Low Res #2
854x480 --bitrate 2160 --ref 6 --merange 16 --me hex --level 31 --subme 9 --threads 5 # Hi Res #1
1024x576 --bitrate 3150 --ref 6 --merange 16 --me hex --level 31 --subme 7 --threads 5 # Hi Res #1
1280x720 --bitrate 4950 --ref 5 --merange 16 --me dia --level 31 --subme 6 --threads 5 # Hi Res #2 Down sampled from inverse telecine 1080i content
1280x720 --bitrate 4950 --ref 5 --merange 16 --me dia --level 31 --subme 6 --threads 5 # Hi Res #2
1920x1080 --bitrate 9000 --ref 4 --merange 16 --me dia --level 41 --subme 7 --threads 4 # Hi Res #2 Inverse telecine 1080i content
(I specifically checked each merange value in multiples of 4. Increaseing them further resulted in no increase of quality based on psnr or ssim.)

Naturally the lower resolutions naturally cannot be threaded as well as the higher resolutions, or so I've learned. I would still like to learn what I can/should tweak in the first two in order to increase threading efficiency if possible and increase their quality a bit if I can.

I added a fixed number of b-frames in my higher res encodings since they actually sped up my encodings while their increasing quality. Now about speed, I have found it to be considerably faster to encode and inverse telecine a 1080i @ 30 fps stream to 720p @ 24 fps than it is to encode a 720p @ 60 fps stream to 1024x576 @ 30fps despite having more macroblocks per second. I would much like to solve this conundrum if possible. However, I can encode at real time or faster speeds when I drop resolutions "one notch" like the above examples and I do not want to loose this speed. and I actually would like to gain more speed in this area if possible without a drop in quality.

I've done my best to understand what each option does to the encoding and these settings are my 3rd revision from when I started this project more than a year ago. By no means am I an expert and in some respects I'm a complete n00b. I would like to ask if you guys don't mind helping me refine these settings to make encoding more efficient for compression and speed.

One last note, I use ABR and not CRF since I much prefer to have some control over final file size output. A 5-10% deviation is acceptable for my needs. Unless CFR provides avery significant difference in quality/speed, I don't see it as a viable alternative.

:) :) Thank you all greatly in advance :) :)

[nm]

One last note, I use ABR and not CRF since I much prefer to have some control over final file size output. A 5-10% deviation is acceptable for my needs. Unless CFR provides avery significant difference in quality/speed, I don't see it as a viable alternative.
Depends on what you consider significant, but generally it does. Why do you need to have control over the size of individual encodes? How about VBV-capped CRF if it's a bandwidth issue?

[Digital Corpus]

That is a thought worth considering. Now I have a lot of tweaks for rate control so I'm not sure how those would be needed or not if I used CRF. Do you think there would be likely drops in quality with the use of VBV? I'm still trying to understand it so I can possibly use it for encoding high[er] quality streaming video.

[nm]

That is a thought worth considering. Now I have a lot of tweaks for rate control so I'm not sure how those would be needed or not if I used CRF.
I don't think they are needed with ABR either. Setting ratetol higher than default may be a good idea in your case, but I wouldn't touch those other settings.

Also, you seem to be tweaking lots of things back and forth between the presets and the resolution-specific options. Personally I'd just copy x264's medium or slow preset and only change things that are required by the playback devices.

Do you think there would be likely drops in quality with the use of VBV?
That depends on many things. If you can set it high enough (perhaps 15 Mbps with a 10 second buffer for 1080p), it would probably be fine considering your OTA sources.

[Dark Shikari]

I added a fixed number of b-frames in my higher res encodings since they actually sped up my encodings while their increasing quality.Oh dear...--no-mbtree --fast-pskip --no-dct-decimate --keyint=$max-key --keyint-min=$min-key --mixed-refs --deblock 2:2 --bframes 1 --b-adapt 0 --b-pyramid strict --cabac --qpmin 1 --qpmax 51 --qpstep 25 --ratetol 35 --qcomp 0.75 --direct auto --partitions i8x8,p8x8,b8x8 --8x8dct --psy-rd 0.4,0 --trellis 0Everything bolded here made me want to stab myself with a rusty spoon. If you don't understand why each of those options makes me want to acquire some corroded utensils, you shouldn't be using them.I would like to ask if you guys don't mind helping me refine these settings to make encoding more efficient for compression and speed.I think you need to throw everything out and settle on one or two simple commandlines with about 1/3 as many options each, because your current options are completely nonsensical. In fact, you could get quite good results simply by dropping every single option listed there.

It is posts like this that make me want to pull a Theora and remove almost all user-facing options completely because if you give someone a button, they will press it. And even after it shoots them in the leg, they'll go limping around for more buttons to press!

[Blue_MiSfit]

LOL!!!!

Oh man..

He's right, you know - all drama set aside :)

Just use something very simple, like the defaults!!!! If it's too slow, speed it up with --preset fast or something. If you can tolerate slower speeds, then use --preset slow, slower, or veryslow.

Add --tune film or --tune animation if you want.

And of course, do specify the rate control mode you're interested in. If you must have control over file size, 2 pass is the way to do it. If you can afford some unpredictibility, then CRF with some VBV isn't a terrible idea.

~MiSfit

[Digital Corpus]

Oh dear...Everything bolded here made me want to stab myself with a rusty spoon. If you don't understand why each of those options makes me want to acquire some corroded utensils, you shouldn't be using them.I think you need to throw everything out and settle on one or two simple commandlines with about 1/3 as many options each, because your current options are completely nonsensical. In fact, you could get quite good results simply by dropping every single option listed there.
I recently found the x264 wiki. I never knew it existed so I've been operating off of documentation in mplayer and google. I was hoping for a more mature response that actually was useful in helping me understand the error of my ways. I gave you a breif overview as to why I have all the options listed as I cannot use the presets via mplayer afaik.

Not using MB-tree was a decision I had to make since it increase encoding time by about 25% when I first started using it. I have a copy of x264 from 11-24-09. I've tried manually compiling mplayer and x264 separately instead of through portage, but mplayer isn't reading the x264 install. Until that is fixed, I cannot easily tell how much the recent fixes have sped up mbtree so I have it disabled.

As for mixed-refs, I didn't see enough of a reason to unset it. I have it listed so I know what is default so I know it is redundant.

The combination of bframe settings was picked due to seeing an increase in psnr and ssim while still maintaining faster than real time encoding speed. I would use my own eye to judge this, but they become glazed over when watching the same clip more than about 5 times. If you want to knock me for this logic, explain why.

Now for the various qp settings, I set those ages ago and see through the wiki that the options I have are fairly stupid. This kind of detail should be listed under the logical "--fullhelp" since it's kinda fairly useful. If poor documentation makes for poor decisions, then don't get mad at the user for uninformed decisions.

Partition selections and 8x8dct are chosen for speed reasons and negligible differences in qualitly. The most understandable explination as to how these related to each other I found in mplayer's man page:
partitions=<list>
Enable some optional macroblock types (default: p8x8,b8x8,i8x8,i4x4).
p8x8 Enable types p16x8, p8x16, p8x8.
p4x4 Enable types p8x4, p4x8, p4x4. p4x4 is recommended only with subq >= 5, and only at low res-
olutions.
b8x8 Enable types b16x8, b8x16, b8x8.
i8x8 Enable type i8x8. i8x8 has no effect unless 8x8dct is enabled.
i4x4 Enable type i4x4.
all Enable all of the above types.
none Disable all of the above types.
Regardless of this option, macroblock types p16x16, b16x16, and i16x16 are always enabled.
The idea is to find the type and size that best describe a certain area of the picture. For example,
a global pan is better represented by 16x16 blocks, while small moving objects are better represented
by smaller blocks.
I am not using i4x4, p4x4, or b4xb for reasons explained in the above excerpt, and for that the resolutions I have setup to work with these settings, a 4x4 pixel area is roughly equivalent to the noise in the source materials I'm working with. Not using them noted a very small penalty in quality and a lot more speed.

It is posts like this that make me want to pull a Theora and remove almost all user-facing options completely because if you give someone a button, they will press it. And even after it shoots them in the leg, they'll go limping around for more buttons to press!
If x264 had a bit more documentation in it's install, I bet post like this would happen less often. The info directly from x264 is cryptic at best. Even mplayer's documentation and explanation is better despite it being outdated. Not all of us know or understand how everything works like you do. I already explained why I have the options set as I do, please read what I have posted. If I had already found a fast resource economical way to keep A/V sync integrity over less than optimal sources from mplayer and passing a YUV stream directly to x264, I wouldn't be pressing all the buttons. When managing over 3TiB of data of video files in my spare while having a life, it's easier to just have everything listed and set in the script instead of find out out one days everything just slowed down; e.g. the mbtree addition caught me off guard and reduced my capacity to encode video by a very large margin. I already asked for a little respect when posting this and indirectly said that something is bound to be "stupid" rubbing it in my face without saying why is highly immature.

[Dark Shikari]

If x264 had a bit more documentation in it's install, I bet post like this would happen less often.It is not x264's job to teach you how video encoding works. That's the job of a textbook.

If you don't want to learn how the internals of x264 work, use the presets. x264 --fullhelp lists them, so you can easily copy them over to mencoder.The info directly from x264 is cryptic at best. Even mplayer's documentation and explanation is better despite it being outdated. Not all of us know or understand how everything works like you do.And here you completely miss the point. The documentation is "cryptic" because it is not intended for you. It is intended for video professionals who know what the options mean. Ordinary users who don't understand the internals should not be messing with dozens of obscure options. The basic options that an ordinary user should need are listed in the --help: everything beyond that shouldn't be touched unless you know what you're doing.I was hoping for a more mature response that actually was useful in helping me understand the error of my ways.What else is there to explain? You arbitrarily poked buttons that you weren't supposed to poke, and then complained when I told you to stop poking them.

[Digital Corpus]

LOL!!!!

Oh man..

He's right, you know - all drama set aside :)

Just use something very simple, like the defaults!!!! If it's too slow, speed it up with --preset fast or something. If you can tolerate slower speeds, then use --preset slow, slower, or veryslow.

Add --tune film or --tune animation if you want.

And of course, do specify the rate control mode you're interested in. If you must have control over file size, 2 pass is the way to do it. If you can afford some unpredictibility, then CRF with some VBV isn't a terrible idea.

~MiSfit
I understand that, but I was hoping for a response a bit more professional such as yours. I know it's a long post, but I stated why I haven't used the presets. I have kept this simple, but unfortunately I need reliability since I'm going through about 15 hours a week of video in several different resolutions. Letting mplayer/mencoder handle possibly broken transport streams is simpler than demuxing the video with projectX, finding where the audio gaps are if just the audio corrupted and then doing the same with video, and then reassembling everything together and passing it through x264 with presets to use. Granted mencoder will have to slew the audio or video to bring them back in sync, but it is simpler since I don't have to deal with that. As I "have" to use mplayer, the presets were not an option. Though at the same time, mbtree isn't an option that exists in mencoder but I can still pass it to the x264 code. That was a surprise and I have to admit I haven't tested if the preset will work the same way since it wasn't broken, I didn't want to break it to fix it with my not so much free time.

[Dark Shikari]

I understand that, but I was hoping for a response a bit more professional such as yours. I know it's a long post, but I stated why I haven't used the presets. I have kept this simple, but unfortunately I need reliability since I'm going through about 15 hours a week of video in several different resolutions. Letting mplayer/mencoder handle possibly broken transport streams is simpler than demuxing the video with projectX, finding where the audio gaps are if just the audio corrupted and then doing the same with video, and then reassembling everything together and passing it through x264 with presets to use.You have yet to explain what is stopping you from using the presets in mencoder. It's a simple matter of reading the --fullhelp and using your copy and paste buttons.Granted mencoder will have to slew the audio or video to bring them back in sync, but it is simpler since I don't have to deal with that. As I "have" to use mplayer, the presets were not an option. Though at the same time, mbtree isn't an option that exists in mencoder but I can still pass it to the x264 code.What do you mean, "option that exists in mencoder"? mencoder passes all options blindly to x264. It does not have its own option system. Any option that is in libx264 is in mencoder.

[Digital Corpus]

It is not x264's job to teach you how video encoding works. That's the job of a textbook.

If you don't want to learn how the internals of x264 work, use the presets. x264 --fullhelp lists them, so you can easily copy them over to mencoder.And here you completely miss the point. The documentation is "cryptic" because it is not intended for you. It is intended for video professionals who know what the options mean. Ordinary users who don't understand the internals should not be messing with dozens of obscure options.What else is there to explain? You arbitrarily poked buttons that you weren't supposed to poke, and then complained when I told you to stop poking them.
Please don't choose your time mocking me instead. That is simply irrational and counter-intuitive. I'm not asking for a full explanation, but even a copy of the wiki into --fullhelp so it is then "full" seems logical. Everyone has to start somewhere. Search works, but only if you think of the exact way that someone else does to pull up the info you need. If you don't want to provide *some* help, why post an answer?

I thought my first post was concise enough to explain why I had to choose the route I did so I would not be mocked. I was wrong. I didn't want to write an essay explaining the countless hours I've spent staring at dozens of video encodings, looking at metrics, and comparing encoding times and the decisions I made based on all of that. Half of what I read on these boards states that the quality is subjective. If that is true, then why mock me for finding something that works for me?

[Digital Corpus]

You have yet to explain what is stopping you from using the presets in mencoder. It's a simple matter of reading the --fullhelp and using your copy and paste buttons.What do you mean, "option that exists in mencoder"? mencoder passes all options blindly to x264. It does not have its own option system. Any option that is in libx264 is in mencoder.
I found that out by fluke, and even though it does do that, it doesn't use the same syntax.

What i have pasted above, looks like this in mencoder:
nombtree:fast_pskip:nodct_decimate:keyint=$max_key:keyint_min=$min_key:mixed_refs:deblock=2,2:bframes=1:nob_adapt:b_pyramid=strict:cabac:qp_min=1:qp_max=51:qp_step=25:ratetol=35:qcomp=0.75:direct_pred=auto:partitions=i8x8,p8x8,b8x8:8x8dct:psy-rd=0.4,0:trellis=0
That is hardly a blind pass.

[Dark Shikari]

Please don't choose your time mocking me instead.

{enormous amounts of tl;dr omitted}Wow. I try to help, I give you instructions on how to get better results, I explain the answer to many of your questions, and now I get accused of mocking you.

I would advise you next time to not attack the people who are trying to help you.

Good luck finding someone to spend hours for free trying to meet your unreasonable expectations.

[Digital Corpus]

Wow. I try to help, I give you instructions on how to get better results, I explain the answer to many of your questions, and now I get accused of mocking you.

I would advise you next time to not attack the people who are trying to help you.

What else is there to explain? You arbitrarily poked buttons that you weren't supposed to poke, and then complained when I told you to stop poking them.
Because I didn't saw how exactly I arrived at each decision by testing each one and looking at the result does not mean I made arbitray decisions. You keep referring to what I've done as arbitrary. You made comments about stabbing yourself with rusty utensils after I said I spent a year of my spare time trying to understand these options and testing them. How is that not mocking me?

[Digital Corpus]

And of course, do specify the rate control mode you're interested in. If you must have control over file size, 2 pass is the way to do it. If you can afford some unpredictibility, then CRF with some VBV isn't a terrible idea.

~MiSfit

I'll probably give this a shot. Will the size output be constrained so that it will not go beyond a certain size due to the VBV algorithms? I have no problem with going smaller and saving bits. I'm mostly worried about over allocation and going beyond what I can reasonably expect. Would I notice any speed difference between CRF and ABR?

[`Orum]

At the risk of possibly making things worse, I suggest you check out MeWiki (http://mewiki.project357.com/wiki/X264_Settings). Just don't use MeGUI for encoding, but some of the info on the wiki is helpful.

[Digital Corpus]

That is the wiki I came across earlier today. Thank you though. I am changing my qp settings to something sane as a result. I have a background in mathematics though through multivariable Calculus and I'm a professional photographer. Already stated I'm by no means perfect, but the only way to make a good decision if I think I want to tweak something to see how it comes out, the more I need to know. cplxblur is the only other option I've considered changing.

[AnonCrow]

check out MeWiki (http://mewiki.project357.com/wiki/X264_Settings).
That is the wiki I came across earlier today.

May also want to check Avidemux wiki (http://avidemux.org/admWiki/index.php?title=H264) on H.264. Though slightly outdated (eg. no subme:10) but IMHO explains things in bit more newbie-friendly way.

[nm]

That is the wiki I came across earlier today. Thank you though. I am changing my qp settings to something sane as a result.

Do try the presets too. Developers have selected them carefully to achieve good results at different speed levels. There should be no need to make additional resolution-specific tunings unless you are targeting certain playback devices. Just use a faster preset for higher resolutions to speed the encoding up.

[Asmodian]

Because I didn't saw how exactly I arrived at each decision by testing each one and looking at the result does not mean I made arbitray decisions. You keep referring to what I've done as arbitrary. You made comments about stabbing yourself with rusty utensils after I said I spent a year of my spare time trying to understand these options and testing them. How is that not mocking me?

I am sorry Digital Corpus but the problem is that Dark Shikari and team has picked the defaults to do exactly what you say you want to do with options to chose the encoding speed and to tune for a variety of sources. From your command line it is obvious that you do not understand what all the options do or how they impact quality vs encode time.

Dark may not phrase everything as politely as one might want but he is offering sincere advice with strong imagery to get the point across that you do not need to mess with weird settings, just use the presets. From the beginning there have been a lot users in the forums posting command lines that have a lot of problems in them which is why the presets were created in the first place. Seeing users not using them and choosing very odd options which results in video that takes longer to compress and/or looks worse then it should for the size would be hard to face after spending all that thought setting up the presets (and improving x264 so much! Dark has a lot invested in the quality of x264).

Ok, my more then "use the presets":

Please DO NOT turn off mb-tree to increase encode speed!

You have no-fast-pskip on and mb-tree off, the first for quality and the second for speed I assume? mb-tree is better than not doing fast-pskip by a long shot. If you are just trying to trade quality for speed at the higher resolutions using the speed presets, or the settings as they set if you cannot use them for some reason, will provide a better trade off; more speed for less loss in quality per byte, eg. which --me and --subme to pick for speed vs. quality. Opps, I did a "use the presets" after all. :p

Just delete everything after --keyint unless you are targeting a device that doesn't support cabac?

Why are you messing with de-block? That looks like like very blurry settings at the higher reses.

If b-frames speed up your encode and improve quality at higher resolutions why turn them off at the lower ones?

Try some crf encodes, it does provides a very significant difference in quality/speed and I have never had a run-away bitrate using crf if set near the default of 23 (though crf <17 can get very large sometimes, noisy sources etc.). crf is the most efficient mode for quality per size per encode time and the videos that are larger will have needed the size to get the same quality. It is sort of like doing a two-pass encode across all the videos in your collection (taking bits from easy to encode videos and giving them to the hard to encode ones), not that it works like that at all but it sort of results in it.

x264 has been changing pretty fast so do not keep too much of what you know of the different options from older versions.

I hope this helps and you can get past the "those options are crazy" sentiment and see why we think that. :)

[Blue_MiSfit]

Let's get this sorted. It doesn't need to be complicated or dramatic.

I'll probably give this a shot. Will the size output be constrained so that it will not go beyond a certain size due to the VBV algorithms? I have no problem with going smaller and saving bits. I'm mostly worried about over allocation and going beyond what I can reasonably expect. Would I notice any speed difference between CRF and ABR?

VBV is more for enforcing peak bitrate limitations for hardware or streaming compatibility. in your case, it may not be necessary.

ABR and CRF are similar in terms of speed, though CRF will give vastly superior quality at the cost of unpredictable file sizes.

Now, as I understand it, you want to use mplayer to decode your sources so you can take advantage of its error fixing magic, and pipe the result into x264's CLI interface. That should work perfectly fine, and will enable you to "use the presets", as we've all been screaming at you to do :)

Do throw everything you know about settings out the window, and make another evaluation of quality:speed using only the presets :) Much changes, and the presets are very well chosen.

~MiSfit

[Digital Corpus]

@Asmodian
Thank you. I was hoping for something a bit more like that. I can understand the built up frustration that takes place by incessant postings like this, but that doesn't excuse the lack of civility.

I have deblocking turned up dues to the nature of MPEG-2 video from my local networks. What is broadcast in my area always has some minor blocking issues and depending on which network I'm recording from, it can actually be pretty bad. That seems to be my optimal range from the tests I've done and it doesn't affect quality more than the visible macroblocks already present.

I turned bframes off due to platform issues. I would like to maintain iPod/iPhone compatibility. I have my server setup with http access...

I'm trying to keep up. Thank you for a this for that take on various setitngs. I was hoping for information such as that to work from.


@Blue_MiSfit
Generally true, but I see VBV as a two fold option. As kinda hinted at, I have this setup for friends/coworkers that don't have their own DVR and would like to grab shows to watch. I'm mostly targeting the Xbox 360 and the iPhone/iPod Classic. Since this kind of requires web access, I thought I might as well make things streaming compatible if I could. That and if I can guarantee that a CRF encode won't surprise me with a large file size, then VBV will serve a purpose there. Unless you have a better suggestion.

If you can provide some help with me getting mplayer to pipe to x264, I'm all for it. My current attempts have failed, but there has to be a way.

Each revision I basically did that. If I have to do it again, I will. I have about 36 hours to play with this until I have to go back to work

[nm]

I have deblocking turned up dues to the nature of MPEG-2 video from my local networks. What is broadcast in my area always has some minor blocking issues and depending on which network I'm recording from, it can actually be pretty bad. That seems to be my optimal range from the tests I've done and it doesn't affect quality more than the visible macroblocks already present.

Have you tried deblocking the source with MEncoder's filters? That should give better results.


Generally true, but I see VBV as a two fold option. As kinda hinted at, I have this setup for friends/coworkers that don't have their own DVR and would like to grab shows to watch. I'm mostly targeting the Xbox 360 and the iPhone/iPod Classic. Since this kind of requires web access, I thought I might as well make things streaming compatible if I could. That and if I can guarantee that a CRF encode won't surprise me with a large file size, then VBV will serve a purpose there. Unless you have a better suggestion.

You also need VBV for device compatibility, even when using ABR or 2-pass rate control.


If you can provide some help with me getting mplayer to pipe to x264, I'm all for it. My current attempts have failed, but there has to be a way.

There are many ways, but using yuv4mpeg and a FIFO is pretty easy on Linux:

mkfifo pipe.y4m
x264 --preset fast --crf 22 -o output.264 pipe.y4m &
mplayer -really-quiet -vo yuv4mpeg:file=pipe.y4m input.ts

Audio needs to be encoded separately and then muxed with the video to an appropriate container.


Or just copy the preset settings over to MEncoder as Dark Shikari suggested. Remember that the defaults (no settings) are the same as the medium preset, also in MEncoder, so you only need to add settings listed by x264 --fullhelp for each preset.

For example, to encode with the fast preset, put rc_lookahead=30:ref=2:subme=6 to x264encopts. The following command does the same thing as the piping procedure did above:
mencoder -ovc x264 -x264encopts crf=22:rc_lookahead=30:ref=2:subme=6 -nosound -of rawvideo -o output.264 input.ts

[Caroliano]

@Asmodian: I have deblocking turned up dues to the nature of MPEG-2 video from my local networks. What is broadcast in my area always has some minor blocking issues and depending on which network I'm recording from, it can actually be pretty bad. That seems to be my optimal range from the tests I've done and it doesn't affect quality more than the visible macroblocks already present.
That is not the correct use for the in-loop deblocking filter of H.264. This deblocking filter is applied after the frame is encoded, and before the frame is used as reference, and is intended to fight only the new blocking created by the encoder.

The best you can do is to use and deblocking/denoising filter before the encoding, in order to feed a cleaner video to x264. And then lower a little your deblock, depending on your taste for bluriness vs noise.

[Digital Corpus]

<snip>
You also need VBV for device compatibility, even when using ABR or 2-pass rate control.

...

Audio needs to be encoded separately and then muxed with the video to an appropriate container.
</snip>
I've never had a problem with any of the video I've outputted on most devices such as the iPod because I'm usuaing only about half of the specified max bitrate. Also, doesn't ratetol take care of this as well? With CRF I will be using VBV though.

Unless you have a way to solve my A/V issues, audio and video get encoded at the same time. My current implementation outputs audio to raw PCM and then I encode that to aac after normalizing and compressing the dynamic range a bit. I then mux everything back together into a mpeg-4 container. Can x264 just ignore a video with audio already in it?


@Caroliano
This would be the pp video filter using vb and hb or va and ha subfilters, yes? I'll start playing with it when I settle on my CRF settings to get the speed I need for my existing video. How much of a slowdown would I be expecting from the extra processing would you guess?

[Blue_MiSfit]

VBV will not clamp down the average bitrate of a CRF encode! It can only control bitrate peaks.

Do not expect VBV to save you from having occasionally oversized files. It will not do this! Its sounds like you need to have some shell scripting logic to determine whether a file is actually "too big", and re-encode it at a higher CRF or whole hog two pass as a fallback. Do use VBV set to the profile and level you're using. For example, if you're making an iPod encode at Baseline @ L3, then set the vbv-bufsize and vbv-maxrate to 10000. This will ensure that the encoder will never use more than 10,000,000 bits in any given second, but still allow for instantaneous spikes etc. If you are doing encodes at High @ 4.1, then you can increase this to 62500 for each - as High 4.1 allows for 62.5mbps. You can see the specs for each profile and level on Wikipedia: http://en.wikipedia.org/wiki/H.264#Levels

Also, I second the notion of using MPlayer's deblocking / denoising filters in place of increasing the deblocker settings in x264. If you can set this up correctly, you will get better overall results, and a sharper overall output.

~MiSfit

[nm]

I've never had a problem with any of the video I've outputted on most devices such as the iPod because I'm usuaing only about half of the specified max bitrate. Also, doesn't ratetol take care of this as well?
Nope, you need VBV to make sure the bitrate never exceeds hardware limits.

Unless you have a way to solve my A/V issues, audio and video get encoded at the same time.
You can very well continue using x264 through MEncoder if you don't want to tweak and test alternative solutions.

Can x264 just ignore a video with audio already in it?
I'm not sure what you mean.

This would be the pp video filter using vb and hb or va and ha subfilters, yes?
That's one alternative. There's also pp7, fspp and spp. Uspp is too slow to be useful.

[Digital Corpus]

Nope, you need VBV to make sure the bitrate never exceeds hardware limits.
Okay, that makes more sense now.
You can very well continue using x264 through MEncoder if you don't want to tweak and test alternative solutions.
Do you know of a way I can automate fixing A/V sync issues from demuxed audio and video when I have OTA broadcasts that brake down?
I'm not sure what you mean.
I know x264 will obviously only work with video, but will it discard the audio if I pass a file/pipe of muxed video and audio, or will it error out? I hope that is explaining it better.
That's one alternative. There's also pp7, fspp and spp. Uspp is too slow to be useful.
I'll compare the two when I find a crf and preset/tune that gives me the same or comparable speed/quality that I had before.


So far, using CRF and defaults is saving me about 20-30% the number of bits, but encoding times are slower. I still have more test encodes to work on.

[Caroliano]

@Caroliano
This would be the pp video filter using vb and hb or va and ha subfilters, yes? I'll start playing with it when I settle on my CRF settings to get the speed I need for my existing video. How much of a slowdown would I be expecting from the extra processing would you guess?
It seems so. I don't have much experience with mencoder/mplayer filters... But they are probably quite fast, as they are intended to work in realtime for postprocessing. At least one of them should be so.

[Digital Corpus]

Preliminary results seem okay. I had to pick a couple options from --preset faster. On my first test video, with deblocking from mplayer, I come closer to my time limit for a 60 minute recording for 1080i to 720p, but according to metrics, and perceptible quality all looks to be okay.

old:
x264 [info]: SSIM Mean Y:0.9863154
x264 [info]: PSNR Mean Y:46.807 U:50.211 V:51.402 Avg:47.680 Global:46.693 kb/s:5054.81
real 58m52.206s

new:
x264 [info]: SSIM Mean Y:0.9880604
x264 [info]: PSNR Mean Y:47.307 U:51.002 V:52.043 Avg:48.217 Global:47.523 kb/s:3704.70
real 59m36.614s

Given the file size difference, the change is fairly dramatic.

Now please correctly my thinking here as I'm finding it hard to find information on why weighted P frames was introduced recently. Some searches I've tried don't reveal much for me unfortunately, but it's is supposed to help fades out, correct? As I'm dealing with sources that tend to need a fair amount of deblocking on scene changes and fades (to black or crosses) and especially those scene changes, the benefits of having weightp on would then be lost (if memory serves). I chose to turn it off over --mixed-refs to help speed up encoding. Is this wise?

I still need to test the combination of settings with other sources/channels, though a fairly noticeable bump in visible quality while droping the bitrate by ~25% is kind of awesome. This is about 50% less bits than when I originally started :thanks:

[nm]

Do you know of a way I can automate fixing A/V sync issues from demuxed audio and video when I have OTA broadcasts that brake down?
I don't think there's a way. Errors would need to be fixed while demuxing by dropping broken audio frames and video GOPs. Perhaps xport (http://forum.doom9.org/showthread.php?p=1255343#post1255343) would do this for you?

You could also try mplayer -vo null -ao pcm:file=out.wav to decode and pipe audio to NeroAACEnc and then pipe video to x264 in another run with mplayer -ao null -vo yuv4mpeg. If that doesn't work well enough, just use MEncoder as you did before.

I know x264 will obviously only work with video, but will it discard the audio if I pass a file/pipe of muxed video and audio, or will it error out?
If you use x264's libavformat or FFMS2 support to input a muxed file or stream, it will discard the audio. When using raw video inputs, there's no way to include audio at all.

[Blue_MiSfit]

Don't use PSNR / SSIM for real quality analysis!!! Especially with psy options turned on. You need to make a visual comparison in cases like this.

Also, don't pick and choose options from --preset faster or --preset fast. Use one or the other :)

If you need more granularity, then we can discuss what can be dropped to be least harmful to quality.

~MiSfit

[Digital Corpus]

I don't think there's a way. Errors would need to be fixed while demuxing by dropping broken audio frames and video GOPs. Perhaps xport (http://forum.doom9.org/showthread.php?p=1255343#post1255343) would do this for you?

You could also try mplayer -vo null -ao pcm:file=out.wav to decode and pipe audio to NeroAACEnc and then pipe video to x264 in another run with mplayer -ao null -vo yuv4mpeg. If that doesn't work well enough, just use MEncoder as you did before.

If you use x264's libavformat or FFMS2 support to input a muxed file or stream, it will discard the audio. When using raw video inputs, there's no way to include audio at all.
I'll look into xport. I basic compilation in my linux box seems to execute it fine (outputted help info given no parameters), but I actually have to try it out. Only reason I don't process the video and audio separately is because mencoder needs both to be encoded at once in order properly keep A/V sync, else I'd demux the audio and video and process them separately. I'm using faac since I'm on a linux box. It's no frills.

Don't use PSNR / SSIM for real quality analysis!!! Especially with psy options turned on. You need to make a visual comparison in cases like this.

Also, don't pick and choose options from --preset faster or --preset fast. Use one or the other :)

If you need more granularity, then we can discuss what can be dropped to be least harmful to quality.

~MiSfit
Fair enough, but I thought that was only when you compare the effects of psy options.

I need that granularity which is a partial reason why I started this thread. Shall I provide a sample of the source in the form of screen to show the lack of quality I have to combat to help ease the decision making?

[nm]

I'm using faac since I'm on a linux box. It's no frills.

FAAC is pretty bad. NeroAACEnc distribution includes Linux binaries (x86 only), or you can run Windows binaries with Wine. Both alternatives work very well.


Shall I provide a sample of the source in the form of screen to show the lack of quality I have to combat to help ease the decision making?

Video clip is better (both the source and your encode). Then we can try things and possibly come up with better solutions. Providing a sample clip for testing is considered fair use.

[Digital Corpus]

I don't have time to check this atm, but does NeroAAC allow CLI interaction? I have everything setup via bash scripts since this is automated with many hours of video each week.


Here is a 5 minute clip of NCIS. It weighs in a 525MB:
http://atlas.selfip.net/NCIS-5min.ts

House, 508MB:
http://atlas.selfip.net/House-5min.ts

Castle, 407MB:
http://atlas.selfip.net/Castle-5min.ts

Each of these are a different network and are the networks I deal with primarily.

I should mention that these are on a 25Mbps up pipe so usually there won't be bandwidth limitations on my end.

[Blue_MiSfit]

Yes, NeroAACEnc is a CLI application. There are various GUIs for it :)

~MiSfit