Can I make these options more efficient?

[Blue_MiSfit]

Let's get this sorted. It doesn't need to be complicated or dramatic.

Originally Posted by Digital Corpus

I'll probably give this a shot. Will the size output be constrained so that it will not go beyond a certain size due to the VBV algorithms? I have no problem with going smaller and saving bits. I'm mostly worried about over allocation and going beyond what I can reasonably expect. Would I notice any speed difference between CRF and ABR?

VBV is more for enforcing peak bitrate limitations for hardware or streaming compatibility. in your case, it may not be necessary.

ABR and CRF are similar in terms of speed, though CRF will give vastly superior quality at the cost of unpredictable file sizes.

Now, as I understand it, you want to use mplayer to decode your sources so you can take advantage of its error fixing magic, and pipe the result into x264's CLI interface. That should work perfectly fine, and will enable you to "use the presets", as we've all been screaming at you to do

Do throw everything you know about settings out the window, and make another evaluation of quality:speed using only the presets Much changes, and the presets are very well chosen.

~MiSfit

[Digital Corpus]

@Asmodian
Thank you. I was hoping for something a bit more like that. I can understand the built up frustration that takes place by incessant postings like this, but that doesn't excuse the lack of civility.

I have deblocking turned up dues to the nature of MPEG-2 video from my local networks. What is broadcast in my area always has some minor blocking issues and depending on which network I'm recording from, it can actually be pretty bad. That seems to be my optimal range from the tests I've done and it doesn't affect quality more than the visible macroblocks already present.

I turned bframes off due to platform issues. I would like to maintain iPod/iPhone compatibility. I have my server setup with http access...

I'm trying to keep up. Thank you for a this for that take on various setitngs. I was hoping for information such as that to work from.


@Blue_MiSfit
Generally true, but I see VBV as a two fold option. As kinda hinted at, I have this setup for friends/coworkers that don't have their own DVR and would like to grab shows to watch. I'm mostly targeting the Xbox 360 and the iPhone/iPod Classic. Since this kind of requires web access, I thought I might as well make things streaming compatible if I could. That and if I can guarantee that a CRF encode won't surprise me with a large file size, then VBV will serve a purpose there. Unless you have a better suggestion.

If you can provide some help with me getting mplayer to pipe to x264, I'm all for it. My current attempts have failed, but there has to be a way.

Each revision I basically did that. If I have to do it again, I will. I have about 36 hours to play with this until I have to go back to work

[nm]

Originally Posted by Digital Corpus

I have deblocking turned up dues to the nature of MPEG-2 video from my local networks. What is broadcast in my area always has some minor blocking issues and depending on which network I'm recording from, it can actually be pretty bad. That seems to be my optimal range from the tests I've done and it doesn't affect quality more than the visible macroblocks already present.

Have you tried deblocking the source with MEncoder's filters? That should give better results.

Generally true, but I see VBV as a two fold option. As kinda hinted at, I have this setup for friends/coworkers that don't have their own DVR and would like to grab shows to watch. I'm mostly targeting the Xbox 360 and the iPhone/iPod Classic. Since this kind of requires web access, I thought I might as well make things streaming compatible if I could. That and if I can guarantee that a CRF encode won't surprise me with a large file size, then VBV will serve a purpose there. Unless you have a better suggestion.

You also need VBV for device compatibility, even when using ABR or 2-pass rate control.

If you can provide some help with me getting mplayer to pipe to x264, I'm all for it. My current attempts have failed, but there has to be a way.

There are many ways, but using yuv4mpeg and a FIFO is pretty easy on Linux:

mkfifo pipe.y4m
x264 --preset fast --crf 22 -o output.264 pipe.y4m &
mplayer -really-quiet -vo yuv4mpeg:file=pipe.y4m input.ts

Audio needs to be encoded separately and then muxed with the video to an appropriate container.


Or just copy the preset settings over to MEncoder as Dark Shikari suggested. Remember that the defaults (no settings) are the same as the medium preset, also in MEncoder, so you only need to add settings listed by x264 --fullhelp for each preset.

For example, to encode with the fast preset, put rc_lookahead=30:ref=2:subme=6 to x264encopts. The following command does the same thing as the piping procedure did above:

mencoder -ovc x264 -x264encopts crf=22:rc_lookahead=30:ref=2:subme=6 -nosound -of rawvideo -o output.264 input.ts

[Caroliano]

Originally Posted by Digital Corpus

@Asmodian: I have deblocking turned up dues to the nature of MPEG-2 video from my local networks. What is broadcast in my area always has some minor blocking issues and depending on which network I'm recording from, it can actually be pretty bad. That seems to be my optimal range from the tests I've done and it doesn't affect quality more than the visible macroblocks already present.

That is not the correct use for the in-loop deblocking filter of H.264. This deblocking filter is applied after the frame is encoded, and before the frame is used as reference, and is intended to fight only the new blocking created by the encoder.

The best you can do is to use and deblocking/denoising filter before the encoding, in order to feed a cleaner video to x264. And then lower a little your deblock, depending on your taste for bluriness vs noise.

[Digital Corpus]

Originally Posted by nm

<snip>
You also need VBV for device compatibility, even when using ABR or 2-pass rate control.

...

Audio needs to be encoded separately and then muxed with the video to an appropriate container.
</snip>

I've never had a problem with any of the video I've outputted on most devices such as the iPod because I'm usuaing only about half of the specified max bitrate. Also, doesn't ratetol take care of this as well? With CRF I will be using VBV though.

Unless you have a way to solve my A/V issues, audio and video get encoded at the same time. My current implementation outputs audio to raw PCM and then I encode that to aac after normalizing and compressing the dynamic range a bit. I then mux everything back together into a mpeg-4 container. Can x264 just ignore a video with audio already in it?


@Caroliano
This would be the pp video filter using vb and hb or va and ha subfilters, yes? I'll start playing with it when I settle on my CRF settings to get the speed I need for my existing video. How much of a slowdown would I be expecting from the extra processing would you guess?

[Blue_MiSfit]

VBV will not clamp down the average bitrate of a CRF encode! It can only control bitrate peaks.

Do not expect VBV to save you from having occasionally oversized files. It will not do this! Its sounds like you need to have some shell scripting logic to determine whether a file is actually "too big", and re-encode it at a higher CRF or whole hog two pass as a fallback. Do use VBV set to the profile and level you're using. For example, if you're making an iPod encode at Baseline @ L3, then set the vbv-bufsize and vbv-maxrate to 10000. This will ensure that the encoder will never use more than 10,000,000 bits in any given second, but still allow for instantaneous spikes etc. If you are doing encodes at High @ 4.1, then you can increase this to 62500 for each - as High 4.1 allows for 62.5mbps. You can see the specs for each profile and level on Wikipedia: http://en.wikipedia.org/wiki/H.264#Levels

Also, I second the notion of using MPlayer's deblocking / denoising filters in place of increasing the deblocker settings in x264. If you can set this up correctly, you will get better overall results, and a sharper overall output.

~MiSfit

[nm]

Originally Posted by Digital Corpus

I've never had a problem with any of the video I've outputted on most devices such as the iPod because I'm usuaing only about half of the specified max bitrate. Also, doesn't ratetol take care of this as well?

Nope, you need VBV to make sure the bitrate never exceeds hardware limits.

Unless you have a way to solve my A/V issues, audio and video get encoded at the same time.

You can very well continue using x264 through MEncoder if you don't want to tweak and test alternative solutions.

Can x264 just ignore a video with audio already in it?

I'm not sure what you mean.

This would be the pp video filter using vb and hb or va and ha subfilters, yes?

That's one alternative. There's also pp7, fspp and spp. Uspp is too slow to be useful.

[Digital Corpus]

Originally Posted by nm

Nope, you need VBV to make sure the bitrate never exceeds hardware limits.

Okay, that makes more sense now.

You can very well continue using x264 through MEncoder if you don't want to tweak and test alternative solutions.

Do you know of a way I can automate fixing A/V sync issues from demuxed audio and video when I have OTA broadcasts that brake down?

I'm not sure what you mean.

I know x264 will obviously only work with video, but will it discard the audio if I pass a file/pipe of muxed video and audio, or will it error out? I hope that is explaining it better.

That's one alternative. There's also pp7, fspp and spp. Uspp is too slow to be useful.

I'll compare the two when I find a crf and preset/tune that gives me the same or comparable speed/quality that I had before.


So far, using CRF and defaults is saving me about 20-30% the number of bits, but encoding times are slower. I still have more test encodes to work on.

[Caroliano]

Originally Posted by Digital Corpus

@Caroliano
This would be the pp video filter using vb and hb or va and ha subfilters, yes? I'll start playing with it when I settle on my CRF settings to get the speed I need for my existing video. How much of a slowdown would I be expecting from the extra processing would you guess?

It seems so. I don't have much experience with mencoder/mplayer filters... But they are probably quite fast, as they are intended to work in realtime for postprocessing. At least one of them should be so.

[Digital Corpus]

Preliminary results seem okay. I had to pick a couple options from --preset faster. On my first test video, with deblocking from mplayer, I come closer to my time limit for a 60 minute recording for 1080i to 720p, but according to metrics, and perceptible quality all looks to be okay.

old:

x264 [info]: SSIM Mean Y:0.9863154
x264 [info]: PSNR Mean Y:46.807 U:50.211 V:51.402 Avg:47.680 Global:46.693 kb/s:5054.81
real    58m52.206s

new:

x264 [info]: SSIM Mean Y:0.9880604
x264 [info]: PSNR Mean Y:47.307 U:51.002 V:52.043 Avg:48.217 Global:47.523 kb/s:3704.70
real    59m36.614s

Given the file size difference, the change is fairly dramatic.

Now please correctly my thinking here as I'm finding it hard to find information on why weighted P frames was introduced recently. Some searches I've tried don't reveal much for me unfortunately, but it's is supposed to help fades out, correct? As I'm dealing with sources that tend to need a fair amount of deblocking on scene changes and fades (to black or crosses) and especially those scene changes, the benefits of having weightp on would then be lost (if memory serves). I chose to turn it off over --mixed-refs to help speed up encoding. Is this wise?

I still need to test the combination of settings with other sources/channels, though a fairly noticeable bump in visible quality while droping the bitrate by ~25% is kind of awesome. This is about 50% less bits than when I originally started

[nm]

Originally Posted by Digital Corpus

Do you know of a way I can automate fixing A/V sync issues from demuxed audio and video when I have OTA broadcasts that brake down?

I don't think there's a way. Errors would need to be fixed while demuxing by dropping broken audio frames and video GOPs. Perhaps xport would do this for you?

You could also try mplayer -vo null -ao pcm:file=out.wav to decode and pipe audio to NeroAACEnc and then pipe video to x264 in another run with mplayer -ao null -vo yuv4mpeg. If that doesn't work well enough, just use MEncoder as you did before.

I know x264 will obviously only work with video, but will it discard the audio if I pass a file/pipe of muxed video and audio, or will it error out?

If you use x264's libavformat or FFMS2 support to input a muxed file or stream, it will discard the audio. When using raw video inputs, there's no way to include audio at all.

[Blue_MiSfit]

Don't use PSNR / SSIM for real quality analysis!!! Especially with psy options turned on. You need to make a visual comparison in cases like this.

Also, don't pick and choose options from --preset faster or --preset fast. Use one or the other

If you need more granularity, then we can discuss what can be dropped to be least harmful to quality.

~MiSfit

[Digital Corpus]

Originally Posted by nm

I don't think there's a way. Errors would need to be fixed while demuxing by dropping broken audio frames and video GOPs. Perhaps xport would do this for you?

You could also try mplayer -vo null -ao pcm:file=out.wav to decode and pipe audio to NeroAACEnc and then pipe video to x264 in another run with mplayer -ao null -vo yuv4mpeg. If that doesn't work well enough, just use MEncoder as you did before.

If you use x264's libavformat or FFMS2 support to input a muxed file or stream, it will discard the audio. When using raw video inputs, there's no way to include audio at all.

I'll look into xport. I basic compilation in my linux box seems to execute it fine (outputted help info given no parameters), but I actually have to try it out. Only reason I don't process the video and audio separately is because mencoder needs both to be encoded at once in order properly keep A/V sync, else I'd demux the audio and video and process them separately. I'm using faac since I'm on a linux box. It's no frills.

Originally Posted by Blue_MiSfit

Don't use PSNR / SSIM for real quality analysis!!! Especially with psy options turned on. You need to make a visual comparison in cases like this.

Also, don't pick and choose options from --preset faster or --preset fast. Use one or the other

If you need more granularity, then we can discuss what can be dropped to be least harmful to quality.

~MiSfit

Fair enough, but I thought that was only when you compare the effects of psy options.

I need that granularity which is a partial reason why I started this thread. Shall I provide a sample of the source in the form of screen to show the lack of quality I have to combat to help ease the decision making?

[nm]

Originally Posted by Digital Corpus

I'm using faac since I'm on a linux box. It's no frills.

FAAC is pretty bad. NeroAACEnc distribution includes Linux binaries (x86 only), or you can run Windows binaries with Wine. Both alternatives work very well.

Shall I provide a sample of the source in the form of screen to show the lack of quality I have to combat to help ease the decision making?

Video clip is better (both the source and your encode). Then we can try things and possibly come up with better solutions. Providing a sample clip for testing is considered fair use.

[Digital Corpus]

I don't have time to check this atm, but does NeroAAC allow CLI interaction? I have everything setup via bash scripts since this is automated with many hours of video each week.


Here is a 5 minute clip of NCIS. It weighs in a 525MB:
http://atlas.selfip.net/NCIS-5min.ts

House, 508MB:
http://atlas.selfip.net/House-5min.ts

Castle, 407MB:
http://atlas.selfip.net/Castle-5min.ts

Each of these are a different network and are the networks I deal with primarily.

I should mention that these are on a 25Mbps up pipe so usually there won't be bandwidth limitations on my end.

[Blue_MiSfit]

Yes, NeroAACEnc is a CLI application. There are various GUIs for it

~MiSfit