Choose the bitrate ?

[sirt]

Hello,

Here I come with another intricate issue ; I've been roaming on this forum since months reading about what I am about to ask you for. Let's say I plan to encode in two pass, like this :

(My file has 100 frames and is at 23.976 img/s)

x264.exe test.avi --sar 10:11 --preset veryslow --slow-firstpass --keyint 240 --min-keyint 23 --scenecut 40
--bframes 16 --b-adapt 2 --b-pyramid normal --ref 5
--deblock 0:0 --rc-lookahead 60 --aq-mode 2 --partitions all --direct auto --me tesa --merange 32 --subme 10 --bitrate ??? --pass 1 -o test_pass1.mkv 2>test_pass1.log

x264.exe test.avi --sar 10:11 --preset veryslow --keyint 240 --min-keyint 23 --scenecut 40
--bframes 16 --b-adapt 2 --b-pyramid normal --ref 5
--deblock 0:0 --rc-lookahead 60 --aq-mode 2 --partitions all --direct auto --me tesa --merange 32 --subme 10 --bitrate ??? --pass 2 -o test_pass2.mkv 2>test_pass2.log

As you see I haven't define any bitrate yet. According to the parameters intended how could I or should choose the bitrate to be consistent with these ?

In my mind, it stands to reason that I should not go below a certain value. For example, it would be insane ft I chose to put --bitrate 1 in such circumstances because - I think - it is similar to ask an eldery people for running the 100 m under 9'58 seconds. On the other hand, if I set an extremely high bitrate like --bitrate 10000 it is clear the encoder will properly achieve his job. So I reformulate my question : when knowing the number of frames of a video, its framerate and the x264 parameters intended is there some way to *calculate* a bitrate that will allow the encoder do its job without fail ? By "job", I mean "work well" with the given parameters. To exemplify : --bitrate 1 will certainly not enable it to do anything good with the comand line below while --bitrate 10000 will but it will be somewhat *too much*. So from where can you be sure the encoder will *really* do the job you ask him for ?

[nm]

So I reformulate my question : when knowing the number of frames of a video, its framerate and the x264 parameters intended is there some way to *calculate* a bitrate that will allow the encoder do its job without fail ?

Yes, CRF.

[Groucho2004]

I've been roaming on this forum since months reading

So, after months of reading about this very subject, having received hundreds of answers to your questions, you come up with insane command lines like this?

You also know that the required bitrate depends heavily on the source.

Is this a joke?

[sirt]

Not at all ! It is not a joke...in fact forget about my comand line (I didn't even plan to use it anyway), the question is : given a comand line with such and such settings, can you do what I asked at the beginning ?


nm

Well I want to encode in two pass.

Groucho2004

Of course it depends on the source. Of course I can use a *bitrate calculator* but that is not my problem. Let's be more simple : when you encode a video how do you choose the bitrate (assuming you want to encode in two pass) ? Would you, for example always use a typical value depending on the lengh (like 2500 for such video) ? Would you just calculate it according to your needs (like getting an output at a given bitrate) ?

[nm]

Well I want to encode in two pass.

Why? CRF does exactly what you asked for and you get the same quality as if you encoded at the resulting bitrate in 2-pass mode.

[sirt]

Perhaps you have misunderstood me ! Let's say I set CRF 80 ; I guess it is not a good idea with the insane settings below. Then, you will probably retort : give a try to several CRF values until you are satisfied. It means I will have to do several encodes until I think it is *good* (and I can be wrong in my judgment). It seems it is impossible to find an answer to my question I guess. In this case, nm how do you encode your videos ?

[SassBot]

Perhaps you have misunderstood me ! Let's say I set CRF 80 ; I guess it is not a good idea with the insane settings below. Then, you will probably retort : give a try to several CRF values until you are satisfied.

Just do a couple of fast first passes using your test CRF values. You won't get the exact same result as what the full settings pass will give you but you can get a decent enough idea of what the quality will be like.

It means I will have to do several encodes until I think it is *good* (and I can be wrong in my judgment).

How can you be wrong about whether or not *you* think something looks good?

[nm]

Perhaps you have misunderstood me ! Let's say I set CRF 80 ; I guess it is not a good idea with the insane settings below. Then, you will probably retort : give a try to several CRF values until you are satisfied. It means I will have to do several encodes until I think it is *good* (and I can be wrong in my judgment).

Sure, but you only need to do such an experiment once to get an idea what different CRF values look like. Then you can either choose one balanced value to use in all encodings or tweak it slightly according to specific needs. Lower the value by a notch or two if you want to play it safe against artifacts.

Let's make it even easier... The sane range for lossy re-encodes is about CRF 16 to 28. Most people use values between 18 and 22. If storage space is not a huge issue, 19 is probably a good choice.

nm how do you encode your videos ?

--preset slower --tune film --crf 21

I mostly archive filtered DVB caps, so even though CRF 21 is generally not quite transparent, it's enough for my purpose. If I had high quality Blu-ray sources, I'd probably go to CRF 18 or 19.

[Atak_Snajpera]

just use --CRF 20 as a base and then go up or down according to your sensitivity to details.

[sirt]

Thanks for your answers, but the fact is, regarding my numerous threads since months, I already know that. In fact I wonder if it would be possible to work out - or does that already exist ? - a new kind of algorithm that would ask you for entering numbers such as datas touching the video source (framerate, number of frames...) and the parameters you want to use to encode ; then, averaging some operations inside, this algorithm would give you a *theorical* bitrate B to use in order to make the encoder achieve its job. If you would go under B, it would mean the encoder is unable to do its job and if you would go above B, the encoder would consume extra time for well-nigh result. Am I an utopian ?

[SassBot]

No, there is no magic algorithm to do that because the complexity of each video is not going to be the same and there is no universally accepted tradeoff in quality level for each viewer.

[detmek]

And after all those questions you still don't have a basic knowledge about how encoders work. So no, you are not utopian, just not knowledgable about video encoding.
Number of frames is irelevant for required bitrate. Resolution, FPS and video complexity (complexity of each frame and correlation between frames) are the most important things. You can enter resolution and FPS but how can you enter complexity and correlation it "your" bitrate calculator?
Edit:
Too late.

[sirt]

It's true I can't but that is not what I should focuse on if somebody imagines such type of algorithm. I can't believe most people just use --crf xx and that's all or --bitrate xx ; anyway, apparently it seems that's the case or that you can't do anything more.

[SassBot]

I can't believe most people just use --crf xx and that's all or --bitrate xx ; anyway, apparently it seems that's the case or that you can't do anything more.

What else do you presume they do? Read tea leaves or tarot cards to guess what the quality looks like?

[SassBot]

To further add, even if this magical algorithm you want did exist, how do you expect it to know how good the output is going to be without running an analysis pass on the video?

[Atak_Snajpera]

It's true I can't but that is not what I should focuse on if somebody imagines such type of algorithm. I can't believe most people just use --crf xx and that's all or --bitrate xx ; anyway, apparently it seems that's the case or that you can't do anything more.

if you are not happy with current algorithms create your own. patches are welcome as dark shikari says.

[SassBot]

if you are not happy with current algorithms create your own. patches are welcome as dark shikari says.

The algorithm he means is one that can magically deduce based on frame count, frame size, frame rate and the x264 parameters some "ideal" bitrate so that he doesn't have to run any tests to see if a certain bitrate or CRF value is acceptable quality.

[sirt]

No, I would expect them to do some tests before such as trying a specifical CRF or bitrate on a given scene of a video. But, look my purpose is mostly experimental. I am aware many people perhaps don't care such sequence would need more bits whereas another one needs less bits. That's why I asked lately how to tweak with zone parameter. It could be a good choice if I want to force something somewhere.

Setting --crf or --bitrare looks too easy ; it is like letting the encoder does his job without paying attention to the *complexity* of the video and the constituent sequences. I mean : I would like to *control* somehow what I do. Of course you will retort (ironically) : read about video compression and make your own point of view. But it will take time to do this !

[SassBot]

Setting --crf or --bitrare looks too easy ; it is like letting the encoder does his job without paying attention to the *complexity* of the video and the constituent sequences. I mean : I would like to *control* somehow what I do. Of course you will retort (ironically) : read about video compression and make your own point of view. But it will take time to do this !

What are you talking about? The whole point of a first pass of a 2 pass encode is to analyze the video for things such as complexity. CRF encoding does as well. Where did you get this notion that the video's complexity isn't determined and used by the encoder? You seem to be very confused.

[detmek]

Let me put this from empirical point of view.
If you need to hit specific file size, you use 2-pass encoding and bitrate is already predetermineted as required filesize/total movie time - audio bitrate - container overhead.
If you need specific quality and filesize is not a problem you just use CRF xx.

Now, lets say you don't need to hit some filesize but you do not want for video to exceed certain size. In that case you can encode with CRF xx option and add --pass 1 --slow-firstpass to CMD. If file size goes over limit you can add --pass 2 and replace --crf xx with bitrate that will ensure for encoder to hit filesize you want.

[sirt]

Atak_Snajpera, unfortunately, as usual, you always understand my expectations as if I was not satisfied with current patches. You even ask me for creating own whereas you fittingly know I can't do that. Futhermore, irronicaly I guess you can't do either because I've read somewhere else you don't code in C/C++ . I am NOT critisizing x264, either its patches. I am JUST asking about because I am curious. But it looks like it is not really welcomed.

[Atak_Snajpera]

i guess you have too much time at home if you want manually tweak for every freaking movie.

crf does perfect job. just set value and rest of black job do algos in x264. more complex scenes receive more bits and vice versa.

[sirt]

No, I don't mind about the final size, either the bitrate. If a video would need a bitrate of 100000 kbits/seconds, I would use such insane bitrate.

SassBoot, did I say the complexity is not taken in account by the encoder ? Not at all, but let's say such complexity requires a specifical bitrate of at least 2500 kbits/s to fit in with my parameters. Then, for some reason I would want to reach some size, and in this perspective, I would choose 1000 kbits/s. Then I would be wrong, wouldn't I ? I don't care about "exceeding a size" either as "a specific quality". I think we are circling in fact ! It is not possible to answer such questions !

EDIT : I am not at home, I am doing encoding tests for a video compression firm.

[SassBot]

I am JUST asking about because I am curious. But it looks like it is not really welcomed.

You presume wrongly. Being told that what you ask for is unrealistic is not the same as being told you can't ask.

[Atak_Snajpera]

they knew who to hire :)

[SassBot]

SassBoot, did I say the complexity is not taken in account by the encoder ?

Yes, you did.

it is like letting the encoder does his job without paying attention to the *complexity* of the video

But specifying a CRF or bitrate is NOT doing that.

Not at all, but let's say such complexity requires a specifical bitrate of at least 2500 kbits/s to fit in with my parameters. Then, for some reason I would want to reach some size, and in this perspective, I would choose 1000 kbits/s. Then I would be wrong, wouldn't I ? I don't care about "exceeding a size" either as "a specific quality". I think we are circling in fact ! It is not possible to answer such questions !

You would only be wrong if you didn't like the quality of the output. If you are perfectly fine with the output at 1000kbit/sec why would you care if some algorithm told you otherwise?

[sirt]

they knew who to hire :)

Do you mean I am inept...


SassBoot, because my eyes are certainly not worth a good algorithm !

[SassBot]

SassBoot, because my eyes are certainly not worth a good algorithm !

Actually they are worth far more. Most algorithms that try to model video quality 'objectively' have fundamental flaws to them especially when a video encoder applies psychovisual enhancements.

[Atak_Snajpera]

every human perceive details differently. some do not notice fine detail loss but others will imediately complain about washed out effect. good luck in creating such algo. don't forget that displays are not perfect either. cheap glossy lcds tend to hide blocking / artefacts more than good mat pva panel

[sirt]

Atak_Snajpera, you may help me to write it then ? :D

[vivan]

So you don't even want to decide what quality you want. Than use crf 18. Any other problems?

[sirt]

vivian, I would like an algorithm to decide it in my place or at least to offer suggestions. I am not sure if it is realistic.

A good idea would be to write a code that would scan the video and, given a CRF value set, estimate the average bitrates that will be used when encoding with this CRF value according to the partitions of the video sequences. But it is probably not possible again.

What if somebody manages to write a patch or a code that will allow you to encode in one pass, a bitrate being set, but with same benefits than encoding in two passes ? It would be a trade-off between CRF and 2 Pass. because in the one hand it will be faster than 2 Pass mode and in the other hand you would have an idea of the bitrate used albeit the encoder can fluctuate while doing its job.

[vivan]

I would like an algorithm to decide it in my placecrf 18

or at least to offer suggestions.... and in case you are not happy with it - lower/increase crf value.

A good idea would be to write a code that would scan the video and, given a CRF value set, estimate the average bitrates that will be used when encoding with this CRF value according to the partitions of the video sequences. But it is probably not possible again.bitrate is just a number, it determines ONLY encoded video size.
E.g. you have video. Magic progamm says that using crf 18 resulting bitrate would be 2153 kbit/s... And now what?

What if somebody manages to write a patch or a code that will allow you to encode in one pass, a bitrate being set, but with same benefits than encoding in two passes ? It would be a trade-off between CRF and 2 Pass. because in the one hand it will be faster than 2 Pass mode and in the other hand you would have an idea of the bitrate used albeit the encoder can fluctuate while doing its job.Use 1-pass bitrate mode. Since it doesn't have first pass to analyze video - it can't encode with constant quality, so result wouldn't be optimal. But it does encode video in 1 pass aiming to certain bitrate.

[SassBot]

A good idea would be to write a code that would scan the video and, given a CRF value set, estimate the average bitrates that will be used when encoding with this CRF value according to the partitions of the video sequences. But it is probably not possible again.

Why don't you just run a fast first pass specifying a CRF value and see what the average bitrate comes out to as I told you try previously?

[sirt]

Because I will have to wait until this one pass encode is done in this case. It should be done *instantly* instead but I can wish..

[SassBot]

Because I will have to wait until this one pass encode is done in this case. It should be done *instantly* instead but I can wish..

And how is it going to scan the video file "instantly"? Are you going to use a computer driven by pixie dust? Do you realize how ridiculous this sounds and how much effort you have put in to avoid doing a couple of hours of testing?

[SassBot]

sirt, I've written you the program you want. It's a CLI app that you use in the following way:

zoltarspeaks --frame-count numFrames --frame-dimensions x,y --video-length length-in-minutes --x264-parameters "parameters list"

It will then use its algorithm to nearly instantly give you the ideal CRF value for your video. It can be downloaded here (http://www.sendspace.com/file/6noeb0).

[vivan]

Awesome work!
Feature request: what about increased accuracy? x264 supports not integer values for crf.

[detmek]

Because I will have to wait until this one pass encode is done in this case. It should be done *instantly* instead but I can wish..

You REALLY need to understand how current encoders work. Then, you will know how much your idea is unrealistic.

[SassBot]

Awesome work!
Feature request: what about increased accuracy? x264 supports not integer values for crf.

Ok, now it does CRF values with 2 decimal places. It can be downloaded here (http://www.sendspace.com/file/6noeb0).

[jethro]

Ok, now it does CRF values with 2 decimal places. It can be downloaded here (http://www.sendspace.com/file/6noeb0).

This program is truly a breakthrough.

[AnonCrow]

Because I will have to wait until this one pass encode is done in this case. It should be done *instantly* instead but I can wish..

Define what you mean by instantly.
Under one second , no way;
in ~10 seconds, maybe, depending on your hardware;
in one minute, a half-decent estimate can be made.
If you make ffmpeg choose a few 1000 frame samples for x264 to encode, then you can start making pretty good estimates - for that particular source.

bitrate 1 will certainly not enable it to do anything good with the comand line below while --bitrate 10000 will but it will be somewhat *too much*. So from where can you be sure the encoder will *really* do the job you ask him for ?
Since you never mentioned any frame dimensions (although --sar 10:11 leaves very few possibilities):
A bitrate of 1 kbit would certainly be enough , even at so-called normal framerates, for frames that are 16 by 16 pixels.
Decrease the framerate to a couple pictures per second and you could even increase the frames to 64 by 48 pixels.

10Mbit might be considered too much for 720p60, but not necessarily even adequate for 1080p60 or 2160p30 video.


If you end up using eg. CRF 18 with your SD encodings, you wouldn't go far wrong with increasing the CRF by one for each step up in the resolution - CRF 19 for 720p , 20 for 1080p , 21 for 2160p.

[kypec]

This program is truly a breakthrough.
Yep, I wonder which license model is he going to use for distribution, LGPL or maybe even open-source? I bet it's multi-platform already but some nice GUI wouldn't harm either!:D

[sirt]

SassBot, I am afraid to download your program...

[Atak_Snajpera]

SassBot, I am afraid to download your program...
This app is truly amaizing !!! It will solve all your troubles ! Don't even hesitate ! Maybe you will get even a raise at work ...

[sirt]

Lol...how about it is so small then ? :D Otherwise, SassBot you should probably provide us with x265.exe because we all know it is a personal projet completed since edges by you :p

Anyway, I'm about to forestall you because I start coding x267, so if somebody is interested in the old x266...it only needs half a second to encode a video with placebo settings (in 6 pass) : http://www.sendspace.com/file/byszou

[SassBot]

It's so small because of my highly advanced algorithms combined with breakthroughs in optimization that have never been done before.

[AnonCrow]

It's so small because of my highly advanced algorithms combined with breakthroughs in optimization that have never been done before.
Even with a little optimization, I could easily see it being 10 times smaller. With a lot of optimization, even a 100 times smaller. A thousand times smaller might be pushing it.