how to use zones in x264 cli?

[JoeBG]

Hi,

I got these informations from -h (Help).
--zones <zone0>/<zone1>/...
Tweak the bitrate of some regions of the video
Each zone is of the form
<start frame>,<end frame>,<option>
where <option> is either
q=<integer> (force QP)
or b=<float> (bitrate multiplier)


It would be very nice if someone could tell me how to create a commandline for zones. An example would be enough -thank you very much :)

[Doom9]

I haven't tried but based on the description I'd create a separate end credits zone as follows

--zones 156034,183123,q=42

that would use quantizer 42 for frames 156034 up to 183123. A more complex example:

--zones 0,10231,b=0.25/156034,183123,q=42

In addition to the previous example, this would create another zone from frames 0 - 10231, and multiply the assigned bitrate by a factor of 0.25 (in other words the first 10232 frames would get 1/4 of the bitrate of frames 10232 - 156034).

[lexor]

are zones still cli only? any news on vfw front?

[akupenguin]

zones are also available in mencoder. (same syntax)

[JoeBG]

Originally posted by Doom9
I haven't tried but based on the description I'd create a separate end credits zone as follows

--zones 156034,183123,q=42

that would use quantizer 42 for frames 156034 up to 183123. A more complex example:

--zones 0,10231,b=0.25/156034,183123,q=42

In addition to the previous example, this would create another zone from frames 0 - 10231, and multiply the assigned bitrate by a factor of 0.25 (in other words the first 10232 frames would get 1/4 of the bitrate of frames 10232 - 156034).

Ok, I understand :) But how can I manage it, that the main movie gets more bitrate when I use zones?

Edit:

Here is my first test with a small clip. When I use the following commandline, the encoder only encodes the main movie and does nothing with the defined zone.
x264.exe --bframe 3 --b-pyramid --ref 3 --filter [0:0] --bitrate 700 --ratetol 1.0 --pass 1 --stats "x264.log" --analyse none --direct temporal --weightb --subme 3 --zones 764,884,q=42 --output nul --progress Film.avs
x264.exe --bframe 3 --b-pyramid --ref 5 --filter [0:0] --bitrate 700 --pass 2 --stats "x264.log" --analyse all --direct temporal --weightb --subme 5 --zones 764,884,q=42 --output Film.mp4 --progress Film.avs

[akupenguin]

x264 will achieve the global bitrate you ask for (unless you apply unreasonable restrictions like putting the whole movie in a fixed quant zone). If you ask for lower bitrate in the credits, it obviously has to spend those bits on the rest of the movie to maintain the average.

[JoeBG]

Originally posted by akupenguin
x264 will acchieve the global bitrate you ask for (unless you apply unreasonable restrictions like putting the whole movie in a fixed quant zone). If you ask for lower bitrate in the credits, it obviously has to spend those bits on the rest of the movie to maintain the average.

Ok, that´s clear now. Thank you very much for the zones and for your explanation :)

Now I have to find out, how to create a working commandline. Sometimes I´m a little stupid with new things :(

Edit:
Works perfect with the above menitioned commandline. It was a mistake in my avisynth-skript (still trim commands from other tests in it).

[bond]

akupenguin, would it be possible to add an "insert keyframe" option to the zones easily? that way you can ensure that each chapterpoint has a keyframe

[Doom9]

that way you can ensure that each chapterpoint has a keyframeAnd why would that be needed? In fact forcing keyframes could easily deteriorate the result (if only by a small margin) because you're manually interfering with the SCD. And at least in the DirectShow case, a properly written filter must be able to jump to any frame and starting playing from that point.. so it might be a bit slower than if it were a keyframe, but imho that's far outweighed by the adverse affect that messing with the SCD can have.

[bond]

yeah, of course its not a good idea to enforce keyframes anywhere, but enforcing keyframes at scenechanges (where chapterpoints belong to) shouldnt do any harm
xvid offers that option too for a long time already, without any big problems coming up

it should be possible to write a filter being able to search to any frame, but still this doesnt mean that its ensured that most parsers will do that
plainly starting decoding from a non-keyframe isnt also really good

after all i dont see a problem in being able to enforce a keyframe at a scenechange, it just makes a rip better if chapterpoints at scenechanges have keyframes imho

[akupenguin]

I could include such an option, but I need more info about what people want:

x264 already puts a I-frame on anything it thinks is a scenechange (xvid does too). I could add a flag meaning "force the nearest I-frame to be IDR".

There's also the case where the chapter point doesn't actually coincide with a scenecut (e.g. if there's a fade-to-black between chapters). In that case, you might actually want to force an extra I-frame.

Enforcing keyframes at scenechanges (where chapterpoints belong to) shouldnt do any harm That assumes you have frame-accurate chapter timecodes. I've seen plenty of DVDs where the chapter breaks were several seconds off from the actual scenchanges.

[bond]

Originally posted by akupenguin
That assumes you have frame-accurate chapter timecodes. I've seen plenty of DVDs where the chapter breaks were several seconds off from the actual scenchanges. indeed, and thats why i always check where the real scenechanges are and adopt the timecodes for the chapterpoints

would it really matter a lot to be able to differentiate between enforcing an idr or an i-frame?
does an "enforce i-frame" option really help with chapters? wouldnt a chapterpoint need to be an idr-frame in any case if you want to ensure that you can seek to it without any hassles. eg with an i-frame only you cant ensure that i assume

[Doom9]

xvid offers that option too for a long time already, without any big problems coming upWhere? I thought I knew all the options xvid offered and giving it a keyframe list isn't one of them the last time I checked. The only thing I can think of is zones and have them all begin with a keyframe.

it should be possible to write a filter being able to search to any frame, but still this doesnt mean that its ensured that most parsers will do that
plainly starting decoding from a non-keyframe isnt also really good
You misunderstood. When you ask a directshow filter to decode frame X (and subsequent ones), it must be able to advance to that point and start playing.. regardless of the frame type and how far away the nearest keyframe is. I suppose this is implemented as "find previous keyframe and start decoding until the frame in question". Either way, it is a non issue. You don't need special frames for decoding reasons.. any frame will work. And let's be honest, how many times do you actually use chapterpoints? A lot of people wouldn't even notice if their DVDs came without them.

[bond]

Originally posted by Doom9
[B]Where? I thought I knew all the options xvid offered and giving it a keyframe list isn't one of them the last time I checked. The only thing I can think of is zones and have them all begin with a keyframe. exactly, in xvid you simply create a new zone, that does exactly the same things as the zone before with the exception to start with a keyframe

You misunderstood. When you ask a directshow filter to decode frame X (and subsequent ones), it must be able to advance to that point and start playing.. regardless of the frame type and how far away the nearest keyframe is. I suppose this is implemented as "find previous keyframe and start decoding until the frame in question". Either way, it is a non issue. You don't need special frames for decoding reasons.. any frame will work.yeah, might all be, but still you cant ensure that every player (hardware or software) you will ever want to play your file on, will handle it like that, so i dont see anything wrong in ensuring to having an i-frame on a chapterpoint (if possible on a scenechange)

for me this is just a matter of wanting an as good as perfect rip. if i spend hours on tuning the codec settings, its not much to invest 20 minutes on finding the scencechanges near the chapterpoints and setting them in the codec

And let's be honest, how many times do you actually use chapterpoints? A lot of people wouldn't even notice if their DVDs came without them.i set them on every rip i do

[Doom9]

set them on every rip i doAnd how many times do you use them? Don't tell me you're watching a movie in bits over days.. most people watch it as a whole (with some breaks but that's where pause comes in). And you always have navigation by timecode as well..

[bond]

well true, but just because i am not always using the chapters it doesnt mean that setting chapters doesnt make any sense
and also this doesnt mean that its bad to have a feature which allows you to enforce a keyframe to be set at a specific scenechanges imho

after all i dont see why such a feature would be really bad and should be opposed?

[Stacey Melissa]

I second Bond's opinion on the matter of forcing IDR at arbitrary frames.

Something like --force-idr 5014,15487,27014,... would work nicely.

It's just something to add polish to encodes. Some of us are anal about these things. :p It's not something I'd call high priority, but it would sure be nice to have at some point.

[redfordxx]

x264 will acchieve the global bitrate you ask forIs there way (when not with setting, at least with calculating) to get in normal movie the bitrate I requested? Example:
clip 200frames, target bitrate 2000:
I want frames 1-100 100%bitrate,
101-150 80% bitrate,
151-200 qp=40

So the total bitrate should be lower.

[bond]

the answer is right in this thread (yeah i know reading is harder than posting) :rolleyes:

striked

[redfordxx]

the answer is right in this thread (yeah i know reading is harder than posting) I am sorry, but I really read the tread BEFORE I wrote that post (well I went faster through the posts discussing IDR frames because that's not the topic in question). But obviously, either we don't understand each other, or I am blind or too stupid to get it. The only solution was in the same (akupenguin's) postunless you apply unreasonable restrictions like putting the whole movie in a fixed quant zone but I don't want to make the movie on fixed quantizer.

[bond]

--zones 0,10231,b=0.25/156034,183123,q=42

In addition to the previous example, this would create another zone from frames 0 - 10231, and multiply the assigned bitrate by a factor of 0.25 (in other words the first 10232 frames would get 1/4 of the bitrate of frames 10232 - 156034).here is your answer

[Audionut]

I have always thought that scene change detection in compressors simply forced an I-frame at said point.

Compressors, unfortunally are not perfect.

Being able to manually enforce an I-frame is a good idea IMO.

[redfordxx]

But I suppose this still does not contradict the akupenguins statement I cited. So again example:
clip 200frames, target bitrate 2000:
zone0: 0-99 100%bitrate,
zone1: 100-149 50% bitrate,
zone2: 150-199 qp=40
Then with the command line--bitrate 2000 --zones 100,149,b=0.5/150,199,q=40 I don't get 2000kbps in main part.

My understanding is that provided zone2@qp=40 makes e.g. 100kbps
avg bitrate(zone0+zone1)=(2000kbps*200frames-100kbps*50frames)/150frames=2633kbps
Then
bitrate(zone0)=(1.*100)*(2633*150)/(1*100+0.5*50)/100=3160
bitrate(zone1)=(0.5*50)*(2633*150)/(1*100+0.5*50)/50=1580
But I want
bitrate(zone0)=2000

I could calculate the target bitrate so that it will match the weight 1 = 2000 but only when I know the bitrate of qp=40. And learning this could be complicated and/or time consuming.
Therefore I asked whether there is another way.

[Doom9]

you're giving the ratecontrol 100 frames to compensate for major bitrate differences in 200 other frames. Why am I not surprised that doesn't work perfectly? Try that again with overriding 200 frames and 150'000 regular frames and the results will be much more satisfactory ;)

[redfordxx]

you're giving the ratecontrol 100 frames to compensate for major bitrate differences in 200 other frames. Why am I not surprised that doesn't work perfectly? Try that again with overriding 200 frames and 150'000 regular frames and the results will be much more satisfactory ;)If understand your post correctly, you wanna say that usually the is difference is negligible...

Of course, I took this example to make it significant and also easy to calculate.
But imagine you make 150min movie on 3CD and there are credits 10min (IIRC LotR EE had credits even much much more). Then the movie on 1st and 2nd CD gets quite a lower bitrate than the last part...

Now, I wonder, how the encoder knows what ABR or VBR should be assigned in the main part to hit the target bitrate, when it does not know yet what will be the credits bitrate.

[stephanV]

redfordxx:

Your sample is too short. Rate control (certainly not 1-pass) isn't that accurate on 300 frames.

Now, I wonder, how the encoder knows what ABR or VBR should be assigned in the main part to hit the target bitrate, when it does not know yet what will be the credits bitrate.
For a 1 pass it can't know, only for a multipass.

[redfordxx]

Your sample is too short. Rate control (certainly not 1-pass) isn't that accurate on 300 frames. well, then multiply the framecount by 100 and the problem remains the same.

The best I can see in my 3CD example is to make fast first pass for CD3.
Then make mp4-->avi, load in VD, read the size of credits, calculate the bitrate for movie without credits and then encode...

I guess probably the bitrate or size can be read from stats file... but I don't know how. Is there somewhere described the syntax of x264 stats file or any reader like for xvid?

[stephanV]

You have tried this out with multipass on a, say, 5,000 frame sample?

Your 300 frame example was only a single pass if your commandline was the one you posted above. If you want to use zones with singlepass then something like you suggest above must be done yes. You could just encode the credits at qp40, substract that size from your targeted filesize and recalculate the bitrate.

[redfordxx]

You have tried this out with multipass on a, say, 5,000 frame sample?

Your 300 frame example was only a single pass if your commandline was the one you posted above. If you want to use zones with singlepass then something like you suggest above must be done yes. You could just encode the credits at qp40, substract that size from your targeted filesize and recalculate the bitrate.It was only calculated example, I didn't do any passes. The part of command line was to show you the intention.

[Doom9]

the goal for multipass encoding is hitting a certain target size, is it not? So why don't you encode your LOTR with end credits, and see if you hit the size or not. My experience is that with intro and end credits using zones, the target size is still hit. Yes, your main movie will have a higher bitrate if you have parts that will be encoded at a lower bitrate/quantizer, but since your end goal is a certain size, that really shouldn't matter, in fact it's the whole point of treating credits separately. The only reason when size is of little consequence is streaming (CBR), there you couldn't care less how large the file gets as long as the stream can fit through your pipe.

The whole point of credits is that instead of having an average bitrate throughout the movie, you lower that bitrate in beginning/end so that the part int he middle gets a higher bitrate.. what you're trying here is to undo that advantage by insisting on a certain bitrate in the middle part, thus effectively nilling the whole advantage rate control adjustments bring, and on top of that getting an unpredictable size. If that's really what you want, it's no wonder no codec will accomodate you and the easiest way is using -seek and -frames in x264 to encode intro and credits separately, then encode the main movie at your given bitrate, and finally merge everything together using mp4box's cat option.

[redfordxx]

The whole point of credits is that instead of having an average bitrate throughout the movie, you lower that bitrate in beginning/end so that the part int he middle gets a higher bitrate.. what you're trying here is to undo that advantage by insisting on a certain bitrate in the middle part, thus effectively nilling the whole advantage rate control adjustments bring, and on top of that getting an unpredictable size. If that's really what you want, it's no wonder no codec will accomodate you and the easiest way is using -seek and -frames in x264 to encode intro and credits separately, then encode the main movie at your given bitrate, and finally merge everything together using mp4box's cat option.I am aware of the goal of zones option and its advantage, but in my case, the avantage would be applied only for CD3. And that is what I don't want. I want the advantage for all CDs.
Spliting the credits is also a good workaround, I will try it. However my source is wmv via avisynth, and according to my experience till now, any seek in that avs caused growing the process memory growth over 1Gig (in VD or various players), until I killed it. Not tried directly with x264 yet, so I'll see.

[Doom9]

I am aware of the goal of zones option and its advantage, but in my case, the avantage would be applied only for CD3. Huh? The bits saved will be distributed over the entire main movie.. I really don't get what you're trying to achieve. You want to use the saved bits only on the last third of the movie, leaving the rest at the normal bitrate? What is the point of that? You can do it (manually of course) but I really don't see the point in it.

[redfordxx]

Huh? The bits saved will be distributed over the entire main movie.. I really don't get what you're trying to achieve. You want to use the saved bits only on the last third of the movie, leaving the rest at the normal bitrate? What is the point of that? You can do it (manually of course) but I really don't see the point in it.No, in opposite. I have more sources and I want to fit them together to certain size. But only the last one has credits. So using zones, the "advantage" would apply only for the last one. To prevent that I should enter lower --bitrate for the last source. But how much lower, that is the question... But I think already I will do it somehow --- I learned there is no direct way. Thanks for all your contributing ideas.

[redfordxx]

Is there somewhere described the syntax of x264 stats file or any stats reader like for xvid?Maybe this fell forgotten... does anybody know?

[Doom9]

I have more sources and I want to fit them together to certain size. And why oh why do you not encode each source separately? It makes little sense to turn multiple movies into one. And if you insist, you can still concatenate the encoded sources when muxing. I'm really getting the feeling that you're missing something and are trying to do things in a more complex way that needs be.

[redfordxx]

And why oh why do you not encode each source separately?There were multiple problems/reasons. In fact I have four pieces of one HDTV video which I want to fit on one DVD. It would be nice to have it in one piece. I wanted to join it, but
1)I expect the the encoding duration far more than one day. Maybe a week? Some unwanted shutdown or restart could cost lot of rework.
2)Framerate of the parts differed a little, so I was afraid of audio sync
3)Audio was a littttttle shorter for all parts, so I was afraid of audio sync
4)I have read somewhere about problems of x264 with files>4GB... but I understood it wrongly I hope

you can still concatenate the encoded sources when muxing.but it does not solve the bitrate issue

[foxyshadis]

So after all this, we finally get to the crux of your problem, which is that you're trying to get 2-pass rate control on something way outside x264's 2pass capabilities. Since rate control only works over one file, you'll have to do it yourself.

The obvious immediate solution is to use a little algebra to manually change the bitrate of each chunk slightly, by the amount of bits saved, down for the last and up for the others. Since you can't easily get that amount, you're going to have to take a guess (and perhaps experiment a little).

The audio problems can be fixed in avisynth with some work, depending on exactly how they're messed up, and I think someone's working on a patch to allow you to stop and later restart encoding. (At least, it was discussed.)

For future reference, explaining what you're doing in a high-level manner like that should be done with your first post, otherwise no one can effectively help you.

[redfordxx]

Now, one more question came in my mind:
Without any zones credits get always lower bitrate, because they are easy to encode. Let's say the expected credits bitrate is half of the movie bitrate.
Now, when I set zone for credits with weight 0.5, the credits get de facto 0.5(=weight) bitrate of the main movie or 0.25(=weight*expected bitrate)?

[stephanV]

Without any zones credits get always lower bitrate, because they are easy to encode.
Not necessarily if you are using 1-pass.

Let's say the expected credits bitrate is half of the movie bitrate.
Now, when I set zone for credits with weight 0.5, the credits get de facto 0.5(=weight) bitrate of the main movie or 0.25(=weight*expected bitrate)?
I assume the weight is applied on the bitrate assigned to that particular scene, not the average.

[redfordxx]

I assume the weight is applied on the bitrate assigned to that particular scene, not the average.I understand it as you are saying the former is correct,i.e.:the credits get de facto 0.5(=weight) bitrate of the main movie

[akupenguin]

weight*expected bitrate.
Consider the default to be putting the entire movie in zone of weight 1. A zone with weight < 1 will always get lower bitrate than if it had weight 1.

[redfordxx]

Now after one (only) test it seems to me, that the weight is not aplied at bitrate only as multiplication, but with some more advanced formula. Maybe in the PSNR/bitrate formula optimization?

So, the expressionweight*expected bitrateis exactly true or mentioned just as a simplification of the issue?