Block decision as setting

[movmasty]

Any encoder have the choice if to use a precedent block or make one new.

Why an user cant choice the sensibility of this?
If to have more new encoded blocks, or use more of the precedent and save bytes.

Fix the threshold himself, i think that will be a very powerful way to customize even more.

[LoRd_MuldeR]

Not quite sure what you mean. You are talking about the selection of macroblock types (I, P and B) within a frame?

[movmasty]

I mean, if code a new block, or use one from a referenced frame, in fact I B frames have only few macroblock really encoded in itself,
the most blocks displayed are from other frames by motion vectors,
now, the code decide if any block from other frames is similar enough to be displayed instead of encode a new block
the user should be able to set what is "similar enough"

[LoRd_MuldeR]

Well, in H.264, the "allowed" macroblock types depend on the type of frame (slice) that we are in:

I-Frames contain only I-Blocks (predicted from neighboring blocks within the same frame), P-Frames can contain I- and P-Blocks (predicted from a single reference frame), and B-Frames can contain I-, P- and B-Blocks (predicted from two references frames). So, selecting the "best" frame type for each frame, as well as selecting the "best" macroblock type for each block inside a frame, is a tough optimization problem that needs to be solved by the encoder!

For each possible decision one needs to evaluate what is the "bit cost" to code that decision in the bitstream and how large is the resulting "prediction error" (residual). Also, decisions made for one frame can have impact on subsequent frames.

I don't think this can be solved by a simple threshold function ;)

See also:
https://books.google.de/books?id=k7nOAiIUo9IC&lpg=SA3-PA104&dq=h264%20%22macroblock%20types%22&hl=de&pg=SA3-PA106#v=onepage&q&f=false

[movmasty]

OK thanks, but who decides the error-bit cost evaluation?
The same movie, with the same codec could have more or less blocks encoded with all the YUV info,
or simply repeated from other frames,
the user should have a part in this decision

This could be interesting also for artistic effects:
a movie encoded re using as many blocks as possible from other frames, also if the error is large,
curious to see what happens, size will go down, dont know how will look :D

The Fans of transparency, dont have to looks only at the CRF, they should look as well at the precision, to have also slightly different block encoded as new

Fans of compression instead could decide to have as many repeated blocks as possible to bring size down.

The people needs this setting! :helpful:

[LoRd_MuldeR]

OK thanks, but who decides the error-bit cost evaluation?

The encoder does!

Keep in mind that video compression standards only define how the decoder works, they absolutely do not define how the encoder has to generate the encoded bitstream from a given source. So, there is always an infinite number of possibilities to encode the very same source. Actually, the encoder is totally free to generate whatever btistream it likes – as long as the resulting bitstream is standard compliant, the decoder is going to decode it. In practice, of course, we do not want an arbitrary (valid) bitstream. We want a bitstream that, after decoding, resembles the original source as closely as possible – while still satisfying our bitrate constraints. How this is achieved, that is a problem left totally up to the encoder developers! That's why "smart" algorithms, such RDO, are used in practice. The goal is always to find the "best" decision (or at least a very good one) from the myriad of possibilities. To make things even more arcane, those decisions often heavily depend on each other.

the user should have a part in this decision

Hopefully not!

Millions of such decisions need to be made to encode a video clip. And finding the "best" decision (or at least a very good one) is a though optimization problem that no user could ever do "by hand".

[foxyshadis]

You can easily do this with the x264 API, it's just not available from the command-line. You can manually micromanage everything about the encoding via the API, from block type to qp offset to which refs to use, as long as it's valid (like you can't create a B-block in an I-frame). The command-line is meant for just getting things done, instead of really deeply hooking into its decisions.

Once you start playing with it, I think you'll find it's not as easy to tune it to large quality increase (without large size increase) just by tweaking with a few things.

[movmasty]

You can easily do this with the x264 API, it's just not available from the command-line. You can manually micromanage everything about the encoding via the API...YOU a programmer, im not, i dont know APIs, just apes.

The encoder does!
Hopefully not!
Millions of such decisions need to be made to encode a video clip. And finding the "best" decision (or at least a very good one) is a though optimization problem that no user could ever do "by hand".The codec will ever do, the user just have to tell to the codec how large to be in the judgement

We want a bitstream that, after decoding, resembles the original source as closely as possible.How close at what cost?
The user shoul have an influence on this.

[LoRd_MuldeR]

How close at what cost? The user shoul have an influence on this.

As close as possible, under the given bit budget.

In other words, the block type decision (just like all the other decisions) is all about using the available bits as efficiently as possible. And the encoder uses sophisticated algorithms to determine the "best" block type for each macroblock, i.e. the block type that results in the most efficient coding of the particular block. But, if the user was manually messing with the encoder's block type decision, this would almost certainly not improve the coding efficiency! Unless, of course, you believe that you can make even better block type decisions then x264's algorithms. I somehow doubt that. And even if you could, you'd better spend your time improving the algorithms in x264, instead of performing millions of block type decisions "by hand".

So, if you want to get even closer to the original source, all you can do is spending even more bits - which means a higher target bitrate (2-Pass/ABR mode) or a lower CRF value (CRF mode).

[movmasty]

As close as possible, under the given bit budget.
But, if the user was manually messing with the encoder's block type decision, this would almost certainly not improve the coding efficiency! Unless, of course, you believe that you can make even better block type decisions then x264's algorithms....performing millions of block type decisions "by hand".
Is my english so bad that i cannot explain?
User just should decide how close will be a good choice,
then the codec will decide, based on users param, if to use an already coded block or encode one new with all the video infos.

So, if you want to get even closer to the original source, all you can do is spending even more bits - which means a higher target bitrate (2-Pass/ABR mode) or a lower CRF value (CRF mode).
You know that above a certain bitrate visual quality doesnt improve,
Here is another setting to improve further,
or bring size down further,
as one likes.

[LoRd_MuldeR]

User just should decide how close will be a good choice

You always want each encoded block to be as close as possible to the original source - under the given bit budget.

So that's not the question. The question is: How to reach that goal?

And that is not so easy to do! If you spend too many bits on one block, there will won't be enough bits left for the other blocks. Also, blocks in one frame may be referenced in subsequent frames. So there are dependencies that have to be taken into account. For example: It makes sense to spend additional bits on a block that will be referenced a lot into future frames (it improves those future frames too), while spending too many bits on a block that won't be referenced is bad.

So it's all about finding the best trade-off for all blocks, in all frames. That's a tough optimization problem. You will not be able to solve this by manually messing with, e.g., the macroblock types. You are probably going to make things worse ;)

then the codec will decide, based on users param, if to use an already coded block or encode one new with all the video infos.

It's seems like you have the idea that a macroblock is either "re-used" 1:1 from another frame, or it will be stored completely in the current frame. But that's not how it things work :eek:

The block is always predicted. The only difference between I-, P- and B-Blocks is how they are predicted. I-Blocks are predicted only from neighboring blocks within the same frame, P-Blocks use one reference from another frame, and B-Blocks use two references from other frames. But you always get a predicted block. And, since that prediction is never "perfect", you always have to store the difference between the predicted block and the original block - that is called "residual".

(BTW: The reason why there actually is some quality loss in that compression sheme is because the "residual" will be subject to quantization)

[foxyshadis]

YOU a programmer, im not, i dont know APIs, just apes.

The codec will ever do, the user just have to tell to the codec how large to be in the judgement

How close at what cost?
The user shoul have an influence on this.

*shrug* It's there if you want to use it. If you care enough, you can learn to use it. (You could also petition someone else to do it for you, but that will be a lot harder.) Any command-line implementation wouldn't be particularly easy to use either.

You can say the user should have influence, but you might well be the only person to have ever wanted this functionality from the command-line, which isn't nearly enough to sway x264 creators, who have to support a mass-market product.

[movmasty]

you might well be the only person to have ever wanted this functionality from the command-line, which isn't nearly enough to sway x264 creators, who have to support a mass-market product.
The 1st.

[LoRd_MuldeR]

*shrug* It's there if you want to use it. If you care enough, you can learn to use it. (You could also petition someone else to do it for you, but that will be a lot harder.) Any command-line implementation wouldn't be particularly easy to use either.

You can say the user should have influence, but you might well be the only person to have ever wanted this functionality from the command-line, which isn't nearly enough to sway x264 creators, who have to support a mass-market product.

How would you even want to set this from the command-line? The block-type needs to be selected for every macroblock in every frame. That makes 8.160 decisions for each frame in a "full hd" video. Multiply that by 30 frames per second and you get 244.800 decisions that need to be made for 1 second of video! If at all, you could specify a file where the MB types are read from. That file would also need to specify the frame types, because MB types depend on the type of frame (slice).

But the biggest question is: How would you create such a file, and for what purpose? I think the x264 API's that allow the caller to adjust things on the block-level exist mainly to enable ROI (region of interest encoding). But to implement ROI, you would probably adjust only the qp offsets, not the MB types. I think it's pretty much impossible to know beforehand what the "effect" of enforcing a particular MB type for a particular block will be...

[movmasty]

I understand that the question in more complex,
but, if there is a difference between a normal encode and an all I-frames one, that could be modulated.

[foxyshadis]

I understand that the question in more complex,
but, if there is a difference between a normal encode and an all I-frames one, that could be modulated.

Like I said, everything is right at your fingertips if you want to make a tiny bit of effort. It's all there, ripe for the taking, you just have to move beyond the CLI.

[LoRd_MuldeR]

but, if there is a difference between a normal encode and an all I-frames one, that could be modulated.

That's a bit different from what we discussed before, but you can do an "all I-frames" encode simply with "--keyint 1". Of course, in such an "all I-frames" encode, there can only be Intra-coded blocks.

BTW: x264 also supports fully compliant AVC-Intra (https://en.wikipedia.org/wiki/AVC-Intra) mode. Set the "--avcintra-class" option to enable.

[movmasty]

That's a bit different from what we discussed before.
Yes, i hypersemplified before