If you just set --bitrate then the encoder will operate in unconstrained VBR.
If you set --vbv-maxrate and --vbv-bufsize to something basic like the H.264 High Profile @ Level 4.1 limitations, the encoder will still operate in ABR mode, but will constrain itself to not go outside these specifications.
If you set --vbv-maxrate to the same value as --bitrate, then the encoder will operate in CBR mode. Note, this is not hard CBR where every picture is the same size. That's a terrible idea, and is basically never necessary. --vbv-bufsize controls the size of the buffer - which allows for bitrate variance while still staying inside the CBR limitations. Also note, this is how every other encoder does CBR, so don't be frightened
