--bitrate sets the average bitrate of the complete encode
--vbv-bufsize sets the size of the decoding buffer
--vbv-maxrate sets the maximum bitrate at which said buffer can be filled
If --bitrate is equal to --vbv-maxrate, CBR encoding will be used. (if --bitrate is greater than --vbv-maxrate, x264 will automatically reduce --bitrate to equal --vbv-maxrate)
You set --bitrate according to the final filesize/average bitrate you want to achieve. (Or use the constant quality mode, i.e. --crf)
You set --vbv-maxrate according to the hardware capabilities of your device (e.g. the CPU/decoder of the player or the speed of your internet connection)
You set --vbv-bufsize according to the hardware capabilities of your device (the memory of the player) and the maximum time you want to spend on buffering. (--vbv-bufsize/--vbv-maxrate = maximum time in seconds it takes to buffer the video before starting to play
*)
H.264 has predefined levels to help choosing the correct --vbv-maxrate and --vbv-bufsize values: