Introduce to VBV

VBV的原理

Video Buffer Verifier是编码器内部的一个模块，用来确保解码器可以smooth播放视频。通常会假设decoder内部有一块buffer，专门用于缓冲输入码流，这块buffer的size是固定的。我们先考虑下解码端的情况，再来看下反推到编码端是怎么样的

解码端是从网络/CD中读入码流，读入的速度可以是固定的，也可以是变化的，我们先考虑固定的情况。假如$R{dec}^{in}$是解码器读入码流的速度，解码器把码流存储在码流Buffer中，Buffer的size是$S{buf}$，另外一方面，当Buffer中的数据够一帧的数据时，解码器就会把整帧的数据一次性消耗掉。

理论上来说，当Buffer中没有空间来缓存数据时，这种情况就叫Buffer Overflow，这种情况是可能是因为$R{dec}^{out}$太小导致; 另外一种情况是，当Buffer中没有数据的时候，就叫Buffer Underflow。这种情况是$R{dec}^{out}$太快，大于$R_{dec}^{in}$，导致没有数据；

以上是解码端的情况，VBV本身就是模拟下解码端的情况，从而来确保编码端的码流符合规范；$R{enc}^{in}$代表送给编码端VBV的速率，$R{enc}^{out}$代表从VBV流出的码率；注意
$R{enc}^{out} = R{dec}^{in},\
R{enc}^{in} = R{dec}^{out}$

因此，当$R{enc}^{in} > R{enc}^{out}$时，对应的是$R{dec}^{out} > R{dec}^{int}$，也就是码流比较高的时候，会导致Buffer underflow;

当$R{enc}^{in} < R{enc}^{out}$是，对应解码器端 $R{dec}^{in} < R{dec}^{out}$，也就是实际编码码率较低的时候，会导致Buffer overflow.

以上就是vbv的基本原理，以下介绍下264/265里面的设置。

X264’s VBV setting

—vbv-maxrate:

Maximum local bitrate (kbits/sec). Will be used only if vbv-bufsize is also non-zero. Both vbv-bufsize and vbv-maxrate are required to enable VBV in CRF mode. Default 0 (disabled)

—vbv-bufsize:
Specify the size of the VBV buffer (kbits). Enables VBV in ABR mode. In CRF mode, --vbv-maxrate must also be specified. Default 0 (vbv disabled)

—vbv-init:

Initial buffer occupancy. The portion of the decode buffer which must be full before the decoder will begin decoding. Determines absolute maximum frame size. May be specified as a fractional value between 0 and 1, or in kbits. In other words, these two option pairs are equivalent:

vbv-bufsize / vbv-maxrate代表了缓冲几秒的视频；

—vbv-end:

Final buffer fullness. The portion of the decode buffer that must be full after all the specified frames have been inserted into the decode buffer.

—min-vbv-fullness

Minimum VBV fullness percentage to be maintained. Specified as a fractional value ranging between 0 and 100. Default 50 i.e, Tries to keep the buffer at least 50% full at any point in time.

Decreasing the minimum required fullness shall improve the compression efficiency, but is expected to affect VBV conformance. Experimental option.

—max-vbv-fullness

Maximum VBV fullness percentage to be maintained. Specified as a fractional value ranging between 0 and 100. Default 80 i.e Tries to keep the buffer at max 80% full at any point in time.

Increasing the minimum required fullness shall improve the compression efficiency,

m_bufferFillFinal: 代表了已经填充的buffer(因为在模拟解码端的处理？)，也就相等于编码器端的可用空间

// Initial
if (m_param->rc.vbvBufferInit > 1.)
    m_param->rc.vbvBufferInit = x265_clip3(0.0, 1.0, m_param->rc.vbvBufferInit / m_param->rc.vbvBufferSize);
if (m_param->vbvBufferEnd > 1.)
    m_param->vbvBufferEnd = x265_clip3(0.0, 1.0, m_param->vbvBufferEnd / m_param->rc.vbvBufferSize);
if (m_param->vbvEndFrameAdjust > 1.)
    m_param->vbvEndFrameAdjust = x265_clip3(0.0, 1.0, m_param->vbvEndFrameAdjust);

m_param->rc.vbvBufferInit = x265_clip3(0.0, 1.0, X265_MAX(m_param->rc.vbvBufferInit, m_bufferRate / m_bufferSize));
m_bufferFillFinal = m_bufferSize * m_param->rc.vbvBufferInit;

// Update
m_bufferFillFinal -= frame_bits;
m_bufferFillFinal += rce->bufferRate;

AV1

rc→starting_buffer_level: 代表了已经填充的Buffer(从解码器的角度来看)，相当于编码端的可用空间；

rc→buffer_level: 代表了已经填充Buffer（从解码端的角度来看),从编码器的角度来看，代表了可用的buffer空间

正常对它的处理如下:

// Initialize
p_rc->starting_buffer_level = starting * bandwidth / 1000;
if (p_rc->starting_buffer_level >= p_rc->maximum_buffer_size) {
  p_rc->starting_buffer_level = p_rc->maximum_buffer_size * init_level / 1000;
}

rc->buffer_level = p_rc->starting_buffer_level;

// Update, 
int64_t new_buffer_lvl = rc->buffer_level - rc->projected_frame_size;
if (cpi->common.show_frame) new_buffer_lvl += rc->avg_frame_bandwidth;

Initialize:

对VBV的一些疑问

1. VBV假设$R_{dec}^{in}$是固定的，这个假设是对的吗？一般对于VCD/DVD/Blu.ray来说，可能需要读取fixed bitrate，而此时这个假设是可以；如果没有这个限制，譬如说live streaming，此时读取的内容就是发送的数据，发送的数据即使少于码率，也不会有Buffer Overflow发生。

2. In video streaming， buffer overflow是不大会出现，可以忽略。这是因为不可能出现解码器的输入码率不可能大于解码器消耗的码率。因为输入码率是等于$Min(bandwidth, R_{enc}^{out})$，

   而$R{enc}^{out} =R{dec}^{out}$，因此必然会有$R{dec}^{in} ≤ R{dec}^{out}$。

Reference

https://www.youtube.com/watch?v=-Q7BuSXdO_8&ab_channel=Demuxed