about real time encoding with 2pass

[seanxu_2010]

Hi, recently, I did some comparison between 1 pass ABR and 2 pass, obviously, 2 pass almostly uses less bit rate to produce more psnr, my issue is if I can use 2 pass method, when I want to implement real time encoding, certainly, it is not really real time, 10 or more second latency can be allowed, and my assumption is one pass generates stat in real time, meanwhile another pass encodes later than 10 seconds based on the stat, can X264 implement it? or somebody has considered it or implemented it based on X264? thanks

[Dark Shikari]

Impossible in the general case since the purpose of 2-pass is to distribute bits over the whole video, while your plan would only distribute bits over, well, 10 seconds.

However, this is already implemented as RC-lookahead in x264, and is used for VBV, where one does in fact care about the next few seconds as opposed to the whole video.

[seanxu_2010]

Originally Posted by Dark Shikari

Impossible in the general case since the purpose of 2-pass is to distribute bits over the whole video, while your plan would only distribute bits over, well, 10 seconds.

However, this is already implemented as RC-lookahead in x264, and is used for VBV, where one does in fact care about the next few seconds as opposed to the whole video.

thanks a lot, I will try!
In fact, I attempt to encode some sport program(such as NBA) from 1080i to D1, as you know, the scene in NBA is always high complexed, my plan is encoding them in a very low bitrate, 800bps maybe, however, the result is awful, too obvious block-effect exist in edge for the complexed scene, I know it's because of the high QP for low bitrate, so I try some special methods to improve the quality, not for the psnr, but for the vision from human, and I denoise the raw data based on freq and time domain to smooth the detail, which maybe decrease the complexity for encoding and more or less can distribute some bits for reducing the block effect, could my attempt be available? or do you have any suggestion for the pre-posting or the encoding process for increasing the quality at low bitrate?

[Blue_MiSfit]

If you're having low quality due to high QPs, then you can do a few things:

1) Slow down x264 to increase its quality
2) Increase bitrate
3) Decrease resolution
4) Add denoising, either directly in x264, or ideally via more fancy stuff like AviSynth

How exactly are you processing your 1080i sources? Real-time is not my specialty, and probably rules out AviSynth, but maybe something can be done. Why the low bitrate restriction? What x264 command line exactly are you using? Your problem may be due to poorly chosen settings.

Are you going to 480p60 or 480p30? If you're coding interlaced - you probably shouldn't if you're trying to squeeze in as much quality as possible. The scaling / deinterlacing process can also heavily impact compressibility. So - how are you doing this?

800kbps is probably enough bitrate for 480p30 sports to look *okay*, not fabulous or anything, always hard to say - and always source specific!!

Moar info pls

~MiSfit

[seanxu_2010]

Originally Posted by Blue_MiSfit

How exactly are you processing your 1080i sources? Real-time is not my specialty, and probably rules out AviSynth, but maybe something can be done. Why the low bitrate restriction? What x264 command line exactly are you using? Your problem may be due to poorly chosen settings.

Are you going to 480p60 or 480p30? If you're coding interlaced - you probably shouldn't if you're trying to squeeze in as much quality as possible. The scaling / deinterlacing process can also heavily impact compressibility. So - how are you doing this?
Moar info pls

~MiSfit

uh...ok. I choose a clip from NBA, mainly including running and shooting, and the source is 1080i 60field, I used some tool to de-interlace, scale, and denoise it to 480p 30 frame(I also compare 480p 30 and 480p 60, it's trade off between quality and fluency, I selected the fore one because it seems more acceptable), then,I use the YUV420 raw data as input which 's quality is very good, and X264 command is below:
1 pass --bitrate 800 --ratetol 10 --qcomp 0.9 --profile baseline --me dia --ref 1;
2 pass --bitrate 800 --ratetol 10 --qcomp 0.9 --profile baseline --analysis all --me umh --ref 3 --deblock 3:3.

800bps is restricted by the network, but it allows 10 second latency, so I choose 10 as ratetol, and baseline is confined by the decoder, cabac and B can' t be used. and deblock alpha and beta is set as 3 3, which can help to decrease the block effect more or less I think.
however, all my attempts do not give me a obvious improvement, now my plan is below:
1 reduce some detail at pre-process, by using some strong denose tool or blur tool, because of scaling down too much.
2 because of NBA scene, what we concentrate on are always the athlete and ball, while the auditorium is ignored usually, so I can adjust the QP higher at center region, on the contrary, lower at around of the video, so maybe I add some code for ratecontrol based on the region adaptive.

that is my assumption, any suggestion?

[Dark Shikari]

Originally Posted by seanxu_2010

uh...ok. I choose a clip from NBA, mainly including running and shooting, and the source is 1080i 60field, I used some tool to de-interlace, scale, and denoise it to 480p 30 frame(I also compare 480p 30 and 480p 60, it's trade off between quality and fluency, I selected the fore one because it seems more acceptable), then,I use the YUV420 raw data as input which 's quality is very good, and X264 command is below:
1 pass --bitrate 800 --ratetol 10 --qcomp 0.9 --profile baseline --me dia --ref 1;
2 pass --bitrate 800 --ratetol 10 --qcomp 0.9 --profile baseline --analysis all --me umh --ref 3 --deblock 3:3.

800bps is restricted by the network, but it allows 10 second latency, so I choose 10 as ratetol, and baseline is confined by the decoder, cabac and B can' t be used. and deblock alpha and beta is set as 3 3, which can help to decrease the block effect more or less I think.
however, all my attempts do not give me a obvious improvement, now my plan is below:
1 reduce some detail at pre-process, by using some strong denose tool or blur tool, because of scaling down too much.
2 because of NBA scene, what we concentrate on are always the athlete and ball, while the auditorium is ignored usually, so I can adjust the QP higher at center region, on the contrary, lower at around of the video, so maybe I add some code for ratecontrol based on the region adaptive.

that is my assumption, any suggestion?

That isn't how ratetol works.

Here's what you actually mean:

--bitrate 800 --vbv-maxrate 800 --vbv-bufsize 8000

[seanxu_2010]

Originally Posted by Dark Shikari

That isn't how ratetol works.

Here's what you actually mean:

--bitrate 800 --vbv-maxrate 800 --vbv-bufsize 8000

thanks, there may be some misunderstanding, I know the ratetol is relative with overflow, what I mean is set a bigger value for ratetol instead of the default value 1, so permit a higher overflow for abr.

And I just view some threads about CBR, you suggest me to use VBV buffer model, and according to my understanding, X264 set a 10 second video buffer for imitating the stream buffer at decoder, in purpose that X264 can distribute less bits for the low complexed scene, and more bits for the high ones, but the distribution is limited by the size of buffer to ensure the decoder stream buffer is not overflow and underflow, in a word, that means it produces a stream with constant bitrate in 10 seconds perspective and if we see the curve of bitrate, it must be fluctuant around --vbv-maxrate. is it correct? if so, what is --bitrate used for? and when we set VBV, has the --rc-lookahead been set as a default value? because, if not, how the x264 can guarantee the constant bitrate in a 10 seconds perspective?

[Dark Shikari]

Originally Posted by seanxu_2010

thanks, there may be some misunderstanding, I know the ratetol is relative with overflow, what I mean is set a bigger value for ratetol instead of the default value 1, so permit a higher overflow for abr.

And I just view some threads about CBR, you suggest me to use VBV buffer model, and according to my understanding, X264 set a 10 second video buffer for imitating the stream buffer at decoder, in purpose that X264 can distribute less bits for the low complexed scene, and more bits for the high ones, but the distribution is limited by the size of buffer to ensure the decoder stream buffer is not overflow and underflow, in a word, that means it produces a stream with constant bitrate in 10 seconds perspective and if we see the curve of bitrate, it must be fluctuant around --vbv-maxrate. is it correct? and if so, what is --bitrate used for?

Maxrate is the maximum rate of flow of the input to the leaky buffer model of the VBV. Bitrate is the overall bitrate of the video. They really have nothing to do with each other, except that if bitrate == maxrate, you have CBR.

[seanxu_2010]

Originally Posted by Dark Shikari

Maxrate is the maximum rate of flow of the input to the leaky buffer model of the VBV. Bitrate is the overall bitrate of the video. They really have nothing to do with each other, except that if bitrate == maxrate, you have CBR.

thanks for your explain, if bitrate < maxrate, what will happened?
and another question is when we set VBV, has the --rc-lookahead been set as a default value? because, if not, how the x264 can guarantee the constant bitrate in a 10 seconds perspective?

[Dark Shikari]

Originally Posted by seanxu_2010

thanks for your explain, if bitrate < maxrate, what will happened?

Then you'll have capped VBR mode, where the local bitrate can't exceed maxrate.

Originally Posted by seanxu_2010

and another question is when we set VBV, has the --rc-lookahead been set as a default value? because, if not, how the x264 can guarantee the constant bitrate in a 10 seconds perspective?

RC-lookahead is on by default, but x264's VBV works without it as well, via a combined reactive/predictive algorithm.

[yaous]

Originally Posted by Dark Shikari

Then you'll have capped VBR mode, where the local bitrate can't exceed maxrate.

if I set bitrate=2000, maxrate=2001, buffer size=4000

is that means video bitrate can't exceed 2001??

[Dark Shikari]

Originally Posted by yaous

if I set bitrate=2000, maxrate=2001, buffer size=4000

is that means video bitrate can't exceed 2001??

Within the constraints of the buffer size, yes. Go read up on how the VBV works.

[seanxu_2010]

Originally Posted by Dark Shikari

Then you'll have capped VBR mode, where the local bitrate can't exceed maxrate.RC-lookahead is on by default, but x264's VBV works without it as well, via a combined reactive/predictive algorithm.

Dark Shikari, here are my test results:

x264 --profile baseline --bitrate 800 --vbv-maxrate 800 --vbv-bufsize 8000 --me umh --analyse all --qcomp 0.90 --subme 6 --trellis 1 --ref 3 --rc-lookahead 200 -o vbv_basketball_8000.mkv 640x360@30.yuv --fps 30 --psnr
x264 [info]: 640x360 (given by file name) @ 30.00 fps
x264 [info]: using cpu capabilities: none!
x264 [info]: profile Baseline, level 3.0
x264 [info]: frame I:1 Avg QP:31.25 size: 21862 PSNR Mean Y:33.30 U:38.34 V:39.47 Avg:34.50 Global:34.50
x264 [info]: frame P:150 Avg QP:31.05 size: 3784 PSNR Mean Y:33.01 U:37.18 V:39.14 Avg:34.13 Global:34.05
x264 [info]: mb I I16..4: 9.3% 0.0% 90.7%
x264 [info]: mb P I16..4: 0.3% 0.0% 1.0% P16..4: 43.6% 14.9% 7.3% 0.9% 0.6% skip:31.4%
x264 [info]: coded y,uvDC,uvAC intra: 72.0% 74.3% 51.7% inter: 14.1% 13.6% 2.9%
x264 [info]: i16 v,h,dc,p: 45% 31% 13% 11%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 22% 26% 17% 6% 5% 6% 6% 6% 6%
x264 [info]: ref P L0: 75.5% 15.6% 8.9%
x264 [info]: PSNR Mean Y:33.012 U:37.188 V:39.139 Avg:34.137 Global:34.051 kb/s:936.92

I find all of the bitrates exceed 800 bps which vbv-maxrate specifies, does it seem accurate? I mean, if the bitrate in continuous 10 seconds are beyond 800bps, the buffer will be overflow!

[Dark Shikari]

Originally Posted by seanxu_2010

Dark Shikari, here are my test results:

x264 --profile baseline --bitrate 800 --vbv-maxrate 800 --vbv-bufsize 8000 --me umh --analyse all --qcomp 0.90 --subme 6 --trellis 1 --ref 3 --rc-lookahead 200 -o vbv_basketball_8000.mkv 640x360@30.yuv --fps 30 --psnr
x264 [info]: 640x360 (given by file name) @ 30.00 fps
x264 [info]: using cpu capabilities: none!
x264 [info]: profile Baseline, level 3.0
x264 [info]: frame I:1 Avg QP:31.25 size: 21862 PSNR Mean Y:33.30 U:38.34 V:39.47 Avg:34.50 Global:34.50
x264 [info]: frame P:150 Avg QP:31.05 size: 3784 PSNR Mean Y:33.01 U:37.18 V:39.14 Avg:34.13 Global:34.05
x264 [info]: mb I I16..4: 9.3% 0.0% 90.7%
x264 [info]: mb P I16..4: 0.3% 0.0% 1.0% P16..4: 43.6% 14.9% 7.3% 0.9% 0.6% skip:31.4%
x264 [info]: coded y,uvDC,uvAC intra: 72.0% 74.3% 51.7% inter: 14.1% 13.6% 2.9%
x264 [info]: i16 v,h,dc,p: 45% 31% 13% 11%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 22% 26% 17% 6% 5% 6% 6% 6% 6%
x264 [info]: ref P L0: 75.5% 15.6% 8.9%
x264 [info]: PSNR Mean Y:33.012 U:37.188 V:39.139 Avg:34.137 Global:34.051 kb/s:936.92

I find all of the bitrates exceed 800 bps which vbv-maxrate specifies, does it seem accurate? I mean, if the bitrate in continuous 10 seconds are beyond 800bps, the buffer will be overflow!

The VBV-init parameter (by default 0.9) determines how full the buffer is to begin with. If the buffer is less full at the end of the video than at the start, the overall bitrate can exceed the maxrate without violating VBV.

[seanxu_2010]

Originally Posted by Dark Shikari

The VBV-init parameter (by default 0.9) determines how full the buffer is to begin with. If the buffer is less full at the end of the video than at the start, the overall bitrate can exceed the maxrate without violating VBV.

I try to follow your mind, we set the VBV as 90% of the size in buffer at initialization, the decoder consume the stream in buffer at the speed of a higher value than 800bps according to the bitrate generated by encoder, while the network can only support the bitrate less than 800bps, then, the balance would be broken, the number of data in buffer must be shifted down, however, it can't induce underflow. Now, we suppose the stream flow is no end, before the buffer underlow, the encoder will produce stream at bitrate under 800bps for filling the buffer, is it correct?

[yaous]

rather OT question...

what is the capped VBR?

is it a kind of CBR?

[J_Darnley]

Originally Posted by seanxu_2010

x264 [info]: using cpu capabilities: none!

What CPU are you running this on?

[seanxu_2010]

Originally Posted by J_Darnley

What CPU are you running this on?

I disable acceleration for debug

I insight into the sequence of sport scene, I find out some frame will consume higher bits than normal because of the photoflash lamp, however I think such special scenes can be eliminated for saving bits, is there any method to drop some frame according to some special conditions for example, set upper limit for total bits per frame or drop some frame according to frame type?

another question is, can I affect or change QP to different slice? for example, I divide a whole frame into 3 slices, the up and down are set as higher QPs, because nobody will take care of them, hence, it can save some bits for the middle slice, is there any method to implement it?

[Dark Shikari]

Originally Posted by seanxu_2010

I disable acceleration for debug

I insight into the sequence of sport scene, I find out some frame will consume higher bits than normal because of the photoflash lamp, however I think such special scenes can be eliminated for saving bits, is there any method to drop some frame according to some special conditions for example, set upper limit for total bits per frame or drop some frame according to frame type?

x264 already does this in about three different ways.

If you're trying to limit bits per frame for network reasons, you'll need to explain the entire use-case before I can give more advice.

Originally Posted by seanxu_2010

another question is, can I affect or change QP to different slice? for example, I divide a whole frame into 3 slices, the up and down are set as higher QPs, because nobody will take care of them, hence, it can save some bits for the middle slice, is there any method to implement it?

"nobody will take care of them"?

[seanxu_2010]

Originally Posted by Dark Shikari

x264 already does this in about three different ways.
If you're trying to limit bits per frame for network reasons, you'll need to explain the entire use-case before I can give more advice.

"nobody will take care of them"?

x264 --profile main --bitrate 800 --vbv-init 0.9 --vbv-maxrate 800 --vbv-bufsize 8000 --me umh --analyse all --qcomp 0.90 --subme 6 --trellis 2 --ref 3 --rc-lookahead 250 -o Estimate_720x576_30fps_NR_max_NRT_max_1.264 Estimate_720x576_30fps_NR_max_NRT_max.yuv 720x576 --fps 30 --psnr --ssim --deblock 0:0 --slices 3

I use VBV to obtain a proper bitrate to accommodate the network. I used ESEyE to observe the size of every frame, I found some B frame used very high bit because of fash lamp, what I expect is when we find it's B frame, and consume an extreme high bits, I will drop it for saving bits for other frame.

because in the match, we will concentrate on the athletes in the middle slice, and up and down are always audience, we can use higher QP to blue them for saving bits