x264 lossless

lexor · 26th January 2006, 22:29

ok using sharktooth's lossless profile I actually get lower speed than HQ-Slow (about half the speed, and that taking in account that HQ-Slow also had a few filters running), is that normal? I mean I always thought of lossless codecs as having to do less work (since they don't have to do intense motion search/compensation and handle too many B frames etc.) so I always thought of lossless as being faster. In other words was my impression wrong and we not supposed to be able to do realtime capture with x264 lossless? (at least not with current cpu's)

akupenguin · 26th January 2006, 22:47

At the same settings, x264 lossless is slower than lossy. It does all the same computations (except DCT, but that's fast anyway). But the time spent in CABAC is directly proportional to bitrate.

Yes, x264 lossless should be faster than HQ-slowest because there's no point in using most of the advanced features for lossless. B-frames are valid, but don't reduce bitrate by more than 1-2%, multiref is similarly disappointing (may even increase bitrate), rate-distortion optimization doesn't do much when you have no distortion, ...
Sharktooth's CQ-Lossless profile includes many of said near-useless options.

My lossless settings are just

Code:

x264 -q0 -m3

If that's not enough for realtime, then try in order:

Code:

x264 -q0 -m1
x264 -q0 -m1 -A i4x4
x264 -q0 -m1 -A i4x4 --no-cabac

Note that the reason most lossless codecs are fast is that they don't do any motion compensation. The exceptions being x264, Snow, and MSU.

Sharktooth · 27th January 2006, 01:58

i thought those "useless" options were automatically disabled by x264.
i'll make a new lossless profile as soon as i can start updating my builds again.

lexor · 27th January 2006, 03:02

on 5000 frames:

x264 -q0 -m1 -A i4x4 => 20.70fps

x264 -q0 -m3 -A i4x4 --no-cabac => 28.81fps

x264 -q0 -m1 -A i4x4 --no-cabac => 28.89fps

source is 23.98fps (and is itself a x264 encode) as reported by x264, man I had no idea cabac used up so much cpu. always thought ME was biggest cpu hog.

akupenguin · 27th January 2006, 05:32

ME is the biggest cpu hog normally. As I said, CABAC's cost is proportional to bitrate. If lossless is 50x the bitrate of a normal encode, then CABAC is 50x as expensive.

moseymosey · 27th January 2006, 19:39

Quote:

Originally Posted by akupenguin

Code:

x264 -q0 -m3

If that's not enough for realtime, then try in order:

Code:

x264 -q0 -m1
x264 -q0 -m1 -A i4x4
x264 -q0 -m1 -A i4x4 --no-cabac

Note that the reason most lossless codecs are fast is that they don't do any motion compensation. The exceptions being x264, Snow, and MSU.

If one chooses lossless encodings using these settings, is it still necessary to do a multi-pass encode? It would seem like there is no added benefit, either in perceived quality or bitrate. What about with -q1 or -q2?

akupenguin · 27th January 2006, 21:06

There is no such thing as multipass with any -q#.
The point of multipass is to let x264 decide the QPs. If you force one, then there's nothing left for ratecontrol to do.

lexor · 27th January 2006, 22:03

also I just tied converting an mpeg2 (about 8mbps) TV capture, just to see what would happen and fps dropped to 10.81 and bitrate was about 40mbps, is that normal? seems obscenely huge.

though normally I wouldn't convert mpeg2 capture to that, my intent is to capture straight to lossless h264, so I wouldn't be affected by the slowdown, still seems weired.

akupenguin · 27th January 2006, 22:42

Lossless being only 5 times the bitrate of mpeg2? doesn't so bad to me.
Consider that 480p/30fps is 124mbit uncompressed.

virus · 27th January 2006, 22:52

does -q0 actually force ipratio and pbratio (where applicable) to 1?
and does it force some other option to a specific value? The loopfilter should already be disabled at such QPs, but I guess disabling stuff like trellis makes sense, too.

(btw: bitrate savings of 1-2% aren't so negligible, as far as pure coding efficiency is concerned, in the lossless world - provided that reductions come at a reasonable speed cost)

akupenguin · 27th January 2006, 22:57

Code:

if( !h->param.rc.b_cbr && h->param.rc.i_qp_constant == 0 ) 
{
    h->mb.b_lossless = 1;
    h->param.analyse.b_transform_8x8 = 0;
    h->param.i_cqm_preset = X264_CQM_FLAT;
    h->param.psz_cqm_file = NULL;
    h->param.rc.f_ip_factor = 1;
    h->param.rc.f_pb_factor = 1;
    h->param.analyse.b_psnr = 0;
    h->param.analyse.i_chroma_qp_offset = 0;
    h->param.analyse.i_trellis = 0;
    h->param.analyse.b_fast_pskip = 0;
    h->param.analyse.i_noise_reduction = 0;
}

virus · 27th January 2006, 23:09

uhm. 8x8 transforms are, of course, disabled - there's no transform at QP 0 in HP, after all. but does that mean that i8x8 blocks are automatically deactivated as the result of clearing h->param.analyse.b_transform_8x8, or x264 just handles the special case for lossless keeping them available?

akupenguin · 27th January 2006, 23:16

Yes, i8x8 is disabled. I would have disabled i16x16 too, but it turns out it actually helps speed to leave it in.
The reason for using larger intra block sizes is to save bits on coding the prediction mode. But in lossless, a couple bits per 4x4 block to improve prediction is almost always a good tradeoff.

Tommy Carrot · 13th February 2006, 15:47

Aku, i think there is a bug somewhere in the b-rdo function, at least in lossless mode. I ran a quick lossless test with different settings, and to my surprise with enabled b-rdo the filesizes were 15-20% lower than usually. I thought such gain is impossible, and indeed, the b-frames are not truely lossless, several blocks are different than in the source, while without b-rdo, the b-frames are working correctly.

akupenguin · 13th February 2006, 18:38

fixed.

asarian · 5th August 2011, 23:36

Perhaps this is a more appropriate thread to ask my question in.

I have an (adapted) command like this:

"c:\x264\avs2yuv.exe" "f:\jobs\%1.avs" -o - | "c:\x264\x264.exe" - --demuxer y4m --frames %_frames% --sar 1:1 --aud --nal-hrd none --level 4.1 --preset placebo --qp 0 --bframes 8 --aq-mode 2 --ref %_reframes% --tune %2 --output "%3:\video\%1.mkv"

In which I merely set "--qp 0" (and removed vbv parameters and crf and such). x264 lists the output as:

x264 [info]: profile High 4:4:4 Predictive, level 4.1, 4:2:0 8-bit

Is it still lossless? Or do all the other parameters, like --aq-mode, still affect the outcome? Another way of putting this, is: what exactly does "--qp 0" make x264 disable/ignore, in terms of other parameters?

Thanks!

Dark Shikari · 5th August 2011, 23:48

Adaptive quantization is disabled with constant QP mode (since the two are, by definition, contradictory), so it's off in lossless.

B-frames are forced off in lossless as they are useless in lossless.

Lossless disables psy, which will force off most of the effects of most common --tune options.

A full list of the effects can be found in encoder/encoder.c, in these code blocks and others:

Code:

    if( (h->param.rc.i_rc_method == X264_RC_CQP || h->param.rc.i_rc_method == X264_RC_CRF)
        && h->param.rc.i_qp_constant == 0 )
    {
        h->mb.b_lossless = 1;
        h->param.i_cqm_preset = X264_CQM_FLAT;
        h->param.psz_cqm_file = NULL;
        h->param.rc.i_rc_method = X264_RC_CQP;
        h->param.rc.f_ip_factor = 1;
        h->param.rc.f_pb_factor = 1;
        h->param.analyse.b_psnr = 0;
        h->param.analyse.b_ssim = 0;
        h->param.analyse.i_chroma_qp_offset = 0;
        h->param.analyse.i_trellis = 0;
        h->param.analyse.b_fast_pskip = 0;
        h->param.analyse.i_noise_reduction = 0;
        h->param.analyse.b_psy = 0;
        h->param.i_bframe = 0;
        /* 8x8dct is not useful without RD in CAVLC lossless */
        if( !h->param.b_cabac && h->param.analyse.i_subpel_refine < 6 )
            h->param.analyse.b_transform_8x8 = 0;
    }

Code:

    if( h->param.rc.i_rc_method == X264_RC_CQP )
    {
        float qp_p = h->param.rc.i_qp_constant;
        float qp_i = qp_p - 6*log2f( h->param.rc.f_ip_factor );
        float qp_b = qp_p + 6*log2f( h->param.rc.f_pb_factor );
        h->param.rc.i_qp_min = x264_clip3( (int)(X264_MIN3( qp_p, qp_i, qp_b )), 0, QP_MAX );
        h->param.rc.i_qp_max = x264_clip3( (int)(X264_MAX3( qp_p, qp_i, qp_b ) + .999), 0, QP_MAX );
        h->param.rc.i_aq_mode = 0;
        h->param.rc.b_mb_tree = 0;
        h->param.rc.i_bitrate = 0;
    }

Code:

    if( !h->param.analyse.b_psy )
    {
        h->param.analyse.f_psy_rd = 0;
        h->param.analyse.f_psy_trellis = 0;
    }

asarian · 5th August 2011, 23:51

Wow, a reply from the man himself.

Thank you; it was most illuminating.

26th January 2006, 22:29	#1 \| Link
lexor Registered User Join Date: Jan 2004 Posts: 849	x264 lossless ok using sharktooth's lossless profile I actually get lower speed than HQ-Slow (about half the speed, and that taking in account that HQ-Slow also had a few filters running), is that normal? I mean I always thought of lossless codecs as having to do less work (since they don't have to do intense motion search/compensation and handle too many B frames etc.) so I always thought of lossless as being faster. In other words was my impression wrong and we not supposed to be able to do realtime capture with x264 lossless? (at least not with current cpu's) __________________ Geforce GTX 260 Windows 7, 64bit, Core i7 MPC-HC, Foobar2000

26th January 2006, 22:47	#2 \| Link
akupenguin x264 developer Join Date: Sep 2004 Posts: 2,392	At the same settings, x264 lossless is slower than lossy. It does all the same computations (except DCT, but that's fast anyway). But the time spent in CABAC is directly proportional to bitrate. Yes, x264 lossless should be faster than HQ-slowest because there's no point in using most of the advanced features for lossless. B-frames are valid, but don't reduce bitrate by more than 1-2%, multiref is similarly disappointing (may even increase bitrate), rate-distortion optimization doesn't do much when you have no distortion, ... Sharktooth's CQ-Lossless profile includes many of said near-useless options. My lossless settings are just Code: x264 -q0 -m3 If that's not enough for realtime, then try in order: Code: x264 -q0 -m1 x264 -q0 -m1 -A i4x4 x264 -q0 -m1 -A i4x4 --no-cabac Note that the reason most lossless codecs are fast is that they don't do any motion compensation. The exceptions being x264, Snow, and MSU. Last edited by akupenguin; 26th January 2006 at 23:11.

27th January 2006, 01:58	#3 \| Link
Sharktooth Mr. Sandman Join Date: Sep 2003 Location: Haddonfield, IL Posts: 11,768	i thought those "useless" options were automatically disabled by x264. i'll make a new lossless profile as soon as i can start updating my builds again. __________________ MPEG-4 ASP Custom Matrices: EQM V1(old), EQM AutoGK Sharpmatrix (aka EQM V2), EQM V3HR (updated 01/10/2004), EQM V3LR, EQM V3ULR (updated 04/02/2005), EQM V3UHR (updated 17/12/2004) and EQM V3EHR (updated 05/10/2004) Info about my ASP matrices. MPEG-4 AVC Custom Matrices: EQM AVC-HR Info about my AVC matrices My x264 builds. Mooo!!!

27th January 2006, 03:02	#4 \| Link
lexor Registered User Join Date: Jan 2004 Posts: 849	on 5000 frames: x264 -q0 -m1 -A i4x4 => 20.70fps x264 -q0 -m3 -A i4x4 --no-cabac => 28.81fps x264 -q0 -m1 -A i4x4 --no-cabac => 28.89fps source is 23.98fps (and is itself a x264 encode) as reported by x264, man I had no idea cabac used up so much cpu. always thought ME was biggest cpu hog. __________________ Geforce GTX 260 Windows 7, 64bit, Core i7 MPC-HC, Foobar2000

27th January 2006, 22:03	#8 \| Link
lexor Registered User Join Date: Jan 2004 Posts: 849	also I just tied converting an mpeg2 (about 8mbps) TV capture, just to see what would happen and fps dropped to 10.81 and bitrate was about 40mbps, is that normal? seems obscenely huge. though normally I wouldn't convert mpeg2 capture to that, my intent is to capture straight to lossless h264, so I wouldn't be affected by the slowdown, still seems weired. __________________ Geforce GTX 260 Windows 7, 64bit, Core i7 MPC-HC, Foobar2000

27th January 2006, 05:32	#5 \| Link
akupenguin x264 developer Join Date: Sep 2004 Posts: 2,392	ME is the biggest cpu hog normally. As I said, CABAC's cost is proportional to bitrate. If lossless is 50x the bitrate of a normal encode, then CABAC is 50x as expensive.

27th January 2006, 21:06	#7 \| Link
akupenguin x264 developer Join Date: Sep 2004 Posts: 2,392	There is no such thing as multipass with any -q#. The point of multipass is to let x264 decide the QPs. If you force one, then there's nothing left for ratecontrol to do.

27th January 2006, 22:42	#9 \| Link
akupenguin x264 developer Join Date: Sep 2004 Posts: 2,392	Lossless being only 5 times the bitrate of mpeg2? doesn't so bad to me. Consider that 480p/30fps is 124mbit uncompressed. Last edited by akupenguin; 27th January 2006 at 22:44.

27th January 2006, 22:52	#10 \| Link
virus Senior n00b Join Date: Jan 2004 Location: Italy Posts: 446	does -q0 actually force ipratio and pbratio (where applicable) to 1? and does it force some other option to a specific value? The loopfilter should already be disabled at such QPs, but I guess disabling stuff like trellis makes sense, too. (btw: bitrate savings of 1-2% aren't so negligible, as far as pure coding efficiency is concerned, in the lossless world - provided that reductions come at a reasonable speed cost)

27th January 2006, 23:09	#12 \| Link
virus Senior n00b Join Date: Jan 2004 Location: Italy Posts: 446	uhm. 8x8 transforms are, of course, disabled - there's no transform at QP 0 in HP, after all. but does that mean that i8x8 blocks are automatically deactivated as the result of clearing h->param.analyse.b_transform_8x8, or x264 just handles the special case for lossless keeping them available?

27th January 2006, 23:16	#13 \| Link
akupenguin x264 developer Join Date: Sep 2004 Posts: 2,392	Yes, i8x8 is disabled. I would have disabled i16x16 too, but it turns out it actually helps speed to leave it in. The reason for using larger intra block sizes is to save bits on coding the prediction mode. But in lossless, a couple bits per 4x4 block to improve prediction is almost always a good tradeoff. Last edited by akupenguin; 27th January 2006 at 23:18.

13th February 2006, 15:47	#14 \| Link
Tommy Carrot Registered User Join Date: Mar 2002 Posts: 863	Aku, i think there is a bug somewhere in the b-rdo function, at least in lossless mode. I ran a quick lossless test with different settings, and to my surprise with enabled b-rdo the filesizes were 15-20% lower than usually. I thought such gain is impossible, and indeed, the b-frames are not truely lossless, several blocks are different than in the source, while without b-rdo, the b-frames are working correctly.

13th February 2006, 18:38	#15 \| Link
akupenguin x264 developer Join Date: Sep 2004 Posts: 2,392	fixed.

5th August 2011, 23:36	#16 \| Link
asarian Registered User Join Date: May 2005 Posts: 1,462	Perhaps this is a more appropriate thread to ask my question in. I have an (adapted) command like this: "c:\x264\avs2yuv.exe" "f:\jobs\%1.avs" -o - \| "c:\x264\x264.exe" - --demuxer y4m --frames %_frames% --sar 1:1 --aud --nal-hrd none --level 4.1 --preset placebo --qp 0 --bframes 8 --aq-mode 2 --ref %_reframes% --tune %2 --output "%3:\video\%1.mkv" In which I merely set "--qp 0" (and removed vbv parameters and crf and such). x264 lists the output as: x264 [info]: profile High 4:4:4 Predictive, level 4.1, 4:2:0 8-bit Is it still lossless? Or do all the other parameters, like --aq-mode, still affect the outcome? Another way of putting this, is: what exactly does "--qp 0" make x264 disable/ignore, in terms of other parameters? Thanks! __________________ Gorgeous, delicious, deculture!

5th August 2011, 23:51	#18 \| Link
asarian Registered User Join Date: May 2005 Posts: 1,462	Wow, a reply from the man himself. Thank you; it was most illuminating. __________________ Gorgeous, delicious, deculture!

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode