how to know the crf of a video

[h3nry]

I'm just a beginner in video encoding and I have many doubts

I've a lot of videos,hd and sd,downloaded from youtube,recorded...and i'm running out of space.so I want to compress the videos as much as possible without quality loss.

first I tried to re-encode a 720p video of 3012kbps using Handbrake v0.9.8 with crf=19.0,without changing the resolution.I used extreme x264 settings and I spent hours for encoding expecting huge decrement in size but the re-encoded video turned out to be nearly double the size.here's the settings I used:-

Encoding settings: cabac=1 / ref=6 / deblock=1:-3:-3 / analyse=0x3:0x133 / me=umh / subme=10 / psy=1 / psy_rd=1.00:0.20 / mixed_ref=1 / me_range=64 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=0 / chroma_qp_offset=-3 / threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=5 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=1 / weightp=2 / keyint=infinite / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=250 / rc=crf / mbtree=1 / crf=19.0 / qcomp=0.60 / qpmin=3 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00 plus I set the interlacing & decombing filters to default.

so I guessed the problem will be with the crf value I use and I tried 21 and 23.but the size is still bigger than the source! so I chose handbrake's high profile settings and let it choose the default settings except I altered ref=1,which is same as source.after many trials i found that a crf value of 25.4 produces a video with same bitrate as the source thus same size(25.4 is way too far than the recommended value).does that mean the encoded video and the source will be of same quality?but after analysing some screen shots it seems there is some loss in detail in high motion scenes even though majority of the frame seems intact.its the property of crf,right?it takes details from high motion scenes where we will hardly notice and thus saving bits creating a vbr mode.or is it quality loss?note that the source is also in variable bitrate mode.

or is the increase in size due to some other issues?please help.is there any recommendations on the settings I use?

[mp3dom]

AVC is lossy (excluding some lossless profiles aimed for professional/broadcast use). This means that you can compress the source at even higher bitrates (50 Mbps for example) but you will still have differences/loss compared to the source. You can at max expect a 'visually transparent' result, which means that during playback you can't detect differences between two sources. CRF have almost the same encoding engine as a 2pass. If you want to target some specific bitrate/filesize, it's better to use the 2pass encode rather than do various different CRF.

[h3nry]

I insist on using crf because it takes only half the time of 2pass encoding and more importantly it compresses more efficiently creating a vbr video which keeps the quality constant taking less size than a 2pass encoded cbr video.and that's exactly what I want.you are saying that there will be difference between re-encoded and the source video,no matter what settings we choose,right?but how to choose the right crf value for a video to be almost identical to the source,ie which isn't too high to waste bits and create size more than source or too low for quality loss.handbrake's recommended values for hd videos are 22+/-1.but that didn't work for me.is my technique right?are there any recommendations about the settings I chose?

[LoRd_MuldeR]

Your understanding of 2-Pass and CRF seems to be wrong

Actually they use the same rate-control algorithm and thus none of them "compresses more efficiently" than the other one. After all, if we have two files, one encoded with 2-Pass mode and one encoded with CRF mode, and both of these files happen to come out at the same file size (i.e. the same average bitrate), then the quality of those files will be pretty much identical!

The difference between 2-Pass mode and CRF mode is that with 2-Pass mode you can know the final average bitrate (i.e. the final file size) in advance, because the encoder will constantly re-adjust the so-called "rate factor" in order to hit the selected (average) bitrate. At the same time, CRF mode, as the name implies, simply uses a constant "rate factor" value. Therefore with CRF mode the final average bitrate (file size) cannot be known in advance. And that's it.

Finally it is wrong to think about the CRF value as a quality indicator/selector! Indeed, as long as you don't change any other options, the same CRF value will give approximately the same visual quality for different sources. But as soon as you change other encoding options, this is no longer the case! For example, many people believe that if they use "slower" encoder settings and the same CRF value, they will get the exactly same quality as with "faster" settings (and that CRF value) but at a lower bitrate (smaller file). In reality, however, the very same CRF value may result in pretty different quality for "fast" and for "slow" settings, which means the resulting bitrates (file sizes) are not even comparable anymore! Therefore you may very well end up with bigger file (higher average bitrate) when using the "slower" settings (for the same CRF value). Only thing you know for sure is that "slower" settings will improve the compression efficiency, i.e. the "quality per bit" ratio. But you don't know how the overall quality will change, for a given CRF value. So you may need to re-adjust your CRF value after changing other settings...

[mp3dom]

2pass is VBR too... If you do two different CRF (assuming that in the second time you've hit your bitrate/filesize target) you have spent more time than a single "2pass" encode. And the result is nearly identical. Please note that the 1st analysis pass of a 2pass encode takes less time than a regular 'encoding' pass (unless you use --slow-firstpass).

[h3nry]

sorry,my bad!

what do you suggest for compressing the video to minimum size without quality loss?

[Guest]

Lossy compression will always produce some quality loss. And what do you mean by "minimum size"?

[LoRd_MuldeR]

h3nry, you need to be aware that "high quality", "low bitrate" (small file size) and "fast encoding speed" are three competing goals. You cannot "optimize" them all at the same time. You can only find a decent compromise/trade-off.

So I think you have to decide what exactly you want:

(1) If you want to hit a specific file size (i.e. a specific average bitrate), then simply use 2-Pass mode with exactly that target bitrate. Use the "slowest" settings you can afford to squish out the maximum quality at that size.

(2) If you want to retain a certain level of quality, first pick the "slowest" settings you can afford and then find the highest possible CRF value that still satisfies your quality needs. This will be as small as it can get for the desired quality.

(3) As others have said, with a "lossy" compression each re-encoding unavoidably causes a certain amount of loss ("generation loss"). Get an external USB-HDD drive, move your files there and avoid re-encoding altogether...

[hello_hello]

Originally Posted by h3nry

I'm just a beginner in video encoding and I have many doubts

I've a lot of videos,hd and sd,downloaded from youtube,recorded...and i'm running out of space.so I want to compress the videos as much as possible without quality loss.

first I tried to re-encode a 720p video of 3012kbps using Handbrake v0.9.8 with crf=19.0,without changing the resolution.I used extreme x264 settings and I spent hours for encoding expecting huge decrement in size but the re-encoded video turned out to be nearly double the size.here's the settings I used:

Handbrake, Handbrake, Handbrake......

Yes, as far as encoding GUI's go it's easy to use, but that's kind of it's only redeeming feature.

I'd guess you loaded Handbrake's default settings, hovered the mouse over the x264 settings, read the tooltips and adjusted each according to what makes the most sense..... which seems logical but really isn't the way to do it. You probably shouldn't touch x264's advanced settings unless you really know what you're doing. I don't, so I rarely do. There's even potential to use settings which will make your encode unplayable using some standalone devices.

x264 has it's own speed presets built in, which I think is possibly what LoRd_MuldeR was referring to when he said to use the slowest settings you afford. It also has built in "tuning" presets for use according to the type of video you're encoding (film, grain, animation). The speed presets automatically adjust the relevant x264 advanced settings accordingly, as do the tuning presets. The problem is, Handbrake doesn't give you access to either via it's GUI, so many Handbrake users are completely oblivious to their existence.

Using MeGUI as an example (as I'm pretty familiar with it), when you open the x264 encoder configuration you're shown the basic x264 settings. There's a button to load x264's default settings (I don't think any of Handbrake's presets load exactly the x264 defaults). The default x264 speed preset is medium, the default tuning is "none" and there's a couple of other basic options. When you tick the "show advanced settings" box, a few new tabs appear under which you'll find the advanced options. If you make note of them and go back to the first tab, change the speed preset from medium to slow, then go back to the advanced settings, you'll see some have changed accordingly, and the new settings effectively become the new "defaults". Same when you adjust the "tuning". Not that it makes it necessary to adjust the advanced settings yourself.... the description is more to explain how it works.

So unless you're really in the "know what you're doing category" it's best to use x264's built in speed presets for the quality/file size/encoding compromise to which LoRd_MuldeR referred. The default speed preset of medium will do a perfectly good job using a low enough CRF value (I use it with CRF18 for HD encodes regularly as my PC is well overdue for an upgrade and a bit slow by today's standards) but as LoRd_MuldeR suggested, it's probably best to first decide on the slowest preset you'd normally use for encoding, then adjust the CRF value for the quality you desire.

If you like using Handbrake, maybe try VidCoder instead. It uses HandBrake as its encoding engine and the GUI is very similar, but it gives you access to the x264 built in speed presets and tunings. Unfortunately when you change the speed preset/tuning it doesn't automatically change the advanced options in it's GUI accordingly, but at least it makes it easy to use them.

There's probably no point looking at the encoder settings used for encoding video you're re-encoding. It's pretty much irrelevant to how it'll be re-encoded. There's kind of only two options. Use the CRF value and speed preset you normally would and live with the resulting file size, or use the speed preset you normally would while choosing the file size and live with the quality.

[Groucho2004]

Originally Posted by hello_hello

It uses HandBrake as its encoding engine

Never knew that it has its own encoding engine. I would have thought that it uses libx264 like all the other graphical user interpolators.

[LoRd_MuldeR]

Originally Posted by Groucho2004

Never knew that it has its own encoding engine. I would have thought that it uses libx264 like all the other graphical user interpolators.

HandBrake is a command-line encoder, similar to FFmpeg/MEncoder. It uses various encoding/decoding libraries, including libx264. VidCoder is just an alternative GUI for Handbrake.

[hello_hello]

Originally Posted by Groucho2004

Never knew that it has its own encoding engine. I would have thought that it uses libx264 like all the other graphical user interpolators.

Well that's the description used by the makers of Vidcoder, so due to an inability to come up with anything more imaginative....
VidCoder often has something such as this in it's release notes, so I guess that's in line with what LoRd_MuldeR is saying:
"Upgraded HandBrake core to 0.9.8 release".

Out of curiosity I ran a couple of quick encodes just to see what x264 settings Handbrake does use. I was curious because some of h3nry's settings aren't ones which you can access via Handbrake's GUI (I don't think).
It's High Profile preset uses all x264 defaults aside from rc-lookahead which is increased from 40 to 50, b-adapt is optimal instead of fast and qpmin is increased form 0 to 3. If you use HandBrake's "reset all" button the only setting which isn't default is qpmin for which it still uses 3. Probably no big deal.

I guess newer versions of x264 might compress a little better. The same small CRF18 encode using MeGUI (x264 core 129) and default x264 settings resulted in a file size around 1% smaller than HandBrake (x264 core 120). I remuxed each using the same version of MKVMergeGUI before comparing them.

[AnonCrow]

Originally Posted by h3nry

first I tried to re-encode a 720p video of 3012kbps

Does Mediainfo show any details on what settings were used when encoding the source video ?

If not , might want to analyze the video to at least get a crude estimate on the quantizers used in it.

[h3nry]

Originally Posted by hello_hello

Handbrake, Handbrake, Handbrake......

Yes, as far as encoding GUI's go it's easy to use, but that's kind of it's only redeeming feature.

I'd guess you loaded Handbrake's default settings, hovered the mouse over the x264 settings, read the tooltips and adjusted each according to what makes the most sense..... which seems logical but really isn't the way to do it. You probably shouldn't touch x264's advanced settings unless you really know what you're doing. I don't, so I rarely do. There's even potential to use settings which will make your encode unplayable using some standalone devices.

hey,all the tuning and preset options are explained in handbrake guide.it's not that hard to understand,only a couple of options are changed.but you cannot simply select some preset and press the convert button.I need to get max quality and compression,but I cannot simply use the placebo or very slow preset because they use reframes=16,as you yourself said "There's even potential to use settings which will make your encode unplayable using some standalone devices".they are known as levels.I am using xperia u.it only supports till high profile @ level 3.2.that means I cannot use reframes >5 for a 720 video and >13 for a 480p video,and I cannot watch a 1080p video cause its high profile @ level >= 4.0.but neither can I use medium or fast presets,cause my device only has 4gb user accessible memory with no external slot.trellis=2 and subq=11/10 is only used in placebo and veryslow presets,which gives great compression without quality loss!also bframes=16 is used in placebo which greatly helps animes but bframes>5 is a waste of time in real life videos.same is in me range where it's clamped to 24 in placebo and veryslow,where you'll get the benefit of increasing to 32 in high motion hd videos,but a waste of time >16 in sd videos.these options given so that it can be changed to meet our needs.instead of not touching them try using it.and I didn't just hover my mouse over them,all options are well explained in mewiki.

of course handbrake is easy to use,but also handbrake have better settings than others.I have also used avidemux,vidcoder,vlc,avs4u...for example in avidemux 'subpixel me and mode decision' (subq) is limited to 9,rd refine in all frames.same's in vlc.but handbrake provides upto 11,no early termination.also there is the command line for setting extra options which's not given.handbrake also comes with high quality filters such as denoise,decomb etc which comes from mplayer.and there is handbrake guide showing how to use each one of them.sometimes I also use vidcoder for large files,cause there is a pause button.

[h3nry]

Originally Posted by AnonCrow

Does Mediainfo show any details on what settings were used when encoding the source video ?

Try using Media player classic-home cinema(mpc-hc).it's a very light video player.it's shows every details same mediainfo. it also shows all the x264 setting I used when encoding using hand brake and vidcoder. don't no about other encoders!

[hello_hello]

Originally Posted by h3nry

hey,all the tuning and preset options are explained in handbrake guide.it's not that hard to understand....

Yeah after a bit of a look around I found a link in the guide which gives instructions for adding them to the command line manually. I wonder how many Handbrake users are aware of the ability to add them though.

Originally Posted by h3nry

only a couple of options are changed.but you cannot simply select some preset and press the convert button.I need to get max quality and compression,but I cannot simply use the placebo or very slow preset because they use reframes=16,as you yourself said "There's even potential to use settings which will make your encode unplayable using some standalone devices".they are known as levels.I am using xperia u.it only supports till high profile @ level 3.2.that means I cannot use reframes >5 for a 720 video and >13 for a 480p video,and I cannot watch a 1080p video cause its high profile @ level >= 4.0.but neither can I use medium or fast presets,cause my device only has 4gb user accessible memory with no external slot

Well the way I understand it, setting the appropriate profile/level when encoding keeps all that under control. If you set High Profile 3.2, it takes precedence, followed by the speed preset. Any advanced options you might specify manually can over-ride the level specified.
So you can in fact use the placebo speed preset when using High Profile, Level 3.2, and it won't allow the number of reference frames etc to be exceeded by the speed preset chosen.
Of course HandBrake doesn't let you specify a Profile or Level via it's GUI, and there's no mention of it in the guide I saw, although no doubt they'd be specified the same way you'd manually add a speed preset to the command line. VidCoder at least lets you choose the Level to be used when encoding via it's GUI.

Originally Posted by h3nry

trellis=2 and subq=11/10 is only used in placebo and veryslow presets,which gives great compression without quality loss!

So there's one reason to use the veryslow or placebo preset. subq=11/10 is probably one of the main reasons they're slow presets.

Originally Posted by h3nry

also bframes=16 is used in placebo which greatly helps animes but bframes>5 is a waste of time in real life videos.

The B frame setting is the maximum number of B frames the encoder is allowed to use, not the number it's forced to use. I'd like to know how you've decided bframes>5 is a waste of time in real life videos but great for animes.

Originally Posted by h3nry

same is in me range where it's clamped to 24 in placebo and veryslow,where you'll get the benefit of increasing to 32 in high motion hd videos,but a waste of time >16 in sd videos

No, it's not clamped. You're free to over-ride any setting used by a speed preset by specifying the desired value for that setting in the command line manually (or via the GUI if it'll let you). If you think it should be changed according to resolution there's nothing stopping you from doing so.

Originally Posted by h3nry

these options given so that it can be changed to meet our needs.instead of not touching them try using it.and I didn't just hover my mouse over them,all options are well explained in mewiki.

By all means do so if you think you really know what you're doing. Otherwise you can expect some unexpected results until you do.

Originally Posted by h3nry

of course handbrake is easy to use,but also handbrake have better settings than others.I have also used avidemux,vidcoder,vlc,avs4u...for example in avidemux 'subpixel me and mode decision' (subq) is limited to 9,rd refine in all frames.same's in vlc.but handbrake provides upto 11,no early termination.also there is the command line for setting extra options which's not given.handbrake also comes with high quality filters such as denoise,decomb etc which comes from mplayer.and there is handbrake guide showing how to use each one of them.sometimes I also use vidcoder for large files,cause there is a pause button.

Never used any of those programs myself. Any decent GUI should allow you to change those settings.
HandBrake's high quality de-interlacer uses Yadif according to the guide, as does I imagine almost every other current encoder GUI. I don't know if it's decomb filter does anything special. According to the guide "it combines several techniques gleaned from tritical's decombing filters for AviSynth", and pretty much every AVISynth based encoder GUI makes use of decombing filters.....
Next time I come across one of those "difficult videos" I'll try running it through HandBrake to see how it manages it with the decomb filter set to auto. Maybe it's very clever, I don't know.

[cold_hell]

Reencoding already crap quality gives you worse quality than encoding from good quality source at the same bitrate (two-pass). Youtube clips quality is very low and it is not good for a source

Originally Posted by h3nry

I am using xperia u.it only supports till high profile @ level 3.2.that means I cannot use reframes >5 for a 720 video and >13 for a 480p video,and I cannot watch a 1080p video cause its high profile @ level >= 4.0

It uses android 2.3+ as far as i know. Just download MX player and use it's Software decoder (yes it can decode even 10bit profiles and they are much better for bitrate/quality). What is the problem with profiles if the player doesn't check it and there is no hardware restriction? Also especially your phone can play 1080p at fullspeed with --no-deblock --no-cabac (tested on my friend's phone - xperia u) However it is worse than 720p with deblock and cabac save alot space, so basicly it is useless, but possible. It can cause problems only with refs since device memory is limited but most clips don't use alot. You can set --level 5.2 on 200x200 video with 1ref if you want but profile doesn't mean "more decoding power required" it is just limitation for hardware playback. If you want to use hardware decoder- that is total different story. Hardware "accelerated" decoding sux at it's current state - my personal oppinion.

also --me umh --merange 64 is insane, slow and useless
I doubt that there will be any benefits --me umh --merange 32 is enough for HD/Full HD

[h3nry]

Originally Posted by hello_hello

The B frame setting is the maximum number of B frames the encoder is allowed to use, not the number it's forced to use. I'd like to know how you've decided bframes>5 is a waste of time in real life videos but great for animes.

yeah but h.264 rearly uses bframes >3.when setting adaptive bframes=optimal,setting more than 4/5 bframe is insane unless you have lot of time to waste.

[h3nry]

Originally Posted by cold_hell

Just download MX player and use it's Software decoder (yes it can decode even 10bit profiles and they are much better for bitrate/quality). What is the problem with profiles if the player doesn't check it and there is no hardware restriction?

yeah,I'am already using mxplayer since the day I bought my phone.true,the player can play all levels,profiles,formats.using s/w 16bit decoder for upto 720p,videos may plays smoothly.but it's like turning off the bravia engine.the color decoding is worse.it looks like the colors are washed out.setting s/w 32bit decoder improves color,but going above sd videos greatly slows down the video playback,while the audio plays at normal speed.changing to h/w decoder is same as s/w 32 bit.then there is the h/w+ decoder from v1.7.7 onwards.it plays upto 720p with enough colors.but any more than level 3.2 the decoder automatically falls back to s/w 16bit,still slower video playback.I also tried 720p with 6 reframes.the default player plays videos upto level 3.2 with awesome colors.so you see there is no reason for me to use mxplayer than my default player.I use mx player for other than mp4 format playback.

Originally Posted by cold_hell

It can cause problems only with refs since device memory is limited but most clips don't use alot. You can set --level 5.2 on 200x200 video with 1ref if you want but profile doesn't mean "more decoding power required" it is just limitation for hardware playback.

may be.but I have to use maximum reframes it can afford cause my phone's only got 4gb user accessible memory.and I have to use all the x264 settings that reduces the size.

Originally Posted by cold_hell

Youtube clips quality is very low and it is not good for a source

true!may be because streaming videos uses lot of I frames.

[hello_hello]

Originally Posted by h3nry

yeah but h.264 rearly uses bframes >3.when setting adaptive bframes=optimal,setting more than 4/5 bframe is insane unless you have lot of time to waste.

I just tried a quick little test encode to experiment. Using a 4 minute 704x384 AVI as the source. I used presetveryslow and tuning=grain. Maximum consecutive bframes was 7, but the instance of more than 3 consecutive bframes only totalled 2.5% of the b frames used. Limiting the number of bframes to 5 shaved 3 seconds of encoding time, bframes 3 shaved off another 7, so no doubt it makes some difference.
In terms of speed decrease I don't think it falls into the "insane" department though. At least not in terms of having a lot of time to waste when compared to some of the settings you prefer.
VerySlow and tuning grain are as close to your settings as I could get using the standard x264 presets. The same encode using x264's default settings reduced the encoding time by 3 min, 20 seconds.

I think you're wrong about level 3.2 needing a limit of 13 ref frames. The way I read it on Wikipedia it only applies to level 3.1 and it's probably when using a 720x? resolution too. When setting 3.1 as the level x264 didn't limit my 704x384 encode to 13 ref frames. To make sure it was working correctly I tried again at 720p and sure enough it limited ref frames to 5 even though the preset says 16.