Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Announcements and Chat > General Discussion

Reply
 
Thread Tools Search this Thread Display Modes
Old 26th October 2015, 22:32   #1  |  Link
rosh
Registered User
 
Join Date: May 2015
Posts: 11
Video sequence that gives worst compression

Hi,

What would be a video sequence that would give the worst possible compression ?
Assuming a modern encoder such as H.264/HEVC is used.

I tried a 10 minute clip of white noise (like Static on TV) but this actually gave 98% compression using an HEVC encoder. Of course it was gray-scale.

What would be worse ? a coloured-noise video ? (I can't find that anywhere)

thanks !
rosh is offline   Reply With Quote
Old 26th October 2015, 22:54   #2  |  Link
vivan
/人 ◕ ‿‿ ◕ 人\
 
Join Date: May 2011
Location: Russia
Posts: 649
And that "98% compression" was lossless? Because otherwise you can compress anything to any size. E.g. in your cause you got something that is really far from your source video.

And, yes, noise is the right answer. Assuming you're encoding 4:2:0 video colored noise should increase size by 50%.
Find it? Just use avisynth...
vivan is offline   Reply With Quote
Old 27th October 2015, 18:19   #3  |  Link
rosh
Registered User
 
Join Date: May 2015
Posts: 11
Hi,

I did another experiment with a random 'coloured noise' video (generated via : ffmpeg) : http://stackoverflow.com/questions/1...ating-tv-noise

which gave me about 74% compression.

This is all lossy compression. Using the HEVC encoder (x265, default parameters).

For coloured noise video :
Avg QP:41.44,

For grayscale noise video :
Avg QP:35.62

This article says that standard H.264 can achieve good quality video at 100:1 compression for normal video sequences. I've tried many test sequences mainly of fast motion video, and they too were around 98%-99%.

I guess is also depends on the level of entropy - maybe /dev/urandom isn't purely random to give low compression.
rosh is offline   Reply With Quote
Old 27th October 2015, 18:33   #4  |  Link
Groucho2004
 
Join Date: Mar 2006
Location: Barcelona
Posts: 5,054
AddGrainC works pretty well for this purpose. Try something like this:
Code:
colorbars(width = 1920, height = 1080, pixel_type = "yv12").killaudio()
addgrainc(var = 50000.0, uvar = 50000.0)

Last edited by Groucho2004; 27th October 2015 at 18:36.
Groucho2004 is offline   Reply With Quote
Old 27th October 2015, 19:35   #5  |  Link
vivan
/人 ◕ ‿‿ ◕ 人\
 
Join Date: May 2011
Location: Russia
Posts: 649
Quote:
Originally Posted by rosh View Post
which gave me about 74% compression.
This is all lossy compression. Using the HEVC encoder (x265, default parameters).
The problem is that this tells you nothing.
Imagine compressing picture with jpg. You compressed it to 10kb. Does it tell you anything?

Quote:
Originally Posted by rosh View Post
This article says that standard H.264 can achieve good quality video at 100:1 compression for normal video sequences. I've tried many test sequences mainly of fast motion video, and they too were around 98%-99%.
1) Everyone have different definition of a "good" quality. For me it's x264's crf 16-17, but there're people that are fine with x265 default crf (28), which I find pretty bad.
(and even though crf is much better metric than bitrate (and even psnr/ssim), it's still far from "constant quality").
2) With modern codecs bitrate doesn't mean much - quality also depends on the video itself and used encoder (x264 is the best H.264 encoder). Difference between videos with high and low complexity exceeds 10x. Difference between encoders and their settings could add 2-3x to that...
1:100 is 0.12 bpp (bits per pixel). For normal, real-life content it's usually low (at good x264 settings). For complex video (like crowd_run) it's very low. For simple video (like old_town_cross) that could be quite good.
3) It also depends on framerate and resolution - 4k@60fps requires less bpp then SD@24fps to look good.

Quote:
Originally Posted by rosh View Post
I guess is also depends on the level of entropy - maybe /dev/urandom isn't purely random to give low compression.
Well, yes, it's depends - there're ways to make noise more compressible (colorless, static, etc), though I don't believe that noise made using bad rng will be any more compressible then when using a good one.
vivan is offline   Reply With Quote
Old 29th October 2015, 01:30   #6  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 9,257
Dont know if this of use (RandomSource):- http://forum.doom9.org/showthread.ph...t=randomsource
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 30th October 2015, 17:25   #7  |  Link
rosh
Registered User
 
Join Date: May 2015
Posts: 11
Quote:
Originally Posted by vivan View Post
The problem is that this tells you nothing.
Imagine compressing picture with jpg. You compressed it to 10kb. Does it tell you anything?

1) Everyone have different definition of a "good" quality. For me it's x264's crf 16-17, but there're people that are fine with x265 default crf (28), which I find pretty bad.
(and even though crf is much better metric than bitrate (and even psnr/ssim), it's still far from "constant quality").
2) With modern codecs bitrate doesn't mean much - quality also depends on the video itself and used encoder (x264 is the best H.264 encoder). Difference between videos with high and low complexity exceeds 10x. Difference between encoders and their settings could add 2-3x to that...
1:100 is 0.12 bpp (bits per pixel). For normal, real-life content it's usually low (at good x264 settings). For complex video (like crowd_run) it's very low. For simple video (like old_town_cross) that could be quite good.
3) It also depends on framerate and resolution - 4k@60fps requires less bpp then SD@24fps to look good.

Well, yes, it's depends - there're ways to make noise more compressible (colorless, static, etc), though I don't believe that noise made using bad rng will be any more compressible then when using a good one.



Hi vivian,
Thank you for your comments, your argument make sense. Maybe a better question would be :

How do I compare the compression of two videos ? (assuming they both have identical specifications : resolution, bit-depth etc.)

I guess it should be a combination of metrics ?
- average QP
- average PSNR
- compression ratio
- bitrate.
rosh is offline   Reply With Quote
Old 30th October 2015, 19:07   #8  |  Link
vivan
/人 ◕ ‿‿ ◕ 人\
 
Join Date: May 2011
Location: Russia
Posts: 649
Compression ratio is basically a bitrate, and you can choose any bitrate so it's doesn't mean anything. However "bitrate required to get a certain quality" is one of the goals of codecs comparisons which is very easy to advertise and very hard to get correctly - many, many people have failed at that.

QP is an internal parameter, it gives some approximation of quality for a given codec.
QP of a single macroblock is basically how much information is lost. Average QP is a simple average for all macroblocks. When using CQP (Constant QP) all macroblocks are encoded with same QP (P/B frames have some constant bias, but that's not important now). But encoding them that way is not very efficient - some areas of image could be compressed more, while others should not. This is what CRF was created for and why using CQP is not recommended.

PSNR is objective metric that was designed to compare videos. It's the simplest and fastest, but it doesn't correlate with HVS (Human Visual System, basically what defines what we consider "good") well. SSIM is an example of a better metric. Still it's not perfect. In fact x264 has --tune PSNR/SSIM option that increases given metric by sacrificing visual quality.

So we are back to the most basic question: "How to perform video encoders comparison?"
Lossy video encode is defined by these 3 parameters:
- quality
- size (bitrate)
- speed
(for lossless there's no quality, this is why it's so easy with lossless - lowest size wins, the end)
Lets assume that we don't care about speed much, and will just use best possible settings that give reasonable encoding speed. Now you have two parameters: bitrate and quality. Comparing videos with different bitrate and quality doesn't make much sense (video with higher bitrate may have better quality, and it will not tell you anything), so you have to fix one. Fixing bitrate is easy, fixing quality is hard.
Okay, you have two encodes with the same bitrate, now how do you compare quality? And this is the hardest question, because it does not have the best answer.
One option is visual comparison - you compare different frames and you decide which version is better (simplest way to do this is to use avsp and avisynth, but you can do it in any way you want). The problem is that people have different opinions, and also different encoders (and even settings) produce different kind of artifacts (e.g. x264 blocks while x265 blurs).
Another option is using objective metrics. It's very easy to do, no subjective opinion is needed, you can make cool plots with percents, etc. Downside is that there's no perfect metric.

Results also will be different for different bitrate/quailty levels (at high bitrate one encoder may be better, while at low - another), and of course for different videos. So it's important to perform several tests.

To sum up:
Comparisons are hard, the easiest and fastest thing I could recommend is encoding at fixed bitrate (2-pass) with SSIM tuning and comparing SSIM score.
For "bitrate required to get a certain quality" test - do lot of encodes with different bitrates (for both encoders), pick ones with closest SSIM and compare bitrates.
Also check this report to see cool plots and meaning behind them.

Last edited by vivan; 30th October 2015 at 19:18.
vivan is offline   Reply With Quote
Old 31st October 2015, 13:51   #9  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,208
Quote:
Originally Posted by rosh View Post
What would be worse ? a coloured-noise video ? (I can't find that anywhere)
Just create it yourself

On a POSIX system, you can use something like:
dd if=/dev/random of=random.yuv count=100MB
__________________
There was of course no way of knowing whether you were being watched at any given moment.
How often, or on what system, the Thought Police plugged in on any individual wire was guesswork.



Last edited by LoRd_MuldeR; 31st October 2015 at 14:32.
LoRd_MuldeR is offline   Reply With Quote
Old 31st October 2015, 15:01   #10  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 9,257
Random source posted above is colored 640x480 @ 24 FPS.

See Mediafire in sig below for dll + source.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 31st October 2015, 17:38   #11  |  Link
raffriff42
Retried Guesser
 
raffriff42's Avatar
 
Join Date: Jun 2012
Posts: 1,375
Code:
LoadPlugin("QTGMC_Plugins\AddGrainC.dll")
ColorBarsHD
ConvertToYV12
Amplify(0)
Tweak(cont=0.5, sat=0.5)
AddGrainC(9000, 9000, 0.01, 0.01)
MediaInfo: encoded w/ x264vfw, ratecontrol="lossless"
Code:
General
Complete name                            : D:\VideoProjects\work\_misc AGC(9000,9000,0.01) x264-LL.avi
Format                                   : AVI
Format/Info                              : Audio Video Interleave
File size                                : 403 MiB
Duration                                 : 10s 10ms
Overall bit rate                         : 338 Mbps
Writing library                          : VirtualDub build 35491/release

Video
ID                                       : 0
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : High 4:4:4 Predictive@L3.2
Format settings, CABAC                   : No
Format settings, ReFrames                : 3 frames
Format settings, GOP                     : N=1
Codec ID                                 : H264
Duration                                 : 10s 10ms
Bit rate                                 : 336 Mbps
Width                                    : 1 288 pixels
Height                                   : 720 pixels
Display aspect ratio                     : 16:9
Frame rate                               : 29.970 fps
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 12.089   // Bits/(Pixel*Frame) for uncompressed YV12 = 12.000
Stream size                              : 401 MiB (100%)
Writing library                          : x264 core 146 r2538bm 121396c
Encoding settings                        : cabac=0 / ref=3 / deblock=0:0:0 / analyse=0x1:0x111 / me=hex / subme=7 / psy=0 / mixed_ref=1 /
me_range=16 / chroma_me=1 / trellis=0 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=0 / chroma_qp_offset=0 / threads=6 / 
lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=0 / 
weightp=0 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc=cqp / mbtree=0 / qp=0
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709
Color range                              : Limited
raffriff42 is offline   Reply With Quote
Old 31st October 2015, 20:47   #12  |  Link
vivan
/人 ◕ ‿‿ ◕ 人\
 
Join Date: May 2011
Location: Russia
Posts: 649
Impressive, but with MagicYUV I've managed to get 11.774 bps
vivan is offline   Reply With Quote
Reply

Tags
compression, encoding, noise

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 08:46.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.