Taking submissions for a small encoder comparison

[Dark Shikari]

With the MSU encoder comparison coming out, we're going to have a good comparison (hopefully) of objective quality (SSIM) between various encoders. Unfortunately, not only does their comparison contain basically only H.264 encoders (plus Theora and Xvid), it also doesn't focus on visual comparisons, so I figured I would make my own small one to get a feel for the "current state of the art". This is a visual comparison only; no measurements of SSIM or PSNR will even be made.

Source: parkjoy (http://media.xiph.org/video/derf/y4m/1080p/park_joy_1080p.y4m) (1080p50)
Target file size: 17000KB (can be a bit over or under if the encoder won't reach the exact value, obviously). Using constant quality mode is fine as long as you can get it close to the target size. If the encoder has no 2-pass ABR mode, use constant quality mode (as I did with Dirac).
Keyframe interval: 250

http://img190.imageshack.us/img190/9841/outzd.png

Stuff I already have:

x264 (PSNR-optimized and psy-optimized)
VP7
Theora (Thusnelda and Ptalabvorm)
Dirac (through Schroedinger)
Xvid
ffmpeg mpeg-4
ffmpeg mpeg-1
Bink
SVQ1
SIF1
CudaH264Enc
Ateme (v1.5 and v2.0)
Elecard
Samsung+BBC H.265 proposal
Mainconcept 8.5
Microsoft VC-1 SDK

Stuff I'm interested in:

GPU encoders (e.g. Badaboom)
Any other comedy options you can think of that can hit the target bitrate
VP8 (if you have the encoder for this, please contact me directly and we can arrange for just the decoded output to be uploaded)
Other H.265 proposals?
Reference encoders (JM, JSVM, etc)
Anything else?

To submit a test, just upload the video file along with the settings you used to create it. All encoders should use their "best" settings possible, whatever they happen to be.

In a few days, after I get a few more submissions, I'll post a blind visual test where people can give scores to each codec.

Rationale for the choice of test clip:

1. It's a lossless source. Using pre-compressed sources results in all kinds of stupid arguments about which encoder benefits the most from the compression artifacts already present in the source. It's also free to distribute.

2. It contains a wide variety of high-detail, highly colorful content that should be a serious challenge for any encoder.

3. It shouldn't bias too heavily towards any one encoder like many of the other standard test clips will:
a. It's relatively high motion, so it won't bias heavily against encoders without B-frames or qpel (as, say, mobcal does).
b. It's not so high motion that it would cripple video formats that don't support motion vectors longer than 16 pixels (e.g. Theora).
c. It's not something that benefits an unreasonably large amount from some of x264's algorithms (which is why I picked this and not parkrun).

Overall it should serve as a good, relatively fair test. Ideally I'd want to test more clips, but I don't have that much time and doing psy comparisons is a lot of effort.

About the source:

The source is from the SVT Multiformat (http://media.xiph.org/video/derf/vqeg.its.bldrdoc.gov/HDTV/SVT_MultiFormat/SVT_MultiFormat_v10.pdf) test set (thanks Sveriges Television (http://en.wikipedia.org/wiki/Sveriges_Television)!). It was filmed using an ARRI ArriFlex 765 (http://www.arri.de/camera/65_mm_film/arriflex_765.html) camera on 65mm film using 1/100th second exposure time, hence the staggering level of detail all the way up to 2160p.

[morqueozwald]

Theora Ptalarbvorm: This was the most recent svn version that affects output, the front end was ffmpeg2theora also the most recent version.
The command line: ffmpeg2theora --optimize -V 13600 --two-pass -K 250 park_joy_1080p.y4m
http://www.mediafire.com/file/qjmnjnzrznn/ParkJoy.ogv

This falls under the "comedy" option.
Quicktime H264 bitrate set to 14200 to hit target size, key frame 250, two pass and no restriction on profile. I first encoded the file with x264 lossless so Quicktime could read it.
http://www.mediafire.com/file/ikynzymzmto/ParkJoy2.mp4

[Dark Shikari]

Theora Ptalarbvorm: This was the most recent svn version that affects output, the front end was ffmpeg2theora also the most recent version.
The command line: ffmpeg2theora --optimize -V 13600 --two-pass -K 250 park_joy_1080p.y4m
http://www.mediafire.com/file/qjmnjnzrznn/ParkJoy.ogv

This falls under the "comedy" option.
Quicktime H264 bitrate set to 14200 to hit target size, key frame 250, two pass and no restriction on profile. I first encoded the file with x264 lossless so Quicktime could read it.
http://www.mediafire.com/file/ikynzymzmto/ParkJoy2.mp4The Quicktime video appears to have completely screwed up brightness...

Ptalabvorm added.

[AnonCrow]

CudaH264Enc through MediaCoder 0.7.3.4640, ABR and VBR (constant quantizer) encodes, Mediacoder preset files included :

http://www.mediafire.com/?sharekey=bf225ce38c0772e5b94117dade8fc2953f70e2d9b9b35f924df0d6082f1c2cd0

... although, the file size produced by the VBR preset varies by a couple KB between encodes.
( brief help on cudah264enc cli options: http://forum.mediacoderhq.com/viewtopic.php?f=2&t=6896#p21699 )


Comedy options:

A M-JPEG encode should always included, either as a comedy option or to make the MPEG-1 encode feel better about itself.
Possibly also MNG, APNG and M-JPEG 2000 encodes to complement the M-JPEG one.

[Dark Shikari]

CUDAH264Enc added.

[shon3i]

Ateme (eavc engine 1.5)

Settings used:
TwoPass encoding
Profile@Level: High 5.1
Entropy Coding: CABAC
Quality: Full
Psychovisual: Best
Adaptive Deblock: 0
WeightP
RefP: 5
RefB: 5
Max GOP Size: 250
OpenGOP

http://www.mediafire.com/?yuykyyzjhzm

Elecard (Converter Studio 3.2)

Settings used:
TwoPass encoding
Profile@Level: High 5.1
Entropy Coding: CABAC
Quality: Highest
Psychovisual: Complexity & Luma, strength: 4
Deblock: 0:0
B-pyramid
Ref frames: 16
GOP Length: 250

http://www.mediafire.com/?y4miontkzmn

[morqueozwald]

The Quicktime video appears to have completely screwed up brightness...

I never noticed this before but this is apparently a very common problem. Finder's information on the file says "Color Profile: HD (1-1-1)" whereas x264 produced files don't say anything (w/r to color) in the info. Could this be responsible?

[Dark Shikari]

@shon3i

Added.

[shon3i]

Mainconcept (SDK 8.5 Brodcast encoder)

Settings used:
TwoPass encoding
Profile@Level: High 5.1
Entropy Coding: CABAC
Performance: 15-Best Quality
Psychovisual: Luma & Contrast & Complexity, strength: 100-100-100
Deblock: 0:0
B-pyramid
Ref frames: 16
GOP Length: 250

http://www.mediafire.com/?zvzgn0omion

Tomorow i will encode with Ateme v2 engine encoder. Maybe someone can post Elecard with Converter Studio Pro, because have more advanced options

[Dark Shikari]

Psychovisual: Luma & Contrast & Complexity, strength: 100-100-100Are you sure this is a good idea? Not only is it insane overkill to combine so many AQ modes like that, but I recall that complexity works on an inverse scale, with negative being a bit like x264's AQ.

Also, the colors are completely wrong on that clip. The way I'm extracting screenshots is using ffmpeg, and even if that doesn't get the right ranges, the range should at least match among all the H.264 encoders. This encode is totally different from all the others.

[shon3i]

I do some test's before with cinevision, and i am happy with results, now i test a bit, and seem that is best combo.

btw new encode from elecard is on a way, i figure out how to change some options that not visible in gui, if some difference exist i will notice you tomorow.

[Dark Shikari]

I do some test's before with cinevision, and produce best results, now i test a bit, and seem that is best combo.

btw new encode from elecard is on a way, i figure out how to change some options that not visible in gui, if some difference exist i will notice you tomorow.Can you try fixing the colors at least?

[shon3i]

which encode? and what is wrong with colors? i didn't notice?

[Dark Shikari]

which encode? and what is wrong with colors? i didn't notice?x264 (http://mirror05.x264.nl/Dark/Flash/compare/x264.png) mainconcept (http://mirror05.x264.nl/Dark/Flash/compare/mainconcept.png)

Command for extracting both:

ffmpeg -i input.mkv -ss 5.74 -vcodec png -f rawvideo -vframes 1 output.png

[shon3i]

x264 (http://mirror05.x264.nl/Dark/Flash/compare/x264.png) mainconcept (http://mirror05.x264.nl/Dark/Flash/compare/mainconcept.png)

Command for extracting both:

ffmpeg -i input.mkv -ss 5.74 -vcodec png -f rawvideo -vframes 1 output.png
aha, i think i found problem, since i use directshow encoder, some colorspace converter are pinned in graph between encoder and source. i will now reencode.

[shon3i]

Link updated colors should now be good. Btw i will play little more with AQ's and i will notice if i made better encode.

[Dark Shikari]

Link updated colors should now be good. Btw i will play little more with AQ's and i will notice if i made better encode.Please don't. I don't want anyone to specifically tweak the encoders for this particular sample; pick settings that you'd use for all videos.

[Biggiesized]

Any particular reason to use the clip at 1080p50? Why not do a Blu-ray compatible shoot-out (that is, 720p50)? That way you can throw in Blu-ray encoders if users have access to them.

[Dark Shikari]

Any particular reason to use the clip at 1080p50? Why not do a Blu-ray compatible shoot-out (that is, 720p50)? That way you can throw in Blu-ray encoders if users have access to them.You can easily use a Blu-ray encoder by setting the framerate to 25 and halving the bitrate ;)

[Biggiesized]

Can x264 do 1080i MBAFF encoding for Blu-ray?

[Dark Shikari]

Can x264 do 1080i MBAFF encoding for Blu-ray?Yes, it can.

[Schrade]

If anyone has access to an Elemental Technologies GPU encoder, I'd love to see that as one of the samples :)

[shon3i]

Please don't. I don't want anyone to specifically tweak the encoders for this particular sample; pick settings that you'd use for all videos.
Ok then, this encode use max possible settings that directshow encoder can have. Use this encode, it's ok.

[shon3i]

Ateme (eavc engine 2.2)

Settings used:
TwoPass encoding
Profile@Level: High 5.1
Entropy Coding: CABAC
Quality: Ultra
Psychovisual: Best
Adaptive Deblock: 0
WeightP
Refs: 16
b-pyramid
Max GOP Size: 250
OpenGOP

http://www.mediafire.com/?ycwxmkndywj

[TinTime]

Here's a VC-1 encode using AVS2ASF (ie. Microsoft's VC-1 encoder SDK).

http://www.mediafire.com/file/2jktjz2agaz/park joy.zip

The settings used are in a text file in the zip.

[radiant chains]

Microsoft Expression Encoder 3
VC1 Advanced
Preset: "Best Quality"

Average Bitrate: 14100
Max GOP: 250
Video Complexity: Best (5)
Adaptive Dead Zone: Conservative
Differential Quantization: Off
In-loop Filter: On
Overlap Filter: On
Denoise Filter: Off
Noise Edge Removal Filter: Off
B-Frame Number: 1
Adaptive GOP: On
Closed GOP: On
Motion Chroma Search: Full True Chroma
Motion Match Method: Adaptive
Motion Search Range: Adaptive

http://www.mediafire.com/?zmywjtfj3mg

[Blue_MiSfit]

You forgot DivX!

I'll do a couple encodes, even though I never ever use DivX :devil:


Scratch that... I can't get the VFW encoder to accept the input. It complains "the source image format is not acceptable". I think it may be a profile issue.

Maybe an HC powered MPEG-2 encode?


~MiSfit

[Dark Shikari]

You forgot DivX!

I'll do a couple encodes, even though I never ever use DivX :devil:Their encoder crashed for me, so I couldn't get it to work. DivX ASP, that is.

DivX H.264 might be interesting.

[Blue_MiSfit]

Yeah, I was talking about their ASP encoder.

MPEG-2 encode
http://www.mediafire.com/?2ymwilj2mam



---------------------------------------
| HCenc - MPEG2 encoder - rel. 0.24.0 |
---------------------------------------

MPEG profile@level: MP@HL
input: d:\parkjoy_lossless.avs
output: D:\crowdjoy_HC.m2v

--------------------
| encoder settings |
--------------------

profile: BEST
frames: 0 - 499
framerate: 50.000
aspect ratio: 16:9
chroma format: 4:2:0
bitrate Kb/s: 14200
max. bitrate Kb/s: 25000
pulldown: no
closed gops: no
VBV check: yes
scene change det: yes
interlaced: auto, TFF
goplen,B-pic: AUTO
dc_precision: 8
intraVLC: auto
scan method: auto
bias: 0
chapter frames: 0
time code: 0 0 0 0
CPU: SSSE3
priority: idle
SMP active: yes
matrix: MPEG, adaptive
luminance gain: yes, level 2
adaptive quantization: yes, strength 2
deadzone quantization: auto

------------------
| source stats |
------------------

nr. of frames in source: 500
width*height: 1920x1080
fps: 50.000
nr. of frames to encode: 500
frames to encode: 0 - 499

---------------------
| encoding - pass 1 |
---------------------

pass 1 encoding time: 0:02:28 (148.11 s)
fps: 3.4

--------------------------------
| encoding - intermediate pass |
--------------------------------

bitrate set to: 14200 kb/s
estimated output file length: 17334 kB
intermediate encoding time: 0.00 s

---------------------
| encoding - pass 2 |
---------------------

pass 2 encoding time: 0:02:23 (143.05 s)
fps: 3.5

------------------
| encoding stats |
------------------


adapted intra matrix used
adapted non-intra matrix used

nr. of gops: 36
nr. of frames: 500
nr. of I-frames: 36
nr. of P-frames: 215
nr. of B-frames: 249
average bitrate: 13747
minimum bitrate: 10291
maximum bitrate: 17599

bytes in bitstream: 17187491
bits in bitstream: 137499928

average Quantizer: 25.665

total CPU time: 0:06:09 (369.30 s)
total elapsed time: 0:04:57 (297.13 s)



~MiSfit

[Atak_Snajpera]

you forgot to include "wonderful" ATI encoder:)

[Snowknight26]

I tried encoding with it but it didn't seem to work with a number of different lossless formats in different containers.

[shon3i]

DivX H.264 might be interesting. Is not since use same or older MC SDK which i use for current MC encode, and it's very restrictive, i think it's impossible to use GOP 250, and other stuff that can help to in compression.

you forgot to include "wonderful" ATI encoderIs avaible to download. I am currently encoding with Elemental/Badaboom. But without success, because is restircted to Level 4.1, which impossible with 50fps according to H264 specs.

btw Dark Shikari how H265/HVC look :) :D

Anyway one thing here maybe will be significant, that is encoding speed.

[mariush]

http://www.mediafire.com/?sharekey=437ae816991c299241446e35a78dc46325661133ff7e923f292b492bd5edc68e

Just for fun I tried Tsunami MPEG Encoder 2.5 (2.524) to encode MPG1 streams... found it surprisingly good... visually looked better than the MPEG2 videos it produced (but maybe i didn't it configure it correctly)

There's a version using CBR 13.8 mbps , highest quality motion search, default matrix (different than mpeg standard) and one using CQ mode (constant quality) and if I remember correctly I've set around 15 mbps as maximum bitrate so that it fits in 17000 KB.

Also there a clip encoded with Real Producer 13.1 (latest from their site), Real 10 codec, the rest is in the filename

Also 2 clips encoded with xvid,the one at 10mbps (max value on slider, too bad, could have gone higher as there's still space up to 17000) and a second version set at quant 12.5

The Real 10 version is made from a YV12 Lagarith encoded copy of the movie because it doesn't accept avs scripts directly (without hanging). The xvid version at 10mbps was made from an avs script with AssumeFPS(25) added.

[Keiyakusha]

But without success, because is restircted to Level 4.1, which impossible with 50fps according to H264 specs.

Slow down video to 30fps and speed it up after encoding is done ;)

[Midzuki]

You forgot DivX!

I'll do a couple encodes, even though I never ever use DivX :devil:

[edit]
Scratch that... I can't get the VFW encoder to accept the input. It complains "the source image format is not acceptable". I think it may be a profile issue.

What version(s) of DivX have you tried?

[shon3i]

Slow down video to 30fps and speed it up after encoding is done ;)
But not be fair then :)

[Blue_MiSfit]

@Midzuki:

I tried the latest official installer from their website as of today. I've since uninstalled it, but I THINK the VFW codec was version 6.5.2, but don't quote me on that :)

~MiSfit

[Mr VacBob]

http://astrange.ithinksw.net/parkjoy/park_joy_1080p_25.divx
DivX 6.8 ASP 2-pass insane quality + psychovisual masking + multiple B-frames

http://astrange.ithinksw.net/parkjoy/park_joy_1080p_3ivx.mov
3ivx 5.0.2 ASP 2-pass + qpel + B-frames + adaptive quant

http://astrange.ithinksw.net/parkjoy/park_joy_1080p_flv1.flv
http://astrange.ithinksw.net/parkjoy/park_joy_1080p_vp6.flv
Flash CS3's FLV encoder set to Spark and VP6

http://astrange.ithinksw.net/parkjoy/park_joy_1080p_svq1.mov
http://astrange.ithinksw.net/parkjoy/park_joy_1080p_svq3.mov
The free SVQ1/3 encoders in QuickTime with no options

I think there are too many samples now, so drop whatever is least interesting, I guess.

I couldn't get QT's own ASP or H.264 encoders to work - they seem to do gamma changes on everything, but strangely in the opposite direction from each other. And I couldn't get Cinepak or Flip4Mac WMV to meet the bitrate target.

Some of them are at 25fps, either because I couldn't type 13600 kbit into the bitrate box (CS3) or because I forgot to close the 25fps movie afterwards.

[Midzuki]

@ Blue_MiSfit:

FWIW, I have version 6.8.0.14, date = November of 2007. :) But I don't have it installed, it's very "meddlesome". :p Just for testing purposes, I re-encoded a very-short clip @ 1920x1080 @ 50fps with DivX 5.2.1 :devil: , final result = 100% OK, no crashes or complaints from the ****ing .dll. :cool:

[shon3i]

btw, this VC-1 encodes, i don't see they are use max GOP 250?

@Blue_MiSfit, what about your mpeg2 hc encode?

[radiant chains]

btw, this VC-1 encodes, i don't see they are use max GOP 250?

@Blue_MiSfit, what about your mpeg2 hc encode?

Ah, mine does indeed use Max GOP of 250. I've edited my post to reflect that.

[Keiyakusha]

What about Chinese AVS? Is there any encoders/decoders?
EDIT: it seems ffmpeg can decode it.

[Midzuki]

Blue_MiSfit wrote:

nr. of frames in source: 500

media.xiph.org says:

park_joy_1080p.y4m 27-Apr-2009 20:55 1.4G

As I said elsewhere, ppl do have forgotten
what the lossless compressors exist for :-/
OK, "everybody" can download at 30MB per minute or faster,
so why should I be complaining alright.

[CruNcher]

The currently newest available DivX ASP Encoder is 8.2.1.1 aka 6.8.1.00003 coming with DivX Plus Converter



Arcsofts Encoder Resuilt (most probably X264 ???)

as with limited Consumer stuff it did most of the decisions itself it's only 30 fps not 50 Main L4

Arcsofts Cuda Encoder crashed with the Input ;)

http://www.mediafire.com/?gyjwno2miym

[Blue_MiSfit]

@revgen

I don't understand what you're complaining about. I downloaded the y4m file quite quickly, then transcoded to lossless h.264 via x264, and used this as source for all other transcodes, including my HC encode. The GOP length obviously doesn't even approach 250 - after all this is MPEG-2 we're talking about ;) I used the default of 15 in HC.

~MiSfit

[roozhou]

17000KB = 17000 * 1000 Bytes or 17000KB = 17000 * 1024 Bytes?

[Dark Shikari]

17000KB = 17000 * 1000 Bytes or 17000KB = 17000 * 1024 Bytes?Latter.

[Blue_MiSfit]

@shon3i

I used the standard GOP settings for HC. The highest the GUI went was 18 or so, so I just left it at the default.

~MiSfit

[Midzuki]

I used the standard GOP settings for HC. The highest the GUI went was 18 or so, so I just left it at the default.

~MiSfit

HCEnc 0.23 supports up to 36 frames per GOP.

OTOH, the old TMPGEnc Plus has "no limits" for the GOP length. :devil: :cool:

The GOP length obviously doesn't even approach 250 - after all this is MPEG-2 we're talking about

MPEG-2 is much greater than the DVD-Forum specs. ;)

[Midzuki]

@ Stephen Savage:

I've produced several cDVDs in which
GOP-length == 24, 28, 30, or 32 frames,
and B-frames per P-frame == 3 or 4.
Yes, the (lack of) quality sucked, but only because of the
insane low bitrates + insane quantization matrices. :o
I think I've read elsewhere that «52 frames per GOP» is allowed by the VCD specifications, and I suppose that, let's say, 60 frames per GOP wouldn't be too terrible for an MPEG-2 encoding.