Taking submissions for a small encoder comparison [Archive]

Dark Shikari

7th May 2010, 08:34

With the MSU encoder comparison coming out, we're going to have a good comparison (hopefully) of objective quality (SSIM) between various encoders. Unfortunately, not only does their comparison contain basically only H.264 encoders (plus Theora and Xvid), it also doesn't focus on visual comparisons, so I figured I would make my own small one to get a feel for the "current state of the art". This is a visual comparison only; no measurements of SSIM or PSNR will even be made.

Source: parkjoy (http://media.xiph.org/video/derf/y4m/1080p/park_joy_1080p.y4m) (1080p50)
Target file size: 17000KB (can be a bit over or under if the encoder won't reach the exact value, obviously). Using constant quality mode is fine as long as you can get it close to the target size. If the encoder has no 2-pass ABR mode, use constant quality mode (as I did with Dirac).
Keyframe interval: 250

http://img190.imageshack.us/img190/9841/outzd.png

Stuff I already have:

x264 (PSNR-optimized and psy-optimized)
VP7
Theora (Thusnelda and Ptalabvorm)
Dirac (through Schroedinger)
Xvid
ffmpeg mpeg-4
ffmpeg mpeg-1
Bink
SVQ1
SIF1
CudaH264Enc
Ateme (v1.5 and v2.0)
Elecard
Samsung+BBC H.265 proposal
Mainconcept 8.5
Microsoft VC-1 SDK

Stuff I'm interested in:

GPU encoders (e.g. Badaboom)
Any other comedy options you can think of that can hit the target bitrate
VP8 (if you have the encoder for this, please contact me directly and we can arrange for just the decoded output to be uploaded)
Other H.265 proposals?
Reference encoders (JM, JSVM, etc)
Anything else?

To submit a test, just upload the video file along with the settings you used to create it. All encoders should use their "best" settings possible, whatever they happen to be.

In a few days, after I get a few more submissions, I'll post a blind visual test where people can give scores to each codec.

Rationale for the choice of test clip:

1. It's a lossless source. Using pre-compressed sources results in all kinds of stupid arguments about which encoder benefits the most from the compression artifacts already present in the source. It's also free to distribute.

2. It contains a wide variety of high-detail, highly colorful content that should be a serious challenge for any encoder.

3. It shouldn't bias too heavily towards any one encoder like many of the other standard test clips will:
a. It's relatively high motion, so it won't bias heavily against encoders without B-frames or qpel (as, say, mobcal does).
b. It's not so high motion that it would cripple video formats that don't support motion vectors longer than 16 pixels (e.g. Theora).
c. It's not something that benefits an unreasonably large amount from some of x264's algorithms (which is why I picked this and not parkrun).

Overall it should serve as a good, relatively fair test. Ideally I'd want to test more clips, but I don't have that much time and doing psy comparisons is a lot of effort.

About the source:

The source is from the SVT Multiformat (http://media.xiph.org/video/derf/vqeg.its.bldrdoc.gov/HDTV/SVT_MultiFormat/SVT_MultiFormat_v10.pdf) test set (thanks Sveriges Television (http://en.wikipedia.org/wiki/Sveriges_Television)!). It was filmed using an ARRI ArriFlex 765 (http://www.arri.de/camera/65_mm_film/arriflex_765.html) camera on 65mm film using 1/100th second exposure time, hence the staggering level of detail all the way up to 2160p.

morqueozwald

8th May 2010, 01:26

Theora Ptalarbvorm: This was the most recent svn version that affects output, the front end was ffmpeg2theora also the most recent version.
The command line: ffmpeg2theora --optimize -V 13600 --two-pass -K 250 park_joy_1080p.y4m
http://www.mediafire.com/file/qjmnjnzrznn/ParkJoy.ogv

This falls under the "comedy" option.
Quicktime H264 bitrate set to 14200 to hit target size, key frame 250, two pass and no restriction on profile. I first encoded the file with x264 lossless so Quicktime could read it.
http://www.mediafire.com/file/ikynzymzmto/ParkJoy2.mp4

Dark Shikari

8th May 2010, 01:38

AnonCrow

8th May 2010, 03:21

CudaH264Enc through MediaCoder 0.7.3.4640, ABR and VBR (constant quantizer) encodes, Mediacoder preset files included :

http://www.mediafire.com/?sharekey=bf225ce38c0772e5b94117dade8fc2953f70e2d9b9b35f924df0d6082f1c2cd0

... although, the file size produced by the VBR preset varies by a couple KB between encodes.
( brief help on cudah264enc cli options: http://forum.mediacoderhq.com/viewtopic.php?f=2&t=6896#p21699 )

Comedy options:

A M-JPEG encode should always included, either as a comedy option or to make the MPEG-1 encode feel better about itself.
Possibly also MNG, APNG and M-JPEG 2000 encodes to complement the M-JPEG one.

Dark Shikari

8th May 2010, 04:11

CUDAH264Enc added.

shon3i

8th May 2010, 20:38

Ateme (eavc engine 1.5)

Settings used:
TwoPass encoding
Profile@Level: High 5.1
Entropy Coding: CABAC
Quality: Full
Psychovisual: Best
Adaptive Deblock: 0
WeightP
RefP: 5
RefB: 5
Max GOP Size: 250
OpenGOP

http://www.mediafire.com/?yuykyyzjhzm

Elecard (Converter Studio 3.2)

Settings used:
TwoPass encoding
Profile@Level: High 5.1
Entropy Coding: CABAC
Quality: Highest
Psychovisual: Complexity & Luma, strength: 4
Deblock: 0:0
B-pyramid
Ref frames: 16
GOP Length: 250

http://www.mediafire.com/?y4miontkzmn

morqueozwald

8th May 2010, 20:50

The Quicktime video appears to have completely screwed up brightness...

I never noticed this before but this is apparently a very common problem. Finder's information on the file says "Color Profile: HD (1-1-1)" whereas x264 produced files don't say anything (w/r to color) in the info. Could this be responsible?

Dark Shikari

8th May 2010, 21:37

@shon3i

Added.

shon3i

9th May 2010, 01:22

Mainconcept (SDK 8.5 Brodcast encoder)

Settings used:
TwoPass encoding
Profile@Level: High 5.1
Entropy Coding: CABAC
Performance: 15-Best Quality
Psychovisual: Luma & Contrast & Complexity, strength: 100-100-100
Deblock: 0:0
B-pyramid
Ref frames: 16
GOP Length: 250

http://www.mediafire.com/?zvzgn0omion

Tomorow i will encode with Ateme v2 engine encoder. Maybe someone can post Elecard with Converter Studio Pro, because have more advanced options

Dark Shikari

9th May 2010, 01:26

Psychovisual: Luma & Contrast & Complexity, strength: 100-100-100Are you sure this is a good idea? Not only is it insane overkill to combine so many AQ modes like that, but I recall that complexity works on an inverse scale, with negative being a bit like x264's AQ.

Also, the colors are completely wrong on that clip. The way I'm extracting screenshots is using ffmpeg, and even if that doesn't get the right ranges, the range should at least match among all the H.264 encoders. This encode is totally different from all the others.

shon3i

9th May 2010, 01:33

I do some test's before with cinevision, and i am happy with results, now i test a bit, and seem that is best combo.

btw new encode from elecard is on a way, i figure out how to change some options that not visible in gui, if some difference exist i will notice you tomorow.

Dark Shikari

9th May 2010, 01:33

I do some test's before with cinevision, and produce best results, now i test a bit, and seem that is best combo.

btw new encode from elecard is on a way, i figure out how to change some options that not visible in gui, if some difference exist i will notice you tomorow.Can you try fixing the colors at least?

shon3i

9th May 2010, 01:37

which encode? and what is wrong with colors? i didn't notice?

Dark Shikari

9th May 2010, 01:44

which encode? and what is wrong with colors? i didn't notice?x264 (http://mirror05.x264.nl/Dark/Flash/compare/x264.png) mainconcept (http://mirror05.x264.nl/Dark/Flash/compare/mainconcept.png)

Command for extracting both:

ffmpeg -i input.mkv -ss 5.74 -vcodec png -f rawvideo -vframes 1 output.png

shon3i

9th May 2010, 01:47

x264 (http://mirror05.x264.nl/Dark/Flash/compare/x264.png) mainconcept (http://mirror05.x264.nl/Dark/Flash/compare/mainconcept.png)

Command for extracting both:

ffmpeg -i input.mkv -ss 5.74 -vcodec png -f rawvideo -vframes 1 output.png
aha, i think i found problem, since i use directshow encoder, some colorspace converter are pinned in graph between encoder and source. i will now reencode.

shon3i

9th May 2010, 02:34

Link updated colors should now be good. Btw i will play little more with AQ's and i will notice if i made better encode.

Dark Shikari

9th May 2010, 05:57

Link updated colors should now be good. Btw i will play little more with AQ's and i will notice if i made better encode.Please don't. I don't want anyone to specifically tweak the encoders for this particular sample; pick settings that you'd use for all videos.

Biggiesized

9th May 2010, 06:13

Any particular reason to use the clip at 1080p50? Why not do a Blu-ray compatible shoot-out (that is, 720p50)? That way you can throw in Blu-ray encoders if users have access to them.

Dark Shikari

9th May 2010, 06:35

Any particular reason to use the clip at 1080p50? Why not do a Blu-ray compatible shoot-out (that is, 720p50)? That way you can throw in Blu-ray encoders if users have access to them.You can easily use a Blu-ray encoder by setting the framerate to 25 and halving the bitrate ;)

Biggiesized

9th May 2010, 07:28

Can x264 do 1080i MBAFF encoding for Blu-ray?

Dark Shikari

9th May 2010, 07:29

Can x264 do 1080i MBAFF encoding for Blu-ray?Yes, it can.

Schrade

9th May 2010, 08:02

If anyone has access to an Elemental Technologies GPU encoder, I'd love to see that as one of the samples :)

shon3i

9th May 2010, 11:19

Please don't. I don't want anyone to specifically tweak the encoders for this particular sample; pick settings that you'd use for all videos.
Ok then, this encode use max possible settings that directshow encoder can have. Use this encode, it's ok.

shon3i

9th May 2010, 13:39

Ateme (eavc engine 2.2)

Settings used:
TwoPass encoding
Profile@Level: High 5.1
Entropy Coding: CABAC
Quality: Ultra
Psychovisual: Best
Adaptive Deblock: 0
WeightP
Refs: 16
b-pyramid
Max GOP Size: 250
OpenGOP

http://www.mediafire.com/?ycwxmkndywj

TinTime

9th May 2010, 14:32

Here's a VC-1 encode using AVS2ASF (ie. Microsoft's VC-1 encoder SDK).

http://www.mediafire.com/file/2jktjz2agaz/park joy.zip

The settings used are in a text file in the zip.

radiant chains

9th May 2010, 14:38

Microsoft Expression Encoder 3
VC1 Advanced
Preset: "Best Quality"

Average Bitrate: 14100
Max GOP: 250
Video Complexity: Best (5)
Adaptive Dead Zone: Conservative
Differential Quantization: Off
In-loop Filter: On
Overlap Filter: On
Denoise Filter: Off
Noise Edge Removal Filter: Off
B-Frame Number: 1
Adaptive GOP: On
Closed GOP: On
Motion Chroma Search: Full True Chroma
Motion Match Method: Adaptive
Motion Search Range: Adaptive

http://www.mediafire.com/?zmywjtfj3mg

Blue_MiSfit

9th May 2010, 19:44

You forgot DivX!

I'll do a couple encodes, even though I never ever use DivX :devil:

Scratch that... I can't get the VFW encoder to accept the input. It complains "the source image format is not acceptable". I think it may be a profile issue.

Maybe an HC powered MPEG-2 encode?

~MiSfit

Dark Shikari

9th May 2010, 20:13

You forgot DivX!

I'll do a couple encodes, even though I never ever use DivX :devil:Their encoder crashed for me, so I couldn't get it to work. DivX ASP, that is.

DivX H.264 might be interesting.

Blue_MiSfit

9th May 2010, 20:19

Yeah, I was talking about their ASP encoder.

MPEG-2 encode
http://www.mediafire.com/?2ymwilj2mam

---------------------------------------
| HCenc - MPEG2 encoder - rel. 0.24.0 |
---------------------------------------

MPEG profile@level: MP@HL
input: d:\parkjoy_lossless.avs
output: D:\crowdjoy_HC.m2v

--------------------
| encoder settings |
--------------------

profile: BEST
frames: 0 - 499
framerate: 50.000
aspect ratio: 16:9
chroma format: 4:2:0
bitrate Kb/s: 14200
max. bitrate Kb/s: 25000
pulldown: no
closed gops: no
VBV check: yes
scene change det: yes
interlaced: auto, TFF
goplen,B-pic: AUTO
dc_precision: 8
intraVLC: auto
scan method: auto
bias: 0
chapter frames: 0
time code: 0 0 0 0
CPU: SSSE3
priority: idle
SMP active: yes
matrix: MPEG, adaptive
luminance gain: yes, level 2
adaptive quantization: yes, strength 2
deadzone quantization: auto

------------------
| source stats |
------------------

nr. of frames in source: 500
width*height: 1920x1080
fps: 50.000
nr. of frames to encode: 500
frames to encode: 0 - 499

---------------------
| encoding - pass 1 |
---------------------

pass 1 encoding time: 0:02:28 (148.11 s)
fps: 3.4

--------------------------------
| encoding - intermediate pass |
--------------------------------

bitrate set to: 14200 kb/s
estimated output file length: 17334 kB
intermediate encoding time: 0.00 s

---------------------
| encoding - pass 2 |
---------------------

pass 2 encoding time: 0:02:23 (143.05 s)
fps: 3.5

------------------
| encoding stats |
------------------

adapted intra matrix used
adapted non-intra matrix used

nr. of gops: 36
nr. of frames: 500
nr. of I-frames: 36
nr. of P-frames: 215
nr. of B-frames: 249
average bitrate: 13747
minimum bitrate: 10291
maximum bitrate: 17599

bytes in bitstream: 17187491
bits in bitstream: 137499928

average Quantizer: 25.665

total CPU time: 0:06:09 (369.30 s)
total elapsed time: 0:04:57 (297.13 s)

~MiSfit

Atak_Snajpera

9th May 2010, 20:47

you forgot to include "wonderful" ATI encoder:)

Snowknight26

9th May 2010, 22:12

I tried encoding with it but it didn't seem to work with a number of different lossless formats in different containers.

shon3i

9th May 2010, 22:14

DivX H.264 might be interesting. Is not since use same or older MC SDK which i use for current MC encode, and it's very restrictive, i think it's impossible to use GOP 250, and other stuff that can help to in compression.

you forgot to include "wonderful" ATI encoderIs avaible to download. I am currently encoding with Elemental/Badaboom. But without success, because is restircted to Level 4.1, which impossible with 50fps according to H264 specs.

btw Dark Shikari how H265/HVC look :) :D

Anyway one thing here maybe will be significant, that is encoding speed.

mariush

9th May 2010, 22:27

http://www.mediafire.com/?sharekey=437ae816991c299241446e35a78dc46325661133ff7e923f292b492bd5edc68e

Just for fun I tried Tsunami MPEG Encoder 2.5 (2.524) to encode MPG1 streams... found it surprisingly good... visually looked better than the MPEG2 videos it produced (but maybe i didn't it configure it correctly)

There's a version using CBR 13.8 mbps , highest quality motion search, default matrix (different than mpeg standard) and one using CQ mode (constant quality) and if I remember correctly I've set around 15 mbps as maximum bitrate so that it fits in 17000 KB.

Also there a clip encoded with Real Producer 13.1 (latest from their site), Real 10 codec, the rest is in the filename

Also 2 clips encoded with xvid,the one at 10mbps (max value on slider, too bad, could have gone higher as there's still space up to 17000) and a second version set at quant 12.5

The Real 10 version is made from a YV12 Lagarith encoded copy of the movie because it doesn't accept avs scripts directly (without hanging). The xvid version at 10mbps was made from an avs script with AssumeFPS(25) added.

Keiyakusha

9th May 2010, 22:30

But without success, because is restircted to Level 4.1, which impossible with 50fps according to H264 specs.

Slow down video to 30fps and speed it up after encoding is done ;)

Midzuki

9th May 2010, 22:54

You forgot DivX!

I'll do a couple encodes, even though I never ever use DivX :devil:

[edit]
Scratch that... I can't get the VFW encoder to accept the input. It complains "the source image format is not acceptable". I think it may be a profile issue.

What version(s) of DivX have you tried?

shon3i

9th May 2010, 23:03

Slow down video to 30fps and speed it up after encoding is done ;)
But not be fair then :)

Blue_MiSfit

9th May 2010, 23:17

@Midzuki:

I tried the latest official installer from their website as of today. I've since uninstalled it, but I THINK the VFW codec was version 6.5.2, but don't quote me on that :)

~MiSfit

Mr VacBob

10th May 2010, 00:02

http://astrange.ithinksw.net/parkjoy/park_joy_1080p_25.divx
DivX 6.8 ASP 2-pass insane quality + psychovisual masking + multiple B-frames

http://astrange.ithinksw.net/parkjoy/park_joy_1080p_3ivx.mov
3ivx 5.0.2 ASP 2-pass + qpel + B-frames + adaptive quant

http://astrange.ithinksw.net/parkjoy/park_joy_1080p_flv1.flv
http://astrange.ithinksw.net/parkjoy/park_joy_1080p_vp6.flv
Flash CS3's FLV encoder set to Spark and VP6

http://astrange.ithinksw.net/parkjoy/park_joy_1080p_svq1.mov
http://astrange.ithinksw.net/parkjoy/park_joy_1080p_svq3.mov
The free SVQ1/3 encoders in QuickTime with no options

I think there are too many samples now, so drop whatever is least interesting, I guess.

I couldn't get QT's own ASP or H.264 encoders to work - they seem to do gamma changes on everything, but strangely in the opposite direction from each other. And I couldn't get Cinepak or Flip4Mac WMV to meet the bitrate target.

Some of them are at 25fps, either because I couldn't type 13600 kbit into the bitrate box (CS3) or because I forgot to close the 25fps movie afterwards.

Midzuki

10th May 2010, 00:04

@ Blue_MiSfit:

FWIW, I have version 6.8.0.14, date = November of 2007. :) But I don't have it installed, it's very "meddlesome". :p Just for testing purposes, I re-encoded a very-short clip @ 1920x1080 @ 50fps with DivX 5.2.1 :devil: , final result = 100% OK, no crashes or complaints from the ****ing .dll. :cool:

shon3i

10th May 2010, 00:12

btw, this VC-1 encodes, i don't see they are use max GOP 250?

@Blue_MiSfit, what about your mpeg2 hc encode?

radiant chains

10th May 2010, 00:27

btw, this VC-1 encodes, i don't see they are use max GOP 250?

@Blue_MiSfit, what about your mpeg2 hc encode?

Ah, mine does indeed use Max GOP of 250. I've edited my post to reflect that.

Keiyakusha

10th May 2010, 00:32

What about Chinese AVS? Is there any encoders/decoders?
EDIT: it seems ffmpeg can decode it.

Midzuki

10th May 2010, 01:49

Blue_MiSfit wrote:

nr. of frames in source: 500

media.xiph.org says:

park_joy_1080p.y4m 27-Apr-2009 20:55 1.4G

As I said elsewhere, ppl do have forgotten
what the lossless compressors exist for :-/
OK, "everybody" can download at 30MB per minute or faster,
so why should I be complaining alright.

CruNcher

10th May 2010, 03:48

The currently newest available DivX ASP Encoder is 8.2.1.1 aka 6.8.1.00003 coming with DivX Plus Converter

Arcsofts Encoder Resuilt (most probably X264 ???)

as with limited Consumer stuff it did most of the decisions itself it's only 30 fps not 50 Main L4

Arcsofts Cuda Encoder crashed with the Input ;)

http://www.mediafire.com/?gyjwno2miym

Blue_MiSfit

10th May 2010, 07:43

@revgen

I don't understand what you're complaining about. I downloaded the y4m file quite quickly, then transcoded to lossless h.264 via x264, and used this as source for all other transcodes, including my HC encode. The GOP length obviously doesn't even approach 250 - after all this is MPEG-2 we're talking about ;) I used the default of 15 in HC.

~MiSfit

roozhou

10th May 2010, 08:42

17000KB = 17000 * 1000 Bytes or 17000KB = 17000 * 1024 Bytes?

Dark Shikari

10th May 2010, 09:01

17000KB = 17000 * 1000 Bytes or 17000KB = 17000 * 1024 Bytes?Latter.

Blue_MiSfit

11th May 2010, 03:31

@shon3i

I used the standard GOP settings for HC. The highest the GUI went was 18 or so, so I just left it at the default.

~MiSfit

Midzuki

11th May 2010, 04:21

I used the standard GOP settings for HC. The highest the GUI went was 18 or so, so I just left it at the default.

~MiSfit

HCEnc 0.23 supports up to 36 frames per GOP.

OTOH, the old TMPGEnc Plus has "no limits" for the GOP length. :devil: :cool:

The GOP length obviously doesn't even approach 250 - after all this is MPEG-2 we're talking about

MPEG-2 is much greater than the DVD-Forum specs. ;)

Midzuki

11th May 2010, 06:30

@ Stephen Savage:

I've produced several cDVDs in which
GOP-length == 24, 28, 30, or 32 frames,
and B-frames per P-frame == 3 or 4.
Yes, the (lack of) quality sucked, but only because of the
insane low bitrates + insane quantization matrices. :o
I think I've read elsewhere that «52 frames per GOP» is allowed by the VCD specifications, and I suppose that, let's say, 60 frames per GOP wouldn't be too terrible for an MPEG-2 encoding.

Biggiesized

12th May 2010, 00:15

btw, this VC-1 encodes, i don't see they are use max GOP 250?
PSE is limited to 120 frames as the maximum GOP length for custom encodes.

Biggiesized

12th May 2010, 00:25

Can anyone encode via broadcast H.264 encoders (in the EU)? Their bit rate ceiling will be much lower but I'm curious how good it will look.

kieranrk

12th May 2010, 09:25

Can anyone encode via broadcast H.264 encoders (in the EU)? Their bit rate ceiling will be much lower but I'm curious how good it will look.

I doubt many people will have 1080p50 broadcast encoders.

roozhou

12th May 2010, 10:20

Real Video 4 (erv4.dll 10.0.0.2) using Easy RealMedia Producer

EHQ = 100
BFrames = 3

http://www.mediafire.com/?qmmddcnzztv

kolak

13th May 2010, 19:54

Stuff I already have:

x264 (PSNR-optimized and psy-optimized)
VP7
Theora (Thusnelda and Ptalabvorm)
Dirac (through Schroedinger)
Xvid
ffmpeg mpeg-4
ffmpeg mpeg-1
Bink
SVQ1
SIF1
CudaH264Enc
Ateme (v1.5 and v2.0)
Elecard
Samsung+BBC H.265 proposal
Mainconcept 8.5
Microsoft VC-1 SDK

Where can I download these files?

Andrew

Dark Shikari

13th May 2010, 20:10

Nowhere yet. Wait until the results are published.

Atak_Snajpera

13th May 2010, 20:18

Nowhere yet. Wait until the results are published.
Will it be a "blind test"? The similar method how audio compression is judged?

Dark Shikari

13th May 2010, 20:22

Will it be a "blind test"? The similar method how audio compression is judged?Here's the planned instructions.

To avoid skewing the results, the most absurd comedy options will probably be omitted. The rest will be posted for users to view (with no names, of course).

"Comparison" should be done by tabbing back and forth in one's browser.

1. Quickly glance over all the images to get an idea of how the quality varies. This will allow you to calibrate your rating scale--to get an idea of how good "good" is and how bad "bad" is.

2. Go through the list rating each on a scale from 1 to 10, 10 being perfect quality. Fractional values are allowed.

3. Go again through the list, but this time in the order of your ratings (from lowest to highest). Compare each pair of neighboring images in said order. Revise your ratings if necessary. For example, it might turn out that something you rated 5.8 actually looks better than something you rated 6.3 when you put the two side-by-side.

4. If you changed any of your ratings in step 3) in such a fashion that it changed the order of your ratings, check again. Repeat until you settle on an order you agree with.

5. Post the results.

Should take 10-20 minutes, I would guess.

kolak

13th May 2010, 20:24

Nowhere yet. Wait until the results are published.

But I already have video from other encoders.
We're going to have all files anyway as a proof, so it does't matter.

What does decode source file- ffdshow?

Andrew

Dark Shikari

13th May 2010, 20:26

But I already have video from other encoders.
We're going to have all files anyway as a proof, so it does't matter.

What does decode source file- ffdshow?

AndrewI'd rather not post stuff too early because that could skew the results of a blind test if too many people look at it early.

I'm using ffmpeg for decoding whenever possible and Virtualdub for cases of proprietary encoders like SIF1, VP7, etc.

kolak

13th May 2010, 20:28

Here's the planned instructions.

To avoid skewing the results, the most absurd comedy options will probably be omitted. The rest will be posted for users to view (with no names, of course).

"Comparison" should be done by tabbing back and forth in one's browser.

1. Quickly glance over all the images to get an idea of how the quality varies. This will allow you to calibrate your rating scale--to get an idea of how good "good" is and how bad "bad" is.

2. Go through the list rating each on a scale from 1 to 10, 10 being perfect quality. Fractional values are allowed.

3. Go again through the list, but this time in the order of your ratings (from lowest to highest). Compare each pair of neighboring images in said order. Revise your ratings if necessary. For example, it might turn out that something you rated 5.8 actually looks better than something you rated 6.3 when you put the two side-by-side.

4. If you changed any of your ratings in step 3) in such a fashion that it changed the order of your ratings, check again. Repeat until you settle on an order you agree with.

5. Post the results.

Should take 10-20 minutes, I would guess.

There has to be quite few, random frames to avoid I v. B frames comparision. Good thing is to have few frames one after another.
We also need files available to watch them in motion.

Andrew

kolak

13th May 2010, 20:30

I'd rather not post stuff too early because that could skew the results of a blind test if too many people look at it early.

I'm using ffmpeg for decoding whenever possible and Virtualdub for cases of proprietary encoders like SIF1, VP7, etc.

..but how do I decode source file- ffdshow, Vdub?

Andrew

Dark Shikari

13th May 2010, 20:32

..but how do I decode source file- ffdshow, Vdub?

AndrewAs I said, I use ffmpeg to decode the source file, except where no open source decoder is available, in which case I use the one they give me as part of the codec.

There has to be quite few, random frames to avoid I v. B frames comparision. Good thing is to have few frames one after another.
We also need files available to watch them in motion.

AndrewThat would be nice, but also a lot of work, which most people who are doing the comparison won't do, so it won't help the results either.

It certainly won't bias in favor of x264, since I intentionally picked a frame that was a B-frame in x264's encode ;)

kolak

13th May 2010, 20:47

As I said, I use ffmpeg to decode the source file, except where no open source decoder is available, in which case I use the one they give me as part of the codec.

That would be nice, but also a lot of work, which most people who are doing the comparison won't do, so it won't help the results either.

It certainly won't bias in favor of x264, since I intentionally picked a frame that was a B-frame in x264's encode ;)

Frames should be chosen randomly- not according to some (whatever) rules.

No files, no fair comparision, but still would like to see results :)

Andrew

Dark Shikari

13th May 2010, 20:55

Frames should be chosen randomly- not according to some (whatever) rules.

No files, no fair comparision, but still would like to see results :)If you want, I can include the files in the comparison to allow people to choose based on them, but I doubt most people will use them.

And there are reasons to have "rules" for picking frames. For example, don't pick one immediately after an I-frame (biases towards encoders that use overly high-quality I-frames).

kolak

13th May 2010, 21:03

If you want, I can include the files in the comparison to allow people to choose based on them, but I doubt most people will use them.

And there are reasons to have "rules" for picking frames. For example, don't pick one immediately after an I-frame (biases towards encoders that use overly high-quality I-frames).

Exactly- so that's why you choose random frames :)
Different encoders will have different "better" frames, so random choice will average them.

Anyway- lets wait for results.

Andrew

Boolsheet

13th May 2010, 21:04

and Virtualdub
Careful with the yuv->rgb conversions in VirtualDub, it's slightly different than ffmpeg.

shon3i

13th May 2010, 21:05

I think there is no problem with this source since all encoders use 2-3 I frames, so i think comparing will be fair enough

kolak

13th May 2010, 21:09

I think there is no problem with this source since all encoders use 2-3 I frames, so i think comparing will be fair enought

Yes- GOP size is massive and source file very short.

Andrew

Dark Shikari

13th May 2010, 21:11

Careful with the yuv->rgb conversions in VirtualDub, it's slightly different than ffmpeg.Yeah, not sure quite what to do here. ffmpeg is defaulting to BT.601, which is wrong, but it's fine as long as every single clip uses the same wrong conversion, since the difference is pretty minor.

Boolsheet

13th May 2010, 21:37

Yeah, not sure quite what to do here.

You can save the frame to avi in raw yv12. VirtualDub doesn't touch the frame if you set the color depth for input and output to yv12.

poisondeathray

13th May 2010, 21:42

another option would be to use avisynth and avsp to take screenshots decoded as rec709 by using converttorgb(matrix="rec709") . This way everything from back to the original film transfer should have rec709 preserved (assuming the various encoders did things correctly as well)

but I agree as long as it's consistently done it shouldn't matter too much

Atak_Snajpera

13th May 2010, 22:16

I think we should judge clips in motion instead of static screenshots.

Dark Shikari

13th May 2010, 22:17

I think we should judge clips in motion instead of static screenshots.It's far easier to measure small differences when comparing screenshots though... I think the best way is to compare screenshots, but to use motion as a supplement. This allows you to catch cases in which it looks much worse in motion than you expected, e.g. if there's a lot of smearing.

Atak_Snajpera

13th May 2010, 22:35

but to use motion as a supplement.
in flash player?

Dark Shikari

13th May 2010, 22:42

in flash player?No, I'm just going to post them for people to view how they want.

poisondeathray

13th May 2010, 22:42

in flash player?

not all the formats are compatible with flash players

maybe you could do "part A" with screenshots, and "part B" with clips.

but playing back clips might make the "blind" nature difficult, because if the clip extension is ".wmv" or ".rmvb" for example....hmmm I wonder what that is :)

CruNcher

13th May 2010, 22:44

I think the best way is to compare screenshots, but to use motion as a supplement.

But what if one of the Encoder is using Psy that makes especially use of Motion and it's effect isn't visible in 1 screenshot itself ?
I agree with kolak the files need to be released everything else doesn't really matter then, the most important ones btw are already posted here kolak and can be compared already :)
From the H.264 ones people expect to see the difference between Ateme, Mainconcept, Nvidia and X264 though the Ateme Encode is the oldest in terms of the Encoder it was done with.
The subjective best of that compared vs the non MPEG stuff especially Dirac, VP7, pre H.265 and SIF1 where VP7 and pre H.265 obviously are the most important in prediction of VP8 as the possible future MPEG contender and MPEGs own future :)
Also if im not wrong Ateme, Mainconcept same as Elecard and maybe also Nvidia by default use VBV restrictions that don't apply for x264 in it's default configuration.

Atak_Snajpera

13th May 2010, 23:12

but playing back clips might make the "blind" nature difficult, because if the clip extension is ".wmv" or ".rmvb" for example....hmmm I wonder what that is
I think we should create simple app/gui for this. Files would be stored without extensions. Order would be randomized by gui. Clips would be viewed in ffplay for example. Result would be automatically send to email address.

CruNcher

13th May 2010, 23:14

MSU has such a application same as everwicked developed Video Quality Studio http://visumalchemia.com/vqstudio/#download for that purpose though neither of both are Cross Platform and MSUs doesn't support HDTV in the free version and everwicked is rather limited of course non of them has E-mail out function of the Subjective rating results gathered :)

creamyhorror

14th May 2010, 08:50

I think we should create simple app/gui for this. Files would be stored without extensions. Order would be randomized by gui. Clips would be viewed in ffplay for example. Result would be automatically send to email address.
Then for safety you'd have to do an encryption of the files so that their type couldn't be easily checked via MediaInfo, and possibly hide their filesize through concatenation/splitting...lots of things to consider :devil:

How about having two neutral/fair/respected parties transcode all the video files to a lossless format, then distributing the full pack via bittorrent? (The two parties would compare hashes of the transcodes to ensure neither did anything wrong.) Would solve the problem of playback codecs and allow for blind(-ish) testing.

Or we could just leave the videos unblinded and do blind testing only for the screencaps.

julius666

14th May 2010, 12:58

How about having two neutral/fair/respected parties transcode all the video files to a lossless format, then distributing the full pack via bittorrent? Would solve the problem of playback codecs and allow for blind(-ish) testing.

Yeah. The playback speed would be sooo low (we are speaking from lossless FullHD-content!), that it would be like comparing screenshots... :rolleyes:
And I myself don't want to download ~1 GB just for testing codecs.

I think comparing well-chosen screenshots is a good compromise. And if anyone is interested, could download the video after the comparison.

BTW Dark Shikari, is there any codec to H265? I can't wait to see what H265 is capable of :)

poisondeathray

14th May 2010, 14:09

BTW Dark Shikari, is there any codec to H265? I can't wait to see what H265 is capable of :)

DS wrote a bit about a h.265 proposal in his blog
http://x264dev.multimedia.cx/?p=360

Dark Shikari

14th May 2010, 16:19

My test uses the Samsung-BBC proposal. It's not a 100% fair test, since the proposal encoder doesn't have b-adapt and some other features, but on the other hand, Samsung/BBC cheated the crap out of that encoder (using optimizations they weren't supposed to, optimizing specifically for the test clips they were given, etc), so perhaps it's fair game ;)

creamyhorror

15th May 2010, 05:50

Yeah. The playback speed would be sooo low (we are speaking from lossless FullHD-content!), that it would be like comparing screenshots... :rolleyes:
Huh? Lossless doesn't mean slow to decode. If anything, it'd likely be faster to decode than a lossy x264 encode of the same material.

And I myself don't want to download ~1 GB just for testing codecs.
1GB isn't much, compared to the x264 Blu-ray.

I think comparing well-chosen screenshots is a good compromise. And if anyone is interested, could download the video after the comparison.
Of course. Chosen screenshots are the essential requirement; this video stuff I was referring to is an additional part.

Dark Shikari

15th May 2010, 06:02

Huh? Lossless doesn't mean slow to decode. If anything, it'd likely be faster to decode than a lossy x264 encode of the same material.Lossless x264 is not fast to decode. Combine that with the disk bottleneck...

And it might be 500MB... per encode. That's a lot of gigabytes to download for all the different sources.

creamyhorror

15th May 2010, 10:36

Lossless x264 is not fast to decode. Combine that with the disk bottleneck...
I'm referring to HuffYUV or maybe Ut. But the disk bottleneck might apply, admittedly.

And it might be 500MB... per encode. That's a lot of gigabytes to download for all the different sources.
Ouch, okay.

IgorC

15th May 2010, 16:56

Maybe I forget something. But what are reasons and advantages to choose 50fps source?

Dark Shikari

15th May 2010, 18:12

Maybe I forget something. But what are reasons and advantages to choose 50fps source?All the SVT clips are high-framerate. If you want it to be 25fps, it can be 25fps, just slow it down 2x ;)

KikeG

15th May 2010, 21:57

I know that you know all of this and I don't know as much as you of video encoding, but wouldn't comparing still frames not take into account that the some encoding schemes take advantage of the fact that our eye is less sensitive to artifacts when there is motion? I mean, the already mentioned B-frames issue for example. I don't know much in x264, but with XVid default parameters isolated B-frames look much worse than I or P frames. Also, some codecs take into account the amount of motion for compressing more or less the frame, and others don't. So depending if you compare high motion or low motion frames this will benefit some encoders or others, benefit that may not be so when looking at the actual moving video. I too believe some artifacts (moving textures, etc) are more visible with moving video than with still frames.

I think that if despite all this the comparison is based in still frames, several different frames, high and low motion, should be used, in order to take into account these issues, and the result for each encoder show be the average result of the different frames.

Also, for blind tests it is always needed at least a control or anchor encoder. This is, a encoder whose quality is known and clearly worse or better than the encoders at test, so that differences between encoders can be made relative to the anchors. Usually a low anchor is employed.

CruNcher

15th May 2010, 23:28

Yep and as the heaviest contenders in the H.264 range here are very heavy developed it will be hard to see differences per 1 test sequence better would be to compare a whole encode versus each other with different complexity stages. I know test sequences x264 and Mainconcept would heavily lose vs Ateme for example too so im not sure if it's good to test such heavy R&D Encoders with just 1 sequence it give a somewhat picture of the state their in but not a complete overview. For example Mainconcept lost some years ago heavily in the ParkRun sample and also x264 recently after that started to optimize for such complexity cases Ateme @ that time already did that before both of them and won visually by a very big margin ;) (i mean would it have been fair comparing x264 this way???). Also comparing Psy (Look & Feel) is hardly possible with just 1 sequence especially not in screenshot differences imho.
I don't think that picking every small detail of difference out and saying here x264 does it visually better is a fair way and it surely shouldn't be sold as such, but combined with MSUs results for example which this time should be much better worked out it can give a nice overall picture, though Ateme isn't participating again i think so 1 major result would still be missing. Anyway for normal average Consumer this comparison is also rather useless as their is no real up2date contender to X264 in that space, except maybe currently the DivX Plus-HD Converter ;)

[ReX]

15th May 2010, 23:57

You could take a screenshot of each video every 2secs (or 1.2, 1.5, etc).

CruNcher

16th May 2010, 00:27

Do you think Hollywood compared solutions this way when they where examining HD-DVD, Blu-Ray participants ? ;)

Atak_Snajpera

16th May 2010, 12:12

At the moment x264 has only one competition Atheme V2. Rest is horrible!
http://img268.imageshack.us/img268/9611/stackhorizontalavssnaps.png

Dark Shikari

16th May 2010, 12:26

At the moment x264 has only one competition Atheme V2. Rest is horrible!
http://img268.imageshack.us/img268/9611/stackhorizontalavssnaps.pngIMO Mainconcept 8.5 stands up rather well too. Too bad you can't actually find any products using it.

Atak_Snajpera

16th May 2010, 12:31

x264 is also amazing in intra-only mode. Original vs x264 intra (~250KB) vs mjpeg (~250KB)
http://img22.imageshack.us/img22/7835/new1pe.png (http://img22.imageshack.us/i/new1pe.png/)

Uploaded with ImageShack.us (http://imageshack.us)

CruNcher

16th May 2010, 12:53

CruNcher

16th May 2010, 12:56

IMO Mainconcept 8.5 stands up rather well too. Too bad you can't actually find any products using it.

DivX Plus-HD Converter uses it since some time now, it seems to be a default 2 pass system because the pre encode stage takes a long time you see a message "Is preparing" and CPU utilization @ 100% (seems to be a analyze pass taking place). So it takes a very long time for the result compared to a well balanced X264 CRF encode :)

gonna upload the default result, that consumers can expect :)

DivX Plus HD Converter result (everything default just pushed start)

http://www.mediafire.com/download.php?wjzmnzndwoh

Mediainfo result:

Allgemein
Vollständiger Name : C:\Dokumente und Einstellungen\Administrator\Eigene Dateien\Eigene Videos\DivX Movies\parkjoy.mkv
Format : Matroska
Dateigröße : 17,0 MiB
Dauer : 9s 960ms
Gesamte Bitrate : 14,3 Mbps
Kodierendes Programm : DivXMKVMux 3.4.1.0004
verwendete Encoder-Bibliothek : libDivXMediaFormat 3.4.1.0004

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format-Profil : High@L4.0
Format-Einstellungen für CABAC : Ja
Format-Einstellungen für ReFrame : 4 frames
Muxing-Modus : Container profile=Unknown@4.0
Codec-ID : V_MPEG4/ISO/AVC
Dauer : 9s 960ms
Bitrate : 14,0 Mbps
Breite : 1 280 Pixel
Höhe : 720 Pixel
Bildseitenverhältnis : 16:9
Bildwiederholungsrate : 50,000 FPS
Standard : PAL
Auflösung : 8 bits
Colorimetrie : 4:2:0
Scantyp : progressiv
Bits/(Pixel*Frame) : 0.305
Stream-Größe : 16,7 MiB (98%)
Sprache : Englisch
colour_primaries : BT.709-5, BT.1361, IEC 61966-2-4, SMPTE RP177
transfer_characteristics : BT.709-5, BT.1361
matrix_coefficients : BT.709-5, BT.1361, IEC 61966-2-4 709, SMPTE RP177

So as Dark_Shikari said as 1080p 50 fps is out of DivX Plus HD specs it changed the output resolution by default to the next supported that is 1280x720p 50 fps

Dark Shikari

16th May 2010, 13:10

DivX Plus-HD Converter uses it since some time now with a default complexity masking setting
Then why does it suck so much?

Atak_Snajpera

16th May 2010, 13:36

except the blocking yeah
I always decompress footage from AVCHD camcorder using --preset superfast --tune fastdecode --keyint 1. I need deinterlaced footage which could be easly (lower cpu utilization) decoded by Sony Vegas. That's why I cannot use deblock and cabac for example. if you see blocks in x264 you better check what you get in mjpeg. Also don't forget that this is 200% zoom.

CruNcher

16th May 2010, 14:43

Yeah but a setting of -f -1 wouldn't have been wrong i would say for your case anyways :)

here is the DivX Plus HD Converter Result in Spec of 1080p 25 fps :)

http://www.mediafire.com/download.php?mzynmnnjj0d

Allgemein
Vollständiger Name : C:\Dokumente und Einstellungen\Administrator\Eigene Dateien\Eigene Videos\DivX Movies\parkjoy(4).mkv
Format : Matroska
Dateigröße : 17,0 MiB
Dauer : 19s 960ms
Gesamte Bitrate : 7 146 Kbps
Kodierendes Programm : DivXMKVMux 3.4.1.0004
verwendete Encoder-Bibliothek : libDivXMediaFormat 3.4.1.0004

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format-Profil : High@L4.0
Format-Einstellungen für CABAC : Ja
Format-Einstellungen für ReFrame : 3 frames
Muxing-Modus : Container profile=Unknown@4.0
Codec-ID : V_MPEG4/ISO/AVC
Dauer : 19s 960ms
Bitrate : 7 004 Kbps
Breite : 1 920 Pixel
Höhe : 1 080 Pixel
Bildseitenverhältnis : 16:9
Bildwiederholungsrate : 25,000 FPS
Standard : PAL
Auflösung : 8 bits
Colorimetrie : 4:2:0
Scantyp : progressiv
Bits/(Pixel*Frame) : 0.135
Stream-Größe : 16,7 MiB (98%)
Sprache : Englisch
colour_primaries : BT.709-5, BT.1361, IEC 61966-2-4, SMPTE RP177
transfer_characteristics : BT.709-5, BT.1361
matrix_coefficients : BT.709-5, BT.1361, IEC 61966-2-4 709, SMPTE RP177

Still the TV scale stays in the Conversion Process.

Hmm why not compare that with the old Consumer Nero Recode (Ateme) ;) result coming :D (in a 2 pass of course like DivX does it)

It's crazy how complex Consumer stuff feels for People who know exactly what they want ;) (you really get lost without a CLI with all the option naming and stages of encoding they put in this stuff that also seem to many degrees totally useless).

poisondeathray

16th May 2010, 15:05

I know it was suggested earlier, and not to derail the thread, but I think a blu-ray comparison should eventually be done and be relevant for more people (than 1080p50). It could be done on this sequence with PAL slowdown

e.g.
SelectOdd() #or SelectEven()
AssumeFPS(24000,1001)

shon3i

16th May 2010, 15:25

CruNcher

16th May 2010, 15:32

maybe Arcsofts Cuda Encoder wouldn't also crash that way ;)
I find recreating the X264 Demo Blu-Ray with each contender would be even a much better idea (especially as it is a very restricted encoding scenario where every encoder needs to be very well optimized) :)

Atak_Snajpera

16th May 2010, 15:40

And more H264 encoders can participate such as Sony, CCE HD.
I doubt they have any chances to look at least as good as Ateme V2.
Btw. This is how i compare samples http://www.mediafire.com/file/mzamgrobjgd/Samples.7z

x264 vs DivX Plus HD Converter
http://img291.imageshack.us/img291/9611/stackhorizontalavssnaps.png

Those blocks on tree remind me of Quake 1 without 3dfx card :)

poisondeathray

16th May 2010, 16:13

I find recreating the X264 Demo Blu-Ray with each contender would be even a much better idea (especially as it is a very restricted encoding scenario where every encoder needs to be very well optimized) :)

I think that's a good idea. IMO, the more tests and data points, the better. As you said earlier , a single test is far from conclusive, and care must be taken not to broadly generalize the results.

Most consumer camcorders don't shoot 24pN , they shoot 60i or 50i. Most people won't have access to film transfers or 24pN professionally shot content. As much as I personally hate interlacing, it's a common format and I think adding a comparison of interlaced blu-ray efficiency would also be relevant.

Anyways, lets finish this thread first, then maybe start a new one for other comparisons

CruNcher

16th May 2010, 16:35

Here is one more fun Encode hehe

Arcsofts (own no Nvidia Core) Cuda Encoder (its fast but you gonna see why ;) )

http://www.mediafire.com/download.php?dyzy24yzhmf

Atak_Snajpera

16th May 2010, 16:50

Arcsofts Cuda Encoder (its fast but you gonna see why )
epic fail :) this peace of s**t is even worse than mpeg2 :) Also cuda sample uses more bits 18mbps vs 14mbps.

CruNcher

16th May 2010, 17:03

yep i guess thinks like no CABAC awful SUBME low quality motion search paired with a bad RC play into this result :P

btw here the same using a CQ of 42 it hit the bitrate much better that way ;) (though visually you still get a blockfest)
http://www.mediafire.com/download.php?22j5jw334wk

Atak_Snajpera

16th May 2010, 17:16

I must say that this sample is absolutely killer for almost all codecs. Only x264 and ateme v2 survived :) Once again x264 is work of art. I didn't expect such huge difference at the beginning :eek:

I can't wait to see Ati's encoder result :)

CruNcher

16th May 2010, 17:28

yep it's even more extreme then Parkrun already was, more motion more details more colors and nicer overall location (tod) :D

Atak_Snajpera

16th May 2010, 17:49

it clearly separates boy from men :)

RunningSkittle

16th May 2010, 19:51

tried several times to get ati encoder to work, fails with every type of input :(

shon3i

16th May 2010, 20:00

I doubt they have any chances to look at least as good as Ateme V2. MC 8.5 by hair does better job on some frames, but that is my taste ;p, anyway CCE HD and Sony encoder are not another encodres in try, they are realy efficient, but are limited to L4.1. I think that this test will be probably more interesting because most of H264 encoder are limited to L4.1 even GPU ones (Badaboom)

We can do this latter ofc :)

mariush

16th May 2010, 22:29

tried several times to get ati encoder to work, fails with every type of input :(

I got it to work but with 25fps... basically just wrote in the avs script assumefps(25).

And as a note, it's the 10.4 avivo encoder version but with an old Control Panel, I believe 10.1 or even older ( it says Catalyst™ Control Center Version 2009.0925.1707.28889) , because I can't upgrade the CCC anymore on Windows 2003 (due to the stupid installer they use now).

The encoder and the driver I can install by running the msi files or using the "update driver" in the Control Panel.

So I don't know if the encoder looks differently now, I see only the basic select source, choose 1080p h264 and bitrate between 8 and 25 mbps.

On the link below, it's h264 and mpeg2 versions, titles say it all.
It's almost impossible to get it to 17000 KB though.

http://www.mediafire.com/?sharekey=437ae816991c299241446e35a78dc4638eddb5f1ab1d0cf6f85d402b9fc3640c

All files should pop up there in about 15 minutes from this post - they're being uploaded as I write this

Boolsheet

16th May 2010, 22:31

tried several times to get ati encoder to work, fails with every type of input :(
Huh, you mean the Avivo thing right?
Works here on Windows XP 32 bit, Radeon HD 4870, Catalyst 10.4 and a VfW codec (UT Video).
The encoder doesn't like 1080p50 at all. The stream gets converted to 1920x2160 with a funky aspect ratio of 2:1 and every odd frame is gone, so it's back to 25 fps.

Slowing it down to 25 helps, size is still a little off: 17842 KiB (http://www.mediafire.com/file/yvjmzwczqgw/park_joy_1080p25_ati_avivo_enc_4870.mp4)
I let it encode a 3 minute loop of park joy and cut out a piece that hit the size: 16949 KiB (http://www.mediafire.com/?41memwetynw) (note: fps set manually to 50)

No CABAC, GOP size of 30, no B frames... I think i'm configuring this wrong.
Can someone help me get that slider in the right position? ;)
http://img100.imageshack.us/img100/551/atienc1.th.png (http://img100.imageshack.us/img100/551/atienc1.png)

I didn't dig around the registry to find more settings and a quick search with google suggested nothing that it could be tweaked.

CruNcher

16th May 2010, 22:53

Didn't the Ati traytools guy coded something that you could directly access the Encoder with his own GUI :) ? (more settings)
on the other side you could try another Stream Encoder or someone that makes use of Atis Encoder API it im not sure though what Arcsofts MediaConverter Platinium is but it supports Cuda as well as Ati Stream, so get the trial http://www.arcsoft.com/estore/software_title.asp?ProductCode=AMC4 and try that too, but if they made it as fast and with the same goal as the Cuda Encoder it wont be very good @ all ;) Another Consumer one with Stream support would be http://www.cyberlink.com/downloads/trials/mediashow-espresso/requirements_en_US.html

kolak

16th May 2010, 23:04

I doubt they have any chances to look at least as good as Ateme V2.
Btw. This is how i compare samples http://www.mediafire.com/file/mzamgrobjgd/Samples.7z

x264 vs DivX Plus HD Converter
http://img291.imageshack.us/img291/9611/stackhorizontalavssnaps.png

Those blocks on tree remind me of Quake 1 without 3dfx card :)

Not <15Mbit, but pass 20Mbit it's a bit different story.

Andrew

Atak_Snajpera

16th May 2010, 23:24

Arcsofts MediaConverter Platinium is but it supports Cuda as well as Ati Stream, so get the trial http://www.arcsoft.com/estore/softwa...oductCode=AMC4
What a peace of s**t! I had to encode first to lossless avi. Maximum bitrate is 10mbps (PS3 high quality :) It was a waste of time. All gpu encoders are some kind of sick joke. Stupid GUIs and terrible quality.

Not <15Mbit, but pass 20Mbit it's a bit different story.
MP2 audio is also good at high bitrates and so what? Compression is supposed to reduce file size and maintain quality. Somehow x264 and atemev2 is able to do good job at 15Mbps.

Boolsheet

17th May 2010, 00:27

ATI Xcoder GUI by Ray Adams has indeed more options, but none of them really change anything. Could be hardware specific.
The other GUIs. Yeah, Atak_Snajpera said it. :)

kolak

17th May 2010, 00:32

What a peace of s**t! I had to encode first to lossless avi. Maximum bitrate is 10mbps (PS3 high quality :) It was a waste of time. All gpu encoders are some kind of sick joke. Stupid GUIs and terrible quality.

MP2 audio is also good at high bitrates and so what? Compression is supposed to reduce file size and maintain quality. Somehow x264 and atemev2 is able to do good job at 15Mbps.

Yes, but try to use F1 car in the city. Some things are design for specific use- x264, Ateme are very good at low bitrates, because this is what they were designed for. BD encoding is different- it's all about transparency with the source, not to achieve watchable video at the smallest file size.
x264 can be also tweaked for this and than it would definitely be ahead of any other BD encoder. For now it's very close, but not a clear winner (as for low bitrates).
Another thing is that BD encoders are limited compared to settings use in this test, so forget about them during this test.

Andrew

mariush

17th May 2010, 02:19

I don't even think it's worth uploading Cyberlink's MediaShow result... it has absolutely NO settings, you can just choose between 720x480, 1440x1080, 1920x1080 and another one.
The bitrates are hard coded and can't be changed - in the 1920x1080's case it's locked at 13 mbps. It does look like it uses ATI Stream but maybe just to decode the source clip (so the answer would be no, because it's a 50fps lagarith encoded park joy), because in the Control Center the GPU usage is 6% (on Radeon 4850).

The result is a M2TS file, 37 MB, at 54.940059 fps, INTERLACED. Ridiculous...

ok , here's the file: http://www.mediafire.com/?gqxgmzibz0e

creamyhorror

17th May 2010, 04:47

tetsuo55

17th May 2010, 06:52

I agree with creamyhorror, scientifically correct research/testing cares about all the results, not only the good ones.

aegisofrime

17th May 2010, 09:40

From my point of view, even bad results should be included, so people know just how bad they are
(especially when they're googling for that encoder to see if it's acceptable). It's good to test as wide a range of common encoders as practically possible, since you don't want to leave the exaggerated claims of lousy products unexamined.

Totally agree. I also hope that somebody will do a speed and quality test of x264 and Badaboom. I am seriously sick of arguing with people who think that Badaboom is the best thing since sliced bread.

Atak_Snajpera

17th May 2010, 13:36

Totally agree. I also hope that somebody will do a speed and quality test of x264 and Badaboom. I am seriously sick of arguing with people who think that Badaboom is the best thing since sliced bread.
I agree.

Dark Shikari

17th May 2010, 14:47

I'm only on planning to omit the ones that are pure comedy, e.g. Bink, since they will ruin the 0-10 point scale ;)

Anything H.264 is not likely to be bad enough to be that laughable.

How about giving me a Badaboom sample so I can use it? :)

Atak_Snajpera

17th May 2010, 14:52

I see no reason to use 0-10 scale. For me: x264 - excellent , atemev2 - very good , rest - awful. End of story.

shon3i

17th May 2010, 17:18

How about giving me a Badaboom sample so I can use it? Level 4.1 30fps? I can send you right now.

Dark Shikari

17th May 2010, 17:27

Level 4.1 30fps? I can send you right now.30? Just use 25 and halve the bitrate.

kolak

17th May 2010, 20:11

30? Just use 25 and halve the bitrate.

Is half bitrate fair?
I would expect that it should be smaller reduction- maybe 30%?

Andrew

MasterNobody

17th May 2010, 20:57

Is half bitrate fair?
I would expect that it should be smaller reduction- maybe 30%?

Andrew
All fair while resulting file is near 17000KB. It doesn't matter at what speed frames played. So half fps mean half bitrate because it course the sample to be two times longer.

kolak

17th May 2010, 21:56

All fair while resulting file is near 17000KB. It doesn't matter at what speed frames played. So half fps mean half bitrate because it course the sample to be two times longer.

I've removed half of the frames, not assumed 25fps.

Andrew

Dark Shikari

17th May 2010, 21:58

I've removed half of the frames, not assumed 25fps.

AndrewDon't do that. Just assume 25fps.

Any removing of frames will not give a fair result.

Blue_MiSfit

17th May 2010, 22:02

Yup. You can just reset the frame rate again when muxing :)

~MiSfit

shon3i

18th May 2010, 07:17

Badaboom 1.2.1.74

Settings used:
Assumed 25fps
RC: VBR
Profile@Level: Main@4.1
Entropy Coding: CABAC
Max GOP Size: 250

http://www.mediafire.com/?ioo1mzmr2ni

aegisofrime

18th May 2010, 16:33

Noob question warning! I have a feeling its going to be asked sooner or later so I will just ask it now.

How do I play .264 files without having to mux it to mkv first? All google tells me is how to play H.264 files.

Midzuki

18th May 2010, 16:57

How do I play .264 files without having to mux it to mkv first?

MPlayer.

aegisofrime

19th May 2010, 13:25

MPlayer.

Thanks! I grabbed Lord Mulder's package and it works great!

Keiyakusha

19th May 2010, 20:13

So, judging by some nice pictures that DS posted somewhere, we already can see that x264 beats most of the participants. Isn't it?

Gser

19th May 2010, 20:51

Gonna do an encode with Elecard Converter Studio as soon as I finish downloading the video file. 1mbps connection ftl :(

shon3i

19th May 2010, 21:14

Gonna do an encode with Elecard Converter Studio as soon as I finish downloading the video file. 1mbps connection ftl :(
I alredy done encode of Elecard.

Neiromaster

21st May 2010, 01:53

Avi file:
http://mysif.ru/Avi/park_joy_sif1.avi

File size 16810 kilobytes.

Compression script:
RawSource("park_joy_1080p.y4m")
AddBorders(0,0,0,8)

Compression settings:
Single pass quality based
Visual distortion: 74
Maximum I-frame interval: 250
Color quantization scaling: 34
Additional quantizer: 64

It is interesting to compare with VP8 :)

Gser

22nd May 2010, 14:47

How about Sorenson h.264?

Biggiesized

22nd May 2010, 17:29

Doesn't Sorenson use the MainConcept SDK?

Gser

22nd May 2010, 17:40

Doesn't Sorenson use the MainConcept SDK?

* Main Concept H.264, VC-1, MPEG-2
* On2 VP6 Pro
* Sorenson H.264 Pro
* Sorenson MPEG-4 Pro (for Visual Simple Profiles and Visual Profile encoding)
* Sorenson Spark Pro
* QuickTime Encoders
* Real Media Encoders

Don't know what their codec is based on though.

poisondeathray

23rd May 2010, 22:41

QT gamma shift "workaround"

Quicktime Pro 7.6.5 on Windows (7.6.6 is known to be buggy)
keyframes every 250
best quality (2pass)

http://www.mediafire.com/?mjyz3wqttyy

Most people don't care for QT, but for anyone interested:

one workaround for the dreaded "quicktime gammashift bug" is to use uncompressed avi with the fourcc "IYUV" . There is one that you use for free with "Drastic Codecs" (google it) . For some reason, some other fourcc uncompressed Y'CbCr video can elicit the shift

morqueozwald

24th May 2010, 00:26

Quicktime 7.5.5 Windows encode without gamma shift I made a while back, which comes closer to target file size than previous post. Used same methodology to get uncompressed into Quicktime.

http://www.mediafire.com/file/2ym5yzzugmm/QTPJ.mp4

Settings used:
Keyframe: 250 frames
Bitrate 14200
Multipass encoding

creamyhorror

24th May 2010, 04:45

QT gamma shift "workaround"
Since Quicktime's gamma shift bug is still not fixed and isn't that easy to get around, shouldn't Quicktime be penalised some way in the test? E.g. by leaving the gamma shift in and making the difference clear in the screenshot comparisons. You'd want to bring more awareness of the bug, after all, since it definitely affects quality.

Tudi

26th May 2010, 09:52

Day 1 noob posting some proprietary encoder results :). Xennon encoders are made for live encoding for mobile streaming. They are not optimized for quality only. Since 3gp standard is pretty restrictive only h264 baseline profile is supported :
MPEG4 normal : http://www.mediafire.com/?y2ltdm0y2gw
MPEG4 sharp : http://www.mediafire.com/?5vunmrdmzqc
X264 : http://www.mediafire.com/?mdyzwyydjhm

Thanks for making this collection of encoders. It really helped me make a decision.

Sagittaire

26th May 2010, 16:30

I have really better result here for Ateme with 2008 core

poisondeathray

26th May 2010, 16:48

I have really better result here for Ateme with 2008 core

why don't you upload it and add it to the comparison?

CruNcher

29th May 2010, 02:17

why don't you upload it and add it to the comparison?

I guess NDA doesn't allow him too

shon3i

30th May 2010, 17:47

@Sagittaire encoder used here is same as you have. its same generation.

CruNcher

30th May 2010, 19:37

It's not the same, or does your encoder have a INSANE mode ;)

shon3i

30th May 2010, 19:58

have, but i didn't manage to use it with 2pass RC. Some options are locked for single pass/VBR mode only. Since i aim to target bitrate/size, i use max possible settings for 2pass. Anyway difference between Insane and Ultra is real placebo, and waste of time.

julius666

5th June 2010, 11:32

when will the comparison start?

Dark Shikari

5th June 2010, 11:45

when will the comparison start?Sorry; been unexpectedly busy with work and some other things. I'll try to get back to it soon.

EricJ2190

16th June 2010, 17:57

Do you already have a Microsoft Expression Encoder Pro 3 H.264 sample? I don't see one in this thread, so I am working on one myself.

I'm done encoding. For a two-pass encoder, it doesn't do a very good job of maintaining the desired average bitrate.

http://www.mediafire.com/?ydgnetqrejo - 17,353,687 bytes

Settings
Output Format: MP4
Video: H.264 Main
Audio: Disabled
Mode: VBR Unconstrained
Frame Rate: Source
Key Frame Interval: 5 seconds
Average Bitrate: 15000 Kbps
Size Mode: Source
Width: 1920
Height: 1080
Video Aspect Ratio: Source
Force Square Pixels: Disabled
Resize Mode: Letterbox

H.264 Settings
Complexity: Best (5)
Number Of Reference Frames: 15
B-Frame Number: 7
Search Range: 64
Entropy Mode: CABAC
ME Partition Level: 4x4
Threads Used: Auto
Force 16 pixel dimensions: Disabled
In-Loop Filter: Enabled
RD Optimization: Enabled

filetree

19th June 2010, 03:41

The Quicktime video appears to have completely screwed up brightness...

Ptalabvorm added.

It's actually a gamma thing. There's a decent workaround (that requires QTPro) but it's still []. It sucks that there are a lot of good broadcast/live hardware that will only allow video in .mov format.
Unless I'm mistaken x264 cannot be held in an mov container.

What about Sorenson3 codec? As old as it is, it is sadly still being used a lot of places.

nakTT

19th June 2010, 19:06

Hi Dark,

Can we expect to see SIF1 v1.10 in the comparison?

:thanks:

hellfred

19th June 2010, 22:29

I have created an up to date theora ptalarbvorm clip:
park_joy_V13600_softtarget.ogv (http://www.mediafire.com/file/jtldzjk2gdn/park_joy_V13600_softtarget.ogv)
command used:
ffmpeg2theora-ptalarbvorm.exe --no-skeleton --speedlevel 0 --soft-target -K 250 -V 13600 --two-pass --noaudio park_joy_1080p.y4m
Binary from:
http://lists.xiph.org/pipermail/theora-dev/2010-June/004164.html
Changes to the theora encoder, which might have had an impact on the video quality since the first theora ptalarbvorm clip provided at page #1:
18.05.2010: https://trac.xiph.org/changeset/17231/experimental/derf/theora-ptalarbvorm
04.06.2010: https://trac.xiph.org/changeset/17275/experimental/derf/theora-ptalarbvorm