Doom9's "Codec shoot-out 2004" test results are out!

[sbp]

Thanks for the reply.

I'm sure that the codecs have come a long way since 2001. But it seems from your shoot-out in 2003 that the codecs got slower in 2003 compared to 2002?
I quote you here from the 2003 comparison:
"The slowest codec last time (WMV9, 17.43 fps) would now be in the faster half. RV9 obviously got slower because of the EHQ mode, but DivX5's speed drop is beyond me. I even tested it twice because I wouldn't believe my results. I'm waiting to hear from DXN on the speed subject (encoding was faster on my 2.8 GHz P4 notebook, but still slower than every other codec). I was positively surprised by XviD, which once again is very presentable in the speed area. ND blew everything else out of the water."

So it seems that the codecs didn't get faster - at least in 2003.

So how is your feeling this year, are they faster now - and especially Divx, which last year were pretty slow (5.1.1)?

I haven't been able to read anything about the speed differences between XVID 1.0 and 1.1 in one pass encoding - does anybody have information on this issue?

Steen

[babayaga]

Originally posted by Doom9
I've also managed to dig up some very old codec comparisons from the year 2001. here's the link: http://forum.doom9.org/old-site.rar
Thanks !!
Your comparisons are consistant for almost 4 years (I guess the 1st shoot-out was done in march or april 2001) : same kind of movies and same protocol. This gives a pretty clear idea of the fast codec evolution :)

[Doom9]

So how is your feeling this year, are they faster now - and especially Divx, which last year were pretty slow (5.1.1)?Look at the DivX settings for the last two years now.. slowest mode and insane mode.. they just can't be fast that way. But it's really a matter of settings.

[babayaga]

Originally posted by sbp
So it seems that the codecs didn't get faster - at least in 2003.

It went almost unnoticed but we did release an impoved version of our MPEG-4 ASP codec in in june 2004.
This version did add the new 1st pass concept that make a 2 passes encoding almost as fast as a one pass encoding on a whole movie.

Unfortunately, the idea came during the christmas season just when Doom9 was making its review and I was testing it when the shoot-out results were published :(

Besides, this version also adds quality improvements that makes it on par with XviD on the detail level, which was the goal.

[Doom9]

Besides, this version also adds quality improvements that makes it on par with XviD on the detail level, which was the goal.Let's look at that in a future codec comparison, shall we? :devil:

[babayaga]

Originally posted by Doom9
Let's look at that in a future codec comparison, shall we? :devil:

With great pleasure :D

[Daniel_HDX4]

Originally posted by eb
What is possibility to get HDX4 1.4 codec for tests with live recording from sattv?


Hi,

we hope to release HDX4 in two or three weeks. Currently we are working on solutions for hardware devices (player) and we have to complete this first...

-Daniel

[bond]

Welcome to doom9!
its always nice to see representatives of a/v implementations on the board :)

Originally posted by Daniel_HDX4
Currently we are working on solutions for hardware devices (player) and we have to complete this first...what exactly does this mean? i saw that on your webpage you seem to offer both .avi and .mp4 output options for your mpeg-4 encoder? is that true?

[SeeMoreDigital]

Hi Daniel and welcome ;)

I hope you've prepared yourself to answer some tough questions?


Cheers

[708145]

Originally posted by babayaga


- sumo : like the football championships but with a reduced set of opponents choosen basically on their rank. There are individual matches at the end in case of ties. The set of player is well sorted but not perfectly though. It further permit to a badly ranked player to finish first in a much more fair way than in tennis.

I prefer the sumo way since it gives a sorted set with a reduced number of individual matches : for 10 codecs, this leads to about 25 individual comparisons, 4h for a single reviewer needing 10mn for each match. This method relies on transitivity (if A is better than B and B better than C then A is better than C) which is highly desirable.

[long explanation skipped]

ok, it's too complex, but I love the idea :D

I prefer the sumo method as well. I don't think it's too complex, maybe it's even simpler than the current scheme doom9 used to compare all codecs. (I boldly assume he compared every codec with every other which might not be the case.)

bis besser,
Tobias

[Fl0ppy]

Thanks for the test, of course xvid r00lez :-D
But i miss the mpeg4 of libavcodec (ffmpeg project) :rolleyes:

[Daniel_HDX4]

Originally posted by bond
i saw that on your webpage you seem to offer both .avi and .mp4 output options for your mpeg-4 encoder? is that true?

Hi,

thanks for your warm welcome. :)

Well, our homepage is not online yet. If you have already a deep link, it's very old and hasn't changed since several weeks (or even months?)... The homepage we will release is completly different. The one you know at the moment is only for the costumers we have already...

Regarding your question: We plan to offer several output formats. .avi and .mp4 are two of them...

-Daniel

[Daniel_HDX4]

Originally posted by SeeMoreDigital
I hope you've prepared yourself to answer some tough questions?

Hi,

yep, I am. ;-) What do you want to know?

-Daniel

[bond]

Originally posted by Daniel_HDX4
What do you want to know?when will we be able to test hdx4 ourselves? :)

[Daniel_HDX4]

Originally posted by bond
when will we be able to test hdx4 ourselves? :)

Hi,

hopefully in a few weeks (still this month!). It depends on one other project we have to finish at first... But it's great to know that the codec seems to be intersting for you...

-Daniel

[CruNcher]

What is Skals position @ Jomigo ?
and is the codebase of hdx4 based on tmp4 + snr (Skals Codec) ?
and of course what's your position ;)

[Daniel_HDX4]

Originally posted by CruNcher
What is Skals position @ Jomigo ?
and is the codebase of hdx4 based on tmp4 + snr (Skals Codec) ?
and of course what's your position ;)

Hi,

Skal is not the lead programmer at Jomigo. In fact, he isn't even working for Jomigo at all. If you look at the "Codec shoot-out" introduction, Doom9 has removed already the information about the relationship of Skal and Jomigo. To be honest, all people at Jomigo were wondering about this information published there...

Anyway, I haven't thought that it's that interesting to know who is behind HDX4. ;-)

HDX4 is developed by a larger team of programmers. And I am sorry, but I haven't heard of tmp4 before...

Addendum on 01.02.2005:

I have looked meanhwile at tmp4. As far as I can see tmp4 can be used as bitstream reader/writer for compatibility tests. But is it a complete codec?

And what do you mean with SNR? Normally this means "Signal-to-Noise-Ratio" - but what has this to do with HDX4 and/or TMP4?

-Daniel

[sbp]

@babayaga
You wrote:
It went almost unnoticed but we did release an impoved version of our MPEG-4 ASP codec in in june 2004.
This version did add the new 1st pass concept that make a 2 passes encoding almost as fast as a one pass encoding on a whole movie."

Thats interesting, is it possible to use this codec for my purpose = real time encoding from my TV-card, or is it also only possible to use this codec from inside Recode?

Thanks
Steen

[Episode]

Heh. It looks like Doom9's 'codec shoot-out 2004' results have been posted to slashdot. I'm not surprised that almost everyone in that thread complains because Doom9 didn't test Ogg Theora and how there weren't any open source codecs included in comparsion. Only one person pointed out that Xvid is also licensed under GPL. I really don't understand those OSS-fanatics. In their opinion Theora is the ultimate codec since it free and doesn't have any licensing issues, no matter how bad image quality it produces.

[Doom9]

I really don't understand those OSS-fanatics. In their opinion Theora is the ultimate codec since it free and doesn't have any licensing issues, no matter how bad image quality it produces.I guess as long as they don't have to watch the content ;) Perhaps next time I should do a more extensive codec research, encode one movie per codec, then throw out codecs that really can't keep up with ND AVC and XviD.

[bond]

lol, again its fun reading slashdot :D

anyways, i consider myself as a fan of theora too and i am not really angry about that it hasnt been included. i think there is no doubt about how it compares to advanced codecs like xvid and nd avc qualitywise (still, quality is not everything), an inclusion might have done more harm than good for theora atm

and also: why is it so hard to differentiate that not "nero digital" was used in the test, nor won it... such an inconcreteness has the potential to confuse newbies i think. it has to be pointed out that nero digital avc was used in the comparison and not the asp version, which is supported on standalone players and wasnt even used in that comparison...
of course doom9 made that clear, but people tend to rush over such important details far too easily :rolleyes:

[CruNcher]

@Daniel_HDX4
Sorry i meant SNS not SNR

[Leak]

Originally posted by Episode
Heh. It looks like Doom9's 'codec shoot-out 2004' results have been posted to slashdot.

Hooo boy, there go my 5 /. moderator points... ;)

How's the site handling the slashdotting? It's up, obviously - but how hard is it being hit? :D

np: Gescom - Keynell 3 (Keynell EP)

[Doom9]

How's the site handling the slashdotting? It's up, obviously - but how hard is it being hit? I'm afraid I don't have more than just the webalizer stats, and it takes at least 24h to see something in those. Yet, I've been talking to an admin of one of the mirrors and the load was virtually zero. I've also not heard anything about a performance impact, and the forum took todays user record with a smile.

[Selur]

@Doom9: thx for another nice comparison :D

[babayaga]

Originally posted by sbp
Thats interesting, is it possible to use this codec for my purpose = real time encoding from my TV-card, or is it also only possible to use this codec from inside Recode?
As far as I know it is only possible to use it from Recode. It may change rapidely (or maybe it's done yet :)).

Besides, there is still the interlaced issue : the current release is about as good as XviD or DivX in the case.

[neo75903]

i did used to used chris-tv to record live tv with Xvid, but the results arent satisfying as using mpeg2 at 2000kbps.
cpu use was high, and also had a out of sync all the time.

i also have a pvr server which uses 3 150mce's to record from 3 different channels, cpu use on a 2500+ hardly over 10 percent.

[fatso485]

very good work doom9

i think it would be very usufull to know the bitrate of the actual clip as appose to the whole movie. this will greatly help us to determine how good a codec is in determine how efficient a codec really is.

also, isn't there something that you can do to make us download the clips.

sorry about my english and if any of these points are address in the extreamly long thread

[Doom9]

i think it would be very usufull to know the bitrate of the actual clip as appose to the whole movie. this will greatly help us to determine how good a codec is in determine how efficient a codec really is.Please think about the effort for a moment.. cutting MP4, RMVB, AVI, then demux? Lots of work. There's a reason why clips are taken throughout the movie.. with a large sample size you can assure that on the average your impression is correct.

also, isn't there something that you can do to make us download the clips.Absolutely not.. think about it.. the comparison itself is 43 MB. The entire 3 movies rate about 40 GB.. so the total clip size would be say 1 GB. Times 10'000 downloads = 10 TB of traffic. And then there's the copyright question.

[Sharktooth]

Originally posted by babayaga
As far as I know it is only possible to use it from Recode. It may change rapidely (or maybe it's done yet :)).
Finally! Thank you.

[sbp]

Originally posted by babayaga
As far as I know it is only possible to use it from Recode. It may change rapidely (or maybe it's done yet :)).

Besides, there is still the interlaced issue : the current release is about as good as XviD or DivX in the case.


That is good news.

If you want someone to help with the testing for real-time capturing from a TV-card (phillips 7134 chip) I will be veryy happy to help you with that :-)

Steen

[SpaceV]

First, thanks Doom9 for the great comparison.
One thing that would it even make better (for me at least),
would be to have 2 differently coded frames on top of each other.
This way you could have the original and ND or ND and VP6 on top
of each other and compare more easily. I know you could do it with
2 explorer windows as well, but its "nicer" on the same page.

Now for the file sizes. I have not read the whole thread, but it seems that some people are still trying to fit a movie onto a CD.

Why is that? DVD burners cost 70$ and DVDs 20 cents. CDs cost about 8 cents. But I can fit 6 CDs onto a DVD. So isnt it more expensive to record onto a CD??? And besides, I rather have a 800-900MB encode with great quality and I can still fit 5 of those on a DVD.

I think the 700MB compression days are over.

What do u guys think?

[Tommy Carrot]

Originally posted by SpaceV
Now for the file sizes. I have not read the whole thread, but it seems that some people are still trying to fit a movie onto a CD.

Why is that? DVD burners cost 70$ and DVDs 20 cents. CDs cost about 8 cents. But I can fit 6 CDs onto a DVD. So isnt it more expensive to record onto a CD??? And besides, I rather have a 800-900MB encode with great quality and I can still fit 5 of those on a DVD.

I think the 700MB compression days are over.

What do u guys think?
It's probably because i like to keep all my movies on my HDD, and larger sizes would take up too much space. 1 or 2 cd-size is a good balance between size and quality. You're right they shouldn't need to be strictly 700 or 1400 MB, but it's like an old habit to me, i don't really like the arbitrary-sized movies. :D

[SeeMoreDigital]

Originally posted by SpaceV

I think the 700MB compression days are over.

What do u guys think? Nowadays, I only buy 800MB CD~R media.

But like a lot of people, I store my encodes on hard-drives so I can view them over a network!


Cheers

[Doom9]

why CD size? I have my doubts that even on this board, with all the people that are into what we do, you'd find a majority to use DVDs for their backups. And if you go to a larger sample of people, like my colleagues at work, or my friends, hardly anybody has ever burned a DVD.

So, from a global perspective, CD sizes are still very much applicable, and 700MB CDs are the most common.

And, regardless of the number of users, CD sizes seem kinda fitting for movies, 800MB CDs would make the test too easy, wouldn't you think?

[SpaceV]

Hm, I am using DVD as backup since about 2 years.
And now with a dual layer 16x burner @ 22MB/s, I see no reason to use CDs unless u want to create an audio CD.
Btw, I was burning @16x on a 8x media over a 100 Mbit LAN.
The recording had to stop and continue many times bc the network could not keep up. Amazingly the DVD works perfectly. (Need to get
a gigabit switch.)

And as I said before burning CDs is more expensive !!

If people still use CDs then they should wake up, it's 2005 !!

I cant wait to use BluRay for back ups, hope the new protection coating is as good as the PRs.

As far as 800MB being to easy, yeah for movies under 90 minnutes. For movies 120min and up, even 900MB can be quite challenging.

[RyFish]

Originally posted by SpaceV

And as I said before burning CDs is more expensive !!

If people still use CDs then they should wake up, it's 2005 !!


Maybe if you'd like to buy me one I could try it out! :) Seriousy , not everyone can afford a DVD burner just like that - especially just after Xmas and when your kids needs new shoes etc!

[SeeMoreDigital]

Originally posted by RyFish
Maybe if you'd like to buy me one I could try it out! :) Seriousy , not everyone can afford a DVD burner just like that - especially just after Xmas and when your kids needs new shoes etc! And by the time you've bought a DVD burner and loads of blank media... you might as well have bought yourself a gigantic hard-drive!


Cheers

[SpaceV]

C'mon you can get them for $70, that cant be much for
anyone on this board. For compressing movies with the new
codecs you need a fairly new machine anyway and a lot of
them even come with DVD burners nowadays.

I agree with the hard drive but: Its tough to carry it around
and go to a friends house to watch the movie there and If
your head crashes then all is lost. I have my stuff on HD but I back it up on DVD.


Off topic: XM Satellite Radio just announced to use On2 VP6.2 for video streaming in cars. Nice !! Congrats ON2, I just wonder why its not VP7??

[gircobain]

Originally posted by SpaceV
Now for the file sizes. I have not read the whole thread, but it seems that some people are still trying to fit a movie onto a CD.

Why is that? DVD burners cost 70$ and DVDs 20 cents. CDs cost about 8 cents. But I can fit 6 CDs onto a DVD. So isnt it more expensive to record onto a CD??? And besides, I rather have a 800-900MB encode with great quality and I can still fit 5 of those on a DVD.

I think the 700MB compression days are over.
Sorry to be off topic, but I think you should perhaps take a trip around the world and realize that these prices in no way apply to everywhere world wide.

[ObiKenobi]

Originally posted by SpaceV
C'mon you can get them for $70, that cant be much for
anyone on this board.

I don't know what brand drive you can find for 70 bucks, but I'd be really concerned about the quality/longevity of such a drive. Any quality brand of drive I've found at Best Buy/CompUSA has always been at least 120 dollars or more. Secondly, not all of us have an endless supply of money to spend on hardware and media so for some of us that is quite a bit.

[SteMa]

Not everybody lives in the US, I have myself a burner, and a few of my friends, but mostly they still have a cd-rw, or a combo drive. a pio 108d costs about $90 here, but a better average monthly paycheck is about $500. 700mb is ok for everybody, U can fit 6 of those on a dvd if you have a dvd burner, and if U have a cd-burner U can still write those on cds (dvd media prices about 70cent illegaly, and legally $1.5 minimum! cd-s are 28cent illegaly, and about 85 legally)

[Sergei_Esenin]

Originally posted by SteMa
700mb is ok for everybody,

Don't generalize in the other direction though--just as it's not accurate to say everyone should do high bitrate 2-4 CD size encodes and burn to DVD, it's not accurate to say 700MB is enough for everyone. I'm lucky enough to view on a 100" SVGA projector system, which makes 700MB rips look like garbage next to non-downsized high-bitrate encodes.

Low bitrate CD encodes are still "mainstream" thanks to media, bandwidth, and installed base, but the near future is high bitrate unresized or processed and upscaled video. It's already the present for many of us quality-loving AVSForum types, and HDTV is accelerating appreciation for high image quality.

[SteMa]

Sorry, you didn't understand me:
What I meant with 700mb was 700mb, 1400mb, 2100mb, etc... meaning, if groups would start releasing 1 gig movies then it whould be ok for ones with dvd-burners, but dividing to cd sizes is a great compromise.

[kieljan]

there are some words like

ºIf anybody claims any codec can deliver DVD quality at 700MB for a 2h movie, have him encode Matrix3 and compare it to the DVD. And if he still thinks so, send him to an eye doctor.º

i´ve ripped Matrix3 by Nero H.264 (714 Mb 1Cd rip). NTSC 720x304, 23.976 FILM fps, 760 kbit/s + HE-AAC 52 kbit/s. Of course it isn´t a DVD quality.
But quality is great and big part of details was preserved.


Ripping material from a DVD(or from any other delivery source for that matter) will never be DVD quality for a simple reason - you are using source material which has already been compressed, in the case of dvd using the mpeg-2 codec, and thus the quality will ALWAYS be lower than the source. And Mpreg2 already introduces compression artifacts (since it is a delivery codec which uses key-frames) so the secondary compression in the ripping stage suffers even more because it doesn't have all the information in terms of original full frames...
So, what i'm getting at - if one wants to compare with "DVD" quality, one would need source material in uncompressed format/intermediate format (such as motion jpg) which doesn't have any keyframes.
I personally encoded some material which was compiled from tiff sequences, and then converted to avi(using the lossless huffyuv codec) and i can tell you that mpeg2 aka DVD quality is miles behind codecs such as wmv 9, avc, and the other newer codecs. Which is why it won't be used as the standard for the emerging HD formats - the codecs of choice will be based upon AVC (h.264) and VC-1 (wmv 9)

If one wants to compare by way of DVD quality

[Doom9]

there's an unwritten rule that you don't pick up a topic after it's been resting for a while. 4 months definitely qualifies...

[kieljan]

sorry, i'm new to this forum, and have been browsing interesting articles - when coming across this whole issue i reckoned that this particular point was quite important, so just wanted to shed some light on it - especially cos it might not be the last time that anyone reads this thread, and are thus misguided...k