PDA

View Full Version : doom9's codec shootout


twist3d
31st December 2003, 10:04
from the codec test:

I contacted each codec maker 2 weeks prior to the start of encoding to give them a chance to suggest the best settings for the movies used in this test

So,

what do you think of the results and settings used on codecs?

ie. would've the results changed if there were lanczos resize instead of bicubic resize & divx 5.1.1 slow setting vs. multipass normal with qpel etc.etc.

comments are welcome.

spectra
31st December 2003, 11:18
yeah it would be interresting if you could comment on the makers settings. altogether that is again a great shootout and the png with zooming feature was worth the hard work for getting the screenshots.

Atamido
31st December 2003, 11:31
It would be nice to get a BitTorrent up of the pages with all of the pictures as the comparison has already slowed to a crawl. And with a hundred or so people seeding, you could get it distributed pretty fast.

Atamido
31st December 2003, 11:42
One thing, it doesn't look like your PNGs have been optimized to get the smallest size. If you run something like PNGCrush on them, you will probably save 2-5% in size, without any quality loss.

Or, if you want to dedicate a few hours to it, then try using this script (http://www.hydrogenaudio.org/index.php?showtopic=16167) from HA and you will likely get 5-7% reduction in file size. It may not seem like much, but that is a lot of cumulative bandwidth.

bond
31st December 2003, 12:15
thanks again to doom9 for again a great test, which would make it once again clear howto judge the quality current codecs provide :)

Originally posted by twist3d
what do you think of the results and settings used on codecs?doom9 used the settings he was told from the codec manufacturers to use (the avs script is the same for all)!
believe me they have no interest in telling him settings that wouldnt result in the optimum...

as i am a qpel nerd, i would also have used it, but you cant blame doom9 for what the codec manufacturers suggested...

jeremymacmull
31st December 2003, 13:24
not to put a damper on this but im sure loads of people would have been more interested to see if the standard mode is as good as divx 5.05 slowest or better as that is the most likely mode to be used by most people

bilu
31st December 2003, 13:29
Originally posted by jeremymacmull
not to put a damper on this but im sure loads of people would have been more interested to see if the standard mode is as good as divx 5.05 slowest or better as that is the most likely mode to be used by most people That's an "Idiot Guide To Codecs" approach, and would require doing every benchmark with default settings. Remember that a lot of codecs have recommended settings that are completely different from outdated default settings.

Bilu

jeremymacmull
31st December 2003, 13:38
Not really im thinking more of time taken to encode a clip

the difference in speed between the slow/slowest mode and the standard mode is ridiculous (not sure of the spelling!! :P)

so a lot of people who were and still are quite happy with divx 5.05 will now encode in 5.1.1 in standard mode if they use it.

i myself will be switching to xvid as the speed differnce is not much and the quality diff is a lot

Basically this is for people to compare or try and compare the difference between the old 5.05 slowest mode and the 5.1.1 standard mode which are meant to be equivalent. so people can see if they want to upgrade

JEREMY

Doom9
31st December 2003, 14:21
Basically this is for people to compare or try and compare the difference between the old 5.05 slowest mode and the 5.1.1 standard mode which are meant to be equivalent. so people can see if they want to upgrade
What question do I ask on page 5? Is it "should you upgrade from DivX 5.0.5 to DivX 5.1.1?" or is it "which codec looks best?"?

Feel free to make another comparison that answers other questions but if you start trying out different codec builds and different settings for the same codec, I won't expect your results until the next decade. You really have to have done this once to appreciate the effort.. the amount of time required is huge.

bilu
31st December 2003, 14:30
Originally posted by jeremymacmull
Not really im thinking more of time taken to encode a clip

the difference in speed between the slow/slowest mode and the standard mode is ridiculous (not sure of the spelling!! :P) People looking for speed should go for Nero Digital.

jeremymacmull
31st December 2003, 14:35
I dont want to cause offence the new codec shootout was most enlightening hence the reason im switching to xvid as my puny machine does not like 25 hours of encoding in a row it much prefers 10-12 hours of encoding :P

I think your work doom9 is top notch and i dont know what id do with all my spare time without this site.

ive conducted a few tests with 5.1.1 standard mode on a couple of films but they werent conclusive.

thanks for all the work and if you dont mind me asking how much time did the codec shootout take from start to finish ???

thanks again

JEREMY

Doom9
31st December 2003, 14:57
thanks for all the work and if you dont mind me asking how much time did the codec shootout take from start to finish ???
I don't mind at all. I started contacting codec makers in late November. They were given two weeks (a few days past the deadline was still acceptable) to provide suggested settings, and updated builds if applicable. I didn't contact everybody at the same time, so the whole process took about 3 weeks until I started encoding. Encoding started on December 13th and was going on pretty much 24/7 until Sunday the 21st (I had to redo the vvfwf stuff as I've mentioned in the article). Some time I also had to delay the start of encoding to ask back (the usual inclarities when it comes to settings).

The week of Christmas I was speed-testing, muxing and taking screenshots. Taking screenshots actually takes more time than writing the article because frame numbers don't match. Frame 5000 in DivX5 isn't equal to frame 5000 in 3ivX (as an example), so for 2/3s of the codecs I had to search for frame offsets to take screenshots (and then those offsets are not always constant, making it really hard to match frames if there's no visually distinguishing mark on screen.. you have to flick between images just like you can in the comparison now). I also quite a few problems taking the screenshots, with overlay and such. Eventually, the VMR9 renderers (thanks DarkAvenger) worked out fine, but they added a 2 pixel border around everything which had to be removed again. And obviously, you screw up a couple of shots which you have to redo in that last minute (yesterday).

This week I then wrote the article (with proof-reading that took perhaps 6 hours), and spent a lot of time with the image switcher / zoomer (talking online to the programmer, bugfixing, adjusting, you name it), and then I had to turn the static image comparison (looking like the previous one) into a dynamic one.. that also took a few hours.

And now I'm having a png optimizer running in the background and I'll create a jpg based comparison as well for people with low bandwidth (all matrix shots weigh more than 10mb.. I can't imagine how long it takes for a 56k user to view them all).

dragongodz
31st December 2003, 15:07
Doom9 - an interesting read as usual.

only 1 complaint, damn switching pictures on 56k is painful. ye i know you already mentioned we 56k users, all i can say is you really dont want to know how long. :(

mf
31st December 2003, 15:07
I find the animation test still not testing the codecs very well. The DVD source itself shows artifacts, Futurama is a show without much detail (and also with thick lines that don't show the individual sharpness of the codecs), and what I heard from superdump is that the codecs don't even have all that much difficulty with it either. And last but not least the DVD is a fine example of crappy authoring, as the source was blurry which was covered up with contrast sharpening (unsharp mask), and of course in interlaced tradition sharpened stronger vertically than horizontally. Not that it actually helped.
As much as you might not like it, asian animation is a bigger test to the codecs, is being encoded by more people (so the results have more significance in the community), and is often authored much better.

As for "I personally haven't seen an animated feature that I liked yet", what have you seen already, then? Can we recommend anything?

snowcrash
31st December 2003, 15:11
I'm disappointed that WMV9 wasn't included in the shootout. It is becoming a mainstream codec with Microsoft behind it, despite the fact that the videophiles around here like to ignore it.

jeremymacmull
31st December 2003, 15:17
if you read doom9s rec he says the reason wmv9 was not included is because no new features or builds etc have been made since the last codec shootout so anyone who wants to compare should look at the last codec shootout images after all its the same movies and frames

same reason why 3.11 is not going to be included after this i believe

JEREMY

bilu
31st December 2003, 15:18
Originally posted by mf
As much as you might not like it, asian animation is a bigger test to the codecs, is being encoded by more people (so the results have more significance in the community), and is often authored much better. It probably would make RV9 win by a larger margin, I've heard that XVID cartoon mode is not good with real anime.

http://forum.doom9.org/showthread.php?s=&threadid=65938


Bilu

Doom9
31st December 2003, 15:38
to the anime crowd: I forgot to add this to the FAQ (will be added when I upload the modem friendlier jpg version):

Why don't you use an anime source instead of Futurama?

Despite what the anime crow (which is really loud) wants you to believe, more people watch regular animated features than Anime. Look at all the cartoons, and mainstream Hollywood movies (Disney anyone), that attract a much wider audience than Anime (which is a special interest genre).

Furthermore, I just don't get Anime and I won't use a source that I don't like.And those FAQ items are my last word on the subjects. I know all the complaints that come every time and I've dealt with them accordingly in the FAQ because I don't want to waste my time with this here.

bilu
31st December 2003, 15:51
IMHO it would be better to invite anime encoders to do their own benchmarkings. I honestly don't like the tone of your FAQ proposal :sly: - Japan is a wide anime market and not a less important one. Of course US, Europe and others do more cartoon then anime, but I've seen lots of anime posting by French users, for example.

Recent article on Japan's cultural growth:
http://slashdot.org/article.pl?sid=03/12/28/1623216&mode=thread&tid=168&tid=186

Bilu

Note: I don't do anime.

mf
31st December 2003, 15:58
Spirited Away was released to the west as a Disney release, and that said, even the original Disney releases are more of a challenge to codecs than Futurama.

Doom9
31st December 2003, 15:59
I suggest you turn on the TV and flick through the channels, then revise your statement. We have millions of kids every day watching good old fashioned cartoons on TV. How many popular movies that were shown in a theater are Anime and how many are regular animated features? I go to the movies quite often but the last Anime flick I remember was Ghost in the Shell and I could give you a long list of normal animated features that has been shown in the past few years.

Come on, don't try to decieve yourself.

bond
31st December 2003, 16:06
Originally posted by snowcrash
I'm disappointed that WMV9 wasn't included in the shootout. It is becoming a mainstream codec with Microsoft behind it, despite the fact that the videophiles around here like to ignore it.it wasnt ignored, it simply didnt change since the last comparison, no need to repeat (and waste bandwidth) for showing that wmv9 provides less details than xvid again :D

btw: i am sniffing a lot around on video related sites, but i dont see where wmv9 is becoming a "mainstream codec" (i guess the reason for this is cause m$ is behind it ;) )
some people prefer to use wma9 because m$ pushes standalone support, but many independant (!) and scientific listening tests showed that wma9 cant cope with vorbis or aac (at as good as all bitrates)

sorry :p

bilu
31st December 2003, 16:07
@Doom9

I agree with you about most of the world except Japan.
But if we think of anime as a harder cartoon, then it would make more sense for cartoon benchmarking, much like SPR or Matrix scenes were chosen.

Now if cartoon has hard parts that anime doesn't have, it becomes a completely different question ...

Bilu

mf
31st December 2003, 16:09
Originally posted by Doom9
I suggest you turn on the TV and flick through the channels, then revise your statement.
Okay.
I flick on Yorin, I see Toonami. Transformers Armada. Anime.
I flick on Fox Kids, I see Crayon Shin Chan. Anime.
I flick on Nickelodeon, I see Bassie en Adriaan. That's not cartoon so it doesn't count.

Kedirekin
31st December 2003, 16:27
I'm a huge anime fan, but (even though I hate to admit it) Doom9 is right.

Despite the surge of popularity of anime in the past 5 years (the selection at my local Best Buy has gone from half a shelf to half a wall), it just doesn't have the same kind of penetration as cartoons.

We've seen the recent theatrical releases of Rugrats, Thornberrys, Kim Possible, Hey Arnold, Recess, Lilo & Stitch, and how many others. The *only* recent anime release I can recall is Spirited Away, and that was a limited release at that.

Teegedeck
31st December 2003, 17:31
Just wanted to say thanks; very meticulous work, must have cost a lot of time. Great layout, too!

sysKin
31st December 2003, 17:51
Just to add my two cents to the anime part of the thread:
Originally posted by Doom9
Furthermore, I just don't get Anime and I won't use a source that I don't like.Sorry but that settles it. I love anime, test every xvid expermient with it and enjoy doing so. If Doom9 says he doesn't enjoy it, I won't mention anime ever again.
This stuff is made for fun, period.

Radek

PS. at least Futurama is the kind of cartoon that works with xvid's cartoon mode :devil: Well Miyazaki's movies would probably also work, but try it with GITS or even modern series (Naruto lol, or SAC) and it b0rks. So I'm not going to complain.

trbarry
31st December 2003, 18:00
I like these shootouts. :)

- Tom

bilu
31st December 2003, 18:03
Originally posted by Kedirekin
The *only* recent anime release I can recall is Spirited Away, and that was a limited release at that. Ghost in the Shell, Princess Mononoke, Dragonball Z were the most noticeable in my country in the recent past.

Do we want to forget Pokemon and Digimon? :D

EDIT: @syskin: like your sig ;)

Bilu

bilu
31st December 2003, 18:17
Originally posted by sysKin
Just to add my two cents to the anime part of the thread:
Sorry but that settles it. I love anime, test every xvid expermient with it and enjoy doing so. If Doom9 says he doesn't enjoy it, I won't mention anime ever again. I agree with that, Doom9 is just a guy like the rest of us and he does what he likes. Everyone is free to do their own benchmarks, and I'm sure that anime specialized benchmarks would be interesting - mf, MasterYoshidino,Gizmotech,Sirber and others are inside the subject and could do it if they're willing to. If not, they won't :D .

I just disagreed with Doom9's FAQ proposal, the text itself seemed like saying that anime is less important than cartoon for everyone. That's not true to everyone - but it is to Doom9 and that is what really matters. This is HIS benchmark done over what HE feels that is important - and EVERYONE is invited to do the same :)

EDIT: But if someone did an anime benchmark, maybe Doom9 would put a link ;)

Bilu

mf
31st December 2003, 18:20
I'm the type of person that just encodes stuff if it's HQ (the only exception is movies like Titanic or Pearl Harbor (:rolleyes::D @ CruNcher), which I won't ever touch for obvious reasons). I'm gonna do ex-driver the movie because the DVD is to lick your fingers for. So I guess I'm not very representative in the anime or cartoon thing. So maybe you might like to have someone else do a subchapter of the "animation" part, that covers anime, to fit in with the subculture thing (subculture->subchapter)? ;)

BoNz1
31st December 2003, 18:21
Thanks doom9 for the excellent codec comparison. Each time you manage to outdo your last comparison. The switching between screenshots and zooming rocks IMO. Thanks again :)

Kedirekin
31st December 2003, 18:31
We anime lovers should probably take this elsewhere, or at least out of this thread.

I just wanted to point out that Doom9 is correct; anime isn't as widely popular as we fans might wish. Of all the people I know face-to-face, I'm the only one I know that really likes anime. I have to go online to find other fans.

Even within the population in these forums (which most likely is heavily weighted compared to the outside world), anime is probably not as popular as we fans mights like to think.

BTW bilu: we're using different definitions of recent. I was thinking of the last 2 or 3 years.

sysKin
31st December 2003, 18:44
Hi again,
If I might, I'd like to point out a single small 'flaw' in the comparison (which actually helps XviD so I shouldn't mind, but I'll tell anyway).
It's the way speed is measured.
The average fps of 1st and 2nd pass seems right but isn't. Take a hypothetical codec that makes 1st pass at 50fps and second at 0.1fps. The average - 25fps - looks good but doesn't really indicate that it takes 20 days for a two-hour movie.

This example is extreme but it does show an advantage given to any codec that can make any pass faster.

The 'better' way would be to measure time of both passes, average the time, and then convert to fps.

BTW Happy New Year everyone,
Radek

[EDIT] As explained by Doom9 himself a bit later, I was wrong. I probably shouldn't write any posts at 4am on 1st of January... I promise I won't do that again, at last for a year :D :p

twist3d
31st December 2003, 18:52
I'm not complaining about the comparison, it's pro and big ups for Doom9 that he spent so much time in it.

I intended this thread to be discussion about what users would've wanted for codec settings, resizing filters etc. that'd effected the final scores of the codecs.

I'd still like comments about "what settings i would have used for encoding" from divx/xvid/3ivx/etc. users

Doom9
31st December 2003, 18:58
@syskin: I'm not sure if I follow but I think your assumption is wrong. I encoded a part of matrix, 10'000 frames long. I set up everything so that the two passes are done consecutively done, then the time it took (for both passes) is divided by the number of frames (in both passes).

Thus, in your example let's say we have a 25fps source that's 10'000 frames long, so it takes 200 seconds for the first pass, and 100'000 seconds for the 2nd pass, total 100'200 seconds divided by 20'000 encoded frames = 5 fps, not 25 fps.

KpeX
31st December 2003, 19:01
Big thanks to Doom9 for the shootout. An excellent read :).

jggimi
31st December 2003, 19:08
I for one would like to thank Doom9 for all his terribly hard work. Each comparison has been as fair as possible, has been meticulous, and has been a lot of work to produce. Each has also been an improvement on his previous comparisons.

This latest one, with it's new display format, is by far the easiest to read and comprehend. And I don't even mind that my most commonly used codec didn't fare nearly as well, in his opinion, as others.

The key is these comparisons were his opinion. We do not see the resulting video, only still images, and are free to make our own comparisons. There is sufficient information in the comparisons that we can duplicate his research, if we wish, or, of course, to do our own. With our preferred settings, too.

Doom9, you've helped me understand your interpretation of the limitations and benefits of the codecs I use, and have given me plenty to think about with codecs I have not yet tried. I appreciate it, and all the work you've put into these comparisons.

Thank you.

Nazgul
31st December 2003, 19:47
Just a quick question, Doom9. Did DivxNetworks mention any particular reason why they suggested Slow rather than Slowest(or Standard, for that matter)? Did they figure it was a good middle ground between quality and speed?

Personally I'd really be interested if someone took a closer look at the differences between the new P/Q options in Divx5.11, but I fully understand that these shoot-outs are no minor task and appreciate the work you've put into them. They're always interesting reads.

Soulhunter
31st December 2003, 21:10
Big THANKS for this shoot-out, really great work !!!

A happy new year ahead... ;)

PS: Could there be also a zip version of it to download ???

Bye

larsc
1st January 2004, 02:27
Originally posted by jggimi
I for one would like to thank Doom9 for all his terribly hard work. Each comparison has been as fair as possible, has been meticulous, and has been a lot of work to produce. Each has also been an improvement on his previous comparisons.

AOL and halleluja.

Soulhunter: There's a torrent link here (you'll need a bt-client):
http://forum.doom9.org/showthread.php?s=&threadid=67855


And I have a question too. It's about Q11 in the FAQ-section.
'Q11: Why did you use postprocessing to review the clips?'

I'm using ffdshow (just straight out of the box, I haven't touched any settings) for DivX/XviD/whatever playback on a box I've got setup for a home entertainment centre (no encoding to be done ever), am I missing out on something?
Should I install seperate decoders instead? I'd really prefer not to. :)
(sorry if this has been asked and answered)

Doom9
1st January 2004, 04:52
Should I install seperate decoders instead? I'd really prefer not to. Well.. I like ffdshow as a generic purpose solution but I think you should give the codec specific DS filters a shot.. having seen the effect those filters can have firsthand it might not hurt trying them out.

@syskin: I redid the calculations in my head when driving to new year's dinner (I was sitting in the back so I had plenty of time).. and even if you have a 200k image movie, the 2nd pass in your example is the only one that would matter so you'd have 200'000*0.1 = 2 mio seconds divided by 2x200'000 frames = 5fps as in my previous example so I think I'm doing the speed measurements is fair.

sysKin
1st January 2004, 09:56
Originally posted by Doom9
@syskin: I redid the calculations in my head when driving to new year's dinner (I was sitting in the back so I had plenty of time).. and even if you have a 200k image movie, the 2nd pass in your example is the only one that would matter so you'd have 200'000*0.1 = 2 mio seconds divided by 2x200'000 frames = 5fps as in my previous example so I think I'm doing the speed measurements is fair. Yes Doom9, I'm sorry bout the fuss - I misunderstood the way you measured speed. Everything's ok :) the comparison is great.

Radek

sh0dan
1st January 2004, 22:50
Great comparison - very detailed and very thorough!

:thanks:

A few minor points I noticed:

*) Scripts - I'd personally prefer lanczosresize - it seems like the codecs can handle the added complexity by now. SPR seems like it's better off with bicubic though.

*) Futurama - I'd personally prefer Donalds KernelDeInt over FieldDeinterlace. It provides better detail and a smaller amount of artifacts. My personal preference is KernelDeInt(threshold=4).

*) I noticed a slight color and luma shift in the Real Video, especially in the futurama toward slightly too bright/saturated images.

Still missing WMV9 in the comparison, even though nothing has changed since the last comparison. I would much rather have it than DivX3.

Doom9
1st January 2004, 23:15
Still missing WMV9 in the comparison, even though nothing has changed since the last comparison. I would much rather have it than DivX3.
I think the reasons are well documented. Until recently DivX3 was my reference codec and you obviously want to compare other codecs against the reference. Now that DivX3 is no longer the reference it will never again be included in a codec comparison (along with every other codec that is not improved). I'd rather swallow a bullet than test a codec twice. It is more than enough work to test new codecs and I am way too underpaid and busy to restest something I have already tested.
I have an estimate of how WMV9 would've rated though.. better than 3ivX and ND but worse than the rest of the codecs. Do you really want to bother with such a codec? At least ND is fast.

Teegedeck
1st January 2004, 23:37
Perhaps it would make sense to keep SBC in as an 'anchor'.

sh0dan
2nd January 2004, 01:02
@d9: I completely agree - but has compression settings changed since the last test? If not, could the images be included in the very excellent comparison you've put up this time?

I'm not expecting you to change anything in the current shootout - just a suggestion for next time.

avih
2nd January 2004, 01:44
:)
amazing job doom9.
congrats for yet another great codecs comparision.
the thumb-zoom is very cool.

cheers mate
avih

Doom9
2nd January 2004, 01:45
No, I will not add the wmv9 screenshot. I just seem to unable to get the message across that images are useless. Maybe I should just not bother with them anymore but then nobody reads the comparison anymore.

People: screenshots are useless in 2/3rd of all cases (at the last)You don't see what I see when I review the clips. Adding images is completely useless if the codec in question has a) not been reviewed, and b) not been compared to the other participants.

I didn't just add the old divx3 screenshots.. I actually reviewed the clip again and compared it against the other codecs. If I hadn't done that I'd never have added the images (and I never will). Without review no images.

Oh and I don't know where any luma shifts might be coming from.. might have something to do with the VMR9 renderer... feel free to investigate. I'm really done with the comparison.. I don't feel the need to invest more time into this (and I didn't have any complaints from the Real people either). One month is more than enough time for something I don't even get paid for.

What I find really interesting is that the people who actually have to gain / loose something because they made the codecs, seem to be the ones with the least amount of complaints in the end. I guess it's vanity to try and satisfy everyone but I can't get that into my head. Maybe I need to bang it against a wall a couple of times...

KAMiKAZOW
2nd January 2004, 17:28
Great work, Doom9. Thanks a lot.

Originally posted by bond
some people prefer to use wma9 because m$ pushes standalone support, but many independant (!) and scientific listening tests showed that wma9 cant cope with vorbis or aac (at as good as all bitrates) http://audio.ciara.us/test/128extension/results.html shows that WMA9 is quite good (better than Vorbis according to this test).

Gaia
2nd January 2004, 18:03
Originally posted by KAMiKAZOW
Great work, Doom9. Thanks a lot.

http://audio.ciara.us/test/128extension/results.html shows that WMA9 is quite good (better than Vorbis according to this test).

WMA9 is very bad. WMA Pro is totally different codec.

Read this quote from that test page:

"It's worth mentioning that the WMA codec used in this test is WMA PRO introduced in Windows Media 9). It has better quality, but is not backwards compatible with WMA standard, that is the codec that is actually supported by portable players, DVD players, etc. It's expected that devices supporting WMA Pro start being manufactured soon."

unmei
3rd January 2004, 03:13
simply great! thanks a LOT doom9 :D

Just laugh at me, but i spent more than 6 hours by now analysing every pic in the matrix test and making a rating to get some picture in how the other codecs differ from xvid. This because i recently thought about trying out ND and VP6 and also was not so sure i shouldn't give divx5 another chance..well my conclusions shall not be here ..do it on your own. The only remarks on this i give here: no need anymore, thanks doom9, you saved me hours and frustration. And RV9 surprised me with it's details in Neo's face close-up.

I know you don't see the whole truth by comparing single pictures, but it is something i always did when evaluating codecs - and a codec that cannot even survive this stage is for me not worth being evaluated any further (read as: i won't use it for actual encodes).

I did this extended examination for matrix even tho i only encode anime. I'm not complaining and think Futurama is a good test to include, since for me, the animes i encode are a mixture of 'sharp and simple' animation like Futurama and the fuzzyness/complexity of real life pictures. Thus the matrix is more important to me, but of course if xvid totally lost out on Futurama it would have made me worry :)

Btw the first scene of Futurama (the rays) is a great test for my type of anime encoding (i still remember how i terribly failed to get that encoded right from TV - like two days only to realise this scene is a killer when combined with TV noise).

mf
3rd January 2004, 15:13
/me wonders if doom9 liked The Animatrix :D.

Doom9
3rd January 2004, 16:15
if doom9 liked The Animatrix Yeah, but some episodes more than others. From a story standpoint the 2nd renaissance was best but I didn't particularly like the visual style. From a visual standpoint I liked the last flight of the osiris best (guess what.. it has nothing to do with Anime style ;).

All in all I didn't like the visuals much, I only watched those episodes for the story content.

mf
3rd January 2004, 16:55
Originally posted by Doom9
Yeah, but some episodes more than others. From a story standpoint the 2nd renaissance was best but I didn't particularly like the visual style. From a visual standpoint I liked the last flight of the osiris best (guess what.. it has nothing to do with Anime style ;).
Drat :p.

SeeMoreDigital
4th January 2004, 20:27
Thanks Doom9, I like the test you did very much.

Very suprised at who came in second for the speed test - didn't see that one comming!

I know you don't encoding like to encode using anamorphic frame sizes but I do wish you had done some comparisons. And yep, DivX5.1.1 in slow and slowest mode is 'as slow as a hearse on its way to a funeral' personally I just stick with standard mode!

I'm still in France, so I can't get to check out the forum as much as I would like.

Cheers you guys and Happy New Year

Doom9
4th January 2004, 21:01
I know you don't encoding like to encode using anamorphic frame sizes but I do wish you had done some comparisons.http://www.doom9.org/codecs-203-6.htm, Q2. I'm dangerously close to use r16 and strike everyone who ever mentions something I put in the FAQ.

bilu
4th January 2004, 21:07
Originally posted by SeeMoreDigital
I know you don't encoding like to encode using anamorphic frame sizes but I do wish you had done some comparisons. Wouldn't be fair if codecs didn't have the same amount of pixels to process.

Bilu

Soulhunter
4th January 2004, 21:16
Soulhunter: There's a torrent link here (you'll need a bt-client):
Thank's !!! ;)

Bye

sillKotscha
5th January 2004, 14:33
Originally posted by Doom9
http://www.doom9.org/codecs-203-6.htm, Q2. I'm dangerously close to use r16 and strike everyone who ever mentions something I put in the FAQ.

:D

hope you had enuff coffee and/or beer during the shootout...

and folks remember one thing - he did it in the time of x-mas!! So congrats to you doom9 spending your time in front of a PC and not with beloved friends (despite the ones here :) )

/me wouldn't have done it and I believe no one else... so please, no complaints!!

cheers Sill

Doom9
5th January 2004, 15:02
actually I don't drink coffee and reviewing codecs with alcohol in my bloodstream would be unprofessional so I never drank a drop of alcohol when reviewing or writing the comparison.

sillKotscha
5th January 2004, 15:06
my statement was more or less rethoric :cool:

no coffee :confused: - and concerning "alcohol in my bloodstream would be unprofessional"... well this one is rethoric as well I guess ;)

Elrik
6th January 2004, 14:29
Curious... In "The Matrix" test, DivX5 keeps the color quite close to the original (especially in the close-up of Neo holding a cellphone), while XviD changes it a bit (darkens and/or saturates it somewhat). In "SPR", however, XviD is the one that keeps the color close to the original (especially in the first underwater scene), while DivX5 does not (sort of lightens it). What causes it? Is it how each codec encodes different sources/scenes or is it post-processing settings?

Could it be that in SPR underwater scene DivX5 lightens it somewhat because it smears it too much and, hence, destroys the darker color details, making the overall scene appear lighter? But then... it smears the details in Neo close-up too, yet it keeps the color nearly identical to the original...

Wilbert
6th January 2004, 22:59
A small remark. Please correct me if I'm wrong, which may very well be possible.

In the encodings, a very low bitrate is used for the Matrix (and almost twice as large for SPR ). The reason is that there is aimed for 1 CD encodings. That's a choice, and there is nothing wrong with this. The results don't say anything abouut 2 CD rips without pp or noisy sources. Of course, nobody claims that it would. (Personally I would love to see also a comparison for these two cases. In case somebody starts complaining: no, I don't have time for it.)

My point is that the post processing (and pre processing) is very important in these lowbitrate encodings. Because, one of the important things in this comparison, is the amount of detail versus blocks.

It seems to me that codecs with a very good deblocking/deringing code (like XviD) has a clear advantage above codecs with less good deblocking code. But, of course, codecs are more than good deblocking/deringing code.

Any thoughts on this?

SeeMoreDigital
7th January 2004, 01:25
Originally posted by Doom9
http://www.doom9.org/codecs-203-6.htm, Q2. I'm dangerously close to use r16 and strike everyone who ever mentions something I put in the FAQ. I don't think proposing to strike members because you either don't think they have read your Codec Shoot-Out FAQ (http://www.doom9.org/index.html?/codecs-203-6.htm) or you don't like (or agree) with what they have posted is very fair Doom9!

I understand your reasons for not generating anamorphic encodes. However, it's my opinion that doing such a test, may be useful and throw up completely different results!

"Push each codec to it's limit I say"!

It's also my opinion that some codecs/applications handle cropping and resizing better than others, which in turn affects the encoding speed, codec performance, encode image quality etc. Which is why I am in favour of encoding to the same image pixel frame size as the source..... But purely for the purpose of testing!

It's just my opinion. I apologise if I upset you!

Cheers

twist3d
7th January 2004, 07:29
Maybe in the next codec comparison it would be useful to share the workload? I think there would be many volunteers in this forum to try different settings and bitrates/resolutions.

Also "real" comparison should use abx-tests using ~100 participants, but this is completely useless as we should first encode video, then convert it to raw .avi for "blind" testing (too large filesizes).

Doom9
7th January 2004, 08:55
@twist3d: As you said yourself, blind testing for video faces major problems, mostly on the bandwidth side. I know of another site to try this but they never finished.

And as for different people trying different settings, that's once again unrealistic. Who reviews the results? Who decides which samples are taken? Every review clip takes hours and hours, for creation, muxing, review, screenshots and writing. Therefore, I'll rather not do a comparison at all than test the same codec with 5 different settings (and multiply that by the codecs to be tested).. it's simply not feasible. In video encoding, we simply have too many variables to compare. Thus instead of taking "some" settings I chose to ask the people who create codecs. They should be familiar with the options they offer to the user and should be able to make an educated guess as to which options might give a good result. It might not be the best possible, but I'm convinced that the results are in the "better" area of all the possible results you can get

@SMD: You, and everybody else who just can't keep their mouth shut about what they think is better should have a look at this thread and specifically colordog's posts: http://forum.doom9.org/showthread.php?s=&threadid=67634. Everyone complaining reduces the chance that I'll ever bother making another comparison. I don't need suggestions that I have already considered, or stuff that's simply useless. Last time there were a few good suggestions at slashdot like PNG images and using tables to present the results. Those things really make sense and they have been integrated. But I really don't need people telling how to do my comparisons.. that is at the least disrespectful of my work. If you disagree with the FAQ, do as colordog says.. keep it to yourself because if you bother me with it, it pisses me off and I take it as a personal insult.

It's also my opinion that some codecs/applications handle cropping and resizing better than others,Cropping and resizing is done in AviSynth for every single test so they have zero influence on the result. On the contrary, with anamorphic encoding you'd actually be evaluating the quality of the codec built-in resizing filter.

dragongodz
7th January 2004, 11:37
jeez this is still going ?

for those that missed my commenst after the previous codecs test i guess i can say them again.
listen carefully for i will say this only. once(name the show if you dare)

if you are unhappy with what Doom9 has done and would do it differently then do it. do the tests, put them up on a website(plenty of free ones around) and post the link here. then wait and see all the flak you get. then maybe you will appreciate how Doom9 feels. :devil:

mf
7th January 2004, 13:29
Originally posted by Doom9
@SMD: You, and everybody else who just can't keep their mouth shut about what they think is better should have a look at this thread and specifically colordog's posts: http://forum.doom9.org/showthread.php?s=&threadid=67634. Everyone complaining reduces the chance that I'll ever bother making another comparison. I don't need suggestions that I have already considered, or stuff that's simply useless. Last time there were a few good suggestions at slashdot like PNG images and using tables to present the results. Those things really make sense and they have been integrated. But I really don't need people telling how to do my comparisons.. that is at the least disrespectful of my work. If you disagree with the FAQ, do as colordog says.. keep it to yourself because if you bother me with it, it pisses me off and I take it as a personal insult.
I'm risking myself here defending SeeMoreDigital, but I'm going to do it anyway. Suggesting is not complaining. I think I understand how you feel as as soon as I show people a webdesign I've been working on I get overloaded by suggestions on readability, colors, etc, while I really did mean all those things to come out like they did. Though they might feel as complaints I have had to understand that they are not, they are simply suggestions on what would improve it for them (which, I must emphasize, is almost never what would improve it for everyone). The best thing to do then is just weigh all the suggestions down by general opinion and weigh in your own (heavier than the rest ;)), and then come to a decision whether you want to change anything. Nobody forces you to anything, and I think it's really best to try not to see any of these suggestions as offensive. After all, they're only suggestions, and if you're not in the mood of explaining why you didn't do things (Senior members and moderators can always back you up on things like the FAQ anyway, I wasn't as sharp as to notice that you actually had a FAQ, but other people will, I think), you're free to ignore them. Also, as a semi-free discussion board it's nothing bad to speculate on how things could be different, when the tendency is not to force anyone to change those things. My own suggestion, though covered by the FAQ (read above about how I totally didn't notice any FAQ, I usually skip to the test itself), also wasn't meant with any pressure in mind. I didn't think that it would actually happen because of me (after all, Futurama has now been in for 2 or 3 tests now), but I did think that if enough people would be of the same opinion, it would be worth considering. Well, if not, pity. No harm done. Or so it should be. Imho. Of course I could still just keep it to myself, but why? As most people here are used to having some kind of freedom in speech, it'll just cause less trouble if you take things friendly ("Better out than in I always say" -- Shrek).
Like SeeMoreDigital, I will also end with my respect to you. I don't wish to upset you, I just want peace in this thread :). I hope my thoughts came across as I intended them through the keyboard and language barrier. Posting like this is always an RGB->YUV->RGB conversion ;).

bilu
7th January 2004, 14:16
@mf
You almost made me cry. :goodpost:

@Doom9

Do you think there are ways to share your workload? Encoding, etc.?
I won't be able to provide hosting, but Im sure lots of guys would be helpful in what they can. You'd do the unreleased codec encodings, of course (such as ND's VFW version). It would be nice if we could at least spare you from some CPU and trouble :) while trying to do things in a impartial way as you have done.

Just trying to help and motivate.

Bilu

Doom9
7th January 2004, 15:09
Suggesting is not complaining.There are different ways you can make a suggestion. There's the productive one: measure X would have the following advantages, and the following disadvantages. And you have to be honest about it. I do have well documented reasons why I do things a certain way and as you insist, I come up with even better ones (I think I have to excent the FAQ now). So in the end what you do is counterproductive.. I've come up with even more reasons against anamorphic encoding in this very thread. I have also come up with more ammo for Q1, Q11 and Q12 based on discussions in this very forum.

Take the tables for the numbers: without tables it is hard to read (everybody agrees with that), adding tables requires only a small amount of work while improving readability considerably. Now that is a positive suggestion (and it does appreciate that changing something actually involves work).

However, had I previously written that I cannot be bothered to add a table then it would be useless to even mention it. It's like people telling len0x to add WMA audio or WMV video to AutoGK. He has previously stated that he won't do that so asking is just a waste of time. And most people don't appreciate if you waste their time.

I know what you're trying to do.. nag away until I change my mind. But that can have two endings, one being that the sheer mention of change makes me shut my ears. For those who care: my MBT type is INJT, so that means I think a lot before I write and thus the answers to my FAQ questions are well thought out and chances of my reversing my opinion are slim. Heck, I even offered the possibility of increasing resolution and I was thinking about anamorphic encoding for a 1 CD source but this constant pestering is really getting on my nerves so that I'm seriously considering to drop that idea just to get back at all the people who've gone on my nerves.

@mf:I was accused of writing unreadable posts as well so I try hard to improve... a few more linebreaks would also do your post some good.

Oh, and not reading an FAQ is a rule 1 violation :devil:

SeeMoreDigital
7th January 2004, 15:48
Originally posted by Doom9
@SMD: You, and everybody else who just can't keep their mouth shut about what they think is better should have a look at this thread and specifically colordog's posts: http://forum.doom9.org/showthread.php?s=&threadid=67634. Everyone complaining reduces the chance that I'll ever bother making another comparison. I don't need suggestions that I have already considered, or stuff that's simply useless. My dear Doom9, your tone appears to be getting a little hostile!

Nobody is belittling all the time and effort you continually put into 'your' forums and websites. But you cannot expect everybody to share the same views as your good self, and/or go about testing in the same way as you!

I thought one of the aspects of having a democratic forum is to openly discuss matters such as this and to be able to put all opinions forward.

Standard anamorphic (720x480/576) encoding is just the first step in seeing how well todays codecs perform when pushed to their limit!

Given that high-def images are gaining in popularity. I think it's fair to say that there are many of us here, on your forum, who are interested in generating encodes using larger image pixel frame sizes than even standard anamorphic. And would like to know how well the new versions of all the codecs perform!

And already it's clear to me that some codecs perform much better than others in this area. Take WMV9 for instance. It's an established fact that this codec can cope much better with larger image pixel frame sizes than say DivX or XviD, in both encoding and playback. So can RV9 and so can VP6.

In a few weeks time I hope to be upgrading from my WVGA (852x480)42" plasma to an WXGA (1280 or 1366x768) plasma screen.

As a PAL user I currently have to compromise my DVD viewing by squeezing down from 576 vertical pixels to 480 vertical pixels. And anybody who has had the opportunity of viewing say the high-def WMV9 version of T2 Extreme on an WXGA (or even WUXGA) screen, can't help being astonished at the quality of the encoded images. Images that just are no possible to view using less pixels!

In conclusion Doom9. Many of us regard you as an independent judge/voice when it comes to video encoding. Many of us look at your test results and obviously use them as a basis to generate and compare against our own encodes. After 25 years in this business I know how easy it is to get upset when somebody says or suggests something you don't like. I can only speak for myself when I say, I have far too much respect for the work that you do to upset you on purpose!


Kind regards

SeeMoreDigital

dragongodz
7th January 2004, 16:12
Doom9 may i humbley suggest you not bother to read this thread anymore ? because the stress levels emeninating from your posts have get to the point where you appear to be ready to burst a blood vesel(at least).
now take 10 slow deep breaths and try to relax.

"the aspects of having a democratic forum"
SMD - democratic forums are blood baths of bad language and flame wars. i dont think anyone really wants that here do they ? open discussion is good but going on once a decision has been made is pointless unless you can come up with a very good NEW reason why the decision should be changed.

SeeMoreDigital
7th January 2004, 16:50
Originally posted by dragongodz
....open discussion is good but going on once a decision has been made is pointless unless you can come up with a very good NEW reason why the decision should be changed. I thought I had forwarded a NEW reason for requesting a 'codec shoot-out' using larger image pixel frame sizes (whether they be standard anamorphic frame sizes or even larger). But just to underline my reason, it is because of high-def sources.

I really don't fancy the prospect of encoding my high-def 1920x1080 or 1280x720 'full 16:9 frame' images to say 640x368. I also don't fancy the prospect of storing hundreds of GB's of raw Mpeg2 streams on dozens of hard drives. So I would greatly appreciate some quality, hopefully impartial, information about alternative methods/codecs.

And I don't think I am alone in this line of thinking. But as usual I'm willing to put my head on the block by suggesting it!

Cheers

bilu
7th January 2004, 18:37
It's time for specialized benchmarks: high-def, anime, interlaced, live-video, and others. There is demand for them.

But Doom9 itself has no obligation in this respect. Others should do them, post the results, and MAYBE Doom9 would be kind enough to host them, if he has spare resources. But specialized stuff should be left in the hands of others - Doom9 has done more than enough already. And has expressed that more than once.


Bilu

Kedirekin
7th January 2004, 19:13
Very well said.

Doom9
7th January 2004, 19:47
Given that high-def images are gaining in popularity. I think it's fair to say that there are many of us here, on your forum, who are interested in generating encodes using larger image pixel frame sizes than even standard anamorphic. And would like to know how well the new versions of all the codecs perform!

And already it's clear to me that some codecs perform much better than others in this area. Take WMV9 for instance. It's an established fact that this codec can cope much better with larger image pixel frame sizes than say DivX or XviD, in both encoding and playback. So can RV9 and so can VP6.

Woah, now you have actually ticked my interest. Needless to say, the test are DVD encoding tests and for that they make a lot of sense.

Now unfortunately we don't get any HDTV in good 'ole Europe so all I get is lower than DVD bitrate DVB usually having a lower than DVD resolution (I'm not really sure about that statement right now.. I haven't installed my DVB card yet, I only know about the bitrates). So I guess I'd need a legit HDTV stream source. However, I do have a handful of XviD files that were taken from HDTV sources (don't worry... I have bought the DVD since so I have at least one license for that content), at resolutions that go beyond that of a DVD and I didn't see any problem with XviD.

I do have my honest doubts about your established facts though, but just for that reason I'd like to test (but not in a form of a codec comparison.. that's way too much work) and disprove such statements.

Just guess why there are no comparable tests available anywhere? Who is willing to sacrify one month of free time for something you don't even get paid for? Did you notice that I never commented on other comparisons in this very forum? There are two reasons for that: a) I prefer to see for myself, and b) I know how frustrating it is if everybody seems to know better than the guy who tested (and you bet your ass I'd find a couple of points I'd do differently). So I'd rather bite my tongue than to make somebody else feel the way certain people make me feel.

SeeMoreDigital
7th January 2004, 19:47
Yes, agreed.

But I think in order for the tests to be fair, each encode will have to be assessed and the results/findings announced from one impartial voice.

I feel that Doom9 (if he can spare the time) would be the ideal voice for such an undertaking.

Now we all know that generating high-def full movie encodes are not going to fit onto an single 700MB CD-R. So maybe a 4.7GB (4470MB approx) DVD~R would offer more realistic encoding bitrates and give us all something to aim for. Especially as both the DVD burners and DVD blank media are getting cheaper with each passing month!

Anybody else have any thoughts about this that we could move onto another thread?

Cheers

EDIT: Doom9 you generated your response while I was writing the above. I'm sorry if we got off on the wrong foot. I guess I just assumed that you knew what I was eluding to!

I hope that both your good self and other members now understand my enthusiasm for the tests now and why I (in the main) encode using such large image pixel frame sizes. It's not because I think they look better (as they obviously do not) it's because that's where my current interest lies and where I think we will be heading in the near future!

As you may also be aware 'large' frame sizes are unfortunately limited to just 720x576 pixels (in hardware) because the current crop of decoding chipsets are limited to output only 414,720 pixels.... bummer!

Which is why I'm interested in both the proposed new Chinese VP6/DVD recorder/players and BluRay.... I wonder what chip sets they will be using and who is manufacturing them. As each technology will need to both encode and decode using more than just 414,720 pixels!

Cheers for now

Doom9
7th January 2004, 20:16
Now we all know that generating high-def full movie encodes are not going to fit onto an single 700MB CD-R. So maybe a 4.7GB (4470MB approx) DVD~R would offer more realistic encoding bitrates and give us all something to aim for. Especially as both the DVD burners and DVD blank media are getting cheaper with each passing month!
Obviously the bitrate would have to be higher, or else we'd be back to the dreaded 300kbit/s RV9 tests from a bits/pixel standpoint. And while using a DVD makes a certain amount of sense I think it might be a bit too high. Just as a guess I think 2-2.5mbit might be enough for a HD source, and that normally doesn't fill an entire DVD-R if I remember my CCE -> DVD-R bitrates correctly.

SeeMoreDigital
7th January 2004, 20:50
Originally posted by Doom9
... Just as a guess I think 2-2.5mbit might be enough for a HD source, and that normally doesn't fill an entire DVD-R if I remember my CCE -> DVD-R bitrates correctly. Yes I think we could get away with the bitrate you suggest. But then there will always be people who will want to squeeze two (or more) 120min HD encodes onto a single DVD~R. In much the same way people now like to put a 120min movie onto a single 700MB CD~R.

For me, the logic behind putting a 120min movie onto a single 700MB CD~R is because I can't be arsed, getting off my fat arse, to change over to another disc. I suspect others might feel the same!

Anyway the HD tests I've generated used the 1440x816 image pixel frame size of the T2 high-def trailer as a source (availble from the M$ website). All of which I have converted to DivX, XviD, RV9 and VP6.

My tests have so far revealed that 1440x816 (which is all image/no matte) is a little too much for both DivX and XviD (1280x720 is about their limit) but there is no such problem with a 2pass RV9 and 1pass VP6 encode!

Cheers

Soulhunter
8th January 2004, 19:56
Originally posted by Doom9
Just guess why there are no comparable tests available anywhere? Who is willing to sacrify one month of free time for something you don't even get paid for?
I would do it...

If I had the bandwidth for it... Uploads with a 56k connection sucks !!! :(

Also, I would need some HD sources... I live also in good 'ole Europe !!! ;)

Bye

SeeMoreDigital
8th January 2004, 20:17
For all those who may be interested, I've started a new thread about the possibilities of creating HD lossy encodes (using RV9, WMV9, Mpeg4, VP6 etc) from an HDTV source here: -

http://forum.doom9.org/showthread.php?s=&threadid=68292

Cheers

mf
8th January 2004, 23:21
Originally posted by Soulhunter
If I had the bandwidth for it... Uploads with a 56k connection sucks !!! :(
I have only double your upload. And I also considered doing an anime shootout (same style same rules as doom9's), but came to the conclusion I'm not really interested in how other codecs perform since the only thing that's a challenge to xvid (high detailed backgrounds from PROPER raws instead of the smoothed stuff that fills 90% of the raw population) is something I'm certain of (especially after reading doom9's test) that other codecs won't even be able to reproduce authentically. The only interesting thing left then is low bitrates but as a quality freak I wouldn't even want to test those (besides, Deen() should be as good as those crazy in-loop filters for anime). So I decided against it. It would still be interesting to read such a test though :D.