Theora: Thusnelda project update 20090507

[buba king]

Interesting updates from the Thusnelda project:
http://web.mit.edu/xiphmont/Public/theora/demo7.html

Seems like Theora is making some progress again. The PSNR chart is quite impressing if it holds true. According to the PNSR chart Theora-svn beats x246 at some bit rates. Of course they don't tell us what kind of settings are used so take it with a truckload of salt. :)

Still, I think it's really great that a completely FREE codec is making so much progress!

Comments?

[Dark Shikari]

They apparently used the worst possible x264 settings (yes, subme 0 and so forth) in order to make Theora look better--if Theora didn't win such a test I would be shocked indeed! Instead, they just proved the fact that they're a bunch of liars who are no better than the worst of the proprietary companies they claim to compete against.

[benwaggoner]

They apparently used the worst possible x264 settings (yes, subme 0 and so forth) in order to make Theora look better--if Theora didn't win such a test I would be shocked indeed! Instead, they just proved the fact that they're a bunch of liars who are no better than the worst of the proprietary companies they claim to compete against.
Where did they list the x264 settings they used?

I also took a gander at their test library:
http://media.xiph.org/video/derf/

Those are a lot of very short clips. The longest SD is 2101 frames: just 84 sec (assuming 25p)! Lots of them are 300 frames, and some 115 frames. That's not going to be a realistic test for VBR rate control, or even VBV in some cases. I hope they concatenate them together or use other longer content for tuning.

Anyway, their test library is right there if you want to make some counterexample demos :).

[Dark Shikari]

Correction: it turns out their PSNR tool was broken (http://www.reddit.com/r/programming/comments/8iphn/theora_encoder_improvments_comparable_to_h264/c09eyvc) and shaved over 4db off of x264 (but not Theora) :p

I have the guy who made the test working on correcting it right now.

[benwaggoner]

Correction: it turns out their PSNR tool was broken (http://www.reddit.com/r/programming/comments/8iphn/theora_encoder_improvments_comparable_to_h264/c09eyvc) and shaved over 4db off of x264 (but not Theora) :p

I have the guy who made the test working on correcting it right now.
You've had quite a fortnight of x264 being assaulted by confused metrics :).

Are they doing a better optimization of settings at the same time? They should really concatenate a bunch of clips together to make this a more interesting test as well.

[Dark Shikari]

You've had quite a fortnight of x264 being assaulted by confused metrics :).

Are they doing a better optimization of settings at the same time? They should really concatenate a bunch of clips together to make this a more interesting test as well.Apparently this wasn't a test of x264 versus Theora at all. Instead, it was a blog post designed to debunk this incredibly awful paper (http://etill.net/projects/dirac_theora_evaluation/include/halbach-2009-dirac_theora-paper.pdf) that claimed H.264 had up to a 20db advantage over Theora.

But then another blogger hijacked the graph and posted it to demonstrate that Theora was now as good as x264. And away it went... :rolleyes:

[benwaggoner]

Ah, one of those...

Well, as long as we're discussing it, what's your take on how far do you think Theora could go? I've been thinking of it being about ASP quality at best; I don't see many tools in there that would let it pull past ASP to make up for the missing B-frames. But it's weird enough that I don't feel confident I understand why it is the way it is, which implies there must have been some magic encoder tweaks they were planning to implement. Superblocks with Hilbert patterns? Encoding bottom-up? They don't seem wrong exactly, but I just don't get why.

It's kinda amazing we're still talking about this bitstream. I was a beta tester for VP3, what 11 years ago? It's kind of like a resurgance in developement for Sorenson Video 3, Indeo 5, or ClearVideo...

[Dark Shikari]

Ah, one of those...

Well, as long as we're discussing it, what's your take on how far do you think Theora could go? I've been thinking of it being about ASP quality at best; I don't see many tools in there that would let it pull past ASP to make up for the missing B-frames. But it's weird enough that I don't feel confident I understand why it is the way it is, which implies there must have been some magic encoder tweaks they were planning to implement. Superblocks with Hilbert patterns? Encoding bottom-up? They don't seem wrong exactly, but I just don't get why.It definitely won't get anywhere near ASP. The entropy coder is likely worse, the motion compensation is definitely worse, the transform is no better, and so forth. The bizarre block ordering and bitplane coding doesn't help.

Straight from the horse's mouth:<xiphmont_> Vorbis still stands up nicely. Theora, OTOH, is a a bit embarrassing.
<xiphmont_> rather, it's a bit embarrassing until you look at the code, then it's alot embarrassing.
<xiphmont_> and that's 70% 'really f***g stupid encoder, really On2, be ashamed' and 40% 'format design flaws'. It's so bad it adds up to 110%.
<xiphmont_> I plan to help Theora limp along not too embarrassingly until it can be replaced for real-- possibly 2-4 years.
<xiphmont_> Theora is actually fixable tho. The amount of low-hanging fruit is staggering.
<xiphmont_> I mean, an entropy backend that results in *more* bits being written than went in? It's just... wow.

I would expect they could possibly get competitive with Xvid if they use sufficient psy optimizations to make up for their lack of real efficiency.It's kinda amazing we're still talking about this bitstream. I was a beta tester for VP3, what 11 years ago? It's kind of like a resurgance in developement for Sorenson Video 3, Indeo 5, or ClearVideo...Indeed...

[squid_80]

You think the theory is bad, try using the library. The documentation includes this note: Security Warning: The Theora format supports very large frame sizes, potentially even larger than the address space of a 32-bit machine, and creating a decoder context allocates the space for several frames of data. If the allocation fails here, your program will crash, possibly at some future point because the OS kernel returned a valid memory range and will only fail when it tries to map the pages in it the first time they are used. Even if it succeeds, you may experience a denial of service if the frame size is large enough to cause excessive paging. If you are integrating libtheora in a larger application where such things are undesirable, it is highly recommended that you check the frame size in _info before calling this function and refuse to decode streams where it is larger than some reasonable maximum. libtheora will not check this for you, because there may be machines that can handle such streams and applications that wish to. This is what I think they should have actually put: We're too lazy to check our mallocs, and we do a lot of them. This will probably cause a crash at some point, unless your app is psychic and knows how much memory our library will require based solely on the resolution.

[akupenguin]

But it's weird enough that I don't feel confident I understand why it is the way it is, which implies there must have been some magic encoder tweaks they were planning to implement. Superblocks with Hilbert patterns? Encoding bottom-up? They don't seem wrong exactly, but I just don't get why.
You're reading too much into it. Most of the things Theora does differently from mpeg are done solely for the purpose of being different, not to improve compression.

[Sagittaire]

there are executable for make test ... ???

[benwaggoner]

You're reading too much into it. Most of the things Theora does differently from mpeg are done solely for the purpose of being different, not to improve compression.
Patent avoidance?

[*.mp4 guy]

Probably, Theora has only two real advantages over ASP, and a lot of obvious problems. The advantages being a not particularly good loop filter, and completely arbitrary quantizer scaling and matrices, the last feature could be exploited a lot, but it would take a lot of work to do so.

[Dark Shikari]

Actually, it's worse than I thought. xiphmont just told me Theora has no MV prediction.

NONE.

Every MV is coded as either 6-bit X and 6-bit Y, or with a global static huffman table. This is worse than MPEG-1.

I retract my statement that Theora can ever get near MPEG-4 ASP. Removing MV prediction from x264, by the way, reduces PSNR by 1db at 500kbps on BlackPearl.

[CruNcher]

What is funny and somehow sad @ the same time is that most of these problems have been fixed/improved for sure already by ON2 themselves i mean come on 11 years ;) and someone is sitting there (monty) and tries to go their same way, he should get more involved into Dirac then trying to reach VP6 state so skip theora and make Dirac the new OS standard and implement it into HTML5 that should be the goal not trying to get this old dumped bistream to it's newer successor quality, how crazy is that ? Redhat should sponsor more into Dirac, but instead they let monty doctoring on Theora crazy that they don't seem to understand what a complete waist of time and resources this is.

[benwaggoner]

What is funny and somehow sad @ the same time is that most of these problems have been fixed/improved for sure already by ON2 themselves i mean come on 11 years ;) and someone is sitting there (monty) and tries to go their same way, he should get more involved into Dirac then trying to reach VP6 state so skip theora and make Dirac the new OS standard and implement it into HTML5 that should be the goal not trying to get this old dumped bistream to it's newer successor quality, how crazy is that ? Redhat should sponsor more into Dirac, but instead they let monty doctoring on Theora crazy that they don't seem to understand what a complete waist of time and resources this is.
Do we have any sense of Dirac's potential? Interframe wavelets seem to have been the downfall of many promising-sounding codecs.

I heard someone has a Dirac MediaStreamSource implementation for Silverlight too. This new Raw AV pipeline is going to force me to learn about too many new codecs :)!

[*.mp4 guy]

Dirac is, if anything, less likely to become useful at any point in the future then theora. When I tested it it was still worse quality then snow, though its been developed for multiples of time longer, it was also still incredibly slow, compared to snow, or anything else. If there are settings for it that give good quality, they must be unimaginably slow, and before anyone says anything, I was using schroedinger, as that is supposed to be the better and faster implementation.

[Dark Shikari]

Do we have any sense of Dirac's potential? Interframe wavelets seem to have been the downfall of many promising-sounding codecs.

I heard someone has a Dirac MediaStreamSource implementation for Silverlight too. This new Raw AV pipeline is going to force me to learn about too many new codecs :)!Dirac has even more problems than Theora.

1. Interframe wavelets suck.
2. Intraframe wavelets suck more.
3. Intra wavelets in inter frames suck even more. No intra prediction.
4. BBC has practically given up on Dirac.
5. Encoder doesn't even implement all of the Dirac spec, let alone well.
6. Doesn't have variable motion partition sizes.
7. Crappy entropy coder.
8. Over-complex motion compensation (even worse than Snow's OBMC in terms of computational cost).
9. Still not better than Snow.

[dragsidious]

I was using schroedinger, as that is supposed to be the better and faster implementation.

Schroedinger encoder is designed to increase speed at the expense of quality. The 'research' codec is the one that is designed for quality and to show new developments.

I don't think I've ever ran into a Dirac-encoded video yet in the wild.. anywere. Even is 'Free Software' circles, which Theora is pretty common.


What is funny and somehow sad @ the same time is that most of these problems have been fixed/improved for sure already by ON2 themselves i mean come on 11 years and someone is sitting there (monty) and tries to go their same way, he should get more involved into Dirac then trying to reach VP6 state so skip theora and make Dirac the new OS standard and implement it into HTML5 that should be the goal not trying to get this old dumped bistream to it's newer successor quality, how crazy is that ? Redhat should sponsor more into Dirac, but instead they let monty doctoring on Theora crazy that they don't seem to understand what a complete waist of time and resources this is.

I think that between Dirac and Theora Redhat figured that Theora has the best bet of actually being used by people. Simply because it exists currently and the encoding performance is not so bad to make it unusable.

Redhat, first and formost, is a "Open Source Software" company. Everything they touch they open source. Sure they charge money for binary downloads, but your not paying for the license your paying for a year of support. Everything they provide you can download, free of charge, from their ftp sites.

As part of what makes 'OSS' useful is the ability to use it with as minimal amount of restrictions and you need to be free to modify and redistribute the software.

I suppose Redhat is looking to ship products that include the ability to stream encoded video. If they used something like H.264 then they would not only be forced to pay licensing costs they would be hurting the end user's ability to modify and redistribute working examples of the software legally. So weither or not Theora is actually competitive is completely secondary. The fact that it is 'Free' in the open source way of looking at things is much more important. Of course if pushing a relatively small amount of money towards the goal of improving Theora will yeild good returns then that is probably worth it in Redhat's eyes.

I expect that, along with pretty much all open source software that I've ever seen in relation with media or whatnot, that they won't restrict the user's ability to add H.264 or anything else at the customer's own risk. Redhat just won't do it by default.

And it's not like the Thusnalda project has not yeilded some successes. Even though I seriously doubt that Theora will ever be a rival to H.264 they have boosted the quality quite a bit.

[Selur]

So weither or not Theora is actually competitive is completely secondary.
Not if you look at it from a video encoding forum perspective, people here like codecs that can give a good quality/size ratio. :)

[IgorC]

I think this PSNR graph isn't representative.
First of all resolution was QCIF 176 × 144
Second Akiyo sample is very low motion video and consequently very easy to compress.
From my experience Theora has big problem with high motion. Quality is droped too much.

Today Theora isn't on par even with Xvid. Let alone x264.

http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=70809&view=findpost&p=625394

[benwaggoner]

Not if you look at it from a video encoding forum perspective, people here like codecs that can give a good quality/size ratio. :)
Also, while I've heard plenty of assertions that Theora is patent free, I've not actually heard anyone who's an expert on video patent IP validate those assertions.

That aside, there's a real cost advantage from improved compression. Any publisher picking a codec would compare the H.264 license fee for them versus the incrased bandwidth costs and reduced audience capable of hitting a minimum quality level. Plus compare the installed base of platforms/devices with the two decoders.

I don't have any stake in this came myself; Silverlight's my baby, and with Raw AV we'd be able to host Ogg + Theora + Vorbis as well as anything else..

But if patent status was that compelling, why not MPEG-1 or H.263? It seems like there are other codecs that are more clearly free-to-implement and have much more refined implementations than Theora or Dirac. Or at least using H.263 as a base and innovating from there.

And I haven't heard much to suggest that Theora could substantially outperform a well-tuned H.263 (MPEG-4 pt Short Header after all...), and we actually have well-tuned H.263 encoders. How's xvid in Short Header mode?

[Dark Shikari]

Also, while I've heard plenty of assertions that Theora is patent free, I've not actually heard anyone who's an expert on video patent IP validate those assertions.I'd personally say Theora is worse-off patent-wise than H.264.

H.264 is the the devil we know. We know exactly what patents cover it because all the companies involved were required to submit their IP as part of the process. There aren't any surprises.

Theora is the devil we don't know. Nobody has any idea what patents it covers. Nobody has done a real analysis. All we have is Xiph's word that it doesn't violate anything, which is pretty much meaningless.

[dragsidious]

But if patent status was that compelling, why not MPEG-1 or H.263? It seems like there are other codecs that are more clearly free-to-implement and have much more refined implementations than Theora or Dirac. Or at least using H.263 as a base and innovating from there.


I am not that familar with the patents around codecs, but I think that H.263 is still patent encumbered. MPEG-1 layer 3 audio is certainly still patent protected and while _legal_ open source implimentations exist (the codecs from Fluendo) they are still subject to redistribution restrictions because of the patent licensing. Although Mpeg-1 is unemcumbered and it's patents are expired.

As for Theora vs Mpeg-1 for streaming internet video.. I don't know which one is better, but I suspect Theora is.


I'd personally say Theora is worse-off patent-wise than H.264.

H.264 is the the devil we know. We know exactly what patents cover it because all the companies involved were required to submit their IP as part of the process. There aren't any surprises.

That is a useful viewpoint for a lot of situations, but it's not really valid in this one.

Remember the goal here is to create something that people can redistribute and modify without restrictions. That is hack it, patch it, sell it, bundle it with consumer goods and services, use it in a professional capacity etc etc. If you try to do that with a H.264 codec then you will have a 100% chance of violating patent law. With Theora you can still use it, right now, in that manner legally.

Nobody has stepped forward yet with claims.

The problems with doing patent searches is that you can never be 100% sure what the patents actually mean. Even if you hire a lawyer who knew what they are doing you can't be sure. There are literally hundreds of new software patents each month and many are not limited in scope to specific programs... that is it's quite possible that a method of, say, compressing text in a database for heuristic searches could apply to a video codec. With literially tens of thousands of new software patents every year the chances of you violating somebody elses patents somewhere, by complete accident, is about 100% for any non-trivial program.

Also the fact that people are pushing patents licenses on H.264 does not isolate you in any way from patent lawsuites coming from other parties. Microsoft has learned that a couple times with regards to MP3...

And there is a trick here:

If you are show in court to violate a patent on purpose your pentalties can be tripled. So if you do end up doing patent audits and later on a court case shows that you do, indeed, violate a patent then that patent search/audit can come back to burn you. It will be easy for the other side to show you did compare the patent against your code and you still violated the patent... so it wouldn't be hard to convince a judge that you did it on purpose and was simply hoping not to get caught. So the general approach that has been recommended to the open source crowd by lawyers is to don't violate patents that people are publicly enforcing, but don't go fishing for patents to avoid either... patent audits and searches will have nearly zero benefit.

So in this case, were you want to have something people can modify, use, and redistribute freely, then yes not knowing you violate licensing is much much much better then knowingly violating patents, and that is the difference between something like Theora vs H.264


--------------------------

Keep in mind that in my personal life, the things I use and create... pfff I've never used Theora for anything. Divx-style or H.264 all-the-way. I've ripped movies for copying them to my cell phone or using on my netbook and I am always trying to find a nice balance between speed, efficiency, compatibility, and file sizes. The avialable tools for Theora simply don't match what I can get withj other open source stuff.

But if I was to recommend doing something like realtime internet streaming to my place at work, for some personal commercial gain, or some non-profit (etc) and we couldn't afford the licensing fees (or in the case for H.264, future licensing fees) then Theora would be the best choice.

Of course that is in the USA. In other counties they are not quite so patent insane so it varies.


I am not a lawyer, obviously. (thank god)

[Dark Shikari]

Remember the goal here is to create something that people can redistribute and modify without restrictions. That is hack it, patch it, sell it, bundle it with consumer goods and services, use it in a professional capacity etc etc. If you try to do that with a H.264 codec then you will have a 100% chance of violating patent law. With Theora you can still use it, right now, in that manner legally.

Nobody has stepped forward yet with claims.But while Theora is the devil we don't know, and H.264 is the devil we know, MPEG-1 is out of patent period and is guaranteed to be safe.

[benwaggoner]

But while Theora is the devil we don't know, and H.264 is the devil we know, MPEG-1 is out of patent period and is guaranteed to be safe.
Anyone done a RD curve against modern MPEG-1 implementations for webish bitrates? I wonder how it would do coupled with a decent postprocessing filter.

Back when I did the Grolier's CD-ROM encyclopedia encodes in MPEG-1 (96-98?), I was able to do pretty good looking 320x240p24 down around 800 Kbps VBR with Heuris MPEG Power Professional. And that was not a good encoder compared to what came in later years once the DVD boom kicked in.

I wonder how it would compare to current Theora builds. Theora certainly has a lot more low-hanging fruit, but if the goal is something unambiguously free to implement...

[benwaggoner]

Anyone done a RD curve against modern MPEG-1 implementations for webish bitrates? I wonder how it would do coupled with a decent postprocessing filter...
Oh, how soon we forget. I just did a test encode, and MPEG-1 and MP2 are still crap at any reasonable web bitrate, even with 2-pass VBR etcetera.

[Shapierian]

But while Theora is the devil we don't know, and H.264 is the devil we know, MPEG-1 is out of patent period and is guaranteed to be safe.

Even MPEG-1 is still susceptible to yet unseen submarine patents filed under the old pre-1995 submarine friendly rules. If course such a submarine would most likely also torpedo any other codec.