Thor: a new codec from Cisco (reference implementation available)

[MoSal]

Cisco presented Thor, a new royalty-free video codec, at the IETF93 meetings in Prague. A high-level description of the codec was submitted as a draft earlier this month.

An open-source reference implementation is now publicly available at github.

The draft's abstract describes the codec as:

Thor is designed to achieve high compression efficiency with
moderate complexity, using the well-known hybrid video coding
approach of motion-compensated prediction and transform coding.

Other than working with the IETF, which implies the willingness to write a usable standard document. I don't think Thor will have any advantages over VP9/VP10. However, some techniques that Thor uses might prove usable to other codecs under development. Daala developers are already experimenting with a couple of those techniques.

[MoSal]

The IETF93 NETVC meeting at which Thor was presented:
http://recordings.conf.meetecho.com/...pter=chapter_1

The slides:
https://www.ietf.org/proceedings/93/...93-netvc-4.pdf

[wiak]

compiled a windows build today
http://nwgat.ninja/thor-video-codec/

[dapperdan]

Interesting, at first glance it appears to be an attempt to tempt existing IP holders in H.264/H.265 to defect to royalty-free since the existing processes have stymied that for the last decade or so.

Bold move.

[pieter3d]

That IETF document is really sparse on details. It appears to be pretty simple, not really any new features. If I had to guess it is probably in between H.264 and HEVC/VP9 in terms of compression efficiency.

[wiak]

Originally Posted by pieter3d

That IETF document is really sparse on details. It appears to be pretty simple, not really any new features. If I had to guess it is probably in between H.264 and HEVC/VP9 in terms of compression efficiency.

yep, did you read the daala hackathlon pdf? hehe

[pieter3d]

I looked at the code - it appears to be largely a dumbed-down version of HEVC. There are even constants and macros in the code with the name HEVC in it.

[wiak]

Originally Posted by pieter3d

I looked at the code - it appears to be largely a dumbed-down version of HEVC. There are even constants and macros in the code with the name HEVC in it.

alot of codecs these days share techniques
yet some, like daala or diarc are alot different, the lapped transform is a pretty interesting technology

[Tommy Carrot]

Thanks for the build, wiak.

I ran a few tests with the encoder. It is horrendously slow in the high efficiency settings, and it has a few issues (cuts off the last few frames if b-frames are enabled), but the quality is pretty good, much better than the current daala encoder, it's actually not that far from x265. Considering the immature state of the encoder (non-adaptive b-frames, no scenecut detection, no psy-rdo), it's quite promising.

[wiak]

Originally Posted by Tommy Carrot

Thanks for the build, wiak.

I ran a few tests with the encoder. It is horrendously slow in the high efficiency settings, and it has a few issues (cuts off the last few frames if b-frames are enabled), but the quality is pretty good, much better than the current daala encoder, it's actually not that far from x265. Considering the immature state of the encoder (non-adaptive b-frames, no scenecut detection, no psy-rdo), it's quite promising.

ooh, well the current daala encoder is basicly a experimental patch work
and the current thor code was posted to githu a week ago

its good that there are more codecs at IETF, the Thor MC seem to fix issues with the current Daala MC

[dapperdan]

Originally Posted by pieter3d

I looked at the code - it appears to be largely a dumbed-down version of HEVC. There are even constants and macros in the code with the name HEVC in it.

Cisco contributed to H.264/5 and have chosen to freely licence that IPR in Thor and any outcome of the NetVC project.

They've also had teams of lawyers deciding what bits of H.264/5 are freely reusable.

So their strategy of sticking close to H.264/5 makes a lot of sense. If they can tempt any of the other IP holders to join in, then it'll make even more sense.

The blossoming of lots of interesting new codecs will happen after the royalty-bearing patent stranglehold on the industry us broken.

[Bloax]

Originally Posted by dapperdan

The blossoming of lots of interesting new codecs will happen after the royalty-bearing patent stranglehold on the industry us broken.

What do you mean patents strangle progress?

Maybe you're in with those filthy pirates that say that copyright hinders creativity!!

*cough*

Based off Tommy Carrot's little writeup this sounds like a very exciting entry to the field, though.
Paint me interested.

[MoSal]

Originally Posted by Tommy Carrot

Thanks for the build, wiak.

I ran a few tests with the encoder. It is horrendously slow in the high efficiency settings, and it has a few issues (cuts off the last few frames if b-frames are enabled), but the quality is pretty good, much better than the current daala encoder, it's actually not that far from x265. Considering the immature state of the encoder (non-adaptive b-frames, no scenecut detection, no psy-rdo), it's quite promising.

Unless someone starts making some crazy claims, I don't think it's very useful at this point to compare Daala with other (traditional) codecs quality-wise.

There are many basic features that are not implemented yet, or still need a lot of work. 64x64 blocks, multiple-references, mixed-predicion, bi-prediction, better intra-prediction, better MC, better MPs, and finally, that promised efficient deringing filter. All this work is on the roadmap. And the plan was to finish most of it by the end of this year. But I think that's a little bit too ambitious of a goal.

[Tommy Carrot]

Originally Posted by MoSal

Unless someone starts making some crazy claims, I don't think it's very useful at this point to compare Daala with other (traditional) codecs quality-wise.

You are right of course, i was just surprised that this codec, despite being almost as early stage in the development as daala, produces such a good quality. It is possible though (i haven't looked at the source code) that many parts were borrowed from the HM encoder, leaving out the features that they considered patent-risks. That would explain the good performance, and the similar characteristics to h.265.

There are many basic features that are not implemented yet, or still need a lot of work. 64x64 blocks, multiple-references, mixed-predicion, bi-prediction, better intra-prediction, better MC, better MPs, and finally, that promised efficient deringing filter. All this work is on the roadmap. And the plan was to finish most of it by the end of this year. But I think that's a little bit too ambitious of a goal.

Daala has improved immensely in the last year or so, but it's still not comparable to the mature codecs. They started to fall behind the roadmap in the last few months though, so yes, i'd not expect to get the main features finished for at least a year from now or so.

[foxyshadis]

Originally Posted by Thor Spec

For the 32x32 and 64x64 transform sizes, only the 16x16 low frequency coefficients are quantized and transmitted.

Interesting idea that could be implemented in HEVC today (explicitly setting them to 0 instead of just tossing them). I wonder if this would have any effect on quality, since those really are mostly insignificant. If Thor intends to set them to random low-level noise on decoding, though, that's impossible for HEVC to copy.

[Gravitator]

Originally Posted by foxyshadis

Interesting idea that could be implemented in HEVC today (explicitly setting them to 0 instead of just tossing them). I wonder if this would have any effect on quality, since those really are mostly insignificant. If Thor intends to set them to random low-level noise on decoding, though, that's impossible for HEVC to copy.

Random noise? Why not transfer the approximate noise filter / decoder...

[dapperdan]

I guess it's just a coincidence that the name fits the Xiph naming scheme of strong female characters from pop culture?

[wiak]

this "MC" is a interesting case of what works works for others tooo
http://i.imgur.com/X6Nzlcf.png
http://i.imgur.com/D45fMKi.png

this forum reaily needs a img scaler (like gmail has)

[foxyshadis]

Originally Posted by Gravitator

Random noise? Why not transfer the approximate noise filter / decoder...

Because there's no point, you don't need to bother. In high frequencies, low level random noise is indistinguishable from grain or shaped noise, and since grain doesn't much change across a single frame, all you need to transmit is one frame-level variance value to fill in all the missing coefficients. It also helps cover up other sources of noise, including compression noise. It's so consistent you can mostly get away with one value for the whole movie, but that's really stepping away from reproduction into stylized.

Shaped noise (and explicit coefficients) matters more for the lower frequencies, for synthesizing film grain and texture, which is usually more trouble than it's worth except in extremely low-bandwidth video. FGM suffers from being complicated enough to be all things to all users, so no one wants it, when a simpler noise generator might have succeeded. I was pretty disappointed when HEVC completely punted on even trying to improve it.

Alternately, instead of messing with the idct you could just sweep random noise across each decoded frame in the same way; it looks nearly as good, but more 'digital' which turns some people off.

Obviously, this doesn't apply to CGI and film stock with low noise and sharp detail. Also obviously, I don't subscribe to the cult of PSNR and perfect reproduction, not when it leaves better subjective quality on the table.

[colours]

I went through the presentation slides and this bit jumped out at me: Thor uses a nonseparable low-pass filter to compute a (0.5,0.5) offset (for luma only). Looking at the source code points to the kernel being [[0,1,1,0], [1,2,2,1], [1,2,2,1], [0,1,1,0]]/16. No negative coefficients at all?

I'm kinda curious about how much this strange choice of a nonseparable interpolation filter for the (0.5,0.5) offset improves on just using the separable 6-tap interpolation filter which is also used for all the other fifteen offsets.