The Next Generation Internet Video (NGIV) Codec project

[schweinsz]

The Next Generation Internet Video (NGIV) Codec project is new video coding project that is designed to get or exceed the new generation international video coding standard, HEVC/H.265 on the coding efficiency while maintaining the similar the enc./dec. complexity. Currently only the intra-prediction part is finished partly.
The main features are as follows:
The max coding unit is 64x64.
High flexible variable block size intra prediction and coding.
Chroma splitting tree is undependent on the Luma splitting tree.
Line by line/colomn by colomn curved intra prediction.
40 directional intra prediction for luma and 6 directional for chroma.
the KLT transform.

Currently the deblocking filter is not finished. I am trying some advanced intra prediction algorithms to get higher coding efficiency. After I finish these, I will finish the deblocking filter, next I will try to invent a in-loop filter that is similar the SAO in the HEVC.
The NGIV software v20130729 (http://www.di-avc.com/NGIV20130729.7z).

[Guest]

So that's what you have been up to! It's an exciting development. Thanks for choosing Doom9 forum to promulgate and discuss it. I certainly plan to closely follow the project's evolution.

[SeeMoreDigital]

Nice one...

[edison]

hmm, how to use it ?

[schweinsz]

Currently you can only read the code and understand the algorithms. Currently the intra prediction part is actually better/more efficient than the HEVC's.
When I finish the test of the advanced intra coding algorithm, the deblocking filter, I will code the intra decoder. At that time, you/I can compare the codec with the HEVC's intra coding part.
Then I will go to the inter coding.
Edit: you can read the video-io.h and the video-io.c, there is some sequences preset. You can change it to yours, then you can code the sequence. You can set the qp. Because it is without any fast algorithm, so it is very slow.

[foxyshadis]

Would you be willing to set up a codebase on github or bitbucket? I'd love to follow along with development, as early as it is.

[LoRd_MuldeR]

I think if you seriously want your format to gain some use on the internet, you will need at least:

(1) A reference decoder library that can easily be compiled and used on all major platforms, i.e. Windows, Linux, MacOS, Android and iOS.

(2) A proper specification of your video format (bitstream syntax, etc) similar to the H.264/HEVC specifications - and please don't say "look at the code", I mean a real specification :p

(3) Convince Mozilla and/or Google and/or Microsoft and/or Apple to add support for the format into their web-browsers :eek:

What are you plans/thoughts on this ???


Furthermore: Even if it's in an early stage, can you provide sample encodes of the typical test sequences (e.g. "Parkrun" or "Parkjoy") any time soon, so we can see how it compares to, e.g., H.264 at the same bitrate?

Before that, I tend to be very skeptical about any claims about "coding efficiency" or "complexity" ;)

[Guest]

The project is just getting started and I'm sure schweinsz knows what is needed to make it successful. It's very interesting theoretically in any case. So I would advise some patience. schweinsz hasn't made any outrageous claims and he shares his work with us. What's not to like?

[foxyshadis]

It seems like it's at the point of "this is AWESOME" rather than locked down right now, kind of like HEVC three years ago when it had 64x64 transforms and other bits that made each frame take ten minutes to encode. Tech previews are cool, let's see what shakes out eventually.

[schweinsz]

Would you be willing to set up a codebase on github or bitbucket? I'd love to follow along with development, as early as it is.
I can, but I must learn about the git because I am not familar with this.

[pieter3d]

I think if you seriously want your format to gain some use on the internet, you will need at least:

(1) A reference decoder library that can easily be compiled and used on all major platforms, i.e. Windows, Linux, MacOS, Android and iOS.

(2) A proper specification of your video format (bitstream syntax, etc) similar to the H.264/HEVC specifications - and please don't say "look at the code", I mean a real specification :p

(3) Convince Mozilla and/or Google and/or Microsoft and/or Apple to add support for the format into their web-browsers :eek:


(4) Input from experienced hardware designers before all features are solidified. Very often a simple feature can cause big headaches for HW designs; conversely a small change that might not be obvious in SW could save a great deal of bandwidth and/or silicon area.

[iwod]

I am going to ask something which i have had in my mind for some time. Why do all Next ( or Next Next Gen ) try to get decode Efficiency of H.264 while offer similar H.265 Quality performance? Surely as we move on, and by the time any of these Next Gen Video codec comes out finalised we would be in 10nm era.

Shouldn't these new codec aim at H.265 similar decode efficiency while offer significantly better quality then H.265?

[Selur]

lower cpu/gpu power to decode -> potentially more devices which can decode via software -> more potential users

[iwod]

lower cpu/gpu power to decode -> potentially more devices which can decode via software -> more potential users

Yes, but that "lower" is comparative. What is consider "higher" today will be "lower" tomorrow.

The future are mobile. Where you expect dedicated Video Decoder or DSP to do these jobs. By the time this Next Gen Video is out we are talking about a few years down the road.

That is of coz assuming NGIV doesn't happen in 1-2 years. And i like to be proven wrong on this one. :D

[Selur]

What is consider "higher" today will be "lower" tomorrow.
Only if you assume that only people with new hardware are your target group. A lot of folks still have a lot of not so up-to-date hardware which they normally use.
Until a few days ago I was using a ~6 year old laptop, the cpu inside my newest pc is 3years old (i7 875k @ 3.6GHz), my mac book is 6 years old, my small tablet is 3 years old and I'm not really in a need to upgrade.
(aside from the laptop which got broken in an accident, but hey if buffered my fall down a few stairs quite well and the sdd is still okay)
-> just because there is new hardware out there which is faster doesn't mean everybody has it and only uses that piece of hardware.

[iwod]

Only if you assume that only people with new hardware are your target group. A lot of folks still have a lot of not so up-to-date hardware which they normally use.
Until a few days ago I was using a ~6 year old laptop, the cpu inside my newest pc is 3years old (i7 875k @ 3.6GHz), my mac book is 6 years old, my small tablet is 3 years old and I'm not really in a need to upgrade.
(aside from the laptop which got broken in an accident, but hey if buffered my fall down a few stairs quite well and the sdd is still okay)
-> just because there is new hardware out there which is faster doesn't mean everybody has it and only uses that piece of hardware.

True. I hope NGIV will be here soon. Need Kickstarter to rise funds? :D

[schweinsz]

I am going to ask something which i have had in my mind for some time. Why do all Next ( or Next Next Gen ) try to get decode Efficiency of H.264 while offer similar H.265 Quality performance? Surely as we move on, and by the time any of these Next Gen Video codec comes out finalised we would be in 10nm era.

Shouldn't these new codec aim at H.265 similar decode efficiency while offer significantly better quality then H.265?
The H.265 is with similar decoding complexity as the H.264.
A little more complex than the H.264.
To the best of my knowledge, there is no need to add complexity to the decoder, the current decoding procedure is very common so that it can fit many advanced encoder algorithms that can give more coding efficiency.

[schweinsz]

That is of coz assuming NGIV doesn't happen in 1-2 years. And i like to be proven wrong on this one. :D
It can not last so much time. Perhaps in several month, I can get it work with similar coding efficiency with the H.265.

[LigH]

Several months already passed ... :rolleyes: ... at least one season. How are you progressing? Steadily, I hope.

And how do you rate the efforts of Xiph Daala, provided you know a little more than one can read on their Wiki page?

[LigH]

Another half year passed. I hope you are still optimistic.

[LigH]

Well, x265 is becoming a mature HEVC encoder; Google is developing VP10; how are you?

[schweinsz]

Well, x265 is becoming a mature HEVC encoder; Google is developing VP10; how are you?
I code the q265 project (the H.265/HEVC encoder project that uses the hierarchical B 8 configuration) for my company, the encoder is 30-40% faster than the x265 medium configuration and 10% gain on bdbitrate.

I waste almost all my spare time during the last several months. I will start to finish NGIV project in next month, and it is expected to beat the HEVC/H.265 on coding efficiency.

[Selur]

bdbitrate <- what's that?
I will start to finish NGIV project in next month, and it is expected to beat the HEVC/H.265 on coding efficiency.
Are there plans for:
a. decoding support in libav ? (or another free decoding library?)
b. a command line encoder/decoder ?

[foxyshadis]

bdbitrate <- what's that?

Bjøntegaard-Delta bitrate, a fancy name for the measure of bitrate difference to match PSNR on average.

[schweinsz]

bdbitrate <- what's that?

Are there plans for:
a. decoding support in libav ? (or another free decoding library?)
b. a command line encoder/decoder ?
Firstly the latter, it is not difficult to integrate it to other platforms.

[Gravitator]

Hi,
Found that there is a common disease in the family HEVC...
Can preliminary check on your release?
Test sample > Park_Joy (http://files.videohelp.com/u/227452/Park_Joy%20(sneaky%20fly).7z) (sneaky fly).
Original sample > Park_Joy (https://mega.co.nz/#!dFdjlL5Q!X-39N69mg22VEH-eX9tvnDBHGmKVyEDgAX6o74k2G78) (mega.co.nz).

[LigH]

Gravitator, you are trying to get below Shannon's informational entropy. Don't complain about loss then. Different lossy codecs rate different parts of the video as acceptable loss, and the human brain will disagree with all of them in specific scenes and with a certain amount of artefacts.

[Gravitator]

Just do not want to see another conveyor with the same repetition of errors /weak points. The author claims about the superiority of his product :)

[LigH]

Well, it is hardly possible to measure the degree of annoyance people feel when watching a lossy video with artefacts, therefore it is hard to write an encoder which knows how to reduce the annoyance of artefacts optimally. The best strategy is to avoid artefacts in general. But that requires to stay close to, not very far below, a certain threshold of bitrate (the "entropy" in Claude Shannon's theory of information), no matter which codec you try to use to encode. There is no witchcraft and wizardry, just maths, and the formula to describe human perception is yet unknown, at least very incomplete. To improve a codec's efficiency, there are different strategies (usually "more efforts to find reducable redundancies" vs. "more detailed metrics to model human perception thresholds"), but both are already quite researched after decades of research and thousands of researchers; it won't be easy to discover another revolutionary method to cut another two-digit percentage off the bitrate. And after all, humans are different, one enjoys what another may find disgusting.

[Gravitator]

The analogy comes from VP9 - it seems that your law is logically expected leads to similar results :) I close this theme.
> sample (http://files.videohelp.com/u/227452/vp9%20(sneaky%20fly).7z)

[schweinsz]

Just do not want to see another conveyor with the same repetition of errors /weak points. The author claims about the superiority of his product :)
When I finish the inter coding module, I will test it. Currently there is only the intra coding module.

[Gravitator]

What characteristic will have the first prototype?

[schweinsz]

What characteristic will have the first prototype?
Full intra encoder/decoder firstly in this winter.

[LigH]

It is winter, and I am still curious...

http://cosgan.de/images/smilie/verschiedene/a030.gif (http://www.cosgan.de/smilie.php)

[Gravitator]

I am also interested to learn :)

[schweinsz]

I still do research on it, but I have no time to implement it because I spend all of my time on my company's project H.265 encoder including my evening. I will code it when I am free.

[LigH]

Hehe, Murphy's recursive law hits...

Every project takes longer than planned. This includes planning this delay.

[madanmohan]

Interesting to notice that alternative to DCT has been considered in video compression. Really wish you the best in bringing up your codec.

[mandarinka]

Alternatives to DCT have been considered for the last 20 years, they just never work :>

[Sulik]

Well, technically, HEVC is using an alternative to DCT in intra modes, it's using a DST, not a DCT :)

[LigH]

Daala is trying hard, too, with "lapped transforms". And Dirac/Schrödinger still believes in Wavelets.

[mzso]

Daala is trying hard, too, with "lapped transforms". And Dirac/Schrödinger still believes in Wavelets.

Overlapping or not the lapped transformation doesn't seem to go to far from DCT and other block transformations.
Transformations that don't try to build an image from tiles (blocks) are more intriguing to me. Too bad no-one spent enough effort to get one right.

[LigH]

Well, wavelets were meant to cover up to the whole dimensions of the image; but they were found to be inefficient when it comes to motion, it was not as easy as hoped to take advantage of only slightly changing wavelet parameters (see: Dark Shikari's blog (http://x264dev.multimedia.cx/archives/317)). The Rududu VfW codec started promising but went extinct, Xiph Snow was merely "a nice try" as well... regarding still image compression, the LuraTech software company was a bit more successful with its LuraWave format (*.lwf), a strong competitor of JPEG 2000.

Unfortunately, I don't know enough about Dirac/Schrödinger, but they may be the current most interesting project for your field of interests.

[mandarinka]

Dirac/Shrödinger has shitload of parameters that will make no sense to somebody who didn't write it, so it is pretty hard to test it (there are no presets or recommendtations). It has interesting behaviour (when it is starving for bits, it doesn'T IIRC blur and drop detail as much, but distorts/artifacts instead, or something like that) so it is not easy to compare its efficiency. I doubt it is competetive with H.264 though, else it would get at least some traction.

Snow is not by Xiph BTW, it was an experimental project of FFmpeg's Michael Niedermeyer, perhaps with some contributions from others (Loren Merritt too I think). It never get polished enough to be useful tho.

[Tommy Carrot]

Snow actually had pretty good quality. It was easily better than xvid and early x264, it had pretty good efficiency with almost no artifacts. Too bad the main developer got disinterested in it, and nobody else wanted (or could) to continue to improve it. With some additions (if they are feasible), like b-frames, adaptive quantization and psy-rdo, imo it could have been easily competitive with x264.

I've played around with dirac/schrödinger, but i was not impressed, no matter what settings i tried, the output was always heavily artifacted, and unusable as an encoding. Snow was a much better codec, too bad they didn't try to improve it instead of wasting their time with dirac.

[mandarinka]

What year are you talking about? Between dunno, 2007-2009, it was horrible. Artifacts everywhere unless you carefully picked settings, pretty bad quality. Also the better quality settings were incredibly slow. I think the only way to get acceptable look out of it was to actually use it in lossless mode (slight hyperbole).

[Tommy Carrot]

If you're asking about dirac, I've tried it about 3-4 years ago. The development was more or less stopped by that time, and i was curious what the "finished" codec looked like. Well, it was still unusable at that point, like the motion prediction didn't work correctly, the inter frames were full of artifacts. Rududu and Snow showed that it's possible to make motion prediction working in a wavelet codec, so if they couldn't solve some obstacles, they could just borrow some ideas from them, but it seems like instead of that they just gave up on dirac.

[mandarinka]

Oh, damn... I was talking about Snow but forgot to state that - sorry.

[Tommy Carrot]

Hmm, i remember if you selected the wrong kind of wavelet transform (there were multiple for different tasks), or wrong prediction combination, it could cause pretty bad artifacts, but with the recommended settings i've found somewhere, it had pretty good quality with minimal artifacting and good detail retention. In my tests it definitely beat xvid and the x264 we had around that time (i guess it was around 2005-2006).

[madanmohan]

Alternatives to DCT have been considered for the last 20 years, they just never work :>

I agree with you on other alternatives considered but they weren't pursued long enough to make an impact, which is unfortunate. It would be wonderful to see videos free from DCT based artifacts at low bit-rates. :)

Hope to see NGIV make an impact. :)