H.26L impressions

[Tommy Carrot]

I've heard so much rumor about the mpeg4 successor, the h.26l codec, so i've tried it. So far i know only about the JVT's reference implementation on http://bs.hhi.de/~suehring/tml/download/.

I've compiled jm40d version btw. The only problem is, that it can be feed with only .YUV files. But i've found avi2yuv and yuv2avi converters on my HDD, so that can be solved. You can use the .CFG files to set the things.

The overall impressions:
The quality is MUCH better than xvid. There is simply no artifact at all in very low bitrates too. It preserve the details much better than simple mpeg4. As lowers the bitrate, it simply became more blurrier due to the inbuilt deblocking filter (something like realvideo9, but the details are not smoothed, only the blocks), but as i said, no artifacts appeared.

BUT! There have to be negative side too. The speed is simply horrible. I know everything is done by brute force right now, but so far it make it useless. I've done a 450 frames long test clip with 352*288 res, and i has been done in cca. 1 hour on my athlonXP 1700+. :eek: The xvid did it in 9 sec.

While it encodes the source to h26l, it creates another YUV file, which can be convert back to avi - TADA- you can check the result.

Yes, it's difficult to use, you can not use it for other than testing purposes, but you can make impressions about this codec.

It would be such a great thing to make an avi wrapper to this! And the speed issue can be solving, because motion estimation would give a huge boost compare to the current brute force method.

Any opinions?

P.s.: I cannot give the compiled thing, and the yuv tools, because they aren't on this computer, but i guess you can find them somehow ;).

P.s.2: Sorry for my bad english:D

[Nic]

Yeah I wrote my own YUVViewer & AVItoYUV to play with it...I even speeded it up a little (I started at 2.1 last time I was on 3.9(

Its very good at low bitarte, doesnt macroblock in the same way, kinda just distorts slightly, alot better to the eye...

everso slow though :)

Cheers,
-Nic

[SirDavidGuy]

So is it Wavelet-Based, or what?

That's the only (good encoding) thing that I know of which blurs instead of creates blocks at low bitrates.

[Tommy Carrot]

Originally posted by SirDavidGuy
So is it Wavelet-Based, or what?

That's the only (good encoding) thing that I know of which blurs instead of creates blocks at low bitrates.

No, it's DCT based, just like the mpeg4, but it can use several block-size, not just 16*16. It's smoothed because the deblocking filter, but it is used at the encoding too, not just at decoding AFAIK.

[SirDavidGuy]

So it "cheats", then ;-) .

I'm really looking forward to a 3-D DCT. Based on some tests, it can really do quite well, but is very slow.

[Nic]

If quantisation matrices are intelligently switched, then a blurring effect rather than macroblocks can be achieved...

(but I dont know if the H.26l is doing this...I dont know the code as well as I might like, but I dont think its doing any pre/post processing in the reference code(?))

-Nic

[SirDavidGuy]

But then this really wouldn't be an "advantage" to MPEG-4.

The only "new" feature that I've seen so far is DCT's which are > 8x8

[TheXung]

But then this really wouldn't be an "advantage" to MPEG-4.

The only "new" feature that I've seen so far is DCT's which are > 8x8

That seems like such an understatement. I actually think the < 8x8 macroblocks make more of a difference. And how can you simply ignore the exponential quantization steps, context-based adaptive binary arithmetic coding, intra MB prediction modes, and multiple reference frames. These are not exactly "obvious" features.


Notice how I refrained from mentioning the deblocking filter which I believe if properly implemented greatly adds to the science of video compression.

[Tommy Carrot]

All i know that it gives outstanding quality even at very low bitrates too. The b-frame handling is much better too, because it doesn't quantize them more than the p-frames, so the PNSR rates are not lower, but the size advantage are there as well.

I did a test at 120 kbit/sec with 320x240x25 sequence:) Yes, it sounds ugly, but it is not that bad! The xvid (quant 14) is a horrible blockfest, full of artifacts, etc,etc. The h.26l is quite clean, no artifacts at all, It keeps details like xvid at 400 kbit, only a little bit smoother, but no artifacts or blocks.

I did only a 50 frame test with dvd (i haven't got a whole day for this:D ), but it seems it's really good at 4-500 kbits (640x352 res IIRC), something like xvid/divx at 900. I don't know how good it is for a whole movie, if you have about a month free time to try it, you should. :D

[TheXung]

If you could post your YUV utilities, it would be a helpful start for me.

As I see it, there are two options, try to hack these new features into an mpeg4 codec or try to speed this one up enough to be halfway usable.

[Tommy Carrot]

Here are the yuv utils.

EDIT: How can i enable the attachment?

[SirDavidGuy]

As I see it, there are two options, try to hack these new features into an mpeg4 codec or try to speed this one up enough to be halfway usable.

In a week or two, I'll try and put in some of XviD's ME routines.

[TheXung]

Just for reference or fun reading: http://www.ubvideo.com/public/h26l-white_paper.pdf

On page 6, they give a description of what I think is a creative way of doing more efficiently storing still images. From what I hear, JVT, using the intra prediction techiques are able to store keyframes at half the filesize of jpeg. I wonder if they have tried using MB prediction to save the residue for inter frames.

[trbarry]

The white paper above says H26L uses a 4x4 integer transform for it's IDCT which does not suffer from mismatch. Anyone know how this works?

- Tom

[SirDavidGuy]

The white paper above says H26L uses a 4x4 integer transform for it's IDCT which does not suffer from mismatch. Anyone know how this works?

It doesn't use a DCT, but a transform similiar to it.

It's too complicated to explain on a forum, but a description can be found here. (http://palms.ee.princeton.edu/plx/submissions/G2/Integertransform-2.html)

[SirDavidGuy]

Also, note that when you encode with the default settings, it skips 2 out of every three frames (Decimating by three) so comparing it with XviD (Skipping 0 frames) isn't entirely fair.

Don't get me wrong, I think H.26L is great, I just don't think it's a huge amount better, and will leave everything else lying in the dust.

[Dali Lama]

Hi, could someone compile that and put it in an attachment for testing?

Thanks,

Dali

[SirDavidGuy]

The source and executable is included in this attachment.

The executable is included in the "bin" folder, and can be configured using the encoder.cfg file.

[Dali Lama]

Thank You, I will wait for Admin approval of the attachment.

Dali

[[Toff]]

Arggg !! Encoding is terribly slow !
I understand now why Tommy said :

if you have about a month free time to try it

It's hard to really test at the moment.

Fortunately decoding is a lot faster.

Anyone know the difference between JM and TML ?

[Nic]

Yeah, TML is the old version & it had its same changed to JM :)
The latest version is the unofficial JM41b :)

-Nic

ps
I strongly recommend compiling with the Intel Compiler....

pps
It isnt skipping the frames, it turns the skipped frames into b-frames (i.e. if you turn skipped frames off, you cant use b-frame encoding)...(I think? lol, I honestly cant remember the source that well :) )

[SirDavidGuy]

Originally posted by Nic
It isnt skipping the frames, it turns the skipped frames into b-frames (i.e. if you turn skipped frames off, you cant use b-frame encoding)...(I think? lol, I honestly cant remember the source that well :) )

You sure?

There is a number of B-Frames setting (setting to 0 will not make any B-Frames, setting to 1 will make a B-frame in between every two anchor frames, etc.)

And there is a seperate setting for number of frames skipped, as well.

[Nic]

in encoder.cfg
FrameSkip = 1 # Number of frames to be skipped in input (e.g 2 will code every third frame)

change that to 0, then try using bframes....but this info is all relying on my memory, which can be exceedingly dangerous :)

-Nic

[SirDavidGuy]

I can't get frameskip = 0 to work even when I'm not using B-Frames, but I can get frameskip = 1 to work, when I'm not using them.

And also, from the code:

int jumpd; /* number of frames to skip in input sequence (e.g 2 takes frame 0,3,6,9...) */

(jumpd takes on "Frameskip").

And also, jumpd isn't declared in B-Frame parameters, but in general parameters, and also, the number of succesive bframes is declared elsewhere, why would they need two variables which have the same value?

[Nic]

Weird? used to work...maybe its changed, oh well :) youve got the source infront of you so im sure your correct...good luck with it :)

-Nic

[SirDavidGuy]

Actually, no, you were right. Frameskip = 1 will code: 1, 3, 5, 7...
Frameskip 1, BFrames 1 will code: 1, 3, 2, 5, 4, 7, 6 ..., as it should normally.

But the default settings (Frameskip = 2) will skip 2 frames, but only fill one of them with a B-Frame. So the defualt settings still don't code some frames which are then irrecoverable.

This makes no sense to me, since the two numbers should be invariably the same.

Maybe it's so someone can make a fast test, using only a fewer numbers of frames, but also add some B-Frames, but not enough to make it "real-time".

Or maybe it's just one of those things that's still there because it was there in the beginning, and nobody's bothered to remove it.

[trbarry]

SirDavidGuy -

10x for the link to the description of the integer transform. I've been puzzling over that page since yesterday but I'm afraid I don't really understand it.

Do you have any idea why they chose those particular numbers? For instance, if you have a row of 4 values <a,b,c,d> then the integer transform is just:

A = 13a + 13b + 13c + 13d
B = 17a + 7b - 7c - 17d
C = 13a - 13b – 13c + 13d
D = 7a - 17b + 17c - 7d


Any idea why they didn't just use the similar but simpler:


A = a + b + c + d
B = a + b - c - d
C = a - b – c + d
D = a - b + c - d


It seems it would have a similar effect (but square wave) and be even more trivial to calculate.

But obviously I'm missing something here, and likely getting out of my own area of competance.

- Tom

[Tommy Carrot]

Some comments:

I strongly suggest to test the latest JM build. TML has many bugs, the quality is slightly worse, slower, etc. And of course you can use frameskip=0 with the latest build. :D

I'm always using 2 frameskip with 2 bframes (just like in mpeg1-2), and everything was ok. No frame was skipped. :)

I've played with the settings a bit. If you set reference frames from 5 to 1, and switch off "adaptive block transform", it will be faster about 4 times (LOL, to about 0.5 frames/sec.), but the bitrate will goes up about 30%. But worth a try, still much better than everything else.

BTW, anyone knows what the hell is Hadamard transform? It slowers the encoding a bit.

[SirDavidGuy]

Originally posted by trbarry
Do you have any idea why they chose those particular numbers? For instance, if you have a row of 4 values <a,b,c,d> then the integer transform is just:

A = 13a + 13b + 13c + 13d
B = 17a + 7b - 7c - 17d
C = 13a - 13b – 13c + 13d
D = 7a - 17b + 17c - 7d


Any idea why they didn't just use the similar but simpler:


A = a + b + c + d
B = a + b - c - d
C = a - b – c + d
D = a - b + c - d


It seems it would have a similar effect (but square wave) and be even more trivial to calculate.

Because the H.26L way focuses the more important frequencies in a way more similiar to the DCT, into the upper-left hand corner. This is the only reason I don't like H.26L, since (IMHO), the DCT performs better in this respect.

BTW, anyone knows what the hell is Hadamard transform? It slowers the encoding a bit.

Again, too complicated for a forum, read this. (http://www.iro.umontreal.ca/~pigeon/science/ondelettes/transformees_rapides.html) (Read the Walsh transform section first, as it makes the Hadamard transform make sense).

I really don't know why the Hadamard transform is used, it's not (once again, IMHO) viable at all as a serious image compression concept.

[TheXung]

I really don't know why the Hadamard transform is used
This is just a stab in the dark since I haven't read much about the hadamard transform but perhaps something about the fact that it's suppose to be a lossless/integer transform allows it to get more accurate results after quanting/dequanting.

I mean just how important is the locality of coefficients for the run length encoding anyway? Sometimes it seems that most of the compression that quanting gives you isn't the fact that it improves the RLE encoding but that quanting makes the coefficients smaller to store.

That might be the advantage that hadamard transform provides, perhaps the inverse transform is less susceptible to coefficient errors due to quanting/dequanting.

Then again, this is all just guessing why they made their design choices.

[TheXung]

After playing with this, did anyone else notice that it takes a beast of a machine to decode these files? It took 25 secs to decode a 9 second clip that's 640x288 on my 1.4 ghz thunderbird. Now even though it was only compiled on MSVC release mode, does anyone forsee it possible to get the decoder to decode such a clip in realtime on say a ghz processor?

[trbarry]

Most MPEG2 & 4 decoders have all the heavy lifting done in assembler or even using hardware support. This does not yet have any of that so it would not be at all surprising to make it go 5-10x faster with just a little optimizing.

And while it is probably not the bottleneck, that integer transform they use can be implemented to be faster than IDCT, and wouldn't need hardware accel.

- Tom

[Vanos_b]

I just compiled TML 9.6 last night (I didn't looked at JM as I didn't knew what was there for and I was very anctious) and tried a little clip (100 frames / 4 seconds at 640x352, and 50 frames / 2 seconds 384x288 from Blue Streak). The quality is clearly better. With the first encode (quant 16/8/8 for I/P/B) I got a 650 KB file and I can say it looked almost as good as XviD const. quant 2 (1.1 MB) - maybe this is because the image is a little smoothered, but no blocks - and certainly a lot better then XviD with quant 4 (530 KB). Too bad for the 0.015 fps vs. 10 fps for XviD on my Duron 900. The real difference is with the second encode: 75 KB for the H.26L with 16/16/16 and 83 KB for XviD with quant 8 for all frames (no B-frames). XviD looks very crapy, full of artifacts (especially ringing) while H.26L seems very clear, of course not so much detail (as in the source video) but edges are very well defined, and the blocks only a little visible, but I guess from how it looked that those could be masked with very light deblocking - I'll have to try ffdshow with raw video, maybe will work. Maybe this is not enought to draw a conclusion, but if it behaves like this on most video clips, even withought further improvements (except for speed - I won't wait a few months to encode a movie) it certainly could beat any MPEG-4 codec available. I really have to try JM 4.1b (I've just downloaded it and I'll compile tonight).

Regards

[Tommy Carrot]

Vanos_b.

Jm 4.1 will seems to be slower, but that is because the adaptive block transform (as i remember it appeared in the jm versions). The encoding is a little bit more slower, but the size gain is 20-25% in my experiences. If you switch that off, it will be faster a bit.

I've discovered the fastest mode:
-reference frames=1
-adaptive block transform=0
-searchrange=8 (instead of 16)
-RDOptimization=0

Ofkoz the quality will be not even close to the best possible, but now i can get 2 fps on a CIF (352x288) source :)

BTW, if it's not too difficult, someone could enlighten me, what
- adaptive block transform and
- RDOptimization
mean, and what they are doing?

[-h]

I haven't checked the source, but I can assume..

- adaptive block transform and

This would try every possible method at the encoder's disposal to encode a block, and afterwards it would choose the most efficient one. Of course permuting through all those block transforms for every single block will take quite a bit of time, but it will save a lot of space. Speed-conscious encoders try to make an educated guess as to which method will be best for a block, so they don't have to brute-force it to achieve similar efficiency.

- RDOptimization

Rate Distortion Optimization - this will determine how much a given quantizer affects the video quality, and hopefully chooses the best tradeoff between visual quality and size. There are a number of ways to determine "quality" of course..

-h

[SirDavidGuy]

- RDOptimization

Rate Distortion Optimization - this will determine how much a given quantizer affects the video quality, and hopefully chooses the best tradeoff between visual quality and size. There are a number of ways to determine "quality" of course..

Not quite :-( .

It's implementation is beyond the scope of a forum (and this time I couldn't find a site which adequately describes it), but it's essentially a technology for sending information over noisy/lossy channels, similiar to the use of CMV's in MPEG.

- adaptive block transform and

This would try every possible method at the encoder's disposal to encode a block, and afterwards it would choose the most efficient one. Of course permuting through all those block transforms for every single block will take quite a bit of time, but it will save a lot of space. Speed-conscious encoders try to make an educated guess as to which method will be best for a block, so they don't have to brute-force it to achieve similar efficiency

Sorry, agaon.

It's an Adaptive Size transform, unless I'm much mistaken.

[-h]

It's implementation is beyond the scope of a forum (and this time I couldn't find a site which adequately describes it), but it's essentially a technology for sending information over noisy/lossy channels, similiar to the use of CMV's in MPEG.

Er, really? First few google matches for words "rate distortion theory video" indicate that it's more along the lines of fancy rate control.. though yes I could find little to nothing regarding "Rate Distortion Optimization". From what Alexis mentioned about it back on the XviD forums, it didn't seem too far removed.

It's an Adaptive Size transform, unless I'm much mistaken.

Adaptive Size of what?

-h

[SirDavidGuy]

Adaptive Size of what?

The Integer transform.

Er, really?

Really.

I dug up a paper on the subject, drop me a line if you want a copy.

[-h]

The Integer transform.

Okay, I thought it was fixed at 4x4. That's adding some real complexity..

Really.

I think I know someone who would enjoy a good read on the subject.. I dropped you a PM.

Just finished reading through http://citeseer.nj.nec.com/sullivan98ratedistortion.html , and what a long boring read it was. Perhaps it'll find its way into XviD some day, but probably only the aggressive INTER4V search that was mentioned. Madly coding to satisfy PSNR or some other objective measurement isn't worth making a slower codec that looks subjectively the same, IMHO. Depends on how many hours someone wants to spend tweaking it I guess.

-h

[SirDavidGuy]

@TRBarry:

This user's mailbox is currently full, and cannot be sent any messages until it is cleaned out. An email has been sent notifying the user of this. Please try your request at a later time.

[trbarry]

darn. I'm too much of a pack rate.

But I just deleted some stuff. Pls try again.

- Tom

[scorchED]

@Tommy Carrot
could you post your Jm 4.1 build please? i would like to put my testing hands on it and maybe try to encode a hole movie.

[slavickas]

Originally posted by scorchED
@Tommy Carrot
could you post your Jm 4.1 build please? i would like to put my testing hands on it and maybe try to encode a hole movie.

Although i'm not Tommy ;) , i've compiled jm. Download @
http://merkys.ktu.lt/~vitslav/jm.html

[Tommy Carrot]

Originally posted by scorchED
@Tommy Carrot
could you post your Jm 4.1 build please? i would like to put my testing hands on it and maybe try to encode a hole movie.

You wont encode a full movie, unless you have a supercomputer. :) The speed is so terrible, that some hundred frame is all you can try. :( An AVI wrapper is missing too to be really usable. It just shows the possibilities of the new technology, which is very promising.

The only thing i'm so excited about this, that some guy at JVT said: with full optimization, h.26l should have the same amount of computation complexity (so encoding speed) as MPEG4 advanced simple profile (divx5 with all features). And this will give much better quality than that.

[-h]

The only thing i'm so excited about this, that some guy at JVT said: with full optimization, h.26l should have the same amount of computation complexity (so encoding speed) as MPEG4 advanced simple profile (divx5 with all features). And this will give much better quality than that.

Of course, if you switch off H.26L's features which aren't in MPEG-4 Advanced Simple :)

I don't see how H.26L could encode as fast as MPEG-4 if you decide to reference 6 times as many frames during ME..

-h

[Tommy Carrot]

Originally posted by -h

Of course, if you switch off H.26L's features which aren't in MPEG-4 Advanced Simple :)

I don't see how H.26L could encode as fast as MPEG-4 if you decide to reference 6 times as many frames during ME..

-h

But h26l doesn't have global motion compensation, which, if properly done (IMO divx5's not), slowers the encoding quite a bit.

Of course i don't believe too that it will be exactly the same, just wanted to make a point that it can have reasonable speed (if half of the advanced simlpe profile, it's still a good deal IMHO (if the optimization doesn't completely ruin the quality :D )).

[SirDavidGuy]

Also, bear in mind that some of TML8's quality gains are due to it's being unoptimized.

eg. DivX 5 and XviD could have much better quality using a Full search, but want to be finished some time this century for the encodings.

[-h]

eg. DivX 5 and XviD could have much better quality using a Full search, but want to be finished some time this century for the encodings.

Back when gruel last tested it, PMVFAST performed better than an full search thanks to MV compressibility. Of course his full search might have been disregarding vector bit cost, so that would certainly hinder it. SAD isn't exactly the best match criterion either.

I can picture a "true" full search encoder.. for every possible MV, compensate the block and fdct/quantize/bitpack to find the true best match. Even gruel's full search slowed down encoding 110 fold, so I can't imagine what that would cost. And yet I have this yearning to try it.

-h

[trbarry]

I don't see how H.26L could encode as fast as MPEG-4 if you decide to reference 6 times as many frames during ME..

Actually I would love to have a working H26L encoder that ran 1/6 as fast as Xvid. ;)

- Tom

[Rrrough]

Yes, me too... :) Hope it will optimized like :devil: and make it's way into xvid project soon...