New video compression thoughts?

[SirDavidGuy]

What about the encoder storing only the difference between the predicted and actual MV's? Some predictors used in ME (eg. Last frames median, Acceleration, MV's of previous adjacent blocks, etc.) don't require having the frame or any current MV's at hand. Based on my guestimations & the calculator, this could save 1-3 bits per MV.

Although, this wouldn't be Mpeg-4 complaint.

Also, what about a 2-D MDCT (to reduce the visibility of blocks)? I saw some tests with it (on images) and it looked alright.

[-h]

What about the encoder storing only the difference between the predicted and actual MV's? Some predictors used in ME (eg. Last frames median, Acceleration, MV's of previous adjacent blocks, etc.) don't require having the frame or any current MV's at hand. Based on my guestimations & the calculator, this could save 1-3 bits per MV.

You mean, for each vector you'd store a code detailing which predictor was closest and should thus be used to add the error MV to? I'm skeptical as to whether this side information wouldn't outweigh the decrease in bits spent on MV error. Or could the median decision be expanded to use 5 inputs, by adding the colocated block and acceleration vectors?

Also, what about a 2-D MDCT (to reduce the visibility of blocks)? I saw some tests with it (on images) and it looked alright.

I wish there was more development surrounding Matching Pursuits for residual coding, the only real demonstration of a mature implementation I've seen was this (http://buffy.eecs.berkeley.edu/IRO/Summary/97abstracts/falcon.1.html) (source code used to be available but was removed), and even though H.26L would outperform it now, the potential is certainly there for improvement. The decoding speed is a big plus too.

-h

[MfA]

Using the colocated block is far from ideal AFAICS (for motion estimation too BTW). It seems better to extrapolate the motion field as was described by Joseph Yeh (http://www.cs.berkeley.edu/~jyeh/mcmvIcip95.pdf) a long long time ago (something I have learned over the years, if you have a smart idea it has almost certainly been described before ... if you find it wasn't a patent will soon be published on it). He came up with a hell of a name for the idea, meta motion compensation.

I am sure such a scheme would work even better with forward motion compensation ("in terms of direction of motion vector, which is from the previous frame to the current frame" for instance as used here (http://vision.ai.uiuc.edu/abstracts/pub6_2_1_a1099yoonvc.htm)). Then you could get your predictors for the next motion field without needing any extrapolation (you would to deal with overlap that way, but I think it would be a small price to pay).

I think PSNR wise matching pursuit would not give much coding gain for H.26L, most of the energy is at edges anyway where small transform block sizes will tend to be used because transform coding with bases with a large area of support is counterproductive leaving little to be gained from matching pursuit, and loop filtering removes most of the potential perceptual gain too.

I dont think matching pursuit is a way forward (I think lapped VQ is way cooler than any transform based coder anyway :) hell some researchers which used dense motion fields went as far as to use a pel level context based entropy coder for the DFD.

[-h]

(something I have learned over the years, if you have a smart idea it has almost certainly been described before ... if you find it wasn't a patent will soon be published on it)

I used to have a long list of zany ideas that I thought no one else had investigated, but it seems every week or so I run across some old paper that not only presented the idea, but gave a better implementation.

Pretty much all I'm left with is adding a "last-chance" subset to the candidates of EPZS/PMVFAST that perform a mean-removed SAD on the median and 0,0 vectors. And that's so small an idea (and really just patching traditional mrMAD techniques onto zonal ME) it's hardly worth calling it such! Pretty much all else I have is testing quantized coeffients against a threshold to see whether they can be decimated into VLCs with lower bitcosts, but that's hardly groundbreaking (see ancient GIF optimization techniques).

Tragic to be so young and so void of zany ideas. That's not to say I'll stop trotting them out when they come to me..

-h

[ReferenceDivx]

I think we need more zany ideas. I think zany ideas will lead the way. Creative problem solving will always be highly regarded. Unfortuanatly many specs have to be obeyed inorder to have a working product that everyone can use. I think we need to think more about quality then computing time as we continue are code evolution. The codecs we design should double in complexity every 18 months.

We have no where to go but to improve.