Digital Subband Video 1 - Wavelet Codec in Less than 5000 Lines of C

[LMP88959]

Hi Doom9, I recently developed this:

https://github.com/LMP88959/Digital-Subband-Video-1

DSV1 is comparable to MPEG-1 / MPEG-2 but uses wavelets instead of the DCT.

Click to expand comparison images:



I intend to use it as a video codec for my game but I believe it might be of interest to some of you here

Here is a little slideshow presentation I made explaining the codec and some of my thoughts on video compression:
https://www.youtube.com/watch?v=zDm4GN-znBo

Please understand I developed this codec out of passion and interest in the field, I didn't develop this for serious/professional use.

Here is a quick description:

- uses mutliresolution subband analysis (aka wavelet decomposition) instead of DCT
- half-pixel motion compensation
- supports 4:1:1, 4:2:0, 4:2:2, and 4:4:4 chroma subsampling formats
- adaptive quantization
- closed GOP with intra and inter frames (no B frames, only P frames for simplicity)
- no complex entropy coding (only interleaved exponential-Golomb coding)
- support for much lower bitrates than MPEG-1 / MPEG-2

The README and the PDF files in the GitHub have more information if you're curious.

I've seen Cineform's codec, Dirac, and Snow but none of these were made to be comparable to the first generation of MPEG video, and Snow ended up being abandoned. DSV1 is the first "complete" wavelet video codec I am aware of that is comparable to MPEG-1 / MPEG-2 and not trying to compete with H.264 or MPEG-4 part 2.


Please let me know your thoughts, positive and negative. If you find any bugs please let me know as well

- EMMIR

[modus-ms325c]

not sure i really need another video codec at this point, but i'm interested.
my superficial look is that DSV1 maintains a lot of detail at expense of blurring out other parts of the frame or worse.

[MoSal]

comparable to MPEG-1 / MPEG-2

You mention this multiple times. Why is it a highly relevant/desirable property for you?

[LMP88959]

Originally Posted by modus-ms325c

not sure i really need another video codec at this point, but i'm interested.
my superficial look is that DSV1 maintains a lot of detail at expense of blurring out other parts of the frame or worse.

Thank you!

Originally Posted by MoSal

You mention this multiple times. Why is it a highly relevant/desirable property for you?

Thank you for your question, I mention it for a few reasons:
1. To let people know DSV1 isn't competitive with codecs newer than MPEG-1 / MPEG-2
2. To help group it into the same era of video codecs.
3. To invite others to compare it with those video coding standards.

I hope that clarifies things for you

[rwill]

This is great.

You probably won't be able to start a business based on that but I am sure you learned a lot.

As you are using HAAR for most things, have you tried to partition the picture in macroblocks like MPEG-1/2 ? This might make things more manageable and HAAR does not have funny edge effects.

And for Inter I know you want short Wavelets, have you tried Daubechies 3 ? I remember that one fondly for Inter, even if used with somewhat small pixel blocks.

If you want to shave of bits from your coefficient coding I recommend Range Coding, maybe look up that IBM paper from G. N. N. Martin / 1979. I think AV1 is using a flavor of that one too. This will make things more complicated and slower though so if speed and simplicity is your goal just pass.

As for no Wavelet based video codecs you know being comparable to Mpeg-1/2, please keep in mind that lots of people made Wavelet based video codecs, block based hybrid ones even, but almost none made it in the spotlight because they just don't perform as well as macroblock based DCT codecs. Wavelet Codecs are mostly some academic thing.

Another thing:

I stalked your Github, what do you think of the following 68K texture span inner loop (texture stride = 255) ?

move.b (%a1,%d2.w),%d6 ; load texel
move.b %d6,(%a0)+      ; write texel
add.w %a3,%d4          ; vfrac inc
scs.b %d7              ; vfrac overflowed ?
add.w %a2,%d5          ; ufrac inc
addx.w %d3,%d2         ; combined u int + v int + ufrac overflow texel step
add.w %d7,%d2          ; additional v texel step if vfrac overflowed

PS: DSV1_spec.pdf, Page 16, 2nd MIN() -> MAX

[LMP88959]

Originally Posted by rwill

This is great.

You probably won't be able to start a business based on that but I am sure you learned a lot.

As you are using HAAR for most things, have you tried to partition the picture in macroblocks like MPEG-1/2 ? This might make things more manageable and HAAR does not have funny edge effects.

Hmm, doing a Haar transform isolated to a macroblock? It's definitely something to experiment with, what are referring to when you say 'more manageable'?

Originally Posted by rwill

And for Inter I know you want short Wavelets, have you tried Daubechies 3 ? I remember that one fondly for Inter, even if used with somewhat small pixel blocks.

I tried a CDF 2.2 and the edge artifacts around intra blocks looked awful

Originally Posted by rwill

If you want to shave of bits from your coefficient coding I recommend Range Coding, maybe look up that IBM paper from G. N. N. Martin / 1979. I think AV1 is using a flavor of that one too. This will make things more complicated and slower though so if speed and simplicity is your goal just pass.

I had heard about, and seen some implementations of, range coding, but since I was prioritizing speed and simplicity like you said I just stuck with EG coding and RLE.

Originally Posted by rwill

As for no Wavelet based video codecs you know being comparable to Mpeg-1/2, please keep in mind that lots of people made Wavelet based video codecs, block based hybrid ones even, but almost none made it in the spotlight because they just don't perform as well as macroblock based DCT codecs. Wavelet Codecs are mostly some academic thing.

Yeah I have no doubt others have written similar codecs, I'm sorry if I made it sound like I've done something novel. I'm mostly referring to DSV1 being publicly available, free, unencumbered, and in a stable 'finalized' state. I personally can't recall seeing another codec like DSV1 that checks those boxes, but if you know of any please let me know, I am always interested to learn about others' projects and efforts.

Originally Posted by rwill

Another thing:

I stalked your Github, what do you think of the following 68K texture span inner loop (texture stride = 255) ?

move.b (%a1,%d2.w),%d6 ; load texel
move.b %d6,(%a0)+      ; write texel
add.w %a3,%d4          ; vfrac inc
scs.b %d7              ; vfrac overflowed ?
add.w %a2,%d5          ; ufrac inc
addx.w %d3,%d2         ; combined u int + v int + ufrac overflow texel step
add.w %d7,%d2          ; additional v texel step if vfrac overflowed

Haha are you a graphics enthusiast too? I'm an assembly novice, but your texture fill looks good, I'm assuming paletted texture due to 8-bit texels?

Originally Posted by rwill

PS: DSV1_spec.pdf, Page 16, 2nd MIN() -> MAX

Thanks for catching that! I just fixed it in the repo

[rwill]

Hmm, doing a Haar transform isolated to a macroblock? It's definitely something to experiment with, what are referring to when you say 'more manageable'?
...
I tried a CDF 2.2 and the edge artifacts around intra blocks looked awful

With 'more manageable' I mean to have some sort of unit that can be somewhat encoded independently. You would lose your global coefficient approach but you then can do things like rate distortion optimization better. That is it makes it more possible to weight the distortion some encode decision generates against the bits it will cost. Now rate distortion was started to be used in the early 00' so most Mpeg-1/2 encoders did not do that. Encoding in blocks also reduces the effects a transform has to the block area. It will reduce wavelet transform efficiency though.

I had heard about, and seen some implementations of, range coding, but since I was prioritizing speed and simplicity like you said I just stuck with EG coding and RLE.

Your plane approach might be suited for something like Huffman variable length codes with a frame optimized table too.. I am just throwing ideas around though. I vaguely recall that the Theora video codec was transmitting its coefficients as a huge size optimized run/level blob too.

Yeah I have no doubt others have written similar codecs, I'm sorry if I made it sound like I've done something novel. I'm mostly referring to DSV1 being publicly available, free, unencumbered, and in a stable 'finalized' state. I personally can't recall seeing another codec like DSV1 that checks those boxes, but if you know of any please let me know, I am always interested to learn about others' projects and efforts.

When I was doing my video codec experiments 20+ years ago there were a couple open source implementations available. But I never have seen one that came with a specification or was as polished as yours. Were different times with different tools though.

Haha are you a graphics enthusiast too? I'm an assembly novice, but your texture fill looks good, I'm assuming paletted texture due to 8-bit texels?

Yeah, somewhat. From time to time I do some graphics stuff on a TI-89 calculator which has a Motorola 68k CPU and is used with 2 bit grayscale graphics.
Like for example I have done this: https://www.youtube.com/watch?v=3WvSVB4nqjw