VDWL video codec

[ntojzan]

Hello,

I've started a research in order to create a new keyframe only codec. It uses a form of 4x4x4 3d wavelets to transform 4 frames into one datastream, and the decompressor can then access each frame individually using the same data but different transformations.

At 100% quality the compression ratio during the tests was from 1:6 to 1:40 depending of the image complexivity and the motion.

However I haven't implemented ANY lossless compression on the given datastream, so it is possible to achieve even better compression ratios. For tests I've tryed to copmress the compressed output with WinRAR, and get 1:10 to 1:60 at 100% quality.

I am searching for people to join me at this research. I especially search for someone who will write the lossless compression part and who will make the OS specific programming - I lack the knowledge of the Windows specific programming.

Any interrested people can contact me to ntojzan@neobee.net

I've also put up a supporting website on angelfire, but since they forbid posting ULR's on newgroups, I'm unable to give you the URL, because they will ban the homepage. Anyone interrested should e-mail me aswell.

Thank's for your time.

Re.

[MfA]

100% quality? What does that mean? (I know what it means in JPEG, but this isnt JPEG ... so I have no idea how it translates to something a little more important, such as MSE.)

I assume it isnt mathematically lossless.

[ntojzan]

Hello

100% quality means: quantizer = 1

That's the best possible quality using this encoding, however it isn't 100% lossles, the error may be +/-1 in RGB for each pixel.

Re

[ntojzan]

Just to correct my previous statement. The +/-1 error is not on RGB data, but on the transformed YUV data. I have not measured PNSR, visually it seems lossless, except for movies with a high noise ratio, there the compressd image gets a bit blury after the transformation.

Sorry for confusing.

[MfA]

Just a comment ...

I dont really see the point anymore in lossy transforms in this day and age ... the loss in compression efficiency for using an invertible transform is negligible, in effect putting a form of pre-quantization in the transform is rather archaic.

[ntojzan]

I disagree. The aim in keyframe only compression is to compress the data to minimal possible datarate, and also to create a FAST compression and decompression routine in order to increase rendering speed in non-linear editing application. Yes, you can achieve good compression ratios without loss, but then the rendering process will be longer. Also you can make it quicker, but then the compression ratio will suffer, and you will need more hard disk capacity to store the same data. If you are working with hours of video data theese things does matters. That is my viewpoint. Offcourse if you are able to achieve average compression ratio around 1:20 with at least 100fps decompression without loss (on average system) then you are right.

[MfA]

That doesnt mean you cant use an invertible transform, H.264 uses an invertible transform too ...

IMO transforms should not tamper with the data, just transform it, that is the role of quantization.

[ntojzan]

Oh, sorry I misunderstood you, and misslead you too. The transformation I'm using IS reversible, but in order to keep the data in bytes I divided the result, so it is actually a bit quantized even at maximum quality. I could make it 100% lossless, however the visual quality is not affected by this division, it only affects noise. Actually at maximum supported quality even the noisy image is OK, just when you increase the quantizer factor it gets blurry, and later (for higher q.) it gets even blocky. That I find a bit strange since it is not blocky even at lowest supported quality if the source is not noisy.

[Atamido]

ntojzan: Your codec sounds very much like Gldm's WARP codec. Although his goal was to do it over a group of 32 frames. I am rather interested in seeing it. What kind of interface are you using, VFW or DirectShow? There were some features that we were thinking about building into Matroska for these types of codecs, so I would be very interested to learn more.

[ntojzan]

No, it has only one similarity with the WARP codec, that it doesn't use 2d transformation. However as I'm informed (migh being wrong) the WARP uses 4d transformation, while WDVL uses only 3d. Also, as General Lee D. Mented stated the WARP codec doesn't aim for high compression / decompression speed, the WDVL does.

As for Matroska, well I called the VDWL routine a "codec" since it works as COmpressor/DECompressor, so it qualifies for that name. However it is not yet a VFW codec or else. I only have an executable which can compress a video file, store it onto the disc without any container format, just the RAW encoded data - and even that without the final lossless compression!!! The main reason why I've started this thread is that I think I'll be compleatly unable to finish this project unless someone joins me. First of all I have a complete lack of C++ knowledge, and secondly I don't know much about windows architecture, I just began coding under this enviroment. What I am good at is assembler. The example code was written in Pascal (that I knew from earlyer) and some parts are rewritten to assembler. What I would actually like is to write the entire compression / decompression routine in assembler, and to find someone to write the OS stuff in C++... THEN it will became a standard codec, until then it is just a compression technique.

If anyone is able to help me to create the entire codec in assembler, or using a combination of assembler and Pascal, I'd be able to finish the codec myself.

Can anyone help me?

[Atamido]

If I understand you correctly, your codec takes 4 video frames and compresses them into a single data block. Then it takes another 4 frames and compresses them into another data block. If this is correct, then you will not be able to use it correctly to encode with VFW. The VFW interface requires that you input a single frame to the compressor, and then the compressor outputs a single frame. The only way you could avoid this in VFW is to buffer frames and output dummy data. Unfortunately this would produce synch issues as your frames would always be 3 behind.

A better option for this type of codec would be something like DirectShow that allows the compressor to input and output as many packets as it wants.

You may try visiting the #matroska channel in IRC to ask about this as some of the developers there have some experience creating DirectShow codec wrappers.

irc://irc.corecodec.com/matroska

[ntojzan]

Yes, I am aware of this problem, but also have a solution. The codec will be able to get one input frame and output one data. Here is the concept:

frame1:
- convert to YUV 4.2.0 and buffer it
- output FRM1 identifier as compressed data

frame2:
- convert to YUV 4.2.0
- perform the first transformation using frame1 and frame2
- buffer the first transformation result
- output FRM2 identifier as compressed data

frame3:
- convert to YUV 4.2.0 and buffer it
- output FRM3 identifier as compressed data

frame4:
- convert to YUV 4.2.0
- perform the first transformation using frame3 and frame4
- perform the second transformation using tr12 and tr34
- output FRM4 identifier and the complete compressed data

Actually this is needed however you approach the codec development, because since you use the same data for each four frames, the compressor must know somehow which decompression transformation to use. So the decompressor checks the identifier, and then performs the decompression. This way you are not using sync with the audio, and it will work except for the last 0-3 frames.

I'm not too familiar to VFW development (as stated before), but my guess is that this should work. (correct me if I'm wrong)

[ntojzan]

Just a correction:
I ment: ...not loosing synch with the audio...
I wrote: ...not using...

Sorry for confusion...

[Zarxrax]

How fast is this codec compared to huffyuv and vble?

[ntojzan]

I'm unable to provide information about the compression/decompression speed until I rewrite the entire routine in assembler. Some parts are still in Pascal, so it runs slower than it will at the final version.

[Koepi]

ntojzan:

take a look at the xvid vfw part. If you want to release the codec under GPL you can directly interface your codec with xvid's vfw. You'll very fast find the necessary interfaces and can use the minimal one.

Also looking at other codecs like Eduardo's Wavelet9/7 might give you some interesting insights.

(Well, or hopefully you find someone with enough spare time to interface your codec with a vfw/dshow front-end).

Regards
Koepi

[ntojzan]

Thanks...

However:
a) XVID is written in C++, and I dont even own a C++ compiler, not to mention complete lack of knowledge... :(
b) I still don't know how will I release it finally, I'd like to have a commercial product, but I also want to share the technique to other developers... So I will open the source eventually, just want to make sure that I'll arive first on the market...

I'll take a look at the codec you've mentioned.

[MfA]

xvid is C through and through.

[ntojzan]

No difference. C++ or C, all the same to me... I guess I could try to rewrite it to assembler or pascal, however there is another issue too: I don't like to work alone... So I'd rather see someone join this project, and write that parts in C, C++ or whatever he likes, that to see the project done by myself alone... (it does sounds like a challenge tough)

[Koepi]

I think one problem could be that you want to go commercial with the codec. There are plenty of open and/or free software/source coders around this forum, but they wouldn't want to see their work make someone else's money possibly.

Making the sources opensource on the other hand isn't an insurance for getting fellow developers to work on it either. Very problematic issue.

Just something to think about when you decide what to do in the hope that it helps.

Regards
Koepi

[ntojzan]

Well, i once again disagree... Why do you presume that I want help for free? If someone joins the project, and it does go commercial, the profit would be shared between the people involved with the project... What is wrong with that?

[Koepi]

Oh, I think there is a misunderstanding.

I just mentioned that because some developers reading this thread could get the impression that way. I didn't want to imply that.

But now you cleared that up, so it's definatly solved/explained IMO.

Regards
Koepi

[ntojzan]

Well, sure it should be specified a bit more, but the thing is that I still haven't decided what to do with the codec if it gets ever compleated...

Actually what I think as the best solution is to:
a) finish the research, and develop the codec
b) make a commercial product using highly optimized asm code
c) make an opensource decompression routine
d) create a homepage with full details about the transformations and compression techniques used withn the compressor

This way the people involved within the project would get some money (hopefully), and also by opening the source and releasing all the information it would make possible anyone interrested to create a freeware version of the same codec. People would be then able to choose between the commercial and possible free version.

My problem is as it seems that I cannot decide between sharing all the information and making money... Does this compromiss sound allright?

[vinetu]

ntojzan,I wish you LUCK!!!

As you mention this type of codec is perfect for NLEditing -
VFW interface,very fast compess/decompress
and every frame-key frame are the keys for this job!
For now(and from years) the best is MJPG
and that's why it is implemented in hardware in many Video Editing systems...

So I thing -the market for this type of codec would be in NLE land.
Here is my vision- You may know that usually there is a many problems with NLE
hardware (clearly Matrox ;)...and not only!)- bad written drivers,hardware uncompatibility,moon phase and so on...
...and to avoid that, sometimes when is possible I start the new projects in software-MJPG "mode" only
and when editing is done ,I use the expensive Video board as a simple TV-out device...
On the other hand every modern VGA card has TV-out ,so...if you succeed with speed and in addition
If you find the way how to activate that TV-out from inside a NLE software (Adobe,Ulead,Pinnacle,Mainconcept...)
-It will be great and you can rely on market :) !!!

May be that man will be interested:
http://www.debugmode.com/

http://www.debugmode.com/wax/

Best Regards!

[ntojzan]

Wow, that Wax is a one hell of a nice idea... I'm surprised I've never heard of it before. I still didn't downloaded any binary, but if it does everything the author claims, it is very usefull.

As for NLE: the constant hardware problems were one of the things which gave me an idea to start my project. There are cheap capture cards, and the TV-output at the GFX cards is better and better. I imagine that the use of hardware editing cards will be not that good idea in future when the CPU power gets enough for realtime-rendering. For example I've contacted Matrox severall times to ask them where can I get information about their editing cards in order to start a project of creating custom drivers an transitions, but eventually I've got no information at all. The set of realtime transitions is VERY limited, and I find it surprising that it uses a 3D chip for rendering, and do nothing with it... That few 3D based effects are too simple for my taste, and realtime rendering in 720x576 is not a hard thing nowadays...

Adobe does support TV out in DV editing mode, however that support isn't as it should be. At the moment the best thing you can do is to export the movie, and then play it from a media player. If you have installed DIVXG, the output will be perfect, with adjustable field order even with the MJPEG codec, so you don't even need the editing card for output.

So my aim _IS_ to make possible video editing on cheap home systems without 320Gb harddisc space - The data rate at 100% quality for 720x576 (25fps) is between 1000-4000kbytes/s, and reducing the quality a bit has only a small effect - especially on the TV output (that is tested) - and the datarate in that case goes 500-2000...

However writing the codec is just the first step, there are some other things to do aswell...

Another idea: if the codec ever sees daylight, I have an idea how could I implement the use of the TV output regardless of the NLE software you are using: It is possible to put an option "agressive TV-out handling" on the codec control panel, using which the codec would change the resolution of the TV output to 720x576 or 720x480 (PAL/NTSC), and opens an always-on-top window, and when a program requests decompression of one frame, the codec would dump that frame to that "default output" aswell. This should make the trick.

[Joe Fenton]

Originally posted by ntojzan
Well, i once again disagree... Why do you presume that I want help for free? If someone joins the project, and it does go commercial, the profit would be shared between the people involved with the project... What is wrong with that?

If you do go commercial, be prepared to hand most of your money over to various patent owners. Almost every conceivable use of wavelets has been patented out the ying-yang for years now. It's probably the reason there aren't any wavelet based codecs seriously being worked on... everyone is waiting for the patents to expire. :D

Seriously, go to the patent search page and do a search on "wavelet" - you will be horrified at the sheer number of patents on one little transform.

[ntojzan]

Well the thing is that I'm not _ACTUALLY_ use wavelets... I'm just using something very similar, but the equations are unique. Searched over the internet to find if anyone used this specific set of transform equations, but find none. (probably noone tought of something that silly that I am actually using)
I used the term "wavelet" because wavelets are the nearest thing to transformation I am using...
So I'm not afraid of patents, I'm rather afraid that I wont be able anyone to cooperate with...
If I'll have to do it all by myself, I will, however it will take years...