WARP explained.

[General Lee D. Mented]

I still haven't gotten my codec working, cause I've been too depressed to deal with it anymore. Some people have told me they can't help me debug it because they don't understand it.

So here's a video that explains the entire algorithm: http://www-e.uni-magdeburg.de/drobek/WARP_explained.avi

52MB, 1:12 play time. It should be a fast enough mirror.

Someone please help me with this thing or I'll probably never get around to finishing it. VFW still does weird crap on me like give me space to store a YUV frame when I'm in RGB mode, inconsistent intitialization between Vdub and Media player, etc.

[General Lee D. Mented]

Oh, the corecodec.org project is apparently down for awhile. Here's an alternate link to the last source I released:

http://www-e.uni-magdeburg.de/drobek/WARP_beta_01_source.zip

[JJ]

Hi!

I've compiled the source you provided, but the image was garbled and displayed upside-down on decompression. I was able to (at least partially) fix it by applying the following changes:

driverproc.cpp

line 59: shouldn't there be a return statement? (this shouldn't matter though)

vfwhandler.cpp

lines 300-301: I think the values for biPlanes and biBitDepth should be 1 and 24 respectively.

line 343: I've changed it to:
if (output->biBitCount == 24)
return ICERR_OK;
else
return ICERR_BADFORMAT;

After that I was able to play the previously created AVIs.

I believe that in warpframe.cpp there should be a 'case 0:' statement around line 76 - this will stop VirtualDub from complaining about not being able to find a suitable format.

What still doesn't work for me:
* setting bitrate to anything but 0 - the codec seems to enter an infinite loop there
* setting block size higher than about 16 results in a black video on playback (maybe it's just getting too slow to be displayed properly?)
* opening the file in VirtualDubMod and viewing individual frames results in a crash

I'll try to look at the last one in the evening.

Regards,
JJ

[General Lee D. Mented]

Originally posted by JJ
Hi!

I've compiled the source you provided, but the image was garbled and displayed upside-down on decompression. I was able to (at least partially) fix it by applying the following changes:

driverproc.cpp

line 59: shouldn't there be a return statement? (this shouldn't matter though)

vfwhandler.cpp

lines 300-301: I think the values for biPlanes and biBitDepth should be 1 and 24 respectively.

line 343: I've changed it to:
if (output->biBitCount == 24)
return ICERR_OK;
else
return ICERR_BADFORMAT;

After that I was able to play the previously created AVIs.


Ok, that's probably what was causing the issue, but now I have to figure out how to test it when the colorspace is something other than RGB. At least you got it to display, which is very helpful.


I believe that in warpframe.cpp there should be a 'case 0:' statement around line 76 - this will stop VirtualDub from complaining about not being able to find a suitable format.


You're right, I think I added it in a switch above and forgot to add it in the other switch there.


What still doesn't work for me:
* setting bitrate to anything but 0 - the codec seems to enter an infinite loop there


Bitrate 0 specifies lossless mode, and the lossy part of the code hasn't been tested and debugged yet, so I expected that. I needed to get the display working right before debugging that part.


* setting block size higher than about 16 results in a black video on playback (maybe it's just getting too slow to be displayed properly?)


It's most likely too slow. If you let it sit for a minute it would probably start playing. It needs to get past a dummy buffer full of blackspace before it can play, and 16 frames will probably decode fairly slow on most machines right now.


* opening the file in VirtualDubMod and viewing individual frames results in a crash


Yes, seeking doesn't work because it screws up the buffering. I may be able to find a way to handle crashes gracefully but I don't know if it will ever work properly in VFW at all.


I'll try to look at the last one in the evening.

Regards,
JJ

Thanks, you've been alot of help. Now I have to reinstall my compiler and see what else is broken.

[JJ]

Keep up the good work!:) This project is too good to be abandoned!

[General Lee D. Mented]

So umm... did anyone actually watch the video? Any thoughts or feedback?


BTW sorry about the delay in fixing code, I lost my CDs for my compiler and am having some trouble getting new ones to reinstall it.

[RadicalEd]

I may be missing something, but it looked like you were comparing a hilbert scan on pixels to a zigzag scan on pixels, which doesn't happen :| The zigzag scan is performed on the transform coefficients, which doesn't interfere with locality of pixels whatsoever.
Video was well done though, IMO. I still have a bit left to go.

[Manao]

I did watch it, it was interesting. Indeed, when speaking of the hilbert scan, the link to the diagonal scan of the DCT coefficients was wrong, but that's not the point, the hilbert scan is important for the encoding scheme he is using.

I wish I had some time left to help.

[bill_baroud]

hmm watched it too, the funny thing is that Gabest AVI Splitter doesn't like this file :)

unfortunatly, i'm not a dshow/vfw/anything else expert, so i fear i can't really help there :/


(and just tried to compile, my visual studio install is borked. geh.)

[virus]

Originally posted by RadicalEd
I may be missing something, but it looked like you were comparing a hilbert scan on pixels to a zigzag scan on pixels, which doesn't happen :| The zigzag scan is performed on the transform coefficients, which doesn't interfere with locality of pixels whatsoever.
yeah I noticed that also, but I'm at a loss understanding spoken English so I thought it was just me...

Anyway, I can only say that I don't understand that "I will find the same color/luma transitions again and again"-thingie. Is this supposed to exploit spatial or temporal correlation?
Because for the spatial corr. it seems a non-sense to me, objects are not uniform in color...
If it's meant to exploit temporal corr., well, and I'm not so sure it works... in a real scene blocks not only move, but also change (slightly) colors due do to the different lighting. A MC'ed, DPCM-based codec can cope very well with that, but what about this scheme? Probably I simply misunderstood this part of the video :D

[General Lee D. Mented]

Originally posted by virus
yeah I noticed that also, but I'm at a loss understanding spoken English so I thought it was just me...

Anyway, I can only say that I don't understand that "I will find the same color/luma transitions again and again"-thingie. Is this supposed to exploit spatial or temporal correlation?
Because for the spatial corr. it seems a non-sense to me, objects are not uniform in color...
If it's meant to exploit temporal corr., well, and I'm not so sure it works... in a real scene blocks not only move, but also change (slightly) colors due do to the different lighting. A MC'ed, DPCM-based codec can cope very well with that, but what about this scheme? Probably I simply misunderstood this part of the video :D


Yeah I'm a bit rusty on the scan order stuff, I had thought mpeg used a zigzag scan of pixels, I guess I'm just too out of date on the subject.

The correlation is temporal, and the reason I believe it will work is because the small variations will result in small coefficients in the transform, which are then dropped by the lossy stage. The idea is that the small variations in brightness from frame to frame are mostly noise and are ignored by the viewer anyway, so they can be dropped. There is some spatial correlation because there's more than one pixel in a transform block (usually) but that's secondary and over a relatively small area anyway.

[virus]

Originally posted by General Lee D. Mented
The correlation is temporal, and the reason I believe it will work is because the small variations will result in small coefficients in the transform, which are then dropped by the lossy stage.
Ok, so you're saying you want to exploit the similarity between trasformed blocks in different frames. Thus you are replacing motion estimation with a LUT search to find a match for the current (transformed) block, right? If it's the case, I guess there may be big complications in the coding stage (i.e. the "zip" compression): the LZ dictionary in two dimensions can be huge.

Also: how can you cope with high bitrates/lossless? If you don't drop enough detail, the number of possible (small, noise-related) permutations of a block will make your memory requirements explode I think...

(please excuse my array of question, but I've never seen a scheme which works _entirely_ on "absolute" signals instead of differential ones. Again, I may have misunderstood your words due to difficulties dealing with a stranger language ;))

[General Lee D. Mented]

Originally posted by virus
Ok, so you're saying you want to exploit the similarity between trasformed blocks in different frames.

Most of the similarity should come from in the same block, as nearby pixels will be in there. The blocks are 4D of data, x/y/t/l.


Thus you are replacing motion estimation with a LUT search to find a match for the current (transformed) block, right?

No, there's no LUT search or any kind of matching for blocks, just straight repetition in string searching.


If it's the case, I guess there may be big complications in the coding stage (i.e. the "zip" compression): the LZ dictionary in two dimensions can be huge.

This is a known problem, hence my stated preference the use of large dictionary systems like 7zip's LZMA or unlimited or dictionaryless systems like burrows wheeler transform (bzip). I'm working on an algorithm of my own design now that has the unlimited full analysis features similar to BWT on the encode side but is a simple find/replace similar to LZH on the decode side, so it's asymmetric. I'm not concerned with encode performance, only decode and even then speed is not a high priority on my list.


Also: how can you cope with high bitrates/lossless? If you don't drop enough detail, the number of possible (small, noise-related) permutations of a block will make your memory requirements explode I think...

There's no permutations because there's no block matching system. If I were to implement that it'd just be another way of doing motion estimation. That doesn't mean I'll never do that in the future but I doubt I'll be doing it anytime soon. Large memory use has been an issue since the start but there's not much of a way around that when doing a temporal compression of any significant number of frames anyway. The encode side has a much higher memory footprint than decode when in lossy mode (lossless is about the same) but decode is still quite large by the standards of other codecs.


(please excuse my array of question, but I've never seen a scheme which works _entirely_ on "absolute" signals instead of differential ones. Again, I may have misunderstood your words due to difficulties dealing with a stranger language ;))

The transform produces a differential signal by its nature. The resulting coefficients are the average and difference from average for pixels in groups of varying size from 2 pixels up to half the number in the block. If we transform two identical sets of pixel values (like say, two neighboring pixels), we get a resulting transform of all 0s. If they differ at all, we get just the differences at the points at which they occur.

[virus]

Originally posted by General Lee D. Mented
The blocks are 4D of data, x/y/t/l.
oh, sorry, I missed that :rolleyes:
So time is already a dimension of the block... this makes sense. I thought they were 2D only.

The transform produces a differential signal by its nature.

yeah, ok, this is true for all block trasforms, but when used in 2D they need to be coupled with a DPCM stage for the third dimension, so you need to explicitly compute a difference. You're just replacing this stage increasing the dimension of the trasform (at least as far as I can understand ;)).

What to say? Some of this stuff is far beyond my skills so I won't comment further on that. Nonetheless, some of these concepts are fascinating. I think the key factor for the success of this scheme is your brand-new entropy coder, but maybe I'm wrong (again :D). Anyway, thanks for your patience. I'd really like to see this codec become a complete project.

cheers :)
virus