about an hybrid codec idea...

[morsa]

I had this idea and I want you to tell me if it is a nonsense.
I wonder what would happen if I made a codec this way:

(source YUV 4:4:4)

Luminance = lossless compression (2:1 or more if posible)
U and V planes = JPEG2000 lossy and very compressed.

Would it have good quality?
How many generations could be supported before becoming trash?

Thank you very much for all your answers..

[Nic]

Could you tell us a bit more? Like why you'd choose 4:4:4 if you were going to have the chroma very compressed? Why you need lossless luminance? etc

Lossless luminance would result in a huge bitrate...but is that what you intend? and if so why? :)

-Nic

[morsa]

@Nic

YUV 4:4:4 because the original (in this case) would be RGB.

I guess compressing the whole resolution color plane instead going the usual way (like 4:1:1 and the kind) would give better results and a sharper image (and no color stairsteps, bleeding or anyother name for it).

Lossless Luminance because it will retain better detail.

I think about it as a capture format, or storage format for later processing,etc,etc.

If we have a 30 MB per second uncompressed RGB stream this would give as 5 MB for the Luminance (or Luma, don't know which is the correct one) and may be 2 Mb for the color planes. 7 Mb sounds great for me!!
Notice that going from 10 MB to 1 MB for JPEG2000 is not an extreme compression.

Hope things look clear now.:)

[Nic]

7mb per second? That would end up using 25+ Gb per hour of video? I don't think this is a good solution...but an interesting idea all the same. :)

[morsa]

think about 1920x1080 :) with better quality than HDCAM..

[bill_baroud]

:eek: you own a cam like that :eek: ?

/me is becoming very envious ;)


it sound a little like ffv1, where you compress something lossy and then compress the difference between the original and the lossy one.

But using jpeg2000 (wavelets) sound a little ambitious (as there isn't any -well almost- mature wavelet video codec). Sound like a good idea to me anyway.

[MasterYoshidino]

how about subsampled luma and full sampled chroma?
(subsampled luma on anime would not compress as well as subsampled chroma but the apparent quality would be higher since chroma artifacts plague homogenous material)

[morsa]

What is the compression ratio of ffv1?
I mean full Luma and lossy full resolution chroma would give the best results in terms of quality.
I think this way a free alternative to Aspect HD (cineforms's codec)would be possible....
This could also open the way for cheap High Def cameras (home made)

I'm planning to use this:

http://www.j2000.org/

Anybody knows how fast is it?
Can I use lossless jpeg for the luma with the same library?
Or is it better to modify Huffyuv to use its lossless part for luma and add the jpeg for chroma?

[bill_baroud]

i would do the latest ... huff the luma and jpeg the chroma.

btw what do you mean by "homemade" hi-def camera ???
i was browsing the web yesterday to find the best tiny camera for an "on-board" system and found some things like : http://www.siliconimaging.com/SI%20Main3.html

i don't know the price, but i suppose it's high :) so i'm pretty interested by your homemade stuff ;)
(diy forever !)

[morsa]

Just in R&D state...
But I would like to have a codec like that for editing purposes and for the open source community.
Cineform's costs around 1200 bucks...

Don't remember well, but I think the 3.1 megapixel model costs around 3,000.

[bill_baroud]

3000$ :eek: i'm eager to hear about your diy hdcam ;)

btw, i just take a quick look at huffyuv, and all the compress part is in assembly, which i have some hard time to understand ;) but at least i can see that it will be hard to separate the channels for compression (i.e. the methods are called asm_compressYUY2, asm_compressRGB24).

better start something from scratch.
(bond mode :sly: : everything but vfw !)

[stephanV]

well, for editing purposes it better be VfW, where are all those opensource DShow editing tools? and it probably would be a keyframe only codec? no problems at all for VfW.

[morsa]

Exactly!!
Keyframe only.
Could someone point me to a simple VFW codec made in C or C++ to start working with?
Also if I could manage to make a proof of concept version I would need some experienced developer to get a decent version of it.

[bill_baroud]

look at huffyuv source code, it's quite simple ... i remember the warp vfw code too, although it's not working it's very clean.

some ms stuff :
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/multimed/htm/_win32_video_for_windows.asp

[morsa]

Well, this looks to be posible.
My last tests show this for 1920x1080x8 24 fps:
Luma (lossless): 22 MB
Chroma (U): 1,5 MB
Chroma (V): 1,5 MB

For a RAW Bayer pattern from a 1920x1080x8 sensor:

Luma (lossless):11.5 MB
Chroma (U): 450 KB
Chroma (V): 450 KB

Both give visually lossless results.
The Bayer gives better results after decompressing and demosaicking!
Now I need to know if J2K can perform lossless on Luma and lossy on chroma at the same time.
See ya...

[bill_baroud]

hi, i played a bit with libj2k, and found something very interesting : you can set the compression you want on the plane you want. i.e if your image is YUV, you can choose to encode luma with xx bits and then the next two plane with yy bits. I still have to found how to access to the lossless mode (if it exists), i didn't find much info ont it, but it's a part of the standard (not like jpeg) so it should be somewhere in the code.

there is a bad point with jpeg2000 though : the speed, it takes about 2sec on my XP 2.2ghz (something like 3400+ :D) to compress a 1024x768x24 image :(
well i've some code for a DWT on gpu using pixel shader, but it would just be 50% faster, and still a no-go.

[morsa]

well, the 5/3 is suppossed to be the reversible transformation (LOSSLESS).
Yes I know it isn't very fast, but 2 seconds on your machine is too much.Don't know what would happen striping the unnecesary parts and optimizing the code.But seeing that MJPEG2000 codecs exist, and that they are really fast on new machines (also lossless compression is faster for these codecs)there must be something really bad with this library.
Good to hear such a good news, cause I'm a real noob about coding and it would have taken me a pair of months to put something together.
Some questions:
what happens when you compress an image of 512x768?
Does it take 1 second to compress?
And 512x384? half a second?
Thank you.

another one:

http://www.ece.uvic.ca/~mdadams/jasper/#software

[bill_baroud]

well i didn't code anything atm, just compiled some tools to play with, and browsed the code. I was planning to wrote a proof-of-concept directshow filter, but i've some linking problem atm so i didn't start to write anything, but it won't take that long. Lib2k is well written and easy to understand, and plain C (iirc, and that's why is slow, well perhaps).

for the speed, well i perhaps exaggerated a little, but that's about it. I'll do more test as you suggested.

i've already looked jasper too, and it's a modified version of it which use pixel shader for DWT (40% of total cpu time, next 40% is for "tier 1" and 20% for the rest, dixit a paper on parallelising DWT). Too bad, the lossless mode doesn't use this shader code, so i suspect something fishy with lossless.

and i'm a n00b at progamming too, just scholar knowledge ;)

[bill_baroud]

here your speed-tests ;)


anime style - flcl-bleu
-----------------------------------------------------------------------------------------------------
tgatoj2k.exe -i flcl-bleu-1280x720.TGA -o f-b-1280x720.j2k -r 1000000
-----------------------------------------------------------------------------------------------------

1280x720 : 512x768 : 512x384 :
tile encoding times: tile encoding times: tile encoding times:
img->tile: 0.050 s img->tile: 0.020 s img->tile: 0.010 s
mct: 0.040 s mct: 0.020 s mct: 0.010 s
dwt: 1.162 s dwt: 0.461 s dwt: 0.220 s
tier 1: 1.522 s tier 1: 0.470 s tier 1: 0.260 s
ratealloc: 0.040 s ratealloc: 0.010 s ratealloc: 0.000 s
tier 2: 0.000 s tier 2: 0.000 s tier 2: 0.010 s
total: 2.814 s total: 0.981 s total: 0.510 s
2 764 818/313 799 1 179 666/84 839 589 842/48 585


real world style - flower
-----------------------------------------------------------------------------------------------------
tgatoj2k.exe -i flower-1280x720.TGA -o flower-1280x720.j2k -r 1000000
-----------------------------------------------------------------------------------------------------

1280x720 : 512x768 : 512x384:
tile encoding times: tile encoding times: tile encoding times:
img->tile: 0.050 s img->tile: 0.020 s img->tile: 0.010 s
mct: 0.040 s mct: 0.020 s mct: 0.010 s
dwt: 1.162 s dwt: 0.461 s dwt: 0.210 s
tier 1: 2.804 s tier 1: 1.362 s tier 1: 0.751 s
ratealloc: 0.060 s ratealloc: 0.030 s ratealloc: 0.010 s
tier 2: 0.010 s tier 2: 0.000 s tier 2: 0.010 s
total: 4.126 s total: 1.893 s total: 1.001 s
2 764 818/898 285 1 179 666/420 003 589 842/224 108


real world style 2 (more complex) - IMG
-----------------------------------------------------------------------------------------------------
tgatoj2k.exe -i IMG-1280x720.TGA -o i-1280x720.j2k -r 1000000
-----------------------------------------------------------------------------------------------------

1280x720: 512x768: 512x384:
tile encoding times: tile encoding times: tile encoding times:
img->tile: 0.050 s img->tile: 0.070 s img->tile: 0.010 s
mct: 0.040 s mct: 0.020 s mct: 0.010 s
dwt: 1.152 s dwt: 0.461 s dwt: 0.220 s
tier 1: 3.645 s tier 1: 1.732 s tier 1: 0.931 s
ratealloc: 0.060 s ratealloc: 0.030 s ratealloc: 0.020 s
tier 2: 0.010 s tier 2: 0.010 s tier 2: 0.000 s
total: 4.957 s total: 2.323 s total: 1.191 s
2 764 818/999 206 1 179 666/509 955 589 842/275 597


damn slow, isn't it ?

[morsa]

Sure, really really slow indeed!
Well, just to point out, a real test would be if your source is 1280x720 to test 640x720 and 640x360 :)
Anyway I don't believe results would vary so much...
One person at Cineform (Aspect HD) told me it would be really fast (but I guess with some tricky optimization and manipulations );)


Cineform guy's example for 4:2:2 :

"Instead of storing all the green as one image, consider the green as two half res. images of separate channels. You now have four channels : R,G1,G2,B. For a 1280x720 image that is four planes of 640x720. Now that they are the same size the compression can be optimized for the human eye by compressing the color planes in this form

G = G1 + G2
Gdiff = G1 - G2
Rdiff = R - G
Bdiff = B - G

Using this lossless transform, G now contains a noise reduced luma approximation green channel (only apply light compression.) Gdiff contains green noise and edge detail (very compressable). Rdiff and Bdiff are approximations of U and V chroma difference channels, these can be compressed more than red and blue native channels has the "luma" component has been removed. Consider a black and white image -- this is the type of image the human eye is most sensitive too; here R,G,B channels contain the same data, you would be compressing the same data three times. In a B&W scene Rdiff and Bdiff would be mostly zero with some edge detail. Now we aren't shooting B&W scenes (the world is color) but our eye can be fooled. Moving the significant image data into G, allow the compressor to optimize to the way you see.

All the can be done easily on the fly in software compression, the trick is to do it fast"

When he says half resolution he is talking about, for a 1280x720, two images of 640x720 created this way: first image will contain all the odd columns and the second all the even ones.


Lookin for more info I 've discovered some other codecs called EZW, SPIHT, and some others I don't remember.They seem to be very fast on a PC. 256x256 image in 0.02 seconds on ! 1GHZ CPU
EZW is supposed to be a part of JPEG2000 standard.


Some implementation:

http://pesona.mmu.edu.my/~msng/EZW.html

Original Matlab:

http://perso.wanadoo.fr/polyvalens/clemens/ezw/ezw.html

[bill_baroud]

huh, really interesting indeed, and tricky :D
...you can easily waste a lot of cpu time with this re-arranging stuff, better to be carefull while programming it.

btw, did you look at this topic : http://forum.doom9.org/showthread.php?s=&threadid=78579 ? it seems he want to achieve something like you want.

anyway, i started to write some vfw stuff, but had to mix c++ and c (libj2k is c) and i don't like the crap i wrote, so i'll restart something from scratch or so instead.

[morsa]

Yes, that guy is talking about his terrific codec since a couple of years.I'm not interested:)

[ntojzan]

Thanks for your most apprechiated opinion. However I'll have to correct you:
a) I'm only registered here since a week or so, so that makes me unable to talk about my "terrific" codec for years.
b) I only started the development a few months ago.

Thanks anyway.

btw: the R G1 G2 B solution will not work, because the RGB components are usualy not related. Also the idea is not too fresh, I used it in several demos for the good old Amiga. That computer had a specific graphic mode called HAM8 (hold and modify), which was used to simulate a truecolour displaymode. It worked on very similar way, just the goal was opposite, for the rendering resolution of 320x256 I used a hardware resolution 640x256 and stored the information just as described by you. (that WAS a couple of years ago)

Concearning the other idea of using lossless compressed Y and JPEG2k U and V, that is not a bad idea, however the maximum compression ratio would be less than 1:6, so it makes it a bit obsolete.

[ntojzan]

Sorry for double posting, but you've stated an opinion on my work, so I'd like to get an explanation.

I'll state right now that if you do convice me that you're right I'll drop the entire project immediatly.

I'm just asking a simple question: why is my codec terrible?

[morsa]

I said "Terrific" not terrible.It is a good thing not a bad one.
Go take a dictionary....
If you don't trust people from Cineform is not my problem.They have a great and really fast codec that sells for around 1,200 dollars.So I guess they are right.Compressing 1920x1080 realtime at a bit depth of 10 bits per color channel is not an easy task.
I'm still waiting for your example videos or frames.This means before an after compression/decompression.
Here, we are looking for an open source alternative to it.Not trying to get the maximum compression possible but a mild compression with the best quality.
If we could go from 300 MB/s to 50 with minimum degradation and 4:4:4 that would be really cool!!
BTW, why are you here with such a bad attitude?
Are you opening your codec?:)

[ntojzan]

> Go take a dictionary....

Oh, sorry, I realy did mussunderstood it... Just please consider that english is far from my primary spoken language, and that I've read your post around 3AM in my timezone...

> Compressing 1920x1080 realtime at a bit depth of 10 bits per color
> channel is not an easy task.

Agree, I just find that compression ratio too low to be usable. However that's only my opinion, so it doesn't matter that much.

> I'm still waiting for your example videos or frames.This means
> before an after compression/decompression.

I can send it via e-mail as soon as you want, I also have a support page, the problem is that it's on angelfire, and they dont allow mentioning homepages on forums - my previous site was banned for that reasons.

> BTW, why are you here with such a bad attitude?

That's just my usual writing style, not ment to harm anyone.

> Are you opening your codec?

I dont think I'm gonna open the source, however I'll surely "open" the technology behind to make it possible for someone else to write it. Also the decompression part will be open sourced, so that is half the job done...

[morsa]

Nice :) .
Just my opinion.I think you will have really a hard time trying to complete your codec to sell it to users.I guess it would be easier to make a working version and trying to license it to other software vendors or something like that.
For me the easiest way would be to make a hardware implementation of it ( as an IP core) may be using FPGA and selling it to hardware manufacturers..

[ntojzan]

Well, I'll do the hard way anyway... :) Thanks for advices tough, and good luck with your work.

btw: I looked at the dictionary, and found:
terrific: 1) scary, 2) horrible, 3) great

[bill_baroud]

well i could propose you my help, but i'm in no way an "experienced" codec/win32 developper ... that's why i'm trying to implement morsa's idea, to code something usefull this time and gain some knowledge ;)

[ntojzan]

Thanks for your proposal, however in the meantime I've found some people willing to join this project, and now the team is complete. :)

[morsa]

Great news!!
Good luck and keep us informed.
BTW, what do you think about EZW?

Another thing:
I've just tested a 640x720 clip with Huffy and it gives me 2.3:1 compression ratio and a speed of 25 fps on a P3 1GHZ with a slow disk....for an RGB compression.

[ntojzan]

I'm not too familiar to the EZW, but from the information I have, it is too computationally expensive for use in realtime encoding. I might be wrong tough. ;)

[bill_baroud]

hmm i did some test today, playing with yuv planar format and ezw.

apparently, ezw is much faster than libj2k, for the same quality (i.e size), but perhaps too slow for real time, yeah. I achieved an 1:6 compression ratio using zlib for the Y plane though.

no test on video atm, and i fear that using ezw as lossy compressor for U/V plane will demand a lot of work (hmm the guy use float everywhere with some funky array structures... i don't really like it, speed wise.)

well, i'll let you know if i've something new.
(and if you have some other algorithms to try on the lossy part, don't hesitate to let me know ;))

[morsa]

Great news!!!
I'll keep searching.By the way if something workable exists, someone here could optimize it for SSE,etc...

Here it comes :) :

http://ai.fri.uni-lj.si/%7Ealeks/Wavelets/index.html

[bill_baroud]

did you take a look to Jpeg LossLess ? the demo code i've here (but unfortunatly it's a mess and hard to use somewhere else -there is no function Compress(char * source, char * dest) :o :D -) is damn fast (0.02sec for 832x480) and achieve an 1:5.76 compression.


anyway, today i fought with ezw and lost... the guy tied all the output code to disk file and i didn't found an (easy) way to output to memory...i'm too lame :(
That's a common problem i encounter with a lot of those "demo" code, everything is tied to picture file and i don't have the mathematical background to rewrite it in something more usefull, or i lack the programming skill to find a work-around.
And rewriting ezw2 if it's too slow ... i'm not that motivated ;)

geez, that wasn't a good day.

[morsa]

Well, if Lossless Jpeg is so damned fast it would be really good!!!
Then we will have a fast codec better than huffy!!
0.02 means 50 fps.On what system?

Can't LS-Jpeg be merged into Huffy structure? (I mean the C++ version).
Anyway there is still the problem with blue/red or U/V planes.
I believe they should be lossy and wavelet compressed.
You can look into the Dirac codec source code.It uses wavelets, I guess.(sourceforge).
If its wavelet part is useful then we've (you've) got the basis for a nice codec!

May be we should decide what would be the way and ask for help?

[Tommy Carrot]

I believe LOCO codec is based on LS-jpeg, and its source code is available here (http://forum.doom9.org/showthread.php?s=&threadid=53305), perhaps you can find useful parts in it for your codec.

[bill_baroud]

yep, Loco is based on jpeg-ls ...thanks for the sources, i'll check that . But for the really quick look, it's look like it's compressing everything in one pass (well i think).

@morsa : the vfw api is very simple, what's i've hard time is to have a function that take a buffer and compress it with wavelets ;)

i'm currently looking at the wavelets compression c++ base code which is used in the ezw2 implementation... much much nicer...but now it's m$ compiler that is complaining ... damn :)

but today is a REALLY good day, i've got my "ticket" to enter that > http://www.egid.u-bordeaux.fr/fdessimage/ \o/

[morsa]

"to have a function that take a buffer and compress it with wavelets"
That`s why I pointed you to Dirac´s source code.It has classes for compressing with wavelets, etc.
I thought that looking at its source, the clue for memory buffering and compression would be there.
Anyway, every news coming from you is a good one.....See ya!

[bill_baroud]

ok, let's take a look at Dirac then :D

news: got a zlib+loco(lossy mode) mode working ... achieving 1:4.5 compression ratio, with a lightning speed of ... 0.06sec for a 832x480 yuv frame. which is a little too slow for real time (and i'm not using a slow computer) :(
i'm working on a vfw version, but i'm somehow stuck with the bitstream to adopt for my compressed frame and some other buffer issues...

anyway, if you had some raw material, so i can test with the content you want to compress, it would be cool ;)
and what's the minimal speed on which processor you want to achieve ?

oh, and from my timing test, ezw2 isn't that faster than libj2k :( ... about 1 sec for one frame.

[morsa]

"zlib+loco(lossy mode) mode working ... achieving 1:4.5 "

I don't think that's a good Idea.Zlib? Why don't LS Jpeg on luma?
You told it takes 0.02 to compress a complete frame(RGB)

"0.06sec for a 832x480 yuv frame."

Huffyuv is faster than that for a 640x720 frame,Why?

I'm trying to get some RAW material.Anyway I would like this codec to be available to anybody that needs to edit HiDef material the cheap and Good way...:D

About timings, I've been told that the most computing intensive task is the entropy encoding,(e.g.Huffman) not a DCT transform or a DWT one.
What would happen if we use the really fast DCT- (lossless means without quantizing)entropy coding scheme for luma, and DWT(with quantization) for colors.Also the entropy encoding of the quantized data is usually faster.
What I'm trying to say is that I don't know how but there must be some way to bypass color conversion routines in Huffy and send it this transformed data.
:D :D

BTW: How fast is your machine?

[bill_baroud]

Why don't LS Jpeg on luma?
because it's slower :)
0.02-0.03sec for loco and 0.013sec for zlib.
if you tried loco vfw, you'll see that it can't achieve real-time compression (22fps here).
Jpeg2000 in lossless mode would be even better, but still, we need a fast library :(

Huffyuv is faster than that for a 640x720 frame,Why?
because the resolution isn't the same :p and that huffman compression is faster than DCT+huffman+some other things

What I'm trying to say is that I don't know how but there must be some way to bypass color conversion routines in Huffy and send it this transformed data.
i'm not really following you ... you want to use the huffman code of huffyuv after dct and dwt ? why not :) ... but huffyuv code is hard to read and the main routines are written in assembly...which is somehow hard for unexperienced person like me.

oh, and my processor is an athlon xp @ 2.2ghz (3400+ like or something)

[morsa]

well, seeing your data a mixture of Zlib with LOCO would give 0.053 for that image color.A total of less than 20 fps on your Athlon.
Anyway a XP 3400 isn't as fast as a P4 of 3.2 GHZ, isn't it?

I know the resolution isn't the same, but a 640x720 RGB image is bigger than what you are using and gives realtime with huffy on a XP 2000+.I can assume that the compression of only an 8 bit image would give at least twice this speed (around 60 fps)

Usually, you first transform with a DCT or DWT and after that, if your compression is lossy, you quantize this data and discard some frequencies, after this you "zip" it.
So I think , if we know that the slowest part of the process is the "zip" part, why not to use the fastest "zip" compressor available for video??
So we go this way a DCT without quantization (lossless) for Luma and a DWT with quantization (lossy) for chroma.Finally the entropy encoding (e.g. Huffman) for both.I guess bypassing the initial image manipulations of Huffy and feeding it with this data would make it.BTW I may be totally wrong.

The exact original purpose of this codec would be the compression of RAW BAYER images from BAYER PATTERN Sensors.So for example for a 1280x720 image your RAW image will have 640x720 for GREEN, 640x360 for RED and 640x360 for BLUE.
I think with these resolutions the code you have now will give realtime for sure!!!!!

Anyway this is just one of the possible examples :D


P.S.: Take note that a 1920x1080 24 fps is around 143 MBytes/s, so in the case of compressing a normal RGB source , guided by your comments we'll get: Around 11 MBytes/s for Luma and (with wavelet lossy compression of 10:1)around 4,5 MBytes/s for Red and the same for Blue.Going from 143 MB/s to 20 MB/s with minimal quality degradation (in fact visualy it would be lossless) would be a great achieve.We could go even to 20:1 compression (for chroma) without seeing the difference.

Here is a link MFA posted.It is a paper about wavelet lifting, blah,bla,bla.Really interesting, might speed up things a couple of fps ;)

http://dali.korea.ac.kr/publication/int_jour/paper/IntJour53.pdf

[bill_baroud]

Anyway a XP 3400 isn't as fast as a P4 of 3.2 GHZ, isn't it?

well it depend which p4 (Northwood or prescott) and for which type of application.
p4 is really great at processing integer stream w/o any unpredictable branchment (i.e video processing) and it need a lot of memory bandwidth... well, i've a nf2-ultra, 200mhz fsb with 2-2-3-11 memory timing, which isn't that bad ;) it should be slower though (256kb l2 cache vs 512kb or 1mb ...).
... hmm just checked, 3400+ is perhaps a little over-estimated (it was when running at 2.4ghz ...)

for your dct/dwt stuff :
1) DCT isn't lossless. well, this is what i always thought. at least, the h264 integer dct IS lossless and reversible, and it's one of the major plus of h264 against old mpeg4 asp. (hm does reversible mean lossless ?)
2) DWT isn't as fast as you think ! remember my speed test, 40% of the cpu time is used in the dwt :(
if you have a recent gpu (gfx 5***), i've some code for a dwt using pixelshader ... +40% speedup with my gfx5900 :) ... but still 1.1sec instead of 1.9sec is too much.

So for example for a 1280x720 image your RAW image will have 640x720 for GREEN, 640x360 for RED and 640x360 for BLUE.

so input is RGB ? it will need a conversion to yuv :( which take some time. why only green have full vertical resolution ? and how you have the 1280 horizontal resolution ? it's stretched ?
i'm confused... more details on the format would be cool.


news : i finished today a first version of my frame C++ class .... which is damn slow :p (0.19sec) ... i've to learn how to make fast code :D (well i know why it slow).

[morsa]

Bill_Baroud, please read again my other post carefully, more or less everything is explained there.
DCT is in fact lossles (or can be) what is not lossless is when you QUANTIZE, discarding selected ranges of information.
The same apply to DWT.
Its principle is very simple, you express the info in a different way, select what you think isn't important and delete it.
About DWT, I discovered the reference code for jpeg2000 is extremely slow.That's why I gave you that link, which explains how to REALLY speed up the DWT!!!!

BTW, Do you know what a Bayer Pattern is??

[bill_baroud]

which explains how to REALLY speed up the DWT!!!!

yeah and you need to read it carefully ... first, it apply to dsp (and there is some things i'm not sure if it's possible on a pc, like changing the behavior of DMA access) and second, take the results tab, and do a little math.
with the maximum speed up, you need about 0.0001ms by pixel ... with a linear scale, that give you 92ms for a 1280x720 image, which is too slow. And it's on a dsp, i don't know how it will perform on a general purpose cpu. and i know nothing about assembly programming, you need to find someone to optimize your dwt :/


In principle, the DCT introduces no loss to the source image samples; it merely transforms them to a domain in which they can be more efficiently encoded.

ok, confirmed. but i'm not sure that dct + huffman is really better than huffman only (dct allow you to easily remove the data you don't want, dunno if this reordering only is easier to compress...i've to test it)

btw, for 1:20 ratio, DCT-based codec perform better against DWT-based codec (http://www.acm.org/crossroads/xrds6-3/sahaimgcoding.html).
perhaps only jpeg (non-ls) on chroma will be enough ?

BTW, Do you know what a Bayer Pattern is??
nop, i tried to googling a little, but didn't try hard enough to find usefull info ;)

edit: ok, now i know ... it's like YUYV but applied to rgb ... funny.

[morsa]

First of all, I'm not trying to attack you, just trying to be helpful :D
That paper was posted by MFA (he always knows what he's talking about) to help in the improvement of Vaguedenoiser, which uses wavelets a lot, using DWT, and gives around 23 fps for a standard image.BTW its code could be useful too :)
In fact the cache management would be more or less the same for a CPU, in fact that TI DSP is no more than a CPU core with some add-ons.

DCT+Huffman is what Lossless JPEG uses in some implementations....
The correct terminology would be DCT+Entropy encoding.

I'm not so sure about DCT being better than DWT, at least visually.That paper you posted is a little old, and makes some assumptions based on PSNR that are not always true.As we all know, PSNR although mathematically correct, doesn't always talk about the "visual quality" for a human viewer.It has been demonstrated than you can have an image with a good PSNR that is terrible awfull to look at.
Another disadvantage of DCT (lossy) would be multigeneration.
After, maybe 3 recompressions the image would be corrupted to the naked eye.A DWT would support far less recompressions without being noticeable.
See it by yourself compress the same image 20:1 with JPEG and with JPEG2000.If you can't see it at first, upsize a portion of it.
Anyway if we can go for a 100:1 compression without degradation I would be more than happy :p
I'm working with some people at an open hardware project to develop a HiDef camera for Cinema and/or TV.
That's why I like having the option of compressing the Raw Bayer array but this is a second goal.The first would be to have a multiplatform open source codec suitable for editing HiDef content with On-Line quality on a normal PC with normal disks, not a Dual opteron with 3 TB disk arrays.;)
So, first a YUV or RGB version is the goal.

Really, I'm not kidding, not trying to be agressive.

P.S.: Let's make a deal, when you don't know about something I'm talking about, say it.The same goes for me.;)

[bill_baroud]

Really, I'm not kidding, not trying to be agressive.
P.S.: Let's make a deal, when you don't know about something I'm talking about, say it.The same goes for me.

eh, no problem, perhaps the tone i used wasn't the one i wanted to use as english is not my native language. anyway you somehow confused me while talking about luminance and B/R planes in the same sentence some post ago :p.

DCT+Huffman is what Lossless JPEG uses in some implementations....
hmm dunno, i always read that lossless jpeg doesn't use DCT, at least the late addition to the standard, but i maybe wrong.


news : dct is lossless (tested it ;)), using xvid routine, giving a 15:1 compression ratio when compressing with zlib afterward.
drawback : zlib is damn slow at compressing it, 7 times slower !

otherwise, i looked at the Dirac code, which is very nicely written and documented ... they use a lifting-dwt ;)

writing fast code isn't easy :(

[morsa]

what would happen if instead of Zlib you use RLE??
can't we balance compression against speed to get a fast enough combo of DCT+Zlib?
I mean a 4:1 compression of a DCT.

Just another link...

http://kondor.etf.bg.ac.yu/~dejaniv/projects/dwt/DWT_VHDL.htm

this is really interesting, it is supposed to be included in the Jasper project.It uses the GPU to accelerate the compression, although you need at least a GeForce FX. Anyway.....

http://www.cse.cuhk.edu.hk/~ttwong/demo/dwtgpu/dwtgpu.html

[bill_baroud]

news : no news, small waves today, didn't work on it at all...perhaps this evening :)

what would happen if instead of Zlib you use RLE??
hmm this is exactly what i'm planning to do ;)
i got some asm code and other libraries to test, but i'll do that ... hmm perhaps tomorrow if i've time.

this is really interesting, it is supposed to be included in the Jasper project.It uses the GPU to accelerate the compression, although you need at least a GeForce FX. Anyway.....

what do you think i'm talking about in my last posts, when talking about a dwt using pixel shader ;) ? i even given you how fast it is (40% speedup) :)

[morsa]

Ohh, sorry, I forgot about it.:D
Anyway, thank you indeed.


Here is a RAW capture of a Raw Bayer image sequence


http://www.geocities.com/morsa20002000/morsafilter.html?1089775887118

The arrange is supposed to be this way:
first line GRGRGRGR...
second line BGBGBGBGBG...
and so on...

[bill_baroud]

ok thanks for the picture :)


... now i'm searching some hint for bayer-rgb to something else conversion ...
here (http://www.fillfactory.com/htm/technology/htm/rgbfaq.htm) they explain some methods, is there one you prefer, or for speed issue, we do not convert anything (keeping the RGB planes as-is, with lossless on G and destructive on R/B or lossless for each ?)


hmmm nice patent (http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PALL&p=1&u=/netahtml/srchnum.htm&r=1&f=G&l=50&s1=6,392,699.WKU.&OS=PN/6,392,699&RS=PN/6,392,699)



news : forget the 15:1 compression ratio. today, as i was playing with asm f/idct, i noticed that my picture after dct+huffman was fucked and blocky... reason is, to perform a dct on a yuv picture i had to convert it from 8 to 16bits (integer dct) and convert it back to 8bits for huffman coding. which lead to heavy quantization lol (everything out of the 0-255 range is zero'ed :rolleyes: )
i need to find a way to compress a integer array now :(


BTW, how do you plan to feed the codec with bayer-RGB ?
i mean, there isn't any VFW (or dshow) codec for that (or only a ST Micro. one)

[metaller]

news : forget the 15:1 compression ratio. today, as i was playing with asm f/idct, i noticed that my picture after dct+huffman was fucked and blocky... reason is, to perform a dct on a yuv picture i had to convert it from 8 to 16bits (integer dct) and convert it back to 8bits for huffman coding. which lead to heavy quantization lol (everything out of the 0-255 range is zero'ed )
What are you talking about? If you have 8 bit source, you will get 8 bit output (DCT itself will not change the range of coefficients). Maybe you just made something wrong?

[bill_baroud]

If you have 8 bit source, you will get 8 bit output

what if the dct function take an uint16 array as input ?
uint8 > uint16 is lossless, but uint16 > uint8 isn't. just take the xvid dct routine and look at it yourself, the resulting dct coefficients aren't in a 8bits range (and strangely xvid as some 8>16/16>8 conversion functions)
well if you have a smarter idea than me, go ahead, i'm open to everything.


news : dct+rice coding (it works on uint16 :D) achieve a 4.5:1 compression ratio, with a really nice speed.
now, i'm putting it in a vfw codec to test the speed further.

[metaller]

The typical 8x8 DCT used in video codecs usually uses 9 bit coefficients (-256 to 255), that's why they are int16.

Suppose you do int8->int16, DCT, then int16->int8.
In this case, int16->int8 conversion can be lossless since int16 values are in range -128 to 127 (you'll probably have to modify DCT to clip them to this range).

[bill_baroud]

hmmmm... i'm so lame with types... well say i have some resulting dct coeffs which are 288 and -53 ... how can you put them in an unsigned char * array (0-255 range) ???

[metaller]

You can clip them: if (x > 127) x = 127; else if (x < -128) x = -128;
This is not lossless, but usually only a few of coefficients (if any) will be out of range.

But I guess you don't deed to do that as your goal is to compress them efficiently (probably using some VLC algorithm). You can use 9 bit DCT output (-256 to 255) for that matter. And don't forget to reorder the coefficients to achieve better compression ratio.

[bill_baroud]

ou can use 9 bit DCT output (-256 to 255) for that matter

hmm, how come i've some 288 values in the resulting coefs then ? (i didn't invented those numbers ;))

for sorting coefficients, i dunno, it can be cpu intensive, and on HD material, there is perhaps not enough time to do it.

[metaller]

How do you convert int8 to int16?
If you have unsigned source coeffs then you need to make them signed first. Here's an example:

unsigned char src[64];
short dst[64];

for (i = 0; i < 64; i++) dst[i] = ((signed char *)src)[i];

for sorting coefficients, i dunno, it can be cpu intensive, and on HD material, there is perhaps not enough time to do it.
Simple zigzag reordering of DCT coeffs can be done at almost no cost with table lookup.

[morsa]

Bayer RGB is not a problem, You just feed the codec with it.
Bayer RGB is no more than a Greyscale image of the total resolution with any bitdepth.
I mean for a 1280x720 color image you get a Greyscale 1280x720 image.
Then you seperate the three color channel and process them thru a demosaicking algorithm.

http://www.insflug.org/raw/software/tools/dcraw.php3

Anyway go for the easiest way for you ;).

I guess various options would need to be supported: Full lossless and Semi-Lossy (lossless G/Luma, Lossy RB/UV)

for a 1920x1080 the codec would need to accept a 1920x1080 grayscale image (most of the times 10 bitdepth or more), decompose the image in its three components ,giving you this: Green 960x1080, RED 960x540, BLUE 960x540.
A full RGB version would need to be supported later I guess....

I don't understand your last question sorry.

[bill_baroud]

I don't understand your last question sorry.

about the vfw thing ? well, iirc, when you want to compress a frame, you got a BITMAPINFOHEADER structure or a VIDEOINFOHEADER2 structure (or something like that), which gave you various information about the frame you're requesting, like the width/height etc... and the format of the frame, which is, iirc, a fourCC. You can then process the picture depending of the input (RGB, YUY2, 21YV ...). Afaik, bayer-RGB isn't a "Microsoft" supported format, i'm asking how you will input a bayer-rgb file into virtualdub to process it ? i mean, you need to put it in an avi file and have the format supported somewhere (natively or by an third-party codec).

what i'm asking is, what will be the fourCC, if any, for a bayer-rgb file so i can add it to my switch case ;)

How do you convert int8 to int16?

something like dst[i]=(short)src[i];

Simple zigzag reordering of DCT coeffs can be done at almost no cost with table lookup.
ok, now i need to search what's a zigzag scan :p

btw, i lost my last contact lenses and i don't have glasses, so i won't be really doing something in front of a computer for the next week or so :(

[morsa]

About FourCC, choose any you like.Maybe BYRV (BaYeR Video), or RAWB (RAW Bayer), BPRV (Bayer Pattern Raw Video), or think a name for the Codec and then we can choose a FourCC!! :D :D :D
But if the codec is gonna support other things appart from Bayer RGB (As I hope,usual RGB or YUV 4:4:4) then It should need a more general name.
Sorry about your optical problems.Hope you luck:)


P.S. I've just discovered this:

http://forum.doom9.org/showthread.php?s=&threadid=79526

It's called Bergwave :rolleyes:

[bill_baroud]

hmm, i think we misunderstood each other :D

how the raw frames will be "served" to a vfw application ???
what output the capture utilities ? is there some VDM drivers ? we need to write those capture utilities ? or a rawvideo2avi utility ?

i'm not asking for a name for a new codec, but if this codec actualy exist ;) (one that handle bayer-rgb input/output, like xvid or divx handle YV12)


for bergware, i dled it when he posted it, but didn't have time to look at it. sound a little like what you want btw ;)


edit : btw, the current fourCC is 'HNLC' for Hybrid Near Lossless Codec.
well isn't anymore hybrid, but that is :)

[morsa]

Well there are no definitions about what the frame grabbers output.
I think it is not a big problem to get an AVI to feed the codec.
Gonna search for an aswer....stay tuned...
No more hybrid??
why?

I don't think Bergwave is related with your (our) proposed codec...:confused:

[bill_baroud]

No more hybrid??
why?


well, at the moment that is, but i fear that some performances issues will arise quickly (in YV12, compressing the whole frame directly without any try to separate luma or chroma is much faster ;))

[metaller]

something like dst[i]=(short)src[i];
If src[i] is unsigned, try this: dst[i] = (short)src[i] - 128;

ok, now i need to search what's a zigzag scan
All JPEG and MPEG codecs use it, see the sources.

(in YV12, compressing the whole frame directly without any try to separate luma or chroma is much faster )
AFAIK in YV12 luma and chroma are already separated.

[morsa]

the source could be avi, tiff, jpeg, maybe cineon.


@Metaller
Are you interested about helping with this?
I mean more actively..:)

[metaller]

Are you interested about helping with this?
Of course, why not?

But right now I'm having troubles recovering data from destroyed HD...:(, and also have some other projects to work on, so please excuse me for being not active enough. Just tell me EXACTLY what you need and maybe I'll be able to help you.

And BTW it would be nice to take a look at the source code you're working on.

[morsa]

After bill_baroud makes a first working version, I guess we'll define the steps to follow.
Thank you for your support.:D :D :D

BTW, does anybody know of a way to store 10 bits per color channel images appart from using the alpha channel?

[bill_baroud]

And BTW it would be nice to take a look at the source code you're working on.

uhuh, i don't think it's a good idea, unless you really like crappy things :p


news : incredible speed of 4 fps for my first vfw yv12 only implementation :D

now, i've to find how xvid achieve more than 30fps in the same conditions :sly:.
and to finish writing the decompression part to verify if it's correctly compressed :whistle:

[metaller]

uhuh, i don't think it's a good idea, unless you really like crappy things
Well, if you want to keep it that way...:p

But without seeing a source code, I can't help you much

[morsa]

"news : incredible speed of 4 fps for my first vfw yv12 only implementation "

YV12 ??????
What are you working on??
A DCT lossless YV12 codec??

Explanation of Huffyuv internals...

http://home.pcisys.net/~melanson/codecs/huffyuv.txt

BTW Huffyuv gives me an amazing speed of 80 fps for a 720x380 RGB source on an Athlon XP 2000 (although it is a really cheap system under $200)

There is also this:


http://www.maven.de/code/wavelet.zip

[metaller]

BTW do you want the codec to support interlaced video?

[morsa]

No, Why?
Well at least not me, but at this moment I don't know what bill is working on.
Anyway I guess interlace treated as progressive (frame)won't be a problem.
I'm getting more and more interested about Directshow interface and what a combination of 10 or 16 bit depth Luma with 8 bit depth U and V would give....( you know always the same approach, lossless luma and lossy full resolution color planes,keyframe only)


BTW this compressor seems quite fast to me ( http://www.maven.de/code/wavelet.zip )

Can't you test it? if it seems reasonably fast a VFW interface can be added and set it to compress lossless on luma and 20 or 30 (may be more) on U and V.

[metaller]

Anyway I guess interlace treated as progressive (frame)won't be a problem.
Simply treating interlaced as progressive will lead to inefficient compression and more artefacts (in case of lossy compression) since odd and even lines correspond to different time monents.

BTW this compressor seems quite fast to me
Looks promising:)

Can't you test it? if it seems reasonably fast a VFW interface can be added and set it to compress lossless on luma and 20 or 30 (may be more) on U and V.
I don't see a way to test its speed before adding some (VfW or whatever) interface first. Maybe you have any ideas?

[morsa]

Well metaller, if we are talking about a single chip camera, and only in that case, I'm sorry to tell you that we have a problem, cause, although Luma/green could be interlaced, chroma can't because we don't have two adjacent lines with Red or Blue, just alternating ones.So it is not a good idea to get an interlaced image from that kind of sensors (unless you want to apply many tricks on camera to get an interlaced video with really crappy resolution ;) )

Remember that my original idea was to compress the RAW BAYER PATTERN from a single chip HiDef camera with LOSSLESS for Green or Luma (don't know yet) and lossy for Red/Blue or U/V.
The other scenario could come second or first, no problem about that :)

PS: When I say interlaced treated as progressive I mean double framerate, half resolution :)

Can't a couple of lines be added to its code to show how much time does it take to compress an image??

[scharfis_brain]

@morsa: why should it be impossible create laced video out of a single chip cam?

all cheap cams are single chip ones...

and since the vertical chroma resolution is crappy with DV (at least for PAL), this

RGRGRG
GBGBGB
RGRGRG
GBGBGB

pattern would not produce that much problems....

[morsa]

Hi Scharfis!!!
Nice to see you here!! :)

When did I say it was impossible?? :)

Well the problem is, as you can see, that we don't have the two color planes for every line, so which solution is the most used??

Right you've guessed it!!
Our dear camera manufacturers blend the lines, giving as a blend of two lines for every fields. :)

[metaller]

PS: When I say interlaced treated as progressive I mean double framerate, half resolution
Well, then there's no problem at all.

Can't a couple of lines be added to its code to show how much time does it take to compress an image??
Yes, but one has to compress many different images and measure the average compression time, otherwise results will be somewhat random.
So, some code needs to be added anyway and that's going to take some time...

[morsa]

But it already supports compression of several images!!! (If I'm not terribly wrong):confused:

[metaller]

But it already supports compression of several images!!! (If I'm not terribly wrong)
Bad news: you are terribly wrong:D
Or maybe not? :confused:
If you know how to compress several images with it, tell me.
As far as I understand it only can compress one image at a time, but with up to 16 channels (some of them can be from other images of the same resolution).

[morsa]

That's it, you are completly right!!!:D

So, I don't know what to do.What do you think, worth the effort?

[metaller]

I think I'll try to do something with it, but I don't have much time currently, so don't expect results very soon. But if you can do it youself, go ahead.

BTW this guy wrote his frontend in Delphi, which makes it harder for me to understand:(

[Nic]

Completely by coincidence I was looking at maven's wavelet code recently. It isn't very quick in general (quick enough for image processing, but might be a little slow for video encoding (Still it would faster than dirac ;) ).

Easy to use C code though :)

-Nic

[bergi]

Hello,

i'm the developer of the BergWave codec. I saw you have already looked at my codec. I have added a quant=1 mode for your purpose. It doesn't use the rle coding, just rice coding. But i have noticed that my wavelet transform isn't lossless. I use a small trick i have found in the w3d (tarkin) code for odd and even line sizes. I think this is the problem, but at the moment i don't have the time to fix this. Also the test program ends with an error, but all frames are decoded?!

Here (http://www.bergos.org/projects/bergwave/files/bw20040723s.zip) the last version of the source code.

[morsa]

@Bergi
I've tested Bergwave and it is really nice!!
I think it needs some tuning cause gives me between 6 and 10 fps for a 720x380 source.And I don't know if it is keyframe only or not.
Quality seems could be really good with a lot of improving (which I believe for sure is posible)!!


@Everybody

Well, it looks we're not so bad in our way.
I have new ideas that have been received with joy from codec developers from a couple of companies I can't name here.;)
I'm thinking about the idea of going Sourceforge, but as I lack the coding knowledge and experience to make it myself, I would like to know how many people in this forum want to get involved in this project.
The goal would be to develop an open source alternative to Prospect HD ($1200 the 8 bit version called Aspect HD), but with some extensions.
This could be included later, (my hope) into Cinelerra project and can be extended to accept every standarized resolution (most common 720x480,720x576,1280x720,1920x1080 and 2K).
It would be interesting for an Open Hardware HiDef camera project I'm working on.

[bill_baroud]

hello there :)

no no, i didn't abandoned the project, it's just i don't have a regular internet access (taking a break :p )

i'm currently re-writing my vfw interface in a more modulable way (c++ ... utterly stolen from warp project :D) and cleaning code. I think the next time i'll show here will be with some code to look at.


now, going back to the beach ;)

[morsa]

Welcome back.
I didn't thought you'd given up :D .
It's just that new stuff is available, apart from many other new tricks ;)

[metaller]

I'm thinking about the idea of going Sourceforge, but as I lack the coding knowledge and experience to make it myself, I would like to know how many people in this forum want to get involved in this project.
It looks like we're thinking about the same thing:)

I reviewed that wavelet encoder more closely today, and I think it cannot be used efficiently in video codec as is, but still the idea is interesting. I now think of rewriting it from scratch, with a better code design (more suitable for video encoding), and probably more advanced entropy coding. I don't see a way to use MMX in it currently, but even a regular assembly will help a lot.

BTW what sampling factor(s) you want to support (4:4:4 or 4:2:2 or something else)?

[morsa]

I guess one of the first supported modes would be YUV 4:4:4.
I was thinking about an hybrid between Huffyuv for Luma (Lossless) and wavelet for U and V.A 20:1 ratio I guess....
Anyway I'm really interested about storing a RAW BAYER PATTERN video stream with a lossless/lossy scheme.Do you understand?

The final idea is to support up to 16 bit per channel (at least for Luma)

[bill_baroud]

hello, due to some debugging issue, no working code atm :(
So if someone could explain me how to debug a vfw codec, i'll be glad, 'cause the way i tried worked only once (let the debugger launch vdub and then intercept my breakpoints, but it doesn't work anymore, i really dunno why)

if you really want to see my crappy code, ask morsa for the link ;)

[maven]

Originally posted by Nic
Completely by coincidence I was looking at maven's wavelet code recently. It isn't very quick in general (quick enough for image processing, but might be a little slow for video encoding (Still it would faster than dirac ;) ).

Easy to use C code though :)

-Nic

That was the idea... I still think it is reasonably fast; at most places I'm memory-bandwith limited (after making the lifting transform more cache-friendly) and I'll be buggered if I can find another way to speed up the Rice / Golomb bitplane encoding... :(

[morsa]

@Maven

Happy to see you here!!!
A couple of questions.
Could you tell me what would be the processing time for an 8 bit image of 1920x1080 20:1 compression? (using Kompressor I mean)
Is there a linear relation on number of pixels?


This is because I want Huffy for Luma, and Kompressor for U and V (YUV 4:4:4)

I don't know how Kompressor internals work, but may be cutting the image into blocks could help....

[metaller]

That was the idea... I still think it is reasonably fast; at most places I'm memory-bandwith limited (after making the lifting transform more cache-friendly) and I'll be buggered if I can find another way to speed up the Rice / Golomb bitplane encoding...
Maybe good optimizing compiler could help out a bit?

[Nic]

@maven: It is fast (& impressive), but for doing high definition 30fps video encoding I think it might be a tad slow. Very nice picture quality with it too (I was trying to add 4:2:0 style YUV instead of each channel being the same size when playing with it ;) ) (very nice and small code BTW :) )

@all: Just as it may be of interest, ffmpeg added a new codec called "Snow" very recently based on wavelets (very experimental, but may be worth looking into :) )

-Nic

[metaller]

Does anybody know of (or have) a short downloadable high def. video sample that could be used for codec testing?

[morsa]

You can download some from Microsoft (wmv) and transcode them with Avisynth and Vdubmod.I do it all the time.Just open them thru Directshow.
Remember that if you plan to read uncompressed RGB 1920x1080 realtime you'll need 200 MB/s tranfer rate...
If you made an utility to convert it to a Bayer pattern of 1920x1080 you'll need around 60 MB/s..

[metaller]

You can download some from Microsoft (wmv) and transcode them with Avisynth and Vdubmod.I do it all the time.Just open them thru Directshow.
How about a little more precise link?

If you made an utility to convert it to a Bayer pattern of 1920x1080 you'll need around 60 MB/s..
Yes, this is exactly what I'm currently working on: a DirectShow filter that does RGB<=>Bayer pattern conversion.
BTW can you tell me how do I interpolate missing color components near the edges of image? Should I duplicate first/last row/column to do this?

[morsa]

here is the link:


http://www.microsoft.com/windows/wi...ntShowcase.aspx


Just repeat the nearest pixel, you won't notice much difference :D


Any news??

[bill_baroud]

not really, it's too hot to code atm ;)

i've at least an encoder working (a "format = format;" was missing in the frame constructor :o) , but still 5 fps, damn :(

[maven]

Originally posted by Nic
@maven: It is fast (& impressive), but for doing high definition 30fps video encoding I think it might be a tad slow.
True. I've thought of doing parts on the GPU (which is reasonably easy for the wavelet transform), but as the transform isn't the bottleneck (anymore), I'll probably become bandwidth-limited when doing the bitplane compression on the CPU.

I'm not quite sure how to improve it. Maybe doing a different sort of encoding (something like arithmetic coding / q-coding / stuff by Wm. Douglas Withers) but that'd need some prediction.

Very nice picture quality with it too (I was trying to add 4:2:0 style YUV instead of each channel being the same size when playing with it ;) ) (very nice and small code BTW :) )
Ta. I thought about adding stuff like that but found it so much easier (and more consistent) to just apply a lower target-mse for the chroma channels. :D

[morsa]

Replacing the Rice encoding with a Huffman Huffyuv style could work?
I mean speed up things a little......?
Some time ago I also read something about optimizing the wavelet compression avoiding cache misses, something about using 64 K windows or tiles...don't remember the link, I guess mfa posted it..

Also the main goal at the moment is to achieve 24 fps..

[maven]

Originally posted by morsa
Replacing the Rice encoding with a Huffman Huffyuv style could work?
I mean speed up things a little......?
It's not quite as easy, as you'd need some sort of prediction / frequency counts to choose the right symbols (which is something you obviously don't need for Golomb / Rice coding of bitplanes).


Some time ago I also read something about optimizing the wavelet compression avoiding cache misses, something about using 64 K windows or tiles...don't remember the link, I guess mfa posted it...
That's the way JPEG2000 does it IIRC. But I've made my wavelet transform cache-friendly by essentially doing the horizontal and vertical transforms in one go (which accesses four image rows in parallel; those usually fit in the cache quite nicely). It is indeed a tremendous speed increase (>200% I reckon).

[morsa]

But isn't Golomb/Rice slower than Huffman or RLE/Huffman combos?
At least that is what I have read everywhere....

[metaller]

Also the main goal at the moment is to achieve 24 fps..
This is meaningless without your target system description.
Replacing the Rice encoding with a Huffman Huffyuv style could work?
AFAIK huffman (or arithmetic) coding doesn't work well with wavelet transform output as it is designed for different data source model.

[bill_baroud]

http://forum.doom9.org/showthread.php?s=&threadid=80690

some interestings posts, if you missed it... :)

[morsa]

It looks that this is loosing its way :)
Much ado about nothing......
Anyway, who knows may be someday....
This days I'm experimenting with the most stupid huffman encoding possible (fixed table) and it doesn't look too bad for me.....
around 1.5, 1.8 ratios....
Next I will add RLE after tweaking the symbils from the table to see what happens.I never imagined Huffman was so easy and fast :D

[bergi]

But isn't Golomb/Rice slower than Huffman or RLE/Huffman combos?

The golomb/rice coding is only slow inside maven's code because it works with bitplanes. If you have no fixed table huffman is always slower than rice.

I have done some test and the best solution is rice coding in lossless mode and rle/rice coding in lossy mode, but all on the complete coefficients. Working with bitplanes is to slow for a video codec.

Rice is nothing else than huffman with a special fixed table, but the table is generated on the fly. If you would generate a fixed table for wavelets it would be the rice table to a high percentage.

Here some results from my codec working in lossless mode (the wavelet transform has still a little bug, but that's just a rounding error):


Source:
640x464 1024 frames

yv12 size:
640x464x1024x12bits = 3649044480bits = 435MB

BergWave size:
1418772480bits = 169MB

Compress ration = 435/169 = 2,57

[morsa]

if someone is using DCT, here is a nice library to see...

http://www.fftw.org/

[bill_baroud]

interesting, but the benchmark results are in mflops and i'm too lazzy to do the math to have the execution duration, to compare with xvid :)

[vsv]

morsa wrote:
I think about it as a capture format, or storage format for later processing,etc,etc.
...think about 1920x1080 with better quality than HDCAM..
morsa, your dream released:)
Elegance solution from Cineform implemented into SI-1920HDVR tapeless HD camera:
http://siliconimaging.com/DigitalCinema/
http://cineform.com/technology/CineForm_RAW.htm

[bill_baroud]

he already knows about it, and about the price too (20K$+ in your face...)