Snow wavelet codec

[Sirber]

Yep. Just H263... :(

[plonk420]

hmm. i must be thinking of something else.

first test of SNOW has been a bit odd. first encode @ 50% wasn't that good (i AM doing a killer clip, tho). 80% was still pretty painful, but getting close. however, the 90% encode was the same size, or to be exact, 6K smaller. going to try 95% after i go to sleep i guess.

[Teegedeck]

Any news, yet?

Fiddled around with HDTV a bit, and Snow seems to be very good with crisp and clean sources. Though I now think the turning point where Snow starts to look better than XviD is a bit higher than I thought - perhaps around H.263 quant=5 or 6. Where savings start to get really good is more in the Realvideo arena. 85% quality is perhaps the lowest I would go. It doesn't have to hide behind Ateme's h.264 at these bitrates, I believe. At that quality percentage Snow still looks quite yummie with clean sources and is 1/3 or so smaller than XviD at h.263 quant=5 -- it doesn't look so nice when the source is of the shabby kind.

[Nic]

Me and trbarry (when he can) are working on Assembly for it.

I've got snow.c compiling in Visual Studio with Intel Compiler, which makes things a little faster...

Slow and steady... ;) (nothing new from Michael though)

-Nic

ps
I'll encode a Matrix scene and put it up, for you to look at when I get chance :)

[Teegedeck]

Thanks for the work, Nic! :) And so good to hear of trbarry; he's only here on the forum once a month or so lately.

[Nic]

grr, my computer's fan's broken so it was getting too hot to create a demo, i'll try it again tonight, might put up my little commandline app I use for testing too (AVS->Snow AVI)

-Nic

[Nic]

Snow Demo:

http://nic.dnsalias.com/snow_860kbps.avi
(SNOW 860kbps, 18mb, 2:54secs, 640x336)

Only try to play this with the latest version of VideoLan or MPlayer and only if you have a fast computer! (and I mean fast ;) )

This example isn't the best, I just coded it up quick, even with a few tweaks quality can be better, and remember that Snow's overlapped motion code doesn't even work yet....the improvements should be impressive when it finally does.

-Nic

[Tommy Carrot]

Originally posted by Nic
Only try to play this with the latest version of VideoLan or MPlayer

The recent FFdshow builds can play it back too. Btw, this (http://www.fw.hu/carrotland/snow_278.avi) is my contribution to show the low bitrate capability of this codec. ;) It is 278 kbps, and the resolution is 720*288. /Actually it's the same clip i made at the beginning of this thread, but that's not decodable with the current version anymore, so i simply remade it with the latest build./

[Nic]

Nice clip :)

I'm thinking about starting a thread in the development forum to help me with mmx'ing of alot of the code and addressing which bits are slow and how to speed them up (only really looked at encoding so far (as that's been easier for me to test))

recent FFdshow builds can play it back too
I can't remember now how I turned it on in the latest build of ffdshow...but I remember it taking more processing power than MPlayer (which plays it the best (for me) anyway)

-Nic

[Mug Funky]

yes, very nice clip. i seem to be getting some jumpiness - sort of halfway between dropped frame(s) and frame-order problem. particularly at the start of the jump scene (around morpheus' jump).

decoding with ffdshow (haven't downloaded an updated mplayer binary yet, and too n00b to compile my own). think i'll give celtic druid's site a visit.

[Tommy Carrot]

Originally posted by Mug Funky
yes, very nice clip. i seem to be getting some jumpiness - sort of halfway between dropped frame(s) and frame-order problem. particularly at the start of the jump scene (around morpheus' jump).

There is no dropped frame or frame-order problem, it's just snow's horrendous playback requirement. :D Your cpu is probably not sufficient for realtime playback. Snow in it's current form is very unoptimized, but Nic and Trbarry are already working on the case.

[billou2k]

It plays fine here (P4 dual 3Ghz;-)
Quite impressive I must say! 720x288 @ 278kbps...
Of course it's not DVD quality but way more watchable than most lower resolution clips based on DCTs codecs with more than twice the bitrate...
The force is strong in this one!

[Nic]

trbarry is working on little bits I show him when he can, I will start posting in the development forum about it, so perhaps other great coders (like sh0dan, etc) can offer help....But for now I've only been working on the encoding code, I think it's the decoding code that really needs attention

(The clip I made runs fine on my 3.2ghz at home, but my 2.4ghz can barely play it)

-Nic

[Mug Funky]

definitely the decoding side needs work. encoding is expected to be slow, but when people start getting useful decoding speeds the enthusiasm for this codec will grow a lot.

@ tommy carrot: yeah, under the mplayer i just downloaded (defaults to no framedrops), it plays every frame. nowhere near realtime of course :) my machine is old and it's going to stay old - i hate throwing things out, and i can't really afford to anyway.

[edit]

OBMC isn't on yet? wow. i can't wait to see it when it works! it's already giving me bitrates that i wouldn't consider even for audio, but delivering good quality. with OBMC i feel that artefacts would be reduced hugely.

[akupenguin]

OBMC is enabled, but the encoder doesn't take it into account. It runs motion estimation and macroblock decisions based on plain block-based motion, and then overlaps them. This is better than nothing, but far from optimal.

[chilledoutuk]

It does use a fair wack the decoder but never the less it only used 80% of my athlon xp barton core running at 2ghz.

nic the qulity of that scene is nice but out of aspect.

[Joe Fenton]

I was playing the two clips in Fedora Core 3 for the AMD64 on my Opteron 240 (1.4GHz CPU). The snow_278.avi clip takes about 50%-55% CPU and the snow_860kbps.avi takes 88%-94% CPU in ffplay (cheesey SDL player for ffmpeg). This is a straight 64-bit compile of ffmpeg without any optimizations. So I would say that any 64-bit machine will handle snow rather well. Looking forward to future updates to this codec.
:)

[Mug Funky]

for a bit of fun, try this with mplayer:

mplayer -idx blurt.avi

and attempt seeking with the arrow keys after a couple of seconds (it crashes if you hit it too fast).

nice effect :) dancing, pulsating gelatinous wavelets.

one thing - i've noticed a tendency for this mess to drift to the upper left. is that a motion estimation thing?

[MfA]

Just curious (and lazy) has snow been profiled? How is decoding time divided between inverse wavelet transform and bitstream decoding?

[akupenguin]

A decode at qscale=4:
7% bistream reader (including dequant)
19% wavelet
74% motion compensation

The same movie, at qscale=2:
7% bistream reader (including dequant)
17% wavelet
76% motion compensation

[Nic]

I'm using Visual Studio 6's profiling and CodeAnalyst to help me profile snow.c, if you'd like a csv dump of the functions / lines cpu usage let me know.

Basically the functions inside snow.c that take the most processing are (from memory):
lift/lift5
mc_block (or it's h264_put_pixel equivalent)
add_yblock
(the vertical compose wavelet functions too)

For decoding...those are the main ones.

-Nic

[trbarry]

Nic -

What percentage of time does mc_block take?

- Tom

[SteMa]

What do you mean by "horrendous playback requirement"
I was doing a nero recode process (to h264), and meanwhile i was able to play both clips fine on my athlon xp 2000+ (latest mplayer: MPlayer-mingw32-dev-CVS-041204.zip)

[Nic]

@Tom:
Some statistics from profiling for you :) :
(This is for decoding)


Profile: Function timing, sorted by time
Date: Mon Dec 06 20:52:52 2004


Program Statistics
------------------
Command line at 2004 Dec 06 20:51: "C:\Development\Projects\SNOW\Release\SNOW"
Total time: 5259.269 millisecond
Time outside of functions: 23.526 millisecond
Call depth: 9
Total functions: 1345
Total hits: 43844295
Function coverage: 8.9%
Overhead Calculated 1229
Overhead Average 1229

Module Statistics for snow.exe
------------------------------
Time in module: 5235.744 millisecond
Percent of time in module: 100.0%
Functions in module: 1345
Hits in module: 43844295
Module function coverage: 8.9%

Func Func+Child Hit
Time % Time % Count Function
---------------------------------------------------------
1001.903 19.1 2742.057 52.4 2048004 _add_yblock (snow3.obj)
761.687 14.5 761.687 14.5 1551723 _mc_block (snow3.obj)
620.398 11.8 620.398 11.8 780600 _lift (snow3.obj)
326.461 6.2 486.348 9.3 7159351 _get_rac (rangecoder.obj)
291.836 5.6 1508.446 28.8 4526556 _pred_block (snow3.obj)
231.708 4.4 231.708 4.4 9184554 _same_block (snow3.obj)
200.973 3.8 200.973 3.8 260200 _lift5 (snow3.obj)
189.875 3.6 666.051 12.7 9600 _decode_subband (snow3.obj)
159.888 3.1 159.888 3.1 7159351 _refill (rangecoder.obj)
136.218 2.6 136.218 2.6 130400 _vertical_compose97iL1 (snow3.obj)
126.815 2.4 126.815 2.4 129800 _vertical_compose97iH1 (snow3.obj)
81.404 1.6 81.404 1.6 129800 _vertical_compose97iH0 (snow3.obj)
79.923 1.5 79.923 1.5 130400 _vertical_compose97iL0 (snow3.obj)
77.051 1.5 77.051 1.5 3962304 _av_log2 (snow3.obj)
65.789 1.3 65.789 1.3 246160 _ff_emulated_edge_mc (extras.obj)
61.215 1.2 61.217 1.2 395408 __intel_fast_memset (vecmem.obj)
60.451 1.2 324.188 6.2 279435 _decode_q_branch (snow3.obj)
49.982 1.0 2792.038 53.3 600 _predict_plane (snow3.obj)
48.142 0.9 193.481 3.7 631993 _get_symbol2 (snow3.obj)
44.996 0.9 175.282 3.3 574188 _get_symbol (snow3.obj)
28.785 0.5 28.785 0.5 49004 _put_h264_qpel8_mc23_mmx2 (dsputil_mmx.obj)
28.680 0.5 28.680 0.5 550527 _put_h264_qpel4_mc00_mmx2 (dsputil_mmx.obj)
27.580 0.5 27.580 0.5 46571 _put_h264_qpel8_mc21_mmx2 (dsputil_mmx.obj)
26.202 0.5 26.202 0.5 47627 _put_h264_qpel8_mc12_mmx2 (dsputil_mmx.obj)
25.142 0.5 25.142 0.5 62559 _put_h264_qpel8_mc22_mmx2 (dsputil_mmx.obj)
24.313 0.5 24.313 0.5 44534 _put_h264_qpel8_mc32_mmx2 (dsputil_mmx.obj)
24.143 0.5 845.514 16.1 260200 _horizontal_compose97i (snow3.obj)
23.213 0.4 23.213 0.4 373798 _put_h264_qpel8_mc00_mmx2 (dsputil_mmx.obj)
21.921 0.4 33.737 0.6 1 ___sti__$E (avsreader.obj)
20.266 0.4 20.266 0.4 49035 _put_h264_qpel8_mc10_mmx2 (dsputil_mmx.obj)
19.951 0.4 19.951 0.4 65588 _put_h264_qpel8_mc31_mmx2 (dsputil_mmx.obj)
19.628 0.4 19.628 0.4 67245 _put_h264_qpel8_mc33_mmx2 (dsputil_mmx.obj)
19.413 0.4 1295.746 24.7 3000 _spatial_compose97i (snow3.obj)
19.364 0.4 19.364 0.4 67889 _put_h264_qpel8_mc13_mmx2 (dsputil_mmx.obj)
18.937 0.4 18.937 0.4 63764 _put_h264_qpel8_mc11_mmx2 (dsputil_mmx.obj)
18.263 0.3 18.263 0.3 608150 _mid_pred (snow3.obj)
17.101 0.3 17.101 0.3 279435 _set_blocks (snow3.obj)
14.358 0.3 14.358 0.3 1 _dsputil_static_init (dsputil.obj)
14.099 0.3 14.099 0.3 117 _av_malloc (mem.obj)
11.884 0.2 11.884 0.2 1 _avpicture_fill (imgconvert.obj)
11.816 0.2 11.816 0.2 1 CAVSReader::CAVSReader(void) (avsreader.obj)
10.770 0.2 10.770 0.2 54939 _put_h264_qpel8_mc30_mmx2 (dsputil_mmx.obj)
10.294 0.2 10.294 0.2 50518 _put_h264_qpel4_mc22_mmx2 (dsputil_mmx.obj)
9.726 0.2 9.726 0.2 63952 _put_h264_qpel8_mc03_mmx2 (dsputil_mmx.obj)


Sorry for the long post,

@SteMa: Good for you :) But for other low spec machines, it takes a little too much processing. (Like a Duron 800 can play DivX just fine, it wouldn't play any Snow)

-Nic

[trbarry]

Nic -

Thanks. I'm still tinkering with mc_block. Should hopefully have something soon.

- Tom

[Nic]

Thanks loads Tom, something's cropped at work, that will keep me busy most of the week, but after that I can get back to working on it. Hope your work on it goes well, let me know if you need any stats or anything...

-Nic

[Paazabel]

I know this isn't exactly the forum for this question, since things here revolve around storage of existing content.

However, the low-bandwidth encodes from this codec are staggeringly impressive. What are the prospects of it being optimized enough at some point to actually stream a live input? Slim to none? Needs 50 tons of hardware? Just a hobby and hadn't thought about it?

I'm simply awed by the quality of the 278k clip. It is really jaw-dropping.

[Paazabel]

Oh, I would also like to beg for smart interlacing support. There are lots of ways to do it, from unfolding the fields to doubling the rate. Having this done at the codec layer would be massively helpful, but I'm sure it's low on your list of things to do (if it's there at all).

Things like classic superbowl games or other sports still look better in a 50 or 60 field format.

[PatchWorKs]

Any friendly interface ideas ?

Check out QuEnc or NuEnc

http://www.pcpages.com/dragongodz2/
http://www.petersplace.000k2.com/projects/

[Teegedeck]

Actually, QuEnc is Nic's baby, so you don't need to worry about the interface.

[dragongodz]

QuEnc is Nic's baby
so does that make me an uncle ? ;)

sorry couldnt resist. :D

[Teegedeck]

Ohhh; I didn't want you to feel left out! :)

[MSlv]

So, you're saying Snow will soon be no. 1 when it's ready? Who's developing it? Is it open source? Will there be a tool to encode DVDs to Snow? How much % is it done?

[Tommy Carrot]

Yep, it's opensource, it's part of the libavcodec, so there are many programs with snow support (ffmpeg, ffdshow, mplayer/mencoder, videolan, etc.).

It's hard to say what will become from it, it's an experimental codec, very promising but not mature enough for practical use, and there is no guarentee that it ever will be (i guess it depends on Michael Niedermayer, the main developer). Dirac, which is a similar codec, will probably be a safer choice.

[Teegedeck]

Aren't Dirac's sources a little developer-unfriendly (or so I remember to have heard)?

[Nic]

Dirac's sources are fine really. Heavy C++ code, but fine...But i've tried Dirac on several occasions, and have never been impressed :( Which is a shame, because i'd like the BBC to succeed :)

Snow wouldn't take much IMHO, to become mature enough for practical use. I can decode FD1 it in over 2x realtime and encode at 10+ fps. And the quality is superb. I'm not sure, unless things have changed dramatically, the same can be said for Dirac.

-Nic

[Tommy Carrot]

No doubt, right now Snow is a lot better, Dirac is really not impressive, but i still feel that Dirac might have a more assured future. It has a commercial support, a roadplan, etc, while snow is developed purely by enthusiasm, so if you or the other developers become bored with it, the development would probably stop, while the involvement of BBC prevents this to happen with dirac. But this is just the worst case scenario, obviously i hope Snow will soon surpass h.264 and take over the video-encoding scene. :D

[Paazabel]

My take on the playout of Snow is that it will spur the commercial companies to make a better product. The likes of Microsoft or DivX may or may not surpass Snow's capability, but it will force them to make something that is at least competitive. Right now, they both look like crap at low bandwidth where Snow may still look very usable.

After playing with it for a week, I really like this codec. There's a lot of promise, here, even if in the long run it is not realized by the Snow developers themselves.

I also think the killer app for Snow is live programming. You can get a very usable video from Snow at bandwidth much below some of the others. Snow would have to be multi-processor-friendly to accomplish this in real time, but it's certainly not out of the question to process in real time.

As said by others, my experience has been that at higher bandwidth, you are better off with other techniques -- so for "DVD backup," the application may be more limited. I am starting to bias toward quality in that arena, particularly as bigger hard drives get ever cheaper. However, having a really good-looking 500kbps live, streaming video -- that is something none of the other codecs can really deliver (despite their claims).

[chilledoutuk]

I must say that having recently updated ffdshow the quality of snow has improved a lot.
I am begining to see how good this codec could become possibly to wavelet codecs what xvid is to DCT codecs.

[PatchWorKs]

Interesting discussion... do you think that Snow could be an incarnation of Tarkin ? :rolleyes:

I mean... what about Xiph ? Could them help ? Or... could you help them ? :cool:

[iwod]

Originally posted by Nic
Dirac's sources are fine really. Heavy C++ code, but fine...But i've tried Dirac on several occasions, and have never been impressed :( Which is a shame, because i'd like the BBC to succeed :)

Snow wouldn't take much IMHO, to become mature enough for practical use. I can decode FD1 it in over 2x realtime and encode at 10+ fps. And the quality is superb. I'm not sure, unless things have changed dramatically, the same can be said for Dirac.

-Nic

Well... Even though i like BBC but i hate the name Dirac :D :D so much and love the name Snow too much :cool: so i really hope Snow can suceed... I would just like to know who is currently developing Snow...

[Nic]

It's all Michael Niedermayer that created and develops Snow. It's part of the ffmpeg project (which he kind of runs/oversees).

I want to develop it further, and hopefully will with christmas coming up (and therefore hopefully, some spare time)

-Nic

[Tommy Carrot]

Originally posted by PatchWorKs
Interesting discussion... do you think that Snow could be an incarnation of Tarkin ? :rolleyes:
I think the they have different goals. Tarkin would've been a royalty- and patent-free codec, but because of this restriction, xiph was unable to make it remotely competitive with the modern codecs (in fact, they couldn't make a working version of it, and it has been proved that making a competitive codec without motion compensation is impossible). I the case of Snow, i doubt Michael cares about patents, i think he just wanted to make a state of the art wavelet codec.

[Nic]

I know Michael is concerned with patents, but he may well ignores the ones he feels are "stupid" ;) . I know he wrote a different arithmetic coder for ffv1 and snow rather than use CABAC, because of patent reasons...

What patents exist on OBMC, Daubechies wavelets, lifting scheme, etc are unknown to me...anyone else know?

-Nic

[Teegedeck]

The codec really has improved a lot - the 85% quality-setting becomes watchable. I encoded a DVB-T movie last night and it came out 200 kbps... quite staggering. I think I'm gonna use Snow for intermediate storage of that bad DVB-T stuff when I don't get around viewing it at once. Good work, Nic and Tom! Thank you.

[chilledoutuk]

Dont forget Michael Niedermayer and yes what to me has improved the most is the stability of the image.

p.s what settings everyone using I am not using qel at the moment as it just use to much cpu to playback.
On my music videos it seems to be evry good indeed and I look forwad to further improvements.

[trbarry]

Good work, Nic and Tom!

Thanks, but I really haven't done anything on it yet except look at optimizing a couple things in asm that have probably not been released yet.

The credit goes to Nic, and of course Michael.

- Tom

[Joe Fenton]

Originally posted by Nic
I know Michael is concerned with patents, but he may well ignores the ones he feels are "stupid" ;) . I know he wrote a different arithmetic coder for ffv1 and snow rather than use CABAC, because of patent reasons...

What patents exist on OBMC, Daubechies wavelets, lifting scheme, etc are unknown to me...anyone else know?

-Nic

Pretty much anything associated with wavelets is patented. Last time I checked, there were several hundred patents covering wavelets for video compression. Even just using the term wavelet is patented. ;) :D

A quick search on the USPTO database on the term wavelet gives a hit on almost 3600 patents. So you can get an idea of just how easy it is going to be to make a commercial wavelet codec in the next 20 years. :p

People could see the promise in wavelets, so every conceivable idea in which wavelets could be used has been patented three or four times over. The sheer quantity of submarine patents on wavelet usage is mind-boggling. Yet another reason I hope that software patents will one day get thrown out.

[trbarry]

People could see the promise in wavelets, so every conceivable idea in which wavelets could be used has been patented three or four times over. The sheer quantity of submarine patents on wavelet usage is mind-boggling. Yet another reason I hope that software patents will one day get thrown out.

That is extremely depressing and I will try not to think about it.

- Tom

[iwod]

Originally posted by trbarry
That is extremely depressing and I will try not to think about it.

- Tom

What happen if we do use it. I mean patent is properly the most stupid thing about Software development. Simple thing now get patent for no reason.

OH... how i now learned to love open source.