my new codec (h.264)

[justin]

ok, since noone will make a h264 codec, I have decided to make a codec named wheatgerm. So far, it is at the stage after the name planning

[Sirber]

is that a joke?

[justin]

well noone else will make one :devil: just started a new project in Visual C++ 6, so far so good

ok I have a question, how do you make a codec? :eek:

[Tommy Carrot]

There were a couple of opensource project intended to create an h.264 implementation. But all of them has been stopped, imo because h.264 is too complex (much more than mpeg4) to do it by enthusiasm alone.

So i think we'll have to wait for a commercial implementation, or perhaps Skal can make something useful, he is our last hope. :)

[Gaia]

Originally posted by Tommy Carrot
There were a couple of opensource project intended to create an h.264 implementation. But all of them has been stopped, imo because h.264 is too complex (much more than mpeg4) to do it by enthusiasm alone.

So i think we'll have to wait for a commercial implementation, or perhaps Skal can make something useful, he is our last hope. :)

Also h.264 is too much for current cpu's. You need one hell of computer to encode full movies. Unless you don't mind encoding weeks one movie.

[justin]

but h264 is better, it's so worth it

[bond]

Originally posted by Gaia
Also h.264 is too much for current cpu's. You need one hell of computer to encode full movies. Unless you don't mind encoding weeks one movie.well simply because current (untweaked) h.264 implementations are really slow it doesnt mean that highly tuned implementations will also need "weeks"

but of course it will need more cpu power than current mpeg-4v2 (but not to forget the cpus are also getting faster and faster)

[Tommy Carrot]

Originally posted by Gaia
Also h.264 is too much for current cpu's. You need one hell of computer to encode full movies. Unless you don't mind encoding weeks one movie.

That's not exactly true. It would be slower indeed, but not that much. Afaik it needs about 2-3x computing power compared to mpeg4.

EDIT: Bond was faster. :D

[Sirber]

my 2000+ can't decode H264 in realtime :(

[Doom9]

My 2800+ has no problem playing Matrix at original DVD res (minus the black bars).. it's only a matter of a good playback filter ;)

@justing: I suggest you have a look at hdot264 and the h.264 reference code if you're serious about this. If you plan to make it open source (I suggest you do because it's a bit much as a one man job), you can use code from the xvid project for the whole vfw and ds stuff.. that should also help you get started.

[bond]

Originally posted by Doom9
My 2800+ has no problem playing Matrix at original DVD res (minus the black bars).. it's only a matter of a good playback filterexactly
envivio also offers such a h.264 decoder filter, the latest version is EnvivioTV-H264-2-1-181 afaik

[el divx]

Originally posted by bond
exactly
envivio also offers such a h.264 decoder filter, the latest version is EnvivioTV-H264-2-1-181 afaik

What I'm gonna say is unrelated, but where do I get those envivio filters? I even got registered the firstt time I tried to download them but now they won't let anyone download them unless he or she is a business associate of them.

[lazyn00b]

Originally posted by el divx
What I'm gonna say is unrelated, but where do I get those envivio filters? I even got registered the firstt time I tried to download them but now they won't let anyone download them unless he or she is a business associate of them.

Google is your friend.

[justin]

I wasn't sure if this should be a joke or not, but it is a good idea, and it's now started, hopefully this will be a very useful codec, I never learned when to quit (/me just keeps going)

wheatgerm (the name is only temporary if anyone would like to change it) is intened to leave where hdot264 left off, but be as open ended as open ended can be. It is a free, completely open source, open to everyone who would like to get involved and program an h264 codec where everything is up to everyone and it's success is going to be determined by the amount of people that join in and try. Hopefully this project will work and bring a constantly improving open source h264 codec to everyone

Some goals:
- try to document the code everything as best I can, so that others can read it and use it in other projects
- I'm hoping that this will be able to be incorporated into Xvid too as an h264 module if it is possible to do this
- And strong support towards Matroska as the container of choice, so that it will be more recognized among h264 as streaming and downloadable video

The big job here is to optimize h264 reference code and I'm not very good at optimizing code at all, I don't think, I can program in c/c++ though. And this should be programmed in c/c++ (not c#) with Visual Studio.NET if possible because it has fastest math operations for now (until the new java version comes out which is supposed to rival c++ in math operations per second) and optimize more with assembly. And right now I am looking for the lastest h264 reference code on the web

- I forget my sourceforge username so I'm going to try to contact sourceforge to try and get it back and then start wheatgerm

[bond]

Originally posted by justin
- I forget my sourceforge username so I'm going to try to contact sourceforge to try and get it back and then start wheatgerm [/B]justin,
if you are really interested in developing a h.264 codec and have the skills to do it, it would be surely the best to join the hdot264 project and continue the work there were it stopped

imho starting a new project is not a good idea, the best is to keep all developers interested in an open source codec together in one project, hdot264, (cause than it has the biggest chances to be successfull in the long run)

[justin]

ok, that's probably a better idea (I still want to call it wheatgerm though -;/ Maybe codename of a development build :p), also want to see if hdot264 crew is still active.. I only heard rumours that it wasn't being worked on

I don't know if I can help or not, I've never tried programming something like this before.. but I want to try (I've done lowlevel assembly and VHDL (for hardware implementations :)) and c/c++ though)

[sysKin]

Originally posted by bond
imho starting a new project is not a good idea, the best is to keep all developers interested in an open source codec together in one project, hdot264, (cause than it has the biggest chances to be successfull in the long run) Just wanted to add my two cents - I don't agree. Hdot's code is very ugly imho, and most of the development would consist of replacing non-gpl code with gpl code, while keeping the overal ugly (and slow) structure.

If I'd start h264, I would start with xvid (surprise, surprise lol). This is definitely the cleanest codec code out there.
Then, I would:
- remove some parts which are not neccessary. Qpel, b-frames, VHQ, more more more. At this point the project should still compile and work as mpeg-4.
- add sub-pixel filters, change header code etc. At this point everything should still work, including decoder, but will not be compiliant with anything.
- change VLC codes for macroblock info, motion info, etc. Add motion prediction etc. This can still be done on encoder's and decoder's side for instant debugging - one thing at a time.
- change block's size to 4x4, add new bitstream reader and writer for coefficients. Lots of debugging here, this is the biggest difference between the two codecs.
- add final h264's specific code, like intra prediction or 4x4-pixel motion mode. Add VHQ back (will definitely be neccessary for advanced motion modes) and start thinking of 8th-pel, b-frames, multi-predictions, picture slicing and more.

This is definitely plenty of work but I'd like to join such project. Just thinking of all the bugs that can be done is thrilling lol.

Radek

[Joe Fenton]

XviD codec clean? Have you looked at it lately? :D

Look at the huffyuv codec - now there is some clean code. I was messing with the hdot264 code at one point. I even had a compile available on my web page for a while for folks to test. However, my system was one of those where hdot264 never worked. It would cause VirtualDub to crash.

So, one thing I am doing right now is taking the latest reference code for h.264 and sticking it in the huffyuv codec. It's considerably easier to see what's going on than messing with all the extra "stuff" in the XviD codec. I also got a paper which describes how to replace certain parts of the reference code with MMX code to make it 3 to 4 times as fast on decoding.

I've got other things I'm doing right now, so don't expect this thing tomorrow. ;)

[BoNz1]

sysKin, what about CABAC? :D and why remove all that stuff when you could start from 0.9x branch :D if you wanted to create your own codec seperate from XviD that way might save you some work. I don't know, just a random and slightly regressive thought :).

[morsa]

Just ask Tom Barry about the project status.He is one of its developers.

[trbarry]

Just ask Tom Barry about the project status.He is one of its developers.

No, I'm not. I first signed up when the project was started but there wasn't much activity. Then when the MPEG-LA took control of the licensing I sort of lost interest. So I have made no contributions there and should probably ask them to remove me from the list of developers.

- Tom

[dillee1]

Actually I am looking at h264 as a possible master research thesis title as well. However I just can't find the spec selling at a reasonable price. (go www.iso.ch and see the price tag yourself).

Anyway, has anyone one here considers using VU/GPUs for h264 number crunching, other than using using the CPU?

Some info some general purpose programming iwth GPU:
http://www.gpgpu.org/

[drmpeg]

You can download a fairly recent draft here:

http://bs.hhi.de/~wiegand/JVT-G050.pdf

Ron

[MfA]

dillee1, funny you should say that ... I just saw some papers about that yesterday :

http://www.mee.tcd.ie/~sigmedia/Publications.html

Cant say Im impressed, a not very efficient search method runs 2 times faster on the GPU ... but all the tricks used to speedup searches on the CPU are neigh impossible on the GPU.

Maybe with the next generation of GPUs.

[BoNz1]

BTW, I found this, http://www.google.ca/url?sa=U&start=1&q=http://www.dspr.com/www/support/download/video_download.htm&e=7413 the decoder doesn't need a password so you could use that. There is an encoder there too but password protected.

[shark37]

Originally posted by drmpeg
You can download a fairly recent draft here:

http://bs.hhi.de/~wiegand/JVT-G050.pdf

It's worth to take a look at http://bs.hhi.de/~wiegand/
and http://bs.hhi.de/~wiegand/JVT.html particularly :p

Originally posted by BoNz1
BTW, I found this, http://www.google.ca/url?sa=U&start=1&q=http://www.dspr.com/www/support/download/video_download.htm&e=7413

Another page from the a/m site:
http://www.dspr.com/www/technology/technology.htm
...
The H.264 / MPEG4 AVC Standard
...
For those of you who want to learn more about video compression and the H.264 standard, the following site is an excellent place to start:
H.264/MPEG-4 AVC Introduction (http://www.vcodex.fsnet.co.uk/resources.html)

For you who are already familiar with video compression standards and want to know more about the H.264 standard, the following publication (19 pages) is an excellent source:
H.264/MPEG-4 AVC Overview (PDF) (http://www.dspr.com/www/technology/csvt_overview.pdf) (2.986KB)

A draft of the whole H.264 Standard (264 pages) from the 'ITU-T 7th meeting: Pattaya, Thailand, 7-14 March 2003', is also downloadable below:
The whole H.264 Standard (draft from 7-14 March 2003) (PDF) (http://www.dspr.com/www/technology/JVT-G050.pdf) (1,816KB)
...

[justin]

lots of responses :o

ok, you are very welcome to join syskin :) for my decision part anyway. Found out that hdot264 is not active at the moment and was welcomed to join the team or start wheatgerm.. Decisions Decisions :eek:

/me has learned a few things about h264, mainly why it takes so long and why the compression is better

h264 is pretty much a new codec, it should probably go under a different category like Mpeg5 than Mpeg4 or Mpeg2. Mpeg1,2, and 4 are all fourier transform based for their encoding and decoding. Also Jpeg and Mp3 are fourier based. H264 and Jpeg2000 are Wavelet Transform based.

I am going to thoroughly explain, I never learned wavelet transform analysis in DSP classes back in university though

ok first, you have to think of an image as a waveform of colours. You can see an audio file as a waveform, but images are also just a waveform of a group of coulours (Red Green and Blue) for each pixel, if you plot these out pixel by pixel, you get the waveform

Fourier transforms work by approximating a waveform with a lot of sine waves (you can get a good approximation of the original waveform with alot less data values than having to sample the wave x times a second (in uncompressed form)).

The wavelet transform does the same sort of thing, but instead of using sine waves, it uses stepwise functions instead to approximate the original waveform (stepwise functions stay at one value for a time and then jump up or down to whatever height you set them at).

There's other things that make H264 take longer than h263, but that's one of the main factors since wavelet transforms take alot longer to compute than fast fourier analysis. Wavelet Based transforms also come out to look better than their fast fourier counterparts do for a given amount of approximations to the original waveform (this is expecially noticible around the edges). For some examples, take a look here (http://www.dspworx.com/primer_jpeg_vs__jpeg2000.htm) for low bitrate, and here (http://www.imagepower.com/technology/jpeg2000/compare/) for extremely low bitrate, to see the differences between fourier based JPEG and the wavelet based JPEG 2000. If h264 stands to reason, then those blocky areas for all your DivX and Xvid movies before should come out to look much nicer with h264. Similarly, as soon as people begin using wavelet based transforms instead of fourier based ones on MP3/WMA/Ogg based codecs, you should see a significant jump in quality at lower bitrates for them as well

[MfA]

Stop right there justin ... Im going to assume you are just being overly enthousiastic, but pretty much everything you said was wrong. Go read up a bit on this stuff.

Try http://www.vcodex.fsnet.co.uk/h264.html which is comparatively clear, although not quite up to date, and google for anything you dont understant.

[Tommy Carrot]

H.264 has nothing to do with wavelets. It's using integer transform instead of DCT, which biggest benefit is that those nasty mosquito noise artifacts are finally eliminated, but it still block-based, just like mpeg4.

[justin]

oh :(, looking at that document you are right, I heard that it was based on wavelets from another site (http://translate.google.com/translate?hl=en&sl=it&u=http://news.hwupgrade.it/11509-10.html&prev=/search%3Fq%3D%2522wavelet%2Btransform%2522%2Bh264%26hl%3Den%26lr%3D%26ie%3DUTF-8%26oe%3DUTF-8)

[RadicalEd]

A wavelet codec is still in MPEG-21's working documents for a scalable coding solution (part 13).
http://www.chiariglione.org/mpeg/working_documents.htm

It looks promising. Wavelet seems to be the next mountain to climb after H.264.

[d'Oursse]

Originally posted by RadicalEd
It looks promising. Wavelet seems to be the next mountain to climb after H.264.
the rududu codec is using wavelets


@justin : a remark about the integer transform of h264 : the calculations used for the integer transform are exactly the same than the fft ones for a matrix of order 4. Indeed, this transform can be found by "rounding the fft" (multiply the matrix which appears in the fft by a real (don't remember which one) and round the coefficients. You obtain the matrix that appear in the integer transform. And, magically, the fast computations of fft works with this matrix). I tried to find similar matrix of superior order (8 and 16) and I have not suceeded in.
H264 gives bloky image (inherent to all transforms where you suppress high frequencies). That's the deblocking algorithm which smoothes the image.

[bobololo]

Originally posted by Tommy Carrot
H.264 has nothing to do with wavelets. It's using integer transform instead of DCT, which biggest benefit is that those nasty mosquito noise artifacts are finally eliminated, but it still block-based, just like mpeg4.

Could you please explain why integer transform would suppress mosquito artefacts ? From my understanding, the biggest advantage of the integer transform (which is DCT matrix scaled and rounded to integer value) is the suppression of the DCT mismatch that occurs between different iDCT implementation.

FYI: http://research.microsoft.com/~malvar/papers/MalvarCSVTJuly03.pdf

[Tommy Carrot]

Originally posted by bobololo
Could you please explain why integer transform would suppress mosquito artefacts ? From my understanding, the biggest advantage of the integer transform (which is DCT matrix scaled and rounded to integer value) is the suppression of the DCT mismatch that occurs between different iDCT implementation.

FYI: http://research.microsoft.com/~malvar/papers/MalvarCSVTJuly03.pdf

Well, as far as i know (but i may be completely wrong) the mosquito artifacts are mostly caused by the rounding error of DCT. Using different implementations can worse the problem, but anyway, it's inevitable with DCT, unless if you're using floating point implementation, but it would be horribly slow.

And don't forget H.264 doesn't have deringing filter, which would be necessary in the case of mosquito artifacts.

[Sulik]

The main difference is that H.264 basically uses a 4x4 DCT instead of an 8x8 DCT, so the ringing artifacts are 4 times smaller, hence less visible. They're still there, but most likely get somewhat lost in the blur induced by the deblocking filter.

[bobololo]

Originally posted by Tommy Carrot
Well, as far as i know (but i may be completely wrong) the mosquito artifacts are mostly caused by the rounding error of DCT. Using different implementations can worse the problem, but anyway, it's inevitable with DCT, unless if you're using floating point implementation, but it would be horribly slow.

And don't forget H.264 doesn't have deringing filter, which would be necessary in the case of mosquito artifacts.

I would say mosquitos are mainly caused by wrong decisions that neglect marginal spots because they don't seem to be important whereas in reality they are -visual perception wise-. An example would be leaves flying on a blue sky. The encoder may produce some mosquitos in the sky caused by some small fragments of leaves covered in a macroblock with 99% of blue.

[MfA]

AFAICS mosquito noise is simply ringing which gets noisy because of highly isotropic basis functions of the DCT.

As for why h.264 has less ringing, Ill agree with Sulik.

[d'Oursse]

Originally posted by Tommy Carrot
Well, as far as i know (but i may be completely wrong) the mosquito artifacts are mostly caused by the rounding error of DCT. Using different implementations can worse the problem, but anyway, it's inevitable with DCT, unless if you're using floating point implementation, but it would be horribly slow.

Even with floating point implementation, you will have a blocky image and mosquito noise (it is the Gibbs phenomenum). It occurs at the edges of the macroblock, and in the macroblock where there is a high gradient of values.

[RadicalEd]

Originally posted by d'Oursse
the rududu codec is using wavelets

Sure, and Rududu is beautiful, but it has it's limits as a one-man experiment. The development and maturation of a wavelet based standard, on the other hand, is what I'd forsee as succeeding H.264, as H.264 is set to succeed Mpeg 4 (A)SP.

[MfA]

Scalability comes at a cost, dont expect h.264 beating compression from that particular MPEG standard.

[Selur]

seems like PixelTools (http://www.pixeltools.com/experth264.html) got a tool that can encode h264.

I know it's really expensive ($995.00) but does anyone know if it's any good ?


Cu Selur

[Sirber]

seems a GUI for hdot264...