Huffyuv for the HD Era: An SMP/Multithreaded Port?

[easy2Bcheesy]

I've been working with high definition video for almost a year now and I'll be honest here - there's not much that can match the performance or compression of the brilliant CineForm HD DirectShow encoder.

Only one codec gets anywhere near the performance level in terms of encoding on the fly and that is... good old Huffyuv. Even a basic 1mb cache 3.0GHz Pentium 530 CPU can encode 720p at 60fps. Indeed, I reckon that with a little bit of tooling (ie multithreading support), Huffyuv could most likely handle 1080p/30.

Filesizes are obviously much larger than CineForm, but why the love for Huffyuv?

1. It reduces HD bandwidth so that even a basic SATA drive can cope with it with zero lost frames
2. It can losslessly handle 4:4:4 or any other HD pixel format I throw at it and offers a quality edge over CineForm - or any HD codec - useful in multigenerational editing
3. Picture quality in HD is simply breathtaking

So the questions I would have are...

1. Is Huffyuv written in a way that would benefit from SMP optimisations?
2. Am I the only person in the world who would find this useful :D
3. Any one willing to give it a shot?

[squid_80]

What about lagarith? It's not much slower for encoding and is meant to be multithreaded already.

[Revgen]

lagarith has a more complicated algorithim and isn't as fast as Huffyuv. It's optimized to compress better at SD resolutions at a normal 30fps speed. Huffyuv isn't optimized for SMP because nobody has needed it until now.

[CruNcher]

Marcfds VBLE test version lg3 is much better in terms of compression factor @ 720p only LocoCodec comes near this yet but don't forget Cineform is visually lossless you can also get LocoCodec in this mode any maybe sooner or later a new VBLE ;) i think it's time for a HD comparsion of the best lossless Codecs vs Cineform HD :P

[easy2Bcheesy]

If people can provide me with download links I will be happy to attempt realtime 720p capture with those compressors. Right now, CineForm is the undisputed king of the lossy compressors (look hard enough and you can see the compression but you REALLY need to look hard enough), but I would still like a mathematically lossless solution.

A multithreaded huffyuv is attractive because not only could it easily handle 720p/60, but enough CPU overhead would be left over for other tasks too.

However, as I said, please let me know what codecs to try and where to get them and I'll be happy to give them a go.

[Revgen]

Loco and VBLE are at: http://forum.doom9.org/showthread.php?s=&threadid=53305

Lagarith is at http://lags.leetcode.net/codec.html

[easy2Bcheesy]

Well the results are thus. I've turned off the Preview Window to provide as much horsepower as possible to the compressors and fed it an Xbox 360 signal running at 720p/59.94. My PC is an Athlon X2 4400 based PC with 2GB of RAM running 32-bit Windows XP.

So any way, results...

CineForm DirectShow (multi-threaded): 59.94fps
Huffyuv (single-threaded): 59.94fps
VBLE (multi-threaded): 52.15fps
CineForm VFW (single-threaded): 49.23fps
Lagarith (multi-threaded): 20.98fps
Loco (single-threaded): 15.58fps

There may be some variance in the exact results as I could not feed the compressors with the exact same video to compress, but I ran the test several times and the results didn't vary THAT much.


Conclusions

1. CineForm HD
It was unfair to include CineForm in this comparison I know, but as a benchmark of the state-of-the-art encoder up against the competition it was interesting to put it up to the same scrutiny as the other codecs. I did run it in max quality mode which has a big impact on its performance compared to lower quality levels. It's a CQ codec and compression varied between 5:1 and 12:1.

2. Huffyuv
God knows how old it is, but my tests confirm that this is still the best codec out there for lossless 4:4:4 capture. Compression is usually around 2:1 to 3:1 and that's fine with me.

3. VBLE
The surprise package in that I reckon on a decent Conroe machine, it'll be able to capture at 720p/59.94 with ease being already multithreaded. However, as the codec is inherently using YV12, the result is definitely not up to the same quality as Huffyuv and as such has little value when you're looking to capture 24-bit RGB losslessly.

And the rest...
I'm sure they have their uses, but it isn't for realtime HD capture. I feel as though I am right back where I started to be honest in that I would kill for an n-way threaded version of Huffyuv.

[dragongodz]

have to tried the huffyuv thats from libavcodec as used in FFDshow ? thats meant to be slightly faster and compress better than the original huffyuv. its not compatible with the old codec of course which is why you can use the fourcc ffvh.

interestingly it doesnt have an option for amount of threads where many others included do. for example flv1 you could set 3 threads and do a constant quant encode and see how that performs etc. of course there is controls such as ME size etc which effect speed and quality.

[easy2Bcheesy]

Huffyuv as it stands in its VFW codec form is a lovely compressor to use in an HD workflow, and while a little bit of extra speed is nice, I need much much more - the kind of speed increase that only multi-threading can provide. I really want a truly lossless codec that will get more powerful as dual and quad core CPUs come out.

CineForm HD will do that as it makes use of all processing paths available (even HT ones on an Extreme Edition 955 Pentium D for example). If I bought a matched pair of 3.46GHz Dempsey CPUs, CineForm would make use of all eight CPU threads that would be available.

But there is no lossless 24-bit RGB equivalent to CineForm HD - another limitation of CineForm is that it's inherently based on 4:2:2 compression and is using CQ lossy compression.

[smok3]

dragongodz, i found no way to force the ffdshow to use anything else than yv12 for compression?, so thats why it compresses better (its not lossless).

[dragongodz]

Huffyuv as it stands in its VFW codec
the compressors in FFDshow are of course also vfw.

while a little bit of extra speed is nice, I need much much more
it would still be interesting to know how much faster it actually is. is it a little or is it substantial ? you would need a clip where normal huffyuv couldnt keep anywhere near framerate to test of course.

i found no way to force the ffdshow to use anything else than yv12 for compression?
the huffyuv in ffdhsow will do yv12 or yuy2, selectable in the generic section.
also in the inout section colourspace input can be forced to many more.

so thats why it compresses better (its not lossless).
hmm my impression was the tests done ages ago on the ffmpeg mailing list was done with the same colourspace and the libavcodec version compressed better and faster since it had been worked on past the final verion of huffyuv.

since we are talking about lossy though yes there would be a colourspace change needed to use the ffdshow version but you woiuld have to test to see if you will notice it or not. also cineform ,which easy2Bcheesy is also testing against, is not lossless. and finally of course for 4:4:4 the colourspace change is needed but for other sources such as dvd/mpeg it would not.

anyway i was simply throwing in some other possible option that may be worth trying.

[foxyshadis]

I was going to suggest FFV1, since it can capture in any colorspace, but testing proves it's slightly slower than lagarith. Shockingly high compression, though, easily the most size-efficient of any lossless I've tried (including AVC). (Unless you use pseudo-lossless xvid, EHR@q2. Which incidentally is about the same speed and 1/3 the size.) Guess it's no good for your purposes, even multithreaded.

Huffyuv on the other hand is shockingly fast indeed. And it's such a brain dead codec that it can't be hard to multithread, especially if there was no requirement to keep the bitstream back-compatible. I can try it, once I finish up a current project, but threading isn't a strong point of mine. ^^;

smok3, ffdshow's (lavc) huffyuv accepts yv12 and yv16 (it converts yuy2 into yv16 first).

cheesy, you have to capture rgb32, and not yv24? Or is that just because huffyuv doesn't support yv24? It'd be easy to adapt lavc's into doing that. Everything output to television is YIQ or YCbCr anyway.

dragongodz, I think the improved compression comes primarily from yv12 support, since the algorithm is essentially the same with minor tweaks.

[easy2Bcheesy]

Yup, no requirement to keep the bitstream back compatible, but just so long as it's fully VFW compliant and the streams will work in Premiere and After Effects etc, that's fine.

I am capturing 24-bit RGB and YUY2 typically.

[Wilbert]

Everything output to television is YIQ or YCbCr anyway.
YIQ is not used anymore by NTSC. NTSC uses the same as PAL, which is YUV (YCbCr is a digital format). Anyway, back on topic :)

[dragongodz]

I think the improved compression comes primarily from yv12 support, since the algorithm is essentially the same with minor tweaks.
from memory its also not compatible with normal huffyuv so i think its a little more than that.

easy2Bcheesy - you can give this a try if you like
http://d.turboupload.com/d/815066/huffyuv-test.zip.html

[easy2Bcheesy]

dragongodz: there's a definite performance increase with your test version of huffyuv. And VirtualDub is reading CPU usage at 172%, so whatever's going on is definitely multithreaded. However, the performance increase doesn't seem to match the increase in CPU usage which is interesting.

Tell me more! :)

[dragongodz]

bit of a quick cheat really. its simply
/Qipo /Qopenmp /QaxKWN /Qparallel

plus libguide.lib linked so it doesnt look for libguide40.dll.

this version i just changed to /QaxKWN to /QxK which should mean use SSE with no checking.
http://d.turboupload.com/d/819233/huffyuv-testSSE.zip.html

[squid_80]

But the real reason huffyuv's so fast is because it's written almost entirely in assembly... The prediction functions could probably be multithreaded fairly easily but I don't know about the encoding functions.

[dragongodz]

i agree to get the most out of it someone would probably have to look at hand threading the asm. i just wanted to see if a quick compiler change plus options would be noticably beneficial.

on that note heres lagarith 1.38(1.39 source is not on the website) with those same first changes. if you could give this a quick try aswell easy2Bcheesy, it would be interesting to see if it gains much over that previous test you did with the normal release.

http://d.turboupload.com/d/819997/Lagarith_138_test.zip.html

[easy2Bcheesy]

Hi,

Great to do some testing!

Bizarrely the SSE huffyuv appears to be a bit slower than the first test version.

VirtualDub, preview window on, 720p/59.94: CPU usage ranges between 75% to 81% on the old version. On the new version it's much the same but with the occasional spike up to 88% CPU usage.

New Lagarith Test: 25.55fps so only a marginal improvement, but I guess it's a massive performance boost for a very simple code tweak.

[Hellworm]

btw ffvhuff compresses better and is not standart huffyuv, because it can use adaptive huffman tables.

[easy2Bcheesy]

But it's not totally lossless 24-bit RGB, is it?

[dragongodz]

New Lagarith Test: 25.55fps so only a marginal improvement, but I guess it's a massive performance boost for a very simple code tweak.
well its an over 20%(of original speed) boost. thats not too bad for a simple compiler change indeed. :)

But it's not totally lossless 24-bit RGB, is it?
looking at the FFmpeg source code it appears for RGB24 and RGB32 it uses normal huffyuv and for YV12 and YUY2 you can use ffvhuff. so i guess really that could be set in FFDshow if Milan wanted to.
but no as i mentione earlier if your source is RGB24 then you need to do a colourspace conversion to either YV12 or YUY2 for FFDshow, which it then compresses lossless. how much do you lose from this conversion really though ? thats something you would have to try for yourself and decide if its worth it.

[easy2Bcheesy]

Lagarith is YV12, I think, and yes, picture quality is good but it is not lossless RGB24 and you can see the difference. If it's not lossless, CineForm rules :) As a straight alternative to capturing uncompressed, Huffyuv is simply superb.

[dragongodz]

but it is not lossless RGB24 and you can see the difference
hmm thats strange, i though Lagarith was for all colourspaces. infact it was written around huffyuv source(which you can easily see in Lagariths source archive) and uses a few of its routines. will have to have a look when i get the chance. pretty busy at the moment though.

EDIT:
Lagarith is YV12, I think,
in its settings you can set the colourspace for it to work in. RGB(RGB24), RGBA(RGB32), YUY2,YV12.

[squid_80]

Lagarith is RGB, YUY2 or YV12. I've used it for capturing YUY2/720x576/25fps with no dropped frames at 60% cpu usage... But that was the 64-bit build, which is out of scope here.

[easy2Bcheesy]

Indeed, Lagarith is way too slow unfortunately :( Faster with YUY2 though, but for any YUY2 capture, CineForm produces astonishing quality and very small filesizes, plus the AVIs drop straight into their Premiere Pro accelerated editing environment.

[squid_80]

Do you have the capabilities to test huffyuv with a 1080p/30fps/RGB24 capture, just to see what framerate it can get?

I hated dealing with huffyuv because it was so complicated, but I'm starting to get interested... Unfortunately I don't have a multi-core machine so I'd be flying blind. But a general huffyuv overhaul would be possible, proper YV12 support (not via ffdshow, which is slower than the original huffyuv) and SSE2 instead of MMX would be a start.

Edit: Also the "predict median" mode for RGB for better compression.

[easy2Bcheesy]

My system is currently optimised for 720p/60 - RGB24 1080P would struggle to top 11 of 12fps in its current form. A proper 1080p digital input is something I'm only really interested in investing money in when I know there's a codec that can handle it. And when I make that hardware leap, I'd really want 1080p/60.

I could certainly simulate 1080p/30 though with a resize filter on the fly though and we could measure the CPU load that induces and weight that against the final results. Alternatively I could hook up my PC and see how it handles an intermidiary resolution such as 1440x900.

[squid_80]

The good news: It's possible to multi-thread both the prediction and encoding functions. At least, I'm using multi-threads on my single cpu machine and it's working.

The bad news: The bitstream's not backwards compatible, and I've got no idea how to make it so.

I'll put some stuff up for testing tomorrow, to find out how it really works on a multi-cpu machine.

[easy2Bcheesy]

Great! Backwards compatibility isn't really a must - the raw speed increase most definitely is :)

I look forwards to testing for you :)

[Revgen]

BTW, Dragongodz, thanks for the Lagarith build. I can now capture in realtime with almost no dropped frames at 720x480 using multithreading.

[foxyshadis]

squid_80: give it the 4cc HFYM or something like that, so it doesn't interfere with normal huffyuv then. The extension to YV12 and MT sounds really cool. =o

[squid_80]

Ok before I get too carried away let's see if multithreading actually helps at all.
http://okejl.dk/dunstan/huffyuv2.zip
That's a slightly modified build of huffyuv with multithreading (2 threads hardcoded) on the YUY2 Median compression method only - the other methods/colorspaces haven't been touched. The 4cc is HFY2, the codec shows up in virtualdub as NEW Huffyuv v2.1.1 and can be uninstalled via control panel->add remove programs without hurting any existing huffyuv installs. This is all because it's a simple test, I wouldn't advise anyone to hang on to it for ongoing use - just try and capture something using yuy2/median, check the speed against capturing in uyvy mode (which is virtually the same but not multithreaded), make sure the file can be re-opened*, and report back.

Sometimes the bottom half can be slightly messed up. This is what happens when the 2nd thread finishes before the first has even started - a very rare case which has only happened to me once. A fix is planned for later.

My tests so far have been sadly non-optimistic. But that's with a hyperthreaded machine, so let's call them inconclusive.

[easy2Bcheesy]

1280x720/59.94 capture from YUY2 source (Xbox 360 dashboard so pretty static with a rolling tint in the background):

Original Huffyuv: CPU usage from 66% to 75%
New Huffyuv: CPU usage is all over the place, starting at 78% and going up to 97%, averaging at 88%

It's difficult to measure in any other way - any transformation filter I invoke requires operating in the RGB domain so by the time it reaches the compressor it's no longer YUY2.

[dragongodz]

easy2Bcheesy - if its not too much trouble can you include one of the builds i did for comparison ?

squid_80 - from what i have seen 3 threads seems to be generally the number for real good gains. thats based off other programs/codes such as Xvid.

[easy2Bcheesy]

hi dragongodz - in terms of CPU usage, the first version of huffyuv you uploaded appears to have extremely similar levels of load as the original huffyuv.

[squid_80]

Without a proper machine to test on I'm really shooting in the dark. It could simply be that the overhead time to create/destroy the threads is longer than the total processing time. If that's the case I might have to do a bit of reworking.

As an aside, huffyuv always assumes any stream with height > 288 is interlaced, and unfolds the fields side by side before compressing. By introducing an interlaced option in the config and turning it off for progressive sources I'm seeing a compression boost of about 10%.

[Sharktooth]

I think inverse hyperthreading will be really usefull... ;)

[easy2Bcheesy]

Heh, is that actually real or just a myth? The facility to have the single-threaded CineForm VFW encoder inversely hyperthreaded would be worth its weight in gold, however, I'm veering more towards 'myth' at the moment. Some things are just too good to be true ;)

[Sharktooth]

it's in the specs of the K8L.

[squid_80]

O RLY? (http://www.reghardware.co.uk/2006/07/17/amd_reverse_ht_not_real/)

[Sharktooth]

O RLY? (http://www.reghardware.co.uk/2006/07/17/amd_reverse_ht_not_real/)
This is the original:
http://www.xbitlabs.com/news/cpu/display/20060714052807.html?www.reghardware.co.uk
However it's all about unreachable/unnamed sources...
And remember AM2 CPUs comes with virtualization (and theoretically it is possible to emulate 1 CPUs with multiple cores - read: inverse hyperthreading).
So, i do not believe a single word from those 2 sites...

[foxyshadis]

The problem is that AMD has to do in hardware what compilers do in software - and compilers are already notoriously bad at automatic multithreading (ie, /Qparallel), even given a holistic view of the entire application and all the time it needs instead of just the current instructions and billionths of a second. </off-topic>

On topic, there isn't any great speedup for me, but then, I can capture 1440x960x60p or 1920x1080x30i in realtime anyway with dragongodz's compile. I get ~50fps on the latter with squid_80's and ~47fps with dragongodz's. (CPU usage wasn't noticeably higher, a little over 50% each run, but that's a red herring anyway; only fps matters.)

[easy2Bcheesy]

What CPU are you using there foxy? If we are moving into the realms of 1080p/60 (as I certainly want to be doing next year) I seriously think we're going to need to go quadcore, or perhaps even dual Woodcrest/Dempsey.

That said, a dual Opteron 275 system can apparently capture 1080p/60 into the n-way threaded CineForm encoder...

[foxyshadis]

Merom. :p The huffyuv compression of 1080p/30 is only ~70% of one core. Any conroe, woodcrest, or recent x2 should have no problem with that part. (Now if you wanted to run some kind of filtering first, along with the bus transfer effects...) Cineform would undoubtedly need all of both cores, but that's why you stack 'em up.

To clarify, as I said in mail to easy, I'm not actually capturing. I tried using both generated noisy video and video captured over GbE through avisynth's tcpdeliver to soft-of simulate capturing.

[squid_80]

I was wondering how you were capturing ~50fps from a 30fps source...

[dragongodz]

I was wondering how you were capturing ~50fps from a 30fps source...
i actually like to see that figure. helps to give a more real view than just cpu percentage usage alone. now if foxyshadis had given the original huffyuv's fps aswell that would have been even better to see how much gain these things(including my simple compiler changes) were giving. ;)

Without a proper machine to test on I'm really shooting in the dark.
my athlon xp 2400+ doesnt exactly allow me to test/see the effectivness of threaded changes either. ;)

EDIT:
squid_80 - out of curiosity what compiler are you using for your compile ?

[easy2Bcheesy]

Merom. :p

Ahhhh Merom... my T7400 can encode difficult 720p material at 84fps with the CineForm encoder :eek: In fact, while I'm at it, here are some official CineForm benchs on the new Intel architecture, all the way from the dual core Merom, to dual next gen Xeons. Shamelessly ripped from CineForm CTO David Newman's Blog (http://cineform.blogspot.com), which is essential reading if you're working day to day with HD

http://img221.imageshack.us/img221/4246/untitled1uf1.jpg

I'd expect encoding performance to match Decoder 3 scores (that's CineForm's n-way threaded reference quality decompressor). Decoder 2 is the slightly less optimal playback decoder for Media Player/Premiere Pro etc.

By way of reference, my Athlon X2 4400 desktop system achieves a score of 69fps using the same benchmark tool (also how I double-checked my Merom performance). Suffice to say, my next desktop system will most likely be based on the quad core 2.66GHz Kentsfield (engineering samples available now if you know where to look - a snip at $1700!!!)

If it gets midway performance between the Conroe and Woodcrest scores, that should be more than sufficient for 1080p/60 capture.

[squid_80]

squid_80 - out of curiosity what compiler are you using for your compile ?
At the moment using VS2003. I have ICL 9.0 and VS2005, just haven't bothered trying them until I get everything working properly (the YV12 stuff is being stubborn and refusing to work).
Most of the heavy code is in the .asm file (except, curiously, the rgb->yuy2 routine), I'm surprised it makes much difference which compiler is used unless it's doing some sort of whole program optimization.

Anyone know of an easy way to generate some default huffman tables for the rgb/median method? I'm thinking an avisynth plugin would probably be easiest, and I can adapt it to create "best fit" tables for any source. It sounds good, but I don't know how practical it would actually be.