Modification of huffyuv codec for YV12?

[HarryM]

Do exists?

[-h]

I'll do this "soon". It's a fairly simple task, but I'll probably fork huffy into a new FourCC / data format just for the job.

-h

[HarryM]

Originally posted by -h
I'll do this "soon". It's a fairly simple task, but I'll probably fork huffy into a new FourCC / data format just for the job.

-h


@-h:

Thanks.
Huffyuv is very good, powerfull codec. I have bitrate about 11-12MB/s (full PAL 720x576) and this is too much. Native support for YV12 can theorethically lowering average bitrate to 3/4 (cca 8-9MB/s). This is acceptable for me.

[trbarry]

I'll do this "soon". It's a fairly simple task, but I'll probably fork huffy into a new FourCC / data format just for the job.


-h

"(-h)uffYV12" would be a nifty addition to our YV12 tool kit. If I'm using a bazillion filters I sometimes spin a first pass out to huffy and then do it the fast way after that.

And this would improve both speed and space. I really hope you do this.

- Tom

[slavickas]

Originally posted by -h
I'll do this "soon". It's a fairly simple task, but I'll probably fork huffy into a new FourCC / data format just for the job.

-h

ffmpeg already have yv12 hyffyuv version

[Suikun]

Originally posted by slavickas
ffmpeg already have yv12 hyffyuv version
Maybe milan could update ffvfw to support it?

[timeToy]

Hi,

As HuffYV12 will have a different FourCC code, it may be the good time to add what missing in the Original HuffYUV, like a simple checkbox for interlaced content and the setting actually saved in the file !

[Koepi]

As huffyuv is lossless, I don't see why you should need an interlaced flag? you won't gain or loose anything due to it's simple huffman tree nature...

Just my 2 cents.
Koepi

[ErMaC]

From what I understand of Huffyuv the compression algo isn't straight huffman, it's more advanced than that. I think the same reason that using interlaced vid in MPEG2 without using an interlaced flag is bad applies to huffyuv's comp algo, however with huffyuv you don't wind up with a worse looking picture, just a larger file. Conversely, setting interlaced when you don't have to will do the same.

[Marc FD]

huffman is useless as-is on images. huffyuv use prediction (gradient/left/median prediction i think) delta prediction would even more efficent, but it would need keyframes/deltaframes.

i'ld love (-h)uffYV12, for the same reasons that tom pointed out, and for captures too. (i think the gain would only be 10% max, but decoding will be faster, and it'ld avoid conversions)

thx -h ^_^

Cheers,
MarcFD

[-h]

The gain will be around 20%.

-h

[Marc FD]

is finished ??

[HarryM]

Hmmm.
We know.
HuffYUV use pure lossless algorithm.

But what using any slightly-lossy algorithm (not JPEG)???:D

[timeToy]

Please, we want our HuffYV12 losseless, that's all the point.
If you do want some near-lossless use DivX or XviD at 1q2 with everything turn off.

[-h]

Allowing pixels to be off by 1 or 2 (out of 255) can increase compression by 50%, if an RLE method is included. I experimented with many variations of the huffy algorithm while working on a lossless codec (which I never completed of course).

-h

[HarryM]

Originally posted by -h
Allowing pixels to be off by 1 or 2 (out of 255) can increase compression by 50%, if an RLE method is included. I experimented with many variations of the huffy algorithm while working on a lossless codec (which I never completed of course).

-h

Of course! ;)

Little 'controlled' deviations in interval e.g. (x-1) to (x+1) can very help to standard lossless algorithms, without visible degradation of captured video, I think.

[HarryM]

@TimeToy:

Principially exists these three section of graphics compression:

a) lossy (e.g. JPEG)
b) lossless (e.g. huff, zip - standard data compression)
c) near lossless (!)

and lossy 'not equal' near lossless, this are diferent algorithms often...

[timeToy]

I'm sorry but Lossless means lossless (no losses), I use lossless compression to store clips that will be included in an After Effects composition, depending on the color space of the original clip (3D rendered pictures=RGB; Captured from SDI source (DigiBeta)=4:2:2 (YUY2); Ripped from a DVD=4:2:0 (YV12) (Yes, some clients bring they own DVD to be ripped !). So the Color Space is important for me as well as the fact that is REALLY lossless.
Compositing several layers of video (often more than 100 layers) will introduce some lossless inhenrent of the compositing process itself, specially when clips are used with transparency.
Lossless is important meaning you can recompress a clip several time without having any losses on this operation, as you WILL have losses during composition, you do not want to add several layers of losses.

So please keep it lossless (or at least give the option)


timeToy

[rocker60]

quote:
--------------------------------------------------------------------------------
Originally posted by -h -AT 19th November-
I'll do this "soon". It's a fairly simple task, but I'll probably fork huffy into a new FourCC / data format just for the job.

-h
--------------------------------------------------------------------------------

Today it's 29th November.
10 days just for a simple task? :rolleyes:

[wotef]

WTF gives with the rolling of your eyes?

did you commission this development? are you paying -h for his personal effort and time?

it'll be done when it's done...

[sh0dan]

Originally posted by rocker60

10 days just for a simple task? :rolleyes:

If it's so simple, then just do it, and attach the result here. That would make a bunch of people happy.

[baz00ie]

timeToy is correct, lossless quality is the single most important factor regarding the huffman codec - anything else should be an option. I would guess that the YV12 conversion request was made with the hope that the full nature and integrity of the original codec would be maintained. That's why we love it so...

take care
baz

[ErMaC]

c'mon guys, I don't think the guy was being greedy or mean when he posted that, sounded more like a friendly jab there, not a demand or anything.

[^^-+I4004+-^^]

Originally posted by ErMaC
c'mon guys, I don't think the guy was being greedy or mean when he posted that, sounded more like a friendly jab there, not a demand or anything.

i agree..also would it be possible to include some kind
of native 4:2:0 DCT system in there...so we get huff+mjpeg in one
package?
(hehe....... just kidding!)

[rocker60]

quote:
--------------------------------------------------------------------------------
Originally posted by: ErMaC
c'mon guys, I don't think the guy was being greedy or mean when he posted that, sounded more like a friendly jab there, not a demand or anything.
--------------------------------------------------------------------------------
That's right! I'm just being a little bit sarcastic... Demanding it's not my intention. Who i'm to demand anything from anyone? My first intention with that (unfortunate) post whas to
bring the thread for the Avisynth first page because is almost forgotten at the end of the second page and to remember -h of is "promesse".
@ErMaC: Thanx for your kind words in my defence.
@ALL: Please let's forget this little incident. Thanx.

[-h]

Hi guys.

I was away for thanksgiving. Should be done this afternoon in some form.

-h

[rocker60]

-h
Don't rush! Take your time but make a good work.
It's great to see you're back mate! :cool:
Cheers.

[Boulder]

I'm also eager to test the codec..one more step towards AVS2.5 for me. With (-h)uffYUV I'll be able to get some more speed for my TV capture encodes which I desperately need;)

[sillKotscha]

Originally posted by Boulder
With (-h)uffYUV I'll be able to get some more speed for my TV capture encodes which I desperately need;)

speed?? aren't we talking about size??? as I understood, a modified version of (-h)uffyuv will result in a less - still huge - *.avi ...

correct me if I'm wrong

[^^-+I4004+-^^]

YV12 makes for faster processing too
(if you use avs2.5)
(less color conversions blah,blah,blah....)

[leo_fischer]

This is only partially related to huffyuv, but has anyone looked at anytning like this?
"Fast Lossless Transform Coding:
The Use of a High Speed Transform in a Finite Field as a Pre-Processor for Huffman Coding"

http://www.eecs.umich.edu/~holtk/reports/ffwavelet/

It looks like something interesting for perhaps making a higher compression lossless video codec (huffyuv on steroids?). It would of corse use more cpu, but from the paper it seems it could increase the compression measurably.
It seems that it would probably be pretty easy to slap this on top of huffyuv to see how well it would work. Unfortunately, I can't find any references about how to code the transform.

[Boulder]

Originally posted by sillKotscha
speed?? aren't we talking about size??? as I understood, a modified version of (-h)uffyuv will result in a less - still huge - *.avi ...

correct me if I'm wrong

Just as -+I4004+- said, YV12 processing is faster. If I want smaller size, I'll just use MJPEG and process in YUY2.

[sh0dan]

@-h:

While you're at it, you could add the "mmx_ConvertRGB32toYUY2" from AviSynth (convert.cpp) instead of the C-one. It uses 24bits, but you should be able to change it, by:

Exhanging:
int lwidth_bytes = w<<2; // Width in bytes
With:
int lwidth_bytes = w*3; // Width in bytes

And:
add RGBOFFSET,8
With:
add RGBOFFSET,6

Also adding RGB -> YV12 conversion would be great! (Currently I use the RGB input -> ConvertToYUY2 all the time).

[-h]

It looks like something interesting for perhaps making a higher compression lossless video codec (huffyuv on steroids?). It would of corse use more cpu, but from the paper it seems it could increase the compression measurably.

I doubt it would increase compression by enough to make it worthwhile - the current "world leader" in image compression is the ERI family of compressors, and in some tests I made a while back, I could almost achieve similar ratios by using context-sensitive huffman coding with the same simple median predictor, all with a very low cost above normal huffyuv speeds.

While you're at it, you could add the "mmx_ConvertRGB32toYUY2" from AviSynth (convert.cpp) instead of the C-one.

I'm not sure what this is in relation to!

-h

[^^-+I4004+-^^]

Originally posted by Boulder
If I want smaller size, I'll just use MJPEG and process in YUY2.

someone in the YV12 FAQ already asked for 4:2:0 version of mjpeg
this means even smaller mjpeg sizes!
let's hope someone will include 4:2:0 in their mjpeg codec
(that was my hint on "including DCT" into this project.....
mjpeg is DCT.....)

morgan works with YV12 as an input,but doesn't bring smaller filesizes (not that i could spot anyway!)
it doesn't have 4:2:0 option in format dialog....

anyhow this is YV12huff thread so let's drop this....

[HarryM]

Originally posted by ^^-+I4004+-^^
someone in the YV12 FAQ already asked for 4:2:0 version of mjpeg
this means even smaller mjpeg sizes!
let's hope someone will include 4:2:0 in their mjpeg codec
(that was my hint on "including DCT" into this project.....
mjpeg is DCT.....)

morgan works with YV12 as an input,but doesn't bring smaller filesizes (not that i could spot anyway!)
it doesn't have 4:2:0 option in format dialog....

anyhow this is YV12huff thread so let's drop this....

Morgan-multimedia's MJPEG2000 codec (J2k based) use YV12 (4:2:0) natively, but speed is dammly slow. I have XP1600+ and I have 40-50% dropped frames :(

[Marc FD]

there are 3 colorspaces in MJPEG's specs : 4:4:4, 4:2:2 and 4:2:0
if the encoder say nothing, you've a lot of chance to have a 4:2:0 version.

[-h]

Most likely it's using 422. Shouldn't be hard to test though.

Oh, and I'm nearly finished on this codec :)

Ended up writing it from scratch, I didn't want to release code as dirty as huffyv12 would be if it was branched from huffyuv.

-h

[Blight]

Any chance of inserting a Delta-Frame mode with optional number of delta frames? Could be useful for re-compression and capture rather than splicing.

[-h]

Any chance of inserting a Delta-Frame mode with optional number of delta frames? Could be useful for re-compression and capture rather than splicing.

I'll try a simple error-against-last-frame mode (i.e. assume zero motion), but I'm not sure how well it will perform. I'd rather keep this as a quick & dirty codec for the time being, and adding motion estimation will make things ugly.

-h

[^^-+I4004+-^^]

Originally posted by Marc FD
there are 3 colorspaces in MJPEG's specs : 4:4:4, 4:2:2 and 4:2:0
if the encoder say nothing, you've a lot of chance to have a 4:2:0 version.

as i see there's some interest,let me say this too:
morgan doesn't have 4:2:0 choice! (yes,M-JPEG2000 has it
but it's too slow!it drops and drops as harry said)
picvideo doesn't even show in compression codecs if
YV12 is selected!

morgan has 4:2:2 and 4:1:1 choice.....

picvideo 1:1:1 and these two.....

or,what's the name of mjpeg that has pure 4:2:0 support if i might
ask?(some link perhaps?)

anyhow....as i can remember i recently did a test with color bars test
pattern and morgan+YV12,and compared that to morgan+YUY2.....
i didn't saw any changes in color saturation or anything.....
the bitrate also didn't dropped with YV12 so it made me doubt that
morgan has native 4:2:0 support.......

[UPollaehne]

@-h:

I would like to lay my hands on your codec since I am more than pleased with the speed of AVISynth 2.5 and mostly do TV captures. Due to this I am desperately looking for a codec that is capable of using YV12 instead of the usual YUY2 as the standard format.

Do not take me wrong, I do not want to put pressure on your. Just a humble question for a release (or alpha or beta) of your codec.

Ullrich.

[-h]

Oh I hear ya, there's nothing I'd rather do than finish debugging and release the thing.

Maybe tonight, but there are a lot of things draining my time just now (leading up to Christmas), and things computer-related get a low priority :(

-h

[UPollaehne]

@-h:

I know what you're talking about. Christmas always comes so unexpected/suddenly. ;-)

Ullrich.

[sh0dan]

Originally posted by -h
While you're at it, you could add the "mmx_ConvertRGB32toYUY2" from AviSynth (convert.cpp) instead of the C-one.

I'm not sure what this is in relation to!

-h

HuffYUV. I see there is a built in conversion from RGB24 -> YUY2 similar to the one in AviSynth in earlier versions. The MMX version is at least twice as fast.

I always convert my huffyuv files to yuy2, if the software exports it as RGB.

[sh0dan]

/me gives Nic a nice hug, and hopes he finds a few hours to finish up this great HuffYUV mod, or posts the source for us happy coders to work on. :cool:

You suggestions for improvements looks very promising, and it would really be great to have it.

[sillKotscha]

/me too ;)

[Suzahara]

Is there any status updates to this project?

[-h]

Sure - it's finished, I just have to run some test video to produce optimized huffman codes. Then make sure it's actually faster than huffyuv :)

-h

[^^-+I4004+-^^]

>I would like to lay my hands on your codec since I am more than pleased with the speed of AVISynth 2.5 and mostly do TV captures. Due to this I am desperately looking for a codec that is capable of using YV12 instead of the usual YUY2 as the standard format

yes,me too..
any speed improvement is welcomed
and if "-h-huff" will be at cca. 5MB/s(of DV) for full pal (too optimistic?)
then even better........

all that's left o say is:
GO -h!

cheers

Ivo