Modification of huffyuv codec for YV12?

[HarryM]

Do exists?

[-h]

I'll do this "soon". It's a fairly simple task, but I'll probably fork huffy into a new FourCC / data format just for the job.

-h

[HarryM]

Originally posted by -h
I'll do this "soon". It's a fairly simple task, but I'll probably fork huffy into a new FourCC / data format just for the job.

-h


@-h:

Thanks.
Huffyuv is very good, powerfull codec. I have bitrate about 11-12MB/s (full PAL 720x576) and this is too much. Native support for YV12 can theorethically lowering average bitrate to 3/4 (cca 8-9MB/s). This is acceptable for me.

[trbarry]

I'll do this "soon". It's a fairly simple task, but I'll probably fork huffy into a new FourCC / data format just for the job.


-h

"(-h)uffYV12" would be a nifty addition to our YV12 tool kit. If I'm using a bazillion filters I sometimes spin a first pass out to huffy and then do it the fast way after that.

And this would improve both speed and space. I really hope you do this.

- Tom

[slavickas]

Originally posted by -h
I'll do this "soon". It's a fairly simple task, but I'll probably fork huffy into a new FourCC / data format just for the job.

-h

ffmpeg already have yv12 hyffyuv version

[Suikun]

Originally posted by slavickas
ffmpeg already have yv12 hyffyuv version
Maybe milan could update ffvfw to support it?

[timeToy]

Hi,

As HuffYV12 will have a different FourCC code, it may be the good time to add what missing in the Original HuffYUV, like a simple checkbox for interlaced content and the setting actually saved in the file !

[Koepi]

As huffyuv is lossless, I don't see why you should need an interlaced flag? you won't gain or loose anything due to it's simple huffman tree nature...

Just my 2 cents.
Koepi

[ErMaC]

From what I understand of Huffyuv the compression algo isn't straight huffman, it's more advanced than that. I think the same reason that using interlaced vid in MPEG2 without using an interlaced flag is bad applies to huffyuv's comp algo, however with huffyuv you don't wind up with a worse looking picture, just a larger file. Conversely, setting interlaced when you don't have to will do the same.

[Marc FD]

huffman is useless as-is on images. huffyuv use prediction (gradient/left/median prediction i think) delta prediction would even more efficent, but it would need keyframes/deltaframes.

i'ld love (-h)uffYV12, for the same reasons that tom pointed out, and for captures too. (i think the gain would only be 10% max, but decoding will be faster, and it'ld avoid conversions)

thx -h ^_^

Cheers,
MarcFD

[-h]

The gain will be around 20%.

-h

[Marc FD]

is finished ??

[HarryM]

Hmmm.
We know.
HuffYUV use pure lossless algorithm.

But what using any slightly-lossy algorithm (not JPEG)???:D

[timeToy]

Please, we want our HuffYV12 losseless, that's all the point.
If you do want some near-lossless use DivX or XviD at 1q2 with everything turn off.

[-h]

Allowing pixels to be off by 1 or 2 (out of 255) can increase compression by 50%, if an RLE method is included. I experimented with many variations of the huffy algorithm while working on a lossless codec (which I never completed of course).

-h

[HarryM]

Originally posted by -h
Allowing pixels to be off by 1 or 2 (out of 255) can increase compression by 50%, if an RLE method is included. I experimented with many variations of the huffy algorithm while working on a lossless codec (which I never completed of course).

-h

Of course! ;)

Little 'controlled' deviations in interval e.g. (x-1) to (x+1) can very help to standard lossless algorithms, without visible degradation of captured video, I think.

[HarryM]

@TimeToy:

Principially exists these three section of graphics compression:

a) lossy (e.g. JPEG)
b) lossless (e.g. huff, zip - standard data compression)
c) near lossless (!)

and lossy 'not equal' near lossless, this are diferent algorithms often...

[timeToy]

I'm sorry but Lossless means lossless (no losses), I use lossless compression to store clips that will be included in an After Effects composition, depending on the color space of the original clip (3D rendered pictures=RGB; Captured from SDI source (DigiBeta)=4:2:2 (YUY2); Ripped from a DVD=4:2:0 (YV12) (Yes, some clients bring they own DVD to be ripped !). So the Color Space is important for me as well as the fact that is REALLY lossless.
Compositing several layers of video (often more than 100 layers) will introduce some lossless inhenrent of the compositing process itself, specially when clips are used with transparency.
Lossless is important meaning you can recompress a clip several time without having any losses on this operation, as you WILL have losses during composition, you do not want to add several layers of losses.

So please keep it lossless (or at least give the option)


timeToy

[rocker60]

quote:
--------------------------------------------------------------------------------
Originally posted by -h -AT 19th November-
I'll do this "soon". It's a fairly simple task, but I'll probably fork huffy into a new FourCC / data format just for the job.

-h
--------------------------------------------------------------------------------

Today it's 29th November.
10 days just for a simple task? :rolleyes:

[wotef]

WTF gives with the rolling of your eyes?

did you commission this development? are you paying -h for his personal effort and time?

it'll be done when it's done...

[sh0dan]

Originally posted by rocker60

10 days just for a simple task? :rolleyes:

If it's so simple, then just do it, and attach the result here. That would make a bunch of people happy.

[baz00ie]

timeToy is correct, lossless quality is the single most important factor regarding the huffman codec - anything else should be an option. I would guess that the YV12 conversion request was made with the hope that the full nature and integrity of the original codec would be maintained. That's why we love it so...

take care
baz

[ErMaC]

c'mon guys, I don't think the guy was being greedy or mean when he posted that, sounded more like a friendly jab there, not a demand or anything.

[^^-+I4004+-^^]

Originally posted by ErMaC
c'mon guys, I don't think the guy was being greedy or mean when he posted that, sounded more like a friendly jab there, not a demand or anything.

i agree..also would it be possible to include some kind
of native 4:2:0 DCT system in there...so we get huff+mjpeg in one
package?
(hehe....... just kidding!)

[rocker60]

quote:
--------------------------------------------------------------------------------
Originally posted by: ErMaC
c'mon guys, I don't think the guy was being greedy or mean when he posted that, sounded more like a friendly jab there, not a demand or anything.
--------------------------------------------------------------------------------
That's right! I'm just being a little bit sarcastic... Demanding it's not my intention. Who i'm to demand anything from anyone? My first intention with that (unfortunate) post whas to
bring the thread for the Avisynth first page because is almost forgotten at the end of the second page and to remember -h of is "promesse".
@ErMaC: Thanx for your kind words in my defence.
@ALL: Please let's forget this little incident. Thanx.

[-h]

Hi guys.

I was away for thanksgiving. Should be done this afternoon in some form.

-h

[rocker60]

-h
Don't rush! Take your time but make a good work.
It's great to see you're back mate! :cool:
Cheers.

[Boulder]

I'm also eager to test the codec..one more step towards AVS2.5 for me. With (-h)uffYUV I'll be able to get some more speed for my TV capture encodes which I desperately need;)

[sillKotscha]

Originally posted by Boulder
With (-h)uffYUV I'll be able to get some more speed for my TV capture encodes which I desperately need;)

speed?? aren't we talking about size??? as I understood, a modified version of (-h)uffyuv will result in a less - still huge - *.avi ...

correct me if I'm wrong

[^^-+I4004+-^^]

YV12 makes for faster processing too
(if you use avs2.5)
(less color conversions blah,blah,blah....)

[leo_fischer]

This is only partially related to huffyuv, but has anyone looked at anytning like this?
"Fast Lossless Transform Coding:
The Use of a High Speed Transform in a Finite Field as a Pre-Processor for Huffman Coding"

http://www.eecs.umich.edu/~holtk/reports/ffwavelet/

It looks like something interesting for perhaps making a higher compression lossless video codec (huffyuv on steroids?). It would of corse use more cpu, but from the paper it seems it could increase the compression measurably.
It seems that it would probably be pretty easy to slap this on top of huffyuv to see how well it would work. Unfortunately, I can't find any references about how to code the transform.

[Boulder]

Originally posted by sillKotscha
speed?? aren't we talking about size??? as I understood, a modified version of (-h)uffyuv will result in a less - still huge - *.avi ...

correct me if I'm wrong

Just as -+I4004+- said, YV12 processing is faster. If I want smaller size, I'll just use MJPEG and process in YUY2.

[sh0dan]

@-h:

While you're at it, you could add the "mmx_ConvertRGB32toYUY2" from AviSynth (convert.cpp) instead of the C-one. It uses 24bits, but you should be able to change it, by:

Exhanging:
int lwidth_bytes = w<<2; // Width in bytes
With:
int lwidth_bytes = w*3; // Width in bytes

And:
add RGBOFFSET,8
With:
add RGBOFFSET,6

Also adding RGB -> YV12 conversion would be great! (Currently I use the RGB input -> ConvertToYUY2 all the time).

[-h]

It looks like something interesting for perhaps making a higher compression lossless video codec (huffyuv on steroids?). It would of corse use more cpu, but from the paper it seems it could increase the compression measurably.

I doubt it would increase compression by enough to make it worthwhile - the current "world leader" in image compression is the ERI family of compressors, and in some tests I made a while back, I could almost achieve similar ratios by using context-sensitive huffman coding with the same simple median predictor, all with a very low cost above normal huffyuv speeds.

While you're at it, you could add the "mmx_ConvertRGB32toYUY2" from AviSynth (convert.cpp) instead of the C-one.

I'm not sure what this is in relation to!

-h

[^^-+I4004+-^^]

Originally posted by Boulder
If I want smaller size, I'll just use MJPEG and process in YUY2.

someone in the YV12 FAQ already asked for 4:2:0 version of mjpeg
this means even smaller mjpeg sizes!
let's hope someone will include 4:2:0 in their mjpeg codec
(that was my hint on "including DCT" into this project.....
mjpeg is DCT.....)

morgan works with YV12 as an input,but doesn't bring smaller filesizes (not that i could spot anyway!)
it doesn't have 4:2:0 option in format dialog....

anyhow this is YV12huff thread so let's drop this....

[HarryM]

Originally posted by ^^-+I4004+-^^
someone in the YV12 FAQ already asked for 4:2:0 version of mjpeg
this means even smaller mjpeg sizes!
let's hope someone will include 4:2:0 in their mjpeg codec
(that was my hint on "including DCT" into this project.....
mjpeg is DCT.....)

morgan works with YV12 as an input,but doesn't bring smaller filesizes (not that i could spot anyway!)
it doesn't have 4:2:0 option in format dialog....

anyhow this is YV12huff thread so let's drop this....

Morgan-multimedia's MJPEG2000 codec (J2k based) use YV12 (4:2:0) natively, but speed is dammly slow. I have XP1600+ and I have 40-50% dropped frames :(

[Marc FD]

there are 3 colorspaces in MJPEG's specs : 4:4:4, 4:2:2 and 4:2:0
if the encoder say nothing, you've a lot of chance to have a 4:2:0 version.

[-h]

Most likely it's using 422. Shouldn't be hard to test though.

Oh, and I'm nearly finished on this codec :)

Ended up writing it from scratch, I didn't want to release code as dirty as huffyv12 would be if it was branched from huffyuv.

-h

[Blight]

Any chance of inserting a Delta-Frame mode with optional number of delta frames? Could be useful for re-compression and capture rather than splicing.

[-h]

Any chance of inserting a Delta-Frame mode with optional number of delta frames? Could be useful for re-compression and capture rather than splicing.

I'll try a simple error-against-last-frame mode (i.e. assume zero motion), but I'm not sure how well it will perform. I'd rather keep this as a quick & dirty codec for the time being, and adding motion estimation will make things ugly.

-h

[^^-+I4004+-^^]

Originally posted by Marc FD
there are 3 colorspaces in MJPEG's specs : 4:4:4, 4:2:2 and 4:2:0
if the encoder say nothing, you've a lot of chance to have a 4:2:0 version.

as i see there's some interest,let me say this too:
morgan doesn't have 4:2:0 choice! (yes,M-JPEG2000 has it
but it's too slow!it drops and drops as harry said)
picvideo doesn't even show in compression codecs if
YV12 is selected!

morgan has 4:2:2 and 4:1:1 choice.....

picvideo 1:1:1 and these two.....

or,what's the name of mjpeg that has pure 4:2:0 support if i might
ask?(some link perhaps?)

anyhow....as i can remember i recently did a test with color bars test
pattern and morgan+YV12,and compared that to morgan+YUY2.....
i didn't saw any changes in color saturation or anything.....
the bitrate also didn't dropped with YV12 so it made me doubt that
morgan has native 4:2:0 support.......

[UPollaehne]

@-h:

I would like to lay my hands on your codec since I am more than pleased with the speed of AVISynth 2.5 and mostly do TV captures. Due to this I am desperately looking for a codec that is capable of using YV12 instead of the usual YUY2 as the standard format.

Do not take me wrong, I do not want to put pressure on your. Just a humble question for a release (or alpha or beta) of your codec.

Ullrich.

[-h]

Oh I hear ya, there's nothing I'd rather do than finish debugging and release the thing.

Maybe tonight, but there are a lot of things draining my time just now (leading up to Christmas), and things computer-related get a low priority :(

-h

[UPollaehne]

@-h:

I know what you're talking about. Christmas always comes so unexpected/suddenly. ;-)

Ullrich.

[sh0dan]

Originally posted by -h
While you're at it, you could add the "mmx_ConvertRGB32toYUY2" from AviSynth (convert.cpp) instead of the C-one.

I'm not sure what this is in relation to!

-h

HuffYUV. I see there is a built in conversion from RGB24 -> YUY2 similar to the one in AviSynth in earlier versions. The MMX version is at least twice as fast.

I always convert my huffyuv files to yuy2, if the software exports it as RGB.

[sh0dan]

/me gives Nic a nice hug, and hopes he finds a few hours to finish up this great HuffYUV mod, or posts the source for us happy coders to work on. :cool:

You suggestions for improvements looks very promising, and it would really be great to have it.

[sillKotscha]

/me too ;)

[Suzahara]

Is there any status updates to this project?

[-h]

Sure - it's finished, I just have to run some test video to produce optimized huffman codes. Then make sure it's actually faster than huffyuv :)

-h

[^^-+I4004+-^^]

>I would like to lay my hands on your codec since I am more than pleased with the speed of AVISynth 2.5 and mostly do TV captures. Due to this I am desperately looking for a codec that is capable of using YV12 instead of the usual YUY2 as the standard format

yes,me too..
any speed improvement is welcomed
and if "-h-huff" will be at cca. 5MB/s(of DV) for full pal (too optimistic?)
then even better........

all that's left o say is:
GO -h!

cheers

Ivo

[Boulder]

-h,

can you give any estimates, how much will the datarate be at 720x576 25fps?

[HarryM]

@Boulder: It is depend on content of video, of course. But (I hope) -h's YV12 modification is surely more sparing than Huffyuv_YUY2.
Only upgrading from YUY2 to YV12 (16bits<>12bits) make sparing effect=25%, theoretically.

[UPollaehne]

@-h: Nice to hear this. Please let us help you with testing. ;-)

Ullrich.

[Boulder]

Originally posted by HarryM
@Boulder: It is depend on content of video, of course. But (I hope) -h's YV12 modification is surely more sparing than Huffyuv_YUY2.
Only upgrading from YUY2 to YV12 (16bits<>12bits) make sparing effect=25%, theoretically.

I know, that's why I said estimates;)

At the moment I'm unwilling to use the YUY2 version for anything longer than 60 mins so hopefully I'll be able to extend the caps using -h's version to a full movie length:cool:

[Suzahara]

Originally posted by -h
Sure - it's finished, I just have to run some test video to produce optimized huffman codes. Then make sure it's actually faster than huffyuv :)

-h

Excellent news. I got kind of worried when this thread wasn't active for a while :D

[wotef]

sign me up, too, if you need help testing, heh

can't wait to get at this! :D

[sh0dan]

Don't know if this actually help, but here goes.

When you have a look at how U/V is mapped in typical movie (non-anime) sources, they are mostly mapped pretty close to 128. (Just try UtoY() in AviSynth 2.5).

Y is typically mapped between 16 and 128 (try histogram()).

Are the tables automatically adjusting to this, or could the tables be optimized for this situation?

IMO it could also be a very good thing to be able to clip invalid YUV values, so that Y and UV values below 16 are mapped to 16, Y values are maxed at 235 and UV are maxed out at 240. For sensible souls it could be made an option (for huffyuv to be truly lossless), but these values are invalid and does not contribute to the visual quality. A very fast ISSE implementation is in "levels.cpp" at the bottom.

Clipping could lead to better compression on sources that have been colorspace-converted, or have compression artifacts left.

[-h]

Are the tables automatically adjusting to this, or could the tables be optimized for this situation?

The prediction is just the median of the left, top and gradient (left + top - topleft) values, with the error being stored. Context modelling could be performed since as you point out some error values are not possible, however it would slow things down and wouldn't save many bits - typical error images are heavily weighted towards small errors (87% of errors are -2 <= x <= 2 in the Y plane, for the U and V planes 94% are -1 <= x <= 1!).

Clipping could lead to better compression on sources that have been colorspace-converted, or have compression artifacts left.

I hadn't considered adding a clipping option, that's a fine idea.

-h

[Boulder]

Let him finish the initial release before you postpone it by your marvellous (I really mean it!) ideas, sh0dan:devil:

[sh0dan]

Ideas never hurt - and I would like to see an YV12-version ASAP too. On the other hand, things like this must be in place before any formal release, otherwise you'll just end up having to do an incompatible version later - with a lot of angry users.

Just look at the 2.0 vs. 2.5 plugin compatibility - how many users haven't complained because the alpha version just crashed on 2.0 plugins - even though it has been clearly marked as alpha material - and I'm just awaiting the uproar, when old 2.5 plugins cannot be loaded because users haven't updated them or the developers haven't changed their code yet. (sorry - just had to get it out ;)

[-h]

I will definitely be releasing incompatible versions after the initial release (too many ideas to try), however new builds will always decode old versions.

I am assuming that people won't mind all that much, since lossless video codecs are rarely used for long-term archiving.

-h

[Boulder]

Originally posted by sh0dan
Ideas never hurt - and I would like to see an YV12-version ASAP too. On the other hand, things like this must be in place before any formal release, otherwise you'll just end up having to do an incompatible version later - with a lot of angry users.


It was pure sarcasm on my behalf, dear friend.. it's like we're all kids waiting for our delayed Christmas present here:D And kids can get so impatient at times!

[cweb]

With the current huffyuv the only valid depth is 24 bits. Could this
be extended to 32 bits (so I can use it in Alamdv2), for YV12
and the current implementation? I was wondering. Or is it the huffyuv which I installed which is perhaps an old one?

[Boulder]

How's the testing progress going, shall we soon be able to enjoy the benefits of YV12 captures?

[Suzahara]

Heh, I'm getting worried since there hasn't been status updates :o ...Is it near release time?

[-h]

I guess it is? I'm not terribly good at hitting release schedules, in case no one's noticed :)

This has actually been my first week at a new job and I've been having a blast, partying most nights and recovering most days. But I have all of tomorrow afternoon off, and I want to release both this and the MSMPEG4V3->MPEG4 convertor in a grand display of bugs and ignoring feedback. Should be a great show.

There is also another eerie secret project which I'm having a lot of fun working on, even though I know it's going to haunt me further down the line.

-h

[Suzahara]

Originally posted by -h
There is also another eerie secret project which I'm having a lot of fun working on, even though I know it's going to haunt me further down the line.

Eerie projects are always good...especially secret ones :D

[Boulder]

Originally posted by -h
There is also another eerie secret project which I'm having a lot of fun working on, even though I know it's going to haunt me further down the line.

-h [/B]

Don't worry, you can tell us - we're really good at keeping secrets:devil:

[ales19]

any news from the front? :D :D

[-h]

Yeah, I'm sick of seeing this in my dev/unfinished folder. Tomorrow I only work 11am-2pm, and have all of the day after off. I will finish this and the MSMPEG4V3->MPEG4 convertor. Or else.

-h

[digitize]

Im glad to hear it -h, and I thank you for all the hard work you've put in :)

[-h]

My working hard is a vicious rumour, and I'm yet to discover who started it.

-h

[digitize]

ah im sorry, well thank you for working period.

[ales19]

Originally posted by -h
Yeah, I'm sick of seeing this in my dev/unfinished folder. Tomorrow I only work 11am-2pm, and have all of the day after off. I will finish this and the MSMPEG4V3->MPEG4 convertor. Or else.

-h

yay, im really impatient to test this out :)
thanks man

[wotef]

:p

[Boulder]

*a shameless bump*

[Boulder]

Any news yet? :devil:

[Suzahara]

No news/release dates yet? :confused:

I'm starting to fear the worst again :( :scared:

[neuron2]

POLITE MODERATOR REQUEST: Gents, please stop the pointless harrassing posts. When and if it is ready it will be announced I am sure. Thank you.

[FishB8]

The work on this lossless codec has me wondering, would it be much work to store this in an Ogg container? Xiph has lossless (FLAC) and lossey (Vorbis & Speex) audio, and lossey video (Theora & Tarkin) but they have no lossless video. If there's no patent issues with HuffYUV they might even consider sponsoring it if you could store it in an Ogg container.

Had to spend my 2¢ somewhere... :)

[Suzahara]

Originally posted by FishB8
The work on this lossless codec has me wondering, would it be much work to store this in an Ogg container? Xiph has lossless (FLAC) and lossey (Vorbis & Speex) audio, and lossey video (Theora & Tarkin) but they have no lossless video. If there's no patent issues with HuffYUV they might even consider sponsoring it if you could store it in an Ogg container.

Had to spend my 2¢ somewhere... :)

Well, I'd figure it's already storable in an Ogg Container. Divx and XVid are storable and I don't think any extra coding was put in just for those two codecs. Although I can't imagine why you'd want to do this. Huffyuv is extrememly high on size since it's around 2.0:1 on compression. This makes filesizes big really fast. So unless you've got a ton of space to use up, most people won't actually do this.

[Belgabor]

Well, what we all want to do at some point (at least I hope all want to..) is replace avi completely, so its perfectly legal to want it usable in other containers. But as you said, thats already possible :)
Another point you have to consider is that a) -h wanted to improve on compression and b) that YV12 in itself is already smaller than YUV2

[Suzahara]

Yeah, avi is an ancient format that needs to be replaced by more flexible formats.

But I still don't believe that you would want huffyuv or huffyv12 as a long term storage thing since I don't think you can improve compression all that much (meaning it will still take up GB's very quickly) since lossless requires space and until we can find a lossless solution that can compress as much as divx or xvid, huffyuy(yv12) will still only serve as an intermediate file between (unfortunately) lossy formats. :(

[FishB8]

But I still don't believe that you would want huffyuv or huffyv12 as a long term storage thing

No I wouldn't. I would want to use it more for a video editing purposes. I'm a linux user and right now all the video editing solutions use a hodge podge of avi, quicktime, DV, and various types of MPEG. I would prefer a container format more native to linux, such as Ogg. I know Ogg was origionally developed to be a delivery container, but I don't imagine it would be too tough to use it for editing.

what we all want to do at some point (at least I hope all want to..) is replace avi completely, so its perfectly legal to want it usable in other containers. But as you said, thats already possible

Exactly, but are there any tools that make this possible on linux...

[^^-+I4004+-^^]

"2003-01-23 07:06 milan_cutka

huffYV12 encoding, graph currently disabled"

from:
http://homepages.pathfinder.gr/ffvfw/

i didn't tested or checked anything,just saw this changelog...

[jcsston]

Originally posted by ^^-+I4004+-^^
"2003-01-23 07:06 milan_cutka

huffYV12 encoding, graph currently disabled"

from:
http://homepages.pathfinder.gr/ffvfw/

i didn't tested or checked anything,just saw this changelog... :( I tried this out, but I couldn't decode will anything. ffvfw gave me a distorted picture and YUV2 Huffyuv crashed

[^^-+I4004+-^^]

Originally posted by jcsston
:( I tried this out, but I couldn't decode will anything. ffvfw gave me a distorted picture and YUV2 Huffyuv crashed

he did put decoding capability few hours (i presume by the changelog)
afterwards,but anyhow,have you tried ffdshow?
(currently i don't have such recent versions of either ffvfw or ffdshow and ,frankly,i cooled down from all this huffyv12 stuff,as
anything larger than 2-3MB/s is too big for me so i'll stuck with mjpeg...)

[fisix]

in agony, i bump.

[Marc FD]

hi.

i'm currently playing with yv12 lossless compression.

about compressibility improvements with yv12.
IMHO, it's not big. because chroma planes are VERY compressible (ratios often over 4.0:1) and luma as hard to compress than in yuy2 ^^

if you have a very hard to compress frame, like one of the frame of my test clip, where luma is 1.6:2, chroma ~4:1, it gives around 2:1 in overall in yv12, but it would be better in yuy2 (~2.5:0), because the additionnal (4bpp) chroma data is more easy to compress ^^

Cheers,
MarcFD

[Boulder]

Marc,

what about near-lossless compression, would that be hard to accomplish? The main problem is that we don't have a YV12 codec for capturing so a possible minor loss of details wouldn't hurt. I believe there was some talk about that elsewhere in this thread. I think people use mainly MJPEG for capturing and with high enough quality settings, the result is near-lossless IMO.

What do you think? I think that -h's too busy nowadays so maybe you could come up with something for us capture freaks?

[Richard Berg]

Indeed, I recall -h saying that merely allowing for +/- 1 error would dramatically increase compressibility. Since YV12 already throws away substantial information with most capture cards vs. HuffYUV, I think such a codec would be a most useful compromise, at least until someone is able to make JPEG2000 run at 30fps :)

[fisix]

Indeed, I recall -h saying that merely allowing for +/- 1 error would dramatically increase compressibility. Since YV12 already throws away substantial information with most capture cards vs. HuffYUV, I think such a codec would be a most useful compromise, at least until someone is able to make JPEG2000 run at 30fps

what are we giving away by using yv12 with capture cards? is this just saying that there is a conversion from yuv2 to yv12 when using most cards because they only output yuy2?

my card is one of the many bt8x8 cards from hauppauge, and the driver gives the option of 4:2:2 packed AND YUV12 planar. if the wintv cards support it, doesn't that almost mean most cards support it?

anyway, a codec that isn't lossless isn't huffyv12. its something else.

as an aside, i saw someone write something about poor decoding ability of huffyuv, and i wondered if the decoding speed was something -h was looking into also. see this thread, post by aktan:

http://forum.doom9.org/showthread.php?s=&threadid=35387

[sh0dan]

The decoding speed of huffyuv is quite good. The missing DirectShow interface is mainly what makes it appear to be slow.

[Marc FD]

>what about near-lossless compression, would that be hard to accomplish?

mhh. not really. dunno how huffyuv works. but i think it's useless.

BTW, the true-lossless thing is what we seek, because for lossy compression, MJPEG will _always_ win.

>The main problem is that we don't have a YV12 codec for capturing

my old PCTV gives me YV12/I420, so with a yv12 lossless codec, you should be able to capture in yv12 ^^.

>so a possible minor loss of details wouldn't hurt. I believe there
>was some talk about that elsewhere in this thread. I think people use
>mainly MJPEG for capturing and with high enough quality settings, the
>result is near-lossless IMO.

yes, but it _is_ lossy. if you reencode 10x a clip with a lossless codec, you still have full quality.

>What do you think? I think that -h's too busy nowadays so maybe you >could come up with something for us capture freaks?

héhé. -h was too slow, so i did something this WE ^^.
honestly, coding a codec from scratch in a WE is really not easy ^___^

compression algo is 100% mine, and it's really not bad (that why i did a codec with). i'll release an alpha for you all to play with tomorow.

VBLE Video Codec, sound good, no ? ^^

>Indeed, I recall -h saying that merely allowing for +/- 1 error would >dramatically increase compressibility.

maybe around 5-10%, but i don't think more. my algo achieve much more, and it's truly lossless.

[Boulder]

Originally posted by Marc FD

if you reencode 10x a clip with a lossless codec, you still have full quality.

compression algo is 100% mine, and it's really not bad (that why i did a codec with). i'll release an alpha for you all to play with tomorow.


I meant near-lossless codec for capturing. As an intermediate codec a true lossless one should definitely be used, I agree.

I can hardly wait, lots to capture:cool:

[vidiot]

Neuron did say that it won´t take long til we see you back MarcFD!

And I´m glad he is right!:D

Knowing all your filters I guess we´ll see a pretty fast "alpha" these days.

Curiously
Harald

[digitize]

Ah cool, i cannot wait to see marc fd's codec, he usually codes good stuff and Im sure this will be the same :D

[McQuaid]

In regards to capping, I checked and I don't have yv12 for my capture card but I do have yuv12 are they the same? Also the other post about near lossless wouldn't be useless for capturing to save space, would be handy feature. It would be nice to have something similar to mjpeg compression sizes while staying in the same colour space throughout.

[Blight]

Regarding decoding, FFDShow decodes huffyuv and is faster than using the huffyuv codec as it is directshow based.

[Marc FD]

>>Neuron did say that it won´t take long til we see you back MarcFD!

wow, wow, not so fast.
i never coded a codec, so i thought it would be fun.

but i'll get a dsl connection very soon, and i don't think i'll have 5 minutes free for devel then ^^.

here's a beta of VBLE. compression is close to huffyuv, with 25% more thanks to yv12 colorspace.

encoding part is pure SIMD (isse processor required, no check)
decoding is pure C, and hard to optimise.

it's way slower than huffyuv, but it should be fast enough for capture on modern cpus ^^.

(attached "VBLE Video Codec (beta version).zip")

Install instruction : dezip & install using the .inf file ^_^

enjoy,
MarcFD

>In regards to capping, I checked and I don't have yv12 for my capture
>card but I do have yuv12 are they the same? Also the other post about
>near lossless wouldn't be useless for capturing to save space, would
>be handy feature. It would be nice to have something similar to mjpeg
>compression sizes while staying in the same colour space throughout.

well, a hyper-fast yv12 native wavelet codec would be cool, sure, but my math skill is way too limited for that (remember i'm 17 years old)

[digitize]

hmmm, guess we'll have to wait for a moderator for the attachment.

[Koepi]

Hm, it works quite well, not a single dropped frame (but i checked max 3 mins for that).

Data rate is a bit too high for my limited space though :-/

Do you think you can do something about that Marc?

Btw.: congrats, it's a nice codec indeed :) And that from a 17 years old... :)

Respect!

Regards
Koepi

PS: I could scan you some of the scientific papers which I had to work through in Uni, but I don't think they're of any use for you %)

[neuron2]

@MarcFD

Ha! Back even sooner than I expected.

Big questions for you:

1. Does your codec use interlaced or progressive downsampling?

2. Can you add an option to select interlaced or progressive downsampling?

3. Any chance of you releasing the source code?

Thank you.

[Schlumpf]

Originally posted by Marc FD
well, a hyper-fast yv12 native wavelet codec would be cool, sure, but my math skill is way too limited for that (remember i'm 17 years old)

You know...I'm already 20 but still have to take tuition in mathematics if I don't want to fail my final exams and I understand less than a fraction of the technical stuff posted here...
Damn, I feel so useless! :(


Anyway, thanks for this sweet codec. I will test it this weekend :)

[Si]

@marcFD
Excellent :D

In a quick test against Huffyuv -
I get about 10% decrease in compression speed with 26% decrease in file size :D

Decompression is 30% slower.

I'm also very interested in answers to neuron2's questions.

regards

Simon

[neuron2]

I did some experiments and am almost certain the codec is downsampling using the progressive algorithm. Since I know MarcFD is quite well aware that his beloved XviD (as well as DivX) does only progressive upsampling, it stands to reason that he would have done it that way.

The problem is that it leads to color corruption if the source is interlaced. To prove this, download the attached AVI and encode it using his codec. Then view the original AVI and the encoded one. You'll see that fields have been blended together (and it is easy to show theoretically that it occurs).

The codec desperately needs an option to specify the downsampling as either progressive-type or interlaced-type.

[neuron2]

Originally posted by Marc FD
well, a hyper-fast yv12 native wavelet codec would be cool, sure, but my math skill is way too limited for that (remember i'm 17 years old) http://www.maven.de/code/#wavelet

Might get you started thinking about things. :)

[Boulder]

This is slightly OT, but how does one enable YV12 format in VirtualDubMod in Windows XP? I tried setting the custom format to YV12 but all I get is a green screen and a very slowly responsive VDubMod. My card's a Hauppauge WinTV Theatre and I'm using the BTWinCap drivers.

Switching the format works fine in Win98 with Hauppauge VfW drivers and the codec seems to work okay:) I just want to get rid of Win98 at last, video capturing is the only reason why I've still kept a dual-boot system.

[ACClarke]

@MarcFD
I see that you reinstall your Visual C++ :D
It's good to see you back

I'm sure you make a great job with this codec but I don't have a computer to test it......

I will be back in april, it's hard to wait !

[Marc FD]

>Data rate is a bit too high for my limited space though :-/

sure, a so simple lossless codec is not going to do miracles ^^.

>Do you think you can do something about that Marc?

well, the whole stuff is a choice. there are 3 parameters : quality, speed and size. you cannot have them together. here, the choice is quality (lossless ^^) and speed (70 cycles per pixel in encoding). size is of course terribly big ^^. but it's temporal storage (i hope ^^).

>Btw.: congrats, it's a nice codec indeed And that from a 17 years old...

^^. i'm more close to 18 than 17 ^^.

> Ha! Back even sooner than I expected.

maybe not, because i'm not really back ^^.

it was just to wait for my dsl line (because i'm in holidays) and you can guess than as soon as i get it, i'll (almost) disappear from devel stuff.

>Big questions for you:
>1. Does your codec use interlaced or progressive downsampling?

progressive.
oups, didn't even thinked at that !

>2. Can you add an option to select interlaced or progressive >downsampling?

er, doing a codec wasn't too hard, but my knowledge of winapi is _so_ bad than i don't even no how to draw a single dialog box. and to be honest, i simply don't want to know ^^.

my advice : use avisynth 2.5 beta, and you would be able to do interlaced downsampling then (i.e. with the stuff i gave to you) and to give yv12 to the codec, it's gonna be (almost) as fast.

>3. Any chance of you releasing the source code?

you know how lazy i am for doc/src/ect.. and for 1000 lines of crap like that, i don't think it does worth it ^^.

but since i'm using GPL code, and because i'm for opensource dev, i'll give the source if someone _really_ wants it.
but it'll take a lot of time, because i don't want to clean the stuff (and i can't release it without some cleaning) and i don't want to play with code (so i'll do it, but slowly ^^).

>I did some experiments and am almost certain the codec is downsampling >using the progressive algorithm. Since I know MarcFD is quite well >aware that his beloved XviD (as well as DivX) does only progressive >upsampling, it stands to reason that he would have done it that way.

i'm not sure, but try to activate interlacing in XviD. you'll be surprised, it's doing much more than only field-based DCT, if i rememeber well. (even if i never tested it ^^)

>The problem is that it leads to color corruption if the source is
>interlaced. To prove this, download the attached AVI and encode it
>using his codec. Then view the original AVI and the encoded one.

well, give interlaced yv12 and everything is okay. btw, if you encode with your TV card, yv12 should be interlaced. so, no problem ^^.

>You'll see that fields have been blended together (and it is easy to
>show theoretically that it occurs).

>The codec desperately needs an option to specify the downsampling as >either progressive-type or interlaced-type.

depends on what you want to do. but since colorspace convertions are not codec-dependant, it's easy to workaround this, so it's not desperate IMHO.

>I see that you reinstall your Visual C++

i never uninstalled it, it's just hidden to avoid tentation (asm virus ^^)

>It's good to see you back

i'm already gone again ^^

>I'm sure you make a great job with this codec but I don't have a
>computer to test it......

well, it's just simple code :

- median predict the image (isse), pack signed/unsigned into intensity (mmx), get the length by level (mmx), pack the bitstream (length+val) (mmx), and that's all. i tried a bit more advanced stuff, but it was too slow or to unefficient (less than -5%)

>I will be back in april, it's hard to wait !

lol ^^. btw, i re-encoded all the slayers try you gave me in xvid q6-9 asp + ogg for 3 cds (instead of the 7 original CDs), and it still looks good !

bye all,
Marc

[^^-+I4004+-^^]

on win98 can't get proper image (only different color
blocks.no real image.....) with YV12 or YUY2 as input..

on w2k crashed VDub, couldn't recover it in that
session........(used btwincap+YV12 as input...)

very poor documantation too.....(heh)

[neuron2]

Originally posted by Marc FD
er, doing a codec wasn't too hard, but my knowledge of winapi is _so_ bad than i don't even no how to draw a single dialog box. and to be honest, i simply don't want to know ^^. Please release the source code and I'll add the dialog handling for you and then give it back.

my advice : use avisynth 2.5 beta, and you would be able to do interlaced downsampling then (i.e. with the stuff i gave to you) and to give yv12 to the codec, it's gonna be (almost) as fast.The stuff you gave me is compiled with AvisynthPluginInit(), making it unusable. And, of course, you didn't give me the source code. :) Finally, you told me not to redistribute it, so others cannot benefit.

i'm not sure, but try to activate interlacing in XviD. you'll be surprised, it's doing much more than only field-based DCT, if i rememeber well. (even if i never tested it ^^)I didn't see any interlaced upsampling code in XviD, but I'll test it just for fun.

Thank you for your response.

[^^-+I4004+-^^]

oups graft,you forgot he's 17?
he has yet to find where his heart is.......


but sure he should give source......(hehe)

[Kurosu]

@Neuron2
Just to clear *one* misunderstanding, although other matters are none of my business:


>what about near-lossless compression, would that be hard to accomplish?
mhh. not really. dunno how huffyuv works. but i think it's useless.

He only stated that doing near-lossless compression (for instance dropping one bit due to noise if it's usefull) would not produce a result worth the implementation for it.

Next point:

The stuff you gave me is compiled with AvisynthPluginInit()

If really compiled for AVS2.5, and if AvisynthPluginInit is the only difference, you can do that ugly hack: replace "AvisynthPluginInit@" ASCII string by "AvisynthPluginInit2@" in the binary (hexadecimal edit). Yes, it _is_ ugly, may not work, but I've just tested it as working right now on several plugins (both loading and processing).

On a side note, he has spent quite a lot of time on coding this last year, and he has some serious exams soon to pass, I guess. Maybe he's hadnling the whole matter a bit lightly, but in the end, wouldn't it produce more good than bad?

Last point, about the codec:
The decoding under W2K produced a totally garbled video, like would do the use of an uncorrect pitch. The decoding in Vdub(mod) was fine though. (Maybe software specific).
Test sequence: a 512x384 xvid encoding of a much cleaned and flat source (anime DVD).

|HUFYUV| VBLE
-----------+------
enc.| 65 | 55
fps | |
----+------+------
play| |
fps | 250 | 160
----+------+------
size| 64.2M| 96.9
----+------+------

I don't know where the size gain comes from, but it sure is impressive, although the material (not a TV cap) helps a lot.

[neuron2]

OK, forgive my impatience. I'll edit my post. I hope you like it better.

[neuron2]

@Kurosu

>If really compiled for AVS2.5, and if AvisynthPluginInit is the only
>difference, you can do that ugly hack: replace "AvisynthPluginInit@"
>ASCII string by "AvisynthPluginInit2@" in the binary (hexadecimal
>edit).

That worked. Thank you for suggesting it.

@MarcFD

Do you have any objection to making your yuy2toyv12i() available to other people? If you give us the source for the codec and for this filter we can integrate them with an option, if you are too busy to do it. Thank you.

[Marc FD]

well, i switched from 56k to 512k this morning, so i'm not gonna touch anything related to dev before several weeks at least. sorry, too much counter-strike to play and to much leeching to do ^^.

but i'll get bored a day, and i'll finalize things who needs to be finalized. BTW, the interlaced upsampling i gave you is in fact just a little piece of a motion compensated denoiser i did some month ago, and i think it's better not to distribute it.

i'll add a config box with interlaced/progressive up/downsampling someday, i could add RGB32/24 & YUY2 native encoding too, and release the code, but now, i'd like to do something else.

thx for comprehension.

bye,
Marc

[c0de_v0id]

case closed... :(

[Zarxrax]

Hrmm, anyone know what happened to -h? It was weird cause he said he would try and have it finished in like a day, then hasn't posted again for 1-2 weeks. I hope nothing has happened to him :eek:
I've seen a lot of people just suddenly up and disappear off the face of the internet, and it never gets out of the back of my head that these people could be dead or something :scared:

[sungey]

yaa ... i like to know that too ...

btw i just tested VBLE in my anime encoding .... the normal size for my huffyuv avi for the show was 5-6 gigs and with vble i got 4.4 gig nice :) ..

after that... i began checking the colorspaces using avisynth
info() ... i just found out that xvid,divx5 and vble are YV12 ( of course they are :) ) but divx3 is rgb32 ? ... i thought mpeg4 are all using YV12 ... ... i tried comparing the encoding speed betweeen huffyuv -> divx3 and vble -> divx3 ... seems like theres no speed gain .... ( or maybe im blind ).

Seems like colorspaces conversion in my divx3 anime encoding is unavoidable :( ... not a big prob though .. i do xvid most of the time now .... just sharing my experience :)

p/s : the source of my anime are usually DivX5 or xvid most of the time .. if divx3 is really rgb32 ... theres a waste of bits there..

[Belgabor]

All MPEG4 codec use YV12 internally, that doesn't mean they accept it. And btw you need a YV12 enabled version of VDub to process without colorspace conversion, which nandub certainly isn't (except someone modded it while i wasn't looking :p)

Cheers
Belgabor

P.S. Hm, I still fail to see why ppl insist on keeping to use Divx3...

[c0de_v0id]

yeah im pretty sure the codec itself is able to use most of the colorspaces. It's just nandub that can't encode any other colorspaces than rgb. i saw a post saying its because of anti-shit.

Also i saw some posts in forum saying nandub can handle yv12(???) with fast recompress+avisynth 2.5 (because ffdshow reports the nandub encode as yuv12)(?????)

btw, i looked at all sbc encodes i have and they are all rgb. i also did some test encodes myself with fast recompress sources being yv12
(one without any conversion, one with converttoyuv2), but they all came out as rgb.

can someone clarify this?

[sungey]

yeah right .... probably its nandub that cant work with YV12 ...
so it converts to RGB32 ... and leave the encode in RGB32 since reconverting to yv12 will make it slower and lowers quality .... that's just my guess though ...

[Si]

I still fail to see why ppl insist on keeping to use Divx3...

Because its got the best quality vs compression factor vs stability factor codec around maybe?

regards
Simon

[neuron2]

But no B frames.

[^^-+I4004+-^^]

Originally posted by sungey
yeah right .... probably its nandub that cant work with YV12 ...
so it converts to RGB32 ... and leave the encode in RGB32 since reconverting to yv12 will make it slower and lowers quality .... that's just my guess though ...

don't know bout yv12,but i can say this:
mjpeg avi->avs205->ND(fastrepack)->ok...
so it's YUY2 input converted directly to yv12...
(off course,fastrepack means avs must do all the processing,and no
antishit...)
but that's no big news....

i think there was some talk on forum on this issue of yv12->yv12
ND.......(or just previewing it?)

[scmccarthy]

@c0de_vi0dbtw, i looked at all sbc encodes i have and they are all rgb. All mpeg-4 codecs use YV12, including divx3.11alpha. Once you've encoded it to sbc, it is always the same format and always has been. AviSynth has nothing to do with that.

On the other hand, players (rather the codecs) convert everything to rgb for the display. So everything gets encoded to YV12 and decoded to RGB.

How did you look at the sbc encodes? Did you open them in a hex viewer and read the fourcc codes?

Stephen

[c0de_v0id]

@siwalters& neuron2
divx3 may not be the best but ppl(at least in fansubs) still use it to keep the leechers happy. especially with xvid instability and leechers' stupidity ( they cant even install ffdshow right)
I use xvid for all my personal encodes btw.

@scmccarthy

How did you look at the sbc encodes? Did you open them in a hex viewer and read the fourcc codes?

No. I used avisynth info(). they are all in rgb colorspace.(unless info() is reporting wrong colorspaces ^^;; ) and their 4ccs are div3 (yes in hex viewer :) ).

here is what i did,
Source = small clip(xvid, yv12)
1) ndub, default settings, 1 pass, avisynth2.5 script with just avisource(clip)
2) ndub, default settings, 1 pass, avisynth2.5 script with avisource(clip) & converttoyuy2()
3) vdubmod, XviD default settings, 1 pass, avisynth2.5 script with just avisource(clip)
4) vdubmod, XviD default settings, 1 pass, avisynth2.5 script with avisource(clip) & converttoyuy2()
:edit: all using fast repack

Colorspace results: ( i used a 2 lines avs script with avisource and info() to check the colorspaces)
1)RGB
2)RGB
3)yv12
4)yv12 (??)
so questions to u guys,
for 1&2 the ndub only operate in RGB?
for 4, how come the output is in yv12? (codec converted it?)

I dont think there is away to find out colorspaces with hex viewer. is there?

@sung
whatchu encoding now man, i miss the old days hehe

[scmccarthy]

@c0de_v0id

I will posit a thought: If you use converttoyuy2() with xvid, you are forcing xvid to convert the YUY2 to YV12 in the encode. It will get stored in the same color space regardless of what you feed it.

Stephen

[sungey]

Originally posted by ^^-+I4004+-^^
don't know bout yv12,but i can say this:
mjpeg avi->avs205->ND(fastrepack)->ok...
so it's YUY2 input converted directly to yv12...
(off course,fastrepack means avs must do all the processing,and no
antishit...)
but that's no big news....

i think there was some talk on forum on this issue of yv12->yv12
ND.......(or just previewing it?)

oh .. it seems like anti-shit causes the colorspace to be converted into RGB32 ... damn anti-shit is important for me though .....

oh well, i seldom use divx3 nowadays though ... just xvid :)

@ c0de_v0id
just the same series as usual .... where have u been lol ..

[digitize]

@sung
hey there, used to talking to you on irc =P
anyway, use ffvfw if you're forced to use divx3, it can encode in divx3 compatible streams. Gives much better results, there is a thread for it in the xvid forum, take a look.

[sungey]

yaa i tried ffvfw 3 months ago IIRC (not sure if theres a newer build) but unfortunately the divx3 encode made with it doesnt work with Divx5, which like 80% of the leechers has installed on their pc.

Divx5 is prioritized higher than divx3 in Directshow (We can change that). Telling leechers to use Registry editor is too complicated and making a Regfile, sometimes work .. sometimes it doesnt. We can use ffdshow. Leechers only has DX50, DivX3 codec ... and Xvid (if they dont get playback jerkiness).. also ffdshow ( not many has this, they say "i cant install this blablabla", wth :( ) ..... anyway ffmpeg is cool ... :)

[digitize]

@sung
quick and easy solution to that, make the fourcc mpeg4 v3 or what not. It'll still be a divx encode basically, just with a different fourcc, and it will not get decoded by divx5, and most if not just about nearly all leechers/people in general should be able to decode mpeg4 v3.

[sungey]

yaa i actually did that once ..the encode was made publicly available and .. surprisingly there are people who
doesnt has mpg4-v3 installed on their pc ... no kidding :(
i had to deal with numerous priv msg on irc saying "waht codec is this? " .. oh man .... that was horrible ... :(

btw... divx3 encoding using ffvfw isnt 100% pixel-flip free .... :( (subtitle-block, whatever u call it)

[digitize]

It's probably better than using nandub... =P

Ermmmm, edit: it is better than using nandub

[sungey]

hell yeah ... :)

[aranaxon]

I feel bad for you fansubbers and the hordes of leechers in your chans :)

In any case, thanks for the codec Marc FD, its quite useful in terms of saving when i have heavy duty filters going.

[digitize]

Originally posted by aranaxon
I feel bad for you fansubbers and the hordes of leechers in your chans :)

and what exactly do you mean by that.....

[ohliuv]

Originally posted by -h
... and I want to release both this and the MSMPEG4V3->MPEG4 convertor ...

anything more for the MSMPEG4V3->MPEG4 ? anyone?

[sh0dan]

@ohliuv: Please stay on topic!

@others: I'm sure -h will release it when it's done! You are not doing anything but annoy him, by constantly demanding him to release it. If you want it so bad, pick up a C-compiler and do the modifications yourselves (as Marc did!)

[aranaxon]

Originally posted by digitize
and what exactly do you mean by that.....

It seems like a relatively thankless job *shrug*

but hey, its a good service for the masses, and its what you like to do.

[litz]

Tried marc's VBLE codec on a Nvidia GF200 capture input, get 1 frame, 1 dropped frame, repeated for the length of the capture.

Almost like it's capturing at 15fps and duping frames. :confused:

Any ideas?

Would like to be able to capture directly to YV12, but with Marc's codec the only one out so far (-h ?) I haven't had much luck so far.

thankx ...

- litz

[mf]

Any idea why this (http://lsoust.hypermart.net/artifact.png) would happen ? This is 1 of 3 setbacks to a super-high-definition version of The Matrix Reloaded trailer and I think I'm gonna give up. AVISynth access violates (hung system), I'm having problem with splitting/appending my broken encodes, and then there's this. These are all great tools but they really don't seem high-definition resolution ready. No offense to anyone.

[sh0dan]

@mf: Could tou describe your problems more in detail - someone might even be able to help you out then. Keeping them to yourself never solve any problems.

[mf]

Originally posted by sh0dan
@mf: Could tou describe your problems more in detail - someone might even be able to help you out then. Keeping them to yourself never solve any problems.
Well, this is in the VBLE thread, so I'm compressing to VBLE. As you can see from the image I posted, the frame size is 1600x864. That's about all the relevant info. The irrelevant info: this is a transcode of the Matrix Reloaded Trailer at 1000x540 from Quicktime to VBLE using TMPGEnc, processed in AVISynth (neither of these things mentioned show this problem), and then encoded to VBLE again. All processing (except maybe the TMPGEnc conversion) is done in YV12. The problem seems to come from the extreme resolution. I don't see a good workaround for it, since AVISynth filtering is already down to 1 frame every 5 seconds.
Oh, and some relevant info I just remembered: if I re-encode the part with the artifact seperately I don't get it, but while appending I run into the VDub problem, which I'm talking to Suiryc now about. Nuff detail now ? :)

Edit: I found the split/append problem, I was splitting at the wrong frame but couldn't see it because Vdub wasn't showing me frame 0. I will now try to fix all the artifacts by means of splitting/appending until I have a clean stream.

[Marc FD]

maybe a error in the codec, but i don't see at all why it would lead to that.

i good idea might be to use smaller and smaller resolutions to see when the bug occurs (how big the resolutions has to be - if it's resolution related)

hope it'ld help.

Marc

[mf]

Originally posted by Marc FD
maybe a error in the codec, but i don't see at all why it would lead to that.

i good idea might be to use smaller and smaller resolutions to see when the bug occurs (how big the resolutions has to be - if it's resolution related)

hope it'ld help.

Marc
Thanks for being around even though you're busy leeching anime :p. I think the problem is memory related, I think one of the 3 (VBLE, AVISynth(and/or plugins), or VdubMod) is having memory leaks or corruptions or whatever (dunno too much about that stuff) causing a total system hang after some time. I really don't know, the errors seem to appear randomly. I can't recreate the problem at a certain frame either. I'm now filtering again (I managed to append everything together properly), so we'll see what results I'll get this time.

[cweb]

Marc,

Tried your codec and it's the best lossless capture codec so far.
I get no dropped frames with it. With the others I always got a small amount (nothing to worry about), but with yours, no drops!

[Valky]

Well, I haven't never captured with Huffy cause it takes too much space from my opinion. But in a little test-sample (3min43sec) when I tried with already encoded file, this Marc's codec gave me 383mb file comparing to huffy's 564mb file so if this is really lossless codec then I am very happy if I can replace my good old huffy with this one.

It saves lot of space,cause I always edit my captured files in VD when cutting out commercials and then save this edited file to one clean avi-file with de-noised audio file.

Thank you for that.

[N_F]

Originally posted by Zarxrax
Hrmm, anyone know what happened to -h? It was weird cause he said he would try and have it finished in like a day, then hasn't posted again for 1-2 weeks. I hope nothing has happened to him :eek:
I've seen a lot of people just suddenly up and disappear off the face of the internet, and it never gets out of the back of my head that these people could be dead or something :scared:
Make that ~11 weeks :(

[Sigmatador]

@MF
i did exactly what you did except my avs script resized the source in 640x272 (from original 1000x540) and i get no artefact.

[bkam]

Just wanted to thank you a ton... I started capturing only last week, and it is so convenient to be able to adapt what I know of encoding from DVDs to capture since now the colorspace is the same. With HuffYUV I was relearning everything and unable to use my YV12 filters (I guess i could have converted, but that's really slow, right? never tried). Also, the first time I used this one I got about maybe 300 frame drops for a 30 minute cap, the second time I messed with some windows settings and got ZERO. Awesome. Great codec, keep up the good work.

[BoNz1]

Well, it seems the attachments are down and I don't want to bother anybody. But, I just recently formatted my computer and I would like to have this modification of huffyuv again but I can't seem to find any mirrors or anything of it anywhere. marcfd doesn't seem to have it on his site either. Maybe someone would be kind enough to find a mirror for me or possibly even mirror it for me. At any rate, I would be very grateful. Thanks in advance.

[Acaila]

This (http://forum.doom9.org/showthread.php?s=&threadid=53305&highlight=loco) thread has the codec attached as well.

[BoNz1]

Thank you very much Acaila. :)

[SILICON]

Y'm look around for lostless codes for YUV12.
I found :

loco004 by TheRealMoh:
http://forum.doom9.org/attachment.php?s=&postid=312334

VBLE by ????
http://www.cc.jyu.fi/~camneely/vble_video_codec.zip

I read about Mark_fd HuffYUV, but i don´t see any link.

Cand anybody post links for Mark_fd HuffYUV and another codecs?

Thanks a lot.

[Wilbert]

VBLE = Mark_fd HuffYUV

The other links can be found at:

http://www.avisynth.org/index.php?page=Section+1%3A+About+AviSynth#q1.26

and

http://forum.doom9.org/showthread.php?s=&threadid=53305&highlight=VBLE

[Marc FD]

hi,

i played with huffyuv some days ago, i added some yv12 support, tried to add new tables and maybe other stuff (prefiltering for near lossless encoding), if you want to test some alpha stuff (bin only) i can post the stuff here.

[sh0dan]

@Marc FD: Sorry - no exceptions - source HAS to be supplied - otherwise you are violating GPL. Source must be downloadable "from the same place" as the binary - it's in the GPL. I hope you still find it worth the efford to release it!

[Boulder]

And while you're at it, you could consider releasing the sources of your other filters too (if they are GPL). Some people have had problems with some of your filters and the sources would help the other experienced coders to figure out what's causing the issues.

[Marc FD]

lol, no pb i will simply keep it here then :/

i've exams and no time to clean the code, and i hate to release dirty code... that's how i am.

and for my filters, they are not GPL until i'll release the source (under GPL) that's as simple as that.

i've not enough time to code, so i'm not gonna spend time on that stuff ^^;

for huffyuv my work was mean as clean as possible (okay looks like a hack on some aspects ^^) and it was pretty easy to do, so i think i'll release with code when i would have finished beta-testing, it's sad that the guy who did it is gone, i would prefer to give him the code directly. btw i don't think it's compatible at all with ffmpeg's yv12 huffyuv, but it seemed the faster and simple way to implement it :]

bye,
marc

[mf]

Originally posted by Marc FD
i've exams and no time to clean the code, and i hate to release dirty code... that's how i am.

and for my filters, they are not GPL until i'll release the source (under GPL) that's as simple as that.
Dirty code is at least code. If it's out there it's just so much more social for ppl who're having problems with it. I am working on a player mimicking Sasami2K (http://www.sasami2k.com) (planning stage, see my sig :D), because the original has bugs, the author disappeared (has no time for coding anymore because he has a job), and no source was ever released :(. Luckily there is MPC to use as a basis, but still there is going to be some duplicate work. Please consider this, nobody cares about how messy your code is, if ppl want to work on it then they'll accept that. And I think that's a pretty big compliment to a coder.

[Si]

@MarcFD
While people are actively developing GPL code, others don't mind if the source is withheld for a short while although this is against the GPL.

But if a GPL binary is not being actively developed then the GPL source code should be released no matter what the state.

Its how we got to the current position with all the GPL programs available (such as Avisynth itself).

Your only choices are to follow the copyright you agreed to when you released your modfied Huffyuv codec or to violate it.

regards

Simon

[outlyer]

Originally posted by siwalters
@MarcFD
While people are actively developing GPL code, others don't mind if the source is withheld for a short while although this is against the GPL.

But if a GPL binary is not being actively developed then the GPL source code should be released no matter what the state.

Its how we got to the current position with all the GPL programs available (such as Avisynth itself).

Your only choices are to follow the copyright you agreed to when you released your modfied Huffyuv codec or to violate it.

This doesn't seems totally correct reading the FAQs at gnu's site.
As I understood it, you don't need to publish the code as long as you're using it privately so, at the initial development stages, even if it gets stalled, there's no need to publish it.

Of course, if he abandons development, it would be nice to publish the code.

[Si]

As I understood it, you don't need to publish the code as long as you're using it privately so, at the initial development stages, even if it gets stalled, there's no need to publish it.

Agreed.

My comments only apply to published binaries derived from GPL works.

regards
Simon

[MfA]

Originally posted by Marc FD
and for my filters, they are not GPL until i'll release the source (under GPL) that's as simple as that.

So we have an implicit license to incorporate any and all of your code in closed source programs then? :) Anything else would be hypocritical now.

[sh0dan]

Stop this off-topic discussion now! If any software is announced here, that violates any licenses it is in violation of rule 6, and will be treated as such. If you need to discuss moral issues, do it in the general disussion forum!

[Longinus]

I'm having a strange BUG using VBLE. I looked around for any msgs talking about it, but I didn't find any... so I'm posting this..

I'm getting a very strange yellow vertical bar (about 1 pixel long) in the left side of the video. This happens only when I open it in virtualdub. Playing in MP is normal. Some video programs show a white bar, instead of yellow (combustion 2).

Another thing that helps is using an Avisynth script to convert to yuv2 or rgb. But this is probably time-consuming and reduces quality.

I've checked in VirtualDub, and it is using the right decoder, VBLE, so what can it be?

Thanks.

[Kurosu]

That's a very well-known bug in XviD YV12->RGB conversion. You see the line, but it isn't actually there in the raw (YV12) data: if you crop on the left, it will still be there! I think it was fixed, and only available in newest "alpha" builds (less than 2 weeks) of XviD. Or maybe it still isn't yet available. In the end, that's not a problem with VBLE. (Very Buggy/Beta Lossless Encoder? :D)

Offtopic, the more and more I read this thread, the more I think it would fit more in the "New A/V codecs" forums (seeing how VBLE is refered to by pointing at this thread).

[Koepi]

My latest XviD build (26062003-1) incorporates sysKin's fix of the color space conversion. Download & install it, the yellow line will be gone.

Regards
Koepi

[AmiRage]

Originally posted by Koepi
My latest XviD build (26062003-1) incorporates sysKin's fix of the color space conversion. Download & install it, the yellow line will be gone.
You mean 24062003-1? Or where can I find this newer build?

[Koepi]

Sorry, of course i typed wrong there. XviD-24062003-1.exe is my latest build.

Regards
Koepi

[Longinus]

Yep, this helps. Thanks!!

This post, is somewhat off-topic, like kuroso said... But I didn't find anywhere else to ask VBLE questions.

By the way, VBLE is a nice codec. I was using huffyuv to save some rendered 3D animation. But with vble I get the almost exact quality (a little bit darker), and sometimes half the size of Huffyuv. Thanks for it Marc.

[maciek_m]

Same impressions like Longinus. Good job

However, is there a newer version of VBLE (the beta version I downloded has been out for several months now).

And, perhaps a newbie's, problem:
I can't access the config option when selecting this codec in VDub(Mod) in capture mode.

VDub does not let me set the YV12 4:2:0 in the "set custom format". So I have to use YUY2 4:2:2.

Perhaps it has something to do with my capture device Radeon 9000 VIVO?

[Wilbert]

However, is there a newer version of VBLE (the beta version I downloded has been out for several months now).
Nope. The coder is gone, and the source is not available :(

VDub does not let me set the YV12 4:2:0 in the "set custom format". So I have to use YUY2 4:2:2.
You don't want to capture in YV12 with vble anyway, since it has no interlace option. Meaning it assumes YV12 progressive, and you will get chroma upsampling artefacts.

btw, it implies that your capture card can't do the yuy2->yv12 conversion, and hence can't output yv12.

[bill_baroud]

Originally posted by Wilbert
Nope. The coder is gone, and the source is not available :(


i think he's still lurking around here (http://atlas2.tgv.net/~media-video/forum2/portal.php)

dunno if he's still coding or anything, but he could perhaps release the source code :)

[mf]

I guess he'll just use the age-old excuse that his code is too dirty or whatnot to release it. Afaik he's only released the source to 2 of all his filters, so I wouldn't really count on it. Quite unnecessary, cause his filters are quite great actually (oxymoron? quite = relative, great = absolute), and the sourcecode can't be too bad.