huffyuv --> x264 lossl. = broken?

[aand]

http://img241.imageshack.us/img241/818/snapshot20081001132225mz0.jpg
This happens when I use a huffyuv avi and x264 lossless compression. If I use CRF there's no artefacts.
Job commandline: "C:\Program Files (x86)\megui\tools\x264\x264.exe" --qp 0 --nf --subme 1 --partitions p8x8,b8x8,i4x4 --merange 12 --threads auto --thread-input --sar 1:1 --progress --no-psnr --no-ssim --output "E:\EM 3 Derain GradFunk.mkv" "E:\EM 3 Derain GradFunk.avs"
I'm using Skystrife's build 994.
The script:
setmemorymax(400)
aviSource("D:\temp2.avi", audio=false).converttoyv12()
Derainbow()
GradFunkMirror(strength=3)

[nm]

Are you using CoreAVC to decode?

http://git.videolan.org/?p=x264.git;a=commit;h=b35a044b95c7eab2c91f55a3ac4100ca26a29d92:
Replace High 4:4:4 profile lossless with High 4:4:4 Predictive.
This improves lossless compression by about 4-25% depending on source.
The benefit is generally higher for intra-only compression.
Also add support for 8x8dct and i8x8 blocks in lossless mode; this improves compression very slightly.
In some rare cases 8x8dct can hurt compression in lossless mode, but its usually helpful, albeit marginally.
Note that 8x8dct is only available with CABAC as it is never useful with CAVLC.
High 4:4:4 Predictive replaced the previous profile in a 2007 revision to the H.264 standard.
The only known compliant decoder for this profile is the latest version of CoreAVC.
As I write this, JM does not actually correctly decode this profile.
Hopefully this lack of support will soon change with this commit, as x264 will be (to my knowledge) the first compliant encoder.

[aand]

Are you saying I should? (I didn't understand almost anything from the list)
I was using ffdshow.

OK, I've tried CoreAVC and it looks ok (the weird artefacts are gone) but I don't like the colours, everything is too dark.

I don't understand why ffdshow's libavcodec wouldn't work now. What's different? AVC lossless--->AVC lossless works OK.

[Keiyakusha]

aand
One lossless has been replaced with another. For now, only last version of CoreAVC supports it.
what is not clear?

[nm]

OK, I've tried CoreAVC and it looks ok (the weird artefacts are gone) but I don't like the colours, everything is too dark.
CoreAVC allows you to manually set PC/TV level conversions if necessary.

[Gabriel_Bouvigne]

FFMpeg is not yet compliant with this revision of the h.264 standard (the previous H4:4:4 has been deprecated, ie it doesn't exist anymore in the standard)

[Dark Shikari]

And the first complaints appear!

/waits eagerly for ffmpeg to support Predictive Lossless :p

[Inventive Software]

So is Lossless 4:2:0 still around then? (i.e x264 -q0)

[Dark Shikari]

So is Lossless 4:2:0 still around then? (i.e x264 -q0)x264 has never supported any colorspace other than 4:2:0, and probably won't for a long time.

[Inventive Software]

So, forgive the ignorance, but what's the difference between 4:2:0 and Predictive 4:4:4?

[Sagekilla]

x264 doesn't actually have lossless predictive 4:4:4 I believe, just 4:2:0. Only difference is we use 12 bits instead of 24. (YV12 vs YV24, in other words)

[Inventive Software]

x264 doesn't actually have lossless predictive 4:4:4 I believe, just 4:2:0. Only difference is we use 12 bits instead of 24. (YV12 vs YV24, in other words)

Um, then what's this?

Replace High 4:4:4 profile lossless with High 4:4:4 Predictive.

This improves lossless compression by about 4-25% depending on source.

The benefit is generally higher for intra-only compression.

Also add support for 8x8dct and i8x8 blocks in lossless mode; this improves compression very slightly.

In some rare cases 8x8dct can hurt compression in lossless mode, but its usually helpful, albeit marginally.

Note that 8x8dct is only available with CABAC as it is never useful with CAVLC.

High 4:4:4 Predictive replaced the previous profile in a 2007 revision to the H.264 standard.

The only known compliant decoder for this profile is the latest version of CoreAVC.

As I write this, JM does not actually correctly decode this profile.

Hopefully this lack of support will soon change with this commit, as x264 will be (to my knowledge) the first compliant encoder.

[Sagekilla]

Dark Shikari's post:

x264 has never supported any colorspace other than 4:2:0, and probably won't for a long time.

I -believe- we don't actually use 4:4:4 precision but we do use predictive lossless.

[LoRd_MuldeR]

I -believe- we don't actually use 4:4:4 precision but we do use predictive lossless.

That's my interpretation as well...

[Dark Shikari]

High 4:4:4 is the name of the profile. It allows any colorspace from 4:0:0 to 4:4:4. x264 uses 4:2:0.

[CruNcher]

lol funny marketing gag you can advertise with High 4:4:4 but only support 4:2:0 hehe i guess it isn't really allowed in the commercial world though as it would be highly misleading, or is it ?

[Dark Shikari]

lol funny marketing gag you can advertise with High 4:4:4 but only support 4:2:0 hehe i guess it isn't really allowed in the commercial world though as it would be highly misleading, or is it ?What, am I supposed to name profiles that don't exist?

"High 4:4:4" was the old profile that allowed lossless.

No other profile allowed lossless.

"High 4:4:4 Predictive" is the new profile that allows lossless.

No other profile allows lossless.

Do you want me to lie or something, will that make you happy? "x264 supports High 4:2:0 Predictive Profile." There, I just lied, is that fine with you now?

[Inventive Software]

Clarity would be helpful, since specifically applying that change, with that commit comment, assumes complete 4:4:4 Lossless encoding support in x264, which is incorrect, since x264 only supports 4:2:0. An average joe who wants to encode losslessly with x264 would assume, with that change, he gets 4:4:4 support, which is incorrect; this is without knowing what the spec profile allows.

What chances are there of x264 supporting other different YUV colourspaces?

[Dark Shikari]

Clarity would be helpful, since specifically applying that change, with that commit comment, assumes complete 4:4:4 Lossless encoding support in x264, which is incorrect, since x264 only supports 4:2:0. What chances are there of x264 supporting other different YUV colourspaces?Maybe clarity would help, but just given the size of the commit its rather obvious it doesn't add 4:4:4 support :p

Alternate colorspace support would be nice, but would require all sorts of modifications across the code, mainly changing constant shifts to non-constant ones. It would be really really really annoying to do. I'd like it, though. Another problem, also, is that libavcodec doesn't support anything other than 4:2:0 either.

[Inventive Software]

Maybe clarity would help, but just given the size of the commit its rather obvious it doesn't add 4:4:4 support :p
I take things at face value (yeah, dumb, but it shows some shortcomings by other people). That commit made me think x264 had lossless 4:4:4 encoding, which from previous experience and reading would mean a very major rewrite of x264's internal workings.

Alternate colorspace support would be nice, but would require all sorts of modifications across the code, mainly changing constant shifts to non-constant ones. It would be really really really annoying to do. I'd like it, though. Another problem, also, is that libavcodec doesn't support anything other than 4:2:0 either.
When one has new support, the other is usually quick to follow. ;) I'd love to say "hey, why don't I work on that", but it would require about 2 or 3 years more understanding of not just x264 but also colourspaces in general, and efficient programming. All 3 take time. :D

[Dark Shikari]

Another problem: Avisynth doesn't support YV24 colorspace, does it?

[Sagekilla]

I think the best Avisynth supports right now is YUY2. You'd need to write a custom filter to output YV24, but then that would mean either A) You have to work in subsampled YUY2 still or B) You have to work with RGB24/32, which isn't much better either I think.


If you were to add support for other colorspaces, wouldn't it become easier to support more once you add the first one?

[LoRd_MuldeR]

Maybe x264 could accept RGB24/RGB32 then and do the required conversion internally? Resolves the colorspace problem on the source-side, once and for all...

[Sagekilla]

Except if we accepted RGB24/32 and converted it to YV12 then it wouldn't be any different still. You'd just move the RGB --> YUV conversion from avisynth to x264.

What we want is to be able to have the full 4:4:4 precision available, not just 4:2:0.

[LoRd_MuldeR]

Wouldn't it be possible to convert from RGB32 to YV24 for 4:4:4 mode? :confused:

[Dark Shikari]

Maybe x264 could accept RGB24/RGB32 then and do the required conversion internally? Resolves the colorspace problem on the source-side, once and for all...Akupenguin has said that colorspace conversion has no place in an encoder.

[LoRd_MuldeR]

Akupenguin has said that colorspace conversion has no place in an encoder.

Right, but ...

[Sagekilla]

But if we did RGB32 --> YV24 conversion inside of x264, what would that solve? That just means x264 gets a YV24 input, and it'd end up throwing an error message saying "unsupported color space" Problem unsolved :(

[CruNcher]

What, am I supposed to name profiles that don't exist?

"High 4:4:4" was the old profile that allowed lossless.

No other profile allowed lossless.

"High 4:4:4 Predictive" is the new profile that allows lossless.

No other profile allows lossless.

Do you want me to lie or something, will that make you happy? "x264 supports High 4:2:0 Predictive Profile." There, I just lied, is that fine with you now?

No surely i didn't want that you lie the other way is the case, more appropriate would have been "High 4:4:4 Predictive (only 4:2:0 currently supported)" :)

[Sagekilla]

I think it would be good to add a disclaimer: "x264 only supports 4:2:0"

[kemuri-_9]

Another problem: Avisynth doesn't support YV24 colorspace, does it?

Avisynth 2.6 and 3.0 both have YV24 support afaik, though they are in various levels of incomplete development