Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 27th June 2011, 01:08   #1  |  Link
xv
Registered User
 
Join Date: Mar 2010
Posts: 98
ConvertToYCgCo - New Colorspace

Iīve created a plug-in for conversion to the YCgCo colorspace.

This colorspace is not derived from PAL/NTSC, but designed for better compressibilty. Iīve made a test with lossless x264 encoding, I tested just a few short samples, but for them YCgCo gives 10% less bitrate than normal bt709/bt601. Quality is of course the same (except some rounding if your source is actually bt709/bt601, for RGB source rounding errors are smaller, because RGB fits better in YCgCo).

Unfortunately most application cannot display this colorspace at the moment (madVR as of version 0.82.5 displays YCgCo correctly, older versions since 0.68 supported it, but is was partly broken and resulted in slightly wrong colors). Most programs will just use bt709/bt601 matrix to decode (like VirtualDub, VLC, any DirectShow-based player without madVR renderer) and colors will be awfully wrong.

If you encode to H.264 using x264 ensure you are telling the encoder to set the correct flags in the H.264 bitstream:
Code:
--colormatrix YCgCo --range pc --input-range pc
Latest version: 0.4 (link to post with download)
Version in this post is outdated, please do not use!
Attached Files
File Type: zip ConvertToYCgCo.zip (30.7 KB, 140 views)

Last edited by xv; 15th March 2012 at 01:24. Reason: version 0.4
xv is offline   Reply With Quote
Old 27th June 2011, 01:32   #2  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,865
Can you give the translation matrix?
Some more info on the theory would be interesting.
jmac698 is offline   Reply With Quote
Old 27th June 2011, 01:50   #3  |  Link
xv
Registered User
 
Join Date: Mar 2010
Posts: 98
The conversion formula:

Y = 0.25 * R + 0.5 * G + 0.25 * B
Cg = -0.25 * R + 0.5 * G - 0.25 * B + 0.5
Co = 0.5 * R - 0.5 * B + 0.5

The 0.5 offset is for [0,1] range, for [0,255] range (8 Bit) itīs 128.
This conversion formula is specified in the H.264 spec (in E.2.1 VUI Parameter Semantics, page 379 in 3/2010 standard).
xv is offline   Reply With Quote
Old 27th June 2011, 15:57   #4  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,865
Ok, that's Chroma Green and Chroma Orange, and they need 9bits to maintain conversion accuracy. They have low mutual correlation. This can be decoded on playback, if we make a script to convert it (with masktools) and load the script in ffdshow. Do you have the formula to go back to rgb?
jmac698 is offline   Reply With Quote
Old 27th June 2011, 16:47   #5  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,412
Quote:
Originally Posted by jmac698 View Post
Do you have the formula to go back to rgb?
Don't you know how to do matrix inversion (or solve simultaneous equations)?

For Cg and Co in [-0.5, 0.5]
R = Y - Cg + Co
G = Y + Cg
B = Y - Cg - Co

For example, see http://de.wikipedia.org/wiki/YCgCo-Farbmodell.
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 27th June 2011, 16:50   #6  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,865
Yes I know, I was lazy
Another sample pic
https://secure.wikimedia.org/wikiped...-1600x4800.jpg
jmac698 is offline   Reply With Quote
Old 27th June 2011, 17:53   #7  |  Link
xv
Registered User
 
Join Date: Mar 2010
Posts: 98
It would not be that difficult to write a plug-in that converts YCgCo back to RGB, but that wouldnīt be more than a testing solution for playback. Hopefully in the next madVR version we have support and automatic detection from H.264 bitstream.

There is another formula for YCgCo-R that needs 9 Bit for chroma and 8 Bit luma and is fully reversable (for 8 Bit source). Unfortunately x264 does not support different luma/chroma bitdepths (yet?). Also there is no colorspace with 8 Bit luma and 9 Bit chroma, conversion would have to be integrated in encoder/decoder and input/output would be RGB.

The interesting question is how does it perform visually with chroma subsampling.
xv is offline   Reply With Quote
Old 27th June 2011, 18:31   #8  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,865
We can experiment with all this in script:
Code:
#RGB<=>YCgCo demo by jmac, requires masktools v2a38+, tested in avisynth 2.58
#Ver 0.1 - does something, but incorrectly - almost finished
#ImageReader("E:\project001a\YCgCo\YCgCo-1600x4800.jpg")#Sample picture stacked as RGB,Y,Cg,Co (Cg and Co are colourized for illustration)
colorbars
rgb=verticalunstack4(0)
y=verticalunstack4(1)
cg=verticalunstack4(2)
co=verticalunstack4(3)
#Testing...
RGBintoYV12(colorbars)
YV12intoRGB
RGB2YCgCo
#for realtime playback in ffdshow, erase all lines above
YCgCo2RGB

function verticalunstack4(clip v, int slice) {
#Return n'th vertical slice of n slices, e.g. 4 videos stackedvertically (opposite of stackvertical)
#note slice=0 is the top block
slices=4
v
blockheight=height/slices
select (slice, \
    crop(0,0,0,-3*blockheight), \
    crop(0,blockheight,0,-2*blockheight), \
    crop(0,2*blockheight,0,blockheight), \
    crop(0,3*blockheight,0,0))
}

function RGB2YCgCo(clip v) {
#convert an assumed RGB clip into Co,Y,Cg
#~ Y = 0.25 * R + 0.5 * G + 0.25 * B
#~ Cg = -0.25 * R + 0.5 * G - 0.25 * B + 0.5
#~ Co = 0.5 * R - 0.5 * B + 0.5
v
#put planes into yuv for masktools (awkward)
RGBintoYV12
#Note: remember b=u, r=v, g=y
mt_lut(yexpr="v 4 / y 2 / + u 4 / +",uexpr="v 2 / u 2 / + 128 +",vexpr="v -4 / y 2 / + u -4 / 128 +")
YV12intoRGB
}

function YCgCo2RGB(clip v) {
#convert an assumed  Co,Y,Cg clip into RGB
#~ R = Y - Cg + Co
#~ G = Y + Cg
#~ B = Y - Cg - Co
v
#put planes into yuv for masktools (awkward)
RGBintoYV12
#Note: remember b=u, r=v, g=y
mt_lut(yexpr="y u +",uexpr="y u - v -",vexpr="y u - v +")
YV12intoRGB
}

function RGBintoYV12(clip v) {
#place r,g,b data directly into v,y,u planes
#this result has no visual meaning (however, closest approximation) but is only for performing arithmetic on the data
v
pointresize(width*2,height*2)#doublesize to allow full 4:4:4 color resolution with u,v
r=ShowRed.converttoyv12
g=ShowGreen.converttoyv12
g=g.pointresize(width*2,height*2)#this is destined for Y which must be larger
b=ShowBlue.converttoyv12
YtoUV(b,r,g)#b->u, r->v, g->y
}

function YV12intoRGB(clip v) {
#place v,y,u data into r,g,b planes
#this result has no visual meaning (however, closest approximation) but is only for performing arithmetic on the data
v
r=vtoy.converttorgb
g=greyscale.converttorgb
g=g.pointresize(width/2,height/2)#Y was doublesized
b=utoy.converttorgb
MergeRGB(r,g,b)#u->b, v->r, y->g
pointresize(width/2,height/2)#doublesize to allow full 4:4:4 color resolution with u,v
}
I'll update later to fix the bugs...
jmac698 is offline   Reply With Quote
Old 27th June 2011, 20:37   #9  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,412
Quote:
Originally Posted by jmac698 View Post
Code:
mt_lut(yexpr="v 4 / y 2 / + u 4 / +",uexpr="v 2 / u 2 / + 128 +",vexpr="v -4 / y 2 / + u -4 / 128 +")
That won't work. The only variable you can use in mt_lut is "x", which refers to Y in yexpr, U in uexpr and V in vexpr.
To combine values from all three channels, you have to use mt_lutxyz, with each channel in a separate clip.
I'm not sure negative constants (-4) are recognised either.

Quote:
Code:
select (slice, \
    crop(0,0,0,-3*blockheight), \
    crop(0,blockheight,0,-2*blockheight), \
    crop(0,2*blockheight,0,blockheight), \
    crop(0,3*blockheight,0,0))
This can be written more simply as
crop(0,slice*blockheight,0,blockheight)
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 28th June 2011, 08:43   #10  |  Link
xv
Registered User
 
Join Date: Mar 2010
Posts: 98
Itīs now more than one and my attachment is still not approved, so I uploaded it here:
http://www.mediafire.com/?d7ididp7o635xcl
Another plug-in is in work "ManualColorMatrix", where you can do any kind of matrix multiplication and offset correction of color triplets (but ConvertToYCgCo should be used for RGB->YCgCo because it will most likely be faster (uses SSE2)).
xv is offline   Reply With Quote
Old 28th June 2011, 12:26   #11  |  Link
pandy
Registered User
 
Join Date: Mar 2006
Posts: 1,044
Avisynth 2.58 please - 2.6 is still far from end, TIA!

Last edited by pandy; 28th June 2011 at 12:30.
pandy is offline   Reply With Quote
Old 28th June 2011, 12:43   #12  |  Link
xv
Registered User
 
Join Date: Mar 2010
Posts: 98
The plugin needs the YV24 colorspace (YUV without subsampling), which is unfortunately not supported by AviSynth before version 2.6, therefore no AviSynth 2.5 support .
xv is offline   Reply With Quote
Old 28th June 2011, 14:21   #13  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,865
Why does it need yv24 when it's an rgb operation? Rgb will work fine. My script works in 2.58 as well.
jmac698 is offline   Reply With Quote
Old 28th June 2011, 14:30   #14  |  Link
xv
Registered User
 
Join Date: Mar 2010
Posts: 98
It does not make much sense to store the YCgCo data as RGB. You cannot use that RGB data, how do you tell other software that RGB is in fact YCgCo?
But like I said I will write another plugin for any matrix conversion that will support RGB/YV24 input/output.
xv is offline   Reply With Quote
Old 28th June 2011, 14:43   #15  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,865
That's a good question, but you have to realize that RGB is only a convention which has little meaning. The actual position of the bytes and what they contain is mostly what's important. I can stuff any data I want into any stream. The only question is how to make it useful with other programs. In our case the most useful thing I see is to feed it to x264 encoder. There's programs which pipe raw video data so we need to know what arrangement of bytes x264 would need. I can rearrange the pixels to get the data right, the color format makes no difference at all.

In my 10bit video importer, I opened the raw data of a quicktime file and rearranged the pixels to display a sensible picture. I then put the extra bits in another clip and wrote tools to manipulate levels, so it's actually useful.

Btw, please consider using the informal deepcolor format other authors use, where MSB is top half of picture and LSB on bottom half. By doing this we can then adjust levels and dither back down to 8 bit with other plugins.

We need an internal format that you can manipulate with plugins and scripts, and an export from this format outside of avisynth to other formats.

Also YCgCo stuffed into G,B,R can still be compressed. It has to be displayed with my script for realtime playback. I also put it into yv12 by doublesizing the video, but I can also subsample the CgCo. Lack of yv24 is not a problem for processing.

Try the script - you can play with subsampling or whatever you want to see the effects.

Last edited by jmac698; 28th June 2011 at 14:48.
jmac698 is offline   Reply With Quote
Old 28th June 2011, 15:47   #16  |  Link
pandy
Registered User
 
Join Date: Mar 2006
Posts: 1,044
Is there any reason to not implement YV12 sub-sampling? for YV24 RGB organization of data can be used...
pandy is offline   Reply With Quote
Old 28th June 2011, 16:42   #17  |  Link
xv
Registered User
 
Join Date: Mar 2010
Posts: 98
x264 for example accepts YV24 and can encode it correctly (with --colormatrix YCgCo --fullrange on). It cannot encode YCgCo in RGB correctly:
-It converts it to a YUV format assuming itīs RGB -> video will be a mess
-It encodes it as RGB without subsampling, but result will be suboptimal since it treats RGB different than YUV/YCgCo
Whatīs the problem with AviSynth 2.6? Any serious issue?

I see that there are many hacks to implement workarounds for missing features and I think thatīs great for testing and AviSynth internal stuff, but like you said outside of AviSynth that doesnīt work well. My goal was to get YCgCo out of AviSynth to feed the data to x264.
xv is offline   Reply With Quote
Old 28th June 2011, 16:52   #18  |  Link
xv
Registered User
 
Join Date: Mar 2010
Posts: 98
Quote:
Is there any reason to not implement YV12 sub-sampling? for YV24 RGB organization of data can be used...
The reason is quite simple, in AviSynth 2.6 thatīs useless, just use ConvertToYCgCo().ConvertToYV12()
Iīll see if I can yust copy the subsampling algorithm from AviSynth into my filter to allow direct subsampling and AviSynth 2.5 support.
xv is offline   Reply With Quote
Old 29th June 2011, 11:19   #19  |  Link
pandy
Registered User
 
Join Date: Mar 2006
Posts: 1,044
It was my point - to re-implement YV12 AviSynth internal filter by Your external YCoCg filter - this should create more flexible environment and more than only YV24 color space supported. Anyway thanks for all - even if i can't use Your filter now.
pandy is offline   Reply With Quote
Old 1st July 2011, 12:30   #20  |  Link
xv
Registered User
 
Join Date: Mar 2010
Posts: 98
Version 0.2 with chroma subsampling and AviSynth 2.5 support

Iīve added support for YUY2 and YV12 subsampling. Iīve done this by calling lanczos to resize U/V plane.
Iīm not sure thatīs correct, but I used src_left=-0.5 for YV12 to have correct chroma position, does anyone know if thatīs correct?

http://www.mediafire.com/?vzgm1po5a3630sw
Attached Files
File Type: zip ConvertToYCgCo_v02.zip (50.1 KB, 82 views)
xv is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 22:58.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.