ColorBars test pattern references

[jmac698]

Hello,
after extensive research, directly of the standards, I have found wrong colors in colorbars(). I have provided references and bmp's with correct values. More available at http://forum.doom9.org/showthread.php?t=107682&page=2 in case you hadn't looked.

[Wilbert]

For clarity, only the +4 IRE values is wrong, no?

[jmac698]

Sorry but all the colors are wrong, in every mode.
http://forum.doom9.org/showthread.php?p=794961#post794961 correct values, directly from table in standard
http://forum.doom9.org/showthread.php?p=795006#post795006 measurements of colorbars()

[Wilbert]

Ok, i didn't read the whole thread carefully :) As you notice, it's all due rounding errors. It will be corrected in the next version ...

[jmac698]

I know it's a lot to read, I have a feeling no one is reading it anymore, but after talking to myself a long time I reached a conclusion :)

[IanB]

As I noted it the Using thread, the core problem is the Adobe NTSCBARS.BMP which all is based on is slightly out of whack. We have compounded the problem by converting back to YUV and doubling the error due to rounding and truncation.

I do not have a problem tweaking the YUV values towards the provided values such that a single YUV[16..235] -> RGB[16..235] conversion return the original NTSCBARS.BMP results.

I am however reluctant to just correct the values used in the original RGB source without understanding where they came from. The errors are more than simple rounding errors! The Adobe BMP's for better or worse are a defacto standard in there own right.

As I also said in the using thread I have wanted to add PAL versions of the patterns. So I propose to leave the NTSCBARS.BMP as the default (with intelligent YUV rounding) and add mathematically correct NTSC and PAL versions of the appropriate patterns as options.

[jmac698]

First, I think you need to be updated. I've downloaded all the actual standards and read them.
Validation of research: I have no doubt my new values are correct. They come directly from a table, there is no room for interpretation, they match by calculation, they match by measurement, they match by two sources of direct listing. Download yourself at http://ecs.itu.ch/cgi-bin/ebookshop, Bt801-1.

Is colorbars(pixel_type = "RGB32") the basis for any conversions, therefore derived from Adobe?

I see no pattern in the errors. Theory: precompensated RGB for rgb->yuv? Otherwise, it's just wrong. Adobe was well known to be wrong in last version.

I *do* see a point to using defacto standards. My version is defacto too, was checked with professional equipment and matched exactly, yet defacto Adobe got known to be wrong, and ppl avoided it.

I suggest direct generation in yuv/rgb spaces.

PAL versions: There are several standards for testpatterns, not all of them are colorbars. A common one, is just the 7 colors, beginning with white, and 75% otherwise. This is valid for both NTSC and PAL. The tvout card automatically corrects the property called "setup".

mathematically correct NTSC and PAL versionsI am no longer advocating mathematically correct, I am simply repeating a table from the standard. There is no room for error. (And they are also mathematically correct).

[IanB]

Is colorbars(pixel_type = "RGB32") the basis for any conversions, therefore derived from Adobe?Yes, the original ColorBars() was apparently ripped from NTSCBARS.BMP, I do not know who originally wrote it, I guess must have been Ben Rudiak-Gould, it was in the original project Richard created source.cpp 1.1 (http://cvs.sourceforge.net/viewcvs.py/avisynth2/avisynth/Attic/source.cpp?rev=1.1&view=markup), Wilbert added the pixel_type="YUY2" fairly recently and directly calculated the YUV values from the existing RGB, I added the the "YV12" soon after.

Any new numbers will be referenced or calculated in the correct initial colour space and converted once to the other colour space.

Can you please concisely tabulate in one post in this thread all your relevant numbers with references urls and/or standard publication names with page numbers.

Also if you have stumbled across some rec.709 patterns in your research please include them as well.

[jmac698]

Rec. ITU-R BT.801-1, Annex 2, P13-15, http://www.itu.int/rec/R-REC-BT.801-1-199510-I/en
Rec. ITU-R BT.801-1
Description of encoded colour-bar signals according to the 4:2:2 level
of Recommendation ITU-R BT.601
100/0/75/0 colour bars
color Y Cb Cr
white 235 128 128
yellow 162 44 142
cyan 131 156 44
green 112 72 58
magenta 84 184 198
red 65 100 212
blue 35 212 114
black 16 128 128

Rec. ITU-R BT.1729, Page16, http://www.itu.int/rec/R-REC-BT.1729-0-200504-I/en
Description of encoded colour-bar signals according to the 4:2:2 level
of Recommendation ITU-R BT.601
100/0/100/0 colour bars
AND
Rec. ITU-R BT.1729
Appendix 2
100% colorbars
color Y Cb Cr
white 235 128 128
yellow 210 16 146
cyan 170 166 16
green 145 54 34
magenta 106 202 222
red 81 90 240
blue 41 240 110
black 16 128 128

Rec.709 colorbars are also there.

SMPTE 170M-1999, Page 7, http://www.smpte.org/smpte_store/standards/index.cfm?scope=0&stdtype=smpte&CurrentPage=5
White Level.....100IRE
Black (Setup) Level... 7.5IRE
Derivation, 92.5IRE=[16,235]

[IanB]

@jmac698,

Good post. Ta muchly!

Also can you please edit the name of this thread to be

ColorBars test pattern references

[hanfrunz]

i found another interesting EBU-document:

http://www.ebu.ch/CMSimages/en/tec_doc_t3305-2005_tcm6-37146.pdf

[jmac698]

EBU Tech 3305,Digital Television Test Pattern Sequence for
Operational Use, Page9, Pattern 2, http://www.ebu.ch/CMSimages/en/tec_doc_t3305-2005_tcm6-37146.pdf , Same as 100/0/75/0 colorbars.

Don't forget, register at http://ecs.itu.ch/cgi-bin/ebookshop to download ITU documents yourself. One of them even includes SMTPE1700 (BT.1700).

[Mug Funky]

just for fun, i wonder what would happen if i get one of the decks here (DVW-A500P) to output PAL colourbars and capture them to 8-bit uyvy? i wonder which spec it'd match. after all, Sony are also big fans of making their own standards.

btw, the avisynth bars look a lot better than those output from premiere pro (they're so wrong it's hilarious). so much so that i'm using colorbars() on heads of DV tapes that i've exported lately. at the very least, they look just like ones that come out of the decks, so rounding/whatever error must be small.

[jmac698]

I suggest to use my colorbars in
http://forum.doom9.org/showthread.php?p=795068#post795068
Avisynth's are inaccurate.
I think Adobe uses a special scaling that can show full Y range.

Please make your test and report back. You will have to use this script to measure YUV, load the script in virtualdub, copy source frame to clipboard, paste to a paint program, then use the dropper tool, and "pretend" G is Y, B is U, R is V.
Y=avisource("...")
U=Y.UToY().pointresize(720,480)#yv12 is scaled 2x, 2y
V=Y.VtoY().pointresize(720,480)#yuy2 is scaled 2x
Return (MergeRGB(V,Y,U))

I think you will see the standard Rec. ITU-R BT.1729, known as 100/100 full field.

[Wilbert]

avisource("...")
pointresize(720,480)
will do fine :)

[jmac698]

It would be fine if you want to measure RGB, but that involves a conversion. If you can read YUV, you can get the values originally stored in the cap. I'm using my script to copy YUV->GBR. Now it looks very strange, but at least you can measure the original YUV in a paint program.

[SeeMoreDigital]

Now it looks very strange, but at least you can measure the original YUV in a paint program.Indeed...

HyperSnap allows you to assign values of anything between 000-255 for red, green and blue. I've been using it to read and change the colour values of individual pixels for some time ;)

[Wilbert]

It would be fine if you want to measure RGB, but that involves a conversion. If you can read YUV, you can get the values originally stored in the cap. I'm using my script to copy YUV->GBR. Now it looks very strange, but at least you can measure the original YUV in a paint program.
Yes, it was a dumb remark. What I wanted to say was, why not use

AviSource(...)
Crop(...) # crop to a color bar
ColorYUV(analyze=true)

to read of the YUV values?

[jmac698]

Yes, that works too. It's the method used by hanfrunz recently to measure his colorbars. Personally I found it easier to use the first method than to figure out the correct crop commands at least 7 times.
Even better would be PixelInfo()

[Wilbert]

@jmac698,

I'm looking at BT.1729, but they only list 100/0/100/0 bars for Rec.709. I can calculate the 100/0/75/0 by hand, that's not the problem. But i was wondering why 100/0/75/0 bars is not listed there? Perhaps we should list it as an option to return the 100/0/75/0 or 100/0/100/0 bars?

[IanB]

The 100/0/75/0 style colour bars are a legacy of crappy domestic televisions, which if ever presented with 100% saturated colours invariable flared or bloomed (some even displayed wrong colours as some video signal path overloaded and distorted). So the industry conspired to limit saturation to 75% everywhere they could. The decision had a lot of financial merit in that acceptable performace for the consumer could be had using much simpler and cheaper circuitry. Remember discreet components ruled, IC were new fangled and fairly simple by todays standards.

Modern equipment using advanced integrated technology no longer needs this compromise and should be expected to correctly display 100% saturated colours. Hence Rec.709 and similar modern specification do not need to make compromises as in the past. 100/0/100/0 patterns for Rec.601 are intended for modern equipment running in the legacy colour space.

The only reason I can think of for having 75% saturation Rec.709 patterns would be to say we have one.

[jmac698]

Remember, hunfranz's Sony DVW-A500P had both 100/75 and 100/100, and this was PAL equipment.

The 75's are listed in BT.801-1 (hmm, but maybe not for 709 colorspace). Why not in Rec.709? Maybe, because 709 is for hdtv, and 75bars are obsolete.
75 colorbars reach 100IRE/1Vpp. 100 bars reach 131IRE/1.22Vpp (in NTSC composite). The only reason for 75 colorbars is to avoid the signal going beyond 100IRE/1vpp. I think Ian's explanation is reasonable.
It's probably just convient to read waveform monitors which are marked with a line at 100IRE.
One thing we could do it try to record them on a VCR and see if it can play back ok. I assume any other modern device will handle it.
On magnetic tape, too high a recording level will lead to nonlinearities and eventually, clipping.
The strongest argument I can give is
1 use 75 to maintain compatibilities with waveform monitors.
2 to be able to label the option with the appropriate standards, such as EIA-189-A or Rec601-1.
However I don't see 100's being superior for any particular reason.
I have no info on PAL. Why not ask hunfranz?
If I had a choice, I'd rather correct 709/601 than a choice of 75/100.

[jmac698]

Ian,
we've now agreed on the colors (almost, but -I and Q), but we've never discussed the small bars or proportions. I also have more info on the I, Q:

http://www.gekco.com/vidprmr.htm#SMPTE%20Bars
SMPTE Bars are split field bars composed of standard EIA 75% amplitude white bars for the top 2/3 of the field, reverse blue bars for the next 1/12 of the field, and the IYQB or “PLUGE” signal for the remainder of the field. This split-field arrangement allows adjustment of color saturation or color intensity and hue or tint on a color monitor that has the feature to allow the blue gun only. The monitor is set to blue only and the hue or phase is adjusted on the monitor until there is no discernable intensity difference between the reversed blue bars and their adjacent color bars. The IYQB section of the bottom pattern consists of a 7.5 IRE (black level) pedestal with a 40 IRE “-I” phase modulation, a 100 IRE white pulse, a 7.5 IRE (black level) pedestal with a 40 IRE “+Q” phase modulation, and a 7.5 IRE pedestal with 3.5 IRE, 7.5 IRE, and 11.5 IRE pedestals. -I and +Q phase modulation signals are helpful to assure the subcarrier processing in correct. PLUGE stands for (Picture Line-Up Generating Equipment). This pattern, at the bottom and to the right side of the SMPTE bars is used to set the brightness of the monitor. The monitor is adjusted so that the black and blacker than black areas are indistinquishable from each other and the lighter than black area is slightly lighter
This isn't a firsthand source, but we'll have to trust it for the proportions.
Also, at first it seems like an option, Rec.601 now and Rec. 709 for another version, but if you code it right, there's no difference in effort. I saw source code for colorbars, which first defined a palette of the used colors, then had a table for each kind of line, then the lines were draw according to the size needed.

bnb_start = height * 2 / 3;
pluge_start = height * 3 / 4;
stripe_width = (width + 6) / 7;
lineY = malloc(width * sizeof(lineY[0]));
lineCb = malloc(width * sizeof(lineCb[0]));
lineCr = malloc(width * sizeof(lineCr[0]));

/* Top: Rainbow */
for (i = 0, x = 0; i < 7; i++) {
for (w = 0; (w < stripe_width) && (x < width); w++, x++) {
lineY[x] = rainbow[0][i];
lineCb[x] = rainbow[1][i];
lineCr[x] = rainbow[2][i];
}
}
for (y = 0; y < bnb_start; y++) {
memcpy(Y, lineY, width);
memcpy(Cb, lineCb, width);
memcpy(Cr, lineCr, width);
Y += width;
Cb += width;
Cr += width;
}

/* Middle: Wobnair */
for (i = 0, x = 0; i < 7; i++) {
for (w = 0; (w < stripe_width) && (x < width); w++, x++) {
lineY[x] = wobnair[0][i];
lineCb[x] = wobnair[1][i];
lineCr[x] = wobnair[2][i];
}
}
for (; y < pluge_start; y++) {
memcpy(Y, lineY, width);
memcpy(Cb, lineCb, width);
memcpy(Cr, lineCr, width);
Y += width;
Cb += width;
Cr += width;
}

/* Bottom: PLUGE */
pl_width = 5 * stripe_width / 4;
/* -I patch */
for (x = 0; x < pl_width; x++) {
lineY[x] = i_pixel[0];
lineCb[x] = i_pixel[1];
lineCr[x] = i_pixel[2];
}
/* white */
for (; x < (2 * pl_width); x++) {
lineY[x] = white[0];
lineCb[x] = white[1];
lineCr[x] = white[2];
}
/* +Q patch */
for (; x < (3 * pl_width); x++) {
lineY[x] = q_pixel[0];
lineCb[x] = q_pixel[1];
lineCr[x] = q_pixel[2];
}
/* black */
for (; x < (5 * stripe_width); x++) {
lineY[x] = black[0];
lineCb[x] = black[1];
lineCr[x] = black[2];
}
/* (black - 4IRE) | black | (black + 4IRE) */
for (; x < (5 * stripe_width) + (stripe_width / 3); x++) {
lineY[x] = neg4IRE[0];
lineCb[x] = neg4IRE[1];
lineCr[x] = neg4IRE[2];
}
for (; x < (5 * stripe_width) + (2 * (stripe_width / 3)); x++) {
lineY[x] = black[0];
lineCb[x] = black[1];
lineCr[x] = black[2];
}
for (; x < (6 * stripe_width); x++) {
lineY[x] = pos4IRE[0];
lineCb[x] = pos4IRE[1];
lineCr[x] = pos4IRE[2];
}
/* black */
for (; x < width; x++) {
lineY[x] = black[0];
lineCb[x] = black[1];
lineCr[x] = black[2];
}
for (; y < height; y++) {
memcpy(Y, lineY, width);
memcpy(Cb, lineCb, width);
memcpy(Cr, lineCr, width);
Y += width;
Cb += width;
Cr += width;
}
free(lineY);
free(lineCb);
free(lineCr);
}

[IanB]

Modern equipment using advanced integrated technology ....From my straw poll of wandering thru the AV dept of a local department store, all Rec.709 related equipment is digital, i.e. DVD, Digital TV, PVR, HDMI, Component Video, etc.

Composite signals I would expect are alway Rec.601 related.

And my doesn't that code look a bit like the AVS source.:p

[Mug Funky]

...hunfranz's Sony DVW-A500P had both 100/75 and 100/100, and this was PAL equipment.

@ hanfrunz: how do you make it do that?? we have 2 here and only 1 can output bars and tone, and they're always (i think) 100/75. can't be sure because there's no (working) vectorscope in the office and i'm too lazy to capture a bit and measure :) is there a hard switch inside the deck somewhere that allows such things? well, it doesn't really matter. i'm not a great fan of these decks as one of them has a habit of not stopping when it's rewound a tape... *SNAP!... whirrrrrrRRRRRRR*

[hanfrunz]

use the super-output to see the MENU. The name of the testpattern is humanreadable then (MENU-ITEM 710). You have to activate the testpattern by holding the video-input-select-button for a few seconds.

[SeeMoreDigital]

I'm looking for a graduated grey scale test pattern ranging from 000/000/000 to 255/255/255. Five-to-ten pixels high and 255 pixels wide, ie: one pixel for each level of grey!

Can anyone oblige?


Cheers

[IanB]

Function Swipe(clip clip, int n) {
n=n+1
StackHorizontal(clip, BlankClip(clip, width=1, color=$010101*n))
return (n>=255)?Last:Swipe(Last, n)
}

Swipe(BlankClip(width=1, height=10, color=$000000, pixel_type="rgb32"), 0)

[jmac698]

That code came from AVS? I got it from mjpegtools. Who stole from who?

[SeeMoreDigital]

Thanks Ian,

Okay, that's the grey scale sorted but I've just got to ask....

How would the code be written for the separate red, green and blue scales?


Cheers

EDIT: You're my 7,000th post.... amazing!

[hanfrunz]

How would the code be written for the separate red, green and blue scales?


Function Swipe(clip clip, int n) {
n=n+1
StackHorizontal(clip, BlankClip(clip, width=1, color=$010101*n))
return (n>=255)?Last:Swipe(Last, n)
}

Function SwipeR(clip clip, int n) {
n=n+1
StackHorizontal(clip, BlankClip(clip, width=1, color=$010000*n))
return (n>=255)?Last:SwipeR(Last, n)
}

Function SwipeG(clip clip, int n) {
n=n+1
StackHorizontal(clip, BlankClip(clip, width=1, color=$000100*n))
return (n>=255)?Last:SwipeG(Last, n)
}

Function SwipeB(clip clip, int n) {
n=n+1
StackHorizontal(clip, BlankClip(clip, width=1, color=$000001*n))
return (n>=255)?Last:SwipeB(Last, n)
}




y=Swipe(BlankClip(width=1, height=10, color=$000000, pixel_type="rgb32"), 0)
r=SwipeR(BlankClip(width=1, height=10, color=$000000, pixel_type="rgb32"), 0)
g=SwipeG(BlankClip(width=1, height=10, color=$000000, pixel_type="rgb32"), 0)
b=SwipeB(BlankClip(width=1, height=10, color=$000000, pixel_type="rgb32"), 0)

stackvertical(y,r,g,b)

[IanB]

Given recursion is pretty savage on the internal working of avisynth, (note the startup delay) a gentler approach could be use RGBAdjust to nuke the unrequired channelsFunction Swipe(clip clip, int n) {
n=n+1
StackHorizontal(clip, BlankClip(clip, width=1, color=$010101*n))
return (n>=255)?Last:Swipe(Last, n)
}

Swipe(BlankClip(width=1, height=10, color=$000000, pixel_type="rgb32"), 0)

R=RGBAdjust(g=0, b=0)
G=RGBAdjust(r=0, b=0)
B=RGBAdjust(r=0, g=0)

stackvertical(Last, R, G, B)Other tricks with RGBAdjust could be to reverse the slope...
R=RGBAdjust(r=-1, rb=255, g=0, b=0) # R=255-Y, G=0, B=0
...

@jmac698,

Nah, it's just a case of great minds think alike (BR-G has a great mind ;) )

[Midzuki]

Maybe this is a silly suggestion/idea, but here it goes anyway:

any plans to include "HDColorBars()" in the next version of Avisynth?

[IanB]

Which particular HDColorBars() where you thinking of?

i.e. state which reference you want implemented.

[Midzuki]

The SMPTE one:

http://upload.wikimedia.org/wikipedia/commons/thumb/6/60/SMPTE_Color_Bars_16x9.svg/500px-SMPTE_Color_Bars_16x9.svg.png

[IanB]

@Midzuki,

You must want this really badly.

You point at a wikimedia .PNG image with levels scaled to make RGB(0,0,0) correspond to -2% black. Okay it conveys a visual impression of what you would like, but it is not a usable specification.

Knocking up the code for the test pattern is easy. Proving and justifying the colour values for each cell requires research and quoted references that are preferably free and open. Read all of this thread to get an idea of what is actually required.

ElseImageSource("500px-SMPTE_Color_Bars_16x9.svg.png")
ConvertToYV24(matrix="Rec709")
Levels(16, 1.0, 235, Round(11.62), 235, Coring=False)

:Edit: Hint arib std b28 v1.0 (http://www.arib.or.jp/english/html/overview/img/arib_std-b28v1.0_e.pdf)

[Midzuki]

ImageSource("500px-SMPTE_Color_Bars_16x9.svg.png")
ConvertToYV24(matrix="Rec709")
Levels(16, 1.0, 235, Round(11.62), 235, Coring=False)

:Edit: Hint arib std b28 v1.0 (http://www.arib.or.jp/english/html/overview/img/arib_std-b28v1.0_e.pdf)

:thanks: :thanks: :thanks:

[Wilbert]

@Midzuki,

This page contains a script producing the test-pattern: http://avisynth.org/mediawiki/HDColorBars (thanks to jmac698).

[pandy]

The 100/0/75/0 style colour bars are a legacy of crappy domestic televisions, which if ever presented with 100% saturated colours invariable flared or bloomed (some even displayed wrong colours as some video signal path overloaded and distorted). So the industry conspired to limit saturation to 75% everywhere they could.

Rather to protect RF broadcast amplifiers against saturation (100/100 color bars equal roughly 134IRE on PAL CVBS which is beyond capabilities of RF - 100/100 is not allowed on RF but it is allowed on component) also 709 is pure component (HD) and it is incompatible from analog TV broadcast (analog TV have only one "hardwired matrix equation" on color decoder)