Noticing colour differences before and after encoding

[jclampy]

I have got a problem and am not sure what is causing it or how to fix it but no that there is a solution and am hoping that someone can enlighten me, please.

I have tried outputing some PAL colour bars from the Avisynth colorbars() filter. Here is my script:

ColorBars()
AssumeFPS("pal_film")
Trim(16924,18423) #SMALL TEST
Converttoyv24(interlaced=false,matrix="Rec601") #lossless conversion?
#ColorYUV(Analyze=True) # Uncomment to check YUV levels
Converttoyv12(interlaced=false,matrix="Rec601",chromaresample="bicubic",ChromaOutPlacement="mpeg2") #lossy conversion?
#ColorYUV(Analyze=True) # Uncomment to check YUV levels
Spline16Resize(352,576)
#info()
#ColorYUV(levels="TV->PC") # Just for viewing when calibrating colors on PC monitor to simulate look on TV

*The ColorYUV(levels="TV->PC") filter at the bottom of the script is just for me to simulate the *view of a TV* like 16-235 on PC monitor while in AVSPmod preview window. I don't use this when encoding the video.

The problem is as follows;
A) Before encoding the colour bars look correct on PC monitor in AVSPmod when full range viewed is 16-235.
B) After encoding and recording to DVD then playing on a TV set the colours 'appear' to be correct 16-235 range.
C) After encoding and playing .M2V file in PowerDVD the colours are wrong as range seems somewhere between 16-235 and 0-255.
D) After encoding and recording to DVD then playing in PowerDVD the colours are wrong as range seems somewhere between 16-235 and 0-255.

I then went to this website where you can download a PAL SD DVD calibration in VOB format: http://merifon.altervista.org/TestDVD.html

I burned this to DVD and then played it in PowerDVD and the colours were correct range of 16-235. It also looked fine when played on TV as well showing 16-235.

This proves that it can be done right and can work perfectly but something I am doing is screwing things up. I need to get this sorted or all my encodes will be wrong.

Here are my Test Result images so you can check the colour range results yourself; .zip file is 33KB compressed.
http://www.mediafire.com/?8kx6copxrll4xd0

I am using HCenc to encode and here are my settings:
interlacing options = progressive
chroma downsampling = progressive
progressive sequence = ticked
colorimetry = BT.470-2BG
MPEG profile@level = MP@ML
--------------------
| encoder settings |
--------------------
profile: BEST
frames: 0 - 1499
framerate: 25.000
aspect ratio: 4:3
chroma format: 4:2:0
bitrate Kb/s: 8000
max. bitrate Kb/s: 8000
pulldown: no
closed gops: no
VBV check: yes
VBV size (bits): 1835008
scene change det: yes
interlaced: no, BFF
goplen,B-pic: AUTO 15
dc_precision: 10
intraVLC: auto
scan method: zigzag
bias: 100
chapter frames: 0
time code: 0 0 0 0
CPU: SSE2
priority: idle
SMP active: yes
matrix: MANONO1, adaptive
luminance gain: yes, level 4
adaptive quantization: yes, strength 4
deadzone quantization: auto
------------------
| source stats |
------------------
nr. of frames in source: 1500
width*height: 352x576
fps: 25.000
nr. of frames to encode: 1500
frames to encode: 0 - 1499

I then use DVD-Lab Pro2 with default PAL wizard single movie with no menu settings.

I then burn the VOB's with Nero v6.6.0.14 to DVD.

All help will be greatly appreciated. I assume problem might be either straight before or while encoding as the .M2V played in PowerDVD looks wrong, but then I don't know if .M2v files played in PowerDVD are normally played back showing full range as 16-235 or not?

[Mounir]

I'm guessing power dvd is the culprit here as it display well on tv

off topic:
I suggest you use avamat6 (for dark scenes) or mpeg matrix (outdoor scenes with plenty of light) (not mpeg std!) and lumgain=1, aq=2 , 2 pass vbr for best results

[jclampy]

Hi thanks for response,

I was thinking that is not PowerDVD because I tried that already encoded PAL colorbars DVD and was getting correct results.

Only my encoded colorbars is not playing right in PowerDVD.

There must be a reason, maybe I *should* be using the last line in my script while encoding instead of commenting it out.

Or perhaps I need to clamp the 16-235 colour range with a filter somewhere in my script, but I am not sure this will solve the problem.

It's got to be something with my Avisynth script, DVDLabPro2, or NERO. That is what I assume, as TV will show 16-235 color range regardless as can't show below or above that.

[jclampy]

I got it working properly but is this how Avisynth is suppose to work?

Here is my Test2 script (same as Test1 but with the last line also used while encoding):
ColorBars()
AssumeFPS("pal_film")
Trim(16924,18423) #SMALL TEST
Converttoyv24(interlaced=false,matrix="Rec601") #lossless conversion?
#ColorYUV(Analyze=True) # Uncomment to check YUV levels
Converttoyv12(interlaced=false,matrix="Rec601",chromaresample="bicubic",ChromaOutPlacement="mpeg2") #lossy conversion?
#ColorYUV(Analyze=True) # Uncomment to check YUV levels
Spline16Resize(352,576)
#info()
ColorYUV(levels="TV->PC") # Used so encode looks correct on TV and also PC in PowerDVD

All other processes to get the encode onto DVD is exactly the same.

I checked and video looks fine on TV and on PC in PowerDVD now.

Here is my Test Result image so you can check the colour range results yourself; .zip file is 12KB compressed.
http://www.mediafire.com/?8zpbmo27a8ir75z

So now my question is;
Is Avisynth suppose to work in this way. I have never noticed anywhere (although that could be my downfall) that it is mandatory to add this line or a line like this at the end of a script to make sure all encodes fall within range of 16-235.
IE; ColorYUV(levels="TV->PC")
Otherwise all encoded videos will be 0-255 color range, even though I was using REC.601 throughout my script.
The colorbars() filter is suppose to be [16,235] REC.601 by default so what is going on here?

If you want to do your own tests to check then please do, it is a simple test using Avisynth internal colorbars() filter.

Is it just me, or has everyone that has ever encoded without a line like or similar to ColorYUV(levels="TV->PC") at the end of their scripts made incorrect video based on a 16-235 range?

PS: I am using Avisynth 2.6a3 if that means anything. Maybe it is a bug in this version or maybe I am the only one in the world who didn't know how to do this properly?

Edit1a:
The above image result shows that in PowerDVD although the resulting colours are not 100% correct (some colors are 3 values out of 255 off or at most 2% off so not that bad), I am wondering if there is a filter or method I could use that is similar to ColorYUV(levels="TV->PC") but would give slightly more accurate color results? If anyone could shed some advice on this I would be much abliged, whether an internal or external filter doesn't matter. Thanks.

Edit2:
Also completed full test comparison with VLC player and got the same results as with PowerDVD. They both are showing same results so it appears is not a problem with PowerDVD. Also I have tested both PowerDVD and VLC with a commercial DVD (Terminator 2 Ultimate Edition: THX calibration feature) and noticed that the color values confer with my results that ColorYUV(levels="TV->PC") is required at the end of Avisynth script even if your source is [16,235] REC.601.

Any response?

Edit3:
The only other feasible explanation I can come up with at the moment is that with the Avisynth script, maybe my PC graphics card (ATI HD4670) is automatically *in hardware* scaling it out to 0-255 and the only way to get true 16-235 is by adding that ColorYUV(levels="TV->PC") or similar to the end of my script. I have heard that Nvidia graphics cards have a setting in their control panel to switch between 16-235 or 0-235 colorspaces but I do not have the hardware on hand to check whether this is the case or whether it can have an effect or solve this issue. So, maybe someone else can check that which could mean that it only remains an issue for ATI or some ATI users? Ofcourse this might not make sense though since the graphics card would just scale it back out to 0-255 after the filter anyway right?

Edit4:
In case this might help someone I have now tested Limiter(16,235,16,240), BT709ToBT601(), and colormatrix("Rec.709->Rec.601"). They all did not produce the result I was looking for. Levels(16,1,235,0,255,coring=false) and ColorYUV(levels="TV->PC") are the only solutions to what I am experiencing. There is a slight difference in color value numbers between the two filters results which I am just checking. It looks like Levels(16,1,235,0,255,coring=false) might be more accurate and therefore the filter of choice.

Edit5:
I can confirm that Levels(16,1,235,0,255,coring=false) is definately the most accurate method. Only a few colors had values that at the most were 1 out of 255 off which is pretty amazing considering all the conversions going on in the script.

Edit6:
Just trying to confirm whether coring=false or coring=true is better for an encode I am going to do. In regards to the internal colorbars() filter, the 'coring' setting of Levels() doesn't appear to make a difference must be because the colorbars() is in RGB32 format and maybe 'coring' requires a YUV format?

Once the colorbars() have been converted to output in YV12 for example the coring=false or coring=true does have an effect, not visually noticeable but when analyzing the color value numbers the ChromaU and ChromaV don't change but LumaY is reduced to 16-235. This is when coring=true. I am not 100% sure but this might be better for MPEG2 or DVD compliance or for some hardware/software reading said video stream? If anyone can confirm or elaborate on this please. At the moment my gut feeling is to go with Levels(16,1,235,0,255,coring=true) for BT601 DVD output.

[nhope]

My NVIDIA driver gives control like this:

http://dl.dropbox.com/u/21489814/NVIDIA-control-panel.png

I have a limited version of PowerDVD that was bundled with an LG Blu-ray writer. I made a test DVD and this is how it plays back DVD folders on my hard drive:

- When I select "With the video player settings", [16-235] is mapped to [16-235].
- When I select "Limited (16-235)", [16-235] is mapped to [16-235].
- When I select "Full (0-255)", [16-235] is mapped to [0-255].

VLC is doing the same thing, but I think you can adjust that in the video settings. I would prefer both programs to display the full range by default. The only way I can get them to emulate a DVD player and TV is to use the NVIDIA 0-255 setting.

If you want to "expand" you luminance I would tend to use matrix="PC.601" at the start of the script instead of adding ColorYUV(levels="TV->PC") at the end.

[jclampy]

Thanks for the response nhope, hopefully that information will help Nvidia users that are reading.

Also, where you suggest:

If you want to "expand" you luminance I would tend to use matrix="PC.601" at the start of the script instead of adding ColorYUV(levels="TV->PC") at the end.

I just tried this if this is what you were refering to;
ColorBars()
AssumeFPS("pal_film")
Trim(16924,18423) #SMALL TEST
Converttoyv24(interlaced=false,matrix="PC601") #lossless conversion?
Converttoyv12(interlaced=false,matrix="PC601",chromaresample="bicubic",ChromaOutPlacement="mpeg2")
Spline16Resize(352,576)
#ColorYUV(Analyze=True) # Uncomment to check YUV levels
I found that the values of the individual colors; green, magenta, etc were still off a little.
I used the values of reference (1) <Rec. ITU-R BT.801-1> from here: http://forum.doom9.org/showthread.php?p=800670#post800670
Ignore the 'Pluge' values from that post and just focus on the colors and black/white.
I took a screenshot of the AVSPmod preview window and checked the RGB and intensity values of each color in GIMP.

From my 'current' findings I would suggest using either Levels(16,1,235,0,255,coring=false) or Levels(16,1,235,0,255,coring=true) at the end of your script instead of the "PC601" or "ColorYUV(levels="TV->PC") methods.

[nhope]

There is a dot in "PC.601" (which is inconsistent with the lack of a dot in "Rec601"). You can also try changing 601 to 709 which might fix your colors in some encoding scenarios.

If you have to "expand" your levels at the end, see Gavino's post (http://forum.doom9.org/showthread.php?p=1484379#post1484379) on why ColorYUV(levels="PC->TV") might give you superior results to Levels(0, 1, 255, 16, 235, coring=false). SmoothLevels from SmoothAdjust (http://forum.doom9.org/showthread.php?t=154971) is another option.

[jclampy]

Hmm, it seems to do the same with PC601 or PC.601, it must be how the filter has been written to work either way. I was expecting it to throw an error message when I first used it but it didn't. Retesting with the PC.601 the results are also the same as PC601. Color values are slightly out for example;

Green: R=3 G=188 B=3 intensity 74 when it should be R=0 G=191 B=0 intensity 75

With Levels(16,1,235,0,255,coring=true or false) then Green is correct R=0 G-191 B=0 intensity 75

With ColorYUV(levels="PC->TV") then Green is R-3 G=189 B=1 intensity 74

So I still stand by my suggestion of Levels(16,1,235,0,255,coring=true or false)

I am not sure how to use SmoothLevels to do the above. In my 'proper video encoding' I am using SmoothLevels earlier in the script to get the initial colours into the correct black/white (or brightness/contrast) areas of the colour range. Before I do my colour calibration with SmoothTweaks.

What I plan to do with the information I am learning in this thread is get the final video 'locked in' before sending to the encoder. I have found that when you run a filter (maybe all/any filters) in the script they change the individual color value attibutes. Even in this colorbars() example where I am using spline16resize, that is enough to change the color values. The way to see this happening is using ColorYUV(Analyze=True).

Because of this it looks like you need to use something like Levels(16,1,235,0,255,coring=true or false) at the end of your script even if you were going to use the PC.601 method earlier in the script.

Unless keeping all values within (L) 16-235 values and all (Y)16-240 values is not important to your particular needs.

[TheSkiller]

I think you are missing one major point about the Colorbars(). They do include super black. In other words they are TV-range but with a patch of 0 in the lower right part.
So as soon as you squeeze the whole thing to 16-235 the color bars have wrong values. If you want to exclude the super black from the test just use Limiter(), it clips the super black.

[jclampy]

Sorry TheSkiller but that is not doing that at my end.

Yes I want to exclude the 'super blacks' since 16-235 would not have them. Limiter() doesn't do that at my end though. It just shrinks/squishes the whole 0-255 range down into the 16-235 range so everything goes way off including colors.

In the meantime I have tested both coring ON and OFF on an old analog CRT TV and can report that there is no visible difference.

Therefore I recommend Levels(16,1,235,0,255,coring=true) when making a BT601 DVD for the reason that it 'tidies up loose ends' (notice the pun). I assume it may also help an encoder or standalone DVD player but would require someone a bit more knowledgeable to confirm that.

Edit:
Here is a test image using Limiter() you can see and analyse the colours that are definately wrong for a 16-235 output:
http://www.mediafire.com/?d8c35blly1bu8de

And to put my money where my mouth is here is with Levels(16,1,235,0,255,coring=true) which is best so far:
http://www.mediafire.com/?07l2sm6vjpz7ak0

[TheSkiller]

Why are you dealing with Levels() at all? There is no need for this because the Colorbars are generated at the right levels. It is preferable to generate them in YV12 instead of RGB32 to avoid an unnecessary color space conversion.

Use this, I would be very surprised if things are still not right then:

Colorbars(width=352, height=576, pixel_type="YV12").trim(0,1499).assumeFPS(25)

Limiter() #exclude super blacks

####Analyzing
#ColorYUV(Analyze=True)
#Histogram("Levels")


Feed this as is to HCenc and you get perfect Colorbars for DVD.

[jclampy]

Hmm, I was under assumption that colorbars() were created in RGB32. According to colorbars documentation here; http://avisynth.org/mediawiki/ColorBars it states this:
ColorBars(pixel_type="YUY2")
is equivalent with;
ColorBars(pixel_type="RGB32")
ConvertToYUY2(matrix="PC.601") # doesn't scale the luma range

I assumed that for colorbars(pixel_type="YV12") with Avisynth 2.6 conversion paths would be equavalent to;
ColorBars(pixel_type="RGB32")
ConvertToYV24()
ConvertToYV12()


Maybe you are right that it can create colorbars in YV12 colorspace with no conversions but I don't see that in the documentation.

Regardless that is not a solution that I am looking for because my source is RGB32 so I need to find a solution that begins from ColorBars(pixel_type="RGB32").

Interesting if limiter() is working in that example, I will have to investigate tomorrow as to why that is. Thankyou for your response.

[TheSkiller]

Maybe you are right that it can create colorbars in YV12 colorspace with no conversions but I don't see that in the documentation.

The two examples taken from the documentation are indeed equivalent in terms of levels, but there is still another kind of difference between them (quote from the doc):

When directly generating YUV format data the color transitions are arranged to occur on a chroma aligned boundary.That's why I said it's preferable to generate them in YV12 in the first place if YV12 is your final storage format.


I assumed that for colorbars(pixel_type="YV12") with Avisynth 2.6 conversion paths would be equavalent to;
ColorBars(pixel_type="RGB32")
ConvertToYV24()
ConvertToYV12()

No, they are not. The point is, ConvertToYV24() converts the RGB input range of 0-255 to 16-235 if you do not specify otherwise by adding matrix="PC.601". It does not matter if your RGB input is already 16-235, it is converted anyway since AviSynth does not know or care.
I think that's what your problem was.
Usually you want a full range RGB input mapped to TV range in YUV to produce correct levels for DVD etc, so the only exeption where you would want to use matrix="PC.601" is with RGB sources which are already in TV range (like Colorbars(pixel_type="RGB32")).

[IanB]

The documentation for ColorBars() is slightly misleading with the RGB32 and YUY2 conversion analogy. Because the output for ColorBars() contains abnormal diagnostic levels for some of the pattern there is no completely correct conversion possible between RGB and YUV. The RGB32 and YUY2 conversion analogy is correct only for the parts of the pattern that contain legal non-diagnostic levels.

The default output mode is RGB32 with the levels for legal non-diagnostic parts scaled to [16..235], the diagnostic parts of the pattern contain levels outside of these limits.

You can crush the diagnostic parts of the pattern with the Levels() filter, thusColorBars().Levels(16,1.0,235, 0,255) # [0..255] pattern, crushed diag elementsThis results in a fully legal [0..255] RGB32 pattern, that you may further process as required.


Similarly you can crush out the diagnostic parts of the pattern in YUV mode withColorBars(pixel_type="YV12").Levels(16,1.0,235, 16,235, coring=false) # [16..235] pattern, crushed diag elementsNote there will still be out of RGB gamut chroma elements in the output.

[jclampy]

Although I did take the comparison in the documentation for granted which I just tested now and found that it is not 'equivalent' because the quality is visibly worse when converting the colorbars from RGB32 to YUY2 instead of creating them in YUY2 in the first place. Which may sound obvious to some, but that same comparison in the documentation also makes it seem that colorbars() can only be created in RGB32 and goes through a conversion to be in a YUV format instead of being able to create them in YUV in the first place. Plus I assumed Avisynth documentation was 'gospel'. I won't make that mistake again and always test everything myself beforehand instead of taking what I read in documentation for granted. Forgive me if I sound angry but I was under the impression that I was getting somewhere yesterday and now find that I have to start from beginning again.

Although my test process yesterday might have got acceptable results because I was working with colorbars, I think the flaw in my test process was also because I was working with colorbars.

The colorbar frame is made up of color values that reach the outer peaks of 0-255 or 16-235 depending on how you have them. When dealing with 'real' video footage frames don't always reach these peaks. In those situations then my methods from yesterday could have been 'boosting' color values to meet these peaks. I will go out on a limb and say this might happen with levels() filter and why limiter() could be the 'proper' solution when dealing with 'real' video footage.

Also, I am guessing for limiter() to work I need to be using PC.601 instead of RC601 in my colorspace conversions.

So, thankyou TheSkiller and IanB for opening my eyes to this and hopefully with today's testing I will be able to understand and have my script setup so it is working 'properly'.

PS: My 'proper' video encoding script is taking an RGB32 source and I want to be outputing to YV12 so I still have to think along those lines when thinking up a new solution/method today.

Edit:
Also this statement in the documentation "By default, a clip is produced of 640x480, RGB32 [16,235],..." is misleading since the default colorbars include values that go below 16. Also, by default these colorbars are incorrect apart from the correct white of 235 and the correct black of 16. All the remaining colour bars of the frame appear to be off whether the values have been squished upwards or downwards as well. It would have been nice if by default colorbars() would have created a fully correct and useable colour bar frame that could have been used straight away, as is, with no further adjustments required. Maybe with a setting so you can choose between having one created with or without super blacks/whites, ie: 16-235 or 0-255.

[2Bdecided]

Also this statement in the documentation "By default, a clip is produced of 640x480, RGB32 [16,235],..." is misleading since the default colorbars include values that go below 16.[16,235] just means black is 16 and white is 235 (unusual for RGB). There are values below 16 because the colour bars include blacker-than-black.

It's beyond me to know whether the values are correct or not...
http://avisynth.org/mediawiki/ColorBars_theory

Cheers,
David.