ColorMatrix v2.3

[yesgrey]

v1.10 contains a text file "coefficients.txt" where those coefficients are derived.

Can you tell me where can I get it?

Thanks.

[yesgrey]

I have found v1.5 and it has the file. I will look at it.

Thanks.

[Wilbert]

I'm not sure whether i changed it in newer version, so i attached it for you.

[yesgrey]

Wilbert,
thanks for the file. It seems to be identical to the one I found. It was very helpfull.

[uncertainty]

Could someone confirm my understanding of this thread when using source material of HDTV TS (mpeg2 720p or 1080i). I should NOT enable any colormatrix conversion of the BT.709 colorimetry?
I'm currently using the latest builds of megui and after reading this thread I have disabled any color correction for these type of source material.


If I'm mistaken let me know...

thanks

[Adub]

What kind of encoding is your destination going to be? AVC, ASP, MPEG2, etc?

From you saying MeGUI, I am assuming AVC or ASP, so I do believe that you should enable Colormatrix for your encodes. Especially if you are using DGindex's hints for the encoding process.

Edit: Use colormatrix according to the source colorimetry, which DGindex displays when you use the "Preview" selection in the file menu.

[uncertainty]

What kind of encoding is your destination going to be? AVC, ASP, MPEG2, etc?

From you saying MeGUI, I am assuming AVC or ASP, so I do believe that you should enable Colormatrix for your encodes. Especially if you are using DGindex's hints for the encoding process.

Edit: Use colormatrix according to the source colorimetry, which DGindex displays when you use the "Preview" selection in the file menu.


Megui with AVC atm and from what Hansolo and Headbanger and others have stated in this thread that using colormatrix for HD source material ( TS 720p , 1080i) will have adverse color effects on AVC output.

This entire thread needs a summary written for it because atm there is so much conflicting information its hard to make heads or tails out of it.

[Adub]

The best thing for you to do is to do a comparison. Use a program like AvsP to compare two scripts. One using colormatrix, the other one not using colormatrix. The best result should obviously be the one with the best colors.

[Wilbert]

This entire thread needs a summary written for it because atm there is so much conflicting information its hard to make heads or tails out of it.
Yes, i will do that one day :) To busy with other stuff atm.

[IanB]

@uncertainty,

Actually, stop and think about the chain of processing your project has taken so far and will take to your ultimate viewing experience.

Typically a camera photographed a scene.

The image was processed and stored as an MPEG2 stream (YUV colour space) which you now have. In the processing a RGB->YUV conversion will have been done, hopefully using a standard conversion and hopefully the correct colourmetry information was included in the stream.

To view a YUV colour space image on a RGB display device there has to be a YUV->RGB conversion. To get the correct original colours back you must use the correct conversion. If the correct colourmetry information was included in the MPEG2 stream, and the display conversion process can correctly act on that information, no problem.

If the colourmetry of the data encoded in the original stream is not as you want/need it to be, the ColorMatrix filter allows you to change the colourmetry of YUV data directly to suit.

If the display process conversion is not as you want/need it to be, again the ColorMatrix filter allows you to change the colourmetry of YUV data directly to suit.

So if the original stream has wrongly documented colourmetry, you can fix it.

If the original stream is correct but simply not as your encoding process needs, you can fix it.

If your Dvd player or display device or what ever uses the wrong conversion, you can fix it.

You need to take a holistic view of the entire process and be clear what conversion matrix was/will be used at each YUV/RGB conversion.

i.e.
RGB ->[601]-> YUV -> ... ->[601]-> RGB
RGB ->[709]-> YUV -> ... ->[709]-> RGB

RGB ->[709]-> YUV -> [ColorMatrix 709to601] -> ... ->[601]-> RGB
RGB ->[601]-> YUV -> [ColorMatrix 601to709] -> ... ->[709]-> RGB

[redfordxx]

Hi, I have question on following:

I have mpg video from digicam which is reported ITU-R BT.470-2 in DGIndex. Acc to manual it means not to apply colormatrix, right?
I encoded it to "x264.mkv" using MeGui.

Then I tested the color with script like:
o=MPEGSource...
c=DirectShow("x264.mkv")
#emhasize difference
mt_lutxy(o,c,"x y - 10 * 128 +")
Then I receive very orange video...the colors are different.

Where could be the problem?

Robert

[foxyshadis]

You must use chroma="process" (or u=2,v=2) with mt tools, or color will be completely meaningless trash.

You can also use levels to do the job.

Even once you have that, there's no guarantee that DirectShow will render colors correctly, or even that mpeg2source will if the stream has the wrong colorspace. Comparing to a film print is literally the only way you can ever be sure the colors are correct.

[redfordxx]

oooooooooooh, beginner's mistake...

mt_lutxy(o,c,"x y - 10 * 128 +",y=3,u=3,v=3)is OK

[Vesi]

When I rip dvd to xvid megui shows these kind of colormetry for defrint type of source, and i don't know i should keep enable the megui's colorcorection and when not, as megui's default is enable always

ITU-R BT.470
ITU-R BT.601
ITU-R BT.709
SMPTE 170 M
SMPTE 240 M

and it is hard for me to know, do i need colormatrix or not. can some one explain this when i need to use colormatrix and when not?
and if there is any other type of color that not named in the following list please add it.
And how i use this filter? I am doing xvids and x264 using megui
and should i replace this with the one i got with megui?

[foxyshadis]

Simple: If it's anything but 601 for SD or 709 for HD you should use it, make sure you always use it with source->dest arguments. The readme mentions BT.470 and 170M are the same as 601, so just refer back to it if you ever forget.

[Vesi]

Will be this Enough
DGDecode_mpeg2source(----info=3)
ColorMatrix(hints=true,interlaced=true).

if I get anything else from the one you mentioned,that colormatrix is not needed for. or I should do it in another way or add some param

[foxyshadis]

Yes, but specify your destination if it's HD (where 709 is preferred). Colormatrix doesn't try to detect HD and will always use 601 by default.

[Vesi]

make sure you always use it with source->dest arguments.

Could you explain this in simple words, coz I am still not familiar with advance language of encoding :).

And I am doing Xvid,x264 and divx.
I did xvid test with source color ITU-R BT.709 and enable colormatrix, i check the file with avsp and found the color close to source, when i remove the info=3 and colormatrix line from the script, then found it a bit off and dark.

The most important thing is for me now, with which type of color I need ColorCorection when I am doing Xvid,Divx and x264.

[tritical]

I updated ColorMatrix to v2.3 (http://bengal.missouri.edu/~kes25c/ColorMatrixv23.zip) based on ideas from josey_wells' BZColorspace filter in this thread: http://forum.doom9.org/showthread.php?t=133389.

Changes:
- Calculate conversion coefficients for PB/PR rows from luma coefficients
- Added inputFR and outputFR parameters allowing for YUV range expansion/contraction
- Changed the clamp parameter from bool to int to allow specifying only pre clipping
or only post clipping or both

The YV12 assembly routines aren't coded such that they can support the case where inputFR/outputFR aren't equal, and I don't have enough time atm to rewrite them or write new ones. So for now, if inputFR/outputFR aren't equal then the c routine gets used. Luckily, the c routine is pretty fast (it should even be slightly faster in this version than prior versions). I also made it so that dest/source can be set equal now as long as inputFR/outputFR are not equal (in which case you just get a range conversion). I set up the defaults for inputFR/outputFR/clamp such that v2.3 acts the same as v2.2 by default.

[EuropeanMan]

Why can't I use the 'clamp' function via CM 2.3? Any idea...since i'm getting an error "argument is wrong type"?

[tritical]

'clamp' was changed from bool to int in v2.3. clamp=3 in v2.3 is the same as clamp=true in v2.2. clamp=0 in v2.3 is the same as clamp=false in v2.2. If you enter it as an int value and still get an error please post your script.

[IanB]

@tritical,

That was a mean thing to do, changing the calling sequence to a well loved plugin. :(

You could define 2 templates for the function and add a small amount of code to the create function to allow old bool clamp to still work along with the new int clamp.

[tritical]

That is a good idea for maintaining backwards compatibility in filter syntax (something I generally pay little attention to), and one that had never occured to me. However, I don't think it is a major problem here since most of the encoder guis that generate scripts with colormatrix don't specify the clamp parameter. In fact, clamp was new to v2.2 (previously it was an integer called 'scaling').

[EuropeanMan]

@Tritical...appreciate your help. I seem sorta dumb for not reading about the change in bool type :)
clamp works for me now...

[hkazemi]

I've compiled a summary to hopefully consolidate the threads and address the confusion over colorspace and the usage of Colormatrix. It is available at http://forum.doom9.org/showthread.php?t=133982#post1090068

[canuckerfan]

if I'm working with a dvd source that i'm upscaling to 720p using x264. what colourspace should the material be set to?

[Adub]

Probably rec 709, but that is not 100%. Try it and if the colors look washed out, try another matrix. It is impossible to set it right 100% of the time, as streams can often be mislabeled as the wrong colorspace.

[Zanejin]

Let's say I have a YV12 Rec.709 [0, 255] source that will be filtered by AviSynth without color-space conversion and then encoded by x264. Am I correct to say that because there are no conversions to RGB, ColorMatrix is unnecessary? And do the majority of LCD monitors display RGB rather than YUV?

[foxyshadis]

Everything displays RGB. Monitors, projectors, scanners, etc, aside from black & white stuff. It's nearly impossible to engineer phosphors that range from one hue to another.

As long as you specify the colorspace in x264's encoding options (see --longhelp), you can keep the source's colorspace and let the decoder worry about it. I believe 709 fullscale is default for HD anyway, with newer video drivers, if you do the conversion on the video card. If doing it in software, you have to use a decoder that uses that information.

[tre31]

Ok I was directed here by neuron2 regarding 'no hints' colormatrix dgindex and HDTV. It seems HDTV and other HD sources in particular are not handled well by colormatrix at all, 'unknown' errors, 'unspecified video (2)', etc. I read in the dgindex thread that hdtv sources were assumed too be bt.709, if that is the case then why not let colormatrix also assume that and then allow 'hints=true' too function properly using bt.709.

Currently I just use Colormatrix() - whether that does anything or is a solution too this problem I don't really know - or just remove it entirely from the script, but the fact remains that dgindex and colormatrix do not handle HD sources.

Perhaps someone can enlighten me on the best way too handle all this.

[neuron2]

Specifically, we are asking if the hints should be set according to the assumptions DGIndex makes for the Info dialog when the stream does not specify colorimetry, or whether the hints should state that the colorimetry is unspecified. The latter is done now. I wanted to get the opinion of the ColorMatrix users before making any change to the hints.

[Wilbert]

Specifically, we are asking if the hints should be set according to the assumptions DGIndex makes for the Info dialog when the stream does not specify colorimetry, or whether the hints should state that the colorimetry is unspecified. The latter is done now.
I prefer the former :) Because it's good and otherwise users can't use hints in an automatic way.

[neuron2]

And so it shall be.

[IanB]

Consider :- If the stream does not specify the colorimetry then Mpeg2Source should NOT set the hints to pretend that it does.

If Mpeg2Source makes a guess and presents it as fact then filters like ColorMatrix cannot be programmed to handle the no colorimetry case, i.e. a default behaviour or a user specified default.

I think the gist of the above thoughts is ColorMatrix is not handling the no colorimetry case as they would like. If that is the case then ColorMatrix should be changed not Mpeg2Source.

If you really must do it this way then at least set an extra bit to say the colorimetry value is "Guessed" so future filters can better handle the decision process.

[~bT~]

does anybody have a working link pls for v2.3?

[buzzqw]

http://www.64k.it/andres/data/avisynth/ColorMatrixv23.zip

BHH

[~bT~]

^ :thanks:

[foxyshadis]

Consider :- If the stream does not specify the colorimetry then Mpeg2Source should NOT set the hints to pretend that it does.

If Mpeg2Source makes a guess and presents it as fact then filters like ColorMatrix cannot be programmed to handle the no colorimetry case, i.e. a default behaviour or a user specified default.

I think the gist of the above thoughts is ColorMatrix is not handling the no colorimetry case as they would like. If that is the case then ColorMatrix should be changed not Mpeg2Source.

If you really must do it this way then at least set an extra bit to say the colorimetry value is "Guessed" so future filters can better handle the decision process.

I'm not sure. If no colorimetry is present, you're pretty hosed, there's no way to find a colormatrix automatically. The spec defaults are the only rational choice unless you've somehow obtained inside information on the real flags, in which case a manual override is needed in the script anyway. So it makes no difference either way except to people stuck with an older dgindex; for that case maybe colormatrix should be updated with the same code.

[canuckerfan]

if an intermediate lossless encoder was used (let's say lagarith) before Xvid or CCE would this produce unexpected results?

[foxyshadis]

Only if you perform a color conversion during lossless encoding. Keep it in YV12 through the whole chain and it'll be just fine.

[kandrey89]

I just started using ColorMatrix and having RTFM I'm still confused in how to apply it to my video.

Video is MPEG 2 Stream from DVD, in d2v format after being Film Forced to get rid of interlaced frames.
Color is indicated as SMPTE 170M(same as 601).
I intend to downsize and encode with XviD.

I'm looking at samples of 601, 709, in both PC and REC(regular-TV?), all are different, the faces in some are more yellow and some are more dimmer/brighter.
Questions:
1. Is SMPTE 170M = Rec.601? or PC.601?
2. If my video is Rec.601, do I need to use colormatrix at all since manual says 601 is for XviD encodes.
3. Which format should I choose, PC (brighter), or Rec (dimmer)? (PC looks really nice on screen, but Rec is more of a standard? and used for TVs?)
4. What color space should I set the video to if I want the video to look good both on PC and on TV, is that possible?
5. Do I need to ConvertToRGB if I want to encode into XviD? (saw this earlier in this thread, got me all confused)

Note: Just realized that ColorMatrix doesn't convert to PC.xxx, it's the ConvertToRGB that does it.

[kandrey89]

Help anyone?

[Ranguvar]

Question: I'm importing a (mostly) telecined DVD through DGIndex. I think I heard that ColorMatrix needs to run even before the IVTC, so should I set interlaced=true? Or not, since it's only telecined?

I would rather not Force Film.

[~bT~]

^ yes, unless u applied force film.

[45tripp]

@kandrey89

2.no
5.no

[pitch.fr]

hi

thank you for this awesome script!

was wondering if you could possibly add "REC-601 EBU" and "REC-601 SMPTE-C" colorspaces ?

I've given real world examples here, where it would very useful :
http://forum.doom9.org/showthread.php?p=1157662#post1157662

TIA,

[yup]

Hi all!

Simple question:
my source analog capture encoded huffyuv, my MPEG-2 encoder is CCE, can I use
ColorMatrix(mode="Rec.601->Rec.709")
in my script, and if yes where, after loading video file or at last string. In CCE setting I use Luminance level 16-235.

With kind regards yup.

[helms]

After reading this http://forum.doom9.org/showthread.php?t=133982#post1090068 I'm still unsure whether I'm correct in using

ColorMatrix(mode="Rec.709->Rec.601", hints=true, interlaced=true)

when downscaling HD (interlaced) to SD resolutions and encoding h264 video.

Is what I'm doing correct?

[Alex_ander]

Simple question:
my source analog capture encoded huffyuv, my MPEG-2 encoder is CCE, can I use
ColorMatrix(mode="Rec.601->Rec.709")
in my script, and if yes where, after loading video file or at last string. In CCE setting I use Luminance level 16-235.


A simple answer would be: if your capture source is in SD format, then you don't need ColorMatrix(); in case you go from HD to SD, you need conversion in the 709 -> 601 direction. Then it's better to use ColorMatrix() in place where the video is progressive or field-separated (if anywhere), or with 'interlaced=true' setting somewhere in the beginning of the script.
CCE is neutral to matrix type (like other encoders, doesn't change it or 'expect something special' as was considered earlier) and is also known to not write matrix type to file header (that is the possible reason why some software players may show input/output difference using their defaults).
That's my conclusion after reading the long discussion here and in similar threads, earlier I used to apply 601 -> 709 conversion with CCE in most cases.

[Alex_ander]

After reading this http://forum.doom9.org/showthread.php?t=133982#post1090068 I'm still unsure whether I'm correct in using

ColorMatrix(mode="Rec.709->Rec.601", hints=true, interlaced=true)

when downscaling HD (interlaced) to SD resolutions and encoding h264 video.

Is what I'm doing correct?

I suspect h264 uses 709 matrix for any resolution, then you probably don't need ColorMatrix in this case.

[OvejaNegra]

HI. After reading all this i'm a little confused about colormatrix. So i will post some questions/doubts and please feel free to correct me:

1- A source (like DVD for example) can use different coefficients for the color. The encorders like XVID, DIVX or X264 (or are we talking about the decoders?) use Rec.601. So colormatrix reads the info from DGindex (using hints or the d2v project) and : Converts the source coef to rec601 (to match the encoders/decoders or IF the source is already rec 601 it stays untouched.

2- The data inside my x264 encoding is YV12 (i will go back here later).
3- When i open a VOB file with DGIndex it converts to RGB (only for the preview of course) AND correct the levels of chroma and luma because the maximum black and white (for example) is just 236 and 16 (that will give me gray and allmost white in a PC with out correction). So it "scales" the range to 255-0. That's why i see a nice picture in my DGindex preview.
4- When i open my favorite avs script in my VDUB the same happens in the preview. (and i see a nice picture).
5- My X264 encoder process the data from my script (it's YV12) and the levels pass untouched, only if i insert colormatrix (hints=true) it will make a correction if necesary (but only in the coeffients, not in the YC levels right?).

6- The encoders won't alter the coefficients of the video or correct anything.

7- In the decoding stage (THIS IS THE PART THAT CONFUSE A LOT OF PEOPLE I THINK).The decoder (like ffdshow) pass the data in YV12 to the video renderer (like VMR9) and the renderer send the data to the video card (something like that). NOW, in some part, someone should scale the levels again (because my PC monitor is 0-255). The renderer or the video card driver BUT sometimes for several reasons that does not happens. And i don't have my nice image (some people think they are not using colormatrix correctly but colormatrix has nothing to do with this).

8-My fresh and new compressed MKV+X264+AAC does not look as nice as in the previews.

What can i do? (Appart of course of having updated drivers, directx and all that stuff)

a) Change the renderer in MPC (VMR9 works bad for me but overlay mixer do the trick, but i lose the subtitles)
b) Cancel the YV12/YUY2 etc output in ffdshow, it will output converted RGB. Looks nice for me but i loose speed with high resolutions.
c) Enable the levels filter in FFDshow and set the input to 16-235 if i remember well and the output to 0-255. That will scale the range. I don't loose speed
and looks fine for me.


If we start talking about bigger resolutions and different matrix things can be more complicated. Enough for me.

9- Haali Renderer makes the corrections? (but i will loose the subtitles if i use it).

10 -Where should i put colormatrix (hints=true) in my script if:

a) I'm encoding a pure interlaced movie (of course i should put the interlaced=true).
b) I'm using forced film in a 100% film movie.
c) I'm using honor pulldown flags with anime/hybrid DVD and TIVTC and all that stuff.

Please fell free to correct me. Use the number of the question to avoid innecessary quotes (10a, 3, etc)

I'm trying to understand the concept but it's a little "complicado" (complicated, in spanish)

Thanks.

[Wilbert]

1- A source (like DVD for example) can use different coefficients for the color.
Yes.

The encorders like XVID, DIVX or X264 (or are we talking about the decoders?) use Rec.601.
If you request RGB from them, they will use Rec.601. Otherwise they will leave them unaltered.

So colormatrix reads the info from DGindex (using hints or the d2v project) and : Converts the source coef to rec601 (to match the encoders/decoders or IF the source is already rec 601 it stays untouched.
Yes.

2- The data inside my x264 encoding is YV12 (i will go back here later).
Yes.

3- When i open a VOB file with DGIndex it converts to RGB (only for the preview of course) AND correct the levels of chroma and luma because the maximum black and white (for example) is just 235 and 16 (that will give me gray and allmost white in a PC with out correction). So it "scales" the range to 255-0. That's why i see a nice picture in my DGindex preview.
Yes. (Btw it's 235, and not 236.)

4- When i open my favorite avs script in my VDUB the same happens in the preview. (and i see a nice picture).
Yes. Note that it assumes Rec601.

5- My X264 encoder process the data from my script (it's YV12) and the levels pass untouched, only if i insert colormatrix (hints=true) it will make a correction if necesary (but only in the coeffients, not in the YC levels right?).
Yes. Irc, it will clip the data if you set levels=false. (I'm too lazy to look it up right now ...)

6- The encoders won't alter the coefficients of the video or correct anything.
Usually they don't. (Some do if you feed them RGB.)

7- In the decoding stage (THIS IS THE PART THAT CONFUSE A LOT OF PEOPLE I THINK).The decoder (like ffdshow) pass the data in YV12 to the video renderer (like VMR9) and the renderer send the data to the video card (something like that).
Yes.

NOW, in some part, someone should scale the levels again (because my PC monitor is 0-255).
Yes, if your source is YCbCr [16,235].

The renderer or the video card driver BUT sometimes for several reasons that does not happens. And i don't have my nice image (some people think they are not using colormatrix correctly but colormatrix has nothing to do with this).
Yes. As you note below, VMR9 doesn't scale the levels back.

8-My fresh and new compressed MKV+X264+AAC does not look as nice as in the previews.

What can i do? (Appart of course of having updated drivers, directx and all that stuff)

a) Change the renderer in MPC (VMR9 works bad for me but overlay mixer do the trick, but i lose the subtitles)
Yes.

b) Cancel the YV12/YUY2 etc output in ffdshow, it will output converted RGB. Looks nice for me but i loose speed with high resolutions.
Yes.

c) Enable the levels filter in FFDshow and set the input to 16-235 if i remember well and the output to 0-255. That will scale the range. I don't loose speed and looks fine for me.[/QUOTE]
Yes. Are you sure about this? Because i suspect you will also loose speed here.


9- Haali Renderer makes the corrections? (but i will loose the subtitles if i use it).
Irc, you can adjust these settings in Haali renderer.


10 -Where should i put colormatrix (hints=true) in my script if:

a) I'm encoding a pure interlaced movie (of course i should put the interlaced=true).
b) I'm using forced film in a 100% film movie.
c) I'm using honor pulldown flags with anime/hybrid DVD and TIVTC and all that stuff.
This should be covered in this thread (probably a few pages back).

[OvejaNegra]

Wilbert: THANK YOU!!!
I have now a new PC with a nice Nvidia GF7300, i don't loose speed using the RBG conversion or the leves correction, but i need more testing with HD resolutions.

I have a a very old Celeron 1.2 GH with Ati RAGE PRO II. With the conversion to RGB my videos with 704 X 480 are allmost unplayable. But with the levels setting everything is fine. Maybe i will loose speed with higher resolutions (like 1280 X 720) but with standard DVD my celeron is ok.


Levels=false? i can't find that here: http://avisynth.org.ru/docs/english/externalfilters/colormatrix.htm#options ???

If i'm using the software player X, most of them use overlay mixer as the default renderer, and i set ffdshow to output RGB: The overlay mixer will overcorrect the levels or with let them untouched? (because it assumes that the correction is already done)? Usually i use MPC but sometimes i test other players.

The overlay mixer ALWAYS correct the levels fine OR it's using the video card for that (and some times will fail if the card is crap or the drivers have bugs). On the other hand, if the correction is done in the overlay mixer, it's software+cpu. Why i dont loose speed? Yes i know that even the most crappy onboard chip decodes YV12 / IYUV but what about the correction of the levels?

I'm just looking for the most safe+trustable+easy way of doing the levels correction (just for telling my friends, not for me). It seems to me that the levels setting in FFDSHOW is safer but since the decoded data is YV12 then the overlay mixer in CoreMedia player (for example) will overcorrect the levels and screw everything. Or it can detect (even with YV12) that the levels are already corrected? I think not.


So, in the end the safer combination is MPC + VMR9 (if you want SSA subtitles in anime) + levels correction or MPC + VMR9 (if you want SSA subtitles in anime) + RGB forced output (if your pc can handle it) or MPC + HAALI (and the ffdshow subtitles). Or don't do anything and leave the work to the Overlay Mixer (IF you can trust in that quartz.dll :) ).

And finally: How can i (by visual inspection) check if my correction is fine? Like comparing the final frame in my player with a preview of the same in VDUB mod (using AVS serving)?

Any advice of how the avs script should be? Or DirectShowSource () it's enough?

NO WAIT! i have one more question: If i open one AVI in VDUB using AviSource () or in Premiere (using of course the VCM codecs) the correction still happens or something may fail? I'm sure Vdub will make the corrections but Premiere, not so sure. I hear rumors about premiere messing with the YV12 colors.


Thank you for your attention.

[tom942]

I don´t really know if this info is useful, but I'm going to post it.

I was playing with Indiana Jones and the Kingdom of the Crystal Skull and The Sleeping Beauty 50th Anniversary (both DVD R2 PAL Spain) and in my case, they are the first DVD's that I found with the colorimetry flag setted and they are really, really new.

When I opened it with DGMPEGDec 1.4.5, I've seen that the Colorimetry field reported: ITU-R BT.470-2. When I saw this, I opened in version 1.5.3, and it reported the same info without asterisk so the stream was flagged, wasn´t it?.

I post this because when I read the threads and FAQ's always it is supposed or assumed that DVD works with 601, but it is not sure, am I wrong?.

If these two new DVD's are correctly flagged, then can we say that BT.601 is the correct color matrix for DVD's MPEG2?.

So when we feed the streams to CCE or HC it should also be set to 601 and not 709?.

I hope this thing helps, if not, just ignore it :).

Regards.

[frank]

BT.601 stands for all standard TV with tube (standard color space) an analog SCART interface.
CCE only encodes BT.601, so you have to use colormatrix in every case for 709 flagged input.

But if you own a modern player with upscaling and HDMI connected to a flat screen TV (only these can show the enhanced color space), then you can use 709 if you have such coder.

[Alex_ander]

CCE doesn't write colormetry data to file header (DGIndex will use * symbol for the assumed info for SD). It will produce the same YUV type, as on input, but the player will always use BT601 coefficients for RGB output, so yes, in case the source before encoding is BT709, you need ColorMatrix for turning it into BT601.

[manono]

When I opened it with DGMPEGDec 1.4.5, I've seen that the Colorimetry field reported: ITU-R BT.470-2. When I saw this, I opened in version 1.5.3, and it reported the same info without asterisk so the stream was flagged, wasn´t it?.

I post this because when I read the threads and FAQ's always it is supposed or assumed that DVD works with 601, but it is not sure, am I wrong?.
ITU-R BT.470-2 is exactly the same as ITU-R BT.601. There's nothing unusual about it and I see it from time to time.

[smok3]

my source is 0-255 RGB huffyuv avi, how should i use this filter for this encoders:
a. hcenc
b. x264
c. wmv
or should i just change the levels to 16-235? tnx.
(this is practical question)

[Adub]

What resolution are your target formats?

Are you going to be creating a DVD with this? Or is it going to be stored on a computer for personal use?

[tom942]

The idea of my initial post was just to give another two evidences that when DVD's MPEG2 is flagged, it reports BT.601. They also reports that are progressive, when they normally were wrongly flagged as interlaced, at least in PAL zone.

I just post it, because when I read the FAQ's about if a DVD uses BT.601 it is only assumed or supposed, but I don´t see a firm statement that says: it is sure that DVD's MPEG2 stream use BT.601 always when you are doing a DVD to DVD backup.

For instance, when I read the FAQ's it says that CCE doesn´t expect a concrete color matrix, but if I continue reading, then says that one people says that it expects 601 and others, 709. In the same FAQ, it says that HC expects 709, although confirmation is needed. Then, I don´t know what to do :).

As these two DVD's have been just released, I thought that it was a good idea to let you know this findings and I would like to know if it is good to "force" to use BT.601 when doing a DVD to DVD backup :).

Regards :)

[smok3]

Merlin7777:

yes, one export will be dvd, x264 will be for flash and one version will be HD, wmv will be web.

source is 1280x720, dvd will be anamorfic pal.

[tom942]

In summary:

If I use CCE, Basic or SP(2), to do a backup of a DVD, doing all the steps by hand or using DVD-RB (for instance), I should use BT.601 because it is ¿sure? that CCE expects as input this color matrix, shouldn´t I?

If I use HC, what color matrix should I use?, which one HC expects, 601 or 709?. I have read the FAQ, and it says 709, but it needs confirmation...

Any advice? :).

[Alex_ander]

There was a separate thread with some useful analysis:
http://forum.doom9.org/showthread.php?p=1193065#post1193065

As for what CCE expects, then according to its manual (SP), in some cases (when it needs conversion between YUV&RGB32) it uses BT601 coefficients. In case you feed YUY2 or any avi file that can be decompressed to YUY2, the manual doesn't mention any calculations of ColorMatrix' style.

[tom942]

@Alex_ander

I was re-reading the link (and some of the threads) that you gave me (this is the FAQ that I was talking about).

If I understood it right, when I'm doing a backup from DVD to DVD, using DGIndex, DVD-RB or other similar, there is no need to use ColorMatrix meanwhile I do not convert to RGB. I don´t talk of upconvert to HD or downconvert to SD.

CCE doesn´t expect a concrete color space. It doesn´t do internal color conversion while encoding.

The same applies to HC encoder, deesn´t it?

The doubt that I've got comes because in that link says HC expects 709, but reading Hanks's posts, he seems to say that HC doesn´t expect a concrete color space, is it right?

In the past I have used the line "Colormatrix (mode=rec.601->rec.709)" but if I undestand it right, I don´t need to use it anymore neither CCE nor HC, is it right?.

[Adub]

@smok3
You may get a kick out of this thread:
http://forum.doom9.org/showthread.php?t=133982&highlight=color+information+bt.601

However, here is what I believe is correct:

Use Rec.601 for HCenc. Be sure to set the flag correctly when you encode it with HCenc.

For the x264 flash, since it will be SD, I believe you also use Rec.601.
For the HD x264, use Rec.709.

For WMV, I have no idea. My best suggestion would be to try both and see which one looks better. This approach can be applied to all of the above encoding situations.


@tom942
Yes, it's sounding like you are right. There should be no need for colormatrix when going from DVD to DVD. The trick with CCE is that it wants YUY2 input, but that is about it. DVD-RB should handle this all correctly for you.

And if you look at HCenc's documentation, I believe that it likes Rec.601 for SD and Rec.709 for HD. Again, refer to the above link that I posted for more information.

[tom942]

Thank you Merlin7777 :).

I was reading that thread (that Alex_ander already point me before) and there was a point that confuse me:

What encoders expect BT.709?
HCenc, any other encoder that by default flags the output stream as BT.709
(verification would be appreciated)

But then, after re-reading nearly all threads, I found tests that Hank done with his encoder and I can undestand that it doesn´t expect anything. It takes what you give him.

http://forum.doom9.org/showthread.php?p=1061520#post1061520

As you said, I suppose that Hank is following the rule of 601 for SD and 709 for HD.

What I'm looking for is that I want do the correct steps with the apps. Later, if my DVD player do other things, it doesn´t mind me, because it wouldn´t be my fault.

Thank you for your time and answers :).

P.S.: This was the table I used to follow. I found it here some time ago:

Source_Colorimetry------------------Encoder-------------------Conversion_to_Use
Rec.709----------------------------CCE/HC/QuEnc-------------No Conversion or 'Rec.601 -> Rec.709 with hints=true'
Rec.601----------------------------CCE/HC/QuEnc-------------Rec.601->Rec.709 or 'Rec.601 -> Rec.709 with hints=true'
Rec.709----------------------------TMPGEnc------------------Rec.709 -> Rec.601 or 'Rec.709 -> Rec.601 with hints=true'
Rec.601----------------------------TMPGEnc------------------No Conversion or 'Rec.709 -> Rec.601 with hints=true'

As you can see, it was the opposite :).

[Adub]

Rec.709----------------------------CCE/HC/QuEnc-------------No Conversion or 'Rec.601 -> Rec.709 with hints=true'

That table seems screwed. Why would one use the 601->709 conversion on a source that is already 709? It makes absolutely no sense.

[tom942]

I don´t know.
It is a table that was in the forum before last year color matrix discussion. I found it when everybody assumed that DVD's MPEG2 was 709 due to the way that old versions of DGMPGDec report the absence of the colorimetry flag.

[smok3]

about luma range (and flash playback);

a. puting the whole 0-255 'RGB' range into yv12 -> x264 and then playing that back via flash seems wrong (the blacks are clipped? dunno...), doing the pc->tv conversion before will give more acceptable results (if little washed out)

about DVD;

b. i fed hcenc with squashed range (16-235) and again it seems a bit washed out (with mpui or vlc), but more black would look wrong again as it seems

:confused:

[fano]

I'm having some problem resing a 632*264 video to 1920*108p and using colormatrix to restore the right colors.
I see a little band on the right part of the image: it seems as a repetition of a part of image.

In this screenshot you can see clearly what is the problem:
http://img155.imageshack.us/img155/6725/garbagedresizehalfresax4.png

This the avisinth script that I'm using:


setMtmode(2)
ColorMatrix(mode="Rec.601->Rec.709")
ffdshow_setDAR(ffdshow_dar_x, ffdshow_dar_y)
MT("hqdn3d(2)", threads=4)
DisposeAxis(1920, 1080, ffdshow_dar_x, ffdshow_dar_y)
MT("Spline16Resize(da_w, last.height)", threads=4)
MT("Spline16Resize(last.width, da_h)", threads=4, splitvertical=true)


If I delete the colormatrix line or pospone it after resize the colored line disappear... but Colormatix after resize has not effect, right?

There is a known solution for this or maybe is a bug?

Thanks for your help,
fanoI

[tritical]

There's a bug in the mmx/sse2 routines causing it not to process the last (width&15) for sse2 and (width&7) for mmx worth of pixels for non mod 16 and non mod 8 widths respectively (you end up with whatever values happen to be at that those memory locations in the destination frame). A temporary fix is to set opt=0 to force the C routines. I'll put up a fixed version later tonight, it just needs one line changed.

[fano]

I've tried to add opt=0 and now works right!

Thanks for your help,
fanoI

[tritical]

I put up v2.4 on my website. Only change was fixing this little problem.

[fano]

Ok I'll try it tonight :)

[vucloutr]

hi, got a problem with colormatrix (using the latest 2.4).

LoadPlugin("..\DGDecode.dll")
DGDecode_MPEG2Source("file.d2v")

ColorMatrix(d2v="file.d2v")
Source is a MPEG2 HDTV .ts and I used DGIndex 1.5.3 for indexing.
With this script, I get an error message "ColorMatrix: d2v and filter frame counts do not match!"
Without the .d2v param and with info=3&hints=true i get strange colors in some frames.
normal.png (2.3 MB) (http://a.imagehost.org/view/0910/normal)
strange.png (2.1 MB) (http://a.imagehost.org/view/0561/strange)

any help is appreciated :)

[tritical]

If you turn on debug output what source colorimetry does it detect for the good frames and the bad frames? That d2v error about frame count mismatch will only trigger if multiple colorimetrys are present. Could you post a sample?

[vucloutr]

debugview log says: 00000173 14.75968838 [1248] ColorMatrix:1784: frame 86432: detected hint = 1 (Rec.709)
00000174 14.75977802 [1248] ColorMatrix:2908: frame 86432: using YV12 Rec.709->Rec.601 conversion (SSE2).
00000175 15.04918671 [1248] ColorMatrix:1784: frame 86433: detected hint = 4 (FCC)
00000176 15.04928112 [1248] ColorMatrix:2908: frame 86433: using YV12 FCC->Rec.601 conversion (SSE2).
first is the "normal" frame and second is the "strange" one.

here's a sample (.7z ~45MB) (http://www.megaupload.com/?d=37043OPD) (P4S5VV0RD: fw48czlqw94bgt g-f45tw-3.6mw.56w-v3mn)

[tritical]

There were two bugs causing problems. First was that the d2v parsing was never updated for v16 DGIndex project files. This didn't effect colorimetry detection in v16 d2vs, but frame count detection was incorrect (only a problem if multiple colorimetrys were present). Second, there was a bug in the mmx/sse2 routines used for FCC<->Rec.601 conversions that caused incorrect U plane values. Both problems are fixed in v2.5 on my website.

[vucloutr]

thanks!

edit: whoops ^^

[canuckerfan]

if a conversion from 601 to 709 has been done. can it be reversed by just calling colormatrix again and changing the destination to 601? I realize it would just be best to go back and not do the conversion in the first place.

[Wilbert]

if a conversion from 601 to 709 has been done. can it be reversed by just calling colormatrix again and changing the destination to 601? I realize it would just be best to go back and not do the conversion in the first place.
Sure. Use mode="Rec.709->Rec.601".

[leeperry]

http://thumbnails14.imagebam.com/5155/6cd6d751549075.gif (http://www.imagebam.com/image/6cd6d751549075)

ColorMatrix(mode="Rec.601->Rec.709")

http://thumbnails9.imagebam.com/5155/7e837451549079.gif (http://www.imagebam.com/image/7e837451549079)

why would it change the levels :confused:

[Keiyakusha]

Hi. I'd like to ask something about colormatrix...

Here (http://img230.imageshack.us/img230/3301/infoor.png) is info from DGIndex about my DVD.
Here (http://img156.imageshack.us/img156/7328/29471546.png) is a screenshot, taken in VirtualDub using avisynth script with only one line in it: Mpeg2source("VTS_01_1.d2v")
Here (http://img156.imageshack.us/img156/667/kkkkkkkk.png) is a screenshot of the same frame, but taken with following script: Mpeg2source("VTS_01_1.d2v",info=3).Colormatrix(hints=true,interlaced=true)
It looks like screenshots are the same, but here (http://img83.imageshack.us/img83/1112/13520879.png) is what I see after overlaying them using "difference" and boosting levels.

I expected to see no difference at all or to see much more difference... So all is fine here, this is how it should be? Am I doing something wrong?

[Keiyakusha]

After reading many nice posts by manono in this (http://forum.doom9.org/showthread.php?t=134684) thread, I think maybe the difference on my screens appears because of clamping some levels after applying colormatrix. Indeed such things shouldn't be done 'behind the back'.

[leeperry]

indeed, RTFM helped me a lot too :D

ColorMatrix(mode="Rec.601->Rec.709",clamp=0,opt=0)

[leeperry]

XhmikosR has been kind enough to compile an ICL11/SSE3 version of ColorMatrix, on the aforementioned call(on 4 threads w/ a 3.2Ghz Q6600) I get a 3% speed improvement(going from 593 to 609 fps): http://www.mediafire.com/?n2gumty1zyn

[Keiyakusha]

XhmikosR has been kind enough to compile an ICL11/SSE3 version of ColorMatrix, on the aforementioned call(on 4 threads w/ a 3.2Ghz Q6600) I get a 3% speed improvement(going from 593 to 609 fps): http://www.mediafire.com/?n2gumty1zyn

Thanks :goodpost:

[Wilbert]

XhmikosR has been kind enough to compile an ICL11/SSE3 version of ColorMatrix, on the aforementioned call(on 4 threads w/ a 3.2Ghz Q6600) I get a 3% speed improvement(going from 593 to 609 fps): http://www.mediafire.com/?n2gumty1zyn
Please, include the source. Thanks.

[XhmikosR]

So you need the ICL project files? Because that's the only thing that's different and that I changed the File Info to distinguish it from the normal build.
Anyway, here (http://www.mediafire.com/?sharekey=3f33c77c2cf9ce25ab1eab3e9fa335ca5d770fd2dc6da976) is the package with the source included.

[JoeH]

Thanks for your great tool. I use it all the time.

I was wondering if it would be feasible to prepare a 64-bit version. I have been looking at squid_80's 64 bit version of AVISynth and related 64-bit plugins that he has prepared, and he has everything I use in AVISynth ready in 64-bit except ColorMatrix.

I have no idea what that would imply, but if it is something feasible it would be great! Thanks again.

[leeperry]

hi tritical, I was wondering if you had any idea about this: http://forum.doom9.org/showpost.php?p=1341393&postcount=1751

problem of chroma alignment possibly? any idea how to get the same exact output as ffdshow please?

:thanks:

[yesgrey]

any idea how to get the same exact output as ffdshow please?
When you use t3dlut, the conversion between BT.601 and BT.709 coefficients is performed like this:
1) up-sample chroma from 4:2:0 or 4:2:2 to 4:4:4
2) convert BT.601 -> BT.709
3) sub-sample chroma from 4:4:4 to 4:2:0 or 4:2:2
I'm not sure about Colormatrix, but I think it will work similarly...
When you use ffdshow it's a different thing, because you are not converting YV12->YV12 or YUY2->YUY2, you are converting YV12->RGB32, and in this kind of conversion, you don't need to downsample the chroma at the end.

Summing up, you only could be able to get the same output as ffdshow if you use it like ffdshow. With t3dlut, you should set it for YV12->RGB32, and not for YV12->YV12 as you are performing in your examples. With Colormatrix, that's not possible, because it only performs conversions between YV12/YUY2->YV12/YUY2.

[leeperry]

ok, but we had the same problem when I wanted to get the same output from your LUT as I was getting from the PS script...and at some point, we did! you told me there was some hiccup in the LUT code and made a bugfix :o

I'll try to fix the 601>709 LUT conversion .ini file, and hopefully I'll have more details about what's "broken" to give to tritical.

[yesgrey]

hopefully I'll have more details about what's "broken" to give to tritical.
I don't think there's anything that tritical could "fix".
From what I understand, the difference between Colormatrix and t3dlut is that the later let's the user define which chroma placement to consider, while the former always assume it internally.
I believe the problem is due to the last step of chroma sub-sampling, so I don't think it could be fixed, because there is nothing to fix...;)

[leeperry]

well, maybe he could allow to choose the chroma alignment? :o

[yesgrey]

well, maybe he could allow to choose the chroma alignment? :o
First, try with t3dlut chroma alignment options. If it doesn't solve the problem, adding the chroma alignment to Colormatrix will not help also...

[leeperry]

First, try with t3dlut chroma alignment options. If it doesn't solve the problem, adding the chroma alignment to Colormatrix will not help also...
I've just tried all the possible settings in t3dlut/yv12toyuy2 and yuy2toyv12...what works the best is t3dlut(cplaceU=1) + yv12toyuy2(itype=0):

http://thumbnails23.imagebam.com/6152/1e2c9361511788.gif (http://www.imagebam.com/image/1e2c9361511788)

but if you compare to these, ffdshow still looks slightly better IMHO: http://forum.doom9.org/showpost.php?p=1341393&postcount=1751

still would be pretty cool if tritical could allow to change the chroma placement in ColorMatrix() http://forum-images.hardware.fr/images/perso/otakonleboss.gif

[tritical]

To match ffdshow you probably need to set itypeU=1 in t3dlut. Since by default t3dlut uses cubic interpolation for 4:2:2->4:4:4, which utilizes 4 pixels, so the end points of the kernel will still be spanning those color blocks. Anyways, I think basing what looks best on that one image is not a good idea. It contains big blocks of color with hard transitions and mpeg1 chroma placement... so of course duplication and mpeg1 chroma placement assumption in t3dlut/yv12toyuy2 will look best on it. However, I could easily generate an image where that combination will look much worse than cubic interpolation with mpeg2 chroma placement assumed. Anyways, with respect to colormatrix there is nothing to change in regards to chroma alignment since it does not upsample. It converts directly YUV->YUV, to get the UV values for computing the new Y samples it uses nearest neighbor. That is, in YV12 each set of 4 luma pixels ((x,y)(x+1,y)(x,y+1)(x+1,y+1)) uses the U and V pixels at (x>>1,y>>1) in the U and V plane respectively.

[leeperry]

ok, thanks for the feedback tritical...if you say that this doesn't matter(even on a wide projection screen), I wholeheartly believe you http://forum.slysoft.com/images/smilies/agreed.gif

[Shinji-kun]

I encoded an MPEG-2 file to MPEG-4 AVC (x264). Would colormatrix be necessary in this situation. The brightness does seem a little off.

[[DB-FR] Nikko]

You answer is here in the first message of this topic ^^:

ColorMatrix corrects the colors of mpeg2 streams of dvds. More correctly, those mpeg2 streams are encoded using a different set of coefficients as used by AviSynth's color conversion routines or by the XviD/DivX decoders, with the result that DivX/XviD clips or mpeg2 clips encoded by TMPGEnc/CCE using AviSynth or VirtualDub are displayed with slighty off colors (which looks like a small difference in brightness ).

[henryho_hk]

If I put "info=3" in mpeg2source() and "hints=true" in colormatrix(), do I need to specify "interlaced=true" in colormatrix() too?

[SilaSurfer]

If you are dealing with interlaced source I think yes.

[henryho_hk]

Oh, I thought the interlace information is put in the hints by "info=3". Thanks for the clarification. (But.... why isn't it?)

One further question about "clamp" and "outputFR". I am encoding AVI from DVDs for viewing on PC monitor (TFT LCD), using madVR (32bit) and VMR9 renderless (64bit XP). Should I specify both "clamp=0" and "outputFR=true" ?

[Wilbert]

Oh, I thought the interlace information is put in the hints by "info=3". Thanks for the clarification. (But.... why isn't it?)
Indeed it's not. It's not, because it's not made that way :) Using the interlaced flag from DGDecode/mpeg2source is not reliable way to determine whether the stream is trully interlaced.

From the documentation:
hints:

DGDecode v1.20 or newer can output colorimetry hints in the video stream which ColorMatrix can read in order to automatically determine the source coefficients.

[Boulder]

But doesn't the source being encoded as interlaced force the use of interlaced=true even if the stream is actually progressive?

[Wilbert]

But doesn't the source being encoded as interlaced force the use of interlaced=true even if the stream is actually progressive?
Do you mean 'force' as in forced by using hints? Then no. No interlaced information is passed by hints.

[Boulder]

I meant "force" as in "force the user to set interlaced=true in CM" since the chroma information has been encoded as interlaced anyway.

[leeperry]

Hi tritical,

Thanks for the great ColorMatrix(), I use it for upscaled SD through madVR :)

I was wondering, is there a way you could implement 16bit? like the hack recently added to SmoothAdjust() that I also use: http://forum.doom9.org/showpost.php?p=1504207&postcount=210

I run Avisynth 2.6 if that matters.

:thanks: for considering it!

[szabi]

Hi

I am using megui, it creates this line to aviscript, when my source is :
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\DGDecode.dll")
DGDecode_mpeg2source("C:\wedding\VTS_01_1.d2v", info=3)
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\ColorMatrix.dll")
ColorMatrix(hints=true, interlaced=true, threads=0)
My question what does "threads=0" mean?

Bye
szabi

[leeperry]

what does threads=0 mean?
it's all explained in the manual.

[steptoe]

Sets the number of threads Colormatrix will use for processing. Can be any value greater than 0 and, for YUY2, less than the frame height, for YV12, less than the frame height divided by 2. If set to 0, ColorMatrix will automatically detect the number of available processors and set threads equal to that value

So if you have a dual core processor, with threads=0 then it will use both cores to process the source, with quad core it will use all 4 cores to process the source and so on


I personally set it by hand then you know it really is using all CPU cores, so threads=4 would be the value set on my quad core

[szabi]

Thnx, for the explaination.
It helped me. :)

bye
szabi

[mandarinka]

Hi tritical,

Thanks for the great ColorMatrix(), I use it for upscaled SD through madVR :)

I was wondering, is there a way you could implement 16bit? like the hack recently added to SmoothAdjust() that I also use: http://forum.doom9.org/showpost.php?p=1504207&postcount=210

I run Avisynth 2.6 if that matters.

:thanks: for considering it!

For encoding where absolute speed isn't critical beyond some basic level, what would suffice would be some script for the most needed conversions between rec709 and rec601.
But of course, somebody who 1) knows what to do 2) can write such a function would be needed...

EDIT:

Actualy I missed this initialy, but since version 1.9, Dither (http://forum.doom9.org/showthread.php?p=1386559#post1386559) includes functions to convert between rgb and yv12, which can be coupled to perform a 16bit (stacked format) conversion between any of FCC, ITU-R BT.601, ITU-R BT.709 and SMPTE 240M.