ColorMatrix v2.3

[Wilbert]

ColorMatrix corrects the colors of mpeg2 streams of dvds. More correctly, those mpeg2 streams are encoded using a different set of coefficients as used by AviSynth's color conversion routines or by the XviD/DivX decoders, with the result that DivX/XviD clips or mpeg2 clips encoded by TMPGEnc/CCE using AviSynth or VirtualDub are displayed with slighty off colors (which looks like a small difference in brightness ). This can be checked by opening the mpeg2 stream directly in VDubMod. More info can be found in the readme.

This filter recalculates the yuv values assuming the coefficients which are used by AviSynth/VDub/DivX/XviD, with the consequence that your final encoding (mpeg2 or mpeg4) is displayed correctly.

http://forum.doom9.org/showthread.php?s=&postid=514595#post514595

Changelog v2.3 (by tritical),

- Calculate conversion coefficients for PB/PR rows from luma coefficients
- Added inputFR and outputFR parameters allowing for YUV range expansion/contraction
- Changed the clamp parameter from bool to int to allow specifying only pre clipping or only post clipping or both


more info: http://forum.doom9.org/showthread.php?p=887227#post887227

http://web.missouri.edu/~kes25c/

[malkion]

Wilbert, if this filter does indeed perform as you intended it, I would like to express some gratitude for future encodes.... :D

Now should I run limiter() before or after colormatrix() ?

[Teegedeck]

I've waited for this a long time! If only I had time right now to try it out...

A heartfelt 'thank you'. :)

[Boulder]

Thanks, this one will be very useful!

[Wilbert]

Now should I run limiter() before or after colormatrix() ?
If you get v1.1, you don't need to use limiter at all (not before or after the filter). The filter clamps input to [16,235] (and output also).

[sh0dan]

Made a few (easy) speedups - it should be a couple of times faster now.

Updated source (http://83.90.202.89/kpo/colormatrix_faster.zip).

- Use float instead of double. They have enough precision and are much faster. It can however be changed back by changing the typedef in ColorMatrix.h

- Simpler algorithms.

- Use internal limiter for output also, instead of very slow if-then.

- Better rounding (adding 0.5 for more exact float to int conversion)

No SSE/MMX this tim, though. :)

[jorel]

:cool:
thank you very much Wilbert !

i was "stuck inside a cloud" here trying to find a way for that problems!
now our encodes will get much, much more quality !
:)
downloading and testing! :D

[Wilbert]

@Sh0dan,

Thanks! I updated the version.

[Inc]

Thanks a lot Wilbert !

:)

[jorel]

Originally posted by Wilbert
@Sh0dan,

Thanks! I updated the version.

of course....

thank you too Sh0dan.:cool:

.seems that we posted at the same time and i saw your post after my last!

:)

[Dark-Cracker]

very nice work :)
just by curiosity does it will harm the final output if u apply this filter on HDTV source or on some raw avi input ?

++

[Wilbert]

just by curiosity does it will harm the final output if u apply this filter on HDTV source
I got a few hdtv sources. Marjority of them uses mpeg2 coefficients (checked with the newest GSpot). So, you should also use the filter on those sources.

However, a few of them uses mpeg1 coefficients. You shouldn't apply the filter on those.

on some raw avi input
Here, you shouldn't use this filter. DivX/XviD assume mpeg1 coefficients when decoding to RGB.

edit: btw, hdtv is YUV [0,255] of course. So you need to use ColorYUV("PC->TV") first.

[FuPP]

I have a question about colormatrix and speed. I use colormatrix() 1.2 and get the following results

with colormatrix : 16.5 fps
without colormatrix : 44.5 fps

should such a difference be expected or is there any problem ?

My script (source is 720x576) :

LoadPlugin("D:\video\avsfilters\yv12\DGDecode.dll")
LoadPlugin("D:\video\avsfilters\yv12\colormatrix.dll")

mpeg2source("d:\test\test.d2v",idct=6)

trim(30000,33000)
colormatrix()

FuPP

[Wilbert]

The problem is that the filter is not optimized. Since I'm not a good coder, I don't know how to do that. I hope someone has some time to do that :)

[Manao]

I added other easy optimisations ( no more float computations, only integer one ). Results *should* be the same as those given by sh0dan's version. It should be almost twice as fast.

http://jourdan.madism.org/~manao/ColorMatrix.zip

[Wilbert]

Thanks! Could you also add the correct source files?

[Manao]

I copy'n'pasted the wrong files :p

You can download it again.

[Wilbert]

Thanks, I added it to the first post! I didn't have time to test it though :)

[FuPP]

with colormatrix : 34.5 fps
without colormatrix : 44.5 fps

Far better ! Thanks a lot Manao ! And thanks a lot Wilbert for that filter :)

FuPP

[ltc]

How to properly use this filter?
Say if I want to re-encode a DVD to DVD-R for TV playing. I assume I should use YUV and TV scale in DVD2AVI. I can find the original matrix coefficient with GSpot which has following type:

I709 = type 1 = MPEG2 default
FCC = type 4 = FCC standard
I470 = type 5 = MPEG1 default
S170 = type 6 = SMPTE 1700M standard
S240 = type 7 = SMPTE 240M standard

I have some DVDs which has S170 coefficient. Should I use ColorMatrix before converting to ConvertToYUY2 for CCE and ConvertToRGB24 for TMPGE?

Some of my encodes are brighter than original. Could it be because I accidentally used RGB PC scale?

[Wilbert]

You need to use it in case your source is: MPEG2 default or SMPTE 240M standard (both set of coefficients are approximately the same).

The other set of coefficients are approximately the same and equal to the MPEG1 coefficients.

It doesn't really matter what your target is. If you use VDub or AviSynth, these matrix coefficient info is lost, and mpeg1 coefficients will be assumed during playback (at least for DivX/XviD/TMPGEnc/QuEnc, not sure about CCE).

Should I use ColorMatrix before converting to ConvertToYUY2 for CCE and ConvertToRGB24 for TMPGE?
Yes. Just directly after loading the source.

Perhaps you can make an avs script, encode it to MPEG2 with CCE, and post the coefficients for me?

[ltc]

Hmn, interesting. The source DVD is NTSC interlaced. Color coefficient is S170 (SMPTE 170M) according to GSpot.
I re-encoded to D1/2 size using KSVCD template. Encoded with tmpgenc the result video has FCC coefficient. Encoded with CCE, no coefficient present.

script:
Mpeg2Source("D:\TietNhonQuy_DVD\TietNhonQuy_01.d2v")
ColorMatrix()
#
crop(8, 0, -32,0)
GripCrop(352, 480, source_anamorphic=false)
GripSize(resizer="LanczosResize")

dgbob(FOrder(),mode=1 )
convert60ito24p(2,0)
DegrainMedian(mode=1)

converttoyuy2()

# Detect field order
function FOrder(clip c) {
fo= GetParity(c)? 1:0
return fo
}

[Wilbert]

Encoded with CCE, no coefficient present.
That's interesting. It means if you are doing DVD -> DVD using AVS_YUY2 -> CCE, the same coefficients are used (in both cases the default MPEG2 coefficients). Thus, in this case, you shouldn't apply ColorMatrix.

[Wilbert]

I put up a new version (v1.4). Changelog:

* Added a mode parameter (default is mode="mpeg2->mpeg1" which gives the same output as older versions).

In case you captured something or you have a XviD/DivX (both are encoded mpeg1 coefficients), and you want to encode it to mpeg2 using CCE (which assumes mpeg2 coefficients):

ColorMatrix(clip, mode="mpeg1->mpeg2")

* Added rgb=true, which converts to RGB24 using mpeg2 coefficients.

The following converts a YV12/YUY2 stream to RGB24 using MPEG-2 coefficients (which might be useful if you want to convert DVD to mpeg2 using TMPGEnc):

ColorMatrix(clip, rgb=true)

It should give the same results as

ColorMatrix(clip, mode="mpeg1->mpeg2")
ConvertToRGB24()

[Boulder]

Is rgb=true the default in the new version? The filter was slower than usual with just ColorMatrix(), adding rgb=false fixed that.

I'd also like to clear some things up, is this the correct way:

1) DivX/XviD clip to MPEG-2 in CCE : ColorMatrix(mode="mpeg1->mpeg2")
2) MPEG-2 capture (no coeffs shown) to MPEG-2 in CCE : ColorMatrix()

[Wilbert]

Is rgb=true the default in the new version? The filter was slower than usual with just ColorMatrix(), adding rgb=false fixed that.
Default is rgb=false.

Btw, rgb=true is the same as

ConvertToRGB(matrix="rec709")

which should be faster. I didn't know that when adding that option :)

1) DivX/XviD clip to MPEG-2 in CCE : ColorMatrix(mode="mpeg1->mpeg2")
Yup.
2) MPEG-2 capture (no coeffs shown) to MPEG-2 in CCE : ColorMatrix()
That's a tricky one. Of course, your cap application (assuming it's just an analog cap) doesn't know which coefficients should be used. Since pal/ntsc standards always (should) use mpeg1 coefficients (ITU BT.601-4), you should use
ColorMatrix(mode="mpeg1->mpeg2")

[Boulder]

Thanks, that cleared things up:)

[Wilbert]

I put up a new version. In the previous ones, the used coefficients were slightly wrong. Although the difference is not noticable in practice :)

btw, I corrected the naming convention. MPEG-2 coefficients are called Rec.709 and MPEG-1 coefficients Rec.601. Thus

ColorMatrix(clip, mode="Rec.709->Rec.601") (default)
ColorMatrix(clip, mode="Rec.601->Rec.709")

I advice you to use the internal

ConvertToRGB(clip, matrix="Rec.709")

instead of

ColorMatrix(clip, rgb=true).

Output is identical (thx IanB), but the former is faster.

[North2Polaris]

@Wilbert,

I recently used your filter on a Mpeg2 to Xvid conversion using AviSynth and VDubMod. The difference, to my eyes at least, was remarkable. I have wondered why past encodes were "less bright" than I expected and needed to be adjusted.

I took a look at the AviSynth plugins that come with Gordian Knot. I did not see ColorMatrix among them, which surprised me given the discussion in the current thread and the following threads:

http://forum.doom9.org/showthread.php?s=&threadid=78220&perpage=20&pagenumber=3

http://forum.doom9.org/showthread.php?s=&threadid=81191

Does Gordian Knot handle this problem another way or not at all?

Thanks for your work on this! Happy New Year!

North

[Wilbert]

Does Gordian Knot handle this problem another way or not at all?
No. Perhaps len0x is not aware of the problem. Did you ask him about this?

[North2Polaris]

Originally posted by Wilbert
No. Perhaps len0x is not aware of the problem. Did you ask him about this?

@Wilbert,

I just sent a PM to len0x about this.

I tried using LumaFilter in DGDecode to adjust the "brightness".

From DGDecode html,

Useful additional YV12 Filters :

To Darken Luminosity.

LumaFilter(-10,0.9)

To Ligthen Luminosity.

LumaFilter(+10,1.1)

Will lightening luminosity address the brightness problem addressed by ColorMatrix or just "cover it up"?

North

[Boulder]

The original problem is not actually the brightness, it's that the colors are off slightly.

[len0x]

Hm... Am I right in understanding that every DVD source needs to be converted to a proper color matrix before encoding into MPEG4?

*Edit* remark about RGB removed

[Wilbert]

Hm... Am I right in understanding that every DVD source needs to be converted to a proper color matrix before encoding into MPEG4?
Yup, well "to the same color matrix" as MPEG4 uses/assumes (Rec.601 coefficients).

[North2Polaris]

Originally posted by Boulder
The original problem is not actually the brightness, it's that the colors are off slightly.

@Boulder,

Thanks. When you don't know that the underlying problem is a color problem, it looks like a problem with brightness, which can lead to unsatisfactory attempts to correct it.

North

[len0x]

Originally posted by Wilbert
Yup, well "to the same color matrix" as MPEG4 uses/assumes (Rec.601 coefficients).

Bummer - now I have a dilemma since speed loss is at least 10% when doing correction so it might not be a good idea to make it default in GK/AGK...

Btw, do you have XML description of the filter? (I can add it to GK then)

[Wilbert]

Bummer - now I have a dilemma since speed loss is at least 10% when doing correction so it might not be a good idea to make it default in GK/AGK...
I wish I could do MMX :)

Btw, do you have XML description of the filter? (I can add it to GK then)
Not yet. I will do it this week and send you a pm.

[len0x]

Originally posted by Wilbert
Not yet. I will do it this week and send you a pm.

cheers

[Steve56]

ColorMatrix does virtually the same as BT709ToBT601 from http://mywebpages.comcast.net/trbarry/downloads.htm, or?

Steve56

[Wilbert]

I corrected the url above :)

Yes that's true, I noticed that too late. His plugin contains optimization code, so it might be faster. (btw, hdtv isn't always Rec.709 as suggested in his docs ...)

[Steve56]

Yes that's true, I noticed that too late. His plugin contains optimization code, so it might be faster. (btw, hdtv isn't always Rec.709 as suggested in his docs ...) [/B]ah, good to know ;-) I did now try to check with GSpot, if a certain MPEG-2 HDTV 1080i stream (Astra 1080 Konzert1), is encodet with the BT709 or BT601 matrix. But I don't know where to read this information from?!

Steve56

[Boulder]

Open the clip in GSpot v2.52b01.

[Wilbert]

1) That info is also available in 'DGMPEGDec 1.0.13 beta 11'. Using


mpeg2source(info=true)

2) Use latest beta of GSpot and look in the upper right corner. In the video section, the five boxes next to NVOP. The fives boxes are from left to right:

1) I709 (= Rec.709 coefficients)
2) FCC (almost the same as Rec.601)
3) I470 (= Rec.601 coefficients [an updated version of Rec.470-6, but coefficients are exactly the same])
4) S170 (= SMPTE 170M; exactly the same as Rec.601)
5) S240 (= SMPTE 240M; almost the same as Rec.709)

see table 6-9 page 56 of http://le-hacker.org/hacks/mpeg-drafts/is138182.pdf

If the boxes remain "black" then the used coefficients are not present in the header, and the default (which is Rec.709) should be assumed.

I hope it's clear now :)

[Steve56]

Wilbert: Thanks for the comprehensive answer!

Before that, I just did a color conversion judged by the look of it, when turned on or off. ;-)

[tritical]

Wilbert, is the source included in the zip linked to at the beginning of this thread the latest? I tried doing some simple optimization and got it running about 40% faster (about 210fps vs 145fps in YV12 on my comp) still with plain c code, but I'm wondering about a few things looking through it... in the constructor it checks mode against "Rec.709->Rec.601" and "Rec.601->Rec.709", yet in the main part it checks mode against "mpeg2->mpeg1" is this intended? In the YV12 mpeg1->mpeg2 it uses the "21" conversions and not the "12" conversions, but in the YUY2 code the mpeg1->mpeg2 does use the "12" conversions? Also, could you tell me or point me to a source that has the numbers for how the coefficients used were derived? I'm just curious, I tried to calculate them myself using coefficients from a few different websites, but they never work out to be exactly the same (usually off by 20-40 after multiplying by 65536). Thanks in advance.

Hm, must be because:

mpeg2source()
clip1 = last.colormatrix(mode="rec.601->rec.709")
clip2 = last.colormatrix(mode="rec.709->rec.601")
subtract(clip1,clip2)
coloryuv(analyze=true)

shows both clips to be identical :confused:.

[Wilbert]

in the constructor it checks mode against "Rec.709->Rec.601" and "Rec.601->Rec.709", yet in the main part it checks mode against "mpeg2->mpeg1" is this intended? In the YV12 mpeg1->mpeg2 it uses the "21" conversions and not the "12" conversions, but in the YUY2 code the mpeg1->mpeg2 does use the "12" conversions?
I corrected all that (v1.6). Thx very much. It's a bit stupid that I didn't notice this :confused:

Also, could you tell me or point me to a source that has the numbers for how the coefficients used were derived? I'm just curious, I tried to calculate them myself using coefficients from a few different websites, but they never work out to be exactly the same (usually off by 20-40 after multiplying by 65536).
I included a text file (coefficients.txt) which contains all calculations (if you know maple i can send you the worksheet). I hope that's clear enough.

Btw, some websites use slightly different coefficients, perhaps that could be the problem.

I tried doing some simple optimization and got it running about 40% faster (about 210fps vs 145fps in YV12 on my comp) still with plain c code
Great! Could you add those changes to v1.6?

[North2Polaris]

Originally posted by Wilbert
Yes that's true, I noticed that too late. His plugin contains optimization code, so it might be faster. (btw, hdtv isn't always Rec.709 as suggested in his docs ...)

@Wilbert,

Other than speed, any advantage to using ColorMatrix over BT709ToBT601?

North

[Wilbert]

Other than speed, any advantage to using ColorMatrix over BT709ToBT601?
Rec.601->Rec.709 conversion (useful for capping -> mpeg2) is possible in ColorMatrix :)

[Boulder]

Originally posted by Wilbert
Rec.601->Rec.709 conversion (useful for capping -> mpeg2) is possible in ColorMatrix :)
However, if you use CCE for encoding, no need to use ColorMatrix as it doesn't specify any coefficients for the encoded file for some reason?

[Wilbert]

However, if you use CCE for encoding, no need to use ColorMatrix as it doesn't specify any coefficients for the encoded file for some reason?
Only if you use CCE :) Remember pal/ntsc uses Rec.601 coefficients. TMPGEnc/QuEnc add fcc (approx Rec.709) coefficients to the mpeg-2 header.

[tritical]

I included a text file (coefficients.txt) which contains all calculations (if you know maple i can send you the worksheet). I hope that's clear enough.
That was exactly what I was looking for :), thanks.

Great! Could you add those changes to v1.6?
Yep, here: [link removed]... most the changes required rearranging things a good bit. You can just skim through it and see if you want to use anything from it. I also made mmx routines for the YV12 conversions (mainly stolen from tom's filter, with some minor changes). On my system the YV12 c code runs around 210-220fps, the mmx versions average somewhere around 350fps). However, due to rounding differences, the mmx and c output are not exactly the same. The maximum difference on the Y plane is +-2 and for UV its +-1. I included an mmx parameter to disable using mmx if you want to though (mmx=false).

[Wilbert]

Great! I updated the first post. Thanks very much :)

[Didée]

However, due to rounding differences, the mmx and c output are not exactly the same. The maximum difference on the Y plane is +-2 and for UV its +-1.[/B]
So, which one is closer to the "exact" value? With the actual performance the filter has reached by now, I don't care for the remaining speed difference (you know my scripts ;) ) - but the value differences could be of importance (ask jorel ... BTW, what happened to him?)

edit: forgot to say: Great work, thank you!

[Wilbert]

So, which one is closer to the "exact" value? With the actual performance the filter has reached by now, I don't care for the remaining speed difference
The c output, thus ColorMatrix(..., mmx=false).

[Dhampir]

I've done a screenshot using mmx and without mmx and I can't spot any difference between using mmx and not using mmx. Are the rounding errors just insignificant? Here are two screenshots.

MMX=false

http://img135.exs.cx/img135/9051/mmxfalse2aq.png

MMX

http://img141.exs.cx/img141/3314/mmxtrue3mj.png

[Wilbert]

The maximum difference on the Y plane is +-2 and for UV its +-1.
The difference is minor, you won't see it by visual inspection :)

[len0x]

Can this filter safely be used after resizing (for speed reasons) or better to put it before?

[len0x]

I just grabbed a bunch of DVDs I have at hand and actually only 50% of them are using default mpeg2 coefficients. There are considerable number of them that uses s170 and i470. Also I had one that uses FCC. So I guess its no good to just apply this filter to any DVD you have.

P.S. Is it possible to write a filter that returns coefficient type, so that conditional usage of ColorMatrix is possible?

[Didée]

Originally posted by Wilbert
The difference is minor, you won't see it by visual inspection :) OOOoooohh. Yes you will.

interleave(last.colormatrix(mmx=false),last.colormatrix(mmx=true))

Step back and forth, and tell me you see no difference. :)


edited stupidity ... grrr!

[tritical]

What about having dgdecode set a global variable colormatrix could read? To make it multi instance compatible the name would need to depend on the d2v file, so colormatrix would need the name of the d2v file as input as well. Good or bad idea?

Wouldn't it be:

interleave(last.colormatrix(mmx=false),last.colormatrix(mmx=true))

for Wilbert's point :confused:.

[Didée]

Originally posted by tritical
Wouldn't it be: Yes of course. Didn't have my brain with me. Corrected.

[Wilbert]

What about having dgdecode set a global variable colormatrix could read? To make it multi instance compatible the name would need to depend on the d2v file, so colormatrix would need the name of the d2v file as input as well. Good or bad idea?
Something like this should be possible. I will ask Neuron2 about it.

Can this filter safely be used after resizing (for speed reasons) or better to put it before?
Doesn't matter.

[jp80]

Hi,

I would like to use matrixcolor in Gordian Knot to encode an interlaced PAL DVD source to xvid. In order to do so I was told by len0x that I had to add this filter at the end of the AVS script. But as I'm a total newbie in modifying avs script with a filter in GK, could somebody please tell me exactly how to do all of those required steps :

-Which avs script should I modify, is it LastCompCheck.avs located at the GK root folder, or the one created after having set up the encoding options for a d2v file in GK? If it's the 2nd case, then at which moment could I do so? Because to me it seems GK starts the encoding as soon as the the customized avs is created?
-What do I need to write for colormatrix in the avs file, knowing that I want to output a 672x496 xvid from a interlaced PAL DVD ? For instance in this case, is it correct to write this?:

----------------------------------------------------------
LoadPlugin("C:\Program Files\GordianKnot\AviSynthPlugins\ColorMatrix\ColorMatrix.dll")
clip(width=720,height=576,pixel_type="YUY2",color_yuv=$954080).ConvertToYV12
ColorMatrix("Rec.709->Rec.601",mmx=false)
ColorMatrix("Rec.601->Rec.709",mmx=false)
ColorYUV(analyze=true)
-----------------------------------------------------------

And about the line "clip(width=720,height=576,pixel_type="YUY2",color_yuv=$954080).", should the width and height match the size of the input file, thus should I write width=720,height=480 for a PAL source? And does the name of "clip" matter and should have the same one as an intermediate file possibly created in the GK encoding process?

[gte024h]

Quick question:

If I am doing no colorspace conversions in my workflow (maintaining YV12 throughout) should I be using ColorMatrix?

From the readme, and the discussions here, it seems that the original MPEG2 source (DVD, DVB, etc.) should have one of many possible coefficient matrices declared in it's header and if not, Rec.709 is to be assumed. The problem is that the decoder (Divx or Xvid) assumes Rec.601. Would it be possible to just copy the flags in the source header into the output header (assuming there are no colorspace conversions in between), then hope that the decoder will honor those flags?

BTW, great work on this filter, thanks!

[Dark-Cracker]

@Wilbert

could you please let us know when DGindex will add a flag in the header of the .d2v file and when to apply colormatrix for the given flag :)

keep up the good work.

Bye.

[len0x]

Its not gonna be a flag, its gonna be a hint that ColorMatrix has to take advantage of (but that will work only for progressive sources).

@jp80

You complicate things a lot. I'm gonna reply to you in GK forum...

[Wilbert]

If I am doing no colorspace conversions in my workflow (maintaining YV12 throughout) should I be using ColorMatrix?

From the readme, and the discussions here, it seems that the original MPEG2 source (DVD, DVB, etc.) should have one of many possible coefficient matrices declared in it's header and if not, Rec.709 is to be assumed. The problem is that the decoder (Divx or Xvid) assumes Rec.601.
I think you gave the answer yourself here. If it's not clear you should propose specific examples, and then i will tell you if you should use it.

Would it be possible to just copy the flags in the source header into the output header (assuming there are no colorspace conversions in between), then hope that the decoder will honor those flags?
If you open your source mpeg2/avi directly in the encoder (application), then yes it should be possible to do this (i guess, but i'm not a coder). If you use AviSynth/dvdavi/vfapi/whatever in between then the frameserver should somehow preserve this info. We are working on this (see response of len0x).

[Dark-Cracker]

ok if colormatrix can find to enable it or not it's perfect i was thinking that a new field (somethink like matrix=709 will be added in the .d2v) thak you len0x for the answer :)

i will stay tuned :)

Bye.

[jp80]

Err did I do something wrong in my post, because I feel like everybody's ignoring it?

[Boulder]

jp80,

len0x told you that he will answer your question in the GKnot forum.

[jp80]

Oh ok sorry I didn't see his reply.

[len0x]

When comparing mmx=true vs false - it seems to me that mmx implementation always makes image darker. I wonder if rounding errors can be directed to make image lighter instead? (coz main usage of this filter will be to correct darker pictures). Mmx implentation is about 10% faster than C in my testing environment (i.e. overall speed of a script is considered), so its more attractive to use it, but not at a cost of still having darker image.

[neuron2]

@Wilbert

DGMPGDec 1.2.0 RC1 hints the colorimetry now. Refer to the Decomb source code for utilities.c for reading the hints. Use FieldDeinterlace() as your model. The colorimetry field is now bits 2-4 inclusive (zero-based). Here is the new header file:

bool PutHintingData(unsigned char *video, unsigned int hint);
bool GetHintingData(unsigned char *video, unsigned int *hint);

#define HINT_INVALID 0x80000000
#define PROGRESSIVE 0x00000001
#define IN_PATTERN 0x00000002
#define COLORIMETRY 0x0000001C
#define COLORIMETRY_SHIFT 2
@Dark-Cracker

If you open the D2V in DGDecode(...,info=1), you can see the colorimetry type displayed.

[tritical]

When comparing mmx=true vs false - it seems to me that mmx implementation always makes image darker. I wonder if rounding errors can be directed to make image lighter instead? (coz main usage of this filter will be to correct darker pictures).
Yes, this is possible. Adding a small adjustment factor to the pixel values before multiplying would shift the resulting output upwards.

[Dark-Cracker]

@neuron2
thank u for the answer. but the result will be hard to catch for an external sofware.

@wilbert
do you think it's possible to made colormatrix to work like :
1) detect if it need to apply the matrix convertion
2) if the conversion is needed apply it
3) if the conversion is not needed only apply the "limiter()" function.

this could allow to always add the colormatrix in the avisynth script without problems.

++

[neuron2]

Originally posted by Dark-Cracker
but the result will be hard to catch for an external sofware. Umm, what do you want, a telegram? :p

I output the info screen, the debug output, and hints. What else can I do?

[Dark-Cracker]

@neuron2
:) lol :) in fact i have to rewrite my previous msg :)

but the result will be hard to catch for an external vb6 sofware. :)

we have enough way to get the infos but in vb there is some limitations it's for that i have say there is no *easy* way to retrieve the info (for exemple somethink like a flag when you read the .d2v header).

but i am sure wilbert will come with an automatic solution and i will could add .colormatrix() in each my script and the filter will (like a big boy) speak with .dgdecode() and decid or not to apply the convertion.

and more :) this could avoid me to have to catch the debugoutputstring (note to myself : "i realize i am a lazzy guy").

++

[neuron2]

Glad you have a sense of humor...

A header line in the D2V won't do because it can change during the stream. It gets yucky to start including more dynamic stuff in the D2V.

I'm sure Wilbert will do it because he is the guy that asked me to add hinting. :D

[Dark-Cracker]

i have a good sense of humor but english is not my mother tonge and my jokes often turn into somethink ...hum not really funny so i prevent myself to made a lot of jokes in this forum :)

i was not thinking the matrix info could change during the stream but i think this could made sense when there is advertisments or trailers at the strart/middle of the video stream.

to sumup all my hopes are in wilbert :)

++

[tritical]

Well, I've been debating putting this up since I don't know if Wilbert is planning to release a new version soon, but I'll just put it up temporarily for those interested in testing. [link removed]. I changed the mmx routines so the rounding is almost a 100% match with the c routine, the difference is now truely indistinguishable. It is slightly slower due to the needed changes but not much, and it seemed like matching the c output was more important.

It also includes experimental support for reading dgdecode hints and an internal interlaced mode so hints are supported for that as well. Extra parameters are: bool debug, bool hints, bool interlaced (defaults: false, true, false). When reading hints, if hints are found then the conversion is only applied when needed... i.e. if you specify mode="rec.709->rec.601" and hints indicate its already rec.601 it doesn't apply the conversion. If no hints are found in the stream, it acts as if hints=false and always applys the conversion. You can use debug=true to check that it is finding the hints.

[neuron2]

Cool, tritical. Just wanted to remind people that they need DGMPGDec 1.2.0 RC2 and that they need to have info=2 on the MPEG2Source() invocation to generate the hints.

The interlaced/progressive hint just follows the pf bit, so if a frame is pulled down, it may not be accurate. I'm not sure how useful that hint is in a following filter. In Decomb it means something else.

[neuron2]

Oops, info=3 is required to output hints.

[tritical]

Yep, I was wondering about the useability of the progressive/interlaced hint from dgdecode if the stream has rff flags. Anyways, the version I put up doesn't check that bit atm (just the colorimetry)... so the interlaced setting needs to be set manually. In fact, whether or not the stream is interlaced only matters for yv12... if the input is yuy2 then it doesn't make a difference and interlaced can be set to false regardless.

[Wilbert]

Well, I've been debating putting this up since I don't know if Wilbert is planning to release a new version soon, but I'll just put it up temporarily for those interested in testing.
Cool thanks! I was away this weekend, and the server was down yesterday evening, so I couldn't look at it :)

so the interlaced setting needs to be set manually.
That's fine. I prefer to use it if the video has combing (which needn't be the case for interlaced material, you know this progressive encoded as interlaced stuff ...).

[len0x]

Great! Will try that one asap :) (not that I need hints, but more accurate mmx routine is always welcomed)

[len0x]

@tritical

I've done some tests and your new version seems to be faster when not using MMX comparing to the official 1.7. Did you change anything in C code?

[Wilbert]

@tritical


Mpeg2source("F:\TestStreams\avs\AguileraGrammies.d2v",info=3) # Rec.709
ColorMatrix(debug=true)


doesn't show any text, although I used dgmpgdec120rc2.zip and

Mpeg2source("F:\TestStreams\avs\AguileraGrammies.d2v",info=1)

gives Rec.709, and although the Rec.709->Rec.601 conversion is done :confused:

[tritical]

Hm, should have said it was the simple/plain kind of debug output, via OutputDebugString(). It doesn't write any text on the frames or anything fancy. The OutputDebugString output is working for me here, if its not working for you not sure what's up.

gives Rec.709, and although the Rec.709->Rec.601 conversion is done
Hm, not sure what your getting at :confused:.

I've done some tests and your new version seems to be faster when not using MMX comparing to the official 1.7. Did you change anything in C code?

That's interesting cause its a straight copy of the 1.7 source. It should in fact be a little slower if you don't set hints=false cause it will spend a little time checking for hints. May be just compiler differences :confused:.

[EpheMeroN]

I am quite scrambled in the brain when it comes to fully understanding this awesome ColorMatrix Plugin. I read all the posts twice now. Can someone clear it all up for me please?

I have an XviD that came from an HDTV Source. I want to re-encode it to DVD. The GSpot Beta doesn't tell me what coefficients are used. Below is the script I'm working on. I'd appreciate it if someone could tell me if it's correct, and if it needs some changes :-) .

Video = Trim(AVISource("C:\Documents and Settings\Blank\Desktop\savages.avi"),0,1433).FadeIn(25).FadeOut(12)
Blank = BlankClip(Video).Trim(0,59).Amplify(0.0)
AlignedSplice(Blank,Video,Blank)

EnsureVBRMP3Sync()
BlindPP()
ColorYUV(Levels="PC->TV")
ColorMatrix(mode="Rec.709->Rec.601")
LimitedSharpen()
LanczosResize(720,480)
UnDot()

Here's a sample of the video as well:
http://users.adelphia.net/~l337/hdtv-xvid-sample.avi

[Wilbert]

Hm, should have said it was the simple/plain kind of debug output, via OutputDebugString(). It doesn't write any text on the frames or anything fancy. The OutputDebugString output is working for me here, if its not working for you not sure what's up.
Ah, that explains why i didn't see anything. I will check if it's working for me.


I have an XviD that came from an HDTV Source. I want to re-encode it to DVD. The GSpot Beta doesn't tell me what coefficients are used. Below is the script I'm working on. I'd appreciate it if someone could tell me if it's correct, and if it needs some changes :-) .
1) Do you still have the hdtv source?

2) What encoder do you use to make this dvd?

[EpheMeroN]

1) Do you still have the hdtv source?
No I do not. It was a clip of something that a friend send me.

2) What encoder do you use to make this dvd?
Well, I have not encoded it entirely yet. I've only run a 1-pass test in QuEnc. I also have CCE.

Did my avi sample provide any help at all in determining the coefficients?

[Wilbert]

Did my avi sample provide any help at all in determining the coefficients?
No, your avi assumed Rec.601 during decoding (which is always the case). These are or aren't the same coefficients as used in your hdtv source. You need the hdtv source to check that.

QuEnc -> fcc coefficients ~ Rec.601 coefficients, CCE -> nothing = Rec.709 coefficients.

So assuming Rec.601 are the correct coefficients for your avi:

1) QuEnc: don't need to use ColorMatrix
2) CCE: ColorMatrix(mode="Rec.601->Rec.709")

I hope it's a bit clear :)

[len0x]

Originally posted by tritical
That's interesting cause its a straight copy of the 1.7 source. It should in fact be a little slower if you don't set hints=false cause it will spend a little time checking for hints. May be just compiler differences :confused:.

When hints=true it is a LOT slower (overall time is ~15-20% more). But I was testing hints=false for that reason. Anyway I like your compiler much more than Wilbert's now :)

[tritical]

When hints=true it is a LOT slower (overall time is ~15-20% more). I noticed that as well, but couldn't find anything obviously wrong. It shouldn't be having that much of an effect... I'm gonna look in to it tonight sometime.

However, since the d2v now has the colorimetry info, it might make since for colormatrix to have a d2v="" option and to be able to get the info from there instead. That would be faster and the filter could be placed anywhere in the filter chain. That's assuming the colorimetry is the same throughout the clip.

[EpheMeroN]

So, if I encode this XviD AVI to MPEG-2 via CCE, will I still have to use ColorYUV(Levels="PC->TV") to squish the color range to 16-235? Or does ColorMatrix do this automatically?

[Wilbert]

However, since the d2v now has the colorimetry info, it might make since for colormatrix to have a d2v="" option and to be able to get the info from there instead. That would be faster and the filter could be placed anywhere in the filter chain. That's assuming the colorimetry is the same throughout the clip.
That's a good idea.

A small request though :) I think it's a good idea if hints=true gives an error when there are no hints. In case the user is using a wrong dvd2avi/mpeg2dec3 version without knowning or realizing that.

[Wilbert]

So, if I encode this XviD AVI to MPEG-2 via CCE, will I still have to use ColorYUV(Levels="PC->TV") to squish the color range to 16-235?
Your XviD is already [16,235]. (btw ColorMatrix *clamps* the luma range to [16,235])

[EpheMeroN]

Your XviD is already [16,235].
Really? I always was to believe all DivX / XviD AVI files were encoded at 0-255. Interesting...

[tritical]

Updated the zip file linked to in my earlier post. Changes include: hints now defaults to false, it throws an error when no hints are detected in the stream when using hints=true, debug output was expanded, added d2v="" option to read the colorimetry info from a d2v file (you can't use both d2v and hints at the same time, only one or the other), if the d2v has multiple types of colorimetry info then it throws an error and on d2v's that have no colorimetry or are not dgindex project files v07 or higher it throws an error. Quick example:

colormatrix(d2v="myd2v.d2v")

[fewtch]

Uggh, I can't stand the look of this filter's output... seems as though black levels are messed up, or something (maybe that looks good on a TV set, but IMO not a PC monitor). Instead, I add something like this if movies look too dark:

Tweak(0.0,1.0,10.0)

Does the trick for me every time.

Edit --

Originally posted by Wilbert
(btw ColorMatrix *clamps* the luma range to [16,235])
Well, no wonder. I hope everyone examines this more limited luma range to see if they like it, because personally I hate it. Would much rather just boost brightness overall. Looks fine to my eyes on a TV set too (probably the DVP-642 automatically performs this luma clamping, and assuming almost any hardware player would do so).

P.S. if Colormatrix filter always clamps to TV standards, how is this "correcting" any picture? If unnecessary, reducing a dynamic range is not an error correction -- rather, it's introducing error. I wouldn't highpass my music files at 80Hz and lowpass at 16KHz without any reason, so why this?

[Wilbert]

@tritical,

Great! Thank you very much!

P.S. if Colormatrix filter always clamps to TV standards, how is this "correcting" any picture? If unnecessary, reducing a dynamic range is not an error correction -- rather, it's introducing error.
Relax man. I will add a pc_range=true option for you. You do understand i needed to clamp it when people are feeding it [16,235] YUV video (and most of them do). If you convert it to DivX/XviD/MPEG it needs to be [16,235] anyhow.

[yaz]

Originally posted by Wilbert
You do understand i needed to clamp it when people are feeding it [16,235] YUV video (and most of them do).would u, pls, make it clear! say, how do u know what ppl feed into cm ? (e.g. i always feed the cropped stream which is never(!) tv scale) & how do u make that 'clamping'? is it a kinda rescaling (as were made w/coloryuv) or just dropping simply the outer range?Originally posted by Wilbert
If you convert it to DivX/XviD/MPEG it needs to be [16,235] anyhow. why should it be ? and what if the input is not tv-scale ?

thx
y

[Wilbert]

would u, pls, make it clear! say, how do u know what ppl feed into cm ? (e.g. i always feed the cropped stream which is never(!) tv scale) & how do u make that 'clamping'? is it a kinda rescaling (as were made w/coloryuv) or just dropping simply the outer range?
Dropping outer range.

why should it be ? and what if the input is not tv-scale ?
It should be possible to make mpeg with luma range [0,255]-YUV. But during playback the luma range will be clamped before decoding to RGB when displaying it on a RGB monitor. When encoding to DivX/XviD the luma range will be clamped to [16,235]-YUV (by the codec itself).

(e.g. i always feed the cropped stream which is never(!) tv scale)
Such as?

[yaz]

Originally posted by Wilbert
Dropping outer range.bad news. that messes my later yuv manipulations :-( would u, pls, make it explicit in the 'readme'.Originally posted by Wilbert
It should be possible to make mpeg with luma range [0,255]-YUV. But during playback the luma range will be clamped before decoding to RGB when displaying it on a RGB monitor. When encoding to DivX/XviD the luma range will be clamped to [16,235]-YUV (by the codec itself). still can't get it. who will clamp ? the encoder or the decoder ? or both ? or neither ?
so, u state that my finely y(uv)-balanced clips are 'blindly' clamped by xvid ? ooouch ... i must check it ...Originally posted by Wilbert
Such as? i always (try to) tune my clips w/coloryuv so as to get a 'bearable' y-scale cus my eyes can't stand if white is 'too white' and if black is 'too black'(of course, it's not the color but the luma) it means a quite arbitrary y-scaling fitting to my taste but it's never 16-234 but rather sg round 8-247.

thx
y

[AsTimeGoesBy]

First, thanks for colormatrix filter!
But about that color range issue, i'm also a bit confused now...

Some basic questions to make understanding the posts here:

1) The PC is using a luma range from 0 to 255 - or let us say a scale with 256 steps. Now all that issue here is caused because the common TV is only using the steps 16 to 235 on that scale? So a TV has less steps/values to 'express' a certain luma level?

2) A DVD content (optimized for TV) has to be expanded for use on PC, but during encoding to Xvid/DivX it will be reduced again on lower TV range of 220 steps/values?

3) A decoder on PC expands/increases the smaller TV range (stroed in Xvid/DivX) to bigger PC luma range?

[Wilbert]

bad news. that messes my later yuv manipulations :-( would u, pls, make it explicit in the 'readme'.
Ok, i will. I will also add a pc_range=true (as i said) which leaves the luma range untouched.

still can't get it. who will clamp ? the encoder or the decoder ? or both ? or neither ?
For XviD/DivX it's the encoder, and for MPEG2/MPEG1 the decoder IF you request RGB from it (it clamps before converting to RGB).

Just make a Y=0,U=128,V=128 clip with AviSynth and convert it to XviD/MPEG. Open your encodings in AviSynth to check it with ColorYUV(analyze=true).

But about that color range issue, i'm also a bit confused now...
Ok, let's confuse you a bit more. The main problem is that this stuff *is* a big mess.


1) The PC is using a luma range from 0 to 255 - or let us say a scale with 256 steps. Now all that issue here is caused because the common TV is only using the steps 16 to 235 on that scale? So a TV has less steps/values to 'express' a certain luma level?
You are right if you say "The PC can be using a luma range from 0 to 255 - or let us say a scale with 256 steps." Whether it does depends on the codec. But in general, codecs assume CCIR input (ie YUV [16,235]), or use the conversion YUV [16,235] -> RGB [0,255] during decoding.

I say 'in general', but this is not always the case. If you request RGB from certain dv-codecs for example, they will use YUV [16,235] -> RGB [16,235]. Also TMPGEnc has an option (which is off be default) which let you do RGB [16,235] -> YUV [16,235].

2) A DVD content (optimized for TV) has to be expanded for use on PC, but during encoding to Xvid/DivX it will be reduced again on lower TV range of 220 steps/values?
DVD content is always stored as YUV with luma range [16,235]. Like i said most codecs assume YUV [16,235] -> RGB [0,255] during decoding (except some dv-codecs).

a) if you don't convert it to RGB in the processing chain, the luma range remains [16,235] (also when you encode to XviD/DivX).

b) if you convert it to RGB and back with VirtualDub or AviSynth, they assume YUV [16,235] <-> RGB [0,255] (at least by default). So, there's no problem here.


3) A decoder on PC expands/increases the smaller TV range (stroed in Xvid/DivX) to bigger PC luma range?
Usually people mean YUV [0,255] when talking about pc_range. But the video is converted to RGB when it is displayed on a PC. I think i answered above what happens to the luma range.

[North2Polaris]

@Wilbert,

I know that this is getting to be a bit off topic, but given your post above, what would be a reason to use levels="TV->PC"?

North

[yaz]

@north2polaris
afaik, levels('tv->pc') will rescale the chroma plane too, but here only the luma scale is considered. or am i wrong (again)?
thx
y

[Boulder]

Originally posted by Wilbert

Ok, let's confuse you a bit more. The main problem is that this stuff *is* a big mess.

Not to mention the fact that many DVDs have values in the 0-255 range:confused:

[yaz]

Originally posted by Boulder
Not to mention the fact that many DVDs have values in the 0-255 range:confused: yep ... i've just checked (rip, d2v, avisynth) 5 dvds of mine (1.vob only) and all(!) of them produced full(0-255) y-scale in coloryuv :confused: :confused: :confused:
that's true that they all open w/(16,128,128) plain black. but afterward, on real scenes (say, on intro titles, white letters on black) the scale is 0-255. who to believe now ?

thx
y

[Wilbert]

I know that this is getting to be a bit off topic, but given your post above, what would be a reason to use levels="TV->PC"?
No, I don't. (Might be useful if you want to make [0,255] YUV clips for some reason.)

afaik, levels('tv->pc') will rescale the chroma plane too, but here only the luma scale is considered. or am i wrong (again)?
No, you are right. I will also rescale the chroma. If you source is trully [0,255] YUV you should scale also the chroma. But if needed, there are ways to scale the luma only.

@Boulder and yaz

yep ... i've just checked (rip, d2v, avisynth) 5 dvds of mine (1.vob only) and all(!) of them produced full(0-255) y-scale in coloryuv
that's true that they all open w/(16,128,128) plain black. but afterward, on real scenes (say, on intro titles, white letters on black) the scale is 0-255. who to believe now ?
I believe you if you say the intro titles are Y255,U=128,V=128, but the intro titles are not part of the movie itself. However you should look at the movie itself. Could you give some examples of movies where the luma range is trully [0,255] (for scenes in the movie itself)?

[Boulder]

From "The Life of David Gale" :

http://www.saunalahti.fi/sam08/gale_1.jpg
http://www.saunalahti.fi/sam08/gale_2.jpg

[len0x]

Which mpeg coefficients does the DVD above use?

[yaz]

@boulder
yep ... sg similar is here w/me :confused:

@lenox
no sign in gspot so i guess rec.709 ( so i would use colormatrix() if i dared ;) )

the bests
y

[Wilbert]

@lenox
no sign in gspot so i guess rec.709 ( so i would use colormatrix() if i dared )
Doesn't matter :) (and yes you should use colormatrix ...)


From "The Life of David Gale" :

http://www.saunalahti.fi/sam08/gale_1.jpg
http://www.saunalahti.fi/sam08/gale_2.jpg
I'm not really convinced that those are trully YUV [0,255]. Ok, the luma of some pixels are way below 16, but it's only a small part of the pixels as you can see by looking at the 'loose minimum'. The loose minimum is 15 in both cases, which means that 255/256 of the number of luma samples have luma larger than 15.

Limiter(12,235,show="luma") colors the samples for which the luma is lower than 12 (with latest binary). Perhaps you can post some screenshots for the values 10, 12 and 14.

[Arachnotron]

The right half of the lowest line of pixels in gale_2 is pure black, though the line above it is part of the grey sweater. It looks like it may very well be te origin of the Y<16 samples. Try cropping off a few pixels at all sides and see what happens.

[Wilbert]

The right half of the lowest line of pixels in gale_2 is pure black, though the line above it is part of the grey sweater.
huh? There's no black line beneath her sweather (the upper line is black though).

[Boulder]

Originally posted by Wilbert

Limiter(12,235,show="luma") colors the samples for which the luma is lower than 12 (with latest binary). Perhaps you can post some screenshots for the values 10, 12 and 14.

Limiter_10
http://www.saunalahti.fi/sam08/limiter_10.jpg
Limiter_12
http://www.saunalahti.fi/sam08/limiter_12.jpg
Limiter_14
http://www.saunalahti.fi/sam08/limiter_14.jpg

As we can see, there are very few pixels with luma below 10, but still there are some. But they shouldn't be there if the DVD was encoded in the 16-235 range? 99% of the dark areas are in legal range though.

Maybe this is just something in the decoding part and not the actual video itself?

[fewtch]

Maybe my eyes are bad (!) but there's something "wrong" with the look of my DVD encodings when using colormatrix... dark doesn't look dark enough, and bright portions look "dimmed" on my PC monitor. If this clamping was happening automatically or was already on DVD, I wouldn't notice it (maybe it's my imagination?).

I do have a fairly high end monitor (Iiyama Vision Master Pro 410), maybe I'm noticing something some people aren't? Anyway, muchas gracias to Len0x for keeping it optional in AutoGK.

[Arachnotron]

@Wilbert: open it in paint, zoom in to 800%, enable the grid and scroll down. It is there allright in the right lower corner.

@boulder:As we can see, there are very few pixels with luma below 10, but still there are some. But they shouldn't be there if the DVD was encoded in the 16-235 range? 99% of the dark areas are in legal range though.

Maybe this is just something in the decoding part and not the actual video itself?16 is the value where black is, the value that on playback will result in an 0 IRE signal(PAL). That does not mean you cannot have values below black (superblack) or that a DVD player cannot output those signal levels. In fact some test DVD's use this to tune a TV set by including pure black background with fields just below and above back. The TV set is ok when the superblack just blends into the background but you can stil see the fields above black.
16 is just a convention where to put the blacklevel. Though many DVD players clamp at 0 IRE, so often you will not see those values (and those test DVD's fail)

The distribution of those values is interesting though. Could this be the result of a sharpening filter causing ringing or something like that? Ringing at an edge at Y=16 might result in values above and below that surrounding the edge.

[edit] tried it out by sharpening a black y=16 object on a dark grey background. After sharpening, the edge is outlined by pixels with Y<16

[Wilbert]

I finally updated the documentation and released ColorMatrix v1.8.

Note that the luma/chroma of the input clip is still clamped in this version. I will add a pc_range=true in a following version.

[Wilbert]

@Boulder,


As we can see, there are very few pixels with luma below 10, but still there are some. But they shouldn't be there if the DVD was encoded in the 16-235 range? 99% of the dark areas are in legal range though.
I don't really know the answer to this, so i'm just guessing a bit. In principle this is true. But as Arachnotron notes, and this post

http://forum.doom9.org/showthread.php?s=&postid=205324#post205324

indicates, processing (sharpening, resizing or whatever) changes the luma range (pixels with luma < 16 can be created). Perhaps this has something to do with it. Perhaps also the compression itself can cause this. But, i don't know much about how the dvd's are created and if it the luma range is clamped after processing ... But, we should be able to find people who know about this.

@Wilbert: open it in paint, zoom in to 800%, enable the grid and scroll down. It is there allright in the right lower corner.
Ok, i see it now.


Maybe my eyes are bad (!) but there's something "wrong" with the look of my DVD encodings when using colormatrix... dark doesn't look dark enough, and bright portions look "dimmed" on my PC monitor. If this clamping was happening automatically or was already on DVD, I wouldn't notice it (maybe it's my imagination?).
I guess you compared it by openening the vob directly in VirtualDubMod?

[Boulder]

Originally posted by Wilbert
[B]@Boulder,


I don't really know the answer to this, so i'm just guessing a bit. In principle this is true. But as Arachnotron notes, and this post

http://forum.doom9.org/showthread.php?s=&postid=205324#post205324

indicates, processing (sharpening, resizing or whatever) changes the luma range (pixels with luma < 16 can be created). Perhaps this has something to do with it. Perhaps also the compression itself can cause this. But, i don't know much about how the dvd's are created and if it the luma range is clamped after processing ... But, we should be able to find people who know about this.

I agree, there's probably people around here who are in the business or know people are there.

I myself didn't do anything but load the d2v project and use Limiter so the "illegal" pixels are either already in the source or introduced when decoding. Maybe Don knows something about this as he's working on the MPEG-2 decoder in question. I don't believe it's a rounding error as the difference is just too big for that.

Nevertheless, it's not a big problem as the stray pixels are quite rare;)

[Jeffster]

It looks like a bug may have been introduced into the latest version (1.8) of ColorMatrix?
I noticed it while converting a DivX source file to MPEG-2 in CCE with the parameter ColorMatrix(mode="Rec.601->Rec.709")

Here is a screenshot of the original without colormatrix (http://homepages.ihug.co.nz/~jefx/temp/without_CM.png) and another screenshot using colormatrix1.7 (http://homepages.ihug.co.nz/~jefx/temp/colormatrix_1.7.png), no problems there... but here is the same script this time using colormatrix1.8 (http://homepages.ihug.co.nz/~jefx/temp/colormatrix_1.8.png) instead.

:confused:

May I make a small suggetsion while I have your ears... for the mode parameter, changing it to something short like mode=1 or mode=2 would be much easier to remember than mode="Rec.601->Rec.709". Currently I have to open the html help file and copy and paste that line each time I use it in a script like the one above.

[tritical]

It looks like a bug may have been introduced into the latest version (1.8) of ColorMatrix?
I noticed it while converting a DivX source file to MPEG-2 in CCE with the parameter ColorMatrix(mode="Rec.601->Rec.709") Indeed, it's an mmx overflow problem with the rec601->rec709 conversion, the other conversion isn't effected. You can use mmx=false for now, I'll fix it right away. I forgot that in the rec601->rec709 mmx conversion there is a 4x scale factor on two of the coefficients and the max it can be after the adjustment for rounding is 2x without risking overflow.

[Jeffster]

Thanks... just confirming that mmx=false corrects it :)

[tritical]

Fixed version... colormatrix 1.8 - mmx fix (http://bengal.missouri.edu/~kes25c/ColorMatrix-18mmxfix.zip). Only change was fixing the overflow in the rec.601->rec.709 mmx conversion. Sorry about not catching that before.

[Wilbert]

Thx! Released v1.9.

[Karyudo]

Originally posted by Jeffster
for the mode parameter, changing it to something short like mode=1 or mode=2 would be much easier to remember than mode="Rec.601->Rec.709".

Or how 'bout mode=6to7 and mode=7to6? Or 627 and 726? Or 67 and 76?

[Boulder]

Would it be possible to just check the first appropriate line in the d2v file and then assume the coefficients do not change during the clip? This would make it slightly faster, probably not much but at least some.

[tritical]

Not sure how that would be any faster then using the current d2v="" option :confused:. It parses the full d2v file once when the filter initially starts and that's it. Checking only the first line of the d2v would only save a second or two when the script is first loaded at most.

[Boulder]

Oh, I thought it processes the file line by line as the encode goes on. In that case, forget what I said;)

The docs could use slight tuning, currently it says that mode is overridden when hints or d2v parsing is used. It might be better to say that the conversion is done if necessary according to what mode the user has specified and what the hints or the d2v file tell.

[Wilbert]

The docs could use slight tuning, currently it says that mode is overridden when hints or d2v parsing is used. It might be better to say that the conversion is done if necessary according to what mode the user has specified and what the hints or the d2v file tell.
I will change that. (The former is still true though, but perhaps a bit confusing :))

[redfordxx]

Please can somebody react on my post 11th March 2005 17:50 in
http://forum.doom9.org/showthread.php?s=&threadid=91238&perpage=20&pagenumber=2

Tnx

[Wilbert]

Of course I poked before... saw many expressions of gratitude etc... that it
helps to repair colors...

But, still I see what I see (No metrics, Didée!).
The capture frame from xvid.avi w/o ColorMatrix is much more similar to
capture frame from vob, than capture frame from xvid.avi with ColorMatrix. I
made same test as Japhsoncross before, moreover with ColorMatrix. (XviD, Q1,
no Bframes) And, as Intervideo makes some postprocessing already, I played
vob with FFDShow too.

I have GKnot 0.35 so I suppose I have DGDecode 1.2.1.
When I look in d2v, first line I see is:
c00 1 0 2048 1 1 b2 b2 92 b2 b2 a2 b2 b2 a2 b2 b2 a2 b2 b2 a2
so I suppose it's ITU-R BT.709. Mpeg2source(...., info=1) confirms it.
So it should be ColorMatrix w/o param. However, I tried ColorMatrix(d2v="D:\record\rip\lotr tt\1\tt1.d2v") too and result the same:

XviD FFDShow capture with ColorMatrix is significantly "brighter" than both Intervideo and FFDShow captures of vob. Conclusion - one of following:
1. FFDShow and Intervideo capture MPEG2.VOB incorrectly
2. FFDShow captures XviD.avi incorrectly
3. ColorMatrix does not do the job I suppose.
4. ???
Which one is it? Suggestions?
Could you make a VirtualDubMod capture of your vob and compare that? I have no idea how ffdshow the YUV->RGB conversion does.

[Malcolm]

does anyone know if there's a standard recommendation (rec601 or rec709) for DVB broadcastings? Neither GSpot, nor dgdecode with info=1 tells me anything about the matrix coefficients in my dvb recordings. so i don't know what to use (rec601 -> rec709, rec709 -> rec601, nothing at all).

Thank you and greetings,
Malcolm

[Boulder]

If you don't see any, rec.709 is assumed IIRC. You can always test it though.

[khmann]

Hi. 2 questions:

1. does anyone know the native matrix of NTSC DV? I guess 601?

2. does anyone know a way to "patch" an MPEG-2 if my stupid encoder doesn't specify the correct matrix? I'd rather avoid conversions if possible...

some info: according to GSPOT 2.5, TMPGENC 2.524 does not set matrix when encoding MPEG1. When encoding MPEG2, matrix is set to "S170".

this chart may be useful to folks trying to diagnose codec colorspace or luma levels: http://www.belle-nuit.com/testchart.html

thankyou!

[Wilbert]

1. does anyone know the native matrix of NTSC DV? I guess 601?
I think it's Rec.601. I suggest you drop a post in that forum to ask for confirmation.

2. does anyone know a way to "patch" an MPEG-2 if my stupid encoder doesn't specify the correct matrix? I'd rather avoid conversions if possible...
No, not yet :(

some info: according to GSPOT 2.5, TMPGENC 2.524 does not set matrix when encoding MPEG1
MPEG1 is always Rec.601. The header doesn't contain such info.

When encoding MPEG2, matrix is set to "S170".
Thx for the info. It's exactly the same as Rec.601 (older versions of TMPGEnc used FCC).

[Daijoubu]

If we only need Rec.709->Rec.601, can we use BT709ToBT601 (http://forum.doom9.org/showthread.php?threadid=50588) instead?
It's faster but the output is slightly different

Which one is more accurate?

[Didée]

Originally posted by Daijoubu
If we only need Rec.709->Rec.601, can we use BT709ToBT601 (http://forum.doom9.org/showthread.php?threadid=50588) instead?
I haven't (re-)tested this one for a long time. But I remember that it once gave me strange results on e.g. some dark blue, noisy backgrounds. Looked somewhat like a mixture of blocking & swimming-wall-effect ... I stopped using it when I discovered that.

[Daijoubu]

Originally posted by Didée
I haven't (re-)tested this one for a long time. But I remember that it once gave me strange results on e.g. some dark blue, noisy backgrounds. Looked somewhat like a mixture of blocking & swimming-wall-effect ... I stopped using it when I discovered that.
Stil version 0.1.0.0, no update since 2003/04/07
I guess I can take your word on this and stick to the more recent ColorMatrix :)

[tritical]

BT709ToBT601 uses slightly different coefficients then colormatrix... I think the coeffecients from Wilbert's calculations are more accurate. Also, BT709ToBT601's mmx code doesn't do any correction for rounding, which is the main reason that is quite a bit faster then colormatrix's mmx code. Finally, BT709ToBT601 does not call limiter (or do any sort of value clamping), which colormatrix calls twice (once at the beginning and once at the end). So the output from BT709ToBT601 will definitely have some differences to that of colormatrix.

[redfordxx]

Originally posted by Wilbert
Could you make a VirtualDubMod capture of your vob and compare that? I have no idea how ffdshow the YUV->RGB conversion does. OK, finally I made these tests...
For VDM I made Printscreen.
For BSplayer I made captures by the SW.
Equations below:
= means (almost) same color
<> noticably different color

VDM:vob=XviD(ColorM.avi)<>XviD(WithoutCM.avi)

BSPlayer:Intervideo(vob)=FFD(vob)=XviD(WithoutCM.avi)=FFD(WithoutCM.avi)<>XviD(ColorM.avi)=FFD(ColorM.avi)

Morover when I set in BSP default color setting (i.e. Brightness=7,5) then
BSPlayer(ColorM.avi)=VDM(vob)

MPClassic has completely different colors than VDM or BSP and I don't know it that much to find some color settings...

Uh oh...

[redfordxx]

As somebody might know:), during my script I want to go to RGB32 to deblock my video...
Doing this and applying ColorMatrix makes three choices --- I think:

1)
ConvertToRGB32("pc.709")
ConvertBackToYUY2("pc.709")
*ConvertToYV12
ColorMatrix

2)
ConvertToRGB32("pc.601")
ConvertBackToYUY2("pc.601")
*ConvertToYV12
ColorMatrix

3)
ConvertToRGB32("pc.709")
ConvertBackToYUY2("pc.601")
*ConvertToYV12

*now I can't remember, whether I tested it with ConvertToYV12

I thought 1) and 2) should be equal and 3) better quality (one conversion less). But in 3) I mentioned sllllllight gradients here and there compared to 1) or 2). Weird...

[Wilbert]

1) bsplayer and ffdshow do it "wrong". I will ask the authors to do it correctly.

2) In theory, the scripts:

a)
ColorMatrix

b)
ConvertToRGB32("Rec709")
ConvertBackToYUY2()

should give the same results. Isn't that the case? Nb, your scripts (using "pc.xxx") make sense if your source is YUV [0,255]. But I don't know why


I thought 1) and 2) should be equal and 3) better quality (one conversion less). But in 3) I mentioned sllllllight gradients here and there compared to 1) or 2). Weird...

I have to look at that.

[redfordxx]

Originally posted by Wilbert
ConvertToRGB32("Rec709")
ConvertBackToYUY2()

should give the same results. Isn't that the case? Nb, your scripts (using "pc.xxx") make sense if your source is YUV [0,255].
when I have in dgdecode the option PCScale? Or should I have there better TVScale anyway?

[Wilbert]

when I have in dgdecode the option PCScale? Or should I have there better TVScale anyway?
That option doesn't do anything when you open the d2v in AviSynth (since no conversion to RGB takes place).

Btw, PC-scale means that the conversion YUV [16,235] -> RGB [0,255] *might* take place *if* a codec requests RGB from the d2v (like TMPGEnc or vfapi, but it doesn't work there (meaning they don't look at the scale setting , and thus the pc-scale is used since it's the default)). I hope this is clear :)

I tested this myself (one of the threads in the dvd2avi forum).

[ronnylov]

Mainconcept encoder 1.4.2 can set the type of colormatrix used at the encoding. Here's what the pdf manual say:


Color Primaries: This field specifies the x,y chromaticity coordinates of the source picture primaries. It is strictly an informative flag to the video decoder and does not affect the video encoding at all. DVD specifies a value of 2 (ITU-R BT.470-2 System M) or 4 (SMTPE 170M) for NTSC or 3 (ITU-R BT.470-2 System B,G) for PAL. See ISO/IEC 13818-2 section 6.3.6 for more information. This option is only valid for MPEG-2.

Transfer Characteristics: This field specifies the opto-electronic transfer characteristics of the source picture. It is strictly an informative flag to the video decoder and does not affect the video encoding at all. DVD specifies a value of 2 (ITU-R BT.470-2 System M) or 4 (SMTPE 170M) for NTSC or 3 (ITU-R BT.470-2 System B,G) for PAL. See ISO/IEC 13818-2 section 6.3.6 for more information. This option is only valid for MPEG-2.


I capture in PAL-B format as huffyuv and feed the avi file through avisynth which use MPEG1-coefficients as default which should be the same as option 3 above? If this is correct colormatrix should not be needed when creating PAL DVD or when opening PAL DVD sources in avisynth? Because for PAL DVD only option 3 is possible?

EDIT: I did a test. The original avisynth script opened in virtualdubmod looks exactly the same as the encoded mpeg-2 file opened directly in virtualdubmod. So I guess I don't need colormatrix in this case.

[freelock7]

THX Wibert!This filter recalculates the yuv values assuming the coefficients which are used by AviSynth/VDub/DivX/XviD, with the consequence that your final encoding (mpeg2 or mpeg4) is displayed correctly.

Very fine results!
I noticed that CCE encoded colors was too satured before that.
So I think that the next release of avisynth could be corrected(?).

[charleski]

I only came across this yesterday after looking at the output of AutoGK and it gave me a bit of a shock.

I have a question, though. What about encodes from PAL DVB material? After scratching around a bit on the web, I get the feeling that these should already be using 601 coefficients. Is that right?

[Didée]

Probably not - check the colorimetry info. For most of the material I capture from DVB-S, DGindex is reporting BT.709 ...

/*hands over smelling salts*

[charleski]

Ah yes, you're right. d2V's made from remuxed mpegs were coming up as Colorimetry: Unspecified Video (2) in DGDecode, but I looked at an original capture and it's 709.

Aargh!

[Wilbert]

New version, v1.10. See first post for download and changelog.

[actionman133]

Hey, I have a question about the use of ColorMatrix. It seems like a simple one, but having searched through about 75% of this thread (and the others that link here too, it's all too big to read everything), I still don't have an answer.

I have a DVD source that DGDecode reveals to be stored using SMPTE 170M (6 in the d2v file). According to your table about the various colorimetrics, it's the same as ITU-R BT.601 (which I assume is the technical name for Rec.601).

My question is, do I need to run ColorMatrix on this clip (or for any clip where the original coefficients are the same as rec.601)? Logic says no, but I did a Subtract (Original.ColorMatrix (d2v = "blah.d2v", mmx = false), Original) and there are some discrepencies. Granted they are minor but I am video purist, and if ColorMatrix is throwing it off a little, I'd rather save the CPU cycles for better playback. :cool:

Thanks, and apologies if this has been answered before. Perhaps a FAQ is in order where ALL the information is. :)

[Wilbert]

Depends on your end format. But if you are encoding to DivX/XviD you don't need to run Colormatrix in your case.

Try the following script to spot any difference:

Original = Mpeg2Source(...).Limiter()
Subtract(Original.ColorMatrix (d2v = "blah.d2v", mmx = false), Original)

Your dvd can have luma values outside [16,235], which will be clipped by ColorMatrix. That's one of the differences you should see in your original script.

If you still see a difference, please upload a small part (20 frames) of that vob.

[actionman133]

Yes, it was for DivX at the end. And Limiter removed those little errors, so the difference was just your filter's clamping. So it's all good.

Thanks Wilbert. :)

[tritical]

I've made a test version of colormatrix that is able to do all colorimetry type conversions (not just BT.709->BT.601 and BT.601->BT.709) with exact coefficients. It also has 3 options for value capping and coefficient scaling:

0.) cap input/output to [16,235(240)] range, and scale conversion coefficients (219/255,224/255)
1.) cap input/output to [0,255] range, and scale coefficients to (219/255, 224/255)
2.) cap input/output to [0,255] range, and don't scale coefficients (255/255)

The current version is always doing the equivalent of option 0. Would anyone be interested in this? If so I'll finish it up (mmx/sse2 conversions aren't finished), if not then I wont :).

[colinb]

I'll shortly be using HC to encode some DV (camcorder) footage. I'll be feeding the DV AVI file to HC via an avisynth script.

Do I need to use colormatrix() in my avisynth script?

It's PAL DV if that is significant.

[TheBashar]

Just a quick note. I saw in the docs that you (Wilbert) had never come across a mixed colorimetry source. I've been transferring some of my TV DVD collections to x264 and I've found some of the season 1 SG-1 DVDs have mixed colorimetry. Thanks for the hints option!

[colinb]

I'll shortly be using HC to encode some DV (camcorder) footage. I'll be feeding the DV AVI file to HC via an avisynth script.

Do I need to use colormatrix() in my avisynth script?

It's PAL DV if that is significant.

Well, I tried loading both the original DV AVI file and HC encoded m2v files into VirtualDub side by side and they appeared identical - this was without a call to colormatrix.

This surprised me because after a bit of searching I found that DV is supposed to be Rec.601 while HC expects its input to be Rec.709, so I would have thought that a conversion rec601->rec709 would be required before feeding HC.

So I then tried passing the original DV AVI file through a colormatrix 601 to 709 conversion and displayed this in VirtualDub alongside the original DV file (without colormatrix conversion). They looked almost identical too. Neither looked darker than the other. I did notice that on frames with intense red/orange these areas seemed slight darker or perhaps just more saturated on the 601->709 converted footage. This was the first time I have tried colormatrix and I was expecting to see a slightly more noticeable difference.

Anyway it looks like for PAL DV at least I don't need to use colormatrix - but I'm not sure why.

[Wilbert]

Just a quick note. I saw in the docs that you (Wilbert) had never come across a mixed colorimetry source. I've been transferring some of my TV DVD collections to x264 and I've found some of the season 1 SG-1 DVDs have mixed colorimetry. Thanks for the hints option!
Great! Could you upload a small vob which has this mixed colorimetry info?

This surprised me because after a bit of searching I found that DV is supposed to be Rec.601 while HC expects its input to be Rec.709, so I would have thought that a conversion rec601->rec709 would be required before feeding HC.
Sounds right (and yes, the difference is small). Could you upload a small part of that dv (10 frames)?

[EpheMeroN]

I have HD content that I am encoding to DVD using CCE 2.70, and was looking into using ColorMatrix on my sources.

DGIndex reports back a Colorimetry = ITU-R BT. 701 (1)

CCE also, by default, assumes ITU-R BT. 701 (1) coefficients correct? So, is there any reason to still use ColorMatrix? Or do I have it all wrong?

[Wilbert]

CCE also, by default, assumes ITU-R BT. 701 (1) coefficients correct? So, is there any reason to still use ColorMatrix? Or do I have it all wrong?
No, you are correct.

[actionman133]

I've made a test version of colormatrix that is able to do all colorimetry type conversions (not just BT.709->BT.601 and BT.601->BT.709) with exact coefficients. It also has 3 options for value capping and coefficient scaling:

0.) cap input/output to [16,235(240)] range, and scale conversion coefficients (219/255,224/255)
1.) cap input/output to [0,255] range, and scale coefficients to (219/255, 224/255)
2.) cap input/output to [0,255] range, and don't scale coefficients (255/255)

The current version is always doing the equivalent of option 0. Would anyone be interested in this? If so I'll finish it up (mmx/sse2 conversions aren't finished), if not then I wont :).

If you created a more comprehensive and precise filter (not to say Wilbert did a bad job, it's a great filter!), I would definitely use it. I am a precision control freak, so those extra options would be much appreciated. :D

[EpheMeroN]

No, you are correct.
Good to know!

...but do you know if HD broadcasts use a range of 0-255 or 16-235?

[jsquare]

I have an old Samurai movie in B/W, when encoded with Xvid it looks a bit too bright and washed out, DGIndex is reporting the colorimetry as "ITU-R BT.470-2" and the frame type seems hybrid(it changes from Interlace to Progressive constantly). By adding ColorMatrix() to the script my encode looks acceptable, but I have a couple of questions:

Is the default command ColorMatrix() all I need or should I include some other settings?

Should I add ColorMatrix() to all my future encodes?

[Boulder]

Does ColorMatrix even affect BW stuff?

[jsquare]

Does ColorMatrix even affect BW stuff?
Probably not, but it seems to "clamp" Luma/Chroma to CCIR-601 values as stated on the docs.

[Boulder]

That's a job for Limiter()..I'm sure using it is faster than using ColorMatrix with B/W stuff.

[Oline 61]

I don't quite understand this. If I have a DVD that uses BT.709 according to DGIndex, do I need to use Colormatrix? If so, what settings?

[Wilbert]

What is your final format? What is your encoder?

As an example: If you are encoding to mpeg-4 (which uses BT.601), you will need the conversion.

[Oline 61]

I am just going AVISynth to MeGUI/x264 CLI. Another thing I just noticed is that when I preview the VOB in DGIndex it says BT.709, but after it finishes creating a d2v project file it says SMPTE 170M. So what would be the proper settings for me to use?

[Wilbert]

I am just going AVISynth to MeGUI/x264 CLI. Another thing I just noticed is that when I preview the VOB in DGIndex it says BT.709, but after it finishes creating a d2v project file it says SMPTE 170M. So what would be the proper settings for me to use?
If true, this would be a bug in DGIndex (since SMPTE 170M = BT.601, and SMPTE 240M = BT.709). Could you check this with the latest DGIndex?

[Oline 61]

Using latest DGIndex version:
The first picture is the windows after completely saving the d2v project.
http://img467.imageshack.us/img467/5964/untitled20tb.jpg
The second picture is a screenshot while running a preview.
http://img382.imageshack.us/img382/9803/untitled8br.jpg

[Oline 61]

Is there a definitive way to tell which of these colorimetries is correct?
BTW, when I try to use ColorMatrix(d2v="VTS_01_1.d2v")
it tells me that multiple colorimetry types were detected in the d2v file.

[Wilbert]

BTW, when I try to use ColorMatrix(d2v="VTS_01_1.d2v")
it tells me that multiple colorimetry types were detected in the d2v file.
Yes, neuron2 hinted at this possibility. You need to find out, by scrolling through the video, when the colorimetry changes. I guess the main movie is BT.709, but after the credits some black frames are SMPTE 170M? Please confirm.

If that's indeed the case, i would apply ColorMatrix to the whole movie (it doesn't do anything when a frame is black, since it corrects chroma).

[Oline 61]

Looks like your right. If I play it starting near the end it changes to 170M right before reaching the end of the file. Thanks for helping, I will use ColorMatrix()

[Boulder]

Another option is to enable hints output in MPEG2Source and set ColorMatrix accordingly.

[15081947]

After going through this entire thread I am trying to tabularise the colormatrix usage for Divx/Xvid conversion.

Anything -> Divx/Xvid
------------------------
Source Colorimetry--------ColorMatrix
ITU-R BT.470-----------------No
ITU-R BT.601-----------------No
ITU-R BT.709-----------------Yes
SMPTE 170 M-----------------No
SMPTE 240 M-----------------Yes

Please feel free to correct it.

Thanks

[Wilbert]

Please feel free to correct it.
It's correct.

[Ginsonic]

Hi Wilbert and Tritical,

Thank You for Your great ColorMatrix-Plugin !

@Tritical:
The current version is always doing the equivalent of option 0. Would anyone be interested in this? If so I'll finish it up (mmx/sse2 conversions aren't finished), if not then I wont
I would be very interested in such a version, it would be great, if You could release it.

[EpheMeroN]

I have an XviD file that was converted from an HDTS .ts file, but ColorMatrix was not used in the script.

Will adding ColorMatrix back in a script to do XviD to DVD correct the colors?

[Wilbert]

Depends. What are the coefficients of your original transport stream?

[EpheMeroN]

Depends. What are the coefficients of your original transport stream?
ITU-R BT. 701 (1)

[Wilbert]

... ITU-R BT. 709 i presume :)

In that case you don't need to correct it (the errors cancel each other out), provided you feed YUV to the mpeg-2 encoder (CCE, QuEnc or HC).

[EpheMeroN]

... ITU-R BT. 709 i presume :)

In that case you don't need to correct it (the errors cancel each other out), provided you feed YUV to the mpeg-2 encoder (CCE, QuEnc or HC).
Is YUV YUY2/YV12 colorspaces?

...and yes, you presumed correctly. :)

[Wilbert]

Is YUV YUY2/YV12 colorspaces?
Yes, it is.

[FredThompson]

How about adding the option to set clamping to either the default of CCIR-601 or simple overflow protection?

[Boulder]

Wilbert,

there seems to be a problem when using hints and either resizing or doing a colorspace conversion, the hints are lost.

See this one for more info: http://forum.doom9.org/showthread.php?p=788356#post788356

[tritical]

there seems to be a problem when using hints and either resizing or doing a colorspace conversion, the hints are lost.
The hint information is stored in the least significant bit of each of the first 64 bytes of the image. For YV12 that means the first 64 luma pixels (Y plane). However, in YUY2 it would be the first 32 luma pixels + 16/16 U/V pixels since YUY2 is a packed format (YUYV). Therefore, doing a YV12->YUY2 or YUY2->YV12 colorspace conversion will make the hints unreadable in the new colorspace. This could be avoided if hints were always stored in the first 64 luma pixels (not just the first 64 pixels), but not sure if its really worth the trouble. Also, any operation that could modify the values of the first 64 pixels could destroy the hints... so resizing, denoising, color correction, trimming that removes the top line or pixels off the left, etc... should all be avoided. Though if you really need to do this there is a filter in TIVTC called MergeHints() that can maintain these types of hints across such filtering and color space conversions by transferring the hint information from a saved clip to a new clip.

I would be very interested in such a version, it would be great, if You could release it. I'd meant to send it to Wilbert to take a look, but I've been busy the last month or so and haven't had the time to finish it. I did finally sit down today and finished the asm routines and made the d2v option able to work with sources that have multiple colorimetry types... so maybe in the next couple days.

[Wilbert]

there seems to be a problem when using hints and either resizing or doing a colorspace conversion, the hints are lost.
That's normal. It's the same story with Decimate and Telecide and why neuron2 advices you to place the two next to each other.

I don't know precisely how it works, so i hope tritical will reply. But hints are added at the start of the stream. Sometimes filters will destroy those hints and sometimes not. I recall tritical made once a filter for this purpose, which can store those hints and pass them to a filter which comes later in the filter chain. I can't find that filter though.

edit: too late :)

[Boulder]

OK, I'll have to refrain from resizing before ColorMatrix.

I've learnt two new things today, not bad ;)

[Wilbert]

tritical, can i ask something stupid :)
The hint information is stored in the least significant bit of each of the first 64 bytes of the image.
(1) what does least significant bit means (i mean why not the first bit)? (2) so there should be one pixel visible (if you don't use the hints) which is actually garbage?

[foxyshadis]

LSB also means least important and least noticeable, ie, changing it only changes the pixel value +-1, below the noise threshold of most videos, whereas changing the first, the most significant, means +-128. Little more noticeable. :p Since it only changes a few pixels' values by 0 or 1, it should never have a noticeable effect on filters or output.

[Wilbert]

LSB also means least important and least noticeable, ie, changing it only changes the pixel value +-1
How does ColorMatrix know which pixel is altered and by how much? I guess i need to read the code again tonight :)

[foxyshadis]

Hinting just overwrites whatever was in the last bit of the first 64 bytes (pixels) of the plane. Then filters read only those last bits, ignoring the rest. They don't really care what was there before, or whether it was modified.

[FredThompson]

Ah, come on, Wilbert, gimme a nod, ok? I can whine if it will help :P

[Ginsonic]

I'd meant to send it to Wilbert to take a look, but I've been busy the last month or so and haven't had the time to finish it. I did finally sit down today and finished the asm routines and made the d2v option able to work with sources that have multiple colorimetry types... so maybe in the next couple days.
Thanks a lot Tritical !

[jackiehcs]

If I use "TMPsource" instead of "mpeg2source" by importing d2v to tmpgenc, is it correct to use "ColorMatrix(d2v="xxx.d2v")" to correct the color difference?

[Wilbert]

TMPGEnc converts it to RGB24 for you. Thus TMPSource also delivers RGB24. TMPGEnc uses fcc coefficients (=Rec.601) for the YV12->RGB24 (and vice versa) conversion.


TMPsource("yourtprfile")
converttoyv12()


converts it back to YV12 using the same (ie Rec.601) coefficients. Ok, that's good and we don't have to worry about this conversion to RGB in between.

So the script:

TMPsource("yourtprfile")
converttoyv12()
ColorMatrix(d2v="xxx.d2v")

should be correct when encoding to XviD/DivX.

[jackiehcs]

I see, thanks.

[Ginsonic]

I'd meant to send it to Wilbert to take a look, but I've been busy the last month or so and haven't had the time to finish it. I did finally sit down today and finished the asm routines and made the d2v option able to work with sources that have multiple colorimetry types... so maybe in the next couple days.

Knock, knock ;) Is there any news ?

[jmac698]

I did some research, and bbmpeg can set the coefficients in advanced settings. Now you just set it the same as your source. If you have dvb/s or hdtv, set rec709, for cap set rec601, and for properly made xvid set rec601.

Only one problem, bbmpg is an older program and we need to increase the motion vector length, and apply some good matrices.

I found that freeenc (based on libavcodec) has no option for color coefficients.

Ideally we need the option in restream, to set the whole stream to a coefficient.

There is one more problem of different coefficients in one stream, and the only solution so far is colormatrix. Use the coeff. of the main movie in the bbmpg settings, and use colormatrix with hints to convert 601->709 when 601 appears.

[Inc]

I found that freeenc (based on libavcodec) has no option for color coefficients.
IF theres an option in the Libavcodec XXXcontext settings to set the coefficients for the encode then this could be integrated in FreeEnc or Qenc as both are open source.

[jmac698]

I see nothing in
http://ffmpeg.sourceforge.net/ffmpeg-doc.html#SEC9
relating to color coefficients. Any other ideas?
Also
http://mplayerhq.hu/~michael/ffmpeg-doxy/structMpegEncContext.html
Has a lot of info, I don't think it's there but I'm not sure.

[Hans Ohlo]

i read trough most parts of the thread. since a hdtv transport stream i recorded showed nothing in gspot i had to determine which coefficents were used. so i loaded the avs script into virtualdub took a screenshot and did the same with the mpeg in mpc. then i compared. i found that when i used colormatrix the result was more a match. now after encoding with x264 and nero digital avc the colors were to bright and i have to do the encode all over again. isnt there a way to determine for shure if to use colormatrix or not?

[Wilbert]

i found that when i used colormatrix the result was more a match. now after encoding with x264 and nero digital avc the colors were to bright
This might be a playback issue. What are you using to play it?

[Hans Ohlo]

This might be a playback issue. What are you using to play it?
CoreAVC (0.0.0.4 alpha)

[Wilbert]

i read trough most parts of the thread. since a hdtv transport stream i recorded showed nothing in gspot i had to determine which coefficents were used. so i loaded the avs script into virtualdub took a screenshot and did the same with the mpeg in mpc. then i compared. i found that when i used colormatrix the result was more a match. now after encoding with x264 and nero digital avc the colors were to bright

I guess the same happens when you don't use ColorMatrix?

[Hans Ohlo]

I guess the same happens when you don't use ColorMatrix?
no, i did a reencode without colormatrix and the colors are ok. strange is what i described before...

[jellysandwich]

1) 1) I709 (= Rec.709 coefficients)
2) FCC (almost the same as Rec.601)
3) I470 (= Rec.601 coefficients [an updated version of Rec.470-6, but coefficients are exactly the same])
4) S170 (= SMPTE 170M; exactly the same as Rec.601)
5) S240 (= SMPTE 240M; almost the same as Rec.709)

If the boxes remain "black" then the used coefficients are not present in the header, and the default (which is Rec.709) should be assumed.


I'm confused about this whole ColorMatrix thing. If the boxes are black, then I should assume Rec.709 was used, and I should use ColorMatrix (Megui x264)?

js

[Wilbert]

I'm confused about this whole ColorMatrix thing. If the boxes are black, then I should assume Rec.709 was used
Yes, indeed.

and I should use ColorMatrix (Megui x264)?
Yes again.

[Hans Ohlo]

Yes, indeed.


Yes again.
yep but in my experience in 90% of the time this is wrong and the colors (especially red) are to bright...

[Wilbert]

yep but in my experience in 90% of the time this is wrong and the colors (especially red) are to bright...
Could you upload a 10 frames vob somewhere showing this problem?

[Hans Ohlo]

it is no vob it is from mpeg2 high def ts streams.

[Wilbert]

Ok, could you cut it with some tool and upload 5-10 frames from that?

[HOLiC]

Hi, I have a quick question!
Is it necessary to use colormatrix right after source, before ivtc?
(I just remebemer someone saying that filter won't do anything if it didn't come right after the source..)

I'm trying to encode something using "tivtc mkv-vfr" method and I can't use colormatrix, since I keep getting an error about clip being not matched, or something like this:


Avisynth Open failure:
TFM: crc32 in input files does not match that of current clip...


So, I placed colormatrix after ivtc, and it seems to work fine:


mpeg2source()

TFM(mode=1,input="tfm.txt")
oldcount=framecount
tdecimate(mode=5,hybrid=2,input="tdec.txt",tfmIn="tfm.txt",mkvOut="timecodes.txt")

colormatrix(mode="Rec.709->Rec.601")

oldfps=framerate
averagefps=(float(framecount)/float(oldcount))*oldfps
assumefps(averagefps).Nicefps()


I'm just worried that if I do it like this, the filter won't do anything...
I would appreciated it if someone would confirm this.
Thanks in advance!!

[Wilbert]

Is it necessary to use colormatrix right after source, before ivtc?
(I just remebemer someone saying that filter won't do anything if it didn't come right after the source..)
Only if you use hints.

I'm trying to encode something using "tivtc mkv-vfr" method and I can't use colormatrix, since I keep getting an error about clip being not matched, or something like this:
I'm not sure. Perhaps tritical can comment?

[foxyshadis]

It's a special function of the tfm 2-pass file, it errors out if you don't use exactly the same filters before tfm as you did on the first pass. tdec has the same check. Colormatrix should have no effect, but tfm's trying to protect against random changes that render the stats useless.

To disable the check, all you have to do is remove the "crc=" section from the beginning of the stats files. (Leave the other info though!)

Colormatrix will still have an effect where it is now, though. It just won't be able to use hints to decide when it actually needs to be used, it'll just affect every frame (which is usually correct).

[jellysandwich]

Is it necessary to use colormatrix right after source, before ivtc?
(I just remebemer someone saying that filter won't do anything if it didn't come right after the source..)

When I was working on FLCL, I actually got small horizontal line artifacts in a few frames using Colormatrix before IVTC. Calling Colormatrix after IVTC fixed it.


js

[Wilbert]

Also, I just got a copy of Ferris Bueller. As it turns out, it's one of those rare DVDs that has changing colorimetry info. What's the recommended action for these types of DVDs? Leave it alone? Use colormatrix anyway?
A small request :) Could you upload a small vob (say 20 frames), where this changing occurs?

Does the change happen at the end or in the middle of the clip?

[rig_veda]

If an avisynth script outputs YV12 and I open it in VirtualDub, Vdub needs to make a YV12->RGB conversion for it's preview image. For this conversion, does it assume that the YV12 material is Rec.601?
In this case, if I process Rec.709 material in my AviSynth script, make a ColorMatrix() call along the line and feed it into Vdub, it would display correctly, right?

If I process Rec.709 material, make a ColorMatrix() call that converts it to Rec.601 and a ConvertToRGB() at the end and open this in Vdub, it would give the same result, right?

And if I process Rec.709 material and just do ConvertToRGB(Rec709) before handing it to Vdub, it would also be the same, yes?

The maximum differences i got from screenshots of vdub's display of these three cases was 3 (for RGB values in the 20-30 area), so i was wondering whether these are still due to rounding errors or whether my assumptions above are just wrong.. :D And, yes, which of the three method's display to trust most for proof viewing the output of a filter chain..?


Edit: I was using VirtualDubMod, not VirtualDub.

[Wilbert]

If an avisynth script outputs YV12 and I open it in VirtualDub, Vdub needs to make a YV12->RGB conversion for it's preview image. For this conversion, does it assume that the YV12 material is Rec.601?
Yes.

In this case, if I process Rec.709 material in my AviSynth script, make a ColorMatrix() call along the line and feed it into Vdub, it would display correctly, right?
Yes.

And if I process Rec.709 material and just do ConvertToRGB(Rec709) before handing it to Vdub, it would also be the same, yes?
Yes.

The maximum differences i got from screenshots of vdub's display of these three cases was 3 (for RGB values in the 20-30 area), so i was wondering whether these are still due to rounding errors or whether my assumptions above are just wrong..
For the last two cases, they are just rounding errors. The first one is wrong.

And, yes, which of the three method's display to trust most for proof viewing the output of a filter chain..?
It seems to me that ConvertToRGB(Rec709) would be the most accurate.
When using ColorMatrix and ConvertToRGB (VDub routines), you will make two rounding erros.

[unplugged]

ColorMatrix corrects the colors of mpeg2 streams of dvds. More correctly, those mpeg2 streams are encoded using a different set of coefficients as used by AviSynth's color conversion routines or by the XviD/DivX decoders, with the result that DivX/XviD clips or mpeg2 clips encoded by TMPGEnc/CCE using AviSynth or VirtualDub are displayed with slighty off colors (which looks like a small difference in brightness ). This can be checked by opening the mpeg2 stream directly in VDubMod. More info can be found in the readme.

This filter recalculates the yuv values assuming the coefficients which are used by AviSynth/VDub/DivX/XviD, with the consequence that your final encoding (mpeg2 or mpeg4) is displayed correctly.
I appreciate your effort, but isn't better to implement an option to ffdshow or XviD decoder to pass YUV data through hardware overlay using the same ITU-R BT standard used in the DVD Player software?

If PC DVD Players show movies with the "right" brighteness and they almost always use overlay card function as well why MPEG4 DirectShow decoders can't simply dial with the overlay subsystem by simply imparting the same YUV->RGB color conversion scheme?

When we see a movie in our PC, most of times we see it through overlay output method. AFAIK "overlay" involves direct video card conversion of YUY2 / YV12 data *directly* into analog RGB signals.
(that's the reason because overlay mode exists, this makes a lot of difference on (analog) CRT displays)

IMHO this is better than manipulate YV12 data and than involve some erratic pixel requantization. (16-235 --> 0-255)

[foxyshadis]

Because the pc-playback streams suck and don't include any form of flagging for this. Same problem, but much worse, with tv/pc yuv translation. A simple bitflag for tv/pc would give decoders & renderers a way to correctly decode everything, instead of essentially randomly picking one method and destroying video that doesn't fit its mold. In this case, adding the colorometry flag of mpeg2 to mp4/mkv, or mpeg4 asp/avc raw streams would be a better way, but afaik it isn't there at all, and good luck calling up ISO and asking them to throw it in.

[Heini011]

hi,

many thanks for your work on colormatrix! but i have 2 questions. regarding to this line from manual:

This filter will clamp (= round) your input video to CCIR-601 compliant values (these ranges are 16-235 for the luma component and 16-240 for the chroma component).

first: i don't understand why luma and chroma limitting is performed and why it is not possible to disable. if i encode for playback on a pc via xvid, i would like to keep the full pc-scale [0..255] of luma and chroma.

second: if i process an avs script within avisynth and save a frame as rgb-bitmap, i got the full pc-scale on luma and corrected color. uhm.. ?

greetings.

[Heini011]

Hi,

many thanks for the working on colormatrix. i have 2 questions regarding to the following line in the docu:

This filter will clamp (= round) your input video to CCIR-601 compliant values (these ranges are 16-235 for the luma component and 16-240 for the chroma component).

1. is that mean tv-scale luma/chroma clipping? i don't want such clipping if i encode material for pc playback!

2. if i open an avs script with colormatrix within virtualdub and save a frame as rgb bitmap, i got the full luma range [0,255] - exactly what i want to have, but in opposite to the docu ?

btw: HCEnc is using ITU-R BT.709 colormetry already.
so i have (not) to use it for hcenc in the opposite way than xvid, right ?

greetings.

[Wilbert]

1. is that mean tv-scale luma/chroma clipping?
Yes.

i don't want such clipping if i encode material for pc playback!
tv/pc-scale has not much to do with pc playback. I know some people claim otherwise, but that's a bit misleading. The point is that the footage is clipped in YUV-space (and when the clip is played back, and thus converted to RGB, the luma/chroma range is expanded again by most players), so it is only a problem if the range of your YUV clip is [0,255].

2. if i open an avs script with colormatrix within virtualdub and save a frame as rgb bitmap, i got the full luma range [0,255] - exactly what i want to have, but in opposite to the docu ?
See above.

btw: HCEnc is using ITU-R BT.709 colormetry already.
so i have (not) to use it for hcenc in the opposite way than xvid, right ?
Yes.

[Ginsonic]

Sorry, that I ask again, but is there any chance to get a special version, which does not clamp to 16-235 ?

Thanks !

[Didée]

Can't decide if that makes sense at all ... but if you desperately must have it, you can do like

a = last
b = a.ColorMatrix()
c = mt_lutxy(a,b, yexpr="x 16 < x 235 > | x y ?",
\ uexpr="x 16 < x 240 > | x y ?",
\ vexpr="x 16 < x 240 > | x y ?", U=3,V=3)

and have the wished result in the "c" clip.

[Ginsonic]

Many thanks Didée, I will try !

[canuckerfan]

Would ColorMatrix be necessary for a DVD9 to DVD5 conversion using CCE SP2?

[unskinnyboy]

Would ColorMatrix be necessary for a DVD9 to DVD5 conversion using CCE SP2?
Only if the MPEG-2 coefficients are different, which is rare. Use it with hints=true so that it is applied only if needed. Also, make sure you call ColorMatrix first before any other filter for the hints to work.

[Wilbert]

Only if the MPEG-2 coefficients are different, which is rare. Use it with hints=true so that it is applied only if needed. Also, make sure you call ColorMatrix first before any other filter for the hints to work.
Don't forget to add mode="Rec.601->Rec.709" in that case. CCE assumes Rec.709 coefficients.

[G_M_C]

Would ColorMatrix be necessary for a DVD9 to DVD5 conversion using CCE SP2?

Only if the MPEG-2 coefficients are different, which is rare. Use it with hints=true so that it is applied only if needed. Also, make sure you call ColorMatrix first before any other filter for the hints to work.

Don't forget to add mode="Rec.601->Rec.709" in that case. CCE assumes Rec.709 coefficients.


These three posts got me to thinking about the way DVD-RB opens its DVD files. This because DVD-RB uses all kinds of encoders, inclusing CCE and Procoder. To stream files to the encoders it generates an AviSynth script that ends with “ConvertToYUY2()”. At the moment it works like this:

Mpeg2Source (“<file>”, div.options)

<some further script options (resizing of instance)>

ConvertToYUY2()

Reading the above posts got me to thinking that something might go wrong there, because AviSynth assumes Rec.601 data when the “ConvertToYUY2()” is invoked (or another conversion), and most DVD’s i’ve checked use rec.709.

Wouldn’t it be wise to use colormatrix in DVD-RB, when opening files ? Like this:

Mpeg2Source (“<file>”, div.options, info=3)
Colormatrix (hints=true)

<from here script as usual>

ConvertToYUY2()

I’ve made a posting in the DVD-Rebuilder part of this forum with this idea. But I am no expert (to say the least), maybe you can shed some light on it ?

Posting is @ : http://forum.doom9.org/showthread.php?p=879894#post879894

[unskinnyboy]

Wouldn’t it be wise to use colormatrix in DVD-RB, when opening files ? Like this:

Mpeg2Source (“<file>”, div.options, info=3)
Colormatrix (hints=true)

<from here script as usual>

ConvertToYUY2()
It's better that you don't cross-post, because the answers can get scattered and discontinuous, but anyway..

Right way is :

Mpeg2Source (“<file>”, div.options, info=3)
Colormatrix (mode="Rec.601->Rec.709", hints=true)

It would be wise to do this by default for the most part, yes. But not all DVDs are Rec.709 and also there is an overhead in checking for hints every time. I think it should remain optional.

[Wilbert]

@G_M_C,

These coefficients are only used in a RGB<->YUV conversion, thus not in a YV12<->YUY2 conversion (which you are dealing with when using DVD-RB).

ConvertToYUY2(matrix="rec709") might be needed when requesting RGB (by using upConv=2) from MPEG2Source. I say "might" because i'm not sure what coefficients are used by MPEG2Source to convert to RGB24.

@Neuron2,

Are the coefficients autodetected and used when requesting RGB24 from MPEG2Source?

[primusmp]

Ok I finished reading the whole post (It`s quite large..) didn´t find answer to my problem... So I`ll be testing tomorrow...

I have PAL DV edited at premiere exported to huffyuv PAL AVI then loaded in avisyth and converted to NTSC ...

Script is being modified now but it used to start with ConverttoYU2() before smoothdeinterlace was applied....

In the end it was fed to CCE... watching tha0 .avs in video player (BS player media playe.. winap.. all teh same.) played the correct levels... The CCE Mpeg 2 file produced VERY distorted levels (No noticeable change using 0-255 or 16-235) Changed CCe to "Try to use RGB" and I get the correct levels (now 0-255 and 16-235 are different also) but Encoding is slower. Tried using ColorMatrix() with several different options... (i.e 6to7 and 7to6) and it doesn´t seem to affect the cce encoding. Colormatrix() line is placed rigth after the resize...

NOTES: when I say "No noticealbe change" it actually means "It`s 2 in the morning and I do not have time to actually pay attention to minor changes" and when it says "VERY distorted" I really mean VERY distorted like oversaturated and really darker...

I`ll send some pics tomorrow and if I find out what the f**k`s going on I`ll let you know.

Please apologize my english It`s not the best it can get but hey... Ive been working for the last 48 ours straight and Im really tired.

[tritical]

@Wilbert
With upConv=2, dgdecode does detect and use the coefficients indicated by the matrix_coefficients value to perform the conversion to rgb. (This is not the case for previous mpeg2decx.dlls that could deliver rgb).

[tritical]

[link removed, read down for a newer version], finally finshed the new version I mentioned back in January. Changes from v1.10:
+ Rewrote a large portion of the code
+ Added ability to convert between any of: Rec.709, FCC, Rec.601, and SMPTE 240M
+ Added source and dest parameters
+ Added scaling parameter (optional clipping and unscaled coefficients)
+ d2v option supports changing colorimetry info (with trim() and frame rearrangement restriction)
- fixed a problem with calculated luma values that were < 0 being set to 255 instead of 0

Main items are ColorMatrix can now convert between any of: Rec.709, FCC, Rec.601, and SMPTE 240M... everything in the mpeg2 specs. The pre/post clipping with limiter is optional, and there is also the option to use non-scaled coefficients if desired. The d2v option supports changing colorimetry (though there are still a few restrictions with using trim() and the like beforehand -- see the read me). The new syntax is:

ColorMatrix(clip, string "mode", int "source", int "dest", int "scaling", bool "interlaced", bool "hints", string "d2v", bool "debug", int "opt")

[G_M_C]

The downloadlink times-out for me, guess the whole world is getting an update :P

[Adub]

Shouldn't you update the title of the thread?
And maybe your website?

[tritical]

hm, I had updated my website but the university's servers had a little outage around then and I guess something screwy happened. Anyways, it's updated now. I can't edit the thread title since it isn't mine. I opted not to post this in a separate thread so that all the ColorMatrix related info would stay together.

[Alain2]

A few questions..
- When using colormatrix with the d2v parameter, do we still have to specify the interlace parameter ?

If so:
- If a PAL interlaced source, for a progressive one I suppose the interlace parameter should be set to false ?
But for a NTSC source, if the d2v was done with honor pulldown flags", should the interlaced parameter be true or false ? Same question for a d2v in forced film ?

I have these questions because the d2v analysed is showing fields, not frames I think, so I am not sure how the interlace parameter is used..

Thanks

[Wilbert]

A few questions..
- When using colormatrix with the d2v parameter, do we still have to specify the interlace parameter ?
Yes.

for a progressive one I suppose the interlace parameter should be set to false ?
Yes.

But for a NTSC source, if the d2v was done with honor pulldown flags", should the interlaced parameter be true or false ? Same question for a d2v in forced film ?
That's a good one. You need to make it progressive first by inverse telecining it. Note that the first frame is not altered by the ivtc process (the first few bytes are altered in the first frame when using hints in dgdecode), so you can use ColorMatrix directly after doing the inverse telecine.

[canuckerfan]

I tried the above suggestion, to use colormatrix after IVTC, but it says no hints detected in red text. It worked fine when I called it before IVTC. here's my script:

mpeg2source("E:\Video\Hindi Movies\Dil.Chahta.Hai.2001.DVD9-bj1988\temp\slow.d2v",idct=5,info=3)

Setmemorymax(512)

AssumeTFF()
Interp = SeparateFields().SelectEven().EEDI2(field=1)
Deinted=TDeint(order=1,field=1,edeint=Interp)
TFM(mode=6,order=1,PP=7,slow=2,mChroma=true,Clip2=Deinted)
TDecimate(mode=1)

ColorMatrix(mode="Rec.601->Rec.709",hints=true)

DeSpot(pwidth=65,pheight=65,p1=35,p2=14,mthres=25)
RemoveDirt()

Tweak(sat=1.12,cont=1.12,bright=10.4)

FFT3DFilter(sigma=2.2,ow=48/2,oh=48/2)
Dehalo_alpha()
LimitedSharpenFaster(Smode=4,Lmode=2,overshoot=14,strength=200)

Crop(0,48,0,-48)
AddBorders(0,48,0,48)

ConvertToYUY2()

[Alain2]

Ok, so if I understand well:

- in case of a d2v in forced film, then I get a progressive clip out of mpeg2source, and thus set interlaced=false in colormatrix ?

- in case of a d2v in honor pulldown flags, then I have a telecined clip. You say it's better to ivtc first (with tfm+tdecimate for instance) before applying colormatrix, so in that case the d2v won't have the same number of frames than what is fed through colormatrix.
-- I can't use the d2v parameter then (so won't detect eventual changes of colorimetry within the clip) ?
-- If I don't ivtc first but place colormatrix with d2v=.. and interlaced=true immediately after mpeg2source, would that be a worse solution and why ?

[Wilbert]

- in case of a d2v in forced film, then I get a progressive clip out of mpeg2source, and thus set interlaced=false in colormatrix ?
Yes.

- in case of a d2v in honor pulldown flags, then I have a telecined clip. You say it's better to ivtc first (with tfm+tdecimate for instance) before applying colormatrix, so in that case the d2v won't have the same number of frames than what is fed through colormatrix.
Yes.

-- I can't use the d2v parameter then (so won't detect eventual changes of colorimetry within the clip) ?
I never thought about this, but i guess that means yes indeed.

-- If I don't ivtc first but place colormatrix with d2v=.. and interlaced=true immediately after mpeg2source, would that be a worse solution and why ?
Yes, a little bit worse. The chroma of the clip will be blurred more in that case (because it's being treated as interlaced).

@canuckerfan,

I tried the above suggestion, to use colormatrix after IVTC, but it says no hints detected in red text. It worked fine when I called it before IVTC. here's my script:
I'm not sure why, but perhaps tritical can explain. In any case, you can store the hints in a different clip:


Setmemorymax(512)

mpeg2source("E:\Video\Hindi Movies\Dil.Chahta.Hai.2001.DVD9-bj1988\temp\slow.d2v",idct=5,info=3)
savedHints = last
AssumeTFF()
Interp = SeparateFields().SelectEven().EEDI2(field=1)
Deinted=TDeint(order=1,field=1,edeint=Interp)
TFM(mode=6,order=1,PP=7,slow=2,mChroma=true,Clip2=Deinted)
TDecimate(mode=1)

MergeHints(hintClip=savedHints) # add hints back for use with ColorMatrix
ColorMatrix(mode="Rec.601->Rec.709",hints=true)

DeSpot(pwidth=65,pheight=65,p1=35,p2=14,mthres=25)
RemoveDirt()

Tweak(sat=1.12,cont=1.12,bright=10.4)

FFT3DFilter(sigma=2.2,ow=48/2,oh=48/2)
Dehalo_alpha()
LimitedSharpenFaster(Smode=4,Lmode=2,overshoot=14,strength=200)

Crop(0,48,0,-48)
AddBorders(0,48,0,48)

ConvertToYUY2()

[tritical]

-- I can't use the d2v parameter then (so won't detect eventual changes of colorimetry within the clip) ?
You can still use the d2v parameter as long as the colorimetry doesn't change at some point in the d2v. The same number of frames requirement is only enforced if colormatrix detects that the colorimetry does indeed change in the d2v file. That requirement is checked for during filter construction, so if you are able to load the script without an error then everything is fine. In the case that the colorimetry does change, colormatrix has to correctly match up input frames to frames in the d2v file to determine which frames need which conversion... thus the requirement that the number of frames be the same.

@canuckerfan
The hints wont survive tfm because by default it writes its own hints into the same bytes that dgdecode uses for hints. Unfortunately, the mergehints() workaround that Wilbert posted wont work correctly either since the frames in the saved clip wont match up with the frames after ivtc... which would be required to correctly handle changing colorimetry. If the colorimetry didn't change, then it would be fine, but in that case you could just use the d2v parameter.