ColorMatrix v2.3

[Wilbert]

ColorMatrix corrects the colors of mpeg2 streams of dvds. More correctly, those mpeg2 streams are encoded using a different set of coefficients as used by AviSynth's color conversion routines or by the XviD/DivX decoders, with the result that DivX/XviD clips or mpeg2 clips encoded by TMPGEnc/CCE using AviSynth or VirtualDub are displayed with slighty off colors (which looks like a small difference in brightness ). This can be checked by opening the mpeg2 stream directly in VDubMod. More info can be found in the readme.

This filter recalculates the yuv values assuming the coefficients which are used by AviSynth/VDub/DivX/XviD, with the consequence that your final encoding (mpeg2 or mpeg4) is displayed correctly.

http://forum.doom9.org/showthread.php?s=&postid=514595#post514595

Changelog v2.3 (by tritical),

- Calculate conversion coefficients for PB/PR rows from luma coefficients
- Added inputFR and outputFR parameters allowing for YUV range expansion/contraction
- Changed the clamp parameter from bool to int to allow specifying only pre clipping or only post clipping or both


more info: http://forum.doom9.org/showthread.php?p=887227#post887227

http://web.missouri.edu/~kes25c/

[malkion]

Wilbert, if this filter does indeed perform as you intended it, I would like to express some gratitude for future encodes.... :D

Now should I run limiter() before or after colormatrix() ?

[Teegedeck]

I've waited for this a long time! If only I had time right now to try it out...

A heartfelt 'thank you'. :)

[Boulder]

Thanks, this one will be very useful!

[Wilbert]

Now should I run limiter() before or after colormatrix() ?
If you get v1.1, you don't need to use limiter at all (not before or after the filter). The filter clamps input to [16,235] (and output also).

[sh0dan]

Made a few (easy) speedups - it should be a couple of times faster now.

Updated source (http://83.90.202.89/kpo/colormatrix_faster.zip).

- Use float instead of double. They have enough precision and are much faster. It can however be changed back by changing the typedef in ColorMatrix.h

- Simpler algorithms.

- Use internal limiter for output also, instead of very slow if-then.

- Better rounding (adding 0.5 for more exact float to int conversion)

No SSE/MMX this tim, though. :)

[jorel]

:cool:
thank you very much Wilbert !

i was "stuck inside a cloud" here trying to find a way for that problems!
now our encodes will get much, much more quality !
:)
downloading and testing! :D

[Wilbert]

@Sh0dan,

Thanks! I updated the version.

[Inc]

Thanks a lot Wilbert !

:)

[jorel]

Originally posted by Wilbert
@Sh0dan,

Thanks! I updated the version.

of course....

thank you too Sh0dan.:cool:

.seems that we posted at the same time and i saw your post after my last!

:)

[Dark-Cracker]

very nice work :)
just by curiosity does it will harm the final output if u apply this filter on HDTV source or on some raw avi input ?

++

[Wilbert]

just by curiosity does it will harm the final output if u apply this filter on HDTV source
I got a few hdtv sources. Marjority of them uses mpeg2 coefficients (checked with the newest GSpot). So, you should also use the filter on those sources.

However, a few of them uses mpeg1 coefficients. You shouldn't apply the filter on those.

on some raw avi input
Here, you shouldn't use this filter. DivX/XviD assume mpeg1 coefficients when decoding to RGB.

edit: btw, hdtv is YUV [0,255] of course. So you need to use ColorYUV("PC->TV") first.

[FuPP]

I have a question about colormatrix and speed. I use colormatrix() 1.2 and get the following results

with colormatrix : 16.5 fps
without colormatrix : 44.5 fps

should such a difference be expected or is there any problem ?

My script (source is 720x576) :

LoadPlugin("D:\video\avsfilters\yv12\DGDecode.dll")
LoadPlugin("D:\video\avsfilters\yv12\colormatrix.dll")

mpeg2source("d:\test\test.d2v",idct=6)

trim(30000,33000)
colormatrix()

FuPP

[Wilbert]

The problem is that the filter is not optimized. Since I'm not a good coder, I don't know how to do that. I hope someone has some time to do that :)

[Manao]

I added other easy optimisations ( no more float computations, only integer one ). Results *should* be the same as those given by sh0dan's version. It should be almost twice as fast.

http://jourdan.madism.org/~manao/ColorMatrix.zip

[Wilbert]

Thanks! Could you also add the correct source files?

[Manao]

I copy'n'pasted the wrong files :p

You can download it again.

[Wilbert]

Thanks, I added it to the first post! I didn't have time to test it though :)

[FuPP]

with colormatrix : 34.5 fps
without colormatrix : 44.5 fps

Far better ! Thanks a lot Manao ! And thanks a lot Wilbert for that filter :)

FuPP

[ltc]

How to properly use this filter?
Say if I want to re-encode a DVD to DVD-R for TV playing. I assume I should use YUV and TV scale in DVD2AVI. I can find the original matrix coefficient with GSpot which has following type:

I709 = type 1 = MPEG2 default
FCC = type 4 = FCC standard
I470 = type 5 = MPEG1 default
S170 = type 6 = SMPTE 1700M standard
S240 = type 7 = SMPTE 240M standard

I have some DVDs which has S170 coefficient. Should I use ColorMatrix before converting to ConvertToYUY2 for CCE and ConvertToRGB24 for TMPGE?

Some of my encodes are brighter than original. Could it be because I accidentally used RGB PC scale?

[Wilbert]

You need to use it in case your source is: MPEG2 default or SMPTE 240M standard (both set of coefficients are approximately the same).

The other set of coefficients are approximately the same and equal to the MPEG1 coefficients.

It doesn't really matter what your target is. If you use VDub or AviSynth, these matrix coefficient info is lost, and mpeg1 coefficients will be assumed during playback (at least for DivX/XviD/TMPGEnc/QuEnc, not sure about CCE).

Should I use ColorMatrix before converting to ConvertToYUY2 for CCE and ConvertToRGB24 for TMPGE?
Yes. Just directly after loading the source.

Perhaps you can make an avs script, encode it to MPEG2 with CCE, and post the coefficients for me?

[ltc]

Hmn, interesting. The source DVD is NTSC interlaced. Color coefficient is S170 (SMPTE 170M) according to GSpot.
I re-encoded to D1/2 size using KSVCD template. Encoded with tmpgenc the result video has FCC coefficient. Encoded with CCE, no coefficient present.

script:
Mpeg2Source("D:\TietNhonQuy_DVD\TietNhonQuy_01.d2v")
ColorMatrix()
#
crop(8, 0, -32,0)
GripCrop(352, 480, source_anamorphic=false)
GripSize(resizer="LanczosResize")

dgbob(FOrder(),mode=1 )
convert60ito24p(2,0)
DegrainMedian(mode=1)

converttoyuy2()

# Detect field order
function FOrder(clip c) {
fo= GetParity(c)? 1:0
return fo
}

[Wilbert]

Encoded with CCE, no coefficient present.
That's interesting. It means if you are doing DVD -> DVD using AVS_YUY2 -> CCE, the same coefficients are used (in both cases the default MPEG2 coefficients). Thus, in this case, you shouldn't apply ColorMatrix.

[Wilbert]

I put up a new version (v1.4). Changelog:

* Added a mode parameter (default is mode="mpeg2->mpeg1" which gives the same output as older versions).

In case you captured something or you have a XviD/DivX (both are encoded mpeg1 coefficients), and you want to encode it to mpeg2 using CCE (which assumes mpeg2 coefficients):

ColorMatrix(clip, mode="mpeg1->mpeg2")

* Added rgb=true, which converts to RGB24 using mpeg2 coefficients.

The following converts a YV12/YUY2 stream to RGB24 using MPEG-2 coefficients (which might be useful if you want to convert DVD to mpeg2 using TMPGEnc):

ColorMatrix(clip, rgb=true)

It should give the same results as

ColorMatrix(clip, mode="mpeg1->mpeg2")
ConvertToRGB24()

[Boulder]

Is rgb=true the default in the new version? The filter was slower than usual with just ColorMatrix(), adding rgb=false fixed that.

I'd also like to clear some things up, is this the correct way:

1) DivX/XviD clip to MPEG-2 in CCE : ColorMatrix(mode="mpeg1->mpeg2")
2) MPEG-2 capture (no coeffs shown) to MPEG-2 in CCE : ColorMatrix()

[Wilbert]

Is rgb=true the default in the new version? The filter was slower than usual with just ColorMatrix(), adding rgb=false fixed that.
Default is rgb=false.

Btw, rgb=true is the same as

ConvertToRGB(matrix="rec709")

which should be faster. I didn't know that when adding that option :)

1) DivX/XviD clip to MPEG-2 in CCE : ColorMatrix(mode="mpeg1->mpeg2")
Yup.
2) MPEG-2 capture (no coeffs shown) to MPEG-2 in CCE : ColorMatrix()
That's a tricky one. Of course, your cap application (assuming it's just an analog cap) doesn't know which coefficients should be used. Since pal/ntsc standards always (should) use mpeg1 coefficients (ITU BT.601-4), you should use
ColorMatrix(mode="mpeg1->mpeg2")

[Boulder]

Thanks, that cleared things up:)

[Wilbert]

I put up a new version. In the previous ones, the used coefficients were slightly wrong. Although the difference is not noticable in practice :)

btw, I corrected the naming convention. MPEG-2 coefficients are called Rec.709 and MPEG-1 coefficients Rec.601. Thus

ColorMatrix(clip, mode="Rec.709->Rec.601") (default)
ColorMatrix(clip, mode="Rec.601->Rec.709")

I advice you to use the internal

ConvertToRGB(clip, matrix="Rec.709")

instead of

ColorMatrix(clip, rgb=true).

Output is identical (thx IanB), but the former is faster.

[North2Polaris]

@Wilbert,

I recently used your filter on a Mpeg2 to Xvid conversion using AviSynth and VDubMod. The difference, to my eyes at least, was remarkable. I have wondered why past encodes were "less bright" than I expected and needed to be adjusted.

I took a look at the AviSynth plugins that come with Gordian Knot. I did not see ColorMatrix among them, which surprised me given the discussion in the current thread and the following threads:

http://forum.doom9.org/showthread.php?s=&threadid=78220&perpage=20&pagenumber=3

http://forum.doom9.org/showthread.php?s=&threadid=81191

Does Gordian Knot handle this problem another way or not at all?

Thanks for your work on this! Happy New Year!

North

[Wilbert]

Does Gordian Knot handle this problem another way or not at all?
No. Perhaps len0x is not aware of the problem. Did you ask him about this?

[North2Polaris]

Originally posted by Wilbert
No. Perhaps len0x is not aware of the problem. Did you ask him about this?

@Wilbert,

I just sent a PM to len0x about this.

I tried using LumaFilter in DGDecode to adjust the "brightness".

From DGDecode html,

Useful additional YV12 Filters :

To Darken Luminosity.

LumaFilter(-10,0.9)

To Ligthen Luminosity.

LumaFilter(+10,1.1)

Will lightening luminosity address the brightness problem addressed by ColorMatrix or just "cover it up"?

North

[Boulder]

The original problem is not actually the brightness, it's that the colors are off slightly.

[len0x]

Hm... Am I right in understanding that every DVD source needs to be converted to a proper color matrix before encoding into MPEG4?

*Edit* remark about RGB removed

[Wilbert]

Hm... Am I right in understanding that every DVD source needs to be converted to a proper color matrix before encoding into MPEG4?
Yup, well "to the same color matrix" as MPEG4 uses/assumes (Rec.601 coefficients).

[North2Polaris]

Originally posted by Boulder
The original problem is not actually the brightness, it's that the colors are off slightly.

@Boulder,

Thanks. When you don't know that the underlying problem is a color problem, it looks like a problem with brightness, which can lead to unsatisfactory attempts to correct it.

North

[len0x]

Originally posted by Wilbert
Yup, well "to the same color matrix" as MPEG4 uses/assumes (Rec.601 coefficients).

Bummer - now I have a dilemma since speed loss is at least 10% when doing correction so it might not be a good idea to make it default in GK/AGK...

Btw, do you have XML description of the filter? (I can add it to GK then)

[Wilbert]

Bummer - now I have a dilemma since speed loss is at least 10% when doing correction so it might not be a good idea to make it default in GK/AGK...
I wish I could do MMX :)

Btw, do you have XML description of the filter? (I can add it to GK then)
Not yet. I will do it this week and send you a pm.

[len0x]

Originally posted by Wilbert
Not yet. I will do it this week and send you a pm.

cheers

[Steve56]

ColorMatrix does virtually the same as BT709ToBT601 from http://mywebpages.comcast.net/trbarry/downloads.htm, or?

Steve56

[Wilbert]

I corrected the url above :)

Yes that's true, I noticed that too late. His plugin contains optimization code, so it might be faster. (btw, hdtv isn't always Rec.709 as suggested in his docs ...)

[Steve56]

Yes that's true, I noticed that too late. His plugin contains optimization code, so it might be faster. (btw, hdtv isn't always Rec.709 as suggested in his docs ...) [/B]ah, good to know ;-) I did now try to check with GSpot, if a certain MPEG-2 HDTV 1080i stream (Astra 1080 Konzert1), is encodet with the BT709 or BT601 matrix. But I don't know where to read this information from?!

Steve56

[Boulder]

Open the clip in GSpot v2.52b01.

[Wilbert]

1) That info is also available in 'DGMPEGDec 1.0.13 beta 11'. Using


mpeg2source(info=true)

2) Use latest beta of GSpot and look in the upper right corner. In the video section, the five boxes next to NVOP. The fives boxes are from left to right:

1) I709 (= Rec.709 coefficients)
2) FCC (almost the same as Rec.601)
3) I470 (= Rec.601 coefficients [an updated version of Rec.470-6, but coefficients are exactly the same])
4) S170 (= SMPTE 170M; exactly the same as Rec.601)
5) S240 (= SMPTE 240M; almost the same as Rec.709)

see table 6-9 page 56 of http://le-hacker.org/hacks/mpeg-drafts/is138182.pdf

If the boxes remain "black" then the used coefficients are not present in the header, and the default (which is Rec.709) should be assumed.

I hope it's clear now :)

[Steve56]

Wilbert: Thanks for the comprehensive answer!

Before that, I just did a color conversion judged by the look of it, when turned on or off. ;-)

[tritical]

Wilbert, is the source included in the zip linked to at the beginning of this thread the latest? I tried doing some simple optimization and got it running about 40% faster (about 210fps vs 145fps in YV12 on my comp) still with plain c code, but I'm wondering about a few things looking through it... in the constructor it checks mode against "Rec.709->Rec.601" and "Rec.601->Rec.709", yet in the main part it checks mode against "mpeg2->mpeg1" is this intended? In the YV12 mpeg1->mpeg2 it uses the "21" conversions and not the "12" conversions, but in the YUY2 code the mpeg1->mpeg2 does use the "12" conversions? Also, could you tell me or point me to a source that has the numbers for how the coefficients used were derived? I'm just curious, I tried to calculate them myself using coefficients from a few different websites, but they never work out to be exactly the same (usually off by 20-40 after multiplying by 65536). Thanks in advance.

Hm, must be because:

mpeg2source()
clip1 = last.colormatrix(mode="rec.601->rec.709")
clip2 = last.colormatrix(mode="rec.709->rec.601")
subtract(clip1,clip2)
coloryuv(analyze=true)

shows both clips to be identical :confused:.

[Wilbert]

in the constructor it checks mode against "Rec.709->Rec.601" and "Rec.601->Rec.709", yet in the main part it checks mode against "mpeg2->mpeg1" is this intended? In the YV12 mpeg1->mpeg2 it uses the "21" conversions and not the "12" conversions, but in the YUY2 code the mpeg1->mpeg2 does use the "12" conversions?
I corrected all that (v1.6). Thx very much. It's a bit stupid that I didn't notice this :confused:

Also, could you tell me or point me to a source that has the numbers for how the coefficients used were derived? I'm just curious, I tried to calculate them myself using coefficients from a few different websites, but they never work out to be exactly the same (usually off by 20-40 after multiplying by 65536).
I included a text file (coefficients.txt) which contains all calculations (if you know maple i can send you the worksheet). I hope that's clear enough.

Btw, some websites use slightly different coefficients, perhaps that could be the problem.

I tried doing some simple optimization and got it running about 40% faster (about 210fps vs 145fps in YV12 on my comp) still with plain c code
Great! Could you add those changes to v1.6?

[North2Polaris]

Originally posted by Wilbert
Yes that's true, I noticed that too late. His plugin contains optimization code, so it might be faster. (btw, hdtv isn't always Rec.709 as suggested in his docs ...)

@Wilbert,

Other than speed, any advantage to using ColorMatrix over BT709ToBT601?

North

[Wilbert]

Other than speed, any advantage to using ColorMatrix over BT709ToBT601?
Rec.601->Rec.709 conversion (useful for capping -> mpeg2) is possible in ColorMatrix :)

[Boulder]

Originally posted by Wilbert
Rec.601->Rec.709 conversion (useful for capping -> mpeg2) is possible in ColorMatrix :)
However, if you use CCE for encoding, no need to use ColorMatrix as it doesn't specify any coefficients for the encoded file for some reason?

[Wilbert]

However, if you use CCE for encoding, no need to use ColorMatrix as it doesn't specify any coefficients for the encoded file for some reason?
Only if you use CCE :) Remember pal/ntsc uses Rec.601 coefficients. TMPGEnc/QuEnc add fcc (approx Rec.709) coefficients to the mpeg-2 header.