ColorMatrix v2.3

[Wilbert]

But, indeed, your script (#5) is identical to #1 & #2.
That's how it is supposed to be, so that's good.

About #3/#4 when opening the script

MPEG2Source("G:\temp\VTS_01_1.d2v")
ColorMatrix("Rec.601->Rec.709") # the interlaced flag is not important

in VDubMod, it is converted to RGB (upon display, but also if you save a frame as a picture). But it does so using Rec.601 coefficients (AviSynth can't pass that information to VDub/VDubMod or to any other encoder, and VDub/VDubMod uses Rec.601 in this case), while it should use Rec.709 coefficients in this case, because you applied ColorMatrix with that option.

In #5 you will let AviSynth do the proper conversion to RGB using the correct coefficients:

#5 (same as #1, #2)
MPEG2Source("G:\temp\VTS_01_1.d2v")
ColorMatrix("Rec.601->Rec.709", interlaced=true)
ConvertToRGB(matrix="rec709")

so that why #5 is the same as #1 and #2.

I hope it's a bit clear :)

[kumi]

Eureka :) Thanks for explaining.

Just one more silly question, and I'll get out of your hair:

Does script #4 represent what a player will do with a DVD with Rec.601/SMTPE 170M coefficients?

[Wilbert]

Does script #4 represent what a player will do with a DVD with Rec.601/SMTPE 170M coefficients?
No, in general, a software dvd player will use the coefficients specified in the stream itself (which is SMTPE 170M here). (I didn't investigate this, so i don't know if all popular dvd players are doing this correctly, but i assume most of them do.)

[kumi]

Ok then, I think I've finally seen the light :cool: Colormatrix() really is necessary for DVD-to-DVD encoding. Well, depending on the source and encoder coefficients of course.

Here's a Rec.601 DVD source:
MPEG2Source("C:\Rec.601.DVD.d2v").ConvertToRGB()http://hachirota.googlepages.com/Rec.601.DVD.png


Here's what HC does when fed a Rec.601 stream. It's borked!:
MPEG2Source("C:\HC.encoded.d2v").ConvertToRGB(matrix="rec709")http://hachirota.googlepages.com/Rec.601.fed.into.HC.encoder.png


And here's what HC does when fed the same stream corrected with Colormatrix(mode="Rec.601->Rec.709"). Looks just like the source:
MPEG2Source("C:\Rec.601.to.Rec.709.then.HC.encoded.d2v").ConvertToRGB(matrix="rec709")http://hachirota.googlepages.com/Rec.709.fed.into.HC.encoder.png


Wow.


@Wilbert: since I would like to avoid manually checking coefficients for each one of my DVDs, can I stack calls to ColorMatrix() to handle all possible 'conversions' to Rec.709?

MPEG2Source("C:\blah.d2v", info=3)
ColorMatrix(mode="FCC->Rec.709", hints=true)
ColorMatrix(mode="SMPTE 240M->Rec.709", hints=true)
ColorMatrix(mode="Rec.601->Rec.709", hints=true)

[Wilbert]

@Wilbert: since I would like to avoid manually checking coefficients for each one of my DVDs, can I stack calls to ColorMatrix() to handle all possible 'conversions' to Rec.709?
The easiest is to set hints=true and dest="Rec.709" (using ColorMatrix 2.1). That covers all the cases you listed.

[kumi]

Hi,

I tried that, but
MPEG2Source("G:\temp\rec.601.d2v", info=3)
ColorMatrix(dest=0, hints=true)
gives a "source and dest cannot be equal!" error.

dest="Rec.709" gives me a "named argument 'dest' had wrong type" error.

[Wilbert]

gives a "source and dest cannot be equal!" error.
Strange. It seems your source is detected as source=0, while it should be 3. Perhaps tritical can comment on this one.

[kumi]

Using the dest parameter with hints is problematic, but the mode parameter with hints corrects the coefficients as expected (and doesn't alter them if the mode target is equal to the hinted coefficients):

This converts a hinted Rec.601 source to Rec.709:
MPEG2Source("C:\rec.601.d2v", info=3)
ColorMatrix(mode="Rec.601->Rec.709", hints=true)

The same function call on a Rec.709 source seems to be a no-op:
MPEG2Source("C:\rec.709.d2v", info=3)
ColorMatrix(mode="Rec.601->Rec.709", hints=true)

[HeadBangeR77]

I've been following the discussion here to check where my problems may lie... If I get everything correctly, then:

1) If I want VDub /VDubMod /AVSP to preview my ITU-R BT.709 DVD using the right colours, then I should use a script like this one:

mpeg2source("I:\DVD\VTS_01_PGC_01_2.d2v", idct=7)
ConvertToRGB32(matrix="rec709")

because otherwise all the above applications will assume Rec.601, what will surely result in wrong colours coefficients, am I right?

2) Using the following scripts I truly get the same colours as with the above preview:

mpeg2source("I:\DVD\VTS_01_PGC_01_2.d2v", idct=7)
ColorMatrix (mode="Rec.709->Rec.601")
ConvertToRGB32(matrix="rec601")

or

mpeg2source("I:\DVD\VTS_01_PGC_01_2.d2v", idct=7, info=3)
ColorMatrix (mode="Rec.709->Rec.601",hints=true)
ConvertToRGB32(matrix="rec601")

I used to get different results from those previewed without ConvertToRGB32(matrix="rec709"), but I guess I was wrong (?).

3) Having read the last few pages once again, I still haven't got the faintest idea why MPC (built-in libmpeg2 decoding), MPC with libmpeg2 ffdshow decoding, Nero ShowTime and PowerDVD (5 & 6) display the colours as in the preview without the above conversion??? Anyone?

:confused:

Since my eyes are used to the colours I see while watching DVD, the results with ColorMatrix look a bit over-saturated to me (much brighter reds, stronger R channel, different greens and blues depending on the scene).

PS. Screenshots are on the previous page (http://forum.doom9.org/showthread.php?p=964856#post964856) - I've taken some new, if anyone is willing to take a look at them.

@ kumi:
Glad you've sorted the things out. :)

[tritical]

@kumi & Wilbert
The problem with dest vs mode is indeed a bug... it should be checking that neither hints nor d2v is being used before throwing the error about source==dest.

@All
I've read through all the latest posts, and as far as I can tell no one has discovered any true bugs have they? Only issues I'm aware of atm are the one above, and the fact that scaling=1/2 are redundant. Also, for everyone comparing colors, mpeg2source(,upconv=2) will output rgb24 using the colorimetry specified by the stream and following the progressive_frame flag.

[HanSolo00]

I've been reading and reading and I think I have things figured out. I'm having a debate with someone about encoding HD 1920x1080 Rec.709 MPEG-2 Transport Stream with x264.

As I understand it there is no difference at all between a SD 720x480 Rec.709 MPEG-2 stream and a HD 1920x1080 Rec.709 MPEG-2 stream from satellite/cable/blu-ray source; that they should both use a ColorMatrix(mode="Rec.709->Rec.601") for proper colors in the H.264 file (playback on a PC/RGB display, or HD monitor set to RGB colorspace, using either CoreAVC or FFDshow.)

Is my thinking correct? Shouldn't HD resolution Rec.709 MPEG-2 transport streams also be corrected to Rec.601 for MPEG-4 encoding? The argument I'm facing is that 'HD is supposed to be Rec.709, so you aren't encoding MPEG-4 properly if you are changing it to Rec.601'.

Edit: I didn't realize DGIndex is incorrectly reporting ITU-R BT.709 Colorimetry for MPEG-2 streams ripped from my SD-DVDs...

[HeadBangeR77]

Edit: I didn't realize DGIndex is incorrectly reporting ITU-R BT.709 Colorimetry for MPEG-2 streams ripped from my SD-DVDs...
Sorry, can't clear your doubts, as I'm pretty inexperienced with ColorMatrix myself. How did you find out that DGIndex was reporting your colorimetry wrongly? Did you compare some still frames or sth' like that?

Please, could anyone explain me, why am I getting different results using ColorMatrix than by MPEG-2 playback??? They are the same as using preview with ConvertToRGB32(matrix="rec709") or mpeg2source(..., upconv=2) , DGIndex reports the colorimetry as ITU-R BT.709, yet all software players and decoders I've tried display the colours differently (more on the previous page, screen grabs included). If someone happens to know the source, it's the 1st part of "Pirates of the Caribbean", NTSC.

many thanks in advance,
HDBR77

[Sharro]

Edit: I didn't realize DGIndex is incorrectly reporting ITU-R BT.709 Colorimetry for MPEG-2 streams ripped from my SD-DVDs...

Neither you realize neither is DGIndex reporting incorrectly, the only thing is if there is no colorimetry flag present it will assume BT.709.

I don't know about H264 but on XVID the problem is that the encoder expects to be feeded with RC.601 and this is the main reason to use ColorMatrix, I've found mpeg2 streams to be in Rec.601 (Star Wars Ep. I to III) pal .

Just my 5 cents.

All the best,

Sharro

[chainring]

Neither you realize neither is DGIndex reporting incorrectly, the only thing is if there is no colorimetry flag present it will assume BT.709.

I don't know about H264 but on XVID the problem is that the encoder expects to be feeded with RC.601 and this is the main reason to use ColorMatrix, I've found mpeg2 streams to be in Rec.601 (Star Wars Ep. I to III) pal .

Just my 5 cents.

All the best,

SharroOh, please say you're kidding about DGIndex not reporting colorimetry correctly....please.

If this is truly the case, then how do we determine it? FYI: I'm primarily messing with R1 DVD's.

[Sharro]

Ufff once and for all:

DGIndex is reporting COLORIMETRY CORRECTLY except when NO colorimetry flag is present in the stream.

Then DGindex will by default show Bt709.

All the best,

Sharro

[chainring]

Ok, that's easy to understand and I gathered that much from other posts. Is there any way to determine the colorimetry when there is NO flag in the stream? I'm assuming not...

[Sharro]

From ColorMatrix Readme.html:

How do you know which set of coefficients were used when encoding a MPEG-2 stream? Usually, the coefficient information is stored in the header of the MPEG-2 file (the "matrix_coefficients" field in the "sequence display extension"). Newer versions of GSpot will be able to read and display this information. Also, DGDecode v1.20+ (with Mpeg2source(info=1)) can be used to view it. If this extension field is not present in the header of the MPEG-2 file, the specs say we are supposed to assume the default of Rec.709

All the best,

Sharro

[chainring]

Cool, that clears it right up. I appreciate your patience and the information.

[HanSolo00]

Neither you realize neither is DGIndex reporting incorrectly, the only thing is if there is no colorimetry flag present it will assume BT.709.

I don't know about H264 but on XVID the problem is that the encoder expects to be feeded with RC.601 and this is the main reason to use ColorMatrix, I've found mpeg2 streams to be in Rec.601 (Star Wars Ep. I to III) pal .

Just my 5 cents.

All the best,

Sharro

Thank you for clarifying how DGIndex handles MPEG-2. I see now some DVDs report SMPTE 170M and others report ITU-R BT.709.

The question still remains for HD material, though. Passing the hints to AVIsynth ColorMatrix will tell it that the source HD transport stream is of course Rec.709, and as I understand it, ColorMatrix will convert that to Rec.601. for subsequent processing. Is this correct? If so, where does the Rec.601 dependency occur? Does x264 or the other H.264 encoders expect Rec.601, or is ColorMatrix even needed at all for Rec.709 MPEG-2 -> H.264 through AVIsynth?

I'll look through the other forums to see if there is any info as well.

[Keepitsimple]

going from rec.709 to rec.601 isnt lossless right? so every encode from hd-dvd has wrong colors if x264 needs rec.601 ..

[HanSolo00]

going from rec.709 to rec.601 isnt lossless right? so every encode from hd-dvd has wrong colors if x264 needs rec.601 ..
I just did a simple visual test without ColorMatrix, taking a frame from a MPEG-2 HD Rec.709 source, and the source frame looks identical in color (in MPC using MPV decoder filter) to the final x264-encoded H.264 frame (viewed using CoreAVC decoder in MPC). So, unless I'm doing something wrong, it would appear the source color is preserved through the whole x264 encoding/playback process without using ColorMatrix.

[Wilbert]

The question still remains for HD material, though. Passing the hints to AVIsynth ColorMatrix will tell it that the source HD transport stream is of course Rec.709, and as I understand it, ColorMatrix will convert that to Rec.601. for subsequent processing. Is this correct? If so, where does the Rec.601 dependency occur? Does x264 or the other H.264 encoders expect Rec.601
All MPEG-4 DEcoders expect Rec.601. That's the whole problem. If (1) the coefficients could be stored in the MPEG-4 stream (just as in any MPEG-2 stream) and (2) those coefficients are passed to the decoder then there would never be a problem.

I just did a simple visual test without ColorMatrix, taking a frame from a MPEG-2 HD Rec.709 source, and the source frame looks identical in color (in MPC using MPV decoder filter) to the final x264-encoded H.264 frame (viewed using CoreAVC decoder in MPC). So, unless I'm doing something wrong, it would appear the source color is preserved through the whole x264 encoding/playback process without using ColorMatrix.
Then you are doing something wrong. Could you upload such a small HD clip?

[HanSolo00]

Then you are doing something wrong. Could you upload such a small HD clip?
:thanks: for replying, I really appreciate it.

Here are some test files I'm playing with:

Sample Files (http://www.darkvault.org/doom9)

The sample clip is a ~5MB MPEG-2 .m2ts (from Blu-Ray) which because it is MPEG-2 HD should be Rec.709. The color in the frames I'm comparing look the same to my eyes, other than a slight difference in brightness on the H264 frame grabbed when using the 'VMR9' renderer instead of 'System Default' renderer in MPC.

When I use ColorMatrix(mode="Rec.709->Rec.601") there is a big difference in color; the image isn't nearly so dark red.

What am I doing wrong :confused:

[G_M_C]

I had a quick look at your sample AVS; Why are you using DirectShowSource ? Isnt it possible to ude DGIndex/DGMPGdec with MPEG2Source () ? When you use that, you can put in MPEG2Source (......, info=3) and colormatrix (......., info=true) wich imho should work better.

Or am i wrong here, and doesnt DGIndex work on those MPEG2 files ? (in wich case, ignore my posst ;) )

Anyway, regarding your earlier posting; Rec601 is expected by for instance XviD, and rec709 is expected by CCE. In the last case for example; When you feed rec601 to CCE you get a (much) darker image. You'll have to convert to the matrix that is required by the encoder you are going to use. Read in the forums/help-file and other places to find this out, or maybe ask about it. You'll only need to find it out once ;)

If your encoder accepts rec709, and your source is 709; Colormatrix shouldn't be needed at all, wich makes the whole affair a bit simpler :)

[HanSolo00]

I had a quick look at your sample AVS; Why are you using DirectShowSource ? Isnt it possible to ude DGIndex/DGMPGdec with MPEG2Source () ? When you use that, you can put in MPEG2Source (......, info=3) and colormatrix (......., info=true) wich imho should work better.

Or am i wrong here, and doesnt DGIndex work on those MPEG2 files ? (in wich case, ignore my posst ;) )
Well, the file isn't a regular Transport Stream (.ts), it's from a Blu-ray disc (.m2ts), and DGIndex didn't like it when I tried to open it. So, I let MeGUI decide how to open it, and it defaulted to DirectShowSource. Incidentally, I suspected something might be up with that, so I tried the same type of frame compares with a regular HD transport stream & DGIndex, and I got the same results--the colors in the source TS looked the same to me as the H264 frame when I left everything Rec.709.

Anyway, regarding your earlier posting; Rec601 is expected by for instance XviD, and rec709 is expected by CCE. In the last case for example; When you feed rec601 to CCE you get a (much) darker image. You'll have to convert to the matrix that is required by the encoder you are going to use. Read in the forums/help-file and other places to find this out, or maybe ask about it. You'll only need to find it out once ;)

If your encoder accepts rec709, and your source is 709; Colormatrix shouldn't be needed at all, wich makes the whole affair a bit simpler :)This is why I'm confused, I thought ColorMatrix is needed for Rec.709 MPEG-2->MPEG-4 encoding for proper color on playback but that isn't what I'm seeing. Probably Wilbert will know why. It's probably something dumb that I've overlooked.

[foxyshadis]

All MPEG-4 DEcoders expect Rec.601. That's the whole problem. If (1) the coefficients could be stored in the MPEG-4 stream (just as in any MPEG-2 stream) and (2) those coefficients are passed to the decoder then there would never be a problem.

There is, actually, and x264 actually exposes those options. xvid_encraw could as well. But no software player has a way to get that information from the bitstream to the renderer, since most windows players use VIDEOINFOHEADER[2] and BITMAPINFOHEADER, with no such field, and others use their own equivalents. Like range conversion, it seems hard to get anyone to care about it. =\

FFDshow could probably stand to properly decode AVC with the VUI flags set, though. I'll file an enhancement request, and hope someone might know how to get that info out of lavc.

[Wilbert]

FFDshow could probably stand to properly decode AVC with the VUI flags set, though. I'll file an enhancement request, and hope someone might know how to get that info out of lavc.
Keep me updated!

@HanSolo00, I will look at your sample. Btw, I have seen some MPEG-2 HD streams that are not Rec.709.

[HeadBangeR77]

DGIndex is reporting COLORIMETRY CORRECTLY except when NO colorimetry flag is present in the stream. Then DGindex will by default show Bt709.
Since I'm not getting any colorimetry info with G-Spot nor MediaInfo, AVInaptic isn't useful for MPEG-2 streams, and DGIndex shows BT.709, then I can assume there isn't any colorimetry information in the video stream I've written about above. Hence, as Didee mentioned once, it could be anything.

Could I assume then, that the colours displayed by all software DVD players/decoders I use are the correct ones? In such a case I should encode without Colormatrix, since it alters the colours differently from those seen by MPEG-2 playback. Or am I missing sth' obvious here? I'm already tired of all this. :/

Thanks in advance,
HDBR77

[Wilbert]

Since I'm not getting any colorimetry info with G-Spot nor MediaInfo, AVInaptic isn't useful for MPEG-2 streams, and DGIndex shows BT.709, then I can assume there isn't any colorimetry information in the video stream I've written about above. Hence, as Didee mentioned once, it could be anything.
Like Sharro said, and like i wrote in the readme file:

If this extension field is not present in the header of the MPEG-2 file, the specs say we are supposed to assume the default of Rec.709.
So in that case it could NOT by anything, but it should be Rec.709.

[HeadBangeR77]

Like Sharro said, and like i wrote in the readme file... So in that case it could NOT by anything, but it should be Rec.709.
Excuse me, Wilbert, but I've read the Readme-file and the last few pages more than once, so I really know, that your assumption is Rec.709, and why you've decided to choose this way. I wrote: "it could be anything" - maybe too strong use of modal verbs, I'm gonna try again then: "it might be anything, although it should be Rec.709".

Now plz, could sb' answer straight: May I assume in such a case, that the decoded MPEG-2 stream is properly decoded as to colorimetry by all the applications /decoders I've mentioned at least twice over the last two pages? If so, may I assume the colours from the original source are Rec.601 then, since I'm getting identical results with MPEG-4 sample encodes without ColorMatrix?

Thanks in advance, and I promise not to interfere in this thread again.

[kumi]

HeadBangeR77: I'm having a hard time understanding your problem. You want to know if various MPEG2 decoders correctly display colors on a clip with unspecified colorimetry?

If they follow the spec, and the source was indeed encoded with Rec.709, then the decoder output should be accurate.

If they follow the spec, and the source was not encoded with Rec.709, then the decoder output should be inaccurate.

As to your question of how to determine the coefficients used in an MPEG2 clip with unspecified colorimetry, my guess is that it's just impossible to tell. Someone correct me if I'm wrong.

[HeadBangeR77]

HeadBangeR77: I'm having a hard time understanding your problem. You want to know if various MPEG2 decoders correctly display colors on a clip with unspecified colorimetry?
Hi! Yes, exactly - I already know my sample encodes without Rec.709=>Rec.601 conversion look the same as the source, yet I've just come to conclusion today (it was yesterday actually ;)) the source itself doesn't contain any colorimetry info, or so it seems at least. Hence I can't be sure the decoded video is properly displayed as to colours, if I don't know what players/decoders (PowerDVD 5 & 6, MPC with internal libmpeg2 decoding, MPC with ffdshow libmpeg2 decoding) assume in such cases.

If they follow the spec, and the source was indeed encoded with Rec.709, then the decoder output should be accurate.
If they follow the spec, and the source was not encoded with Rec.709, then the decoder output should be inaccurate.
In other words, in case there is no colorimetry info present in the stream, they assume Rec.709, right? Or at least they should - mind they never obey ITU PARs, no matter what.

As to your question of how to determine the coefficients used in an MPEG2 clip with unspecified colorimetry, my guess is that it's just impossible to tell. Someone correct me if I'm wrong.
I was just trying to find out how the original should look like. Since there's whole lot of assumptions and uncertainties in the given case, I think I will just encode the film as it is, without ColorMatrix.

:thanks: anyway ;)

[kumi]

I was just trying to find out how the original should look like. Since there's whole lot of assumptions and uncertainties in the given case, I think I will just encode the film as it is, without ColorMatrix.That's probably the right choice. If your MPEG2 source is Rec.709, encoding to MPEG4 with and without colormatrix(mode="Rec.709->Rec.601") will look different. If your source is Rec.601, both encodes should look the same.

[foxyshadis]

If it's already Rec.601 and there's no hints/d2v to stop colormatrix, it'll happily convert it. Not to any particular standard now, since it just left all that behind, but you can stack ten of them to get some very rich color, or dull if you go the other way. (At which point even blind foxy here can notice the effect, finally.)

[Wilbert]

@HeadBangerR77,

Please upload a short sample source clip.

1) VOB - MPC (latest Celtic Druid's build), ffdshow (I can't recall the revision, but it was from December/January) libmpeg2 decoding, high quality YUV to RGB32 conversion. (...)
Hi! Yes, exactly - I already know my sample encodes without Rec.709=>Rec.601 conversion look the same as the source, yet I've just come to conclusion today (...)

I'm pretty sure this is false, and you are doing the comparison incorrectly. I think the problem is that you use ffdshow's "high quality YUV to RGB32 conversion". I'm pretty sure that this conversion uses Rec.601 coefficients, and you can't choose the Rec.709 coefficients. If this is true, then of course, you won't see any difference, because in BOTH cases (and both of them are wrong) Rec.601 coefficients are used for conversion to RGB.

[HeadBangeR77]

Hello, Wilbert!

A sample is still available here, in the "Holy War" ;) thread:
http://forum.doom9.org/showpost.php?p=947397&postcount=85

If you could figure sth' out from that crappy stream, I would be very much obliged. :)
The problem is all MPEG-2 decoders, with or without the mentioned conversion (MPC's internal decoding doesn't do any conversion AFAIK, and so PowerDVD), display the source as if it was Rec.601 (they look like my sample encodes w/o ColorMatrix, or preview in VDub /AVSP without Rec.709=>Rec.601), though it's Hollywood's blockbuster, so I would expect it to be Rec.709 (or at least my nose tells me so ;)).

thanks very much in advance,
HDBR77

[HanSolo00]

Just did another little test on an HD broadcast MPEG-2 1080i transport stream. DGindex reports Rec.709 (as one would expect with HD MPEG-2), and I used DGindex to framegrab instead of MPC, thinking perhaps MPC's MPEG-2 decoder was introducing something unexpected. Then I compressed the clip to H.264 (with x264), without using ColorMatrix. The output color matches the source color exactly (visually, anyhow.)

When I instead let ColorMatrix detect and use the (Rec.709) hints from the D2V, it of course adjusts the colors (Rec.709->Rec.601) and the resulting H.264 colors don't match the source frame (difference is quite easy to see, the adjusted reds are more of an orange than the original deep red in the original Rec.709 MPEG-2 frame.)

This would seem to tell me:
1) my process for comparison is flawed because I have overlooked something (perhaps both the MPEG-2 decoder and MPEG-4 decoder assume Rec.601 even though it is Rec.709), or
2) there is no need for ColorMatrix adjustment for Rec.709 MPEG-2 -> x264 MPEG-4, because without it, the decoded H.264 color already matches the source MPEG-2 color.

[HeadBangeR77]

(...)difference is quite easy to see, the adjusted reds are more of an orange than the original deep red in the original Rec.709 MPEG-2 frame.
I already thought I was the only one crazy here. :D
I've got exactly the same results - reds are too bright and more sort of orange; have a look at greens - they also change.

This would seem to tell me:
1) my process for comparison is flawed because I have overlooked something (perhaps both the MPEG-2 decoder and MPEG-4 decoder assume Rec.601 even though it is Rec.709)
This is exactly what I'm thinking about both in your and my case, though we encode with different codecs.

cheers,
HDBR77

EDIT:
@ Wilbert,
I've come to conclusion the other sample isn't suitable enough for the task. I'm just uploading another one, with much reds and greens, about 1 minute long. I will update this post, when I've uploaded it (Force Film for 2dv project works flawlessly).

The Sample (http://xasonline.info/headbanger/BlackPearl.m2v)

PS. Don't be surprised with the poor quality - it's been shrunk a long time ago - the original lies broken into two pieces, and I'm at fixing the poor back-up atm.

[Wilbert]

EDIT:
@ Wilbert,
I've come to conclusion the other sample isn't suitable enough for the task. I'm just uploading another one, with much reds and greens, about 1 minute long. I will update this post, when I've uploaded it (Force Film for 2dv project works flawlessly).
Could you tell me exactly (meaning decoder, player, script, etc...) what i should do with your clip to make the problem visible?

If i make a screenshot of avs (with ColorMatrix(mode="Rec.709->Rec.601")) and of DGIndex, the picture is exactly the same (as it should). If i make a screenshot with MPC (playing the m2v) it is different (as it should because you didn't correct anything).

[HanSolo00]

If i make a screenshot of avs (with ColorMatrix(mode="Rec.709->Rec.601")) and of DGIndex, the picture is exactly the same (as it should). If i make a screenshot with MPC (playing the m2v) it is different (as it should because you didn't correct anything).

I'm seeing the same (expected) behavior as Wilbert with the test clip.

[HeadBangeR77]

Hello,
The matter is we're obviously getting different results using the same software. Could you post the screen grabs you've mentioned? Plz save mine (beneath) to HDD and view at full-screen...

If i make a screenshot of avs (with ColorMatrix(mode="Rec.709->Rec.601")) and of DGIndex, the picture is exactly the same (as it should).
The problem is, I'm not getting the same picture in terms of colours with ColorMatrix(mode="Rec.709->Rec.601") avs preview (the same applies to sample encodes I've made using ColorMatrix):

1) Just ColorMatrix(mode="Rec.709->Rec.601") (AVSP 1.3.7, ColorMatrix v2.1)

http://www.hidebehind.com/thumbs3/E91CC480.png (http://www.hidebehind.com/E91CC480)

2) As above + hints enabled (looks exactly the same to my eyes)

http://www.hidebehind.com/thumbs3/19D0F8D2.png (http://www.hidebehind.com/19D0F8D2)

and DGIndex:
3) DGIndex 1.4.9 beta 14

http://www.hidebehind.com/thumbs3/E02D339E.png (http://www.hidebehind.com/E02D339E)

If i make a screenshot with MPC (playing the m2v) it is different (as it should because you didn't correct anything).
So you're saying MPC (I've got the same results with MPC and e.g. PowerDVD 5/6) isn't decoding the source properly in terms of colours? I don't understand what should I change in the m2v, since it's just a part cut out of the original VOB (?). I don't know how I could made myself clearer? If the above players fail with this source, I can also get wrong colours with other MPEG-2 sources - that's the matter.

4) MPC (internal libmpeg2 decoding), VOB/m2v ( both look the same, really):

http://www.hidebehind.com/thumbs3/6AE7CF37.png (http://www.hidebehind.com/6AE7CF37)

5) MPC, sample encode, ffdshow rev. 1006, ICL9, XviD lib:

http://www.hidebehind.com/thumbs3/D22CFE3C.png (http://www.hidebehind.com/D22CFE3C)

1 & 2 look different (to my eyes there's too much orange in the reds there) than 3-5 (the reds are deeper and seem more natural to my eyes). So to your eyes the first two ones look correctly (plz, forget about technical matters, and just have a look at them)?

cheers,
HDBR77

PS. AviSynth 2.5.7 final.

[Wilbert]

Regarding 1,2,3. Could you redo the screenshots in the following way:

a) open the scripts in 1,2 in VDub and: video -> snapshot source frame.

b) regarding 3: use latest DGIndex (i'm using 1.4.6). Open m2v: file -> save bmp.

My apologies if you have done it this way, but it's not clear from your post.

So you're saying MPC (I've got the same results with MPC and e.g. PowerDVD 5/6) isn't decoding the source properly in terms of colours?
That depends on who is doing the YCbCr -> RGB conversion in MPC. I don't think MPC does this conversion itself, but i might be wrong.

1 & 2 look different (to my eyes there's too much orange in the reds there) than 3-5 (the reds are deeper and seem more natural to my eyes). So to your eyes the first two ones look correctly (plz, forget about technical matters, and just have a look at them)?
I'm not saying which ones *look* correct, that's a different question.

My screenshots:

screenshot of avs with ColorMatrix(mode="Rec.709->Rec.601"):

http://img180.imageshack.us/img180/1963/blackpearlavsio2.th.png (http://img180.imageshack.us/my.php?image=blackpearlavsio2.png)

and of DGIndex:

http://img126.imageshack.us/img126/9489/blackpearldgindexvm1.th.png (http://img126.imageshack.us/my.php?image=blackpearldgindexvm1.png)

MPC:

http://img126.imageshack.us/img126/3695/blackpearlmpcjd4.th.jpg (http://img126.imageshack.us/my.php?image=blackpearlmpcjd4.jpg)

[Alain2]

If ffdshow is used by mpc for the video, by default its yv12 output is tv scaled, not PC scaled, contrary to all other output modes. Only way around I use is to disable in ffdshow all output colorspace but rgb32, and tick the high quality yv12->rgb convertion box. That way it properly scales the yv12 input. It is slightly more expensive on the cpu usage as it's software convertion but it's not very significant I found on my average PC. This is a known "bug" (or developper's choice).

I didn't read the thread completely, if this is not related to this post I apologize

[Wilbert]

If ffdshow is used by mpc for the video, by default its yv12 output is tv scaled, not PC scaled, contrary to all other output modes.
If you mean that it is converted to RGB [16,235] by default, then you are wrong. It depends on the driver of your card:

http://forum.doom9.org/showthread.php?t=106111

[HeadBangeR77]

Hi! I'll be home tomorrow afternoon, and I'll post the screen grabs here.

Regarding 1,2,3. Could you redo the screenshots in the following way:

a) open the scripts in 1,2 in VDub and: video -> snapshot source frame.
I can already tell you they look the same.

b) regarding 3: use latest DGIndex (i'm using 1.4.6). Open m2v: file -> save bmp.
I could use 1.4.8 (the latest stable) - the older ones I've deleted from my HDD (that's why I'm using 1.4.9 beta14, cause it seems stable enough).

That depends on who is doing the YCbCr -> RGB conversion in MPC. I don't think MPC does this conversion itself, but i might be wrong.
I've excluded the conversion - the VOB/m2v screen shot is pure source YV12 (look at the "nice" stair-stepping on the soldiers' outfit); while the encode is forced YUY2.

I'm not saying which ones *look* correct, that's a different question.
Right, yet I'm the kind of guy the eyes have a priority before all other technical matters. ;) that's why I've asked you, which one looked more natural to your eyes.

My screenshots:
screenshot of avs with ColorMatrix(mode="Rec.709->Rec.601"):

http://img180.imageshack.us/img180/1963/blackpearlavsio2.th.png (http://img180.imageshack.us/my.php?image=blackpearlavsio2.png)

and of DGIndex:

http://img126.imageshack.us/img126/9489/blackpearldgindexvm1.th.png (http://img126.imageshack.us/my.php?image=blackpearldgindexvm1.png)
Both look ok, yet more similar to my results without ColorMatrix, so I'm just wondering could it be some kind of e.g. graphic card drivers issue (?)

MPC:
http://img126.imageshack.us/img126/3695/blackpearlmpcjd4.th.jpg (http://img126.imageshack.us/my.php?image=blackpearlmpcjd4.jpg)
This one looks different because it's flawed with TV-scale bug (sorry, I've know the bastard for so many years I recognize it immediately), so the colours are dimmed and washed out, I've got no doubt... You've used VMR9 renderless in MPC, haven't you? I stick to VMR7 windowed, which works fine for me (PC-scale) - the same as overlay. VMR9 is too buggy because of our gracious Nvidia/ATI 2d drivers to be reliable enough for such a comparison (they're fixing it and then breaking it down again with mad irregularity).

thanks very much for your attention.

PS. I'm not using ffdshow for MPEG-2 decoding, as I've stated above, yet ffdshow libmpeg2 looks exactly the same as MPC's internal libmpeg2 decoding & Power DVD 5&6 & Nero ShowTime.

[Alain2]

If you mean that it is converted to RGB [16,235] by default, then you are wrong. It depends on the driver of your card:

http://forum.doom9.org/showthread.php?t=106111
Yes sorry that's what I meant, was a bit fast. And I think it's not possible to change this with my card driver (ati catalyst) ; I think it's the same for a lot of people (maybe nvidia as well) as this issue often comes back when I discuss on other boards. Also I still think it's ffdshow specific, using something else to read the same file does not convert to 16..235 (well last time I tried anyway, don't have anything else than ffdhow now)

[HeadBangeR77]

@ Alain2
I don't think it's ffdshow specific: I've got similar washed out & dimmed colours in MPC with its internal MPEG-2 decoding, or just using XviD 1.1.2 VfW decoding. It affects both VMR7 & 9 renderless. No problems here while using VMR7 windowed or Overlay. And yes, foxyshadis has mentioned more than once it's 2D drivers specific issue.

@ Wilbert
I was having hard time today trying to grab exactly the same frame in all applications, especially with DGIndex and Nero ShowTime, yet I've just managed it. Will upload the screen grabs soon.

1) Just ColorMatrix(mode="Rec.709->Rec.601") (VDub 1.7.1, ColorMatrix v2.1)
2) As above + hints enabled
3) DGIndex 1.4.8 preview

The above three up-sample chroma to RGB 24/32 by default - I've just discovered - there's no sign of colour bleeding or stair-stepping.

4) MPC (internal libmpeg2 decoding, YV12, no chroma up-sampling nor any other colour conversion)
5) Nero ShowTime (Nero Digital's internal splitter and decoder, default colour setting, YV12)

cheers,
HDBR77

Here we go:

1) http://www.hidebehind.com/thumbs3/605B2C09.png (http://www.hidebehind.com/605B2C09) 2) http://www.hidebehind.com/thumbs3/BC1D2A08.png (http://www.hidebehind.com/BC1D2A08)

3) http://www.hidebehind.com/thumbs3/2E05C2DB.png (http://www.hidebehind.com/2E05C2DB) 4) http://www.hidebehind.com/thumbs3/61DEE52D.png (http://www.hidebehind.com/61DEE52D) 5) http://www.hidebehind.com/thumbs3/4B89255C.png (http://www.hidebehind.com/4B89255C)

[HeadBangeR77]

1) http://www.hidebehind.com/thumbs3/6296135B.png (http://www.hidebehind.com/6296135B) 2) http://www.hidebehind.com/thumbs3/44702F4C.png (http://www.hidebehind.com/44702F4C)

3) http://www.hidebehind.com/thumbs3/3BEDF564.png (http://www.hidebehind.com/3BEDF564) 4) http://www.hidebehind.com/thumbs3/C0E28100.png (http://www.hidebehind.com/C0E28100) 5) http://www.hidebehind.com/thumbs3/124C296.png (http://www.hidebehind.com/124C296)

***
Here the orange reds look the worst imo:

1) http://www.hidebehind.com/thumbs3/99F38CB3.png (http://www.hidebehind.com/99F38CB3) 2) http://www.hidebehind.com/thumbs3/84BE031C.png (http://www.hidebehind.com/84BE031C)

3) http://www.hidebehind.com/thumbs3/5B7CCCE0.png (http://www.hidebehind.com/5B7CCCE0) 4) http://www.hidebehind.com/thumbs3/713A8BEA.png (http://www.hidebehind.com/713A8BEA) 5) http://www.hidebehind.com/thumbs3/C1451EE0.png (http://www.hidebehind.com/C1451EE0)


cheers,
HDBR77

[Wilbert]

Ok, first a possible misconception:

I've excluded the conversion - the VOB/m2v screen shot is pure source YV12 (look at the "nice" stair-stepping on the soldiers' outfit); while the encode is forced YUY2.
What ever you do, the clip is always converted to RGB upon display. If you enable forced YUY2 in ffdshow, then your system default YUY2 codec is converting it to RGB. (You can change that in your registry by setting "VIDC.YUY2"="huffyuv.dll" for example; see faq for more on this.)

1) Just ColorMatrix(mode="Rec.709->Rec.601") (VDub 1.7.1, ColorMatrix v2.1)
2) As above + hints enabled
3) DGIndex 1.4.8 preview
Ok, I also installed DGIndex 1.4.8: 1,2 is the same and 3 (saves as a bmp) is different (i guess the behaviour changed wrt to 1.4.6). I think it's a bug in DGIndex. I will inform neuron2 about it.

As for your pics 4 and 5, i think that both of them are wrong. I'm not sure who is doing the conversion to RGB here, but i think that Rec.601 is used.

Right, yet I'm the kind of guy the eyes have a priority before all other technical matters. that's why I've asked you, which one looked more natural to your eyes.
Don't get me wrong, since this is an important question. But if the colorimetry is wrong, it doesn't mean it is always wrong. You have to manually judge whether it is wrong in a particular case an act upon that. It's certainly wrong to assume it is always wrong.

[HeadBangeR77]

Thanks very much for the quick answer. :)

(...)
What ever you do, the clip is always converted to RGB upon display. If you enable forced YUY2 in ffdshow, then your system default YUY2 codec is converting it to RGB. (You can change that in your registry by setting "VIDC.YUY2"="huffyuv.dll" for example; see faq for more on this.)
I must admit I don't understand this entirely. Previewing with VDub didn't give me any artefacts typical for 4:2:0 colour compression, because it defaults in options to RGB24. I assume DGIndex does very similar or even exactly the same thing, so no colour bleeding nor stair-stepping.

Yet while decoding MPEG-2 (most of them are YV12 for compression's sake) or MPEG-4 ASP (Divx, XviD) I always get the mentioned artefacts, no matter what player and decoder I use. How could it be a clip is converted to RGB upon display and keeps artefacts typical for chroma sub-sampling (as in JPEGs)? The only thing I can do then is to use ffdshow and force either YUY2 output colour space or the already mentioned HQ YV12=>RGB conversion, again as output colour space. Only then the artefacts disappear (I made dozens of screen-shots once). Standard MPEG-2 decoding with either MPC or PowerDVD or Nero ShowTime also has stair-stepping. Where is the conversion then? :confused:


Ok, I also installed DGIndex 1.4.8: 1,2 is the same and 3 (saves as a bmp) is different (i guess the behaviour changed wrt to 1.4.6). I think it's a bug in DGIndex. I will inform neuron2 about it.
I've seen your post and backed you up. ;)

As for your pics 4 and 5, i think that both of them are wrong. I'm not sure who is doing the conversion to RGB here, but i think that Rec.601 is used.
I also think Rec.601 is used, yet the stream doesn't seem to have its own colorimetry info, so it's an assumption it should be Rec.709 (which is true in most of the cases).

Hence...
Don't get me wrong, since this is an important question. But if the colorimetry is wrong, it doesn't mean it is always wrong. You have to manually judge whether it is wrong in a particular case an act upon that. It's certainly wrong to assume it is always wrong.
I get it. After all that discussion, from which I've learnt a lot in the meantime, I think I'm gonna encode this one into two versions (with and without ColorMatrix). Most of the effects with CM are pleasant to my eyes, yet the orange reds look totally unrealistic to me.

Thank you very much for all the help.
cheers,
HDBR77