Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 14th August 2015, 17:38   #21  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,060
shit won't happen to the color if you applied a casual twisted color system to some vid, it shares the same CIE result like the one with a "scientific" color system anyways, doesn't really matter if the vid is 4:4:4, scientific or not, it always leads to the same CIE value, and that's why matrix like "YCgCo" exists.
but if the vid got a subsampled chroma, that would be a whole new story, the wrong matrix would send some luminance to chroma planes, and chroma planes will be subsampled and we all know what will happen next...
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 15th August 2015, 07:22   #22  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 3,483
I'll confess I'm not really sure if that was an answer to my question.
hello_hello is offline   Reply With Quote
Old 15th August 2015, 07:48   #23  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,060
And I'm not really sure what was the question
Like I said, shit won't happen color wise, the down side is subsample related, if subsample didn't exist, YCgCo would be way better than 601 or 709, it gives zero rounding error converting between YUV and RGB
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 15th August 2015, 22:10   #24  |  Link
Wilbert
Moderator
 
Join Date: Nov 2001
Location: Netherlands
Posts: 6,295
Quote:
No. ColorMatrix(hints=true) is placebo and in most real-life situations does nothing with regard of the colorimetry because most streams on dvds do not have colorimetry info specified and assumed to be Rec.601 by DGDecode. (and if it will be specified, it will be Rec.601 anyway)
I'm pretty sure it will assume Rec.709 when the colorimetry is not specified in the header. Perhaps that was only the case in older versions, but i'm too lazy to look it put in its documentation.
Quote:
I guess only author of the plugin can answer that. Other than that, we can't even guess. That quote makes absolutely no sense.
"corrects the colors of mpeg2 streams of dvds" - there's nothing to correct
"encoded using a different set of coefficients as used by AviSynth's color conversion routines" - What different set? What routines? Avisynth is not converting anything. This is only matters when you convert from YUV to RGB, and you don't. And even if you do, avisynth supports both conversions, Rec601 and Rec709
Guess i'm the guilty one. Of course it only matters when converting from YUV to RGB (or back). That should be clear when reading the rest of the documentation. There used to be a time when the consensus here was that dvd's were encoded assuming Rec709. The next to last version of the MPEG-2 specs said that Rec709 should be assumed when the colorimetry was not specified in the header of the stream. In the last version they changed that to 'assumed to be implicitly defined by the application', in other words can be anything. See http://avisynth.nl/index.php/Colorimetry for more details.

The general consensus nowadays is that Rec601 should be assumed for dvd's, regardless when the colorimetry is present in the header. I'm not saying that's wrong, but we can't be 100 percent sure unless some actually got that from the dvd specs itself.

Btw, where is the quote "encoded using a different set of coefficients as used by AviSynth's color conversion routines" coming from? Perhaps i wrote that one day, but it is not in the current (or 2007) documentation.

Last edited by Wilbert; 15th August 2015 at 22:52.
Wilbert is offline   Reply With Quote
Old 16th August 2015, 03:48   #25  |  Link
Keiyakusha
契約者
 
Keiyakusha's Avatar
 
Join Date: Jun 2008
Posts: 1,567
Quote:
Originally Posted by Wilbert View Post
I'm pretty sure it will assume Rec.709 when the colorimetry is not specified in the header. Perhaps that was only the case in older versions, but i'm too lazy to look it put in its documentation.
Latest version definitely assumes 601 for SD, 709 for HD. Not sure about older ones... I'm curious to check, but no idea where to get them.
Edit: found some old ones. dgmpgdec150+ assume rec601.


Quote:
Originally Posted by Wilbert View Post
Btw, where is the quote "encoded using a different set of coefficients as used by AviSynth's color conversion routines" coming from?
Well I simply re-quoted the post above but I do recall seeing it somewhere.

Edit: By the way, first english-language link in google leads here, where it says something different:
Quote:
ColorMatrix corrects the colors of MPEG-2 streams of dvds. More correctly, many MPEG-2 streams use slightly different coefficients (called Rec.709) for storing the color information than AviSynth's color conversion routines or the XviD/DivX decoders (called Rec.601) do, with the result that DivX/XviD clips or MPEG-2 clips encoded by TMPGEnc/QuEnc are displayed with slighty off colors (which looks like a small difference in brightness). This can be checked by opening the MPEG-2 stream directly in VDubMod.
... but even if we'll ignore Rec.709 and TMPGEnc (no idea how it works) parts, the bold part is still unclear. Why avisynth is mentioned here? I suppose some old avisynth version always assumed Rec601 and documentation was never corrected. But if it was meant as a workaround, it is strange that the plugin itself isn't performing YUV-RGB conversion. Then, there are decoders that always assume Rec.601 if RGB was requested? Even if yes, that's just a broken decoder, not to mention that usually RGB will not be requested (so the conversion will be done by downstream software using it). Adjusting video to match the behavior of some specific playback software is not really a good idea. So this part of the documentation is needlessly confusing (assuming this one is up to date)

Last edited by Keiyakusha; 16th August 2015 at 04:36.
Keiyakusha is offline   Reply With Quote
Old 16th August 2015, 05:11   #26  |  Link
OvejaNegra
ekTOMBE STUDIOS
 
OvejaNegra's Avatar
 
Join Date: Dec 2005
Location: Cuba
Posts: 250
Quote:
Not sure if I got you right, but assuming you have Rec601 YV12 source, to make it 709 this should do the job:
ConvertToRGB32(matrix="Rec601").ConvertToYV12(matrix="Rec709")
But this is more destructive than using colormatrix.
Got it, yes its very destructive. I was asking for the more "99.9999%" accurate conversion, but off course is too much.

Quote:
You're free to do it however you like, but me.... I try to make sure my SD encodes are BT.601 and my HD encodes are BT.709. If that means converting the colours when downscaling/upscaling, I convert them. That way, if a player isn't paying attention to the info written to the video stream and the colorimetry used is based on resolution, the video should display correctly.
i always assumed that was the main porpuse of colormatrix (because Xvid / MPEG4 dont have that info on the stream) even now with x264, there is the risk of decoders ignoring the info and work according to resolution.
__________________
So, it works or not???
OvejaNegra is offline   Reply With Quote
Old 16th August 2015, 10:18   #27  |  Link
DarkSpace
Registered User
 
Join Date: Oct 2011
Posts: 204
I see so many people here who ask themselves what can be done for people who use buggy software (video decoders that don't pass the colorimetry information to the video renderer, video renderers that ignore colorimetry information that they get from the decoder), so I feel I should raise the question of whom this truly benefits:
  • Those who use buggy software are unlikely to care about precise colorimetry. If they do care, they should switch to a different software anyway.
  • Those who use software that properly handles this stuff will, at best, experience no negative effects from such a conversion. Usually, I'd expect that the matrix adaption will indeed harm quality.
So, the matrix adaption brings no real benefit to those whom it's supposed to help (those who should change their playback environment anyway*), while harming those who are already doing it right.
* If anyone is going to mention ffdshow and avisynth processing, I'm just going to tell them to manually insert colormatrix into the avisynth script themselves, if ffdshow can't handle colorimetry.

You'll notice that I have little to no sympathy for people who use broken software. I think that is all right, though, because it's already 2015, and stuff like LAV Filters, which respects colorimetry information, has been around for a long time already. Additionally, I remember that, also quite a while back, MPC-HC fixed EVR (perhaps only EVR-CP, I don't remember that well) to respect colorimetry information also, so the point that there are too few common video renderers that respect colorimetry information, if present, is also invalid.

You'll also notice that I only mentioned software. If the goal is playback on hardware players, then my reasoning becomes invalid, because then the act of "simply switching playback environments" becomes that much harder. And although I'm still in favor of using something that actually does it right for playback, even in those cases, I can acknowledge that this is unreasonable.
DarkSpace is offline   Reply With Quote
Old 17th August 2015, 07:30   #28  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 3,483
I'm still runing XP on the PC I use as a media player. I have no particular need to change OS. It works fine. I'm fairly certain the colorimetry info is ignored when using the MPC-HC/EVR combination just as it is with most other renderers. I could use MadVR, but I prefer not to.

I care about colorimetry. It's why,
- I always use rec.601 for SD and rec.709 for HD.
- If I need to correct the colorimetry on playback, I can do it manually with a pixel shader. Or several other ways.

I'm still not clear as to how converting the colours "harms those who are already doing it right" (those using software that obeys the colorimetry, I assume), and in the real world encoded video is rarely only played back using a single device, software or hardware, and my switching to different playback software has no effect on the colorimetry that'll be used by the media player built into the TV in the family room, or the Bluray player connected to the TV in the living room etc. Using a colorimetry which is "most likely" to be displayed correctly seems logical to me.

Quote:
Originally Posted by Wilbert View Post
Btw, where is the quote "encoded using a different set of coefficients as used by AviSynth's color conversion routines" coming from? Perhaps i wrote that one day, but it is not in the current (or 2007) documentation.
I'd be guessing "AviSynth's color conversion routines" refer to Avisynth's colour conversions from RGB to YUV etc being rec.601 by default. "Encoded using a different set of coefficients" possibly refers to encoders that always write rec.709 to the video stream, or assume RGB should always be converting using rec.709, so if you want the video to display correctly by a player that obeys the colorimetry info......
Maybe there's no longer encoders like that, but the HCenc encoder is mentioned here:

Color coefficients and Colormatrix usage summary (Q and A)

Last edited by hello_hello; 17th August 2015 at 07:37.
hello_hello is offline   Reply With Quote
Old 17th August 2015, 07:36   #29  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 3,483
Quote:
Originally Posted by feisty2 View Post
And I'm not really sure what was the question
I have a Bluray video. I re-encode it. I use no other filtering aside from colormatrix to change the colorimetry to rec.601.
(Of course normally I wouldn't do so unless I was downscaling to SD).
On playback I use a player that obeys the colorimetry info written to the video stream. I use the same TV/monitor each time.

The original Bluray video would look like this:

Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709

After I convert the colours, I'd tell x264 to write this:

Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.601

The question was, how/why would they display differently?
I'm not sure how your references to subsampling apply.

Last edited by hello_hello; 17th August 2015 at 07:48.
hello_hello is offline   Reply With Quote
Old 17th August 2015, 07:48   #30  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 3,483
Quote:
Originally Posted by OvejaNegra View Post
i always assumed that was the main porpuse of colormatrix (because Xvid / MPEG4 dont have that info on the stream) even now with x264, there is the risk of decoders ignoring the info and work according to resolution.
Exactly, which is why it's a good idea to use rec.601 for SD and rec.709 for HD when encoding with Xvid, and convert the colors accordingly when downscaling/upscaling, because Xvid can't write the colorimetry info, so the playback device must assume a colorimetry based on resolution.
I still don't understand why that was best practice for years, but because x264 has the ability to write the colorimetry info, it's now a sin to follow the same rule.
hello_hello is offline   Reply With Quote
Old 17th August 2015, 07:56   #31  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,060
Quote:
Originally Posted by hello_hello View Post
I have a Bluray video. I re-encode it. I use no other filtering aside from colormatrix to change the colorimetry to rec.601.
(Of course normally I wouldn't do so unless I was downscaling to SD).
On playback I use a player that obeys the colorimetry info written to the video stream. I use the same TV/monitor each time.

The original Bluray video would look like this:

Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709

After I convert the colours, I'd tell x264 to write this:

Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.601

The question was, how/why would they display differently?
I'm not sure how your references to subsampling apply.
there will be NO color difference between these 2 copies
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 17th August 2015, 11:35   #32  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 3,483
Quote:
Originally Posted by feisty2 View Post
there will be NO color difference between these 2 copies
Well that makes me feel a bit better. If there was a reason why converting the colours would have a detrimental effect, I'd reconsider whether doing so was a good idea. Is there a circumstance when converting the colours that way while re-encoding video would be detrimental?

Of course much of the time the colour primaries, transfer characteristics and matrix coefficients aren't known. I just checked a Bluray rip on my hard drive and MediaInfo doesn't show info for any of them. So I guess all I can do is assume the source matrix coefficients are rec.709 and write that to the video stream when encoding, or write rec.601 if converting the colours. My understand is it's best not to specify anything for colour primaries or transfer characteristics unless you actually know what they should be, so unless I do, I don't. I assume that's the right thing to do?
hello_hello is offline   Reply With Quote
Old 17th August 2015, 12:00   #33  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,060
Quote:
Originally Posted by hello_hello View Post
Is there a circumstance when converting the colours that way while re-encoding video would be detrimental?

I assume that's the right thing to do?
Q1: like I said earlier, standalone matrix conversion will do some extra harm to luminance if your source file and encoded file are both not 4:4:4, because some luminance will be assumed as "chrominance" and get subsampled, and you lose some luminance data for this

Q2: I guess so.

EDIT:
the precise and more correct way to convert from HD to SD, SD to HD likewise
0. expand to PC range
1. resample to 444
2. convert to gamma compressed RGB with matrix "709"
3. convert to linear RGB with "709" transfer
4. convert to CIE XYZ with "709" primaries
5. convert to CIE Lab
6. scale to SD
7. convert back to CIE XYZ
8. convert to linear RGB with "601 NTSC" or "601 PAL" primaries
9. apply "709" transfer
10. apply "601" matrix
11. resample to 422/420 or keep it as 444
12. shrink to TV Range
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated

Last edited by feisty2; 17th August 2015 at 12:13.
feisty2 is offline   Reply With Quote
Old 18th August 2015, 05:21   #34  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 3,483
So when I asked earlier if the example source and encode will display the same way, the answer was yes, because the colours will display the same way, but now it's no because the luminance might be a little different?

To be honest, about the only time I convert the colours is when downscaling, or if for some reason I think the source was encoded with the wrong colorimetry, and if I'm re-encoding/downscaling I'm probably going to be applying other filtering such as noise removal and/or sharpening anyway. I might try the 12 step colour conversion method one day, but if I'm taking a high definition video and downscaling it to standard definition it'd have to offer a noticeable quality improvement. Considering the "damage" already being inflicted on the picture through downscaling and other filtering, I'm not sure it'd make enough difference to matter.
hello_hello is offline   Reply With Quote
Old 18th August 2015, 07:41   #35  |  Link
Sparktank
47.952fps@71.928Hz
 
Sparktank's Avatar
 
Join Date: Mar 2011
Location: Planet Express, Inc.
Posts: 788
Quote:
Originally Posted by feisty2 View Post
the precise and more correct way to convert from HD to SD, SD to HD likewise
0. expand to PC range
1. resample to 444
2. convert to gamma compressed RGB with matrix "709"
3. convert to linear RGB with "709" transfer
4. convert to CIE XYZ with "709" primaries
5. convert to CIE Lab
6. scale to SD
7. convert back to CIE XYZ
8. convert to linear RGB with "601 NTSC" or "601 PAL" primaries
9. apply "709" transfer
10. apply "601" matrix
11. resample to 422/420 or keep it as 444
12. shrink to TV Range
Interesting.
This almost looks like the dither_resize16 process.
Dither_convert_yuv_to_rgb(...)
Dither_y_gamma_to_linear
Dither_resize16
Dither_y_linear_to_gamma
Dither_convert_rgb_to_yuv(...)

Except, I dont apply 444 and PC range first (and TV range last, then 420/422). (and also don't do that CIE XYZ & CIE LAB thingy)
I've always been curious about that, though.
For DVD processing.
__________________
Win10 (x64) | GPU Caps Viewer v1.32.0.0
Crucial M500 240GB SSD | Kingston SSDNow V300 (Marvell) 120GB | NVIDIA GeForce GTX 750 Ti | R375.95 (Nov 18, 2016)
NTSC | DVD: R1 | BD: A
Sparktank is offline   Reply With Quote
Old 18th August 2015, 07:48   #36  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,060
Quote:
Originally Posted by Sparktank View Post
Interesting.
This almost looks like the dither_resize16 process.
Dither_convert_yuv_to_rgb(...)
Dither_y_gamma_to_linear
Dither_resize16
Dither_y_linear_to_gamma
Dither_convert_rgb_to_yuv(...)

Except, I dont apply 444 and PC range first (and TV range last, then 420/422). (and also don't do that CIE XYZ & CIE LAB thingy)
I've always been curious about that, though.
For DVD processing.

Dither_convert_yuv_to_rgb(...) = 0. 1. 2.
Dither_y_gamma_to_linear = 3.
Dither_resize16
Dither_y_linear_to_gamma = 9.
Dither_convert_rgb_to_yuv(...) = 10. 11. 12.
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 18th August 2015, 08:05   #37  |  Link
Sparktank
47.952fps@71.928Hz
 
Sparktank's Avatar
 
Join Date: Mar 2011
Location: Planet Express, Inc.
Posts: 788
saving.txt
__________________
Win10 (x64) | GPU Caps Viewer v1.32.0.0
Crucial M500 240GB SSD | Kingston SSDNow V300 (Marvell) 120GB | NVIDIA GeForce GTX 750 Ti | R375.95 (Nov 18, 2016)
NTSC | DVD: R1 | BD: A
Sparktank is offline   Reply With Quote
Old 18th August 2015, 08:14   #38  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,060
Quote:
and also don't do that CIE XYZ & CIE LAB thingy
CIE XYZ is required to convert primaries
CIE Lab is required to prevent linear light artifacts (dark halo, see http://www.imagemagick.org/Usage/resize/#resize_lab)
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 18th August 2015, 09:44   #39  |  Link
Sparktank
47.952fps@71.928Hz
 
Sparktank's Avatar
 
Join Date: Mar 2011
Location: Planet Express, Inc.
Posts: 788
Interesting reads.

Although, tyring to implement to YV12 movies (AVC/VC-1/MPEG2) with avisynth, I can't see myself converting a whole movie to PNGs for a process.

Still going through some limited google searches to this forum for CIE LAB.
There's a lot to catch up with and read.
Got myself a good mini-ebook to compile here.
__________________
Win10 (x64) | GPU Caps Viewer v1.32.0.0
Crucial M500 240GB SSD | Kingston SSDNow V300 (Marvell) 120GB | NVIDIA GeForce GTX 750 Ti | R375.95 (Nov 18, 2016)
NTSC | DVD: R1 | BD: A
Sparktank is offline   Reply With Quote
Reply

Tags
blu-ray, color, colormatrix, correction, h264

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 06:36.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2017, vBulletin Solutions Inc.