How necessary or important are the BT.709 parameters?

[sebazvideo]

Excuse my ignorance in this, but I realized from doing some tests that including "ConvertToYV12(matrix="pc.709")" in the Avisynth script as opposed to just "ConvertToYV12()" as MeGUI sets by default is absolutely necessary to ensure that the encode will have the same gamma as the original. Not knowing about this previously, I encoded something that had a lot of dark areas at some points in the footage and I could tell that the encode was noticeably brighter than the original. So I remembered reading about the matrix="pc.709" in a forum added to the avisynth script and after adding that to the script the gamma shift didn't occur anymore, and it looked just as good as the orignal.

But then I remembered reading about these parameters (--colorprim "bt709" --transfer "bt709" --colormatrix "bt709") in another thread and I'm wondering if it's necessary to add them or not, and what difference would it make one way or the other, given that I already specified BT.709 in the avisynth script. Can anyone clarify this for me? I searched for this in other threads and while there's related things here and there, I cannot find a definitive answer on this.

Thanks

Edit: sorry, after reading this post again I realize I had failed to see the difference between bt.709 (in the x264 parameters) and pc.709 (in the Avisynth script)

[mp3dom]

Personally I've never used pc.709 parameter in the ConvertToYV12 since the source material (mainly HD) is already bt709. I've never had a color shift from the master source anyway. I think it make sense if you comes from an RGB source which is quite uncommon (maybe for graphics). The BT709 in the h264 header is not mandatory... some pro-encoders set it and some others not (they also set the standard to NTSC). By default, 720p and up are bt709 but it could be that some decoder assume bt601 even if the source is HD. In that case the explicit bt709 flag could let the decoder understand the correct color primaries.

[G_M_C]

I use it allways, even though it might not be nessesary. It's not too much trouble to take this into account, and type the extra "matrix=xxxx". Guess i like to make shure to endresult is correct, and i take control over this to make shure. (Nothing wrong in making sure is there ?).

[poisondeathray]

Excuse my ignorance in this, but I realized from doing some tests that including "ConvertToYV12(matrix="pc.709")" in the Avisynth script as opposed to just "ConvertToYV12()" as MeGUI sets by default is absolutely necessary to ensure that the encode will have the same gamma as the original. Not knowing about this previously, I encoded something that had a lot of dark areas at some points in the footage and I could tell that the encode was noticeably brighter than the original. So I remembered reading about the matrix="pc.709" in a forum added to the avisynth script and after adding that to the script the gamma shift didn't occur anymore, and it looked just as good as the orignal.


I think you might be mistaken, unless you had CGI footage as a source? Most live action and HD camcorder footage will be Rec.709 (not PC.709), but using PC.709 will stretch the values 0-255 instead of 16-235

If this is an export from a NLE, the footage has been converted to RGB internally most likely and treatment will vary according to the different editors and what settings you have (this is opening a huge can of worms).

If you convert to PC.709 , these values <16 and >235 are illegal for DVD or blu-ray video (small excursions are fine) , and will look incorrect on regular TV setups.

You should do some background reading on PC vs. TV ranges.


But then I remembered reading about these parameters (--colorprim "bt709" --transfer "bt709" --colormatrix "bt709") in another thread and I'm wondering if it's necessary to add them or not, and what difference would it make one way or the other, given that I already specified BT.709 in the avisynth script. Can anyone clarify this for me? I searched for this in other threads and while there's related things here and there, I cannot find a definitive answer on this.


These parameters are important for some devices. E.g. popcorn hour needs these specified otherwise it displays as rec.601 and colors look slightly "off"

[Keiyakusha]

Afaik this is only for RGB-yuv conversion.
The only possible scenario for using ConvertToYV12(matrix="pc.709") i see is when you have HD (720p and up) RGB source and you want to convert it to YV12 colorspace for encoding and keep full luma range. But IMHO keeping full luma range is not really good idea since then h264 stream should be flagged accordingly and not every decoder respects this flag (at least thats how it was some time ago). So just to be safe i would use Rec709.

[sebazvideo]

I think you might be mistaken, unless you had CGI footage as a source? Most live action and HD camcorder footage will be Rec.709 (not PC.709), but using PC.709 will stretch the values 0-255 instead of 16-235

If this is an export from a NLE, the footage has been converted to RGB internally most likely and treatment will vary according to the different editors and what settings you have (this is opening a huge can of worms).

Well, this was from Vegas Pro 9.0d (latest version) and it came from footage I shot myself at a circus, sending it to MeGUI through Debugmode Framesever. I realized after encoding in MeGUI and authoring the BD9 that the luminance in the encode was quite higher than what I had seen in Vegas using the Secondary Monitor Preview. At first I thought that it may have been a matter of calibrating the gamma or brightness controls in my ATI card, but then I thought it could be related to the encoding, so after running some tests, the first encode that matched the original perfectly was after adding the "ConvertToYV12(matrix="pc.709")" in the Avisynth script. I also tried not adding the "matrix="pc.709" (MeGUI adds the ConvertToYV12() automatically) but setting Debugmode Frameserver to output YUV2 instead of RGB24, and that still resulted in luminance brighter than the original, so I concluded that adding "ConvertToYV12(matrix="pc.709")" to the Avisynth script is the only way to ensure the luma levels are correct.

I also tried bypassing the frame server and exporting one minute from Vegas to an Uncompressed RGB file and feed that into Megui, and the results were exactly the same.

[ronnylov]

Why are there three different parameters (--colorprim "bt709" --transfer "bt709" --colormatrix "bt709") instead of only one?
In what situations could they be set differently?

When downscaling from HD to SD I guess it is valid to convert the colormatrix to bt601.

[Keiyakusha]

after running some tests, the first encode that matched the original perfectly was after adding the "ConvertToYV12(matrix="pc.709")" in the Avisynth script

This could indicate that you have playback problems which is pretty common.
i assume that you know the difference betwen PC and TV luma ranges so you should use ConvertToYV12(matrix="pc.709") if your source is HD and you want PC levels. This is not something you can use all the time with any source.

[poisondeathray]

vegas treats footage and levels very differently if you are in 32-bit mode vs 8-bit mode. (debugmode only works in 8-bit mode, so I'm assuming you're using 8-bit mode). It does the Y'CbCr => internal RGB conversion differently depending on the source file, and what mode you are using

is your secondary monitor calibrated? if this is on a pc monitor, a calibrated adjusted video for dvd / blu ray on a typical pc monitor will usually look washed out

what do the scope and other waveform monitor devices look like? vegas' preview notoriously doesn't correlate with actual results.

There are a few articles by glenn chan which address vegas' quircks with colorspace (there are a few others on his website, I didn't link them all)
http://www.glennchan.info/articles/vegas/colorspaces/colorspaces.html
http://www.glennchan.info/articles/vegas/v8color/v8color.htm
http://www.glennchan.info/articles/vegas/v8color/vegas-9-levels.htm

But if you think you've figured out a workflow that works for you , then that's all that matters - but be aware you might be making not legal range video

[sebazvideo]

This could indicate that you have playback problems which is pretty common.
i assume that you know the difference betwen PC and TV luma ranges so you should use ConvertToYV12(matrix="pc.709") if your source is HD and you want PC levels. This is not something you can use all the time with any source.

What playback problems are you referring to?

[mp3dom]

If you're working with Vegas and DebugMode, set the DebugMode to output YUY2 colorspace. This will output a YUY2 colorspace with the same colors as your original timeline.

[Keiyakusha]

What playback problems are you referring to?

Problems with levels. Normally they should be scaled on playback. It is possible that they not scaled for you so pc.709 looks right.

Here is HD RGB Image (http://imgur.com/85xuc.png) (you can do some tests with it) and some encoding scenarios you should get when previewing on PC monitor using CoreAVC decoder on default settings (since it supports fullrange flag for sure). If we use:
Converttoyv12() which is equal to Converttoyv12(matrix="Rec601"), h264 stream is not flagged. -> result should have wrong colors but right luma levels.
Converttoyv12(matrix="Rec709"), h264 stream unflagged -> result should have right colors and right luma levels
ConvertToYV12(matrix="pc.709"), h264 stream unflagged -> result should have right colors but wrong luma levels
ConvertToYV12(matrix="pc.709"). h264 stream flagged as --fullrange on -> result should have right colors and right levels.
But do not use avisynth for previewing result.

Any other flags in h264 streams like --colorprim are not important since decision often done judging by resolution, however they may be absolutely required for your target hardware device. Or if you want to make HD with SD coefficients (or vice versa), but this is unlikely.

[sebazvideo]

vegas treats footage and levels very differently if you are in 32-bit mode vs 8-bit mode. (debugmode only works in 8-bit mode, so I'm assuming you're using 8-bit mode). It does the Y'CbCr => internal RGB conversion differently depending on the source file, and what mode you are using

I use only 8 bit mode, Vegas as it is is very slow with AVCHD footage, if I were to use 32 bit I couldn't even play it.

is your secondary monitor calibrated? if this is on a pc monitor, a calibrated adjusted video for dvd / blu ray on a typical pc monitor will usually look washed out

I didn't calibrate it, however, the luma levels (at least to my eye) are the same when viewing the exact frame of footage from the Vegas timeline and from the Sony BDP-S300 Blu-Ray player, if I either encoded the footage with Sony's AVC encoder, or with DFS>MeGUI>x264 using the ConvertToYV12(matrix="pc.709") line in the Avisynth script. This is obviously with both the computer and the BD player connected to the TV set's HDMI inputs, and both inputs set to exactly the same picture settings.

Now, if I don't enter the ConvertToYV12(matrix="pc.709") line in the script, then the resulting encode, both when seen from the computer or burned to a BD and played in the blu-ray player, has higher luma levels.

[sebazvideo]

If you're working with Vegas and DebugMode, set the DebugMode to output YUY2 colorspace. This will output a YUY2 colorspace with the same colors as your original timeline.

If you see one of my posts above, I already tried setting DFS to YUY2 but it didn't make a difference. If I don't ad the ConvertToYV12(matrix="pc.709") line in the script, it still results in the luma levels being brighter than the original.

[mp3dom]

Keep the DFS to output YUY2 and try to open the reference AVI with VirtualDub, set it to Fast Recompress and compress with HuffYUV or other lossless codec that works with YUY2 (e.g. Lagarith set to YUY2 or Ut Video Codec for YUY2). Then compare the file and the timeline. To me it keeps the originals luma/chroma values.

[poisondeathray]

If you are using 8-bit mode, native camcorder files (avchd, dv, hdv) when imported are decoded in vegas 8 & 9 using "studio rgb" or 16-235. By using ConvertToYV12(matrix="pc.709") , you would be streching the range when you do the conversion back to YV12 to 0-255 ("illegal" for video). But some hardware setups (players and TV's) will accept full range (0-255) signals, but I would say for the majority of setups this is problematic. If you use histogram() in your avs script (and just preview it in avsp or vdub from the debugmode output, you will see the illegal excursions <16, >235 as marked in the brown area). Anyways, if you've found something that works for your setup, then that's all that matters

[Keiyakusha]

Oh by the way. If it is possible that RGB output from Vegas have TV levels (which would be mistake but still...) then pc.709 would work!
Personally I never use non-rgb colorspace as output from NLE since I prefer to do all conversion in avisynth and there is no need to inspect the streams to be sure that they exported as I want.

[sebazvideo]

I'm trying to bypass MeGUi by using x264 from a command prompt, but it seems that it's a bit over my head. I send this command:

C:\Users\Sebastian\x264>x264.exe --level 4 --pass 2 --bitrate 16900 --stats ".stats" --deblock -6:-6 --keyint 30 --min-keyint 2 --b-pyramid strict --ref 4 --weightp 0 --vbv-bufsize 20000 --vbv-maxrate 20000 --no-fast-pskip --aud --nal-hrd vbr --tff --sar 1:1 --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --output d:\circus\YUY2 direct test.264 d:\Circus 2-28-10 multicam audio.avi

And I get this:

ffms [error]: could not create index
lavf [error]: could not open input file
yuv [error]: rawyuv input requires a resolution.
x264 [error]: could not open input file `direct' via any method!

I'm trying to search in forums and wikis for the right syntax, but if anyone can tell me what am I doing wrong it would save me a lot of time.

Edit: if I use the Avisynth script referencing to the dummy avi file created by DFS, it does the same thing.

[poisondeathray]

If it is possible that RGB output from Vegas have TV levels (which would be mistake but still...) then pc.709 would work!


Well I haven't used it in a while (debugmode doesn't work with 32bit per channel mode, which is nicer for color correction), but my earlier tests showed it decodes native camcorder files as TV levels (16-235), without clamping (so excursions are kept) which is in line with those Glenn Chan articles. BUT... the preview decodes with with Rec.601 (16-235) , so the preview is inaccurate, both in terms of levels and color. If debugmode in RGB mode is outputting TV levels, this would be the same as what vegas was using internally, (but different from what is shown on the normal preview)

[sebazvideo]

I tried bypassing the frameserver by exporting one minute of footage to Lagarith in each color space available in it, RGB, YV12 and YU2. I encoded all three in MeGUI, and then I modified the AVS script in the RGB file to ad the matrix="pc.709" to the Convert to YV12, which MeGUI inserted to be able to encode in x264. Except for the one where I added the matrix="pc.709", the rest of them show the brighter luma level, the one with matrix="pc.709" is the only one that looks exactly as the original.

What sucks is that I can't use "ConvertToYV12(matrix="pc.709")" in the Lagarith YV12 or YUY2 files, because they are less than half the size of the RGB Lagarith file. If I could use it I wouldn't care much about using the frameserver (which works only 1 in 5 times) because one minute in Lagarith YV12 or YUY2 is 1.29 GB, which is acceptable as far as drive space goes for a longer project.

[Keiyakusha]

Can you post small unprocessed RGB source that "looks like original only if you add matrix=pc.709"? Even screenshot will be fine.
EDIT: Oops, and already encoded one too! (here screenshot is not fine)
Otherwise, try to encode the image I posted before and check if result matches to what I posted.

[poisondeathray]

I just played with debugmode and vegas , and what he's describing is happening...but it's because the export from vegas seems to have a reduced dynamic range in 8bit mode (the highest whitepoint is reduced lowest dark point is raised), so when the PC.709 contrast expansion occurs in avisynth, it almost brings it back to "normal" (normal as in compared to the original footage in a proper decoder, not vegas). You can see this by examining a waveform monitor

Contrary to what he's been explaining "the others have a brighter luma" it's actually the opposite - a reduced dynamic range, and lower luma values

[Keiyakusha]

Ahh thats what I thought before. Anyway using PC.709 is not a good solution IMO...

[poisondeathray]

Anyway using PC.709 is not a good solution IMO...

I agree, for several reasons

Each colorspace conversion and expansion /compression in levels will cause more posterization and banding in 8-bit mode => you have to dither more, eats more bitrate

Overshoots are very common with PC.709 => not legal range, loss of detail and they will clip on regular setups

But given that he wants to do it this way with 8bit mode in vegas, there is no other way to "fix" the current behaviour of vegas

[sebazvideo]

I agree, for several reasons

Each colorspace conversion and expansion /compression in levels will cause more posterization and banding in 8-bit mode => you have to dither more, eats more bitrate

Overshoots are very common with PC.709 => not legal range, loss of detail and they will clip on regular setups

But given that he wants to do it this way with 8bit mode in vegas, there is no other way to "fix" the current behaviour of vegas

So you would say I don't have much choice but to use it that way since I'm using Vegas, correct?

Also, is there a way to do the matrix="pc.709" without the "ConvertToYV12"? Because that way I can export to Lagarith in either YUY2 or YV12 (I would like to know which of these would be best) and bypass the frameserver, at least until it works reliably in Vegas.

[poisondeathray]

So you would say I don't have much choice but to use it that way since I'm using Vegas, correct?


The way you are using it in 8-bit mode, I can't see any other way around it (except to do a levels conversion to stretch the contrast by a similar amount). 32-bit mode acts very differently.

[sebazvideo]

I just found something in an old Avisynth Wiki that seems to make sense in this situation (correct me if I'm wrong):

"In v2.56, matrix="pc.601" (and matrix="pc.709") enables you to do the RGB <-> YUV conversion while keeping the luma range, thus RGB [0,255] <-> YUV [0,255] (instead of the usual/default RGB [0,255] <-> YUV [16,235])."


To me this means that my original footage is using 0-255 levels, and Vegas keeps working in those levels, and exports also in those levels, but maybe Avisynth introduces the change in levels. I think that probably Avisynth assumes 16-235 levels unless you tell it to do 0-255. In that phrase you can read above it basically says that if you want to keep the full range then you have to specify the Matrix pc.709 in the Convertto YV12 command, so if I don't specify it, that's when the change occurs.

I'm trying to follow the path of the footage to troubleshoot this. In Vegas I see a certain luma level, and the exported file in Lagarith keeps the same levels because if I import that file into the same Vegas project and align it perfectly with the section that I exported, when I compare the original footage with the exported Lagarith footage, the luma levels are the same, at least to my eye. However, when I encode it to h.264 that's when it seems brighter, so I'm guessing it comes that way from Avisynth, because if x264 was introducing that luma shift, then the file that has the Convert to YV12 with Matrix PC.709 would also look brighter, and it doesn't.

Does this make sense?

[poisondeathray]

I just found something in an old Wiki that seems to make sense in this situation (correct me if I'm wrong):

"In v2.56, matrix="pc.601" (and matrix="pc.709") enables you to do the RGB <-> YUV conversion while keeping the luma range, thus RGB [0,255] <-> YUV [0,255] (instead of the usual/default RGB [0,255] <-> YUV [16,235])."

To me this means that my original footage is using 0-255 levels, and Vegas keeps working in those levels, and exports also in those levels, but maybe Avisynth introduces the change in levels. I think that probably Avisynth assumes 16-235 levels unless you tell it to do 0-255. I'm trying to follow the path of the footage. In Vegas I see a certain luma level, and the exported file in Lagarith keeps the same levels because if I import that file into the same Vegas project and align it perfectly with the section that I exported, when I compare the original footage with the exported Lagarith footage, the luma levels are the same, at least to my eye. However, when I encode it to h.264 that's when it seems brighter, so I'm guessing it comes that way from Avisynth, because if x264 was introducing that luma shift, then the file that has the Convert to YV12 with Matrix PC.709 would also look brighter, and it doesn't.

Does this make sense?

Not really, it doesn't correlate with my quick tests.

YOur footage does have ranges <16 and >235 however, (I downloaded from another thread), this is normal.

Avisynth only assumes 16-235 when you do the colorspace conversion to and from RGB (i.e. forgetting to stipulate the matrix in convertoyv12)

I can only explain share my observations. There might be other difference in our setups for example.

Lagarith isn't treated as completely lossless in vegas (the same is true in Premiere, although for slightly different reasons). It has a levels conversion and decoded to full range instead of studio RGB. So the compressed range on your first go round in 8-bit mode is re-expanded upon the lagarith import, that's probably why it "looks" similar. The problem is vegas.

If you want the bottom line, look at a scope or waveform monitor. They don't lie. Ie. compare the original footage with the various outputs like lagarith, debugmode dummy avi in various color spaces etc.... HOWEVER, if you read those previous articles, you will understand that using vegas' scopes can be problematic - i.e. it depends if you are using 8-bit vs. 32-bit mode, and what the input format is. Those factors are what determines if vegas converts Y'CbCr footage => RGB using studio RGB vs. Full range RGB.

[Keiyakusha]

sebazvideo
I don't know what is going on with your Vegas, but I already explained avisynth's behaviour with colorimetry and levels when source is fullrange.

Yes, avisynth assumes TV levels by default when you converting from RGB. Yes, pc.709 will keep full luma range. But if you keep full luma range, you must flag h264 stream accordingly otherwise result will be wrong on playback. Put some screenshots, some files we can check. or something. I'm on 99% sure that x264 is not introducing any shifts, unless you using some broken custom build.
By the way, full luma levels is almost not used anywhere so don't see much reason in doing so.

[poisondeathray]

Even if you flag it correctly in the x264 encoding settings, many players (software and hardware) will ignore the flag.

I agree with keiyakusha, that you should strive to make legal range video, which will look ok on the majority of setups. The way you are going about, I strongly suspect blacks will be crushed and whites will be blown out , and you will be losing detail

[sebazvideo]

This is confusing the hell out of me. If I render from Vegas to the Sony AVC encoder, which has barely any options compared to x264, and it tops at 16 Mbps, the luma levels look exactly the same as the original, and they look the same as any file I encode in x264 using the Matrix pc.709 option. Now, this may be because of the way Vegas handles levels in 8 bit, but when I tried setting the project to 32 bit (with the option video levels) and exporting to Lagarith, it crashed Vegas, and exporting to uncompressed RGB from a 32 bit project takes forever even for the one minute loop I set. So even if I took the time and hard drive space for that, it doesn't matter because it would never be practical for me in a real workflow. To me what's practical is to export using Debugmode Frameserver, which as of now it works whenever it pleases, or to export to a YUY2 or YV12 file to then feed to MeGUI.

However, you guys are telling me that if I set the Matrix pc.709 option the levels are not "range legal", so I don't know what other options do I have.

[Keiyakusha]

This is confusing the hell out of me. If I render from Vegas to the Sony AVC encoder, which has barely any options compared to x264, and it tops at 16 Mbps, the luma levels look exactly the same as the original, and they look the same as any file I encode in x264 using the Matrix pc.709 option.
I can explain only like this:
Vegas keeping full luma range when encoding using Sony AVC and you have playback problems that I mentioned before or you previewing result with some broken tool.
By using Converttoyv12() without matrix and encoding using x264, video looks brighter because of the same problem as above. Also colorimetry probably wrong if your video is HD, but we talking about levels now.

[Warperus]

Check your level ranges in Vegas. In 8-bit mode Vegas processes video at full range (PC Levels, aka Computer RGB in Vegas terms) keeping in mind most video sources are in Studio Levels (Studio RGB).
There is no 709/601 issue inside Vegas as far as you import video in standard formats with proper configuration. There is PC Levels/Studio Levels issue though.
Regular DV and AVCHD input stays in 16-235 ranges for each plane (R, G, B) inside Vegas. All necessary conversions are done by Vegas plugins/codecs.
If you add CG, pictures, generated media etc. you get pc levels event.
Default preview is done in PC levels, so regular DV/AVCHD input looks wrong. External preview monitor works in different modes, for default settings and computer monitor it looks like default preview, so regular DV/AVCHD input looks wrong.

Also, is there a way to do the matrix="pc.709" without the "ConvertToYV12"?
Lagarith (in YUV mode) expects picture in studio levels. If your picture is in PC levels, you can correct it inside Vegas. Use Color Corrector FX with "Computer RGB to Studio RGB" preset. You can use it on media files, events, tracks, or even whole project output.
But before you do so make sure conversion is needed. Put cursor to your media, use Histogram (Ctrl+Alt+2) and check levels. If it's mostly 16-235 then media is already in studio levels. If it's 0-255 then it's PC levels.

[shon3i]

Maybe Sony AVC encoder use fullrange VUI, which every decoder respect AFAIK, that can make video look as 0-255 (PC levels)

[Gavino]

Also, is there a way to do the matrix="pc.709" without the "ConvertToYV12"? Because that way I can export to Lagarith in either YUY2 or YV12
I may have misunderstood your requirements, but levels change without RGB conversion can be done with ColorYUV(levels="TV->PC") or ColorYUV(levels="PC->TV")

[TheFluff]

Why are there three different parameters (--colorprim "bt709" --transfer "bt709" --colormatrix "bt709") instead of only one?
In what situations could they be set differently?
They mean different things.

--colormatrix specifies what coefficients are used when converting YUV to RGB. This is usually the one you want to use.

--colorprim specifies what color primaries the RGB uses. Setting this properly most likely requires knowledge of the studio equipment used to master the stream and/or the settings for said studio equipment. I doubt that any decoders actually read this flag.

--transfer specifies the gamma curve used for the RGB. Again, setting this properly probably requires knowledge of the studio equipment used to master the stream.

[sebazvideo]

I've been doing some more testing, and the whole thing seems even more confusing, and I still don't know which software is to blame. It could be Vegas, or it could be that other software doesn't read the information in the exported file properly. The behavior is so inconsistent between different programs that I don't know what to make of it.

Here's what I'm talking about: I found a small clip that begins with a static shot and about one third of the frame is group of trees, and the rest is white from the snow. I loaded that clip in Vegas Pro 9.0d 64 bit and opened the histogram. The levels showed that they were PC range, although they didn't go down to zero but because there isn't anything really black in that clip, but still they were down to about 6 or so, below 16 for sure. They went all the way to 255. I selected just the first 15 seconds of the clip, because the half tone area with the trees was good enough for me to test for this.

I exported those 15 seconds to RGB uncompressed, to Lagarith in YV12 (to avoid any further conversion for x264), and the Sony AVC codec. I converted the RGB uncompressed and the Lagarith YV12 clips to x264 using MeGUI, both in straight mode and with the "ConvertToYV12(matrix="pc.709")" for the RGB clip and the ColorYUV(levels="TV->PC") for the Lagarith YV12 clip. So I ended up with four clips encoded with x264.

I loaded all those back in Vegas and put them in different tracks one on top of the other, at the same point in the timeline, to compare frames properly. I went 3 seconds into the clip and using the solo switch in each track I compared them to the original, as well as checked the histogram at the same time. The original clip, the uncompressed RGB, the Lagarith YV12, and the two encodes in x264 with the matrix and levels parameters looked identical, with no shift in the levels. Still, the histogram showed a bit of difference between them (except for the original and the uncompressed RGB), but to my eye they all looked the same. However, there was a difference between the those and the two x264 encodes without the added matrix and ColorYUV parameters, in these two looked the dark areas looked brighter than the rest. However, according to the Vegas' histogram, it's not Studio level, because it doesn't cut at 235, it goes up to 244.

So from these tests in Vegas I thought that Avisynth was causing the shift, but I decided to do a few more tests, and that's when it got even more messed up.

I loaded all these clips in Premiere CS4, and here the levels shifted differently. The uncompressed RGB, the Lagarith YV12, and the x264 encodes of these two looked brighter than the original. So here's a difference with Vegas, which had shown the unc. RGB and Lagarith YV12 with the same levels as the originals. The respective x264 encodes of these two also looked brighter in Premiere. As far as I could tell, Premiere doesn't have a histogram like Vegas, so I couldn't check the levels that way, but the shift was still obvious to my eye. The x264 with the added parameters (ColorYUV and matrix pc.709) looked the same as the original.

The test in Premiere suggested that either Vegas was exporting the uncompressed RGB in studio levels but expanding the levels when it displayed the clip in the timeline, or that it exported it using full range but Premiere read it as studio levels.

And then to complicate things even further, I loaded the same clips in Edius Neo Booster. Here the level shifts were different. The original and the uncompressed RGB looked the same, but the Lagarith YV12 looked brighter. The x264 encodes behaved the same as in the other two NLEs.

So basically the unc. RGB looks the same as the original in Vegas and Edius, but brighter in Premiere. The Lagarith YV12 looks the same in Vegas, but brighter in Premiere and Edius.

Now, I don't know what to make of this. If Vegas was narrowing the levels when exporting to unc. RGB and YV12, then how come the histogram between the original and the uncompressed RGB looks exactly the same bit by bit? It doesn't look exactly the same with the Lagarith YV12, but it looks quite similar. The levels go all the way to 255. But then, what does Premiere do, does it shrink the levels when it reads these files? And why Edius reads the full levels in the unc. RGB file but narrows the levels in the Lagarith YV12?

Again, this is a real mess. For now I will use the parameters in the avs file to make sure that the levels look the same in the encode as the original.

[poisondeathray]

If you haven't guessed already, each NLE has different quircks when handling different types of footage formats (ie. they usually handle uncompressed differently than lagarith for example; I already explained how "lossless" formats aren't necessarily treated as lossless in NLE's). I've already explained what vegas does in 8-bit mode. If you haven't already, read those articles, and look at the waveform monitor, and play with 32-bit mode....then you will understand what is going on.

The waveform monitor in vegas has a setup ; it's possible to set it up differently (for studio rgb or computer rgb) so your results may differ depending on how you have yours setup. There is a button right next to the top. Read the manual if you can't find it

Be careful when using vegas' waveform monitor to test the clips you exported., because depending on the format, and the mode (8-bit vs. 32-bit) it either expands to computer RGB or studio RGB (again look at the articles). ie. Your re-imported clips are treated differently. e.g. uncompressed is treated differently than lagarith

If you want to be more consistent, use a 3rd party decoder and histogram for YV12 formats, e.g. in avisynth. This will help to solve a lot of your confusion from the different behaviour of NLE's. This is a good habit for you to get into, because all the levels shifts , color shifts etc... are because of the colorspace conversion and levels compression/expansion in the NLE, NOT avisynth or x264. When you check the input format before your final encode, you can be sure things will turn out the way you want them to.

Premiere does have a waveform the YC waveform. You can toggle it in the program monitor , or secondary monitor for example. Premiere will convert the levels on Y'CbCr formats differently than Vegas. If you use v210 (uncompressed 4:2:2 10-bit) , it will allow pass through of levels, every thing else gets clamped to 16-235 or 0-100 IRE. Besides the levels, it also uses Rec.601, so the color is off as well. (v210 will preserve Rec.709 coefficients). I don't use Edius, so I can't explain what's happening there.

Finally, seeking to achieve something that looks like your original isn't always the best thing to do. For example, in the sample footage I downloaded , it had significant blow outs. You might not want to keep that way for a blu-ray or DVD. or maybe you had little control because you used an "auto" mode during shooting instead of doing things manually

[sebazvideo]

The waveform monitor in vegas has a setup ; it's possible to set it up differently (for studio rgb or computer rgb) so your results may differ depending on how you have yours setup. There is a button right next to the top. Read the manual if you can't find it

I did find it, but even setting it to PC levels it gives me the same levels for the source footage and all the rest.

If you want to be more consistent, use a 3rd party decoder and histogram for YV12 formats, e.g. in avisynth.

Are you talking about one of those plugins for Avisynth? I really don't know how to use Avisynth, not more than the AVS Script creator in MeGUI. But if you point me to where I can download it I can give it a try. I would like to find another software that gives me a histogram to check the levels in these files.

Finally, seeking to achieve something that looks like your original isn't always the best thing to do. For example, in the sample footage I downloaded , it had significant blow outs. You might not want to keep that way for a blu-ray or DVD. or maybe you had little control because you used an "auto" mode during shooting instead of doing things manually

What sample footage are you talking about? But anyway, when I talk about keeping it as the original I mean the footage as I see it in Vegas whether it's in its original form or with color correction and others filters added. If I apply a levels filter in Vegas to adjust the dark areas to what I like but then when I encode it to x264 the dark areas look brighter then it's no good to me. I need consistency all across my workflow.

[poisondeathray]

I did find it, but even setting it to PC levels it gives me the same levels for the source footage and all the rest.


Are we talking about the same thing here? the luminance waveform in vegas? Setting it to Studio RGB (16-235) setup should show a different pattern than Computer RGB (uncheckmarked)

The waveform monitor is a monitoring device, it doesn't change the actual levels of the underlying video by itself. It's used in conjunction with your color correction, levels adjustments etc...But you have to know what the values mean in order to use it. You have to know if it was setup one way vs. the other.

Watch it change as you hit the studio RGB setup. Watch your preview change as you toggle 32bit vs. 8-bit mode. This is the key to understanding what's going on in and what those articles are saying. Read the manual if you need more info.

The problem with looking at the scopes in an NLE, is it's showing what the NLE "sees". This will vary depending on what the file format is, and what settings you are using. The colorspace conversion is already done by the NLE and will different in 8-bit vs. 32-bit mode, for example. This will even differ between different NLE's as you've seen. Very inconsistent.

I'll give you another example, even the exact same video stream can be decoded completely differently in a different wrapper. An h.264 stream in a .mov wrapper looks much different than when in a .mp4 or .ts wrapper in premiere, even though the actual video stream is bit for bit identical. You have to learn how your NLE handles different formats.


Are you talking about one of those plugins for Avisynth? I really don't know how to use Avisynth, not more than the AVS Script creator in MeGUI. But if you point me to where I can download it I can give it a try. I would like to find another software that gives me a histogram to check the levels in these files.


Google for Histogram() or Videoscope() . Histogram takes YV12 an YUY2, but Videoscope takes YUY2() only. I think histogram comes with native install. Videoscope is an external plugin. I use histogram as a quick check because it's the last thing before the actual encode (as I mentioned earlier areas in brown are video "illegal", but small excursions are fine)

If you want a numerical value, you could use coloryuv(analyze=true)

In avisynth, you can stay in the same original YV12 colorspace and see what the histogram shows without all the problems caused by the NLE internal RGB conversion (that's the bottom line, that's what's causing ALL the problems and confusion- the colorspace conversion and how each NLE
handles different types of footage)

YV12 Lagarith is treated lossless in avisynth, because there is no colorspace conversion, and no levels expansion/compression. Even RGB lagarith isn't lossless in Premiere, at it does the levels clamp to 16-235.


What sample footage are you talking about? But anyway, when I talk about keeping it as the original I mean the footage as I see it in Vegas whether it's in its original form or with color correction and others filters added. If I apply a levels filter in Vegas to adjust the dark areas to what I like but then when I encode it to x264 the dark areas look brighter then it's no good to me. I need consistency all
across my workflow.


It was in another thread, there was a statue of a guy pointing...

I realize you're looking for consistency, your're not going to find it easily with NLE's. I don't use vegas much these days, but I think 32-bit mode with linear workflow is the most consistent - you find many vegas users say the same. Also, by having access to the full range, you have complete control over color correcting the "look" of your piece. Some cameras have detail in highlights and those can be accessed brought out when using linear workflow.

Doom9 doesn't seem to have a lot of vegas users, but search other forums, you will see that color grading is a nightmare in vegas because if you're not using a calibrated 2nd monitor, you'll never get quite what you dial in. According to my tests, the preview is using full range, but internally vegas is
using studio rgb (16-235) in 8-bit mode! This is consistent with what the Glenn Chan article says. Regardless, there is a definite disconnect, and you're going to have problems without a broadcast monitor.

For exporting, uncompressed is the best. It passes levels though untouched. Most other options can do some strange things with levels. Maybe you'll find one that works for you.

BTW your ConvertToYV12(whatever) , should be using interlaced=true if you are using interlaced material, or your chroma is going to get scrwed up. This is very important.

Just experiment and find something that works out ok for you. If you're happy with when you use debugmode, then converttoyv12(interlaced=true, matrix="pc.709") , then use it. You can always to some final tweaking in avisynth before your final encode (maybe tone down some illegal overshoots for example) , but at least please be aware of what is "legal" and "illegal" in terms of video levels and the potential problems you're going to get on various setups

Good luck

[sebazvideo]

If you're happy with when you use debugmode, then converttoyv12(interlaced=true, matrix="pc.709") , then use it.
Good luck

To set the interlace to true when not doing a convert to YV12 but instead a ColorYUV(levels="TV->PC"), would the syntax be ColorYUV(interlaced=true,levels="TV->PC")? Doesn't the field order have to be specified like in x264?

[poisondeathray]

To set the interlace to true when not doing a convert to YV12 but instead a ColorYUV(levels="TV->PC"), would the syntax be ColorYUV(interlaced=true,levels="TV->PC")? Doesn't the field order have to be specified like in x264?


The reason for interlaced=true when using converttoyv12 is because the way chroma is handled interlaced video . This only does something when you do RGB<=>YUV conversions

If you're already in YV12 colorspace, you obviously don't have to converttoYV12... , you can use AssumeTFF() for the field order. This maybe necessary for other filters as well (e.g. deinterlacers). ColorYUV doesn't have an interlaced argument

[Warperus]

According to my tests, the preview is using full range, but internally vegas is
using studio rgb (16-235) in 8-bit mode! This is consistent with what the Glenn Chan article says. Regardless, there is a definite disconnect, and you're going to have problems without a broadcast monitor.With all due respect I can't agree with your conclusion concerning 8-bit processing mode. Vegas processes images in PC range (0-255) even though they might be in TV range.
Some generated media is not in PC range naturally, background is R=0/G=0/B=0, PNG is imported in PC range etc.
For the most part YUV input is converted to TV levels because it mostly has TV levels inside, but it depends on codec and source used. For example, xvid produces PC levels and quicktime files can easily produce PC levels as well.
Even more, fade-in and fade-out in Vegas produce PC levels effects, that is you will get illegal values (0-15) as result.
People making TV-compliant movies have to use some weird techniques: still images are processed through "Computer RGB to Studio RGB", default background is substituted by 16-16-16 grey color bar in lowest video track, after all that processing they use Broadcast Colors filter to be sure no illegal values are left...

To set the interlace to true when not doing a convert to YV12 but instead a ColorYUV(levels="TV->PC"), would the syntax be ColorYUV(interlaced=true,levels="TV->PC")? Doesn't the field order have to be specified like in x264?First of all you want YV12+TV levels in the end, since it's the natural mode for x264.
So if your source is in PC levels RGB interlaced you do not want to convert it TV->PC, you want quite the opposite!

Actually Converttoyv12(matrix="pc.709") does all you need - it converts PC range RGB into TV range YUV (YV12, that is 4:2:0) with BT.709 conversion matrix (it's the default color standard for x264). RGB full white (R=255, G=255, B=255) will be converted into YUV white (Y=luma=235).

[Gavino]

The reason for interlaced=true when using converttoyv12 is because the way chroma is handled interlaced video . This only does something when you do RGB<=>YUV conversions
No, it is any conversion to/from YV12, including YV12<->YUY2.
It's because of the vertical chroma subsampling in YV12, and the differences between interlaced and non-interlaced in how this is done.

[Keiyakusha]

Converttoyv12(matrix="pc.709") does all you need - it converts PC range RGB into TV range YUV (YV12, that is 4:2:0)
No. It leaves PC range. Rec709 for TV. That's the problem. If on Vegas side all is done right, original TV source was TV->PC converted during importing in Vegas, edited, saved with PC range RGB, then Rec709 should be used.
EDIT: If exporting to non-rgb colorspace, it should be 100% clear what levels are. If TV, then just encode and all should be fine, if PC, PC-TV conversion should be done. All as simple as that. However saving to YUV may be tricky, as I working with video for some time, and for me saving in RGB is a best practice.
Also it should be impossible to get RGB with TV levels as output from Vegas (unless done on purpose by applying levels filter to all footage), otherwise it just broken.

[poisondeathray]

With all due respect I can't agree with your conclusion concerning 8-bit processing mode. Vegas processes images in PC range (0-255) even though they might be in TV range.
Some generated media is not in PC range naturally, background is R=0/G=0/B=0, PNG is imported in PC range etc.
For the most part YUV input is converted to TV levels because it mostly has TV levels inside, but it depends on codec and source used. For example, xvid produces PC levels and quicktime files can easily produce PC levels as well.
Even more, fade-in and fade-out in Vegas produce PC levels effects, that is you will get illegal values (0-15) as result.
People making TV-compliant movies have to use some weird techniques: still images are processed through "Computer RGB to Studio RGB", default background is substituted by 16-16-16 grey color bar in lowest video track, after all that processing they use Broadcast Colors filter to be sure no illegal values are left...

First of all you want YV12+TV levels in the end, since it's the natural mode for x264.
So if your source is in PC levels RGB interlaced you do not want to convert it TV->PC, you want quite the opposite!


I never mentioned still images. My comments were specfic to what he was using, i.e AVCHD video. Native AVCHD is treated as studio RGB (16-235) in 8-bit mode internally, but the preview display is computer RGB ,hence the big confusion.

GFX and still images are treated as computer RGB (0-255). This is exactly what those articles say, and my tests have shown as well.

The export format makes a difference as well. e.g. exporting in debugmode will result in different levels then uncompressed or h.264 using mainconcept or sony.

[sebazvideo]

First of all you want YV12+TV levels in the end, since it's the natural mode for x264.
So if your source is in PC levels RGB interlaced you do not want to convert it TV->PC, you want quite the opposite!


No, the scenario I was referring to is when I export in Lagarith YV12 and feed that into MeGUI through Avisynth. In my tests, if I don't add ColorYUV(levels="TV->PC") to the avs script, I end up with narrower levels, and thus it looks different than the original.

[Warperus]

No. It leaves PC range.Thx for correction.

If on Vegas side all is done right, original TV source was TV->PC converted during importing in VegasUnfortunately, it's not that simple :( For most camcoders (including Sony AVCHD camcoders) Vegas keeps TV Levels in 8-bit and 32-bit Studio Levels modes. That is, importing codec does no TV->PC conversion.
Quicktime sources mostly produce PC levels.

Also it should be impossible to get RGB with TV levels as output from Vegas (unless done on purpose by applying levels filter to all footage), otherwise it just broken.Studio RGB inside Vegas is just like that. If you render such project to uncompressed RGB in .avi file you'll get RGB with TV Levels.

[Keiyakusha]

Well, then it looks like sebazvideo will never find consistency with Vegas :Р

[Warperus]

In my tests, if I don't add ColorYUV(levels="TV->PC") to the avs script, I end up with narrower levels, and thus it looks different than the original.Looks different where? The same mp4 file opened in QuickTime player or KMPlayer may look different. If you check video in MeGUI preview it might look even more different.
Regular MPEG video is like this: YV12 colorspace & TV levels. When such video is played decoder produces YV12 output that is then processed by YUV->RGB converter that also applies TV->PC color conversion. It is default behaviour for players like Media Player Classic when they deal with x264 output (at least for encodes without fullrange flag).

Keiyakusha
I'm afraid he wants full range x264 output. And I'm not sure if it's a reasonable wish...

My comments were specfic to what he was using, i.e AVCHD video. Native AVCHD is treated as studio RGB (16-235) in 8-bit mode internally, but the preview display is computer RGB ,hence the big confusion.It's not that simple as well :devil:
Small preview is done in computer RGB. Full screen preview on OCHI compliant IEEE 1394/DV monitor is done in studio levels.
Full screen preview in widnows secondary display can have color correction applied. If you select icm profile it will look different. If you check "Use studio RGB (16-235)" it will... well, you get the idea.