The MPEG2 encodes of my VHS-C captures need more than 10000 of max bitrate ! ! ! ! !

[crespo80]

I capture VHS-C tapes with my lifeview prime30 (Philips SAA7130) at the resolution of 704x576 (cropped from 720x576) with fly2000tv 2.36 software and HuffyUV codec (YUY2).
I want to preserve the maximum possible quality so I mantain the interlaced video and compress to MPEG2 with the same capture size and no filtering (the source is quite good). I add 16 pix of black borders to left/right/bottom side to hide some tape noise and increase encoder compression. I tried CCE 2.67.00.27, Procoder 2.0 and TMPGenc Plus 2.521.
CCE and procoder alter brightness and/or saturation of the video, so my choice is TMPGenc which preserves the source intact.

For all the encoders i set a Costant Quality factor (30 for CCE, 80 for TMPGenc and 97 for procoder), leave all the filters off and set the min/max bitrate to 1000/9656 (audio is Mp2 mono 48KHz 192Kbps).
All the other settings are common (Top Field First, 15frames GOP, no preprocessing, default matrix, ecc.)

With these settings, all encoders output same sized videos.

The problem is:
In quite static scenes the medium bitrate is around 5000, with peaks of 6000/7000 and it is mantained a constant quantization of 5 (analyzed with bitrate viewer).
In high motion scenes (generally due to the the camera that moves too fast) the medium bitrate grows up to 8000/8200 with peaks of 9800/10000!!! The medium quantization also gros up to 7 and, in those peaks, can reach 10/12 and the video gets blocky.
With CCE and Procoder the video never becomes blocky in those scenes, but i prefer TMPGenc because it mantains the correct luminance/saturation level (I posted this in CCE section).

If I set the maximum bitrate to 15000, then the quantization of 5/6 is mantained, but the medium bitrate is around 9000 and the peaks are of 12000/13000 and it's not DVD compliant.
This means that the encoder, to mantain the CQ setting, need very high bitrate.

Is it normal such a high bitrate to achieve best quality video?
To analyze the video I open it in VirtualDubMOD and BitrateViewer.

[Kika]

CCE and procoder alter brightness and/or saturation of the video, so my choice is TMPGenc which preserves the source intact.

Sound's like you have trouble with the Lumarange. If that's the case, that's the reason why TMPGEnc-Encodings become more blocky in High-Motion Scenes - the Encoder has a lot of more Details to encode.

If you are using CQ-Mode, i suggest you to set B-Picture Spoilage to 0.

In quite static scenes the medium bitrate is around 5000, with peaks of 6000/7000 and it is mantained a constant quantization of 5 (analyzed with bitrate viewer).

Wow, that's much too much... Guess your Video is noisy and a bit shacking (also shaking Lines, that's an VCR-Effect).
Without a TBC or a Noise-Filter, there's nothing you can do.
Do a research on the Capture-Guide here in the Board, you will find some usefull hints.

[crespo80]

get a look to this thread, and tell me is my source is noisy or not (the images are just copied from VirtualDubMode and jpg compressed with a factor of 15).
http://forum.doom9.org/showthread.php?s=&threadid=82834

I tried playing with luminance level in CCE but with no results. It's a problem of CCE itself.
With Procoder i didn't find a setting to correct that, and the result is worse even than CCE, because it alters also the saturation and smooth the video.

I also tried some filters with avisynth 2 (PixieDust and GoldDust) and some of avisynth 2.5 (PeachSmooth, Convolution3D, TemporalSoften) but the size of the video decreases only a little and the video is too much soften).

Note, however, that the blockiness is only noticeable in VirtualDubMOD with 2x zoom in still image and in very few frames of those points, but is annoyng because the required bitrate is so high.

The blocky frames are the B frames, of course.
But I think B-picture Spoilage set to 0 cannot solve the problem, because the problem is the lack of bitrate...

Or not? What is the error?

[crespo80]

i just tried mainconcept mpegencoder 1.4.2 (last version) but the constant quantization setting is ridicolus.
Just for curiosity, it also alters the brightness, just as CCE (but less than procoder).
It mantais the constant quantization with no care for the maximum bitrate (that meand that BitrateViewer shows a flat curve for Q facor). So I had a video with an average bitrate of 8450 and peaks of 15000 !!!

I ask you to try my settings to see if it's common.


But, thinking a moment, isn't the DV standard derived from MPEG2? Even if it doesn't uses B frames, its bitrate is det to 25000 to mantain maximum quality.
So I think 8000 of average bitrate isn't too high if I want the maximum possible quality.

Or not?



HELP ME!!!!!!!!!!!!!!!!!!!!!!!

[rfmmars]

If VHS-C is your source,you can use a fairly low bit rate 4500 - 6000. Your only dealing with 230 lines (Max) of resolution anyway.

I capture 8000 bps VBR I frames only with a ATI 9000 card MEPG2 format, and I never run into any prolems like you are discribing.

However with camcoder footage nothing is standard, and problems vary within brands and between brands, and formats.....VHS,VHS-C, 8MM, and DV.

Some help may be found at www.100fps.com

richard
www.photorecall.net

[crespo80]

At which resolution do you capture and encode?
Do you leave interlaced frames or deinterlace?
What program do you use to encode and which settings?
Please make this test:
open you encoded file in VirtualDubMOD and go to a fast motion scene.
Then go to a B frame and see if there is any blockyness (at that so low bitrate you should see a loto of blocky images).
Copy the source frame to clipboard and paste it to Paint Shop Pro.
Save to Tiff format and post the image on the forum so i can compare with my encodes.
:)




P.S.
I'm making a test with a VHS-C capture, encoding with these encoders:
TMPGenc Plus 2.521.58.169
CCE-SP 2.67.00.27
MainConcept 1.04.02.00
Procoder 2.01.30.0

I use a 2-pass VBR encode (0/6000/9656) and the results are similar. All the encoders produce blockness at this low medium bitrate of 6000, i'll post the test results this evening.

When you analyze the encoded video, you only see it in a player or also compare frame-to-frame with VirtualDubMOM?

[rfmmars]

Capture at 720 x 480 29.97fps interlaced.

Since I capture I FRAMES ONLY, there are no B FRAMES. This is most likey why I don't have these problems.

Capture program is ATI's bundled MPEG2 software, many settings.

richard

[fccHandler]

Originally posted by crespo80
CCE and procoder alter brightness and/or saturation of the video
Did you say you use VirtualDubMod to judge the result? If so, you may be seeing the effect of MPEG-2 "matrix coefficients." There was a discussion about it here (http://forum.doom9.org/showthread.php?s=&threadid=81191).

[crespo80]

Originally posted by fccHandler
Did you say you use VirtualDubMod to judge the result? If so, you may be seeing the effect of MPEG-2 "matrix coefficients." There was a discussion about it here (http://forum.doom9.org/showthread.php?s=&threadid=81191).

thanks for the link, but I think that the problems isn't with the "matrix coefficients".
Otherwise, how do you explain that TMPGenc outputs a file identical to the source?

[Wilbert]

thanks for the link, but I think that the problems isn't with the "matrix coefficients".
Otherwise, how do you explain that TMPGenc outputs a file identical to the source?
Maybe you are right here. But, perhaps you can check it.
Get the latest GSspot which gives the matrix coefficients.

btw, you opened your capped mpeg2 directly in the encoder right (so, no avs)?

The point is here the TMPGEnc adds FCC (almost the same as MPEG-1) coefficients to the header, while CCE adds MPEG-2 coefficients.

I think, but you should test it by opening the mpeg2's in the latest GSpot:

1) open your capped mpeg2 in GSpot and post the coefficients.

2) open your TMPGEnc-mpeg2 in GSpot and post the coefficients.

3) open your CCE-mpeg2 in GSpot and post the coefficients.

One of the two is "wrong" (either CCE or TMPGEnc). Which one depends on how the source is created. But we don't know, because it's stuff from tv. But in this case it appears it is made (RGB->YUV) by using MPEG-1 coefficients because TMPGEnc has it right.

Of course you can't see that by looking at the header of your cap, because your capture device doesn't know that.

It is possible to modify ColorMatrix() to do MPEG-1 coefficients -> MPEG-2 coefficients (instead of the other way around), but I don't have time before next week to look at that.

Now, I don't know how (and if) CCE does the RGB->YUY2 conversion if you feed it with RGB. You could check that by making

mpeg2source("***.d2v")
converttorgb32()

and feed it in CCE, and post the same screenshot as in the other thread.

[fccHandler]

According to the samples Wilbert sent me, TMPGEnc was using the value 4, which is so close to the MPEG-1 values that you probably can't see the difference. It's possible that CCE doesn't define "matrix coefficients" at all, in which case VirtualDubMod selects the default for MPEG-2, which is quite different (brighter?) than MPEG-1.

There is a way to test if "matrix coefficients" is causing you grief: Convert the MPEG-2 to AVI format using "fast recompress" mode and Xvid (or DivX) codec. This will avoid the "matrix coefficients" entirely, because there is no conversion to RGB involved.


EDIT: Same time post as Wilbert! ;)

[crespo80]

Originally posted by fccHandler

There is a way to test if "matrix coefficients" is causing you grief: Convert the MPEG-2 to AVI format using "fast recompress" mode and Xvid (or DivX) codec. This will avoid the "matrix coefficients" entirely, because there is no conversion to RGB involved.


EDIT: Same time post as Wilbert! ;)

I just made this test compressing to XviD an MPEG2 video from CCE (fast processing in virtualdubMOD).
The luminance of the result video was diffent from that of the original CCE file and near to the source, but not exactly the same as the source and TMPGenc.
But, in this case, took place two compressions, so it's possible that a part of the information has been lost.
Though I used a very high quality XviD:
I choosed the quality based encode, with a quantization factor of 3 (or 2, I do not remember, I'll do another test) and the result video had an average bitrate of 10000!

[fccHandler]

Originally posted by crespo80
The luminance of the result video was diffent from that of the CCE file and near to the source, but not the same as TMPGenc.
To be 100% sure, do the same recompression in "full processing mode." If the result appears brighter than your previous Xvid, then "matrix coefficients" is surely the source of your problem.

I don't know a fix, because the proper setting of this value is the encoder's responsibility. However, I really think VirtualDubMod may be one of few decoders which actually uses this value. I doubt very much that any DirectShow applications do.

[crespo80]

Originally posted by fccHandler
To be 100% sure, do the same recompression in "full processing mode." If the result appears brighter than your previous Xvid, then "matrix coefficients" is surely the source of your problem.

But, in this case, the "matrix coefficients" are not applyed because there's no MPEG1/2 source....
Or I'm missing something?
:confused:

[fccHandler]

What I mean is, do the same recompression from your MPEG-2 source to Xvid, but this time use "full processing" mode, and compare that to your previous result (which used "fast recompress" mode).

The matrix coefficients are applied in VirtualDubMod only if the source is MPEG-2 and it is being converted to RGB during processing.

[crespo80]

I made other test, opening in VirtualDubMOD the MPEG2 file encoded by CCE and recompressing in many formats:


COMPRESSION MADE IN FULL PROCESSING MODE

- Conversion to uncompressed RGB24 AVI:
the video is perfectly identical to the MPEG2 created by CCE, so the luminance error is still present.

- Conversion to HuffyUV (same settings of the capture):
the video is perfectly identical to the MPEG2 created by CCE, so the luminance error is still present

- Conversion to XviD with best quality options (target quantization=2, max I frame interval=25, motion search precision=6, VHQ=3):
the video luminance is almost identical to the MPEG2 created by CCE, so the luminance error is still present.


COMPRESSION MADE IN FAST PROCESSING MODE:

- Conversion to uncompressed RGB24 AVI:
the uncompressed video is perfectly identical to the MPEG2 created by CCE, so the luminance error is still present.

- Conversion to HuffyUV (same settings of the capture):
the video luminance is different from the MPEG2 created by CCE, and now its luminance is almost identical to the source video!!!

- Conversion to XviD with best quality options (target quantization=2, max I frame interval=25, motion search precision=6, VHQ=3):
the video luminance is different from the MPEG2 created by CCE, the luminance is almost identical to the source (even if a bit influenced by the XviD compression).




Take your conclusions!
I had the perception that isn't a brightness error, but a gamma error!

So it seems that there's not an error of CCE, but an error of VirtualDubMOD with its preview mode and FULL PROCESSING MODE!

If you read the encoders test I made, you'll see that also procoder and mainconcept show a different luminance level in VirtualDubMOD, so we can adfirm that the problem is VirtualDubMOD!

[Wilbert]

If you read the encoders test I made, you'll see that also procoder and mainconcept show a different luminance level in VirtualDubMOD, so we can adfirm that the problem is VirtualDubMOD!
No, wrong.

Please can you open all mpeg2 streams (source, CCE and TMPGEnc) and post those coefficients, then we can explain you what's going on?

The whole problem is that in VDubMod (or any other player) the mpeg2 is converted (from YUV) to RGB when being displayed. In order to do this, you need to know *how* this should be done (meaning which set of coefficients should be used).

Usually this info is present in the header of the mpeg2 file. This info ia added by the encoder (like CCE and TMPGEnc) when encoding the mpeg2 file. CCE adds mpeg2 coefficients and TMPGEnc adds fcc coefficients (which are appr. equal to the mpeg1 coefficients). Which one is correct depends entirely on the source, ie how the mpeg2 (your source) is created in the first place.

Your capture device has nothing to do with this, because it gets the yuv data from your tv. So, you have to know how the broadcasts are created.

Let's look at the specs of pal/ntsc where this conversion is prescribed. In table 2 of ITU-R_BT.470-6, you will see for example

Y = 0.299 * R + 0.587 * G + 0.114 * B

which are precisely the mpeg-1 coefficients.

conclusion 1) mpeg-1 coefficients should be used when displaying.

conclusion 2 ?) your capture software should add those coefficients to the header of the mpeg-2 stream. Apperently it does that. To check this you should open your capture in (latest) GSpot. Please do that, and post it.

conclusion 3) If conclusion 2 is true, then at least CCE does something wrong. If you open the mpeg-2 file in CCE it should keep the coefficients as given in the header of the mpeg-2 file.

Of course if you used AviSynth, the header is lost, and *no* encoder can know what the correct coefficients are.

conclusion 4) Apperently TMPGEnc also doesn't keep the coefficients, since it replaces them with fcc coefficients. In this particular case, it doesn't matter because they are approximately equal.

But if you feed TMPGEnc a dvd-mpeg2 file, it will be TMPGEnc which adds the wrong coefficients, and CCE adds the correct ones.

I hope it's a bit clear.

But, I really hope you can open all those mpeg2s in GSpot and post those coefficients.

[crespo80]

- I dowloaded Gspot but I can't see any info about the matrix coefficients of the files.
All the informations about the matrix coefficients are greyed (both for the HuffyUV source and CCE MPEG2 encoded file).

- Since my card captures in YUY2 format and the capture software directly compresses in HuffyUV YUY2 format, no color space conversion takes place during capture, so no matrix coefficients are written in the avi header. So, the only coefficients involved are those written by the encoding softwares like CCE or TMPGenc.

- Since we are talking about matrix coefficients and color space conversions, I think that the problem involves the gamma rather than the brightness.
You said maybe the matrix coefficients were lost in avisynth.
But tell me why, if I open the CCE MPEG2 file in VirtualDubMOD and recompress it in FAST RECOMPRESS MODE to HuffyUV or XviD, than the new encoded files look identical to the source in relation to the gamma (not the same could be said about the quality)?
And why, in FULL RECOMPRESS MODE, the gamma of the encoded file is still incorrect?
I think the problem is with VirtualDubMOD, since there shouldn't be any difference between FAST and FULL RECOMPRESS MODE.
Or I'm wrong again?

[Wilbert]

1) This coefficient box in GSpot is only used for mpeg2 files.

2) I thought your cap was mpeg-2. Sorry about that :)

But tell me why, if I open the CCE MPEG2 file in VirtualDubMOD and recompress it in FAST RECOMPRESS MODE to HuffyUV or XviD, than the new encoded files look identical to the source in relation to the gamma (not the same could be said about the quality)?
That's because the source (ie broadcast) is created using mpeg-1 coefficients. The huffyuv and XviD (also DivX) decoders assume mpeg-1 coefficients when decoding.

And why, in FULL RECOMPRESS MODE, the gamma of the encoded file is still incorrect?
fcchandler can answer this probably. The YUY2 -> RGB conversion is done by VirtualDubMod itself (thus not by huffyuv). Apperently it is doing the conversion by assuming mpeg-2 coefficients. But fcchandler should comment on this.

I think the problem is with VirtualDubMOD, since there shouldn't be any difference between FAST and FULL RECOMPRESS MODE.
Or I'm wrong again?
Summarizing. In the first case the used end codec is decoding the stream (xvid or huffyuv) using mpeg-1 coefficients, and the second case VDubMod itself is doing the color conversion using mpeg-2 coefficients. Which one is correct, depends on how the source is created.

In this case the first one is correct, because the broadcast is created using mpeg-1 coefficients. Just coincedence.

[Kika]

@crespo80
Just open your Video with AVISynth, force it to RGB24 and encode the AVS-File.

avisource("video.avi", true, rgb24)

Try both Lumarange-Settings (0-255, 16-235). One of them MUST give you correct colors.
And don't judge the colors on the PC, use a DVD-Player. Some PCs do have problems with MPEG-Video an Luma/Chroma.

[fccHandler]

@crespo80: I'm just a little angry that I have to explain this again. Did you read the whole thread I linked to? The explanation is already there, along with links to the MPEG-2 video specification so you can see for yourself.

First of all, understand that anything you watch on your monitor or television is being converted to RGB (at some level) for the phosphors on the picture tube. This matrix conversion (which we've had since the origin of color TV) uses coefficients 0.299 0.587 0.114. Not surprisingly, these are the default coefficients for MPEG-1 video.

But an MPEG-2 stream may carry its own coefficients within itself; this is the "matrix coefficients" field of "sequence display extension", and it's optional. If the field is present, VirtualDubMod honors the request and uses whatever matrix the stream has specified. If the field is NOT present, VirtualDubMod uses the default matrix for MPEG-2 which is NOT the same as the MPEG-1 matrix.

TMPGEnc seems to specify the FCC's coefficients, which are nearly the same as MPEG-1. CCE doesn't seem to specify any coefficients, and that's probably why the field is grayed out in Gspot. However, without explicit coefficients VirtualDubMod assumes the default for MPEG-2, thus there is still a brightness difference.

"Fast recompress" mode doesn't convert to RGB (unless you explicitly select RGB, as you did in one of your experiments). As I said before, if the decoder isn't converting to RGB, then "matrix coefficients" doesn't apply. "Full processing" mode always converts to RGB.

Now, what's happening with Xvid in "fast recompress" mode is that it's receiving YUV data from VirtualDubMod, and storing it in MPEG-4 format. But in this case, the MPEG-2 "matrix coefficients" don't get used; furthermore, they don't get saved with the video data and they are lost forever. On playback, the Xvid video is converted back to RGB for your CRT, using the de facto MPEG-1 coefficients which may not be correct for this source. In effect, the whole process is b0rked.

In "full processing" mode, VirtualDubMod converts the MPEG-2 video to RGB correctly using the stream's specified coefficients, then passes that along to Xvid. Xvid converts to YUV using MPEG-1 coefficients, on playback it's converted back to RGB using MPEG-1 coefficients, so nothing is lost in that process.

The real source of the problem is that CCE isn't using the correct coefficients for your particular video. From what you've said about its origin, clearly this video shouldn't have MPEG-2 coefficients applied to it. But the fault doesn't lie with VirtualDubMod! I'll repeat what I said before, it's the encoder's responsibility to specify this field correctly.

Unfortunately there doesn't seem to be a way to tell CCE (or any encoder that I've seen) which coefficients it should use.


P.S. Sorry for shouting. This has caused such controversy that I almost wish I hadn't implemented "matrix coefficients" support in the first place... :rolleyes:


EDIT: Accidentally misspelled crespo's nick; sorry. :)

[Wilbert]

Thanks for the lengthy explanation fcchandler! I was in a rush yesterday, and didn't have time to give a detailed explanation.

@Kika,

The issue is not the 'preserving' of the luma range here. Dig up the thread called 'MatrixColor' in the avs forum to see what this coefficient stuff is about.

[Kika]

@Wilbert

Yeah, i know this stuff (a little), but i think, that's not the problem here. Have you seen the Screenshots of crespos Videos? I bet, the problem here IS preserving the Lumarange. The differences are imho much too big to be explained only by incorrect settings of the sequence display extensions.

[crespo80]

Thanks for your explanation, now I understand more things; I also spent the morning reading some guides on color space conversions, both in the doom9 site and in the doom9 forum (the links you posted), and also with google.

VirtualDubMOD (like all the applications that show a video/image on a PC RGB monitor) always converts the video to RGB24 to preview it (both for a YUV or RGB source). That because PC monitors use the RGB color space. All right.
If I open my YUY2 native source (the captured video) VirtualDubMOD applies the standard coefficients (0.299 0.587 0.114) to make the YUY2->RGB conversion both for preview and full recompress mode. So there are no problems. The video is identical to the source.
Or not? Maybe the error is here?
I have tried to play the source video in windows media player and it looks the same as in VirtualDubMOD, so media player uses the same YUY2->RGB coefficients of VdubMOD. That's right?

If I encode this YUY2 native source, CCE shouldn't do any color space conversion (because the subsampling YUY2->YV12 doesn't implicates any matrix coefficients, right?), so it shouldn't write any matrix coefficients in the avi header (and infact Gspot doesn't find them).

If I open this encoded file in VirtualDubMOD, Vdub doesn't find any matrix coefficients in the avi header (because CCE hadn't write them) so it applies the standard coefficients for the YV12->RGB conversion needed for preview. But these coefficients aren't the same of the YUY2 ->RGB conversion (0.299 0.587 0.114), otherwise the video should be identical to the source. Instead VdubMOD now shows a video with a different luma/chroma, so maybe something is wrong with the YV12->RGB conversion.
The same thing happens if I full recompress this video to a YV12 or RGB format, or if I fast recompress the video to uncompressed RGB AVI format. All right, the coefficients used are the same of the preview mode, so the error is the same. Or not?.
Now the thing I cannot understand at all:
If I fast recompress this video to a YUV format like HuffyUV or XviD the resulting video, opened again in VirtualDubMOD, should look identical to the CCE encoded file.
Instead it looks identical to the source!


I can't understand this last passage (and probably I didn't fully understand the others).

[fccHandler]

Originally posted by crespo80
If I open this encoded file in VirtualDubMOD, Vdub doesn't find any matrix coefficients in the avi header (because CCE hadn't write them) so it applies the standard coefficients for the YV12->RGB conversion needed for preview. But these coefficients aren't the same of the YUY2 ->RGB conversion (0.299 0.587 0.114), otherwise the video should be identical to the source. Instead VdubMOD now shows a video with a different luma/chroma, so maybe something is wrong with the YV12->RGB conversion.
No, there's nothing wrong with the code. If the source is an MPEG-2 with no sequence display extension, VirtualDubMod uses the default coefficients for MPEG-2, not the standard MPEG-1 coefficients. The default for MPEG-2 is 0.2125, 0.7154, 0.0721 (c.f. section 6.3.6 of ISO/IEC 13818-2).


The same thing happens if I full recompress this video to a YV12 or RGB format, or if I fast recompress the video to uncompressed RGB AVI format. All right, the coefficients used are the same.
Once again, it uses whatever coefficients CCE specified for the stream, or the default for MPEG-2 (0.2125 0.7154 0.0721) if none are specified.

But, if I fast recompress this video to a YUV format like HuffyUV or XviD then the resulting video, opened again in VirtualDubMOD, should look identical to the CCE encoded file.
No, because that process is b0rked. The MPEG-2 coefficients are not being transferred to the Huffy/Xvid file. From now on, the display of the Huffy/Xvid file will use the MPEG-1 coefficients, and won't look the same as the CCE video (which uses MPEG-2 coefficients).

Instead it looks identical to the source!
That proves that CCE should have encoded the source with MPEG-1 coefficients in the first place. (There is actually a setting for that; matrix coefficients = 5.) What you are doing is forcing MPEG-1 coefficients for conversion (by skipping VirtualDubMod's application of MPEG-2 coefficients), so it all looks good.

[crespo80]

Clarify me this point:
the matrix coefficients could be present in the header of all the types of YUV video streams (mpeg1/2, HuffyUV, XviD, ecc.)
And they cannot be present in RGB video streams because they are used only to do the YUV->RGB conversion.
It's all right?

If it's ok, I understood the most part of the discussion and can ask you the last things.
Otherwise, there is something I have to understand yet.

Thanks for your great help!
I appreciate it a lot
:)

[Wilbert]

That proves that CCE should have encoded the source with MPEG-1 coefficients in the first place. (There is actually a setting for that; matrix coefficients = 5.)
Really? Can you set this in CCE? (I don't have it myself ...)

the matrix coefficients could be present in the header of all the types of YUV video streams (mpeg1/2, HuffyUV, XviD, ecc.)
No, *sometimes* it's only present in a mpeg-2 header. (I don't know about mpeg-1 headers though.)

Huffyuv and XviD are open source. We know that the decoders assume mpeg-1 coefficients when decoding.

@Kika,

Yeah, i know this stuff (a little), but i think, that's not the problem here. Have you seen the Screenshots of crespos Videos? I bet, the problem here IS preserving the Lumarange. The differences are imho much too big to be explained only by incorrect settings of the sequence display extensions.
Yes, I have seen the screenshots. I could be wrong, but the CCE screenshot is more orange than the TMPGEnc screenshot. That can't be caused by preserving luma change?

If it would be such a luma change, then I don't understand this

- Conversion to HuffyUV (same settings of the capture):
the video luminance is different from the MPEG2 created by CCE, and now its luminance is almost identical to the source video!!!

- Conversion to XviD with best quality options (target quantization=2, max I frame interval=25, motion search precision=6, VHQ=3):
the video luminance is different from the MPEG2 created by CCE, the luminance is almost identical to the source (even if a bit influenced by the XviD compression).
because CCE should also keep the luma range.

I know that there's a lumarange setting in CCE, but it only does something if you feed it with RGB?

@crespo80,

Could you upload 10 frames huffyuv cap somewhere? Then we can take a look at it.

[crespo80]

here are 10 seconds of my captured video (opened in VirtualDubMOD, cutted and saved in direct stream copy)
Hope can help


right click in Internet Explorer and save object (left click doesn't work, FireFox may not work)
http://www.crespo80.altervista.org/doom9.avi 3,7MB

[Kika]

@Wilbert

I could be wrong, but the CCE screenshot is more orange than the TMPGEnc screenshot. That can't be caused by preserving luma change?

Hm... it may be an effect caused by encoding with super black and super white (illegale Range). But i'm not sure about this.


HuffYUV / Xvid
I'm not familiar with Xvid, but isn't Xvid compressing the range to 16-235? As far as i know, HuffYUV did'nt do that. Pleas correct me, if i'm wrong.

I know that there's a lumarange setting in CCE, but it only does something if you feed it with RGB?

That's correct, yes. This setting works only on RGB-Source. But on YUY2-Source with a wrong Lumarange, CCE does it always wrong. That's not a fault of CCE, is a fault of some Codecs (do you remember? I Had this Problem with PicVideo MJPEG)

But again: The effect on crespo's Screenshots is imho too strong to be only caused by incorrect matrix coefficients.

To bring up an other point: I have noticed such an effect on an other PC. A MPEG2-Video encodet by me which is looking quite perfect on a TV was displayed much to dark on te PC while the Source was shown correct. I installed new Drivers for the Graphics Card and did a reinstall of DirectX and all Codecs. After that, the Source and the encodet Video came out perfectly.

[Wilbert]

I'm not familiar with Xvid, but isn't Xvid compressing the range to 16-235?
No, it just clamps it to 16-235 (0-16 -> 16 etc). I'm quite sure it doesn't scale.

But on YUY2-Source with a wrong Lumarange, CCE does it always wrong.
Meaning that when feeding a YUV [0,255] the lumarange is untouched? Just want to be sure ...

@crespo80,

Could you make the following script

AviSource("your_huffyuv.avi")
ColorYUV(levels="PC->TV")
ConvertToRGB()

and open this in VDub/VDubMod and post the same screen shot as the ones in this thread:

http://forum.doom9.org/showthread.php?s=&threadid=82834

[fccHandler]

Originally posted by crespo80
Clarify me this point:
the matrix coefficients could be present in the header of all the types of YUV video streams (mpeg1/2, HuffyUV, XviD, ecc.)
Sure they could be, but they aren't. The "matrix coefficients" field is something peculiar to MPEG-2 video. Every other YUV-based format (that I'm aware of) just assumes the traditional 0.299 0.587 0.114 matrix.

And they cannot be present in RGB video streams because they are used only to do the YUV->RGB conversion.
Correct. ;)

[fccHandler]

Originally posted by Wilbert
That proves that CCE should have encoded the source with MPEG-1 coefficients in the first place. (There is actually a setting for that; matrix coefficients = 5.)
Really? Can you set this in CCE? (I don't have it myself ...)
Sorry, bad wording on my part. I meant to say that a "matrix coefficients" value of 5 would index the same standard coefficients which MPEG-1 uses. But unfortunately I don't see a way to set this in CCE, or any other MPEG-2 encoder I've tried.

[Kika]

Meaning that when feeding a YUV [0,255] the lumarange is untouched? Just want to be sure ...

Yes. Fortunatly, most Codecs are providing a correct Lumarange in YUY2-Mode.

[Wilbert]

@crespo80,

Could you make the following script

AviSource("your_huffyuv.avi")
ColorYUV(levels="PC->TV")
ConvertToRGB()

and open this in VDub/VDubMod and post the same screen shot as the ones in this thread:

http://forum.doom9.org/showthread.php?s=&threadid=82834


Still waiting :)

[crespo80]

escuse me, i no longer read the thread because after the last post of FccHandler I understood everything. The thing is very very simple in the end:
1) I capture in HuffyUV YUY2 and no matrix coefficients are written into the avi header.
So VDMOD uses standard matrix coefficients to convert it to RGB.
2) CCE encodes the file to mpeg2 and does not specify any matrix coefficients in the avi header (so media players don't know which coefficients they have to use in the YV12->RGB conversion).
3) VDMOD opens the captured huffyuv file (like any other video file except for MPEG2) using standard coefficients, while opens the encoded MPEG2 file using the standard MPEG2 coefficients (because CCE hadn't specified the correct MPEG1 coefficients to use). So the output video displays wrong, but it is internally right!
TMPGenc encoded files look identical to the source because it specifies mpeg1 coefficients in the file header, so VDMOD uses those coefficients to do the YV12->RGB conversion and not the incorrect MPEG1 coefficients.
4) So, source and CCE encoded video files look different when displayed in VDMOD, but the two files are innerly perfectly identical, the only thing that changes is the way they are outputted to RGB by VDMOD.

I made the ultimate test to confirm that:
I encoded in CCE the captured huffyuv with best quality settings (constant quality=10 etc.) then I opened this encoded video in VDMOD (where it appears different from the source) and fast recompressed it again in huffyuv.
The resultant file was PERFECTLY identical to the source video, there wasn't ANY difference at all!
So, for me, the story is closed, I'll abandone TMPGenc and go for CCE!!!


I noticed another thing.
I tryed to capture in RGB mode (instead of YUY2) and the resulting video was different from YUY2 capture. The colors seem to be greyed, the final effect is worse than YUY2 capture.
I also made the test wilbert asked me, converting the captured YUY2 huffyuv video into RGB with his avisynth script (it's his last post above) and the resulting video was identical to that captured in RGB mode, so worse than the YUY2 one.
Here the samples as displayed in VDMOD (outputted to PSP and jpg compressed=5):

CAPTURED HUFFYUV VIDEO (YUY2)
http://www.crespo80.altervista.org/cap_huffyuv.jpg

CAPTURED VIDEO ENCODED BY CCE (YV12)
http://www.crespo80.altervista.org/cce_encode.jpg

CAPTURED VIDEO CONVERTED TO RGB WITH AVISYNTH (the script by wilbert)
http://www.crespo80.altervista.org/cap_huffyuv_to_rgb.jpg

[crespo80]

Why does the video look different if captured in RGB mode or in YUY2 mode? Shouldn't these format be nearly equivalents (being the color downsampling of YUY2 practically invisible to the eye)?

The same identical difference is visible when I convert the captured YUY2 video to RGB with AviSynth.

Im' confused again

[Wilbert]

Why does the video look different if captured in RGB mode or in YUY2 mode? Shouldn't these format be nearly equivalents (being the color downsampling of YUY2 practically invisible to the eye)?
Probably because your capture is YUY2 [0,255]. You can check this in AviSynth using histogram and looking up some very bright/dark scenes (see analogue capture guide).

This means you have to scale it to YUY2 [16,235] in some way. This can't be done if you feed it directly to CCE. You can use AviSynth for this (as described above using ColorYUV(levels="PC->TV")). For TMPGenc there's a setting for this which converts [0,255] YUY2 to [0,255] RGB (and back to [16,235] YV12).

I guess that VDubMod clips (= rounds) a [0,255] YUY2 stream to [16,235] YUY2 before converting it to [0,255] RGB (for display). fccHandler can you confirm this?

If the latter is true it means that VDubMod shows you a video which is too dark (in your case).

[fccHandler]

Originally posted by Wilbert
I guess that VDubMod clips (= rounds) a [0,255] YUY2 stream to [16,235] YUY2 before converting it to [0,255] RGB (for display). fccHandler can you confirm this?
I'm really only familiar with the MPEG decoding in VirtualDub(Mod)(-MPEG2). But the way I understand the traditional CCIR-601 conversion, a Y below 16 does get clamped to black. But it happens after the conversion to RGB, not before. (Because the RGB values end up being negative.)

[fccHandler]

Originally posted by crespo80
Why does the video look different if captured in RGB mode or in YUY2 mode?
I have a BT878 using BTWincap drivers. I've noticed that if I set its capture colorspace to RGB24, the video looks very different from its YUY2 mode. I can see the difference in VirtualDub's preview even before capturing. Don't know why...

[Wilbert]

I'm really only familiar with the MPEG decoding in VirtualDub(Mod)(-MPEG2). But the way I understand the traditional CCIR-601 conversion, a Y below 16 does get clamped to black. But it happens after the conversion to RGB, not before. (Because the RGB values end up being negative.)
Ok. (I guess it doesn't matter whether you do it before or after.)


I have a BT878 using BTWincap drivers. I've noticed that if I set its capture colorspace to RGB24, the video looks very different from its YUY2 mode. I can see the difference in VirtualDub's preview even before capturing. Don't know why...
I thought I explained it in my previous post. In YUY2 mode it caps the full luma range [0,255]. So, you get:

1) cap RGB [0,255]

2) cap YUY2 [0,255] -> RGB [-18,278] -> clamp to RGB [0,255]

and the second one is not correct (because Y=U=V=0 should be mapped to R=G=B=0 for correct display).

[fccHandler]

Great; now I'm confused... :p

Remember I'm not capturing anything to disk, I'm just looking at VirtualDub's capture preview (not overlay) while switching the BT's capture format from YUY2 to RGB24. The RGB version looks much darker. I think I understand what you're saying, but does this mean the card itself is b0rked?

[Arachnotron]

I thought I explained it in my previous post. In YUY2 mode it caps the full luma range [0,255]. So, you get:

1) cap RGB [0,255]

2) cap YUY2 [0,255] -> RGB [-18,278] -> clamp to RGB [0,255]

and the second one is not correct (because Y=U=V=0 should be mapped to R=G=B=0 for correct display).
The BT878 always caps yuy2, and converts afterwards to RGB internally if you reguest RGB. According to the BT878 specsheet:

RGB Conversion:
R = 1.164(Y–16) + 1.596(Cr–128)
G = 1.164(Y–16) – 0.813(Cr–128) – 0.391(Cb–128)
B = 1.164(Y–16) + 2.018(Cb–128)
Y range = [16,235]
Cr/Cb range = [16,240]
RGB range = [0,255]

In which case 1 and 2 should look the same? :confused:

[Wilbert]

The BT878 always caps yuy2, and converts afterwards to RGB internally if you reguest RGB.
Yes, I know. That's the case with every chip.


According to the BT878 specsheet:

RGB Conversion:
R = 1.164(Y–16) + 1.596(Cr–128)
G = 1.164(Y–16) – 0.813(Cr–128) – 0.391(Cb–128)
B = 1.164(Y–16) + 2.018(Cb–128)
Y range = [16,235]
Cr/Cb range = [16,240]
RGB range = [0,255]
The OP is using a Philips SAA7130 chip. But I suspect the same holds for this philips chip (page 30-31 saa7130HL datasheet).

I don't understand it either, that's why I asked crespo80 to look at it with Histrogram (in avs).

I think that the Y range is [0,255], while it should be [16,235].

As you know I still have the saa7108. I'm quite sure that the luma range is [0,255] when capping YUY2 with that driver (although according to the specs it should be [16,235]).

Do you think VDub upscales the lumarange somehow during capping?

[Arachnotron]

As you know I still have the saa7108. I'm quite sure that the luma range is [0,255] when capping YUY2 with that driver (although according to the specs it should be [16,235]). I think it is (16-235 I mean) in the sence that pure black is at 16. But since we are dealing with analog sources, you may get out of range values which are not clamped.

At least with my 7134 black generally is around 16 but you do get values between 0 and 16 in captures.

[fccHandler]

Originally posted by Arachnotron
According to the BT878 specsheet
Where can I get this specsheet?

[Arachnotron]

There are some links in chapter 5.3.1 of the capture guide.

see

http://www.conexant.com/products/entry.jsp?id=404

[fccHandler]

Wow, thank you! Looks like I'm going to be reading for days... ;)

[Wilbert]

I think it is (16-235 I mean) in the sence that pure black is at 16. But since we are dealing with analog sources, you may get out of range values which are not clamped.
I disagree.

1) If you look at dark spots, a significant part is below Y=16. If you wish I can upload a few huffyuv frames.

2) If I don't scale it to [16,235], the video is simply too dark. Details in certain (parts of) scenes are invisible because of that.

I will upload a few huffyuv frames tomorrow, then you can judge for yourself :)

[Arachnotron]

1) If you look at dark spots, a significant part is below Y=16. If you wish I can upload a few huffyuv frames.

Er.. that was what I was saying too...

But I wanted to check anyway.

I took a test pluge from the digital video essentials DVD which contains superblack as a reference. It contains superblack bars (could not find exactly the value) on a pure black background with also 2% and 4% grey bars, and a pluge from 20% grey to 100%white.
I used a s-video lead to connect the DVD player to my SAA7134 card.

First thing I found out my old pioneer DVD players clamps, so the superblack was gone from the signal. :(
Second thing I found out is that the soft option on my Philips DVD is just low brightness. After getting the player running ok:

Capping yuy2/huffyuv puts black on y=16, white on 235. The superblack gives a nice line below 16 in the histogram.

Capping the same pic as raw rgb24:
superblack 5,5,5
black 14,14,14
4% 24,24,24
2% 20,20,20
white 234,234,234

Converting the previous yuy2/huffy cap with ConvertToRGB():
superblack 0,0,0
black 0,0,0
4% 8,8,8
2% 3,3,3
white 253,253,253

Now that I have a good reference signal, maybe I'll try out a few other cards to see how they handle this.

[Wilbert]

So, in other words, the superblack and superwhite values are already present in the broadcast itself.

1) Why are there some many superblack/white parts present in the broadcast itself (where parts doesn't mean a few pixels per frame, but a significant part of a frame)?

2) Does that mean that the broadcast is displayed incorrectly on the tv? Because like I said "(...) the video is simply too dark. Details in certain (parts of) scenes are invisible because of that.".