Pixelated Video in VMR 7/9

[cmw]

Ok, I'm at a loss here!

I have tried to get DXVA to work and well - it does, however I've run into some rather annoying issue with that. I'm using Media Player Classic HC, and until now, I used the Haali Renderer, but now i have to use VMR7 or 9 to get DXVA to work (well ok, Overlay works too, but mhh ;) ).

Now the problem is that I somehow get very pixelated Video when using VMR7 or 9.

Screen: http://molkemon.mo.funpic.de/vmr9.bmp
(especially visible in the left half of the picture)

While with Overlay, Haali and EVR (I have .net 3.5 installed so I can use it on XP), the Video is fine:

Screen: http://molkemon.mo.funpic.de/haali.bmp

I've tried every possible combination of settings in VMR9 renderless (and of course always restarted the MPC afterwards)... Yuv Mixing, every possible resizer, 2d surfaces etc, nothing of that had any effect at all.

I also tried using other players, and that is where it gets REALLY confusing!

Windows Media Player 11 plays the file without pixels. I've set it to use Quality Mode, so it *should* use VMR9 and not the default VMR7.

mplayer2 (Windows Media Player 6.4) plays the file pixelated (it should be using VMR7 allthough there is no way to tell for sure).

graphstudio doesn't allow for fullscreen, but the pixelation with vmr7 and 9 is visible there aswell, and again, no pixelation with haali.

ANY suggestions? oO

[Dark Shikari]

Its because of bad YV12->RGB conversion, it seems.

Why do you need to use VMR9? Its well-known as an oft-broken renderer.

[cmw]

I want to use DXVA on XP, and I have the same issue with VMR7, and I really don't want to use Overlay since I can't make Image stills then and it's having issues with my multimonitor setup. Can I somehow fix the bad conversion?

[Leak]

Its because of bad YV12->RGB conversion, it seems.
Shouldn't it be possible to do this the right way using a shader in MPC-HC when using VMR9 renderless with custom renderer, just like bicubic resizing?

[cmw]

No. It didn't matter which resizing filter I'd use (PS 2.0 or not), that didn't work.

Eventually, the solution was to use ffdshow and go to the output configuration there and disable ALL yuv output, so it's forced to convert to rgb with the cpu. (check high quality yv12 to rgb converison though). this instantly fixed the pixelation. for avc i choose decoding with internal mpc decoder rather than ffdshow, so dxva is used, which is nice aswell :)

[Leak]

No. It didn't matter which resizing filter I'd use (PS 2.0 or not), that didn't work.
I didn't mean your problem, I meant this in general - MPC-HC could do the YV12->RGB conversion itself in the renderer with proper interpolation of chroma values instead of just point resizing.

Also, I've found that ffdshow's YV12->RGB conversion, even with high quality turned one, is worse than using AviSynth for the job... at least for big, bright red text from a DVD I still get pixelated diagonals when using ffdshow's conversion vs. smooth outlines when doing the conversion via ConvertToYV12... :(

If MPC-HC could do it properly on the graphics card it would be perfect, though.

np: Amon Tobin - Displaced (Chaos Theory)

[cmw]

Can you give me avisynth script i have to set in ffdshow to test? Also, how demanding is that method? Atm I get ~5% cpu load for sd content without dxva and I don't want to get it any higher.

Also, compared to the original pixelation, the picture is extermely superior, making it even better would be sweet, but it's not as important now anymore^^

Of course I agree that it would be better when mpc-hc could do it properly.

[Leak]

Can you give me avisynth script i have to set in ffdshow to test? Also, how demanding is that method?
Dunno, I haven't really tested that aspect - resizing, adjusting gamma and converting to RGB along with adding a bit of noise in ffdshow to DVDs takes about 13% on my Core 2 Quad.

For color conversion just use
ConvertToRGB32()
at the end of your AviSynth script.

Atm I get ~5% cpu load for sd content without dxva and I don't want to get it any higher.
That has to be a typo - if the quality can be improved via filtering why would you only want to use your CPU to 1/20 of it's capacity? :confused:

[cmw]

Becuase I need my CPU for other tasks that run in the background ;)
Video Quality is important to me, but there is a limit how much CPU time I will dedicate to it ;)

[pitch.fr]

I've found that ffdshow's YV12->RGB conversion, even with high quality turned one, is worse than using AviSynth for the job... at least for big, bright red text from a DVD I still get pixelated diagonals when using ffdshow's conversion vs. smooth outlines when doing the conversion via ConvertToYV12... :(
do you have a sample please ?

do you mean it's better to do ConvertToRGB32() than RGB32HQ in ffdshow ? :eek:

RGB32HQ looks fine to me :

http://pix.nofrag.com/8/0/6/aea08863ddeef671ad19e79af9b85.png

here's a compare between ffdshow and AVS :

http://img261.imageshack.us/img261/6462/plllqa9.png

they look identical :cool:
which would make sense, considering RGB32HQ is using ConvertToRGB32() from what I understood :confused:

even the CRC32 sum of the 2 lossless screenshots is identical :)

sample is available here :
http://rapidshare.com/files/122925763/Bronz_s.mkv.html

[Leak]

do you have a sample please ?
Gah, I think I broke my brain today playing DROD RPG. (And it was late to boot...)

I'll post a sample tomorrow.

EDIT: But the more I think about it - don't the MPC shaders work on an RGB image? A quick look at the grayscale shader suggests they do...

If that's the case it shouldn't be hard to cook up a shader that takes the (pixellated) RGB image, converts it back to YUV (at 16 bits/channel), keeps the luma and properly interpolates the chroma for all pixels that don't have even x and y coordinates then converts the results back to RGB...

That should fix both EVR and VMR without the CPU performance impact of converting to RGB in ffdshow...

(Someone please stop me if I'm totally wrong here... :))

np: Bomb The Bass - Hold Me Up (feat. Paul Conboy) (Future Chaos)

[pitch.fr]

cool thanks, looking forward to it :)

[Leak]

Okay, so I didn't even take the time to find out if ffdshow RGB conversion is wonky on my end, but here's the shader I've come up with to combat bad chroma upsampling via point resize:


sampler s0 : register(s0);
float4 p0 : register(c0);
float4 p1 : register(c1);

#define width (p0[0])
#define height (p0[1])

float4 getPixel(float2 tex, float dx, float dy)
{
tex.x+=dx;
tex.y+=dy;

return tex2D(s0, tex);
}

float4 rgb2yuv(float4 rgb)
{
float4 yuv=0;

yuv.r= 0.299*rgb.r+0.587*rgb.g+0.114*rgb.b;
yuv.g=-0.147*rgb.r-0.289*rgb.g+0.436*rgb.b;
yuv.b= 0.615*rgb.r-0.515*rgb.g-0.100*rgb.b;

return yuv;
}

float4 yuv2rgb(float4 yuv)
{
float4 rgb=0;

rgb.r= 1.000*yuv.r+0.000*yuv.g+1.140*yuv.b;
rgb.g= 1.000*yuv.r-0.395*yuv.g-0.581*yuv.b;
rgb.b= 1.000*yuv.r+2.032*yuv.g+0.000*yuv.b;

return rgb;
}

float4 main(float2 tex : TEXCOORD0) : COLOR
{
float dx=1/width;
float dy=1/height;

float4 yuv00=rgb2yuv(getPixel(tex, 0, 0));
float4 yuv01=rgb2yuv(getPixel(tex, 0,dy));
float4 yuv10=rgb2yuv(getPixel(tex,dx, 0));
float4 yuv11=rgb2yuv(getPixel(tex,dx,dy));

float4 yuv=yuv00;

yuv.g=(yuv00.g+yuv01.g+yuv10.g+yuv11.g)/4.0;
yuv.b=(yuv00.b+yuv01.b+yuv10.b+yuv11.b)/4.0;

return yuv2rgb(yuv);
}

Open the shader editor in MPC, enter a name in the dropdown list (I used "Deblock YV12"), hit return, select and delete the sample code MPC inserts and paste the above, then turn on your new shader.

What it does is actually quite dumb - it takes each pixel from the (blocky) RGB image the shader gets fed as well as the ones to the right, bottom right and bottom, converts them back to YUV, averages the U and V values of those 4 pixels then converts the result back to RGB and returns it, effectively turning the ugly point resize that creates the blocky edges into a bilinear resize that is much easier on the eyes...

Oh, and it should work with DXVA decoding... :D

(Okay, you could optimize that a bit since pixels with even coordinates don't change anyway and those only having one odd coordinate could be done by averaging only two pixels, but I'm happy with it as it is now...)

np: Soul 223 - In Search Of Slowly (Soul Jazz Records Singles 2006-2007 (Disc 2))

[MatMaul]

:eek::eek::eek:
awesome !
I just made a test and it is nearly as good as rgb32hq or nv12.
it blurs a little more in the red part compared to them but it is anyway a LOT better than without it.

[Leak]

:eek::eek::eek:
awesome !
I just made a test and it is nearly as good as rgb32hq or nv12.
it blurs a little more in the red part compared to them but it is anyway a LOT better than without it.
Yep, I just gave it a thorough test drive as well... I wonder if Casimir666 might want to add it to MPC-HC?

One thing I'm still not sure about is how the chroma is downsampled in the first place, as I might have to shift the source chroma by half a pixel or so in order to get the calculation fully correct...

Well, that's something to try for tomorrow... :) *yawn*

np: Kode 9 - Magnetic City (Soul Jazz Records Singles 2006-2007 (Disc 3))

[Japhsoncross]

I noticed this problem as well, and I sent a ticket to ATI driver team, but don't get any reply.
And if you use the latest driver for HD2XXX graphic card, even overlay has the same problem at the original picture size(but it will disappear if users change the size).
I also wrote a shader to make it better, basically it blurs the chroma channel, but detects if it's BT601 or BT709.
But it look a little bit too blurred, then I wrote a sharpen shader, and sharpens the chroma channel a little bit. MPC allows multi shaders combined together. And now it looks good to me.

PS: Pixel Shader 3.0 is required, too many instructions :(

[Leak]

I also wrote a shader to make it better, basically it blurs the chroma channel, but detects if it's BT601 or BT709.
Ummm... exactly how is one supposed to tell BT601 or BT709 apart? Not even ffdshow's YUV->RGB conversion manages to do that automatically... (Can't look at your attachments - yet...)

EDIT: Wait, let me guess - image size? I guess I can easily add that... :)

But still - blurring the chroma is overkill if the problem is caused by the chroma values being simply doubled instead of interpolated; simply undoing the doubling then doing a proper resize (which basically is what I'm doing above, even though it could probably use an even better resizer than bilinear) should yield the same result as doing the chroma upsampling properly (for varying values of "properly", I know - but point resize definitely isn't the best choice for playback) in the first place.

I'll take a look at your attachments once they're approved... :)

[Japhsoncross]

:) it detects the image height>= 720, then BT709, otherwise BT601.
There are a lot of limitations of such decision. but AFAIK, the colorimetry information can't be detected by the shader, or even from some file stream.
ATI driver assume image height>=720 is BT709 for the VMR9 Y'CbCr->R'G'B' conversion, so I should do the reversion according to it:)

[pitch.fr]

I've run some compares on my side :)

ConvertToRGB32(rec709) and RGB32HQ(16-235 BT709) don't use the same levels ?! one outputs a clearer picture than the other(ffdshow looks better).

and using Ulevels() in 0-255, they look identical, but when you compress them to .PNG, the ConvertToRGB32() pictures are always 20% bigger ?!

still hoping that you can make that sample please.....but indeed, you might be right :)

would be most interesting to compare your PS script/RGB32HQ/ConvertToRGB32 :D

[Leak]

would be most interesting to compare your PS script/RGB32HQ/ConvertToRGB32 :D
Ummm... what's preventing you from doing that right now? :confused:

[pitch.fr]

well from my tests with the 720p red credits, converttorgb32/rgb32hq look identical.

even the CRC32 sum is identical.

you said you have some DVD that looks better with AVS than with ffdshow, and you would be willing to make a sample ?

[Leak]

well from my tests with the 720p red credits, converttorgb32/rgb32hq look identical.
I thought you wanted to compare those two with my pixel shader? That's what I meant when I was wondering what was keeping you from doing it...

[pitch.fr]

well I do :D

any chance for that sample please ? :p

[Leak]

well I do :D

any chance for that sample please ? :p
Anything with big red letters on a black background should do - in my case it was the big fat "WARNING" message at the beginning of the UK DVDs for "Terminator: Sarah Connor Chronicles" season one - which I've lent to a friend so I won't get them back before Friday...

As for testing the shader while I threw it together I just fired up GIMP, created an 800x600 image of big honking red letters (and a few small ones) on a black background, imported that image into AviSynth and turned it into a 10 second 25fps video, encoded it to XviD via VirtualDub and played the result in MPC-HC - I'll post that video when I get home.

But yeah, doing a "ConvertToRGB32()" via the AviSynth filter in ffdshow-tryouts beta 5 looked better while doing the RGB32HQ conversion still looked blocky - I'll re-test that when I get home.

[pitch.fr]

OK, this is LSF + ConvertToRGB32(matrix="rec709") :

http://pix.nofrag.com/d/4/6/0991a49475a6df22a8590fa10fa16t.jpg (http://pix.nofrag.com/d/4/6/0991a49475a6df22a8590fa10fa16.html)

now LSF + ffdshow RGB32HQ BT.709 "standard" :

http://pix.nofrag.com/3/1/9/ac5d0ac22f871728b0cc55702b354t.jpg (http://pix.nofrag.com/3/1/9/ac5d0ac22f871728b0cc55702b354.html)

right off the bat, you can see that they don't use the same levels mapping ?
god knows which does it the "proper" way ?! ffdshow does look better :devil:

this is LSF + Ulevels(tv2pc) + ConvertToRGB32(matrix="PC.709") :

http://pix.nofrag.com/c/a/3/9e6036cb45bef2a19cf2baffa0041t.jpg (http://pix.nofrag.com/c/a/3/9e6036cb45bef2a19cf2baffa0041.html)

now LSF + Ulevels(tv2pc) + ffdshow RGB32(BT.709 full range) :

http://pix.nofrag.com/a/d/7/3b34018880dbf4f814fde1c1e7afbt.jpg (http://pix.nofrag.com/a/d/7/3b34018880dbf4f814fde1c1e7afb.html)

they look identical this time, but each time ConvertToRGB32 yields 10% bigger PNG files...

[Leak]

OK, this is LSF + ConvertToRGB32(matrix="rec709") :
now LSF + ffdshow RGB32HQ BT.709 "standard" :

right off the bat, you can see that they don't use the same levels mapping ?
god knows which does it the "proper" way ?! ffdshow does look better :devil:
No offense, but other than a slight difference in levels they look exactly identical to me - and I couldn't care less about that, to be honest...

Also, I thought this thread was about pixelated/blocky chroma upsampling? :confused:

[pitch.fr]

No offense, but other than a slight difference in levels they look exactly identical to me - and I couldn't care less about that, to be honest...

Also, I thought this thread was about pixelated/blocky chroma upsampling? :confused:
well yeah, there's a level difference.....which one does it right ? ffd or AVS ?

well I ran a compare on the red stuff :

ConvertToRGB32 rec709 :

http://pix.nofrag.com/f/4/0/9213fe22e85c58818e1ab0c8365f8t.jpg (http://pix.nofrag.com/f/4/0/9213fe22e85c58818e1ab0c8365f8.html)

ffdshow RGB32HQ BT709 standard :

http://pix.nofrag.com/f/4/0/9213fe22e85c58818e1ab0c8365f8t.jpg (http://pix.nofrag.com/f/4/0/9213fe22e85c58818e1ab0c8365f8.html)

CRC is identical.

you said you had some sample where it wasn't, so I was surprised ?!

then I made these screenshots, where they look identical in real life examples(0-255 only)......yet AVS yields 10% bigger .PNG files :confused:

still hoping for your sample(s) that show a visible improvement of AVS over ffdshow at this point :D

[CruNcher]

When you do these comparisons with Windows you have to take care of all the colorspace and level conversions that can occur in it's chain of vfw/directshow Rendering those are Decoder->Colorpsace input Renderer (without it nothing works and different colorspace renderer can have their own pros/cons ffdshow,xvid,divx,helix,microsoft (default),mainconcept and many more)-> the main output Renderer (which behavior can be influenced by the driver).
if they don't work hand in hand you can get different results for different inputs so first of all it's important to show of your complete chain of how you took these screens and from where :)

[pitch.fr]

When you do these comparisons with Windows you have to take care of all the colorspace conversions that can occur in it's chain of vfw/directshow Rendering those are Colorpsace input Renderer (without it nothing works and different colorspace renderer have their own pros/cons ffdshow,xvid,divx,helix,microsoft,mainconcept and many more)->Decoder-> the main output Renderer (which behavior can be influenced by the driver).
if they don't work hand in hand you can get different results for different inputs so first of all it's important to show of your complete chain of how you took these screens and from where :)

no sh*t ? :D

well it's BT.709 YV12 h264 in all cases.

for the conversions, I've given all the details already.

I didn't use DDCC gamut conversions in these screenshots ;)

[Leak]

still hoping for your sample(s) that show a visible improvement of AVS over ffdshow at this point :D
Gah... I think I know what's going on here...

Here's the FBI warning from Planetes Vol. 5, converted with ffdshow's HQ RGB conversion:

http://leak.no-ip.org/Stuff/ffdshow%20RGB32HQ%20Bad.png

Using ConvertToRGB32() in AviSynth looks totally the same, I agree... but!

http://leak.no-ip.org/Stuff/AviSynth%20ConvertToRGB32%20Good.png
(note the red parts on the emblem and the "WARNING"...)

I think we can agree that this looks better - and it's the output of ConvertToRGB32(interlaced=false)...

It seems as if ffdshow somehow always does interlaced upsampling with DVDs, even though progressive would look better with most PAL stuff and of course with stuff that's been IVTCed via AviSynth - maybe there should be an option for interlaced/progressive chroma conversion? :(

EDIT: Yeah, it seems as if ffdshow only considers DVDs (or at least MPEG2 video) to be interlaced, as this AVI I talked about in my last post (http://leak.no-ip.org/Stuff/lipsum.avi) gets upsampled progressively and looks pretty bad when using ConvertToRGB32(interlaced=true) instead...

Here's the result using my shader...

http://leak.no-ip.org/Stuff/Shader.png

I guess I should shift the chroma down and right by a pixel or so... (EDIT: Okay, make that half a pixel...)

np: Bomb The Bass - So Special (feat. Paul Conboy) (Future Chaos)

[pitch.fr]

how would ffdshow know that the source is DVD ?

is there some cases when interlaced chroma upsampling works better than progressive ?

I've tried your AVI sample and it looks better progressively than interlaced, as intended.

indeed an option to force progressive upsampling in ffdshow would be a good idea...I thought that was already the case, actually :eek:

still doesn't explain why Convert yields 10% bigger .PNG's than ffdshow :D

[Leak]

Okay, so my shader was off by 0.5 pixels - here's a corrected version:

/*
YV12 chroma upsampling fixer
by Kurt Bernhard 'Leak' Pruenner

Use with YV12 output if the half-resolution chroma
gets upsampled in hardware by doubling the values
instead of interpolating between them.

(i.e. if you're getting blocky red edges on dark
backgrounds...)
*/

sampler s0 : register(s0);
float4 p0 : register(c0);
float4 p1 : register(c1);

#define width (p0[0])
#define height (p0[1])

float4 getPixel(float2 tex, float dx, float dy)
{
tex.x+=dx;
tex.y+=dy;

return tex2D(s0, tex);
}

float4 rgb2yuv(float4 rgb)
{
float4x4 coeffs=
{
0.299, 0.587, 0.114, 0.000,
-0.147,-0.289, 0.436, 0.000,
0.615,-0.515,-0.100, 0.000,
0.000, 0.000, 0.000, 0.000
};

return mul(coeffs,rgb);
}

float4 yuv2rgb(float4 yuv)
{
float4x4 coeffs=
{
1.000, 0.000, 1.140, 0.000,
1.000,-0.395,-0.581, 0.000,
1.000, 2.032, 0.000, 0.000,
0.000, 0.000, 0.000, 0.000
};

return mul(coeffs,yuv);
}

float4 main(float2 tex : TEXCOORD0) : COLOR
{
float dx=1/width;
float dy=1/height;

float4 yuv00=rgb2yuv(getPixel(tex,-dx,-dy));
float4 yuv01=rgb2yuv(getPixel(tex,-dx, 0));
float4 yuv02=rgb2yuv(getPixel(tex,-dx, dy));
float4 yuv10=rgb2yuv(getPixel(tex, 0,-dy));
float4 yuv11=rgb2yuv(getPixel(tex, 0, 0));
float4 yuv12=rgb2yuv(getPixel(tex, 0, dy));
float4 yuv20=rgb2yuv(getPixel(tex, dx,-dy));
float4 yuv21=rgb2yuv(getPixel(tex, dx, 0));
float4 yuv22=rgb2yuv(getPixel(tex, dx, dy));

float4 yuv=
(yuv00*1+yuv01*2+yuv02*1+
yuv10*2+yuv11*4+yuv12*2+
yuv20*1+yuv21*2+yuv22*1)/16;

yuv.r=yuv11.r;

return yuv2rgb(yuv);
}

(I also had the genius idea of using matrix calculations, which helps to keep the instruction count down and is a forte of shader units anyway... :D)

np: Bomb The Bass - Burn The Bunker (Toob's Whatgoesoninhemsbystaysinhemsby Mix) (Future Chaos Remixes)

[Leak]

how would ffdshow know that the source is DVD ?
Because DVDs are usually MPEG2 streams which can flag interlaced encoding?

is there some cases when interlaced chroma upsampling works better than progressive ?
Well, definitely on already deinterlaced material - and I don't think ffdshow's deinterlacers (or the AviSynth filter) discards any indication that the video is interlaced... :(

still doesn't explain why Convert yields 10% bigger .PNG's than ffdshow :D
Don't get so hung up on PNG size - maybe Convert just rounds more accurate than ffdshow; you'd also get smaller PNGs if you reduced the bit depth from 8 to 7 bits per channel, but quality would surely suffer...

np: Bomb The Bass - So Special (Specialized By Michael Fakesch) (Future Chaos Remixes)

[pitch.fr]

Because DVDs are usually MPEG2 streams which can flag interlaced encoding?

Well, definitely on already deinterlaced material - and I don't think ffdshow's deinterlacers (or the AviSynth filter) discards any indication that the video is interlaced... :(

Don't get so hung up on PNG size - maybe Convert just rounds more accurate than ffdshow; you'd also get smaller PNGs if you reduced the bit depth from 8 to 7 bits per channel, but quality would surely suffer...

yeah I also think that this is a possible indication that Convert rounds more accurately than ffdshow ?!

yeah, OK if you deinterlace 29.97 NTSC DVD in ffdshow with TomsMoComp....then the stream still has the "interlaced" flag ?

but you said interlaced chroma upsampling works better then ?

I use Convert() in all cases, so my TomsMoComp'ed DVD's suffer from progressive chroma upsampling then ? they would look better in interlaced ?

I think I misunderstood what you said..and that for progressive native/deinterlaced output, progressive chroma should be used ?!

[Leak]

but you said interlaced chroma upsampling works better then ?
No, why would it?

The deinterlacer should have already converted the interlaced chroma to progressive chroma while doing it's work (which is creating progressive frames) - after all, you're taking the odd lines (and then the even lines in case of bobbing) and calculating new values for where the other field was, so you'll also have to make the chroma progressive.

And that's when interlaced chroma upsampling gets ugly, as the vertical position of each line of chroma differs between interlaced and progressive video...

Just try both ways with ConvertToRGB32(interlaced=true/false) on progressive and interlaced material and you'll see the quality depends on using the correct upsampling method...

Interlaced conversion is for converting interlaced YUV images to interlaced RGB images; but if the interlacing is already gone you need to convert progressive YUV images to progressive RGB images...

np: Bomb The Bass - Star (Future Chaos Remixes)

[CruNcher]

@ptch.fr
So the chain for this was, Source H264 YV12 BT709 input->ffmpegsource/dgavcdecode/ConverttoRGB32 respectively ffdshow hq rgb32->VMR??? 7 renderless or 9 renderless ? and you took them via MPC as BMP and converted them to PNG ?

[Leak]

@ptch.fr
So the chain for this was, Source H264 YV12 BT709 input->ffmpegsource/dgavcindex/ffdshow->ConverttoRGB32->VMR??? 7 renderless or 9 renderless ? and you took them via MPC as BMP and converted them to PNG ?
I think we're talking about using "ConvertToRGB32()" inside of ffdshow directly when playing back the video in a media player here...

BTW - have you tried my shader with YV12 directly instead of converting to RGB?

np: Tocotronic - Hier Ist Der Beweis (Tocotronic)

[CruNcher]

Leak nope not yet normally i use Nvidias Hardware Decoding it does that Chroma upsampling internally for all supported formats for both cases interlaced/progressive correct, you can also see that in my very old decoder compare i made (with Nvidias 1st Generation Hardware Decoder) if you find it ;)

[pitch.fr]

I think we're talking about using "ConvertToRGB32()" inside of ffdshow directly when playing back the video in a media player here...
indeed, and I simply made printscreens of Haali's Renderer in RGB32.

OK, so you're saying that ffdshow does interlaced chroma upsampling even if deinterlacing is being processed with one of its filters ?

so 2 options : make its deinterlace filter kill the interlaced flag, or put an option to force progressive upsampling ?

I'm sure clsid should be able to manage one of these two ?

[MatMaul]

ok now YV12 with your shader looks exactly the same as NV12.
that's great, thanks for that and I vote for incorporate it in mpc-hc.

[Leak]

ok now YV12 with your shader looks exactly the same as NV12.
that's great, thanks for that and I vote for incorporate it in mpc-hc.
Already done according to Casimir666... :) (I had PMed him about it...)

np: Tocotronic - Dringlichkeit Besteht Immer (Tocotronic)

[Kado]

@Leak
Nice shader man!
Shader off:
http://pwp.netcabo.pt/kado/off.png
Shader on:
http://pwp.netcabo.pt/kado/on.png
RGB32HQ:
http://pwp.netcabo.pt/kado/rgb32hq.png
Pics are from "Mahoromatic" Season 2 episode 9 @ min 12:04.

[pitch.fr]

no wonder I've always hated DXVA :eek:

how about a LSF PS script ? :D

[DigitalDeviant]

Is there a downside (video quality-wise) to this using this shader?

Oh, and how 'bout a debanding shader :p

[Kado]

@pitch.fr
I use DXVA whenever possible, anyway what's the source for the screenshots on this post (http://forum.doom9.org/showthread.php?p=1184867#post1184867)?

If I use MPC HC DXVA to decode h264 video there's no need for the shader from some tests I've been doing, it seems the GPU outputs in RGB32HQ, because there's no blocking.
Blocking occurs with YV12and software mode.

[Japhsoncross]

@Leak
your latest shader does almost the same as my my blur shader do, the only difference is mine checks the image height and use another matrix for 709.

@Kado
i checked your picture, the red flower in "shader on" is less saturated then the one in "RGB32HQ".

i think that's why i want to use different matrix for different height of the image.
VMR9 doesn't do any conversion if the input color space is RGB32. but if the input source is YV12, it does the YV12->RGB conversion, and this procedure is controlled by the driver. And for the ATI card (the reason i mention ATI card is that nVidia cards (GF8) doesn't seem to have such issue, they upsample the chroma channels well.), the driver checks the image height and applies different matrix for the picture, which means if we have an RGB source which is in SD size, and we convert it by Rec.709, then we playback the file with an ATI card, the picture is always incorrect. And if it's BT601 for SD and Rec.709 for HD sources, which may work well with an ATI card, the VMR9 first convert it to RGB32, and then, here comes the shader, if we want to extract the exact chroma channels and do the blurring, we have to follow the conversion what's been done by the VMR9. Otherwise, we may get the non-pixelated picture, but the result could still be wrong, because the colors are changed.

[Leak]

Is there a downside (video quality-wise) to this using this shader?
I wouldn't recommend using it on non-YV12 video (since YUY2 would need a shader that only does horizontal interpolation, and RGB doesn't need any at all), but for YV12 it shouldn't have any downsides.

As for the BT.601/.709 difference: even if you're using the wrong colorspace it should only have minimal impact since I'm only averaging values after conversion followed by converting back to RGB using the same (wrong) conversion afterwards.

But I'll add a height check to use BT.709 for HD material when I get home... (and then I'll locate whoever was responsible for that whole 601/709 brouhaha in the first place and will taunt him fiercely... :mad:)

[pitch.fr]

@Kado : Appleseed 2 BD .m2ts

I think there's a 601/709 goof up in your screenshots.

@Leak : well they went BT.709 for HD because the possible gamut is much bigger.
actually reds are orangey in BT.601/SMPTE-C gamut......due to all major companies(JVC/Sony/etc..) refusing to give their gamut coordinates.
so the SMPTE went with SMPTE-C/BT.601, and fixed their mistake later on with the HDTV standard(a bit too late, I'll give you that)

if you look at Disney's CARS, the hero will NEVER be as red in the movie as he is on the cover(BD use BT.709.....but SMPTE-C primaries) :D

[Japhsoncross]

@pitch.fr
are you really sure BD uses SMPTE-C primaries? that must be confusing.

from what i know, BT601 luma coefficients is still (0.299, 0.587, 0.114), which ware actually calculated from the obsolete 1953 NTSC primaries. but then the primaries were changed.(NTSC uses SMPTE RP145, PAL uses EBU 3213, they both nearly match Rec.709 primaries. And now we capture SD by the new primaries and recover them to RGB by (0.299, 0.587, 0.114).
I'll locate whoever was responsible for that whole 601/709 brouhaha in the first place and will taunt him fiercely... :mad:
i do agree :D

[pitch.fr]

@pitch.fr
are you really sure BD uses SMPTE-C primaries? that must be confusing.
US/ASIAN are SMPTE-C, european are EBU....just like DVD.

it's due to the Sony CRT monitors being used in mastering houses.

the ISF makes lists for their users so they know which gamut to switch to.

it's all been discussed in here, and tritical made an Avisynth plugin that I use to get proper gamut conversion in ffdshow :
http://forum.doom9.org/showthread.php?t=139389

there's also a PS script, that Haali told me he would add to his renderer later this month :
http://www.avsforum.com/avs-vb/showthread.php?t=912720

[Kado]

I only noticed this after reading Japhsoncross post (http://forum.doom9.org/showthread.php?p=1185187#post1185187) about the ATI/nVidia situation but I was using YUV instead of YV12. If I use YUV I get blocks that the shader can fix but with YV12 I get none of that maybe because I have a Geforce 9800GTX?

@pitch.fr
AS for the BT.601/709 range I was using BT.709 on SD material (maybe that's the problem) and luma range was "Standard" (16-235) on ffdshow and "Full" (0-255) on the nVidia control panel to remove the washed out image with EVR Custom Presenter.

[DigitalDeviant]

I wouldn't recommend using it on non-YV12 video (since YUY2 would need a shader that only does horizontal interpolation, and RGB doesn't need any at all), but for YV12 it shouldn't have any downsides.

Well, my problem is I have a HD3650 and I'm getting bad chroma upsampling even using the NV12 colorspace in certain situations using EVR (both custom and regular). Can I use the shader for this?

[pitch.fr]

I was using BT.709 on SD material (maybe that's the problem)
despite what some obscure MPEG2 specs say, SD=601 / HD=709.

at least there's no fuss & fighting on this.

Well, my problem is I have a HD3650 and I'm getting bad chroma upsampling even using the NV12 colorspace in certain situations using EVR (both custom and regular). Can I use the shader for this?
I'm surprised they can get progressive chroma on nvidia.
ATi can only do interlaced.

[DigitalDeviant]

I'm surprised they can get progressive chroma on nvidia.
ATi can only do interlaced.

I don't quite understandyou here. Do you mean that I should really only be getting correct upsampling on interlaced video? Come to think of it, I might only have been noticing it on progressive material.

Since nVidia doesn't have this problem, have they fixed they're lack of VC-1 acceleration? Might be clue to swap cards.

[pitch.fr]

this is with 720p, EVR and an ATI HD card :
http://forum.doom9.org/showpost.php?p=1183500&postcount=10

you can't get progressive chroma upsampling AFAIK...or maybe the media player needs to tells the drivers it's progressive ?

MPC HC and KMPlayer fail at progressive chroma on the ATi...

[Leak]

Well, my problem is I have a HD3650 and I'm getting bad chroma upsampling even using the NV12 colorspace in certain situations using EVR (both custom and regular). Can I use the shader for this?
Ummm... why don't you just try it? It's not as if you can't just toggle it on/off or as if it left permanent damage in your video...

[DigitalDeviant]

@pitch.fr
Interesting. I usually convert most of my video to RGB in ffdshow but had been trying out the NV12 colorspace since I use it for HW deinterlacing and wondered if I couldn't just use it for everything. From my experimenting, it illustrated bad upsampling on all EVR custom settings but on regular EVR it only showed up when the video was upsized beforehand with ffdshow. VMR9 Renderless seemed OK all around but I haven't really put it through the ringer since I won't really be using it.

Now I need an interlaced sample to test. If it is only progressive video affected then I suppose that it doesn't matter since I only use NV12 for interlaced video (unless, the thought occures, that DVXA decoders output NV12).

And I suppose if auto BT.601/709 selection is based on resolution then resizing it with ffdshow before hand will throw that off but I'll figure that one out when I get to it.

@Leak
Tried it and tentatively it looks OK but I never trust my own eyes. So far it looks like you've done a great job.

[Japhsoncross]

US/ASIAN are SMPTE-C, european are EBU....just like DVD.

it's due to the Sony CRT monitors being used in mastering houses.

the ISF makes lists for their users so they know which gamut to switch to.

it's all been discussed in here, and tritical made an Avisynth plugin that I use to get proper gamut conversion in ffdshow :
http://forum.doom9.org/showthread.php?t=139389

there's also a PS script, that Haali told me he would add to his renderer later this month :
http://www.avsforum.com/avs-vb/showthread.php?t=912720

Oh, my... I missed that great post:)
I used to work for some of the SONY professional monitors design, mainly color space conversion designing.
I missed the stage of mastering. Hmm... then it will be easy to recover everything theoretically. I'll try to get a Minlota CA-210 to measure my monitor first to make sure they are still stable now.
Thanks for the information:P

BTW:
10-bit is not enough for linear processing, at least 11bits. Oh, a little bit off topic ;)
nice projector.

[pitch.fr]

BTW: 10-bit is not enough for linear processing, at least 11bits. Oh, a little bit off topic ;)
well I'm in contact with the french ISF CEO and several ppl involved in professional color management, here in France.

they said that DCI projectors work in 10 bit, and that this was more than enough ?!

Sanyo and other consumer brands like to boast about 12 bits, but this would be overkill from what they say ?

but anyhow both tritical's and PS plugins work in 32 float ;)

what I'm interested in, at this point, is knowning how to recognize at a glance whether there's a gamut hiccup.

basically for BT601/709, it's quite easy.

if ppl faces look greenish, you're converting 601 to 709
if reds look faded and colors washed out, you're converting 709 to 601

http://pix.nofrag.com/1/8/b/89ac689b298066012fefad1f061b6t.jpg (http://pix.nofrag.com/1/8/b/89ac689b298066012fefad1f061b6.html)

but a friend of mine recorded some HD episodes of Seinfeld on satellite, they've been reframed to 16/9 and obviously converted from SMPTE-C to HDTV.

if I play them in SMPTE-C, ppl faces look greenish to death.

so this is one : HDTV in SMPTE-C = greenish faces

also the Doomsday BD looks terrible in SMPTE-C, and a lot better in EBU.
considering it's a UK movie, that would make sense I guess...

[Japhsoncross]

the picture you posted is a typical case.
actually, there are a lot of things to consider.

SMPTE is a good pattern for me to show if the primaries are the same for two different sources.
but for a real life footage, serveral mistakes can be made in several stages.
SMPTE-C and Rec.709 primaries are very close to each other, if your monitor's primaries are not accurate enough, and you will miss the point.
if you can see big change, the problem might comes from other parts. but different people has different color sensitivity, we do better trust color meters. CA-210 is my good friend, though i can only use it in my office and lab:(
The most accurate way is trying to recover the exact XYZ, and remap them to the R'G'B' values according to your monitor primaries.

BTW: New Sony professional monitors can set primaries for different type of sources. Probably we won't have to do such brain killing things, finding everything possible wrong, in future:)

[Leak]

Okay, I'll drop the idea of discerning between BT.601 and BT.709...

If you give the following script a try in MPC...

ColorBars().Trim(0,-150).ConvertToYV12(matrix="PC.601").ConvertToRGB32(matrix="PC.601").Subtitle("601/601") + \
ColorBars().Trim(0,-150).ConvertToYV12(matrix="PC.709").ConvertToRGB32(matrix="PC.601").Subtitle("709/601") + \
ColorBars().Trim(0,-150).ConvertToYV12(matrix="PC.601").ConvertToRGB32(matrix="PC.709").Subtitle("601/709") + \
ColorBars().Trim(0,-150).ConvertToYV12(matrix="PC.709").ConvertToRGB32(matrix="PC.709").Subtitle("709/709")

...you'll notice that the colors of the bars for all conversion combinations stay exactly the same with my shader turned on or off since I'm using BT.601 conversion in both directions instead of only converting to YV12 and I'm doing no clipping either...

(I.e. the colors for each of the four 5-second segments will of course be different, but my shader doesn't alter them either...)

np: ESG - Insane (Tambourine Mix) (Soul Jazz Records Singles 2006-2007 (Disc 2))

[pitch.fr]

Samsung offers automatic 601/709 detection on their 800B projector.

god knows how they managed to do that :eek:

[Japhsoncross]

for one block with the same color, it will be fine, as you convert and revert back with matrix A and matrix A-1, and the shader does nothing. but for the edges, where the chroma channels are going to be traded between different pixels, then you will find the difference, as the shader always assumes the RGB pixels are BT601 and extract chroma channels in that manner. after getting a blurred result, the difference is shown.
you may try this avs script and check the center pixel to see the difference between shader only does by 601 and the one only does by 709.
blankclip(width=640, height=720, color=$7F0000).addborders(0, 0, 640, 0).converttoyv12(matrix="rec709")

[Japhsoncross]

Samsung offers automatic 601/709 detection on their 800B projector.

god knows how they managed to do that :eek:

would you please provide more detailed information?
if it detects 601/709 in HDMI source, I don't think there's anything difficult, as the colorimetry information is included.
is it able to detect in the YPbPr source if I use 601 conversion for an HD source, and transfer it by YPbPr to the projector?

[pitch.fr]

good point, it only works through HDMI.......they just confirmed it.

http://www.ultimateavmag.com/videoprojectors/908sama800/

The user can also choose one of three color standards: SMPTE-C, HD, or EBU.
I used SMPTE-C for most of my tests and viewing because, according to Kane, most post-production and mastering facilities still use monitors calibrated to this color gamut.

[totozero]

I wouldn't recommend using it on non-YV12 video (since YUY2 would need a shader that only does horizontal interpolation, and RGB doesn't need any at all), but for YV12 it shouldn't have any downsides.

As for the BT.601/.709 difference: even if you're using the wrong colorspace it should only have minimal impact since I'm only averaging values after conversion followed by converting back to RGB using the same (wrong) conversion afterwards.

But I'll add a height check to use BT.709 for HD material when I get home... (and then I'll locate whoever was responsible for that whole 601/709 brouhaha in the first place and will taunt him fiercely... :mad:)

Hello Leak,
been following this thread from the beginning and now I've added your shader in my setup i.e. MPC-HC/CUSTOM EVR/FFDSHOW input yv12/decoding -> output yv12 (B4 I used to output rgb32hq due 2 this chroma stuff)

Been making some comparisons and in the end yuy2+yer shader performed slightly better than rgb32hq.

What's bugging me is that you told us this shader was inaccurate while outputting yuy2.

If so, would you be kind enough to mod. your shader to fit correct yuy2 output ?

TIA.

[Leak]

If so, would you be kind enough to mod. your shader to fit correct yuy2 output ?
Well, it's so straightforward that a child could do it (Fetch me a child! None here? D'oh - do I have to do everything myself? :() but I'm at work, so this is untested:

/*
YUY2 chroma upsampling fixer
by Kurt Bernhard 'Leak' Pruenner

Use with YUY2 output if the half-resolution chroma
gets upsampled in hardware by doubling the values
instead of interpolating between them.

(i.e. if you're getting blocky red edges on dark
backgrounds...)
*/

sampler s0 : register(s0);
float4 p0 : register(c0);
float4 p1 : register(c1);

#define width (p0[0])
#define height (p0[1])

float4 getPixel(float2 tex, float dx)
{
tex.x+=dx;

return tex2D(s0, tex);
}

float4 rgb2yuv(float4 rgb)
{
float4x4 coeffs=
{
0.299, 0.587, 0.114, 0.000,
-0.147,-0.289, 0.436, 0.000,
0.615,-0.515,-0.100, 0.000,
0.000, 0.000, 0.000, 0.000
};

return mul(coeffs,rgb);
}

float4 yuv2rgb(float4 yuv)
{
float4x4 coeffs=
{
1.000, 0.000, 1.140, 0.000,
1.000,-0.395,-0.581, 0.000,
1.000, 2.032, 0.000, 0.000,
0.000, 0.000, 0.000, 0.000
};

return mul(coeffs,yuv);
}

float4 main(float2 tex : TEXCOORD0) : COLOR
{
float dx=1/width;

float4 yuv0=rgb2yuv(getPixel(tex,-dx));
float4 yuv1=rgb2yuv(getPixel(tex, 0));
float4 yuv2=rgb2yuv(getPixel(tex, dx));

float4 yuv=(yuv0*1+yuv1*2+yuv2*1)/4;

yuv.r=yuv1.r;

return yuv2rgb(yuv);
}

[totozero]

Many thanks leak.
Guess I'm not even skilled as a child.
I'll give it a try it and tell you if it makes a difference.

C'ya.

[STaRGaZeR]

That last update doesn't work right for me, only a part of the bloking is gone. The previous one was spot on.

[tetsuo55]

Do i understand this correctly?

There are basically 5 problems we are facing:
1.Blocking due to incorrect rgb<>others conversion.
2.Difficulty in accurately detecting BT type
3.Drivers converting between BT's regardless of the fact if its needed or not
4.Incorrect BT conversion due to incorrect primaries
5.Lack of chroma upsampling for progessive sources(Does this help image quality?)

[Leak]

That last update doesn't work right for me, only a part of the bloking is gone. The previous one was spot on.
That "last update" is a shader for YUY2, not YV12 - it'll only interpolate horizontally, as in YUY2 the chroma has the same height but only half the width of the luma.

Don't tell me I changed the comments at the beginning of the shader for nothing... :(

np: Matias Aguayo - Uno (Soul Jazz Records Singles 2006-2007 (Disc 1))

[STaRGaZeR]

That "last update" is a shader for YUY2, not YV12 - it'll only interpolate horizontally, as in YUY2 the chroma has the same height but only half the width of the luma.

Sorry, don't bite me :p I though it was an incremental update, with support for both YUY2 and YV12. But now thinking about it it's possible to see if the source is in one format or another and apply the shader accordingly?

[Leak]

Sorry, don't bite me :p I though it was an incremental update, with support for both YUY2 and YV12. But now thinking about it it's possible to see if the source is in one format or another and apply the shader accordingly?
Bite you? Nah, I already had breakfast today... :p

And no, it's not really possible to detect anything here since all the shader gets to see is the final RGB image, and at that point there's no information about the original colorspace left...

You could, however, just use the YV12 shader for both since it can't look worse than what it would look like if the video had been in YV12 in the first place...

[STaRGaZeR]

Yeah, I'll use the YV12 for everything.

How about my question here (http://forum.doom9.org/showpost.php?p=1186058&postcount=4303)? Let's use this thread only.

[arfster]

:) it detects the image height>= 720, then BT709, otherwise BT601.

Better to use horizontal res as Haali renderer does imo (say 1000), else you get bt601 used on bt709 widescreen 720p content encoded without bars - eg 1280 * 550ish

[JohnLai]

Why the YUY2 Chroma sampling fixer is not added into the mpc-hc as well?

[Leak]

Why the YUY2 Chroma sampling fixer is not added into the mpc-hc as well?
Probably because nobody has poked Casimir666 to do it...

Why don't you send him a PM with a link to my post? :)