Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Hardware & Software > Software players

Reply
 
Thread Tools Search this Thread Display Modes
Old 12th June 2014, 16:11   #81  |  Link
Shiandow
Registered User
 
Join Date: Dec 2013
Posts: 753
Yes the YCbCr shader should be after Chroma-NEDI. Glad it's working!
Shiandow is offline   Reply With Quote
Old 12th June 2014, 16:24   #82  |  Link
FireFreak111
Registered User
 
Join Date: Apr 2014
Posts: 48
Small thing, your YCbCr to RGB shader is bugged for SD, its only outputting 16:235 (example being a fully black 1st frame is grey). On HD, Blacks is fully functional.

For this SD content, the primaries are apparently SMPTE C (not BT.709, as usual). The shader isn't handling this type.

Last edited by FireFreak111; 12th June 2014 at 16:26.
FireFreak111 is offline   Reply With Quote
Old 12th June 2014, 16:54   #83  |  Link
Shiandow
Registered User
 
Join Date: Dec 2013
Posts: 753
I added a different RGBtoYCbCr shader to the first post that should also work for SD content, but again likely not all. You could also use the RGBtoYCbCr shader I added instead of using the YCbCr hack, this should at least keep the colours the same.
Shiandow is offline   Reply With Quote
Old 13th June 2014, 02:40   #84  |  Link
FireFreak111
Registered User
 
Join Date: Apr 2014
Posts: 48
Absolutely perfect, working completely without the hack (which was messing with my gamma for some reason). Thankyou for this shader.
FireFreak111 is offline   Reply With Quote
Old 13th June 2014, 19:28   #85  |  Link
Shiandow
Registered User
 
Join Date: Dec 2013
Posts: 753
Minor update: the shaders chromaNEDI-I and II have been improved a bit by using a slightly larger (more disk shaped) window, the difference is most noticeable when not using chromaNEDI-III (which seems to remove some detail, including aliasing).
Shiandow is offline   Reply With Quote
Old 14th June 2014, 09:20   #86  |  Link
FireFreak111
Registered User
 
Join Date: Apr 2014
Posts: 48
Is there a way to view Chroma only, so I can compare different algorithms? Is there a shader I could use?
FireFreak111 is offline   Reply With Quote
Old 14th June 2014, 14:22   #87  |  Link
Shiandow
Registered User
 
Join Date: Dec 2013
Posts: 753
You can just modify the YCbCrtoRGB shader slightly so it will remove the luma channel:

Code:
sampler s0;
float2 c0;

float4 main(float2 tex : TEXCOORD0) : COLOR
{
	float3 s1 = tex2D(s0, tex).rgb;// original pixel
	s1.yz -= 0.5;
	s1.r = 0.3; // erase luma
	if(c0.x < 1120 && c0.y < 630) return float3(s1.x+1.402*s1.z, dot(s1, float3(1, -.202008/.587, -.419198/.587)), s1.x+1.772*s1.y).rgbb;// SD Y'CbCr to RGB output
	return float3(s1.x+1.5748*s1.z, dot(s1, float3(1, -.1674679/.894, -.4185031/.894)), s1.x+1.8556*s1.y).rgbb;// HD Y'CbCr to RGB output
}
Shiandow is offline   Reply With Quote
Old 14th June 2014, 14:42   #88  |  Link
FireFreak111
Registered User
 
Join Date: Apr 2014
Posts: 48
This works perfectly except when the Chroma-NEDI shader is in the middle, then everything shows as normal, so I cant compare your shader to others. (Luma is included when using your shader, with just RGB-YCbCr-RGB I see chroma.

Also, alot of grey in this chroma, no blacks, is that normal? First frame of a movie for example is grey, it's black when the new value you added is at 0.0


Last edited by FireFreak111; 14th June 2014 at 14:54.
FireFreak111 is offline   Reply With Quote
Old 14th June 2014, 14:59   #89  |  Link
Shiandow
Registered User
 
Join Date: Dec 2013
Posts: 753
What I usually do when I want to compare is to add a RGBtoYCbCr shader followed by the chroma only shader post-resize, then I can toggle chromaNEDI on/off by enabling disabling the pre-resize shaders. You could also use it instead of the YCbCrtoRGB shader but that makes it harder to compare.

You see a lot of grey because it fixes the luma to 0.25, if you set it to 0 then it's usually too dark to see anything.
Shiandow is offline   Reply With Quote
Old 14th June 2014, 16:01   #90  |  Link
FireFreak111
Registered User
 
Join Date: Apr 2014
Posts: 48
Using Frozen for chroma investigation, due to the movie's good chroma compression and heavy use of it, especially in the pink cloak constantly used.

NEDI seems to fare well against NNEDI3, except for some blur on distinct edges (pillars, characters lower cloak, reflection, shoes). Jinc3 (not shown) has the same definition here as NNEDI3, so its distinctly NEDI's problem, not an advantage of NNEDI3.

NEDI



NNEDI3-16



To be fair, NEDI is 2-3x faster then NNEDI3, entire chain taking only 11.7ms on 1080p24 content (at 1080p48 with no SM).

Last edited by FireFreak111; 14th June 2014 at 16:04.
FireFreak111 is offline   Reply With Quote
Old 14th June 2014, 17:50   #91  |  Link
Shiandow
Registered User
 
Join Date: Dec 2013
Posts: 753
One key difference between NEDI and other chroma upscaling algorithms is that NEDI only makes edges sharp if the luma channel is sharp. Without the luma channel it's hard to tell if NEDI is blurring an edge that it shouldn't or NNEDI3 is sharpening an edge that it shouldn't. Both are very well possible. I also found it interesting that NEDI seems to be better at bringing out detail on the pillars itself.
Shiandow is offline   Reply With Quote
Old 14th June 2014, 23:45   #92  |  Link
leeperry
Kid for Today
 
Join Date: Aug 2004
Posts: 3,477
Quote:
Originally Posted by Shiandow View Post
A slight adaptation of chromaNEDI-II should do it; it currently shifts things half a pixel vertically, but it could just as well shift them horizontally. Although I would like it better if the shifting was done simultaneously with scaling. Anyway the following code should shift the chroma channels half a pixel to the right. It expects the input to be in YCbCr, and should be used pre-resize. It tries to convert the result to RGB but I can't guarantee that that part will always work; it might be wrong about the colour space.
Sweet, thanks! All MPEG1 movies are using BT.601 but fair enough, if there can't a fail-safe way to work it out afterwards I'll just feed mVR with RGB32 for MPEG1. Would need to run test patterns in order to ensure that ffdshow does it properly duh ^^
leeperry is offline   Reply With Quote
Old 15th June 2014, 01:45   #93  |  Link
FireFreak111
Registered User
 
Join Date: Apr 2014
Posts: 48
Chroma.rar

Here is some screenshots with Luma shots for NEDI and NNEDI3, Chroma for NEDI, NNEDI3, Jinc3 and Bicubic, compressed into a RAR. This is after disabling the no DXVA copyback option in madVR, which was bluring the Chroma channel (still a blur difference between NEDI and NNEDI3, causing some minor artifacts like the bottom of the character's cloak in the Luma shot)

On another note, is it possible to combine the first two shaders, considering there's only a one line difference between them? Functions possibly?

Last edited by FireFreak111; 15th June 2014 at 06:30.
FireFreak111 is offline   Reply With Quote
Old 15th June 2014, 16:03   #94  |  Link
Shiandow
Registered User
 
Join Date: Dec 2013
Posts: 753
Quote:
Originally Posted by FireFreak111 View Post
On another note, is it possible to combine the first two shaders, considering there's only a one line difference between them? Functions possibly?
The second shader needs the results of the first one so it's not really easy to do it in just one shader. It would be possible to split them in a slightly different way but I think it will obfuscate the code and it will only be faster because I haven't found a way to make the shader skip processing for a particular pixel.
Shiandow is offline   Reply With Quote
Old 19th June 2014, 19:46   #95  |  Link
Shiandow
Registered User
 
Join Date: Dec 2013
Posts: 753
With the current discussion around chroma channel alignment I thought it would be useful to add support for different chroma patterns. I've therefore updated the chromaNEDI shaders.
Shiandow is offline   Reply With Quote
Old 21st June 2014, 16:23   #96  |  Link
cyberbeing
Broadband Junkie
 
Join Date: Oct 2005
Posts: 1,859
Quote:
Originally Posted by Shiandow View Post
One key difference between NEDI and other chroma upscaling algorithms is that NEDI only makes edges sharp if the luma channel is sharp.
Is this why the NEDI chroma shaders don't remove distinct 2x2 block aliasing steps like the others do? The source video in question was a 640x360 x264 1200Kbps stream.

__________


Nearest Neighbor Chroma (400% zoom)
__________


NEDI Chroma (400% zoom)
__________


Catmull-Rom Chroma (400% zoom)
__________


NNEDI3 64 Chroma (400% zoom)
__________

madVR's YCbCr hack also seems to result in an elevated purplish black level on SD video with the provided YCbCrtoRGB shader as the last step pre-resize. [Edit: This was caused by using a 3DLUT with the YCbCr hack, which I guess must not be supported]. The above images used madVR in normal mode with both RGBtoYCbCr (first step) and YCbCrtoRGB (last step) shaders to workaround this.

Last edited by cyberbeing; 21st June 2014 at 16:34.
cyberbeing is offline   Reply With Quote
Old 21st June 2014, 17:09   #97  |  Link
madshi
Registered Developer
 
Join Date: Sep 2006
Posts: 9,140
@cyberbeing, it might be interesting to look at the luma channel, to check if the blocking is visible there. That might explain (or not) why the NEDI Chroma shader produces that kind of output.
madshi is offline   Reply With Quote
Old 21st June 2014, 17:21   #98  |  Link
cyberbeing
Broadband Junkie
 
Join Date: Oct 2005
Posts: 1,859
Quote:
Originally Posted by madshi View Post
@cyberbeing, it might be interesting to look at the luma channel, to check if the blocking is visible there. That might explain (or not) why the NEDI Chroma shader produces that kind of output.
Okay, I've just sent a PM with a sample to you and Shiandow.
cyberbeing is offline   Reply With Quote
Old 21st June 2014, 17:22   #99  |  Link
Shiandow
Registered User
 
Join Date: Dec 2013
Posts: 753
There seems to be a mismatch between what the shader thinks the chroma channels are and what they actually are. As a result some part of the chroma channel isn't scaled. Using the RGBtoYCbCr shader will ensure that the colours remain accurate even if it guesses the colour space incorrectly, but instead it will use the shaders on the wrong chroma channels. If you use the YCbCr hack you should get a smooth(er) result but the colours might be inaccurate.

If you use the chroma only shader I posted a few posts back, you'll see that the "chroma" part looks perfectly smooth. Unfortunately it is the wrong chroma channel.

Edit: Using the shaders with the YCbCr hack seems to remove all chroma aliasing, and changes the colour. I'm guessing that the BT.601 part of the RGB <-> YCbCr shaders doesn't work correctly.

Last edited by Shiandow; 21st June 2014 at 18:07.
Shiandow is offline   Reply With Quote
Old 21st June 2014, 19:47   #100  |  Link
cyberbeing
Broadband Junkie
 
Join Date: Oct 2005
Posts: 1,859
Quote:
Originally Posted by Shiandow View Post
Edit: Using the shaders with the YCbCr hack seems to remove all chroma aliasing, and changes the colour. I'm guessing that the BT.601 part of the RGB <-> YCbCr shaders doesn't work correctly.
If you think those shaders are incorrect, you could try adapting NVIDIA's sample vertex/pixel shaders for RGB <-> YCbCr conversions which I'd assume should be accurate.

NVIDIA RGBA to YUVA sample | NVIDIA YUVA to RGBA sample

The actual pixel shader portion of these samples seems to be the following, but would need to be made MPC-HC compatible:
Code:
#define QUAD_REAL float
#define QUAD_REAL2 float2
#define QUAD_REAL3 float3
#define QUAD_REAL4 float4

QUAD_REAL4 ToYUV(QuadVertexOutput IN,
		    uniform sampler2D SceneSampler) : COLOR
{   
    QUAD_REAL4 rgba = tex2D(SceneSampler, IN.UV);
    QUAD_REAL3 ctr = QUAD_REAL3(0,.5,.5);
    return QUAD_REAL4(rgb_to_yuv(rgba.xyz)+ctr,rgba.w); // don't lose alpha
}

QUAD_REAL3 rgb_to_yuv(QUAD_REAL3 RGB)
{
    QUAD_REAL y = dot(RGB,QUAD_REAL3(0.299,0.587,0.114));
    QUAD_REAL u = (RGB.z - y) * 0.565;
    QUAD_REAL v = (RGB.x - y) * 0.713;
    return QUAD_REAL3(y,u,v);
}

QUAD_REAL4 FromYUV(QuadVertexOutput IN,
		    uniform sampler2D SceneSampler) : COLOR
{   
    QUAD_REAL4 yuva = tex2D(SceneSampler, IN.UV) - QUAD_REAL4(0,.5,.5,0);
    return QUAD_REAL4(yuv_to_rgb(yuva.xyz),yuva.w); // don't lose alpha
}

QUAD_REAL3 yuv_to_rgb(QUAD_REAL3 YUV)
{
   QUAD_REAL u = YUV.y;
   QUAD_REAL v = YUV.z;
   QUAD_REAL r = YUV.x + 1.403*v;
   QUAD_REAL g = YUV.x - 0.344*u - 1.403*v;
   QUAD_REAL b = YUV.x + 1.770*u;
   return QUAD_REAL3(r,g,b);
}

Last edited by cyberbeing; 21st June 2014 at 19:53.
cyberbeing is offline   Reply With Quote
Reply

Tags
chromanedi, nedi, shader, superres, upscaling

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 15:48.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.