Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Hardware & Software > Software players

Reply
 
Thread Tools Search this Thread Display Modes
Old 8th June 2014, 22:39   #61  |  Link
Asmodian
Registered User
 
Join Date: Feb 2002
Location: San Jose, California
Posts: 4,372
Quote:
Originally Posted by madshi View Post
I always do the RGB->YCbCr->RGB conversion because you usually want to use more neurons for luma than for chroma, even if you enable both luma+chroma doubling.
Wow, I had badly misread your explanation.

Of course separate luma and chroma doubling (only doubling luma) has always been better in my tests but if it is low effort maybe a special case for the future when we can all run NNEDI3 256 Luma + Chroma doubling? Or does the math workout the same either way?

Quote:
Originally Posted by Shiandow View Post
On what channel does MadVR perform nnedi3 resizing in YCbCr mode? Because if I use the chromaNEDI shaders and add a pre-resize shader that converts to RGB the resulting image is quite noticeably better: resized from YCbCr / resized from RGB.
From my limited understanding there would be a difference between NNEDI3 or NEDI on R, G, and B vs NNEDI3 or NEDI on Y, U, V converted to RGB. The HVS important luma information is more connected to R, G, and B than it is to U or V and both NNEDI3 and NEDI might be sort of human like as far as edge recognition goes? Also the source usually starts with half resolution U and V. Combining the chroma with the higher resolution luma information (via YUV -> RGB) might result in sharper easier to recognize edges for NNEDI3 or NEDI? Of course there is no longer a luma channel so all channels would have low resolution chroma mixed into them but your example seems to say RGB is better.
Asmodian is offline   Reply With Quote
Old 9th June 2014, 08:53   #62  |  Link
madshi
Registered Developer
 
Join Date: Sep 2006
Posts: 9,140
Quote:
Originally Posted by Asmodian View Post
if it is low effort maybe a special case for the future when we can all run NNEDI3 256 Luma + Chroma doubling? Or does the math workout the same either way?
I think the math would either work out the same, or if it does not, YCbCr might be better than RGB. Just a guess, though.

Quote:
Originally Posted by Shiandow View Post
On what channel does MadVR perform nnedi3 resizing in YCbCr mode? Because if I use the chromaNEDI shaders and add a pre-resize shader that converts to RGB the resulting image is quite noticeably better: resized from YCbCr / resized from RGB.
Hmmmm... I'm not sure what you mean exactly with this. The pre-resize shader should already run in RGB. All of madVR's custom pixel shaders (both pre-resize and post-resize) are always fed with RGB data. So why would you convert to RGB, if the data is already RGB? Can you list the exact processing chain for both of these images?
madshi is offline   Reply With Quote
Old 9th June 2014, 14:24   #63  |  Link
Shiandow
Registered User
 
Join Date: Dec 2013
Posts: 753
Quote:
Originally Posted by madshi View Post
I think the math would either work out the same, or if it does not, YCbCr might be better than RGB. Just a guess, though.
Since NNEDI3 is nonlinear the math doesn't exactly work out to be the same. I think all other algorithms in MadVR (except spline) are linear so in those cases it doesn't really matter which colour space you use (unless you change the gamma).

Quote:
Originally Posted by madshi View Post
Hmmmm... I'm not sure what you mean exactly with this. The pre-resize shader should already run in RGB. All of madVR's custom pixel shaders (both pre-resize and post-resize) are always fed with RGB data. So why would you convert to RGB, if the data is already RGB? Can you list the exact processing chain for both of these images?
I fear I've been slightly unclear. This is happened when I put an empty file called YCbCr in the MadVR folder, and used NNEDI3 for luma doubling and Jinc3AR for image upscaling. It seems that MadVR skips the YCbCr -> RGB conversion but doesn't skip the RGB -> YCbCr conversion after the pre-resize pixel shaders. Which means that it's performing NNEDI3 on the wrong colour space which lowers the picture quality.

It seems that this is also the reason I thought NEDI looked superior, and it may be the reason that improving chroma upscaling seemed to slightly improve the luma upscaling in some of those images.
Shiandow is offline   Reply With Quote
Old 9th June 2014, 14:56   #64  |  Link
madshi
Registered Developer
 
Join Date: Sep 2006
Posts: 9,140
Quote:
Originally Posted by Shiandow View Post
Since NNEDI3 is nonlinear the math doesn't exactly work out to be the same.
Yeah, it won't be identical, but probably in the end it will look similar.

Quote:
Originally Posted by Shiandow View Post
I fear I've been slightly unclear. This is happened when I put an empty file called YCbCr in the MadVR folder, and used NNEDI3 for luma doubling and Jinc3AR for image upscaling. It seems that MadVR skips the YCbCr -> RGB conversion but doesn't skip the RGB -> YCbCr conversion after the pre-resize pixel shaders. Which means that it's performing NNEDI3 on the wrong colour space which lowers the picture quality.
Oooh, I think you're right! Sorry about that. I had implemented the "YCbCr" empty file hack just a long time ago for testing purposes and when implementing the RGB->YCbCr->RGB conversions for NNEDI3 I didn't think of that the "YCbCr" would need special treatment. So it seems NNEDI3 image doubling doesn't work well with the "YCbCr" empty file hack atm. Will put that on my to do list to fix...
madshi is offline   Reply With Quote
Old 9th June 2014, 21:21   #65  |  Link
Shiandow
Registered User
 
Join Date: Dec 2013
Posts: 753
It seems that the current versions of the ChromaNEDI shaders only use the directional information very weakly, apparently this is still enough to outperform most other algorithms but it causes a loss of detail. I think I've been able to fix this using a few of the same tricks I used in the NEDI shaders (and then some). I'll post these improved versions and a couple of comparisons when I have time, they still require some fine-tuning.

Edit: Later I'll post a more detailed explanation of what I changed but first some comparisons between the new and old method:
(Chroma only)
New method
Old method
Bicubic

With luma

One thing I find worrying is that Bicubic seems to be shifted slightly to the left, I've been unable to confirm if this is correct. For my implementation I assumed that the chroma channel corresponded to the average of each 2x2 block.

Last edited by Shiandow; 10th June 2014 at 20:05. Reason: Added examples
Shiandow is offline   Reply With Quote
Old 12th June 2014, 04:54   #66  |  Link
pie1394
Registered User
 
Join Date: May 2009
Posts: 212
Quote:
Originally Posted by Shiandow View Post
Edit: Later I'll post a more detailed explanation of what I changed but first some comparisons between the new and old method:
(Chroma only)
New method
Old method
Bicubic

With luma
Personal opinions ...

[High-frequency details]
New method > Bicubic >> Old method

[Low-frequency details]
Bicubic > New method > Old method



This time I use the Sony Fit 11A's 11.6" 1920x1080 to view these samples. This monitor's backlit is QD-LED --- so richer colors than most LCD monitors, and more similar to a good TV set. But it has ridiculous high gamma value by factory default setting. The object / skin-tone colors also look weird (too vivid) if its RGB LUT settings are not adjusted in the display driver.

Anyway it looks impressive with your new method's sample on this monitor. The little dinosaur's head looks more pop/stereo. The girl's hair colors are also distinguishable one by one even on such small-sized FHD panel.
pie1394 is offline   Reply With Quote
Old 12th June 2014, 06:56   #67  |  Link
madshi
Registered Developer
 
Join Date: Sep 2006
Posts: 9,140
Quote:
Originally Posted by Shiandow View Post
One thing I find worrying is that Bicubic seems to be shifted slightly to the left, I've been unable to confirm if this is correct. For my implementation I assumed that the chroma channel corresponded to the average of each 2x2 block.
All modern compression algorithms (MPEG2, VC1, h264, h265) have Chroma moved half a pixel to the left for 4:2:0 content:

chroma placement image
madshi is offline   Reply With Quote
Old 12th June 2014, 10:41   #68  |  Link
Shiandow
Registered User
 
Join Date: Dec 2013
Posts: 753
Quote:
Originally Posted by madshi View Post
All modern compression algorithms (MPEG2, VC1, h264, h265) have Chroma moved half a pixel to the left for 4:2:0 content:
Then it seems that I have made the current version for the wrong "type" of 4:2:0. Although, on the bright side, that means that the image quality can be improved even further.

Edit: Well luckily that was somewhat easy to fix, and the resulting image is definitely better.

Last edited by Shiandow; 12th June 2014 at 11:39.
Shiandow is offline   Reply With Quote
Old 12th June 2014, 13:25   #69  |  Link
leeperry
Kid for Today
 
Join Date: Aug 2004
Posts: 3,477
Quote:
Originally Posted by Shiandow View Post
Then it seems that I have made the current version for the wrong "type" of 4:2:0
While on that subject, would that be possible to write a PS script that would map the chroma alignment from MPEG2/H264 to MPEG1? There is no way to get properly aligned MPEG1 chroma in mVR when not feeding it RGB so a "hotfix" PS script would be full of win
leeperry is offline   Reply With Quote
Old 12th June 2014, 13:26   #70  |  Link
Shiandow
Registered User
 
Join Date: Dec 2013
Posts: 753
Allright, I think the improved chroma shaders are now more or less ready. The new shaders only use a 2nd order interpolation instead of a 4th order, the linear equation that you needed to solve for a 4th order interpolation has the annoying property that it becomes unstable if you get close to an edge, which means that you either have a lot of artefacts or you can only use the directional information very weakly. The new shaders also use a more stable method for solving these equations which also reduces artefacts. I've also added a smarter way of limiting the resulting coefficients, which forces the chroma intensity to stay the same and prevents it from interpolating too strongly in one direction.

I'll add the new shaders to the first post.

By the way, the only purpose of the third shader is to shift the chroma channel half a pixel upwards; it would probably be better to do this simultaneously with scaling.
Shiandow is offline   Reply With Quote
Old 12th June 2014, 13:47   #71  |  Link
huhn
Registered User
 
Join Date: Oct 2012
Posts: 7,568
Quote:
Originally Posted by leeperry View Post
While on that subject, would that be possible to write a PS script that would map the chroma alignment from MPEG2/H264 to MPEG1? There is no way to get properly aligned MPEG1 chroma in mVR when not feeding it RGB so a "hotfix" PS script would be full of win
a h264 stream with: chromaloc 1 should provide all infos needed. and mpeg1 input should work too.
huhn is offline   Reply With Quote
Old 12th June 2014, 13:54   #72  |  Link
Shiandow
Registered User
 
Join Date: Dec 2013
Posts: 753
Quote:
Originally Posted by leeperry View Post
While on that subject, would that be possible to write a PS script that would map the chroma alignment from MPEG2/H264 to MPEG1? There is no way to get properly aligned MPEG1 chroma in mVR when not feeding it RGB so a "hotfix" PS script would be full of win
A slight adaptation of chromaNEDI-II should do it; it currently shifts things half a pixel vertically, but it could just as well shift them horizontally. Although I would like it better if the shifting was done simultaneously with scaling. Anyway the following code should shift the chroma channels half a pixel to the right. It expects the input to be in YCbCr, and should be used pre-resize. It tries to convert the result to RGB but I can't guarantee that that part will always work; it might be wrong about the colour space.

Code:
// $MinimumShaderProfile: ps_3_0
sampler s0 : register(s0);
float4 p0 :  register(c0);

#define width  (p0[0])
#define height (p0[1])

#define px (1.0 / (p0[0]))
#define py (1.0 / (p0[1]))

#define sqr(x) (dot(x,x))
#define I (float2x2(1,0,0,1))

//Conjugate residual 
float2 solve(float2x2 A,float2 b) {
	float2 x = 1/2.0;
	float2 r = b - mul(A,x);
	float2 p = r;
	float2 Ar = mul(A,r);
	float2 Ap = Ar;
	for (int k = 0;k < 3; k++){
		float a = min(100,dot(r,Ar)/dot(Ap,Ap));
		x = x + a*p;
		float2 rk = r; float2 Ark = Ar;
		r = r - a*Ap;
		Ar = mul(A,r);
		float b = dot(r,Ar)/dot(rk,Ark);
		p = r + b*p;
		Ap = Ar + b*Ap;
	}
	return x;
}

float4 toRGB(float4 s1) {
	s1.yz -= 0.5;
	if(width < 1120 && height < 630) return float3(s1.x+1.402*s1.z, dot(s1, float3(1, -.202008/.587, -.419198/.587)), s1.x+1.772*s1.y).rgbb;// SD Y'CbCr to RGB output
	return float3(s1.x+1.5748*s1.z, dot(s1, float3(1, -.1674679/.894, -.4185031/.894)), s1.x+1.8556*s1.y).rgbb;// HD Y'CbCr to RGB output
}

#define Col(xy) (tex2D(s0,tex+float2(px,py)*(xy)).yz-0.5)
#define Get(xy) (tex2D(s0,tex+float2(px,py)*(xy)).x+0.25)
#define Get4(xy) (float2(Get(xy+dir[0]),Get(xy+dir[1])))

float4 main(float2 tex : TEXCOORD0) : COLOR {
	float4 c0 = tex2D(s0,tex);

	float2 dir[2] =  {{-1,0},{1,0}};
	float2 wind[4] = {{-1,0},{1,0},{0,1},{0,-1}};
	float2 pos[2] = {{0,0},{1,0}};
	
	float w = 2;
	float2x2 R = w*mul(float2x1(Get4(0)),float1x2(Get4(0)));
	float2 r = w*Get(0)*Get4(0);
	float4x2 C = {Get4(wind[0]),Get4(wind[1]),Get4(wind[2]),Get4(wind[3])};
	float4 y = {Get(wind[0]),Get(wind[1]),Get(wind[2]),Get(wind[3])};
	R += mul(transpose(C),C);
	r += mul(y,C);

	//Normalize coefficients
	float n = 16;
	R/= n; r /= n;
	
	//Solve equations
	float2 a = solve(R+0.000001*I,r);

	//Limit the coefiicents of a
	float2 b = float2(a[0]+a[1]-1,a[0]-a[1]);
	b[0] = clamp(b[0],-.02,.02); 	//Clamp intensity
	b[1] = clamp(b[1],-1,1); 	//Clamp "directedness"
	a = .5 + float2(.5,.5)*b[0] + float2(.5,-.5)*b[1];

	c0.a = dot(c0.gb,1);
	c0.gb = 0.5+mul(a,float2x2(Col(pos[0]),Col(pos[1])));

	return toRGB(c0);
}

Last edited by Shiandow; 12th June 2014 at 15:03. Reason: It shifted things in the wrong direction, it should be fixed now.
Shiandow is offline   Reply With Quote
Old 12th June 2014, 14:57   #73  |  Link
FireFreak111
Registered User
 
Join Date: Apr 2014
Posts: 48
Running the chroma shaders in MPC-HC in pre-resize in order, the first two blur the image, and with the third one the image looks blown out and grainy. With the YbCbCr file, basically the same problem. Chroma set to NN.

Clearly not working here.

GTX 660, newest Madvr, nightly MPC-HC, Windows 8.1.1.

From the samples shown however, this looks like a great middle-ground between Chroma Jinc and NNEDI3, which for DVD and lower quality content I just cant combine with Luma NNEDI3 16N with both double and quad enabled on this 660 and i7 870.

Wish NNEDI3 ran on DirectCompute.

Last edited by FireFreak111; 12th June 2014 at 15:07.
FireFreak111 is offline   Reply With Quote
Old 12th June 2014, 15:11   #74  |  Link
Shiandow
Registered User
 
Join Date: Dec 2013
Posts: 753
Could you post a few screenshots of what happens?
Shiandow is offline   Reply With Quote
Old 12th June 2014, 15:30   #75  |  Link
FireFreak111
Registered User
 
Join Date: Apr 2014
Posts: 48
Nearest Neighbour


Chrome-NEDI
FireFreak111 is offline   Reply With Quote
Old 12th June 2014, 15:30   #76  |  Link
madshi
Registered Developer
 
Join Date: Sep 2006
Posts: 9,140
Thanks for your efforts so far, Shiandow, I'll definitely play with your shaders when I find some time!
madshi is offline   Reply With Quote
Old 12th June 2014, 15:32   #77  |  Link
Shiandow
Registered User
 
Join Date: Dec 2013
Posts: 753
Oh it turns out that I copied the wrong shader for chromaNEDI-III, it should work with the current one. Thanks FireFreak111 for noticing that.
Shiandow is offline   Reply With Quote
Old 12th June 2014, 15:54   #78  |  Link
FireFreak111
Registered User
 
Join Date: Apr 2014
Posts: 48
Sorry for the full size images, photobuckets linking is messed when trying to use a full size image.

There is a distinct blur when using these shaders, it doesn't appear to be working correctly.

Nearest Neighbour:


Chroma-NEDI
FireFreak111 is offline   Reply With Quote
Old 12th June 2014, 15:58   #79  |  Link
Shiandow
Registered User
 
Join Date: Dec 2013
Posts: 753
Are you sure that the input is in YCbCr? Because it looks like the kind of images you get when the input is RGB.
Shiandow is offline   Reply With Quote
Old 12th June 2014, 16:02   #80  |  Link
FireFreak111
Registered User
 
Join Date: Apr 2014
Posts: 48
Thankyou, I had named the YCbCr file wrong. The YCbCr to RGB should be post Chroma-NEDI right? It seems to be working correctly now.
FireFreak111 is offline   Reply With Quote
Reply

Tags
chromanedi, nedi, shader, superres, upscaling

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 06:07.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2023, vBulletin Solutions Inc.