AviSynthShader + SuperRes

[MysteryX]

I was using
nnedi3_rpow2(2, nns=4, cshift="Spline16Resize", Threads=2)

I have added the results with Sinc above.

[luquinhas0021]

it stayed sharper than all others algorithms! As I expected. After, I will test your SR algorithm with Lanczos4, because Sinc makes much ringing.
Only one question: You alrady put etype=1 (minimize squared error)? Comparing with etype=0, what you like more?

I JUST SEE YOU USED SINC EP0 =4.

[MysteryX]

I copy/pasted your command. I personally prefer the Spline16 over Sinc which looks more artificially processed.

I'm adding another test: NNEDI3(spline16)+SuperRes(Passes=3, Strength=1, Softness=.85)

Shiandow added Softness for the purpose of being able to use higher strength and passes and then softening it down. So far I wasn't convinced... I'll see better with this test. The previous tests I made, these settings looked good too, but not better than Passes=2, Strength=.42. Let's see what we get! HD pictures might make it better.

OK. With Softness, Clown and Eclipse look a LOT better, but it makes Lighthouse look like a painting. It works for some content but not all. Passes=2 with Strength=.42 gives more consistent results.

[luquinhas0021]

The algorithm you just post generates more ringing in letters of Eclipse image. But increases sharpness on Clown image. Maybe use 3 passes of superres don't be a gorgeus thing to do in all images. I suggest to you use my script, instead (...cshift="SincResize", ep0=4...), use (...cshift="Spline144resize"...) with passes=2, strenght=1 and softness=0,4.
In Lighthouse image, you last script generated aliasing in some parts.

[MysteryX]

Isn't Spline144 a broken algorithm? You're free to play with the algorithms and post your results.

Here are some tests with the lighthouse with NNEDI3(nns=4, cshift="Spline16Resize")

SuperRes(passes=2, strenth=XXX, softness=0)
Strength=30, 40, 50, 60, 100
http://s22.postimg.org/yct0gczkd/Lighthouse_Str30.png (http://postimg.org/image/yct0gczkd/) http://s22.postimg.org/61c1x251p/Lighthouse_Str40.png (http://postimg.org/image/61c1x251p/) http://s22.postimg.org/bqsahdb7x/Lighthouse_Str50.png (http://postimg.org/image/bqsahdb7x/) http://s22.postimg.org/fofk6ry19/Lighthouse_Str60.png (http://postimg.org/image/fofk6ry19/)http://s22.postimg.org/jfkjfcmbx/Lighthouse_Str100.png (http://postimg.org/image/jfkjfcmbx/)

SuperRes(passes=3, strenth=1, softness=XXX)
30, 40, 50, 60, 70, 80, 90, 100
http://s16.postimg.org/l1rsiyavl/Lighthouse_Soft30.png (http://postimg.org/image/l1rsiyavl/) http://s16.postimg.org/893oj0z9t/Lighthouse_Soft40.png (http://postimg.org/image/893oj0z9t/) http://s16.postimg.org/gz0rjvgr5/Lighthouse_Soft50.png (http://postimg.org/image/gz0rjvgr5/) http://s16.postimg.org/b5qpjb53l/Lighthouse_Soft60.png (http://postimg.org/image/b5qpjb53l/) http://s16.postimg.org/fg5dewa6p/Lighthouse_Soft70.png (http://postimg.org/image/fg5dewa6p/) http://s16.postimg.org/hlzo9edn5/Lighthouse_Soft80.png (http://postimg.org/image/hlzo9edn5/) http://s16.postimg.org/ao05h74wh/Lighthouse_Soft90.png (http://postimg.org/image/ao05h74wh/) http://s16.postimg.org/v6qg8itg1/Lighthouse_Soft100.png (http://postimg.org/image/v6qg8itg1/)

Which one is your favorite?

When playing with madVR, I found that NEDI+SuperRes were doing a good job together; would be worth a try to see how it compares to NNEDI3. Jinc+SuperRes, however, don't go well together.

SuperRes(passes=2, strength=.4, softness=0)

NNEDI3(nns=4, cshift="Spline16Resize")
NNEDI2(cshift="Spline16Resize")
NNEDI3(nns=3, cshift="Spline16Resize")
NNEDI3(nns=1, cshift="Spline16Resize")
http://s22.postimg.org/61c1x251p/Lighthouse_Str40.png (http://postimg.org/image/61c1x251p/)http://s8.postimg.org/4w126dncx/Lighthouse_NNEDI2_Str40.jpg (http://postimg.org/image/4w126dncx/)http://s13.postimg.org/wcfc9exkj/Lighthouse_NNEDI3_Str40.jpg (http://postimg.org/image/wcfc9exkj/)http://s14.postimg.org/xjpow0wct/Lighthouse_NNEDI3_NNS1.jpg (http://postimg.org/image/xjpow0wct/)

Honestly... NNEDI2 is doing ALMOST as good as NNEDI3(nns=4), but NNEDI3 with lower NNS gets blurrier. NNEDI2 gives a sharp output. The only downside is distortion on the white bars.

[MysteryX]

I have added a variant of SuperRes that does the YUV conversion via shaders. Performance is slightly lower, but the quality of colors is better. When doing YUV conversion on the CPU, I get 21fps. However, it is doing Rec.601 color conversion on Rec.709 content! When doing YUV conversion via shaders, I get 17fps (including processing NNEDI3). The first implementation might cause a very slight color distortion, and the 2nd implementation makes the colors more vivid.

To use this variant, use file SuperResYUV.avsi

Here's the comparison. Using SuperRes(passes=2, strength=.42, softness=0)

CPU conversion / GPU conversion

http://s24.postimg.org/3qryk6k41/Lighthouse_Super_Res.jpg (http://postimg.org/image/3qryk6k41/) http://s21.postimg.org/5kcn9j643/Lighthouse_YUV.png (http://postimg.org/image/5kcn9j643/)

http://s24.postimg.org/n3iuqg9qp/Clown_Super_Res.jpg (http://postimg.org/image/n3iuqg9qp/) http://s21.postimg.org/8bvx054mr/Clown_YUV.png (http://postimg.org/image/8bvx054mr/)

http://s10.postimg.org/amjeq6j05/Eclipse_Super_Res.jpg (http://postimg.org/image/amjeq6j05/) http://s21.postimg.org/499heszw3/Eclipse_YUV.png (http://postimg.org/image/499heszw3/)


EDIT: Now this is embarrassing and strange... when I put both versions side-to-side on my computer, I can clearly see a difference. But once converted to PNG, I honestly can't see any difference at all! Perhaps whatever color this makes is being discarded by PNG compression? But PNG is supposed to be lossless. Not sure on this one.

[luquinhas0021]

I don`t know if you saw things like me, but there`s a strange effect in lighthouse when use softness with passes=3. The bigger is amount of softness, a little, little more sharper it`s this photo. Don`t know why!
You asked me about which parameters I like. Then, for not take risk, I preffer passes=2, strenght=1, softness=0. From passes=3, looks like algorithm blurs the photo, and, with passes=3, softness sharp then. I may be wrong, but this is what I`ve seen.

[MysteryX]

Yeah, I don't like the effect of Softness, it's not working well. Shiandow might improve it in the future. For now, Passes=2 and Strength=.42 works best for noisy material, and for clear images, you can get away with 2 passes of strength up to 1.

I figured out what I did wrong in the last test: I was upscaling with nns=1 and comparing with the previous results that had nns=4!

Here's a new test of frame quadrupling.

Spline16
edi_rpow2(2, nns=4, cshift="Spline16Resize") uses NNEDI3 but fixes a few details
SuperRes(passes=2, strength=.42)
SuperRes (YUV conversion done on GPU)
SuperRes (YUV conversion done on GPU) + NNEDI2

http://s18.postimg.org/nf8rjtcet/4x_Clown_Spline.png (http://postimg.org/image/nf8rjtcet/) http://s18.postimg.org/3tfp9vklx/4x_Clown_EDI.png (http://postimg.org/image/3tfp9vklx/) http://s18.postimg.org/sp94xd79x/4x_Clown_Super_Res.png (http://postimg.org/image/sp94xd79x/) http://s18.postimg.org/4k8ffnmz9/4x_Clown_Super_Res_Yuv.png (http://postimg.org/image/4k8ffnmz9/) http://s15.postimg.org/uw6w8pfrb/4x_Clown_Super_Res_NNEDI2.jpg (http://postimg.org/image/uw6w8pfrb/)

http://s23.postimg.org/xgk2zw7qf/4x_Lighthouse_Spline16.png (http://postimg.org/image/xgk2zw7qf/) http://s23.postimg.org/xulezhrtz/4x_Lighthouse_NNEDI3.png (http://postimg.org/image/xulezhrtz/) http://s23.postimg.org/sx7uedpuv/4x_Lighthouse_Super_Res.png (http://postimg.org/image/sx7uedpuv/) http://s23.postimg.org/ljsgff5t3/4x_Lighthouse_Super_Res_NNEDI3.png (http://postimg.org/image/ljsgff5t3/) http://s23.postimg.org/nzu9t9nvr/4x_Lighthouse_Super_Res_NNEDI2.png (http://postimg.org/image/nzu9t9nvr/)

Interestingly enough, the 3rd one (which is the I had been testing before) has some distortion with frame quadrupling! Not sure where that's coming from... The newer implementation doesn't have that distortion... or is that the distortion I saw when using NNEDI2? Here's also with NNEDI2 to compare.

EDIT: NNEDI2 is *slower* than NNEDI3(nns=4), so we can discard it, although it gives *almost* identical results. The new implementation with YUV conversion on the GPU does give considerably better image quality. There's a lot less color distortion on the Lighthouse.

[luquinhas0021]

Can you provide me the file I should paste in Avisynth plug-in folder? Or can you tell me how I install your algorithm in Avisynth?

[MysteryX]

I have improved the performance of the YUV conversion via shaders.

You'll need Shader.dll and the AVSI and CSO files within "Shaders\SuperRes". You might have to specify the "folder" argument to tell SuperRes.avsi where to find all the CSO files. Ideally I'll want to automate that parameter.

SuperRes converts colors on the CPU, while SuperResYUV converts via Shaders. I'll probably remove the CPU-conversion implementation as the other one gives better quality, and now performance is similar. It "was" slower but then the CPU usage was also lower so you could just increase the amount of threads to make it up. Now it runs even better.

[foxyshadis]

MysteryX, I didn't speak any SR algorithm creates artificial details. What I wanted say is common SR algorithms analizes a set of neighbour frames or, in a single image, a lot of similar patterns, in order to, let's say, replicate details from a frame to another.
Your algorithm first upscale image using some algorithm, then downscale it using BiCubic and compare it to original image, creates a difference map, upscale the missing parts and paste into upscale image. This was what I understood.
But, generally, upscaling are detail lossy and downscaling, too. So, probably, the difference map will show so much difference. What I really want know is how the details that are in original image, but aren't in downscaled upscaled image, are pasted into upscaled image, i.e, how this details are upscaled.
David Horman, I don't see so much difference too. The maximum I see was some ringing, in SuperRes image, disappear.

You should be using the term temporal super-resolution to minimize everyone's confusion over your posts. The SuperRes shader could be considered non-linear spatial super-resolution (the way deconvolution is also spatial super-resolution). Please open another thread if you'd like to discuss research into temporal super-resolution, since that's drifting way off topic; if you can come up with something good, I'm all ears. I'd suggest actually having a prototype to discuss instead of just throwing open the discussion, though.

[foxyshadis]

http://s23.postimg.org/sx7uedpuv/4x_Lighthouse_Super_Res.png (http://postimg.org/image/sx7uedpuv/) http://s23.postimg.org/ljsgff5t3/4x_Lighthouse_Super_Res_NNEDI3.png (http://postimg.org/image/ljsgff5t3/)

Any reason why CPU vs GPU results in such a strong red->blue color shift? The same applies to clown, too. Different color matrices maybe?

http://s18.postimg.org/sp94xd79x/4x_Clown_Super_Res.png (http://postimg.org/image/sp94xd79x/) http://s18.postimg.org/4k8ffnmz9/4x_Clown_Super_Res_Yuv.png (http://postimg.org/image/4k8ffnmz9/)

The clown also doesn't seem to make a lot of sense: SR CPU has a ton of aliasing, while SR GPU has none. Obviously that's a point in GPU's favor, but any idea why that is?

[MysteryX]

I didn't create the name for the shader; if you have any complaints about the name, contact Shiandow directly :) It's not temporal, it only processes frames one at a time.

The CPU color conversion is for Rec.601 while the content is Rec.709. Obviously, this creates some slight color distortion. It also creates some overflows. The Shader conversion is Rec.709 and doesn't create overflow. I also saw that the CPU conversion code was creating some image distortion, not sure exactly why.

I just realized. Since the GPU conversion doesn't overflow, I can use UINT16 data instead of half-float for the video buffers. This may slightly increase processing speed, and mostly it will greatly save on data conversion time as it currently needs to convert to/from half-float 7 times when running 2 passes!! This should greatly improve performance!

[MysteryX]

v0.9.2 is released. SuperRes.avsi now uses GPU conversion; you can still test CPU conversion with SuperResCPU.avsi

It now converts to UINT16 data instead of half-float. Convert with precision=2 for UINT16, precision=3 for half-float. "convertYuv" parameters are now false by default.

Performance went from 21.5fps to 27fps!

The output appears to be exactly the same.

[MysteryX]

Further improved performance, especially on the UINT16 conversion. Also fixed a mix-up in HLSL files that might have corrupted the output if you tried it yesterday.

I'm now getting 33fps. It's getting very decent!

By the way, this might also cause color distortion. I don't know if it's using the right color matrix here.
ImageSource("Lighthouse.png").ConvertToYV12()

[luquinhas0021]

MysteryX, you`ve done the test with eedi3 sclip nnedi3 + superres? Both nnedi3 and eedi3 in parameters which gives maximum quality. If yes, stays sharper and retains more details than nnedi3 only or eedi3 only?

[MysteryX]

I had tried EEDI3 a while ago and it didn't give good results. EEDI3 gives a softer image. SuperRes works better with something more sharp. Jinc doesn't work well either.

[sqrt(9801)]

Hi.

I've been trying to convert Super-xBR for use with AvisynthShader, and it looks like there's an issue when using command chains on shaders that take parameters.

Here is my script :

edge_strength = 1
weight = 1
third_pass = true

ColorBars(width=640, height=480, pixel_type="RGB32")

input=ConvertToFloat(precision=3)
input

Shader(path="Shaders/Super-xBR/super-xbr-pass0.cso",\
clip1=1,output=1,\
param1="args0="+string(edge_strength,"%.32f")+","+string(weight,"%.32f")+"0,0f",\
param2="size0="+string(width/2)+","+string(height)+","+string(2./width,"%.32f")+","+string(1./height,"%.32f")+"f",\
height=width,width=height*2)

Shader(path="Shaders/Super-xBR/super-xbr-pass1.cso",\
clip1=1,output=1,\
param1="args0="+string(edge_strength,"%f")+","+string(weight,"%f")+"0,0f",\
param2="size0="+string(width/2)+","+string(height)+","+string(2./width,"%.32f")+","+string(1./height,"%.32f")+"f")

third_pass ? Shader(path="Shaders/Super-xBR/super-xbr-pass2.cso",\
clip1=1,output=1,\
param1="args0="+string(edge_strength,"%f")+","+string(weight,"%f")+"0,0f",\
param2="size0="+string(width/2)+","+string(height)+","+string(2./width,"%.32f")+","+string(1./height,"%.32f")+"f") : nop

last.ExecuteShader(input,precision=3)

ConvertFromFloat(format="RGB32",precision=3)

First, you'll notice that the width/height parameters taken by the Shader function are actually reversed. Second, here's the output :

http://abload.de/thumb/chaino9s3d.png (http://abload.de/image.php?img=chaino9s3d.png)

Outputting the results of the first two passes to a new clip makes thing worse, since it seems to ignore the width/height parameters specified in the first command.

Shader(path="Shaders/Super-xBR/super-xbr-pass0.cso",\
clip1=1,output=2,\
param1="args0="+string(edge_strength,"%.32f")+","+string(weight,"%.32f")+"0,0f",\
param2="size0="+string(width/2)+","+string(height)+","+string(2./width,"%.32f")+","+string(1./height,"%.32f")+"f",\
height=width,width=height*2)

Shader(path="Shaders/Super-xBR/super-xbr-pass1.cso",\
clip1=2,output=3,\
param1="args0="+string(edge_strength,"%f")+","+string(weight,"%f")+"0,0f",\
param2="size0="+string(width/2)+","+string(height)+","+string(2./width,"%.32f")+","+string(1./height,"%.32f")+"f")

third_pass ? Shader(path="Shaders/Super-xBR/super-xbr-pass2.cso",\
clip1=3,output=1,\
param1="args0="+string(edge_strength,"%f")+","+string(weight,"%f")+"0,0f",\
param2="size0="+string(width/2)+","+string(height)+","+string(2./width,"%.32f")+","+string(1./height,"%.32f")+"f") : nop

Output :
http://abload.de/thumb/chain2rosmj.png (http://abload.de/image.php?img=chain2rosmj.png)

If I execute the shaders one by one, like so :

input=ConvertToFloat(precision=3)
input

Shader(path="Shaders/Super-xBR/super-xbr-pass0.cso",\
clip1=1,output=1,\
param1="args0="+string(edge_strength,"%.32f")+","+string(weight,"%.32f")+"0,0f",\
param2="size0="+string(width/2)+","+string(height)+","+string(2./width,"%.32f")+","+string(1./height,"%.32f")+"f",\
height=width,width=height*2)

input=last.ExecuteShader(input,precision=3)

input

Shader(path="Shaders/Super-xBR/super-xbr-pass1.cso",\
clip1=1,output=1,\
param1="args0="+string(edge_strength,"%f")+","+string(weight,"%f")+"0,0f",\
param2="size0="+string(width/2)+","+string(height)+","+string(2./width,"%.32f")+","+string(1./height,"%.32f")+"f")

input=last.ExecuteShader(input,precision=3)

input

third_pass ? Shader(path="Shaders/Super-xBR/super-xbr-pass2.cso",\
clip1=1,output=1,\
param1="args0="+string(edge_strength,"%f")+","+string(weight,"%f")+"0,0f",\
param2="size0="+string(width/2)+","+string(height)+","+string(2./width,"%.32f")+","+string(1./height,"%.32f")+"f") : nop

third_pass ? last.ExecuteShader(input,precision=3) : nop

ConvertFromFloat(format="RGB32",precision=3)

I get this (same output as Super-xBR in MPDN, as far as I can tell) :
http://abload.de/thumb/not-chainbssao.png (http://abload.de/image.php?img=not-chainbssao.png)

Sure, it works, but the increased VRAM usage is not negligible.
Now, if I hardcode the shaders' parameters into the HLSL files and chain the resulting shaders :

input=ConvertToFloat(precision=3)
input

Shader(path="Super-xBR/super-xbr-hardcoded-pass0.cso",\
clip1=1,output=1,\
height=width,width=height*2)

Shader(path="Super-xBR/super-xbr-hardcoded-pass1.cso",\
clip1=1,output=1)

third_pass ? Shader(path="Super-xBR/super-xbr-hardcoded-pass2.cso",\
clip1=1,output=1) : nop

last.ExecuteShader(input,precision=3)

ConvertFromFloat(format="RGB32",precision=3)

I get exactly the same output (verified with ImageMagick's compare function), so I won't link another image. ;P


TL;DR :
1) There's an issue when using command chains on shaders that take parameters.
2) The width and height parameters taken by the Shader function are reversed.
3) The width and height parameters taken by the Shader function are ignored when not outputting to clip #1.

[MysteryX]

Keep in mind that by default it's not converting pixel type from YUV to RGB anymore, but you're using RGB as the image source so that's not an issue. If you do need to convert, the recommended approach is to either
A) Chain a call to YuvToGamma with other shader calls (performance cost)
or even better
B) Include a call to YuvToGamma within the first Shader being called (no extra overhead)

As for precision, you can use 3 (half-float) for testing and development but this code path is considerably slower. Once it's working, Precision=2 (default) is recommended unless you need to have values that overflow the 0-1 range.

I was told by Shiandow that in HLSL, it's difficult to get some code to *not* execute, so if parts of code are optional, he'd rather compile a different version of the file (such as the final pass of SuperRes).

As for the parameters, you're right: Width and Height were reversed. This is now fixed.

As for the rest, could you send me your files so that I can reproduce the issues locally? Send me both the files that work and the files that don't work.

[sqrt(9801)]

Sure thing. CSO and HLSL files should be attached to the post (it's just a lazy hack of MPDN's Super-XBR code). Hardcoded versions is meant for 2X upscaling 640x480 clips.


Additionally, I think I've figured out what's going on with the shader parameters.

I made a simple shader, which takes a single float value as input and multiplies every pixel by it.

sampler s0 : register(s0);
float mul : register(c0);

float4 main(float2 tex : TEXCOORD0) : COLOR0
{
return mul*tex2D(s0,tex);
}


And here's my test script :
ColorBars(width=640, height=480, pixel_type="RGB32")

ConvertToFloat()
input=last

Shader(path="Shaders/test.cso",\
clip1=1,output=1,\
param1="mul=.5f")

last.ExecuteShader(input)

ConvertFromFloat(format="RGB32")

And here's the output :
http://abload.de/thumb/test13pybx.png (http://abload.de/image.php?img=test13pybx.png)


Now if I chain the shader twice with different values for mul :
Shader(path="Shaders/test.cso",\
clip1=1,output=1,\
param1="mul=1f")

Shader(path="Shaders/test.cso",\
clip1=1,output=1,\
param1="mul=.5f")

It looks like mul is set to 0.5 for the entire chain :
http://abload.de/thumb/test2atzco.png (http://abload.de/image.php?img=test2atzco.png)


So I compiled the same shader with "register(c0)" replaced with "register(c1)."

# c0
Shader(path="Shaders/test.cso",\
clip1=1,output=1,\
param1="mul=1f")

# c1
Shader(path="Shaders/test2.cso",\
clip1=1,output=1,\
param1="mul=.5f")

Which gives me the correct result.

So I'm guessing that basically, a register is set to the last value entered in the chain... or something like that. Bummer.

[MysteryX]

Ah... got it. The ConstantTable containing parameters isn't switching properly when setting the pixel shader...

[MysteryX]

Not yet sure what's the right way of solving this, but I made a temporary fix. It's now working, although the design and performance may not be optimal. For now I'm re-parsing and re-setting all parameters on every call.

[sqrt(9801)]

Doesn't seem to work on my end, I'm not sure why.

Anyway, I've attached a shader for bicubic resampling since you've been looking for one. Since this my first time coding anything more complex than an inversion shader in HLSL, I may (read "must") have gotten some things wrong. :p

Example of use :
input_width = 1600
input_height = 1024

output_width = 640
output_height = 180

ColorBars(width=input_width, height=input_height, pixel_type="RGB32")
Subtitle("Réfrigérateur !", align=1, size=80)

source=last

Shader(path="bicubic.test.cso",\
param1="out_size=" + string(output_width) + "," + string(output_height) + "," + string(1./output_width,"%.32f") + "," + string(1./output_height,"%.32f") + "f",\
param2="in_size=" + string(input_width) + "," + string(input_height) + "," + string(1./input_width,"%.32f") + "," + string(1./input_height,"%.32f") + "f",\
param3="BC=" + string(1/3.,"%.32f") + "," + string(1/3.,"%.32f") + "f",\
width=output_width,height=output_height)

last.ExecuteShader(source.ConvertToFloat())

ConvertFromFloat(format="RGB32")

Compare(source.BicubicResize(640, 180, b=1./3., c=1./3))

Output :
http://abload.de/img/testwduvz.png (http://abload.de/image.php?img=testwduvz.png)


This should theoretically enable running multiple SuperRes passes in a single command chain, among other things, but I keep running into that issue with shader parameters. orz

[MysteryX]

Thanks! Are you sure your Bicubic downscaling is working? You may contact Shiandow if you need help with the specifics of the implementation. Or he might review your final code to ensure it's designed right.

Heck, better as well produce a shader that works both for upscaling or downscaling.

Once you're sure the Bicubic downscaling is working properly on its own, I can integrate it into the chain and test/debug/benchmark it at the same time. I also want to see whether there is a noticeable quality difference.

For handling color spaces, is there a AVS function to get the current color space? Right now ConvertToFloat supports YV24 and RGB32, while SuperRes only supports YV24. "SuperResDiffCpu.cso" could be renamed "SuperResDiffRgb.cso" to process RGB clips that require no YUV conversion. Then within SuperRes.avsi, I run the regular shaders if the source is YV24, and the 2nd version if the source is RGB32. But how do I get the source format?

As for all other formats, they are also supported if you just call ConvertToYV24() or ConvertToRGB32() before SuperRes.

By the way, whenever you see a large image like this, keep in mind that once converted to 16-bit-per-channel, Clip.Width is twice bigger than it should be so it needs to be divided by 2.

[sqrt(9801)]

Upscaling/downscaling works in the sense that I don't get any visible difference compared to Avisynth's internal bicubic resizer in my tests. PSNR is usually around 60 dB, which is okay, I guess ? Seriously though, this is probably due to the higher precision, and the way the shader treats borders. I think Avisynth also resizes in two passes ??

I've also noticed some difference when using it in SuperRes. The current version using the internal bicubic filter actually produces incorrect color on some parts of the ColorBars clip, probably because the clip is converted to linear RGB (via the GammeToLinear shader) beforehand. That issue doesn't appear when using the bicubic shader.

As for getting a clip's colorspace from a AVS script, you have the IsRGB24/IsRGB32/IsYV12/IsYV24 etc. functions. They should get the job done.

[MysteryX]

I tried your Bicubic. First, better call the main function "main" so that we don't need any special arguments to compile.

Then, with your CSO file, setting the arguments failed. Not sure why. I tried compiling the file myself to know I'm working with the right thing.

If I type in this command, then it loops indefinitely and never compiles.
"C:\Program Files (x86)\Microsoft DirectX SDK (June 2010)\Utilities\bin\x86\fxc.exe" /T ps_3_0 /Fo "Bicubic.cso" "src\SuperRes\Bicubic.hlsl"

I think something is wrong.

[sqrt(9801)]

I'm not sure why the CSO doesn't run. Have you tried with the sample AVS script I posted earlier ?

Compilation takes longer than usual because the loops can't be unrolled. It takes ~90 seconds to compile on my end with the same command (more or less, I just have an additional /O3 switch.)

[MysteryX]

OK I got it working. I also fixed the issue with shader parameters. This fix may have a slight performance cost for now but at least it's finally working.

As expected, running all shaders in a chain with HLSL Bicubic downscaling does improve the quality very slightly. With MT=8, memory usage went down from 921MB to 688MB which is considerable. Performance is slightly better with CPU usage considerably lower.

I have also added support for YV24 and RGB32 in SuperRes.avsi. However, I can't test it with RGB32 because NNEDI3 doesn't support RGB32!

Could you send me your Super-xBR code when you got it fully working? I'll add it to the project.

[MysteryX]

In practice, the benefits are greater. In a more complex script that calls SuperRes twice combined with a bunch of other processing, memory usage went down from 1700MB to 1000MB, and performance went up from 5fps to 7fps!

[MysteryX]

I have improved the handling of shader parameters to fix a few issues
- Improved performance by avoiding to parsing string values on every call
- Invalid values now return an error
- INT vectors now supported
- Default values specified within the shader should now be respected

**BREAKING CHANGE**
The format of parameters has changed. Instead of
Param1="input=640,480f"
it will now be
Param1="640,480f"

Parameters are now configured by index only, we don't need to pass the names anymore.

I'm still hesitating whether the first param should be called Param1 or Param0... and also for clips.

[StainlessS]

For handling color spaces, is there a AVS function to get the current color space? Right now ConvertToFloat supports YV24 and RGB32, while SuperRes only supports YV24. "SuperResDiffCpu.cso" could be renamed "SuperResDiffRgb.cso" to process RGB clips that require no YUV conversion. Then within SuperRes.avsi, I run the regular shaders if the source is YV24, and the 2nd version if the source is RGB32. But how do I get the source format?


Here something that might assist (OK in both v2.6 and v2.58).

Function ColorSpaceIx(clip c) {
cs=c.IsRGB24?0:c.IsRGB32?1:c.IsYUY2?2:c.IsYV12?3:-1
return(cs>=0||VersionNumber<2.6)?cs:c.IsY8?4:c.IsYV16?5:c.IsYV24?6:c.IsYV411?7:-1
}


EDIT: Above is for script, which is what I thought you meant.
If was meaning C/CPP then you will have more problems unless providing both v2.5 and v2.6 plugins.

[sqrt(9801)]

"Parameters are now configured by index only"

What does that mean exactly ? Param1 sets c0, param2 sets c1, etc... ?

[MysteryX]

It's already working for RGB colorspace.

"Parameters are now configured by index only"

What does that mean exactly ? Param1 sets c0, param2 sets c1, etc... ?
Yes

I have just added support to convert to/from RGB24. Now we can test NNEDI3 and SuperRes without YUV conversion.

[MysteryX]

Here are some image tests in RGB24, without YUV conversion.

ImageSource("Lighthouse.png").ConvertToRGB24()

1. Spline16Resize(Width*4, Height*4)
2. edi_rpow2(2, nns=4, cshift="Spline16Resize", threads=2), twice
3. edi_rpow2(4, nns=4, cshift="Spline16Resize", threads=2)
4. SuperRes(2, .43, 0, """edi_rpow2(2, nns=4, cshift="Spline16Resize", threads=2)""", PluginPath), twice
5. SuperRes(2, .43, 0, """edi_rpow2(4, nns=4, cshift="Spline16Resize", threads=2)""", PluginPath)

http://s21.postimg.org/mwcg0d91v/Lighthouse_Spline16.png (http://postimg.org/image/mwcg0d91v/) http://s21.postimg.org/pg82ugwlv/Lighthouse_NNEDI3_2.png (http://postimg.org/image/pg82ugwlv/) http://s21.postimg.org/xbim9a68j/Lighthouse_NNEDI3_4.png (http://postimg.org/image/xbim9a68j/) http://s7.postimg.org/hfs3ar1lj/Lighthouse_Super_Res_2.png (http://postimg.org/image/hfs3ar1lj/) http://s7.postimg.org/bqbuqfvfb/Lighthouse_Super_Res_4.png (http://postimg.org/image/bqbuqfvfb/)

EDIT: That's wrong, SuperRes doesn't appear to be running!

[sqrt(9801)]

Yeah, it looks like that last update broke something. Shaders don't seem to do anything at all right now.

[MysteryX]

OK it's fixed and I updated the images.

**Breaking Change**
You now set C0 with Param0 and C1 with Param1, instead of C0 with Param1 and C1 with Param2. For clips, it's still S0=Clip1, S1=Clip2.

The SuperRes image is considerably darker than the NNEDI3 version, not sure why. If the pixel conversion would slightly darken, then there would be a slight color tone between running it once or twice, which is not the case.

[Suicycle]

hello everyone. this looks like a great app. thanks for the hard work and sharing it.

I am getting this error every time I try to launch the codes below and can't figure out what is wrong.

Error line :
ConvertFromFloat : Source must be float-precision RGB

These are the codes I am using. (got them from this thread) I am not a programmer. I don't know anything about coding. So all I am doing is to copy paste the codes I find on the internet (mostly in this forum). But I have checked out the basic usage of Avisynth for beginners.

I have Avisynth 2.6 installed with VirtualDub 1.10.4
I am on a win 8.1 64bit pc but avisynth and virtualdub are 32 bits.
I have tried the same codes with other avisynth versions. I always got an error (not the same error tho)

LoadPlugin("C:\Program Files\AviSynth\Plugins\nnedi3.dll")
LoadPlugin("C:\Program Files\AviSynth\Plugins\Shader.dll")
Import("Shaders\SuperRes\SuperRes.avsi")
SetMTMode(3,4)

ImageSource("1.png").ConvertToYV12()

SetMTMode(2)

SuperRes(2, 1, 0, false, """nnedi3_rpow2(rfactor=4, cshift="Spline16Resize", Threads=1)""" )

Spline64Resize(2560,1380)


I get the same error even if I take out the .ConvertToYV12() code. I have tried it with jpgs, tifs, pngs etc. I get the error every time. What should I do? any ideas?

[sqrt(9801)]

ImageSource outputs to RGB24 by default. Try with ConvertToRGB32 or use ImageSource("1.png", pixel_type = "RGB32")

Also, it looks like you're using an outdated version. You should update it, since the new version supports more colorspaces.

[MysteryX]

Don't forget to pass the "folder" parameter to SuperRes so it knows where to find the CSO files. Also make sure you have the latest DLL as I have been updating it many times.

[Suicycle]

Thank you, it works now. I updated all the files, added the file path and then realized there was something extra in the codes. I removed it and it worked.

[MysteryX]

sqrt(9801), do you have Super-xBR working? If so, can you send it over?

[sqrt(9801)]

Sure thing. A (haphazardly made) AVS function is included.

Btw, it looks like your version of the SuperRes shader is outdated compared to the one currently found in the MPDN_Extensions repo.

[MysteryX]

The SuperRes code I have is Oct 2, there's a Oct 10 versio. You're right.

What is super-xbr-params.inc ?

Could you also include a .BAT file to batch-compile the files?

[MysteryX]

The new version of GPU-Z now detects my GPU usage on my Radeon HD 7670M.

I've done a benchmark test.
SetMTMode(3,8)
AviSource("F:\AVSMeter\Preview.avi", audio=false, pixel_type="YV12")
SetMTMode(2)
SuperRes(2, .43, 0, """nnedi3_rpow2(2, cshift="Spline16Resize", Threads=2)""")
Distributor()

It runs at 34fps, with GPU usage between 60-70%. Increasing threads to 16 doesn't help much; at first it's considerably higher and then it goes back down.

[Groucho2004]

The new version of GPU-Z now detects my GPU usage on my Radeon HD 7670M.

I've done a benchmark test.
SetMTMode(3,8)
AviSource("F:\AVSMeter\Preview.avi", audio=false, pixel_type="YV12")
SetMTMode(2)
SuperRes(2, .43, 0, """nnedi3_rpow2(2, cshift="Spline16Resize", Threads=2)""")
Distributor()

It runs at 34fps, with GPU usage between 60-70%. Increasing threads to 16 doesn't help much; at first it's considerably higher and then it goes back down.
You do remember what the "Distributor()" call does (http://forum.doom9.org/showthread.php?p=1728771#post1728771), right?

[sqrt(9801)]

super-xbr-params.inc is just part of the super-xbr.hlsl file (you can see it has an #include "super-xbr-params.inc" directive.) Not sure why Hyllian felt the need to make a separate file just for the shader's parameters.

As for batch compilation, this is the command I've used.
fxc.exe /T ps_3_0 /O3 /E main_fragment /D Pass=0 /Fo super-xbr-pass0.cso super-xbr.hlsl

fxc.exe /T ps_3_0 /O3 /E main_fragment /D Pass=1 /Fo super-xbr-pass1.cso super-xbr.hlsl

fxc.exe /T ps_3_0 /O3 /E main_fragment /D Pass=2 /Fo super-xbr-pass2.cso super-xbr.hlsl

[MysteryX]

Super-xBR is working. Here are some tests of image quadrupling

NNEDI3
Super-xBR (default settings)
NNEDI3 + SuperRes
Super-xBR + SuperRes
Super-xBR(edgeStrength=.6, weight=.6)
Super-xBR(edgeStrength=.6, weight=.6) + SuperRes

http://s9.postimg.org/e0sq5p41n/Clown_NNEDI3.png (http://postimg.org/image/e0sq5p41n/) http://s9.postimg.org/vteadkla3/Clown_Super_XBR.png (http://postimg.org/image/vteadkla3/) http://s9.postimg.org/44rn61y9n/Clown_Super_Res.png (http://postimg.org/image/44rn61y9n/) http://s9.postimg.org/e4mjly9iz/Clown_Super_Res_XBR.png (http://postimg.org/image/e4mjly9iz/) http://s1.postimg.org/qi5wic04b/XBR_6.png (http://postimg.org/image/qi5wic04b/) http://s1.postimg.org/6z1b8z1cr/Super_Res_XBR_6.png (http://postimg.org/image/6z1b8z1cr/)


Way too much ringing. So far NNEDI3 + SuperRes is what works best.

Edit: added Super-xBR with edgeSrength=.6 and weight=.6

SuperRes reduces Super-xBR's halo, and this video is almost as good as NNEDI3+SuperRes

[MysteryX]

I have released Super-xBR and updated SuperRes AVS script. I also applied some updates to the SuperRes HLSL code.

Right now for Super-xBR I'm not doing YUV-RGB conversion... and it looks pretty much the same in YUV or RGB.

[MysteryX]

sqrt(9801), if you got some time, perhaps you could look into doing the same with NEDI?
https://github.com/zachsaw/MPDN_Extensions/blob/master/Extensions/RenderScripts/Shiandow.Nedi.NediScaler.cs

[MysteryX]

You do remember what the "Distributor()" call does (http://forum.doom9.org/showthread.php?p=1728771#post1728771), right?
Removing Distributor() does reduce the amount of threads and considerably reduces CPU usage, although performance is about the same overall. Thanks