Log in

View Full Version : AviSynthShader + SuperRes


Pages : 1 2 [3] 4 5 6 7 8 9 10 11 12 13

MysteryX
15th November 2015, 06:54
I fixed the slight darkening issue.

Conversion to UINT16 gave values between 0 and 255*256=65280. Add 128 to avoid darkening when converting back.

sqrt(9801)
15th November 2015, 15:19
Why not just multiply the UINT8 value by 257 ?

As for NEDI, I'll check it out. Thanks for the suggestion.

MysteryX
15th November 2015, 17:50
Why not just multiply the UINT8 value by 257 ?
Interesting suggestion. Multiplying by 257 and dividing by 256 would produce "valid" results but would cause a subtle shift in colors.

Suicycle
15th November 2015, 18:07
Hello again, guys I have a question. I hope this is not out of topic but say that I have a 1080p footage which is nicely shot with a DSLR.

Even though the footage has no quality problem, when I upload it to Youtube, the quality will be reduced automatically.

So what I am planning to do is that I will upscale the 1080p footage (say by 2x) by using SuperRes and then downscale it back to 1080p using one of the good downscale functions (blackmanresize maybe?). Since SuperRes will enhance the quality when it upscales, it will look better when downscaled. And then I'll upload it to Youtube like that.

Does this make sense? Is my logic right?

MysteryX
15th November 2015, 20:10
Your output when you upload it to YouTube will still be 1080p. What benefit will you get? YouTube won't reduce your resolution, but it will greatly reduce the bitrate.

The only way for YouTube to use a higher bitrate is to upload it in higher resolution, such as in 2160 resolution. In which case, it will re-encode using a higher bitrate.

Suicycle
15th November 2015, 21:53
OK I got it, since it's only about bit rate, then downscaling won't change anything. Thank you.

MysteryX
15th November 2015, 23:42
I tested with the updated SuperRes code and with the HLSL Bicubic code. Quality is *considerably* better. The shape of objects is much more defined than before. Not sure if it's due to the Bicubic code or due to the SuperRes core update.

Edit: Interesting... in my script, I'm running InterFrame after SuperRes, and the update is causing InterFrame to generate more accurate frames. I didn't expect this update to have so much impact. In theory, the new encoded video (after all my processing) should be about the same size, and in practice, the new video is 87.3MB instead of 85.4MB with fixed quality encoding, because there are more details to encode. That's a big difference!

The readme file will need to be considerably updated.

MysteryX
16th November 2015, 03:29
Version 1.0 released.

luigizaninoni
18th November 2015, 13:05
great plugin, still slowish with intel HD4600, so no hope of real time, but results are great.

I guess that porting to Vapoursynth and/or to Avisynth+ 64bits would not really help, since the bottleneck is the graphic card, isn't it ?

MysteryX
18th November 2015, 14:59
You can use SuperRes for real-time playback with madVR, but both NNEDI3 and SuperRes are very expensive so don't count on it unless you have a top-end graphic card!

SuperRes makes even Bicubic upscaling look decent. If you upscale with Bicubic and run a single pass, then you'll get much higher performance.

That being said, with my Radeon HD 7670M, I can't get GPU usage to go above 60-70%. Not exactly sure where the bottleneck is.

luquinhas0021
19th November 2015, 21:25
MysteryX, is the superresolution plugin you are discussing here able to take off the artifacts which are introduced by some upscaling algorithm? Example: bicubic -1, 1 is sharpest, but generates a lot of aliasing. If I use this same type of bicubic, with your superresolution plugin, will result image be free of aliasing, or, at least, with less aliasing compared with result image of only bicubic?

MysteryX
20th November 2015, 01:03
MysteryX, is the superresolution plugin you are discussing here able to take off the artifacts which are introduced by some upscaling algorithm? Example: bicubic -1, 1 is sharpest, but generates a lot of aliasing. If I use this same type of bicubic, with your superresolution plugin, will result image be free of aliasing, or, at least, with less aliasing compared with result image of only bicubic?

It does appear so. That's the case with Super-xBR. It generates very sharp images but massive ringing, and applying SuperRes on top of it reduces the ringing and makes it very decent. Experiment with it.

jinkazuya
23rd November 2015, 06:02
I also have some problem using this script too. According to the error message, it is stated that "There is no function named ConvertToFloat". I checked the script, there is no such a function but there is such a function call within the code. Besides there is also no function named ConvertFromFloat either.

StainlessS
23rd November 2015, 16:44
jinkazuya
You will also need to say where the relevant plugins are, and if you are audoloading or explicitly loading (in your script).
You best at least provide your script so that MysteryX can see what you are doing wrong.
It sounds like the plugin is not being loaded.

MysteryX
24th November 2015, 07:42
I also have some problem using this script too. According to the error message, it is stated that "There is no function named ConvertToFloat". I checked the script, there is no such a function but there is such a function call within the code. Besides there is also no function named ConvertFromFloat either.
You're not currently loading Shader.dll

MysteryX
24th November 2015, 23:20
Version 1.1 released.

What's new:
- Shader will now attempt open relative paths in the same folder as the DLL.
- Folder argument removed from SuperRes and Super-xBR. CSO files must be in the same folder as Shader.dll
- Added ColorMatrix.avsi for high bit dept color matrix conversion to avoid banding
- SuperRes now has srcMatrix601 argument allowing to convert color matrix while running SuperRes with no performance cost.

https://github.com/mysteryx93/AviSynthShader/releases/tag/v1.1

SSH4
25th November 2015, 09:24
Am i only one here who think that original NNEDI 2x/4x looks MUCH better in any case than SuperRez(NNEDI,EEDI,Makemehappy,pass=100) ?

Groucho2004
25th November 2015, 09:29
Am i only one here who think that original NNEDI 2x/4x looks MUCH better in any case than SuperRez(NNEDI,EEDI,Makemehappy,pass=100) ?You may be the only one who bothered to compare them. :D

SSH4
25th November 2015, 09:50
You may be the only one who bothered to compare them. :D

Ah, my mistake. This is public diary? ;)

But serious.
SuperRes version of Clown picture looks like billinear upscaled awarpsharped and blurred after this picture.

"In the Nightside Eclipse" upscaled cover looks good because original source have uniform noise from scanned offset print. And such median/awarpsharp filters can help "remove" this noise without losing much details.

Groucho2004
25th November 2015, 10:26
SuperRes version of Clown picture looks like billinear upscaled awarpsharped and blurred after this picture.

"In the Nightside Eclipse" upscaled cover looks good because original source have uniform noise from scanned offset print. And such median/awarpsharp filters can help "remove" this noise without losing much details.
I assumed that you compared the scalers with real footage, not static images. So, nothing new really...

SSH4
25th November 2015, 11:16
I assumed that you compared the scalers with real footage, not static images. So, nothing new really...

Spatial "filter" have better result in "real" video footage than on static frame?

Idea of diff between NNEDI3 and billinear scale have rights to live. At least for creating mask for problematic part of image that you can refine. But "real" mpeg encoded footage will produce non usable mask in this method, because of noise and encoding blocks.
So faster and better just create edge mask with masktools and "smart" sharp edges on image.
Oops i just reinvented Deede's scripts for sharpen ;)


Anyway, sorry. I just asked for opinion about quality in results of this "SuperRes".

MysteryX
25th November 2015, 17:01
Spatial "filter" have better result in "real" video footage than on static frame?

Idea of diff between NNEDI3 and billinear scale have rights to live. At least for creating mask for problematic part of image that you can refine. But "real" mpeg encoded footage will produce non usable mask in this method, because of noise and encoding blocks.
So faster and better just create edge mask with masktools and "smart" sharp edges on image.
Oops i just reinvented Deede's scripts for sharpen ;)


Anyway, sorry. I just asked for opinion about quality in results of this "SuperRes".
It's actually working very well on noisy videos. NNEDI3 cannot be applied for noisy MPEG sources, what worked was EEDI3 followed by NNEDI3. With SuperRes, I'm able to do it only with NNEDI3.

MysteryX
25th November 2015, 21:10
Here's a sample noisy 288p MPEG video re-encoded with this script.

PluginPath=""
LoadPlugin(PluginPath+"Shader.dll")
LoadPlugin(PluginPath+"KNLMeansCL.dll")
LoadPlugin(PluginPath+"nnedi3.dll")
Import(PluginPath+"edi_rpow2.avsi")
Import(PluginPath+"ResizeX.avsi")
Import(PluginPath+"SuperRes.avsi")
LoadPlugin(PluginPath+"svpflow1.dll")
LoadPlugin(PluginPath+"svpflow2.dll")
Import(PluginPath+"InterFrame2.avsi")

SetMTMode(3,8)
AviSource("Preview.avi", audio=true, pixel_type="YV12")
SetMTMode(2)
Crop(0, 0, -8, -0)
SetMTMode(5)
KNLMeansCL(D=2, A=1, h=3, device_type="GPU")
SetMTMode(2)
SuperRes(2, 0.43, 0, """edi_rpow2(2, nns=4, cshift="Spline16Resize", Threads=2)""", srcMatrix601 = true)
InterFrame(Cores=8, Tuning="Smooth", NewNum=60000, NewDen=1001, GPU=true)
SuperRes(2, 0.43, 0, """edi_rpow2(2, nns=4, cshift="Spline36Resize", fwidth=944, fheight=724, Threads=2)""")
Spline36Resize(940, 720, 0, 4, -4, -0)

Original video (https://www.spiritualselftransformation.com/files/media-encoder-old.mpg)

New video (https://www.spiritualselftransformation.com/files/media-encoder-new2.mkv)

Previously, the best approach was to use EEDI3+NNEDI3 which gave a more blurry result. Afterwards, I improved with 2-3 passes of fine-tuned sharpening at various stages of the scaling. The SuperRes version looks better and I can go without EEDI3 and without sharpening.

SSH4
26th November 2015, 03:12
It's actually working very well on noisy videos. NNEDI3 cannot be applied for noisy MPEG sources, what worked was EEDI3 followed by NNEDI3. With SuperRes, I'm able to do it only with NNEDI3.

NNEDI3 Can and Work perfectly with noisy MPEG sources. Just check some great upscales made by anime release groups.

But "Noise MPEG Source" + NNEDI3 != "perfect upscale".

You should divide work for steps.

Analyzing source and understanding problems it have.

Healing noised, blocked mpeg source (if you can't find/buy better quality source for you work).
Usually in this step you will use temporal denoisers with MVDegrain/Dfttest/etc with motion compensation with detail protection with masks etc.

After healing source you can upscale with NNEDI.

After this you can sharpen or refine upscaled picture.
And in final you can add some noise back for hide some upscale/deblocking errors and for better encoding.

Just check Example scripts from MVTools2, dfttest.

Sometime you need use different methods for different part of source video. For video - one method, for hardcoded subtitle another. And merge results in final step. But not one bullet for all.


Hmm, just checked on your source. dfttest(sigma=30) and nnedi3 give better result than your SuperRes 8)

foxyshadis
26th November 2015, 10:03
Hmm, just checked on your source. dfttest(sigma=30) and nnedi3 give better result than your SuperRes 8)

If you're going to claim something's better, at least post a screenshot comparison so we can judge too.

MysteryX
26th November 2015, 16:22
If you're going to claim something's better, at least post a screenshot comparison so we can judge too.
I agree. There are standard protocols to follow on this forum. If asking for help on a video, post that video. If claiming better results, post exact script and screenshots to compare. Standard stuff you should know.

I've previously tried working with any EEDI3/NNEDI3 combination, KNLMeans and FF3DFilter denoisers. NNEDI3 was too sharp for that source. The result of Denoise+EEDI3+NNEDI3 was fine... except that when playing the unprocessed video with madVR with SuperRes activated, I couldn't say for sure whether the original video or the processed video looked better. Perhaps my processed video looked 'slightly' better, but the one passing through madVR with SuperRes was definitely sharper. Now the new processed videos are definitely better.

If you find something that works better, I'd love to see it!

MysteryX
26th November 2015, 18:58
v1.2 released (https://github.com/mysteryx93/AviSynthShader/releases/tag/v1.2)

What's new:
- Added precisionIn and precisionOut arguments to ExecuteShader, allowing converting data on the GPU
- SuperRes, Super-sBR and ColorMatrix scripts adapted to convert with precision=1 while doing the processing with precision=2
- Removed D3DCREATE_DISABLE_PSGP_THREADING flag from DirectX9 device
- Performance is similar, memory usage is slightly lower and CPU usage is considerably lower

Here's a benchmark comparison while running SuperRes
SuperRes(2, .42, 0, """nnedi3_rpow2(2, cshift="Spline16Resize", Threads=2)""")

Before
FPS (min | max | average): 1.882 | 1000000 | 28.72
Memory usage (phys | virt): 598 | 679 MB
Thread count: 158
CPU usage (average): 57%

With data conversion on the GPU
FPS (min | max | average): 1.778 | 1000000 | 27.33
Memory usage (phys | virt): 590 | 662 MB
Thread count: 158
CPU usage (average): 51%

Without the D3DCREATE_DISABLE_PSGP_THREADING flag
FPS (min | max | average): 1.882 | 1000000 | 27.38
Memory usage (phys | virt): 595 | 666 MB
Thread count: 166
CPU usage (average): 45%

The performance isn't better. In fact it is 'slightly' slower. However, memory usage is slightly lower (679 to 666), and CPU usage is considerably lower (57% to 45%).

EDIT: This update now makes it possible to do the processing with half-float data (precision=3) since the conversion doesn't need to be done on the CPU. This results in higher performance. This change will be in the next release, and for now you can apply this fix by replacing precision=2 with precision=3 in SuperRes.avsi on ExecuteShader.

FPS (min | max | average): 2.667 | 1000000 | 29.09
Memory usage (phys | virt): 594 | 666 MB
Thread count: 169
CPU usage (average): 45%

SSH4
27th November 2015, 03:05
If you're going to claim something's better, at least post a screenshot comparison so we can judge too.
really strange seen this from such skilled doom9 member. I thought you better than me understand difference between any kind spatial "magic" and temporal denoisers.

Ok.

I did not preprocess source or postprocess results.

http://s14.postimg.org/rjpcn8359/New_File_3_002340.png (http://postimg.org/image/rjpcn8359/) http://s14.postimg.org/epratvppp/New_File_3_002341.png (http://postimg.org/image/epratvppp/)

http://s14.postimg.org/uzhgws0dp/New_File_3_006180.png (http://postimg.org/image/uzhgws0dp/) http://s14.postimg.org/ftbf5uact/New_File_3_006181.png (http://postimg.org/image/ftbf5uact/)

Both results are weird. But at least dfttest(sigma=30) don't have so much weird mpeg blocks. And this is only simple sigma=30. dfttest can be adjusted in spatial and temporal parts use noise sample etc., or if you need better results can use with motion compensation.


UPD:
Hmm. I did not seen any words about realtime upscaling.
But if SuperRes is realtime upscaler all my question are useless.

bxyhxyh
27th November 2015, 07:25
That superpupermagicshaderscript actually looks better for me.

When I doing 720p DVD upscales on animated source, I prefer nnedi3+"debilinear" combination.
DVD animations 2x upscaled by nnedi3 looks like it is upscaled from 720p source using bilinear.
And Debilinear helps to reduce "blown up" effect of them. Because it inverts blown up effect caused by Bilinear.
But SuperRes seems widen the lines.

For live action sources, nnedi3 is not really good choice to upscale for me. So I choose better one by comparing them.
Actually I don't even upscale them.

This is a comparison
DGDecode_mpeg2source("VTS_01_1.d2v", info=3)
#SuperRes(2,upscalecommand="""nnedi3_rpow2(2,nns=4,cshift="spline36resize",fwidth=960,fheight=720)""")
#nnedi3_rpow2(2,nns=4).Dither_convert_8_to_16().Dither_resize16(960,720,-0.5,-0.5,kernel="bilinear",invks=true).Ditherpost(mode=6)
#Spline36Resize(960,720)
source - https://dl.dropboxusercontent.com/u/58215671/Other/source.png
Spline36 - https://dl.dropboxusercontent.com/u/58215671/Other/spline36.png
nnedi3+deb - https://dl.dropboxusercontent.com/u/58215671/Other/nnedi%2Bdeb.png
SuperRes - https://dl.dropboxusercontent.com/u/58215671/Other/superres.png

SSH4
27th November 2015, 10:42
Well if you prefer dark artifacts on edges. Than SuperRes is right choice!

http://s1.postimg.org/p9yczusiz/image.png (http://postimg.org/image/p9yczusiz/)http://s1.postimg.org/41koife23/image.png (http://postimg.org/image/41koife23/)
Spline36 SuperRes

I tried upscale non-animation source, and eedi+nnedi is probably best for this. But for Anime NNEDI still unbeatable.

That superpupermagicshaderscript actually looks better for me.
BTW.
This is comparison between simple temporal denoiser + nnedi3 upscale with "BlackBox" pre/post processing shader.
Just add LimitedSharpenFaster or any other wonderful Didée sharpen script to "temporal denoiser + nnedi3" and result will be amazing.

bxyhxyh
27th November 2015, 11:10
Well if you prefer dark artifacts on edges. Than SuperRes is right choice!

I tried upscale non-animation source, and eedi+nnedi is probably best for this. But for Anime NNEDI still unbeatable.


BTW.
This is comparison between simple temporal denoiser + nnedi3 upscale with "BlackBox" pre/post processing shader.
Just add LimitedSharpenFaster or any other wonderful Didée sharpen script to "temporal denoiser + nnedi3" and result will be amazing.

For your sample pictures, SuperRes actually looks better. Those black artifacts are already in the source and SuperRes strengthened it due to its "line widening" artifact.
Then you destroyed picture details and artifacts with a denoiser when you use nnedi3.
I think you should call superres after you denoise it like you did with nnedi3. Then that would be a fair comparison.

Now you say you'll sharpen it? It's just comparison between resizer/upscalers, you know.

For my one picture sample, nnedi3+debilinear (debilnear is a resizer) definitely looks better. SuperRes widened the lines. I think old versions wasn't like that.
Edit: When I used Debilinear with SuperRes, it have created halo, since SuperRes's result is very sharp.

MysteryX
27th November 2015, 18:03
Both results are weird. But at least dfttest(sigma=30) don't have so much weird mpeg blocks. And this is only simple sigma=30. dfttest can be adjusted in spatial and temporal parts use noise sample etc., or if you need better results can use with motion compensation.
Post the full script you're using for both.

To compare apples with apples, are you using a denoiser in both cases? Here's we're comparing the upscaler, not the denoiser. This does NOT replace the denoiser, it is still required in both cases.

As for blocking, a lot of it "disappears" when converting to AVI with FFMPEG. I don't know why or how, but this mysterious deblocking appears to be doing a better job than when playing with AviSynth deblocking filters.

For the line widening, we could compare with the previous version of the code, as it was updated at some point.

bxyhxyh
27th November 2015, 20:26
About this line widening and creating "black snow" (I don't know what it is called properly) artifact, was it like this in previous versions? I used the very first version once for test purpose and I really don't remember this artifact.

Shiandow
27th November 2015, 20:55
About this line widening and creating "black snow" (I don't know what it is called properly) artifact, was it like this in previous versions? I used the very first version once for test purpose and I really don't remember this artifact.

Well it isn't in the MPDN version (or the MadVR version for that matter), so something seems to be going wrong.

MysteryX
27th November 2015, 20:59
I recompiled the previous version of SuperRes HLSL code, see attached file. This is for when you apply on Rec.709 YUV content. Just replace the 3 CSO files and see what difference you find.

I also asked a few questions to Shiandow and he replied this
There's something wrong with the SuperRes implementation though, the examples in the AviSynthShader thread don't look anything like I've seen before. SuperRes isn't supposed to be *that* different from Spline or NNEDI.

Not sure what to say about that, or what could be different.

I'm also wondering something about the color conversion. The YUV frames in AviSynth, are the levels 0-255 or 16-255? If that's wrong, it could cause distortion in the conversion. Although for the PNG samples in the first post, I'm doing it on RGB so it's not doing any YUV conversion. By default it assumes 0-255 levels.

foxyshadis
27th November 2015, 21:02
really strange seen this from such skilled doom9 member. I thought you better than me understand difference between any kind spatial "magic" and temporal denoisers.

That's why I asked you to back up those claims; different people have wildly different opinions on upscaled images, making a flat "better" inaccurate, and it became instantly obvious that you were comparing with a bad quality source when you posted the pics. The better your input to SuperRes, the better your output. SuperRes is just a replacement for nnedi3, not dfttest+nnedi3.

MysteryX
27th November 2015, 21:12
sqrt(9801), Shiandow also said this about your Bicubic code

That downscaler seems about right.

Although to make it faster it would probably be best to split it into two parts, first scaling horizontally then vertically. And to make it compile faster you could try replacing "[loop]" with "[fastopt]", although you might want to check if this gives the same output.

loneboyz
28th November 2015, 03:28
I'm testing resize from old NTSC DVD source, convert ratio 4:3 fullscreen up ratio 16:9 widescreen 576p
Between two version of SuperRes, ver 1.2 had darker than on the edge although same script

eedi3 + sclip nnedi3:
http://1.t.imgbox.com/l1jHVLC3.jpg (http://imgbox.com/l1jHVLC3)

SuperRes v1.0
http://0.t.imgbox.com/fqEvlmOy.jpg (http://imgbox.com/fqEvlmOy)

SuperRes v1.2
http://7.t.imgbox.com/tEGNZ6ht.jpg (http://imgbox.com/tEGNZ6ht)

MysteryX
28th November 2015, 04:58
Yeah. That's probably due to taking 8-bit input/output and doing the conversion on the GPU in the latest version.

Try editing SuperRes.avsi and play with precisionIn and precisionOut parameters of ExecuteShader (and match the corresponding ConvertToFloat functions)

MysteryX
28th November 2015, 19:18
PrecisionOut=1 works fine, but using PrecisionIn=1 causes that distortion. I have edited the scripts to use PrecisionIn=2 until the bug is corrected.

v1.2.1 released. Same DLLs, only scripts updated.
https://github.com/mysteryx93/AviSynthShader/releases/tag/v1.2.1

loneboyz
29th November 2015, 07:40
@MysteryX, thank for your work!

after tried again, it's seem ok but have little different color of output

bxyhxyh
29th November 2015, 15:39
How different?

Interesting suggestion. Multiplying by 257 and dividing by 256 would produce "valid" results but would cause a subtle shift in colors.
BTW, why divide by 256, if you multiplied by 257?

wonkey_monkey
29th November 2015, 19:18
I think it's because integer division usually (according to the C++ spec) rounds down.

If you started with a value of 0xff (full white), multiplying it by 257 gives you 0xffff (full white). Dividing by 257 again gives you 0xff again, which is fine, but what if you subtracted 1 first, before the division? Then you'd be at 0xfffe, which, after dividing by 257, rounds down to 0xfe (which is way off what you'd expect as an 8-bit approximation of 0xfffe). If you instead divide by 256, the rounding down works in your favour by getting you back to 0xff.

MysteryX
29th November 2015, 20:41
I just did a comparison between my implementation and MPDN. Wow, there's a big difference!

AviSynthShader
http://s20.postimg.org/jjig2f8bd/Clown_Super_Res_Avi_Synth.png (http://postimg.org/image/jjig2f8bd/)

MPDN
http://s20.postimg.org/gdxuc7pp5/Clown_Super_Res_MPDN.png (http://postimg.org/image/gdxuc7pp5/)

I really have no clue where the difference could be coming from!

Edit: The problem is with the HLSL Bicubic downscaler

Here's a screenshot with the HLSL downscaler disabled and doing the downscaling in AviSynth. Almost identical to MPDN
http://s20.postimg.org/sh35zs0rd/Clown_Bicubic.jpg (http://postimg.org/image/sh35zs0rd/)

MysteryX
29th November 2015, 21:44
Version 1.2.2 released
- For SuperRes, HLSL Bicubic downscaler is broken and has been disabled. Downscaling is now done in AviSynth
- There will be a performance hit and it will take a lot more memory, but the image will be good. Until HLSL Bicubic is fixed.

Someone will need to fix the Bicubic downscaler HLSL code.

On top of giving a corrupt output, it could use a few performance tweaks. I know nothing about HLSL programming so someone else must do it.

That downscaler seems about right.

Although to make it faster it would probably be best to split it into two parts, first scaling horizontally then vertically. And to make it compile faster you could try replacing "[loop]" with "[fastopt]", although you might want to check if this gives the same output.

To compare the output of SuperRes with and without Bicubic.cso, compare Shaders\SuperRes\SuperRes.avsi and Shaders\SuperRes\SuperResBicubic.avsi

bxyhxyh
30th November 2015, 02:03
I think it's because integer division usually (according to the C++ spec) rounds down.
but what if you subtracted 1 first, before the division? Then you'd be at 0xfffe, which, after dividing by 257, rounds down to 0xfe (which is way off what you'd expect as an 8-bit approximation of 0xfffe). If you instead divide by 256, the rounding down works in your favour by getting you back to 0xff.
Then I think it should be converted to float and divided by 257.0.
Or can we directly divide integer by float? (I forgot)
Then round result to the closest integer.

MysteryX
30th November 2015, 02:09
Then I think it should be converted to float and divided by 257.0.
Or can we directly divide integer by float? (I forgot)
Then round result to the closest integer.
The conversion is working. Doing such a division would be more expensive performance-wise than bit-shifting. No need to change the current code unless we add assembly optimization to process 8 pixels at a time.

MysteryX
30th November 2015, 02:21
The latest version isn't as bad as I thought on memory, now that the format conversion code had been optimized and runs very fast. I ran a test and would get about 200fps with ConvertToFloat().ConvertFromFloat(), doing bit-shifting on each pixel and doing no YUV-RGB conversion; and if I remember that was in non-MT mode. It's not a bottleneck anymore.

As for memory usage, I'm running 8 threads of complex scripts and it's taking only 1GB, even though there are many devices getting created. Whatever memory bottleneck we were experiencing before isn't such an issue now, apparently.

Performance is in fact slightly better, as the "broken" Bicubic downscaler isn't well optimized.

MysteryX
30th November 2015, 03:11
I have updated the screenshots on the first page. The default in MPDN is SuperRes with 2 passes of strength=1 with Super-xBR with EdgeStrength=1, Sharpness=1. I tried with that and it's giving good results. See screenshots on first page.

There are a few things I had to tweak to get SuperRes+SuperxBR to run properly, which are not yet released. You can get the updated SuperRes.avsi here. (https://github.com/mysteryx93/AviSynthShader/blob/master/Shaders/SuperRes/SuperRes.avsi)

MysteryX
30th November 2015, 05:32
Now that's weird... after doing a bunch of tests and tweaks, when I'm re-testing the original script (SuperResBicubic.avsi in the repository), it gives a good output. Almost identical to MPDN except that it's very slightly sharper, which may be due to slightly different NNEDI3 configuration.

Can someone else test the latest version, and compare SuperRes.avsi (with AviSynth downscaling) with SuperResBicubic.avsi (with HLSL downscaling), and tell me whether the distortion is still there? I've done plenty of tests and now I'm confused as to what happened.

Another weird thing: if I disable YUV-RGB conversion, the output is *exactly* the same!! It does a GammaToLinear conversion on the YUV content and it doesn't seem to cause any distortion, which brings a few questions... but that works, somehow. In SuperRes.avsi, we just have to set "convertYuv=false" within the code.