AviSynthShader + SuperRes [Archive] - Page 7

luquinhas0021

21st May 2016, 19:45

Both in Windows Media Player, last version, and in MPC-BE, last stable version, shows: "it's not possible render the file", translating from portuguese.

MysteryX

22nd May 2016, 01:03

Both in Windows Media Player, last version, and in MPC-BE, last stable version, shows: "it's not possible render the file", translating from portuguese.
Have you tried playing other scripts? Have you tried running SuperResXBR without arguments? We don't even know whether the problem is related to AviSynthShader.

luquinhas0021

22nd May 2016, 01:22

I think the problem, or an it piece, is in Avisynth 2.6.0, because I've tried run a simple Bicubic upscaling, and I didn't can make it.
As I told you, I installed it again, but I didn't have time enough to play with it.
What I guessed so strange was the instability of SuperResXBR. One time, the script I wrote worked; other time, it didn't do.
I was with MPC-BE 1.4.6 beta (From this I ran SuperResXBR script). I was think it was the problem, then I go back to MPC-BE 1.4.5 stable, but the problem wasn't it. I will install the MPC-BE 1.4.6 beta again.

luquinhas0021

22nd May 2016, 01:55

You wrote that SuperRes with passes = 2 generates a bigger file, when compressed, that SuperRes with passes = 3. This means passes = 2 makes a sharper (Or more detailed) image?
Can I say that this script is like a SuperResXBR visible sharpest (Or more detailed) settings:
SuperResXBR(Passes=2 or 3, Str=1, Soft=0, XbrStr=5, XbrSharp=1.5, fDownscaler="SSim", fStr=1, fSoft=false, fB=0, fC=1)?
Ah, your last image comparison show the clown and lighthouse images, but it's four by four times bigger than the original ones, no-upscaled, that you show in first image comparison. But the SuperResXBR has factor 2 of upscaling. So, can you put here the original images you used in last image comparison?

MysteryX

22nd May 2016, 06:31

You wrote that SuperRes with passes = 2 generates a bigger file, when compressed, that SuperRes with passes = 3. This means passes = 2 makes a sharper (Or more detailed) image?
I find it sharper with Passes=3 after encoding.

Passes=5 loses some details somehow.

I'm running it twice: Double+InterFrame+Double. I just tried the first double with 5 passes and the 2nd with 3 passes and the result was great!

Here's the file size with each pass settings
2 passes: 89.6MB
3 passes: 88.3MB
4 passes: 87.8MB (loss of details)
5 passes: 87.6MB (loss of details)
5 passes + 3 passes: 87.4MB (sharp!)

http://s20.postimg.org/r65pjljvd/Passes2.png (http://postimg.org/image/r65pjljvd/) http://s20.postimg.org/nqdla1mmx/Passes3.png (http://postimg.org/image/nqdla1mmx/) http://s20.postimg.org/rk71j73yx/Passes4.png (http://postimg.org/image/rk71j73yx/) http://s20.postimg.org/5oz1pzhzt/Passes5.png (http://postimg.org/image/5oz1pzhzt/) http://s20.postimg.org/pto0hpmft/Passes53.png (http://postimg.org/image/pto0hpmft/)

Can I say that this script is like a SuperResXBR visible sharpest (Or more detailed) settings:
SuperResXBR(Passes=2 or 3, Str=1, Soft=0, XbrStr=5, XbrSharp=1.5, fDownscaler="SSim", fStr=1, fSoft=false, fB=0, fC=1)?
This will look distorted with big halos.

In the screenshots above, I'm using
SuperResXBR(3, 1, 0.15, XbrStr=2.7, XbrSharp=1.3, fWidth=1012, fHeight=778, fDownscaler="Bicubic", fB=0, fC=.75)

Can you put here the original images you used in last image comparison?
The original images are in the first post of this thread.

luquinhas0021

23rd May 2016, 01:49

MysteryX, honestly, I didn't see any difference between 3 passes and 5 + 3 passes, unless artifacts shift. My screen is good and has optimal color/luminous calibration, and I saw with zoom 100%.
Mystery, I used this above script for Clown and Lighthouse images, but, fortunately, it didn't stay distorted or with big halos:
SuperResXBR(Passes=3, Str=1, Soft=0, XbrStr=4, XbrSharp=1.5, fDownscaler="SSim", fStr=1, fSoft=false, fB=0, fC=1)
Maybe I down XbrStr from 4 to 3.7, in order to avoid any possible distortions, without loss of details and sharpness. But this is it!
You had spoken SSIM retains more details that Bicubic; after, spoke Bicubic 0.75 and SSIM 0.8 are almost equal, but the edges in Bicubic 0.75 are sharper (I imagine 0.75 and 0.8 refers to "fC" parameter). This why I put SSIM with fC=1: retain more details and to be equal or sharper than Bicubic 0.75.

MysteryX

23rd May 2016, 04:34

SSim doesn't use fB and fC parameters

Are you really seeing better results with SSim than with Bicubic before applying SuperRes? I'm considering changing the default downscaling method to Bicubic.

or maybe, to avoid confusion, I could share the same fB and fC parameters for both SSim or Bicubic, and remove fStr and fSoft.

luquinhas0021

23rd May 2016, 05:05

I didn't test Bicubic, because, when I tried run the script, happened the error I told you.
I'm considering the technical information you spoke.
SSIM doesn't use fB and fC parameters in your plugin or, really, it, originally, don't have b and c parameters? Where is 0.8, from SSIM 0.8, from? There's way like I increase or decrease this number?
I don't advice you remove fStr neither fSoft anyhow, unless you put, by default, fSoft=0 and fStr=1.

MysteryX

23rd May 2016, 08:21

SSim uses fStr=.5 and fSoft=false by default.
Bicubic uses fB=0 and fC=.75 by default.

"maybe" I could change the syntax so that SSim uses fB instead of fStr and fC instead of fSoft (0 or 1)

This syntax simplification could become more significant if another downscaler gets added into the mix.

You cannot run it with Bicubic but it works with SSim? What error are you getting?

luquinhas0021

23rd May 2016, 20:17

MisteryX wrote:
"SSim uses fStr=.5 and fSoft=false by default."

Change fStr to 1: people doesn't matter with a few rings if sharpness and detail maintenance is bigger!

MisteryX wrote
"maybe" I could change the syntax so that SSim uses fB instead of fStr and fC instead of fSoft (0 or 1)"

MisteryX wrote
In Bicubic, "b" is relative to softness and "c" to sharpness. I think you will make trouble in user's head if you put the SSIM parameters in way you told.

MisteryX wrote
"You cannot run it with Bicubic but it works with SSim? What error are you getting?"

There's was a time when SSIM ran; there's was a time when it didn't run; with Bicubic, never run.

MysteryX

24th May 2016, 04:07

There's was a time when SSIM ran; there's was a time when it didn't run; with Bicubic, never run.
I can't help unless you give me more details.

Using fB and fC arguments for both kernels isn't different from the way other shared resizers work such as ResizeX, where you have 2 parameters to configure whatever kernel is selected.

In Bicubic, "b" is relative to softness and "c" to sharpness. I think you will make trouble in user's head if you put the SSIM parameters in way you told.

As long as these parameters are documented.

And that way, it's lot easier to implement other resizers.

luquinhas0021

25th May 2016, 03:17

I was using a broken Avisynth, I realized. So, I will test with unbroken Avisynth and, if some error occur, I will tell you.

MysteryX

25th May 2016, 14:42

Version 1.4.4 is ready! (https://github.com/mysteryx93/AviSynthShader/releases/tag/v1.4.4)

What's new:
- Renamed SSimDownscaler to ResizeShader
- Removed fStr and fSoft arguments. With SSim, fB = Strength (0 to 1), fC = Soft (0 or 1)
- Renamed fDownscaler argument to fKernel and Downscaler to Kernel
- fKernel and Kernel default value is now Bicubic
- DLL now specifies its supported MT modes to AviSynth+ so SetFilterMTMode is no longer necessary

bilditup1

2nd June 2016, 04:46

Version 1.4.4 is ready! (https://github.com/mysteryx93/AviSynthShader/releases/tag/v1.4.4)

Quick question MysteryX - you mentioned in the other thread that AviSynthShader may be used for color matrix operations. Should a resize down (or up) to the desired size happen before or after calling ColorMatrixShader()?
I ask because it a resize is supposed to be done before color matrix processing when using Dither.

MysteryX

2nd June 2016, 05:55

AviSynthShader needs to convert from YUV to RGB to process anything anyway. If you want to downscale the image, you might as well use SSim and convert colorspace at the same time. This will convert from Rec601, resize using SSimDownscaler, and convert back using Rec709

ResizeShader(Width, Height, Kernel="SSim", MatrixIn=601")

bilditup1

2nd June 2016, 08:13

AviSynthShader needs to convert from YUV to RGB to process anything anyway. If you want to downscale the image, you might as well use SSim and convert colorspace at the same time. This will convert from Rec601, resize using SSimDownscaler, and convert back using Rec709

ResizeShader(Width, Height, Kernel="SSim", MatrixIn=601")

And converting to 601 (downscaling), I guess it would be

ResizeShader(Width, Height, Kernel="Bicubic", MatrixIn="709", MatrixOut="601")

right?

Also: should this be done before or after deinterlacing? After, is what I'd guess.

One more thing -

ColorMatrixShader(input, MatrixIn, MatrixOut, FormatOut)

Converts the color matrix with 16 bit depth to avoid banding. Source can be YV12, YV24, RGB24 or RGB32.

How much of a concern is banding when using ResizeShader instead of ColorMatrixShader?

MysteryX

2nd June 2016, 11:21

De-interlace first.

The difference between ColorMatrixShader and ResizeShader is that the various Shader functions allow processing the color correction at the same time without any performance penalty. And since the bottleneck is in memory transfers, performing various operations at once is a great performance benefit.

Use ColorMatrixShader only when you don't need any of the other functions to perform that operation alone.

bilditup1

2nd June 2016, 12:19

De-interlace first.

The difference between ColorMatrixShader and ResizeShader is that the various Shader functions allow processing the color correction at the same time without any performance penalty. And since the bottleneck is in memory transfers, performing various operations at once is a great performance benefit.

Use ColorMatrixShader only when you don't need any of the other functions to perform that operation alone.

Aha, great. Thanks for the explanation.

MysteryX

3rd June 2016, 11:11

Version 1.4.5 is ready! (https://github.com/mysteryx93/AviSynthShader/releases/tag/v1.4.5)

What's new:
- DitherTools (lsb_in/lsb_out) support was broken for the various functions and has been fixed
- ConvertToShader and ConvertFromShader now use Bicubic for chroma resizing when lsb=true
- ConvertToShader and ConvertFromShader now give an error if lsb=true and Precision=1

Note: There is still an issue where the x64 version takes over twice as much memory

When processing YV12 data, there is a slight loss of chroma data during the ConvertToYV24 conversion. If you want maximum quality, you should first convert to YV24 using DitherTools and set lsb_in and lsb_out to true. However, this may degrade performance by half because memory transfers are the bottleneck.

Dither_convert_8_to_16()
Dither_resize16(Width, Height/2, kernel="Bicubic", csp="YV24")
SuperResXBR(5, 1, 0.15, XbrStr=2.7, XbrSharp=1.3, MatrixIn="601", lsb_in=true, lsb_out=true)
Dither_resize16(Width, Height/2, kernel="Bicubic", csp="YV12")
DitherPost()

When upscaling SD material such as 288p, it is recommended to keep the maximum amount of data for the first image doubling as the size of each frame is limited so the performance cost is low. You can also use KNLMeans with lsb_inout before calling SuperResXBR. Then for the second double, you can do it directly on YV12 or YV24 data as the performance cost would be higher and the quality retention would be smaller.

So the rule of thumb is: with the smaller image, try to preserve as much of the details before extrapolating it. With the larger image, you can trade quality for performance as the loss of details is only extrapolated data anyway.

MysteryX

6th June 2016, 11:05

Version 1.4.6 is ready! (https://github.com/mysteryx93/AviSynthShader/releases/tag/v1.4.6)

What's new:
- Minor performance improvements
- Removed ColorMatrixShader
- ConvertFromShader no longer uses 'invks' when converting to YV12 using DitherTools

About ColorMatrixShader, it was designed to shift the color matrix in 16-bit to avoid banding. However, because it was calling ConvertToYV24 and ConvertTo12, rounding occured there which nullify any advantage over the standard ColorMatrix function or over the DitherTools conversion method. As a work-around, I could make ColorMatrixShader convert between YV12 and YV24 using DitherTools, but if it depends on DitherTools, it has no advantage over simply doing it the DitherTools way. Thus, I removed this function.

The best way to do shift the color matrix is to do it while calling another function such as SuperResXBR; or to use the standard methods (ColorMatrix or DitherTools).

If you want to avoid rounding errors (loss of data) when processing YV12, first convert it to 16-bit LSB format using DitherTools (Dither_convert_8_to_16) and call the shader method with lsb_in=true, lsb_out=true. It will then automatically perform the conversion between YV12 and YV24 in high-bit-depth.

ConvertToShader/ConvertFromShader will then internally convert using these functions

Dither_resize16nr(Width, Height/2, kernel="Spline36", csp="YV24")
Dither_resize16nr(Width, Height/2, kernel="Spline36", csp="YV12")

MysteryX

8th June 2016, 15:47

OMG. I found the performance bottleneck.

Replaced the flag D3DPRESENT_INTERVAL_DEFAULT with D3DPRESENT_INTERVAL_IMMEDIATE (https://msdn.microsoft.com/en-us/library/windows/desktop/bb172585(v=vs.85).aspx)

The driver will wait for the vertical retrace period (the runtime will "beam follow" to prevent tearing). Present operations will not be affected more frequently than the screen refresh; the runtime will complete at most one Present operation per adapter refresh period. This is equivalent to using D3DSWAPEFFECT_COPYVSYNC in DirectX 8.1. This option is always available for both windowed and full-screen swap chains. See remarks.

Performance went WAY up and I'm doing all kind of other performance improvements. A new version will be out soon!

MysteryX

11th June 2016, 12:16

Here are some preliminary benchmarks on the latest developments (available in the GitHub repository)

ColorBarsHD(352, 288).KillAudio()
ConvertToYV24()
SuperResXBR()
SuperResXBR()
Prefetch(8)

Current version (1.4.6)

FPS (min | max | average): 2.286 | 1000000 | 22.36
Memory usage (phys | virt): 311 | 656 MiB
Thread count: 131
CPU usage (average): 26%

Version 1.5

FPS (min | max | average): 3.200 | 1000000 | 26.71
Memory usage (phys | virt): 333 | 344 MiB
Thread count: 113
CPU usage (average): 5%

4 threads instead of 8

FPS (min | max | average): 5.818 | 1000000 | 27.04
Memory usage (phys | virt): 193 | 200 MiB
Thread count: 69
CPU usage (average): 5%

Let's try with Stack16 input and output. As of right now the output is corrupt because of a bug in the new ConvertShader functions but hopefully that will be fixed soon; it shouldn't affect the GPU performance.

ColorBarsHD(352, 288).KillAudio()
ConvertToYV24()
Dither_convert_8_to_16
SuperResXBR(lsb_in=true, lsb_out=true)
SuperResXBR(lsb_in=true, lsb_out=true)
DitherPost()
Prefetch(8)

Version 1.4.6

FPS (min | max | average): 0.207 | 1000000 | 3.208
Memory usage (phys | virt): 478 | 916 MiB
Thread count: 140
CPU usage (average): 3%

Version 1.5

FPS (min | max | average): 0.800 | 1000000 | 13.15
Memory usage (phys | virt): 481 | 496 MiB
Thread count: 122
CPU usage (average): 13%

4 threads instead of 8

FPS (min | max | average): 1.684 | 1000000 | 13.36
Memory usage (phys | virt): 312 | 318 MiB
Thread count: 78
CPU usage (average): 12%

So far, we have a 21% performance increase and 70% memory reduction in 8-bit, and a 316% performance increase and 65% memory reduction in 16-bit.

I'll see if I can turn this into a MT_NICE filter. It would require creating a memory pool for all the textures similar to a pool cache that was implemented in AviSynth+. If I can get that to work, memory usage will go considerably further down.

MysteryX

13th June 2016, 13:53

Got it working with MT_NICE_FILTER and a dynamic allocation pool. Here's the result with 4 threads.

FPS (min | max | average): 5.333 | 1000000 | 25.57
Memory usage (phys | virt): 136 | 153 MiB
Thread count: 31
CPU usage (average): 4%

Performance is slightly lower than 27fps but memory usage is considerably lower.

if I run this new code design with MT_MULTI_INSTANCE, I get this

FPS (min | max | average): 6.400 | 1000000 | 27.07
Memory usage (phys | virt): 193 | 218 MiB
Thread count: 71
CPU usage (average): 5%

So far MT_MULTI_INSTANCE is still working better unless I can do some more optimization, but either way, memory usage is a lot lower than before.

MysteryX

14th June 2016, 07:23

MysteryX

14th June 2016, 10:21

Woah... once integrated into a complex script (and the threads are busy working on other filters), the picture is different!!

MT_MULTI_INSTANCE, 4 threads
13.43fps, 478MB, 15% CPU

MT_MULTI_INSTANCE, 8 threads
14.43fps, 677MB, 18% CPU

MT_NICE_FILTER, 4 threads
14.81fps, 435MB, 16% CPU

MT_NICE_FILTER, 8 threads
17.13fps, 555MB, 18%

We got a winner!!!

MysteryX

15th June 2016, 17:01

MysteryX

18th June 2016, 14:47

Convert functions are now working.

Now this is embarassing. There are still image differences with the previous version, but as I investigate, it appears LSB wasn't working at all in the last version!

- Fixed ConvertToShader(2) was generating noise in the 2nd byte
- Fixed ConvertToShader(2, lsb=true) was causing brightening
- Fixed ConvertFromShader(2, lsb=true) was corrupting the lsb data

Chikuzen's version of the converters looks OK... Kind of hard to compare when the previous version was wrong.

Well... view it this way. Version 1.4.6 is auto-applying dithering :)

MysteryX

21st June 2016, 18:05

Version 1.5 is ready! (https://github.com/mysteryx93/AviSynthShader/releases/tag/v1.5)

Major improvements:
- Major performance improvements (~double)
- Replaced D3DPRESENT_INTERVAL_DEFAULT with D3DPRESENT_INTERVAL_IMMEDIATE which removed the performance cap
- Added Ordered Dithering (16x16) before returning 8-bit-depth data
- Shader: Clip1-Clip9 can now have values between 0 and 100
- Shader: Added Precision parameter to specify different output precisions per shader
- Shader: Now detecting whether to compile file or not based on extension (.hlsl or .cso)
- Shader: ShaderModel default value is now 'ps_3_0'
- ExecuteShader: Full support for MT_NICE_FILTER under AviSynth+. When MT is enabled, it will alternate threads between 2 engines for optimal performance. AviSynth 2.6 will use it with MT=2.
- ExecuteShader: Added PlanarOut parameter to return output as YV24 planar data
- ExecuteShader: Can now transfer YV24 source clips as planar data
- ExecuteShader: Precision can now be 0 to process with Y8
- ConvertToShader/ConvertFromShader: Added assembly optimizations (thanks to Chikuzen!!)
- ConvertToShader/ConvertFromShader: Added Planar parameter to convert to YV24 (thanks to Chikuzen!!)
- ConvertToShader/ConvertFromShader: Added opt parameter to specify the optimization path to execute
- ConvertToShader/ConvertFromShader: Now support Precision=0 for Y8
- ConvertToShader/ConvertFromShader: Precision default value is now 1
- Added PlanarIn and PlanarOut parameters to all shader scripts, mostly useful for performance tweaks
- Fixed ConvertToShader(2) was generating noise in the 2nd byte
- Fixed ConvertToShader(2, lsb=true) was causing brightening
- Fixed ConvertFromShader(2, lsb=true) was corrupting the lsb data
- ConvertFromShader: Format default value is now YV24 instead of YV12
- SuperXBR and ResizeShader: Added ConvertYuv parameter for consistency

Note: This version comes with 4 DLLs: Regular (SSE2), AVX, x86 and x64. The AVX brings increased performance when using ConvertToShader with Precision=3 which is not used by default in Shader.avsi. The AVX version will fail if your CPU doesn't support it. The SSE2 version is recommended.

The Dither feature was added at the last minute and hasn't been fully tested. Give it a try and let me know if something needs to be tweaked.

luigizaninoni

22nd June 2016, 12:35

Issue with SuperRes in 1.5.0: The following script immediately crashes with AVS+ 64-bit (inside staxrip):

LoadPlugin("C:\Users\luigi.TZMS\Desktop\Video\Stax64\Apps\Plugins\avs\aWarpSharp2\aWarpSharp.dll")
LoadPlugin("C:\Users\luigi.TZMS\Desktop\Video\Stax64\Apps\Plugins\avs\Decomb\Decomb.dll")
LoadPlugin("C:\Users\luigi.TZMS\Desktop\Video\Stax64\Apps\Plugins\avs\L-SMASH-Works\LSMASHSource.dll")
Import("C:\Users\luigi.TZMS\Desktop\Video\Stax64\Apps\Plugins\avs\QTGMC\QTGMC.avsi")
LoadPlugin("C:\Users\luigi.TZMS\Desktop\Video\Stax64\Apps\Plugins\avs\masktools2\masktools2.dll")
LoadPlugin("C:\Users\luigi.TZMS\Desktop\Video\Stax64\Apps\Plugins\avs\mvtools2\mvtools2.dll")
LoadPlugin("C:\Users\luigi.TZMS\Desktop\Video\Stax64\Apps\Plugins\avs\nnedi3\nnedi3.dll")
LoadPlugin("C:\Users\luigi.TZMS\Desktop\Video\Stax64\Apps\Plugins\avs\RgTools\RgTools.dll")
SetFilterMTMode("DEFAULT_MT_MODE", 2)
LWLibavVideoSource("Y:\work\prova_temp\prova.m2v", format = "YUV420P8")
vid0=last
vid=(QTGMC(vid0,Preset="Slower",EzDenoise=0.9,denoiser="dfttest",NoisePreset="Slower").SelectEven())
vid2=QTGMC(vid0,Preset = "Slow",InputType=1,EzDenoise=0.6,denoiser="dfttest",NoisePreset="Slower")
ConditionalFilter(vid0,vid,vid2,"IsCombed()" , "=" , "True")
Crop(0, 2, -0, -2)
MergeChroma(aWarpSharp(depth=10), aWarpSharp(depth=20))
Overlay(last, ColorYUV(off_y=-8, off_u=8, off_v=-2), 0, 0, GreyScale(last).ColorYUV(cont_y=30))
SuperRes(2,0.65,0,"""nnedi3_rpow2(2, cshift="Spline16Resize")""",MatrixIn="601")
Prefetch(3)

Same script works fine in 1.4.6

MysteryX

22nd June 2016, 13:32

It may have to do with ExecuteShader now running as MT_NICE_FILTER.

nnedi3_rpow2 fails under AviSynth+ MT, while instead calling it through edi_rpow2 does work. The reason is a bit complicated.

Reel.Deel

22nd June 2016, 13:45

nnedi3_rpow2 fails under AviSynth+ MT

This issue was fixed in the latest nnedi3 version: http://forum.doom9.org/showthread.php?p=1768995#post1768995

Mystery Keeper

22nd June 2016, 13:59

I don't get you people comparing x64 builds against x86. x64 is not supposed to be faster or slower. It is needed to access more memory. Because with x86 if you hit 4GB (that's the maximum you can get only on x64 OS with properly built executable) - you crash. With complex scripts and FullHD content you can easily exceed that memory load.

MysteryX

22nd June 2016, 14:38

I don't get you people comparing x64 builds against x86. x64 is not supposed to be faster or slower. It is needed to access more memory. Because with x86 if you hit 4GB (that's the maximum you can get only on x64 OS with properly built executable) - you crash. With complex scripts and FullHD content you can easily exceed that memory load.
x64 is generally a bit faster when implemented properly.

In the case of AviSynthShader, however, there are no benefits there as the CPU isn't the bottleneck.

The x64 version in fact takes a LOT more memory. DirectX just manages the memory differently in x64 and I haven't found any way to fix that...

Mystery Keeper

22nd June 2016, 19:23

MysteryX, simple. x64 doesn't take "a lot" more memory. The memory usage increase comes from the doubled pointer size. The more pointers used - the more memory they take.

MysteryX

23rd June 2016, 01:15

There aren't many pointers in my program; yet AVSMeter showed and memory usage is over TWICE HIGHER! So instead of taking 900MB, the script might take 2.3GB. What takes memory are the texture buffers on the CPU and GPU which are managed by DirectX 9 and the graphic driver.

Although I haven't found the cause nor the solution, v1.5 takes a lot less memory in both cases.

MysteryX

24th June 2016, 08:25

I'm seeing two bugs already. Lsb output is broken (corruption of dark colors) and ResizeShader(kernel="SSim") corrupts a line at the bottom when not using lsb_out. Will have to investigate.

Edit: the lsb issue occurs when using only one of lsb_in/lsb_out. Using it twice cancels the distortion. There is also a bug in SSimDownscaler.

MysteryX

24th June 2016, 09:46

I found an easy way to make dither and banding visible: upscaling ColorBars

First, the Ordered Dither shader is doing a pretty good job. In fact, it's doing better than Dither Tools' ordered dither (mode=0). Not sure how mode=0 got implemented; perhaps with a matrix that is too small. I'm using a 16x16 matrix.

Second thing to notice is that there is color distortion between DitherShader and Dither0. This was really confusing because I only see a very slight color difference in VirtualDub, and once I copy both into PhotoShop, the color difference becomes greater!??

Third thing to notice is that Dither6 completely distorts the dark colors. Would love to hear experts' opinion on this one!

ColorBarsHD(300, 200)
ResizeShader(600, 500, kernel="bicubic")

https://s32.postimg.org/rjqa84je9/Dither_Shader.png (https://postimg.org/image/rjqa84je9/)

ColorBarsHD(300, 200)
ResizeShader(600, 500, kernel="bicubic", lsb_out=true)
DitherPost(mode=0)

https://s32.postimg.org/sjgl3tyk1/Dither_Mode0.png (https://postimg.org/image/sjgl3tyk1/)

ColorBarsHD(300, 200)
ResizeShader(600, 500, kernel="bicubic", lsb_out=true)
DitherPost(mode=6)

https://s32.postimg.org/wh3ut8ldd/Dither_Mode6.png (https://postimg.org/image/wh3ut8ldd/)

But then, to make it even more confusing, this one looks fine.

ColorBarsHD(300, 200)
Dither_convert_8_to_16
dither_resize16(600, 500)
DitherPost(mode=6)

https://s31.postimg.org/uxbh88nlz/Dither_Resize6.png (https://postimg.org/image/uxbh88nlz/)

Maybe ShaderResize is returning out-of-range values that DitherPost doesn't know what to do with?

MysteryX

29th June 2016, 09:11

Version 1.5.1 is ready! (https://github.com/mysteryx93/AviSynthShader/releases/tag/v1.5.1)

What's new:
- Shader: Added Defines parameter to configure HLSL shaders before compilation
- Restructured the files to compile several HLSL files on-demand with compilation parameters. Reduced the amount of shader files from 41 to 26.
- Fixed: Texture overflow method wasn't properly reset to CLAMP after dithering with WRAP
- ExecuteShader: Changed default values Clip1Precision=1, OutputPrecision=1, Precision=3
- ResizeShader: Changed default values to Kernel="Bicubic", B=1./3, C=1./3. Defaults for downscalers remain B=0, C=.75
- MatrixIn and MatrixOut now support PC colors for matrix conversion. Valid values are now 601, 709, Pc601 and Pc709.
- ResizeShader: Added Kernel="ColorMatrix" to perform color matrix conversion without resizing

jrodefeld

29th June 2016, 14:56

Okay, I've been trying to get this filter to work for me with no success. I'm not even entirely sure I have it installed correctly. If anyone could give a basic, clear explanation of how to install and how to write a basic script for this plugin, I'd really appreciate it.

I've been trying to add this command to other AVS scripts I have that I know already work. Then I want to load the script into Virtualdub. But Virtualdub is crashing every time. It is the same basic crash and error message that I get every time I make an error in a script. I'm honestly not entirely clear what all the commands do for this filter. I've been basically copying and pasting different script examples I've found in this thread. Some of the documentation is a little over my head, so I think I need a dumbed down version.

Any help would really be appreciated.

luigizaninoni

29th June 2016, 15:02

Okay, I've been trying to get this filter to work for me with no success. I'm not even entirely sure I have it installed correctly. If anyone could give a basic, clear explanation of how to install and how to write a basic script for this plugin, I'd really appreciate it.

I've been trying to add this command to other AVS scripts I have that I know already work. Then I want to load the script into Virtualdub. But Virtualdub is crashing every time. It is the same basic crash and error message that I get every time I make an error in a script. I'm honestly not entirely clear what all the commands do for this filter. I've been basically copying and pasting different script examples I've found in this thread. Some of the documentation is a little over my head, so I think I need a dumbed down version.

Any help would really be appreciated.

Can you post your full non-working script ?

jrodefeld

29th June 2016, 15:02

Okay, here's the script I'm using right now:

DirectShowSource("D:\Downloads\-- RAW DM2 ENDING --.avi")

LoadPlugin("C:\Program Files (x86)\AviSynth\plugins\Shader.dll")

QTGMC(Preset="Slow")
SelectEven()

Decimate()

Crop(0,90,0,-90)

SuperRes(1, .85, 0, false, """nnedi3_rpow2(rfactor=4, cshift="Spline16Resize", Threads=1)""")

Any ideas?

MysteryX

29th June 2016, 15:04

First, does it work without SuperRes? Can you get it to work by removing some components?

What version of AviSynth are you using?

luigizaninoni

29th June 2016, 15:07

Did you put all the files that came with shader.dll in the same folder as shader.dll ? that is, in your case, in C:\Program Files (x86)\AviSynth\plugins\Shader.dll ?

Also, try removing "false" from your syntax

luigizaninoni

29th June 2016, 15:11

My syntax is usually

LWLibavVideoSource("Y:\work\prova_temp\prova.m2v", format = "YUV420P8")

SuperRes(2,0.65,0,"""nnedi3_rpow2(2,shift="Spline16Resize")""",MatrixIn="601")

And it works fine

jrodefeld

29th June 2016, 15:21

Okay, I discovered something else. Even when I remove the SuperRes line from my script but keep the "LoadPlugin" line, VirtualDub still crashes. So there must be some problem with the DLL files, my version of AviSynth or something else. Just for the record, when I remove the SuperRes line AND the LoadPlugin line, the script loads just fine so I know the other commands are working. I've used this script many times minus the SuperRes stuff.

And yes, I've copied over all the other files that came with the shader.dll file into the same directory.

I'm using either AviSynth 2.5.8 or 2.6. I can't recall at the moment, but it's never given me any trouble before.

Any other ideas?

raffriff42

29th June 2016, 15:24

This is what worked for me (using the actual path to the shader folder of course)SetWorkingDir (http://avisynth.nl/index.php?title=Internal_functions/SetWorkingDir)("c:\...\shaders")
...

An alternative, not tested: adding the shader folder to your PATH (https://www.google.com/search?q=windows+set+path+variable)

luigizaninoni

29th June 2016, 15:24

Maybe your are loading the plugin from your autoload plugin folder, so AVS crashes trying to load the same plugin twice ?

Groucho2004

29th June 2016, 15:37

Then I want to load the script into Virtualdub. But Virtualdub is crashing every time. It is the same basic crash and error message that I get every time I make an error in a script.
How about telling us what that error message is? Or, even better, analyse your Avisynth install with AVSMeter (http://forum.doom9.org/showthread.php?t=165528):
"avsmeter -avsinfo -log". This will create a log file which you should post in case there are errors ("avsinfo.log" is created in the current (working) directory).

jrodefeld

29th June 2016, 15:43

How about telling us what that error message is? Or, even better, analyse your Avisynth install with AVSMeter (http://forum.doom9.org/showthread.php?t=165528):
"avsmeter -avsinfo -log". This will create a log file which you should post in case there are errors ("avsinfo.log" is created in the current (working) directory).

The error message is: "Oops - VirtualDub has crashed.", "An out-of-bounds memory access (access violation) occurred..."

This is the same message I get any time there is any problem with a script, so I doesn't seem to pinpoint the problem.

I'll try the AVSMeter program and see if that gives any idea what could be the problem.

jrodefeld

29th June 2016, 16:08

Okay, I've made some progress. I decided to re-install AviSynth and try it again. I actually installed AviSynth+. Anyway, I am not getting the same crashes in Virtualdub that I was getting before. However, I am getting some other error messages.

I'm using the exact script luigizaninoni recommended, but I'm getting this error: "There is no function named ConvertToShader"

Any idea what this means and what I can do about it?