Jinc Resizer - Avisynth Plugin [v0.1.1]

[innocenat]

So I know a lot of people (including me) want the new Jinc resizer from madVR in Avisynth. So after some research and some help from madshi, here I present the Jinc Resizer

This plugin require Avisynth 2.6.


Download (https://github.com/innocenat/jinc-resize/releases)
GitHub repo (https://github.com/innocenat/jinc-resize)


IMPORTANT: Downscaling isn't currently implemented.

Changelog

v0.2
- Core rewritten to use quantized lookup table for coefficient
- Support SSE2, SSE3, AVX2 and FMA3.
- Basically, MUCH faster.

v0.1.1
- Binary now compile with ICC14
- More optimized code, should rune much faster.
- Thanks to tp7 and others for optimization tip.

v0.1
- Initial release

Documentation
The plugin currently exposes 3 functions:


Jinc36Resize(clip, int width, int height)
Jinc64Resize(clip, int width, int height)
Jinc144Resize(clip, int width, int height)
Jinc256Resize(clip, int width, int height)


Crop-style syntax like in internal resizers also works. You can control sub-pixel quantization option by quant_x and quant_y parameter, default is 256.

Note

It should be fast enough for encoding now, especially if you have Haswell CPU or newer. Though the quality without anti-ringing filter is debatable.

Roadmap

Add anti-riging filter
Possibly add OpenCL version

[burfadel]

Looks interesting :) Will be looking forward to when the stuff in the roadmap is completed.

If making an OpenCL version, it would probably be worth adding related things like sharpen and deband of some description. I know Madshi is currently implementing debanding in MadVR. I suggest this because a small amount of sharpening and debanding can be a good thing when resizing, and might as well get that done on the graphics side of things.

[PetitDragon]

OMG! This is just f*cking great!:thanks:

[innocenat]

v0.1.1 released, with more optimization that should runs at least two times faster.

[Gavino]

Thanks for this, innocenat - a useful addition.

I notice that only planar YUV clips are currently supported.
Will support for YUY2 and RGB be included eventually?

Also worth noting it is a 2.60 plugin only.
Any reason why it couldn't be made to work on 2.58 too?

[innocenat]

I notice that only planar YUV clips are currently supported.
Will support for YUY2 and RGB be included eventually?

I don't think I will add support for YUY2 and RGB data soon. The problem is that interleaved format is hard to load quickly in SIMD. And the filter is already slow as it is.


Also worth noting it is a 2.60 plugin only.
Any reason why it couldn't be made to work on 2.58 too?

First, a lot of things I use differ from 2.5 header and 2.6 header, such as accepting floating point parameter, get subsampling detail etc. Not that it couldn't be done, but unless there are actually people who want it, I don't think it's worth the effort. Also, there might be AVX optimization in future, and you'd require Avs+ or post-a5 Avisynth 2.6 anyway.

Second, I think most users of Jinc Resizers will be hardcore user and will probably be using 2.6 already anyway.

[Keiyakusha]

After putting it into plugins directory, I'm getting this, even if JincResizer is not used:

The program can't start because libmmd.dll is missing from your computer. Try reinstalling the program to fix this problem.

[innocenat]

After putting it into plugins directory, I'm getting this, even if JincResizer is not used:

Sorry, binaries updated.

[Keiyakusha]

Well, for me on core i7, when doing 720x480 -> 1920x1080, Jinc36 is 2-3 times (depending on content complexity due to prescreener) slower than icl-compiled nnedi3 with fturn (4x upscale + bicubic downscale) and the visual quality is obviously worse.
Edit: nnedi was set to use only 1 thread. I hope I'm not sleeping again, cause as of late I'm getting weird results when trying new avisynth resamplers ^^

[turbojet]

Thanks for this, do you plan on implementing downsizing?

[innocenat]

@turbojet

Downscaling will be added later, that's for sure.

@Keiyakusha

Yes, nnedi3_rpow2 should be much better choices right now. Maybe until I implement an AR filter and add a prescreener and/or OpenCL version.

[Keiyakusha]

Yes, nnedi3_rpow2 should be much better choices right now. Maybe until I implement an AR filter and add a prescreener and/or OpenCL version.
Mmm... but if this is exactly the same stuff as in madvr, it won't be able to beat nnedi quality-wise even with AR. It simply doesn't connects lines as good. And judging from the feedback in madvr thread, Jinc downscale sux and sticking to something like spline36 is a better idea. Also now that we have OpenCL nnedi3, do you think Jinc is still will be competitive? So far Jinc performs exactly as I was afraid of...

[innocenat]

Mmm... but if this is exactly the same stuff as in madvr, it won't be able to beat nnedi quality-wise even with AR. It simply doesn't connects lines as good. And judging from the feedback in madvr thread, Jinc downscale sux and sticking to something like spline36 is a better idea. Also now that we have OpenCL nnedi3, do you think Jinc is still will be competitive? So far Jinc performs exactly as I was afraid of...

It's nice to have options, after all.

Personally, I don't care if people will use it over nnedi3ocl; coding this is fun, and help me learn a lot of thing.

[mandarinka]

Oh, this could be very interesting.

Correct me if I am wrong, but this really shouldn't be that much slow, no? Since it is still just a resampler, not a complex interpolator like nnedi variants and other EDI methods, I would expect it to be faster.

[madshi]

Jinc has to be slower than other linear resamplers because due to how it works (can't be split into 2 separate passes) it has to read and process more source pixels. E.g. compared to Lanczos3, Jinc3 has to read and process 4x as many source pixels.

One thing to keep in mind is that NNEDI3 can only do exact 2x enlargements while Jinc can handle any up- and downscale factor you want.

[innocenat]

In addition to what madshi said, other traditional resampler can also be normalize to integer operation and has their coefficients cached prior to actual resampling.

Jinc and other 2d resampler unfortunately can't, and has to calculate everything in floating point without any coefficients cache.

[cretindesalpes]

BlankClip (width=640, height=480, pixel_type="YV12")
ShowFrameNumber ()
Jinc64Resize (1920, 1080)

The result is often corrupted:
http://s29.postimg.org/reh24qtcz/jincresize.jpg (http://postimg.org/image/reh24qtcz/)
Sometimes it works, but there is no obvious pattern.

I quickly hacked a version with precalculated coefficients (https://ldesoras.fr/src/avs/JincResize-0.1.1-quick-hack.zip), compiled with MSVC 2012. It’s faster but very memory hungry (don’t even think about Jinc256 on a HD frame). Enable the precalculation with table=true. However the memory usage can be easily reduced when the scale is a rational fraction or by accepting a small phase error (sub-pixel shift), which would allow using a same set of coefficients for different locations.

EDIT: Second attempt (https://ldesoras.fr/src/avs/JincResize-0.1.1-quick-hack2.zip) with position quantization (1/256). Much more memory-friendly and faster. Use pquant=true to enable the quantization.

[innocenat]

Yes, a Jinc36Resize on 1920x1080 frame would produce roughly 490MB worth (49 coeff * 4 bytes (float) * 1920x1080 pixel). That's why I don't even think about doing that. I guess I could add option for that. The precalculated coeff version could be simplify further by using integer operation instead of floating point (accepting a small rounding error).

Regarding the error, I guess it's because some reckless assumption I made concerning memory layout. Will have to investigate further.

[Nevilne]

Anti-ringing filter:



Function nrJinc36Resize(clip input, int "target_width", int "target_height", float "src_left", float "src_top", float "src_width", float "src_height"){

Assert( input.IsPlanar(), "nrJinc36Resize: only planar color spaces are supported!" )

target_width = Default( target_width, width(input) )
target_height = Default( target_height, height(input) )
src_left = Default( src_left, 0 )
src_top = Default( src_top, 0 )
src_width = Default( src_width, 0 )
src_height = Default( src_height, 0 )

return input.Jinc36Resize(target_width, target_height, src_left, src_top, src_width, src_height)
\ .Repair(input.GaussResize(target_width, target_height, src_left, src_top, src_width, src_height, p=100), 1)

}

[Gavino]

src_width = Default( src_width, 0 )
src_height = Default( src_height, 0 )

The built-in defaults for src_width and src_height are not zero, they are respectively width(input) and height(input). In the case where src_left or src_top is non-zero, this gives a different result.

If you want your function to have the same defaults as the actual resizers, the simplest way is just to pass on the function parameters src_left, src_top, src_width and src_height unchanged. That way, you don't even need to know what the actual default is.

[NicolasRobidoux]

...
And judging from the feedback in madvr thread, Jinc downscale sux and sticking to something like spline36 is a better idea.
...
I don't know, specifically, how it performs with video content, but fully implemented (possibly slightly deblurred, as it is in madVR---where it only upsamples---and by the EWA LanczosSharp method of ImageMagick---where it can do everything) Jinc-windowed Jinc 3 lobe is an amazing downsampler with digital photographs, esp. if used through linear light.

[NicolasRobidoux]

I know I am preaching for my own church of one, but a cheaper EWA filter that works fairly well, at least for downsampling, is the Robidoux filter, which is the EWA default (chosen by Anthony Thyssen, the dev. responsible for resampling, not by me) in ImageMagick. Given that it uses a disk of radius 2 (instead of a bit more than 3 like most EWA Lanczos variants) and that the coefficients are computed with a Keys spline, it should run quite a bit faster.
Don't get me wrong: With good quality input content (and output not overly compressed destructively), EWA Lanczos and close relatives gives better results with the types of images I work with (not video). But EWA Robidoux may give good bang for the buck.

[madshi]

Jinc downscaling is just not (yet) implemented in madVR. It might be in the future. I have no specific opinion about how it performs yet. I'll cross that bridge when I come to it.

[NicolasRobidoux]

If you base your look up table on the square of the distance, you should be able to live with a 1D array.

[NicolasRobidoux]

It should be possible to base a pretty nice AR (applied to Jinc-windowed Jinc 3-lobe, deblurred or not) on the key components of the bicubic interpolator LBB (Locally Bounded Bicubic) built into the NoHalo method (found in the open source GEGL and VIPS libraries).

[innocenat]

You should really consider editing your post instead of adding new reply.

Downscale isn't implement because it requires dynamic disk size. The more downscale ratio, the larger the disk need to be. Compared to upscale, where disk size is constant, this poses more problem for optimization. The performance, at least for this filter, runs directly proportional to disk size, because the coefficient is calculate with LUT.

What I am currently working on:
- Integrating the quantized-table-lookup approach from cretindesalpes, without using STL container like he is using.
- Figuring out the the deringing algorithm.
- Figuring out what cause non-deterministic behaviour (sometimes it produces wrong result)

I will consider other EWA-based scaling too; the code is written in such way that it's easy to add other resizing kernel.

The reason I have been so slow on this is because real-life, and I have been spending free time optimizing Avisynth+ internal resizer.

[NicolasRobidoux]

Downscale isn't implement because it requires dynamic disk size. The more downscale ratio, the larger the disk need to be. Compared to upscale, where disk size is constant, this poses more problem for optimization. The performance, at least for this filter, runs directly proportional to disk size, because the coefficient is calculate with LUT.
I am fully aware that the main bottleneck is memory traffic when downsampling. A reasonably elegant solution, based on power of 2 box filtered mipmaps, is built into the GEGL EWA components of the filters NoHalo and LoHalo. Whether this approach could pay off within Avisynth, I have no idea. It obviously depends on whether (local) mipmaps can be created and used efficiently.

[NicolasRobidoux]

...
- Figuring out the the deringing algorithm.
...
It is my hunch that anti-ringing is much less useful when downsampling. At least, much less useful than going through linear light.

[Seedmanc]

Is it better than spline144resize?

[Asmodian]

Depends of course. I would say yes but you need to say what you mean by "better". Jinc is not sharper but it has lower aliasing and ringing.

[innocenat]

Jinc is not sharper but it has lower aliasing and ringing.

lolno. Jinc (and all lanczos-based resizer) has hallelujah ringing. But less aliasing, yes.

[Asmodian]

Ah it must have been madshi's anti-ringing filter which came out at a similar time that gave me the impression Jinc had less ringing. :o

[innocenat]

Long time no update. I presented v0.2. Thanks to cretindesalpes idea, now it runs much faster.

I have rewritten the core to be only quantized, table-lookup coefficient only. Even though this is very slightly less accurate, it's not noticeable at all and it runs MUCH faster, on my Haswell laptop the 360p->720p upscale with Jinc36Resize runs at near 40fps. It now has acceleration for SSE2, SSE3, AVX2 and FMA3. While FMA3 offer very slight to unnoticeable advantage over AVX2, AVX2 is huge leap over SSE3, which is also huge leap over SSE2. Non-deterministic behaviour should have also disappeared.

Sorry, anti-ringing filter is not yet implemented.

I have also add Jinc144Resize which is 6-tap filter. You can control quantization option by quant_x and quant_y option. Default is 256 for both value.

[zerowalker]

What's the main difference between this and Lanczos / Spline?

I haven't used MadVR much so haven't got any experience with the options there.

[DarkSpace]

I think the answer you're looking for is "Lanczos / Spline scale width and height separately, while Jinc uses some sort of elliptical weight averaging, which means it scales both width and height in one step" (it's also called EWA Lanczos with certain parameters).

[zerowalker]

And, how does that change the result?
I mean, what is the point of this?

[innocenat]

And, how does that change the result?
I mean, what is the point of this?

Please read the thread. It has already been answered.

[zerowalker]

Guess i am blind or assuming a different kind of answer, as i can't actually find anything.
I just see the differences in how it calculates, but not the actual difference in the results qualitywise.

If perhaps, not this is the answer.
lolno. Jinc (and all lanczos-based resizer) has hallelujah ringing. But less aliasing, yes

[madshi]

Jinc is a bit softer than Lanczos, has a bit less ringing than Lanczos (but still some), but has noticeably less aliasing. Jinc has a more "analog" look to it. Even the ringing looks more natural compared to Lanczos. But some people prefer Lanczos because it's a bit sharper. Also Jinc is much slower than Lanczos because Jinc can't separate X and Y scaling operations.

Here's a comparison image from madVR, upscaling Monsters AG to 4K:

http://madshi.net/madVR/monsters.png

[jpsdr]

After, it's a matter of personnel choice, but mine goes to NNEDI3.

[madshi]

NNEDI3 is quite great, IMHO. It does have disadvantages, too, though. E.g. in some image areas (trees, grass, leaves) it can produce fractal like artifacts. That doesn't happen with Jinc. Also NNEDI3 is a lot slower than Jinc, when using madVR. I'm not sure how speed compares in AviSynth, though. Finally, NNEDI3 can only upscale by exactly 2.0x, while Jinc can up and downscale with any factor you like. So although NNEDI3 is great, there's a place for Jinc, too, IMHO.

[zerowalker]

Hmm, well guess i say the same, Lanczos looks similar, but has more artifacts.
Jic looks less detailed, so softer indeed.

But i think it depends on content, some things looks better in a soft detailed scale, and harder need that sharp edge.

[innocenat]

Also NNEDI3 is a lot slower than Jinc, when using madVR. I'm not sure how speed compares in AviSynth, though.

In my test (on Mobile Haswell, which uses AVX2 that is ~25% faster than SSE3 on same CPU), Jinc36 720p->1080p runs at ~20fps. With nnedi3 doubling with spline64 to 1080p average to 3-5fps prescreener on, <1fps off. I can't test the OpenCL version because I can't force it to use my external graphic over integrated (I'm on optimus setup).

So yeah, right now performance is MUCH better. But tbh, because of current optimization I don't think it will be nearly as fast on CPU when anti-ringing filter is implemented due to branchy nature of the filter. I am looking into OpenCL right now, but no promise since I don't have much free time nowadays.

[Reel.Deel]

Hi innocenat,

Thanks for the update. I was wondering what's the purpose of the version parameter? When I set it to true it gives this message.
[Jinc Resizer] [7] Compiled Instruction Set: FMA3 AVX2 SSE3 SSE2 x86

[innocenat]

It currently show Jinc's internal CPU flag (the [7]) and instruction support it compiles with. I was not sure if AVX2 will really be faster at first so I make this mechanism to tell which version of plugin you have. The Jinc's internal CPU flag is because Avisynth and Avisynth+ cannot currently detect FMA3 and AVX2. Granted, its name is misnomer.

[Reel.Deel]

Thanks for the information. One more question if you don't mind. Whats the license for JincResize? Apache 2.0 license? The reason I'm asking is because I want to add JincResize it to the wiki.

[innocenat]

Put it as Apache 2.0 I guess.

The Jinc function calculation (JincFilter.cpp) are Apache 2.0 since they are from ImageMagick. The main resampling code I wrote (EWAResizer.h, FilteredEWAResize.cpp, etc.) is also under MIT license. But the combination (i.e. the project itself) is under Apache 2.0. I guess should put a LICENSE file on the repository.

On a side note, you might encounter line artefact with large upscaling factor. It can be fixed by increasing quant_(x|y) option, depending on what direction the line is. I am still not sure if this is bug in my code, or limitation of quantization. I think it's the former, but I still can't pinpoint it yet.

The code on the GitHub actually now support downscaling, but I haven't thoroughly check/test it for correctness yet.

[Groucho2004]

I decided to play around with this a little which usually includes getting the code and compiling it myself. I used VC10/ICL13 with PGO to build the DLL.

Test script:
SetMemoryMax(1700)
LoadPlugin("JincResize.dll")

w = 1280
h = 720
colorbars(width = w, height = h, pixel_type = "yv12").killaudio().assumefps(24000, 1001)
trim(0,99)
fadeio(49)
trim(0,99)
v = last

a = v.Jinc36Resize(1920, 1080)
b = v.Jinc64Resize(1920, 1080)
c = v.Jinc144Resize(1920, 1080)
d = v.Jinc256Resize(1920, 1080)

return a++b++c++d

Results with innocenat's DLL:
Frames processed: 400 (0 - 399)
FPS (min | max | average): 3.965 | 18.09 | 7.212
CPU usage (average): 25%
Thread count: 1
Physical Memory usage (peak): 1328 MB
Virtual Memory usage (peak): 1327 MB
Time (elapsed): 000:00:55.463

Results with my DLL:
Frames processed: 400 (0 - 399)
FPS (min | max | average): 3.998 | 18.29 | 7.263
CPU usage (average): 25%
Thread count: 1
Physical Memory usage (peak): 644 MB
Virtual Memory usage (peak): 643 MB
Time (elapsed): 000:00:55.075

This was tested on a i5 2500K @ 4GHz (on XP, so AVX was not used.)

The speed is more or less the same but the memory usage is less than half with the DLL I built, no idea why.

FYI, Here is the makefile I used to build the DLL:
CPP=@icl.exe
CPP_FLAGS=/MT /EHa /W0 /O3 /Qipo /arch:IA32 /Qprof-use /D "NDEBUG" /nologo

LINK=@xilink.exe
LINK_FLAGS=/dll /nologo

JincResize.dll: JincFilter.obj AvisynthEntry.obj cpuid.obj FilteredEWAResize.obj
$(LINK) $(LINK_FLAGS) JincFilter.obj AvisynthEntry.obj cpuid.obj FilteredEWAResize.obj /out:JincResize.dll

JincFilter.obj: JincFilter.cpp
$(CPP) $(CPP_FLAGS) JincFilter.cpp -c

AvisynthEntry.obj: AvisynthEntry.cpp
$(CPP) $(CPP_FLAGS) AvisynthEntry.cpp -c

cpuid.obj: cpuid.cpp
$(CPP) $(CPP_FLAGS) cpuid.cpp -c

FilteredEWAResize.obj: FilteredEWAResize.cpp
$(CPP) $(CPP_FLAGS) FilteredEWAResize.cpp -c

[innocenat]

I decided to play around with this a little which usually includes getting the code and compiling it myself. I used VC10/ICL13 with PGO to build the DLL.

I use VC12/ICL14 right now. I am surprised it works on WinXP, though, since I did not select vc120_xp as a base platform, though it is statically compiled.

This was tested on a i5 2500K @ 4GHz (on XP, so AVX was not used.)

There are no AVX code anymore, it's AVX2 only now so you require Haswell. I might try to see if SSE2 integer pack/unpack and AVX processing is faster than pure SSE3, but I doubt that.

The speed is more or less the same but the memory usage is less than half with the DLL I built, no idea why.

This commit (https://github.com/AviSynth/jinc-resize/commit/ef39dab27340a82e6d8762a5712af3842f8ef811) is not in the release built yet.

EDIT: Also, FYI my official built are build with /arch:SSE btw, but important functions are #pragma to specific instruction set anyway (which is SSE minimum)

[Groucho2004]

This commit (https://github.com/AviSynth/jinc-resize/commit/ef39dab27340a82e6d8762a5712af3842f8ef811) is not in the release built yet.
I see, that might explain the difference.

Also, FYI my official built are build with /arch:SSE btw, but important functions are #pragma to specific instruction set anyway (which is SSE minimum)
Just checked the ICL13 documentation, "arch:IA32" is the same as "arch:SSE".

[NicolasRobidoux]

If you are looking for a method of minimizing haloing that does not rely on some sort of a limiter, you may want to check http://www.imagemagick.org/discourse-server/viewtopic.php?f=22&t=25935&sid=23bf275e21d4e6e26ab4c5d6db824d9b and the linked discussions on the Luminous Landscape web site, possibly starting with the later posts.

[DTL]

Finally found that I call '2D' processing. Unfortunately current Jinc being Lanczos weighted is not ideal for most high quality work : it is bad for production/downsizing because do not produce 'conditioned' spectrum and not enough for reference display work because has too few sinc taps and additionally weighted with Lanczos window which is with too few taps do not open full potential of sinc kernel for correctly prepared input data. But it have implementated correct '2D/round' resampler engine as I see from quick testing. So we need to add more practical kernels for this resampler - same as used in SinPowResize for production work and same used in SincLin2Resize for reference displaying. And current Jinc is better to rename to some like JLanczosResize.

As for processing speed it is strange to see the runtime-calculated kernel samples. At least for pow2 resize can use static pre-computed kernel and just convolve in 2D matrices using SIMD that is faster. When I do not know what Jinc do I made simple C-based 2D convolution project for Sinc2D resize with static pre-computed kernel as an example. It is on github.
https://github.com/DTL2020/Sinc2D-master

Do developer of JincResize project still active ?

Jinc has to be slower than other linear resamplers because due to how it works (can't be split into 2 separate passes) it has to read and process more source pixels. E.g. compared to Lanczos3, Jinc3 has to read and process 4x as many source pixels.

One thing to keep in mind is that NNEDI3 can only do exact 2x enlargements while Jinc can handle any up- and downscale factor you want.

In practice with too large input arrays the 1pass '2D' processing may be even faster in compare with V+H 1D passes + 2 full memory 90degree rotation as we have in old avisynth resampler.

For 1pass 2D we have read taps*2 lines from mem for convolution and stream 1 line for writing. If taps*2*width is fits into cache the things must run fast. For typical taps around 20 and width up to 10000 we have 10000*40 bytes to cache for planar 8bit encoding.

lolno. Jinc (and all lanczos-based resizer) has hallelujah ringing. But less aliasing, yes.

Jinc is sinc-based filter and it will ofcourse rings if feeded with 'unconditioned'/illegal content. So it is ofcourse not general resize filter for content from unknown sources. But it good for high image quality data processing and reference video monitor emulation. As such monitors are strictly prohibited to mask any ringing or aliasing caused by illegal video data supplied. So for badly conditioned content it is better to use any other simple resizer like bilinear/bicubic/etc.

In the 'perfect world' it is responsibility of image data producer to supply such (compressed to limited samples) video image data that not rings, not aliases and with controlled over/under shooting per producer's taste with 'decompressing' to analog image data with sinc (pure non-weighted sinc) filter. Practically it is upsizing to infinity number digital samples. With upcoming 8K displays we finally can see FullHD 1080 sampled video data with up to 4x restoration closer to analog form - that is usually good enough. It simulate 54 MHz DAC for DVD-player.

[DTL]

Well - I see this thread is not updated but there was significant plugin update. It looks it finally re-written for much more stable 2D resampler core and capable for downsizing.

Now I think using the current 2D resampler core it is good to add all common family of resizers - at least downsizers suitable for production work (having properties for Gibbs-neutralisation) like Gauss kernel and SinPow kernel. So we at least will have a complementary family of 'true-2D' resizers for downsample and upsample work. Because current Jinc resizer as well as Sinc in 1D is not suitable for downsize (antializing/production) work (without additional pre-filtering).

Which version of Visual Studio is required to build current version 1.1.0 ? (https://github.com/Asd-g/AviSynth-JincResize/releases/tag/1.1.0) I tried VS2013 but it looks uses old std library and even can not found std::cyl_bessel_j(1, x) function.

[kedautinh12]

Visual studio ver 16

[DTL]

Visual studio ver 16

It is from VS 2019 software pack ? v16.8.1 is good/enough ?

[kedautinh12]

Yeah ������

[real.finder]

Now I think using the current 2D resampler core it is good to add all common family of resizers - at least downsizers suitable for production work (having properties for Gibbs-neutralisation) like Gauss kernel and SinPow kernel. So we at least will have a complementary family of 'true-2D' resizers for downsample and upsample work. Because current Jinc resizer as well as Sinc in 1D is not suitable for downsize (antializing/production) work (without additional pre-filtering).


IIRC z.lib is do resize as 2D https://forum.doom9.org/showthread.php?t=173986

[DTL]

IIRC z.lib is do resize as 2D https://forum.doom9.org/showthread.php?t=173986

I look into its description at http://avisynth.nl/index.php/Avsresize and do not found how to use special '2D' mode. I do not think it is default processing because typical 2D processing is much slower in compare with typical 1D+1D V+H 'fast video resizers' processing and usually have wide adjustment of 'plane of processing' size because it greatly inflence on processing speed. Like with JincResize it is tap parameter. And it cause tap*tap plane of processing size and speed decreasing. May be as squared number of tap or larger when processing block do not fits into cashes.

[real.finder]

I look into its description at http://avisynth.nl/index.php/Avsresize and do not found how to use special '2D' mode. I do not think it is default processing because typical 2D processing is much slower in compare with typical 1D+1D V+H 'fast video resizers' processing and usually have wide adjustment of 'plane of processing' size because it greatly inflence on processing speed. Like with JincResize it is tap parameter. And it cause tap*tap plane of processing size and speed decreasing. May be as squared number of tap or larger when processing block do not fits into cashes.

https://forum.doom9.org/showthread.php?p=1784190#post1784190 and here https://forum.doom9.org/showthread.php?p=1788992&highlight=z.lib#post1788992

[jpsdr]

Humm... Maybe i'll try to MT this one... Maybe... A quick look at the code, it seems not so hard at first glance, but there is allways tricky things possible... ;)
How is it realy better to standard resizer, NNEDI3 ? Or just different ?

[DTL]

https://forum.doom9.org/showthread.php?p=1784190#post1784190 and here https://forum.doom9.org/showthread.php?p=1788992&highlight=z.lib#post1788992

Hmm - it may be really use some 'one-pass' processing but still uses very visible V+H kernel instead of 'ideally 2D isotropic' as Jinc in this resampler.

Here is simple testing of sort of 'point spread function' of resizer: For given 1-sample image buffer:
https://i2.imageban.ru/out/2020/12/04/cc0be9916e7122d9c3f7b7d9d212096d.jpg



LoadPlugin("avsresize.dll")
ImageReader("1sample_230.bmp")
ConvertToYV12()
z_ConvertFormat(width*10,height*10, resample_filter="lanczos", filter_param_a=10)

It gives (leveled a bit for better ringing viewing)
https://i6.imageban.ru/out/2020/12/04/e1e3445ff03b82cd6dde28d689bbece1.png

And 'true-2D' jinc upsizer gives ideal round response - isotropic for V and H axes as well as with any angled direction.
https://i5.imageban.ru/out/2020/12/04/34102c933aa47eb08cffaed6411b9843.png

[DTL]

Humm... Maybe i'll try to MT this one... Maybe... A quick look at the code, it seems not so hard at first glance, but there is allways tricky things possible... ;)
How is it realy better to standard resizer, NNEDI3 ? Or just different ?

I finally got working built binary today setting Platform Toolset to VS2019 instead of llvm as in the downloadable project. So I can try to modify its kernel to make move to Gauss and SinPow kernels for this resampler. I made quick test with Jinc/Bessel J1(x)/x resampler in my web-scripted kernel-summing testbench and see typical kernels from 1D resamples prepares data good enough for 'Jinc-2D-planar image restoration' with just small parameters tweaking. So I hope I will make some Gauss-kerneled or SinPow-kerneled downsampler in some future using this 2D resampler engine to show difference.

It is different but 'linear mathematics' resizer I think instead of 'non-linear' like NNEDI and many others. It is some step further to 2D space from simple Sinc (that is really spherical Bessel J0 function as I see in the wiki).

The resampler in Jinc is much slower in compare with V+H processing but Jinc (Bessel J1) decays/fades significally faster in compare with Sinc so we will need less taps - may be up to 5..10 will be very good.

For upsampling may be weighting the edge of kernel also required as done in SincLin2 kernel to fix computational errors at the end of kernel that are still visible now. It required additional testing.

[jpsdr]

if (args[src_left_idx + 0].Defined())
out_args->add(args[src_left_idx + 0], "src_left");
if (args[src_left_idx + 1].Defined())
out_args->add(args[src_left_idx + 1], "src_top");
if (args[src_left_idx + 2].Defined())
out_args->add(args[src_left_idx + 2], "src_width");
if (args[src_left_idx + 3].Defined())
out_args->add(args[src_left_idx + 3], "src_height");
if (args[src_left_idx + 0].Defined())
out_args->add(args[src_left_idx + 4], "quant_x");
if (args[src_left_idx + 1].Defined())
out_args->add(args[src_left_idx + 5], "quant_y");


Are you sure of going to 3 to 0...?

[DTL]

It may be simply error like copy-paste lines of code without correct re-numbering ?

[jpsdr]

For now, it's my guess...

[DTL]

One possible speed-up of 2D-convolution with typical 8 or even 10 bit unsigned integer input data: To make not kernel_x_input+output_sum but LUT addition to output sum.

But I do not know is there significant difference with todays CPUs in Mul+Add operation in compare with Add only. It looks only implementation and testing required.

Because for 2D convolution we need to multiply each kernel sample with each input sample but all possible 8bit input samples are only 256 numbers limited count - we can make pre-multiplied 256_x_kernel_size LUT and just read-index this LUT instead or Mul. For even 10 taps 2D-kernel we have 20x20x4byte_float_x_256=about 400 kbyte LUT that is good cacheable on most CPUs in season of 201x years and may be later.

So the main computational line of convolution (from C-routine)
result += src_ptr[lx] * coeff_ptr[lx];
may be replaced with something like
result += LUT[src_ptr[lx]]; //- no multiplication - just cache read and addition

The LUT start pointer is valid for all line of kernel so it may be calculated once per summing of full kernel line if using SIMD ASM processing.

Addition:

I think there may be 2 significally different approaches for 2D convolution. They give same output result but may be very different in speed on different platforms:

1. Each output sample got kernel-weighted and input-area covered by kernel_size (filter size/support size) sum.
2. Each input sample 'casts' (add) kernel weighted by input sample to output buffer.

The 1. needs significant memory-read traffic (to both input buffer and kernel buffer) and produces very small output write traffic to memory (write once - may be uncached). Kernel buffer is read-only and can be easily shared between all cores in multi-core processing. Input memory buffer is also read-only and cached memory image may be shared by many cores proceses neibour input samples. For LUT using I still not understand if it can be used in this approach.

The 2. produces very small read memory traffic for input buffer and produces read_(mul+)add_write traffic for output buffer. If this traffic is good cashed - the actual memory writes depends on CPU memory manager. This approach allows to use LUT for weighting by small number of input variants kernel buffer. But for multi core processing it mostly require each core process far enough input and output memory arrays because for read_(mul+)add_write memory access to output buffer may reqiuire many resources to keep cache coherence between cores. Also this approach allows for easy skip zero input samples processing with simple compare_and_continue. Because zero input sample makes all-zero kernel addition to output buffer and do not changes it.

May it good to test both approaches on todays hardware platforms to compare its processing speed.

As I see from C-resampler subroutine it uses 1. approach:

for (int y = 0; y < dst_height; y++)
{
for (int x = 0; x < dst_width; x++)
{
const T* src_ptr = srcp + meta->start_y * static_cast<int64_t>(src_stride) + meta->start_x;
const float* coeff_ptr = coeff->factor + meta->coeff_meta;

float result = 0.f;

for (int ly = 0; ly < coeff->filter_size; ly++)
{
for (int lx = 0; lx < coeff->filter_size; lx++)
{
result += src_ptr[lx] * coeff_ptr[lx];
}
coeff_ptr += coeff->coeff_stride;
src_ptr += src_stride;
}

if (!(std::is_same_v<T, float>))
dstp[x] = static_cast<T>(lrintf(clamp(result, 0.f, peak)));
else
dstp[x] = result;

[DTL]

Having tests today with different kernels. It looks easy way to enter different kernels into short void Lut::InitLut(int lut_size, double radius, double blur) function.

So I temporary use 'blur' variable as control variable for different kernels in the range 0..10. And also for switching from jinc-kernel (upsizing) to other kernels (downsizing). Just remove (comment-out) its conversion to 1..0.9 range in the function JincResize::JincResize - // blur = 1.0 - blur / 100.0;. So is my tests:

{
auto radius2 = radius * radius;
auto blur2 = blur * blur;

for (auto i = 0; i < lut_size; ++i)
{
auto t2 = i / (lut_size - 1.0);

if (blur == 0.0f) // if blur value !=0 - use non-jinc kernel
{
double filter = sample_sqr(jinc_sqr, radius2 * t2, blur2, radius2);
double window = sample_sqr(jinc_sqr, JINC_ZERO_SQR * t2, 1.0, radius2);
lut[i] = filter * window;
}
else
// lut[i] = pow(2.7, -4.0f * blur * radius2 * t2 * t2); // Gauss kernel (aligned in 0..10 blur-param range)
{ // or some sort of SinPow kernel - required more tweaking of many 'magic-numbers'
float value = radius * t2 * M_PI / blur;

if (value < (M_PI / 2)) lut[i] = pow(cos(value), 2.2);
else
{
if (value < (M_PI/1.2)) lut[i] = -(cos(value*1.3) * cos(value*1.3)) / (2.5 * value);
else lut[i] = 0;
}
}
}
}

[DTL]

I remember about OpenMP and tried to use it in the release version 1.1.0. To use it I rewrite a bit convolution subroutines to make 'dst' and 'meta' variables local for each processing row. So it is now compatible with OpenMP at least with main processing routines. The preparation routines still untouched and single threaded.

Current sources and compiled x64 executable with Intel C++ Compiler Version 2021.1 - https://cloud.mail.ru/public/5nwM/2CBA6o3sc
libiomp5md.dll is required to load JincResize.dll - taken from Intel C++ Compiler redistributable.

Unfortunately the VS2019 can not compile working release build with full optimization enabled. And with partially disabled optimisations it compile very slow executable. Do not know why.

Test script:

LoadPlugin("JincResize.dll")

function Ast2(clip c, int isize)
{
return Subtitle(c, "Text",font="Arial",size=isize,x=5,y=20,halo_color=$FF000000, text_color=$00e0e0e0)
}

BlankClip(100,200,180,"RGB24",25,color=$00202020)

Animate(last, 0,100,"Ast2", 35, 180)

AddBorders(150,140,100,100,color=$00202020)

ConvertToYV24()
GaussResize(width/4,height/4,p=15)
JincResize(width*10,height*10,tap=5)

[DTL]

Here's a comparison image from madVR, upscaling Monsters AG to 4K:


The main sad idea about using either '1D' resizers or '2D' is about full image data processing workflow:

The 2D upsizers (for image viewing) better be used with 2D downsizers (for moving pictures content production). But unfortunately old and possibly current motion pictures industry still do not have standard on motion pictures production downsizer (like digital video camera data source and all other motion picture data source like 3d-render/2d-rasterizer etc). Being inherited from 20-th century partially-digital TV (being digital - sampled on frames in time and on lines in 2d space vertically and analogue inside horizontal lines) it mostly based on 1D digital processing of analogue 1D digital/analogue forms of TV signal. And most tv engineers work in 20 and beginning of 21 century was put in keeping 1D signal in best form in terms of ringing/sharpening. And for 1D processing of 1d video signal (and also 1d audio signal) we have good working Sinc-based workflow and appropriate resizers.

Unfortunately with 2d sampled moving pictures data things goes not very nice if just use 1D+1D V+H approach or even 'radius-based 1D' used. As I read from ads at about end of 20 th century and beginning of 21 century there was some commertial products as data-sources for moving pictures workflows used 'elliptical filters' for sampled data production. It was CG product - font rasterizers. I.e. convertors of vector image/object data into 2d sampled form acceptable for broadcast TV. But the producers of broadcast video cameras and 3d-rendering engines for moving pictures keeps its internal processing for data-downsizing in secret. And publicity available international engineers/expert groups for moving pictures data processing documents also do not covers the exact form of downsizer for 'video'. So each 'video content' provider uses whatever it want and I think mostly based on faster and simplier 1d-solutions.

It the 'free world of video content rippers / community' directly applicable to this internet forum it also means sad things: Because type of downsizer for low-res video rip (or some other way of producing video data) was not and even still not standartized and even not signaled in metadata of provided content so the content viewing person do not straight and correct way of upscaling for viweing. It can use either 1d or 2d based upsizers or 'non-linear' upsizers at its own taste.
Even worse - we still almost do not have any good '2d-production downsizer' for creating low-res rips directly applicable to 2d-viewing-upsizer like this Jinc-resize filter. Because current Jinc in 2D as well as Lanczos in 1D are not applicable for 'conditioned' dowsizing and will produce data causing alising/ringing if attempt to view-upsizing using Jinc or Sinc upsizers.
When persons feed unknown-source video content to Jinc-upsizer they definetly may and usually will got alising/ringing artifacts. And attempt of using AR-filtering is just attempt to fix distorted result from not very correct workflow.

I make some testbench script for future testing of workflows for moving pictures data. I.e. combinations of production dowsizers and viewing upsizers. It have source of 1-point elementary 'video object' being moved in some directions in 2d-space with size (radius/diameter) changed from almost zero to some visible 2d-circle like 2..3 sampling-steps in raduis/diameter. The moving speed is low enough and object get many non-integer positions in 2d space in relation of 2d sampling grid. The best output result of moving pictures handling workflow is - the full round form of object in any frame of animation. Any 'ghosts/ringing' is not allowed. The only allowed is 1 round form undershoot for better visual sharpness. The raising level after end of undershoot higher that base level (0x20) is treated as starting of ringing and not allowed too.

function SourceObject(clip c, float fArg)
{
xpos_rot = c.width/3+(10*Sin(Pi()*fArg/(50)))
ypos_rot = c.width/3-(10*Cos(Pi()*fArg/(50)))

xpos_diag = c.width*2/3+(10*Sin(Pi()*fArg/(50)))
ypos_diag = c.width*2/3-(10*Sin(Pi()*fArg/(50)))

xpos_h = c.width*2/3+(10*Sin(Pi()*fArg/(50)))
ypos_h = c.width/3

xpos_v = c.width/3
ypos_v = c.width*2/3+(10*Sin(Pi()*fArg/(50)))


ch_size = (Int(fArg/100))+1

xpos_rot_i=Int(xpos_rot)
ypos_rot_i=Int(ypos_rot)

xpos_diag_i=Int(xpos_diag)
ypos_diag_i=Int(ypos_diag)

xpos_h_i=Int(xpos_h)
ypos_h_i=Int(ypos_h)

xpos_v_i=Int(xpos_v)
ypos_v_i=Int(ypos_v)


c=Subtitle(c, CHR($95),font="Arial",size=ch_size,x=xpos_rot_i,y=ypos_rot_i,halo_color=$FF000000, text_color=$00e0e0e0)
c=Subtitle(c, CHR($95),font="Arial",size=ch_size,x=xpos_diag_i,y=ypos_diag_i,halo_color=$FF000000, text_color=$00e0e0e0)
c=Subtitle(c, CHR($95),font="Arial",size=ch_size,x=xpos_h_i,y=ypos_h_i,halo_color=$FF000000, text_color=$00e0e0e0)
c=Subtitle(c, CHR($95),font="Arial",size=ch_size,x=xpos_v_i,y=ypos_v_i,halo_color=$FF000000, text_color=$00e0e0e0)

return c

}


BlankClip(30000,800,800,"RGB24",25,color=$00202020)

Animate(last, 0,30000,"SourceObject", 0, 30000)

ConvertToYV24()

# production-downsizing
#SinPowResizeMT(last,width/10,height/10,p=3.2)
#GaussResizeMT(last,width/10,height/10,p=11)
#LanczosResizeMT(last,width/10,height/10)

# restoring vewing-upsizing
#SincLin2ResizeMT(width*8,height*8,taps=15)
#SincResizeMT(width*8,height*8,taps=15)
#JincResize(width*8,height*8,tap=5)

#Levels(25, 1, 55, 0, 255) # can be used for better viewing of processing errors


To use it uncomment one of line of production downsizer and one of viewing-upsizer. The /10 in downsizers gives 0.1 sample grid movings of test object. The upsample ratio may vary as user need. At higher upsample ratios the defects of processing usually better visible.

The 'ideal non-ringing/non-alizing' both 1d and 2d compatible production downsizer example is GaussResize(last,width/10,height/10,p=11) i.e. p=11 at gauss at current avisynth. But it is very soft. And do not provide 'undershoot' that helps to raise visual sharpness a bit. If trying to make more sharp content at given resolution it easy to got ringing/aliasing. So the task for developer some workflow for limited digital frame size is to use such combination of downsizer+upsizer that gives best possible visual sharpness at limited or system-fixed digital frame size in samples (usually called 'pixels') with limited distortions like aliasing/ringing.

[DTL]

NNEDI3 is quite great, IMHO. It does have disadvantages, too, though. E.g. in some image areas (trees, grass, leaves) it can produce fractal like artifacts. That doesn't happen with Jinc. Also NNEDI3 is a lot slower than Jinc, when using madVR. I'm not sure how speed compares in AviSynth, though. Finally, NNEDI3 can only upscale by exactly 2.0x, while Jinc can up and downscale with any factor you like. So although NNEDI3 is great, there's a place for Jinc, too, IMHO.

Current fastest version of planar convolution with Jinc64 (taps=4, may be close to Jinc4 in madVR it exist) kernel runs at about film-realtime with upscaling Full HD->8K at 6 cores 3.4 GHz CPU (near 2018 year of production, 2x256bit FMA processing units per core) with AVX2+FMA processing on host CPU only. There is some place for speed improvement and moving to AVX512. I not sure if NNEDI3 have close progress in performance for using in displaying motion pictures content.

For currently theoretical performance: 2x256 FMA units can give sustained stream of 16 float32 FMAs per clock so 4 core typical 3..4 GHz CPU I think can give FMA processing rate up to 192..256 G-FMAs per second.

For upsampling 1920x1080 YV12 frame of about 3e6 input samples with taps=4 kernel to 8K needed about 1024 FMA ops per input sample so about 3 G-FMAs per frame. For 25fps - about 76 G-FMAs per second.

Currently with 6 core CPU and about 326 G-FMAs per second theoretical peak performance achieved about 20 fps 1920x1080 YV12 input frame processing. It required about 60 G-FMAs per second. So practical performance looks like about 0.18 of possible peak and there is place for some improvement. May be even about 2x more to 0.5 of peak performance.

As I see moving to AVX512-capable core is not still added to FMA performance because it only have same 2x256 bit FMA units.
Though at intel IntrinsicsGuide there is strange table for _mm512_fmadd_ps performance:
IceLake Throughput 1 CPI
SkyLake and Knights Landing Throughput is 0.5 CPI - like real true 2x512 FMA units available. Needs testing.
Though Knights Landing was Xeon Phi non-consumer-grade CPUs. From Wiki: Xeon Platinum, Gold 61XX, and Gold 5122 have two AVX-512 FMA units per core. Xeon Gold 51XX (except 5122), Silver, and Bronze have a single AVX-512 FMA unit per core. So looks like still no 2x512 FMA units per core at consumer-grade cores for now.

[DTL]

Well - finally some milesone in performance: Function for planar/true-2d upsampling with avx(2?) instructions with multiplier=4 and taps=4 running as single thread and with multithreading frame-based in Avisynth+ reach about 33% of theoretical G-FMAs performance of CPU like i3-9100T and i5-9600K. It allow to upsize FullHD 1920x1080 YV12 frame to 8K with taps=4 with about 45fps at i5-9600K running at 4.3 GHz in 6 threads. The most performance boost in compare with previous AVX FMA 8-samples engine looks like reached because of smaller memory using with 'circulating buf' of size only out_width_x_KernelSize instead of full output frame size in floats. With C-code upsample using of circulating buf makes very small difference may be because of too slow CPU processing without good AVX FMA engine. I think of making multithreaded of the '_cb' functions in the future - I still poor in C++ so need to found how to make array of vectors for each thread. Also some work need for aligning edges of processed frame stripes by each thread. Also some more ideas for future optimizations may add to performance. I made some attempt to release binaries at https://github.com/DTL2020/AviSynth-JincResize/releases/tag/v0.3-alpha for testing.
Only x64 builds available because need for all 16 ymm registers for processing.

Test script with AVX CPU and Avisynth+:

LoadPlugin("JincResize.dll")
BlankClip(1000, 1920,1080,"YV12")
JincResize(width*4,height*4,tap=4,opt=2,threads=1,ap=2)
Prefetch(_num_cores_)

[kedautinh12]

thanks

[DTL]

How is it realy better to standard resizer, NNEDI3 ? Or just different ?

Got EBU tech 3325 test pattern 7 for testing scalers of broadcast control monitors and made test with madVR 2x upscalers and Sinc and Jinc from Avisynth plugins.

Results combined to 1 image file: https://i1.imageban.ru/out/2021/02/06/51daae063fda26b70933ff03581cfc0a.png
https://i1.imageban.ru/thumbs/2021.02.06/51daae063fda26b70933ff03581cfc0a.png (https://imageban.ru/show/2021/02/06/51daae063fda26b70933ff03581cfc0a/png)

In compare with some slow in processing with high number of taps sinc and jinc it looks EBU engineers live in separate perfect world and not very good connected to real sad world of digital moving pictures content production companies. So most of build-in madVR upsamplers works bad enough on EBU scaler test dataset and mostly designed to fight against ringing on real world badly 'conditioned' digital moving pictures data while trying to keep sharpness as great as possible.

Also the last pictures shows significant difference between sinc and jinc upsamplers at this EBU scaler test pattern - jinc filter-out all 'diagonal' frequencies outside 1/sample_step (1D Nyquist limit). Both sinc and jinc with high enough taps values restore vertical and horizontal frequencies good. Looks I need to add ratio 2x with taps 8 AVX-enchanced multi-sample function too. It will fit with ymm AVX registers with about 8 input samples processing without bulk memory load-store operation.

[kedautinh12]

JincResize 2.1.1
https://github.com/Asd-g/AviSynth-JincResize/releases

[kedautinh12]

JincResize 2.1.2
https://github.com/Asd-g/AviSynth-JincResize/releases