Zsmooth - cross-platform smoothing functions written in Zig!

[Adub]

Hello Doom9!

After being part of this community for many years, I finally got the time to start writing my own plugins, and so today I present something I've been working on for the past few months.

Zsmooth - Cross-platform smoothing functions written in Zig

Current Version: 0.12 (https://github.com/adworacz/zsmooth/releases/tag/0.12)

Currently implemented functions

CCD
Clense
DegrainMedian
FluxSmooth
InterQuartileMean
Median
RemoveGrain
Repair
SmartMedian
TemporalMedian
TemporalRepair
TemporalSoften
TTempSmooth
VerticalCleaner


All functions take 8-16bit integer, and 16-32 bit float clips, in RGB, YUV, and GRAY formats. Parameter scaling is optional, based on a 'scalep' parameter, so you can choose to pass 0-255 values or get more specific.

Every function has SIMD support on x86_64 and ARM processors, thanks to the power of Zig's Vectors. This means that every function should be as fast, if not faster than existing plugins, and it should be fast everywhere.

Note: 16bit float support is a work in progress. All functions take it, but some are very slow due to some compilation bugs. I'll be working on ironing these out over time.

Please feel free to test these functions out and report back! I've been testing these myself over the past few months, and I *think* I've ironed out the bugs, but please report back any issues that you find, or major drops in performance compared to existing plugins. Also, I've only been testing on Linux, so I'm honestly curious to hear how things are on Windows and Mac.

Benchmarks
https://github.com/adworacz/zsmooth/blob/master/BENCHMARKS.md


Bugs / Feature Requests
Feel free to either open an issue on Github or post in this thread.

https://github.com/adworacz/zsmooth/issues

Downloads
All releases are on Github. I've provided pre-compiled binaries for Windows, Mac and Linux, with the latter offering x86_64 and aarch64 (ARM) binaries.

https://github.com/adworacz/zsmooth/releases

[Julek]

wow, incredible work!
Your code will help me a lot, since i'm still a beginner in the dev world,
I was working with RemoveGrain too, only @Vectors code, but i was stuck at mode 20 hahaha

If you happen to use Discord, we have a vapoursynth devs server:
https://discord.gg/XTpc6Fa9eB
it would be great to have you there

[Myrsloik]

Interesting. I would however request that you don't spread TemporalSoften(2) around. That function was only ever used with the maximum thresholds and was simply a cargoculted discount AverageFrames. Promotes bad scripts.

But very interesting to see zig used for things.

[Julek]

About printf memory leak (https://github.com/adworacz/zsmooth/blob/c4148f94d67e572d0c1fe068afafb8cb038aff3e/src/common.zig#L330-L332), you can avoid this by using bufPrintZ instead of allocPrintZ.
I've added a function (https://github.com/dnjulek/vapoursynth-zig/commit/91c4f5249ac84416d654c1c5dc8b4b8dd90a7c2b)to the zig module and updated the example (https://github.com/dnjulek/vapoursynth-zig/commit/116d94b96c15694d0fbc704304ddcfe4c3eb48e1), so you can import from vsh now.

[Adub]

@Julek - Thank you for your comments, and help in creating the vapoursynth-zig project. It definitely made my work a lot easier, and I'm happy to contribute as well. I just opened a few issues that I noticed while looking at the code, and I can submit a PR with fixes once I get a spare moment.

Also, good call on the memory leak thing. I'll look at updating.

@Myrsloik - Good to know on TemporalSoften / AverageFrames. I have seen some uses of TemporalSoften with the threshold (https://github.com/Jaded-Encoding-Thaumaturgy/vs-rgtools/blob/a9a5fd235df86f097c4669dea89de14c3a0d613f/vsrgtools/func.py#L102-L105), but I'll keep in mind that AverageFrames exists as well.

And yeah, Zig is a surprisingly excellent language for video processing filters, in my opinion. The C-interop, compile-time language execution, native SIMD vector support, comptime type passing, f16 support, etc etc etc make for a really nice development environment.

[Adub]

FYI - Selur has already found a bug (thank you!) in TemporalMedian:

https://github.com/adworacz/zsmooth/issues/1

I suspect I know the issue, and will be testing/fixing over the next day or so.

[Julek]

It definitely made my work a lot easier, and I'm happy to contribute as well. I just opened a few issues that I noticed while looking at the code, and I can submit a PR with fixes once I get a spare moment.

I'm glad my code was useful for something, I was making a zig module for AVS too, but this one is much more complicated because AVS include is a mess, I'll have to turn the enums into variables because they have repeated values inside the same enum and Zig doesn't allow this.
I'll wait for your PRs :thanks:

A question about the benchmark, how was your RGVS compiled? Is the SSE code active? Because in my tests the zig version has pretty much the same speed.
My RGVS code in zig was also getting the same performance as the c++ plugin with SSE.

[Adub]

Version 0.3 is up: https://github.com/adworacz/zsmooth/releases/tag/0.3

This should fix all stride + vector related handling (turns out I had a few bugs wrt non-mod 32 handling).

[Adub]

I'm glad my code was useful for something, I was making a zig module for AVS too, but this one is much more complicated because AVS include is a mess, I'll have to turn the enums into variables because they have repeated values inside the same enum and Zig doesn't allow this.
I'll wait for your PRs :thanks:

A question about the benchmark, how was your RGVS compiled? Is the SSE code active? Because in my tests the zig version has pretty much the same speed.
My RGVS code in zig was also getting the same performance as the c++ plugin with SSE.

Good to know that you're working on an Avisynth version! I really appreciate your efforts here, it will make writing "dualsynth" filters that much easier.

I'll work in a PR with fixes to vapoursynth-zig in the next day or so.

As for my RG benchmarks - I simply compiled my version locally using "zig build -Doptimize=ReleaseFast". I've noticed that the performance difference can really depend on the CPU in use. On my laptop I see a massive speedup, but on my desktop the performance difference between Zsmooth's version of RG and RGVS seems negligible. Floating point is still generally faster with Zsmooth due to using single precision vs RGSF's double precision.

[Selur]

This should fix all stride + vector related handling (turns out I had a few bugs wrt non-mod 32 handling).
still broken, just in another way now (see details on github)

[Adub]

Version 0.4 is up: https://github.com/adworacz/zsmooth/releases/tag/0.4

Should have fixes for all bit depths, and now uses Zig slices for array bounds checking in debug mode. Also added unit tests to assert the stride handling behavior as a double check.

I've tested with RGB, YUV420, YUV422, and YUV444, and I can't replicate any more artifacts (green pixels or black bars).

Hopefully everything is fixed this time. Thank you for bearing with me as I iron out these kinks.

[Selur]

Sadly it's still broken. :( (more details over at github)

[Adub]

Version 0.5 is out with the last of the stride fixes - https://github.com/adworacz/zsmooth/releases/tag/0.5

Big thank you to NSQY, Selur, and Julek for testing and helping to diagnose the issue.

[Adub]

Minor update: I'm working on a few updates for 0.6, mainly adding DegrainMedian support (mode 0 is *much* faster than the existing plugin), along with some minor dependency updates and Zig version bump.

After that I'll work on adding Dogway's IQMV (Inter Quartile Median) filters, and then maybe CCD after. I still need to finish the RemoveGrain functions (Repair and Cleanse) as well.

[Adub]

Version 0.6 is now available, including prebuilt binaries for x86_64 and aarch64 Mac - https://github.com/adworacz/zsmooth/releases/tag/0.6


* Add DegrainMedian implementation.
* Optimize TemporalMedian even more. Latest tests show a bump from 340fps to 413fps in
single core tests on my laptop. Essentially I heavily reduced branching calculations by
moving diameter calculations to comptime. This does blow up the size of the binary by about double,
from about 850kB to 1.6MB, but oh well.
* Refactored internal common code significantly, and introduced a helper type called Grid for operating on a 3x3
grid of pixels as either scalars or vectors.
* Added more testing to many of the common functions.
* Add generic builds for Mac OS, on x86_64 and aarch64 architectures.

[Selur]

Nice! Thanks for the update!

[Adub]

Just so I don't pollute https://forum.doom9.org/showthread.php?t=186119, bit of a status update:

1. I've started my work on implementing Repair. It's going extremely well, with some pretty insane speedups over the older plugins, especially RGSF. We're talking a frequent 10-20x improvement. Again, not a typo.
2. Community members have requested that I change how Repair (and potentially RemoveGrain...) is implemented. Prior implementations of RG/Repair simply copy the edge pixels (so first/last row/column) to make the underlying algorithms easier to write. Since those plugins were written an alternative technique has arisen for filling in the requisite information for edges, which is "mirroring". I'll be experimenting with this approach as it makes sense and I should be able to implement it in the next version or two. More details can be found on the Github issue: https://github.com/adworacz/zsmooth/issues/6
3. As part of investigation into #2, I've uncovered some more speedups in RemoveGrain as part of a refactor I started. I'm seeing a good 10%+ improvement in RG speeds (and I think I'm bottlenecking on something else now so it could be even higher).

[Selur]

Looking forward to it, faster RemoveGrain and Repair would be nice. :)

[Selur]

btw. if possible would be nice if you could create Vapoursynth versions of TemporalRepair and AutoLevel which atm. lack any Vapoursynth port :)

[Adub]

TemporalRepair seems reasonable. AutoLevel seems outside the scope of Zsmooth in particular, but I'll consider it for another plugin at some point.

I've added TemporalRepair to a pinned issue on Github, so feel free to up vote or add others there: https://github.com/adworacz/zsmooth/issues/7

[Selur]

Thanks, for considering. :)
(another thing that could always be faster is mvtools and all its functions)

Cu Selur

[Selur]

Seeing that Repair, VerticalCleaner and Clense were added any eta for a new release? :)

Cu Selur

[Adub]

Version 0.7 is now available: https://github.com/adworacz/zsmooth/releases/tag/0.7

A ton of updates on this one, but in short, we essentially have full RGVS/RGSF parity now. And some pretty insane speed improvements, if I my testing hasn't completely lied to me.

Also RemoveGrain + Repair now use a mirror padding to ensure that edge pixels are processed properly. This came out of a community request, and luckily wasn't too tricky to implement.

Also, we now have builds from pretty much every operating system and architecture at this point.

Essentially, I don't think there's a reason to use RGVS/RGSF any more...

I'll be looking at adding the remaining modes from Avisynth's RgTools over time (and based on demand), with TemporalRepair being next on my radar, per Selur's request.

[Adub]

Seeing that Repair, VerticalCleaner and Clense were added any eta for a new release? :)

Cu Selur

Haha, basically now.

[Selur]

Nice! Got it! Thanks, will do some testing tomorrow. :)

Cu Selur

[Adub]

Version 0.8 is up: https://github.com/adworacz/zsmooth/releases/tag/0.8

This is a minor release, with one bug fix for TemporalSoften, and mostly focused on cleaning up build artifact names to make things easier for everyone to consume.

[Selur]

nice, thanks for the update.

[Adub]

Version 0.9 is up: https://github.com/adworacz/zsmooth/releases/tag/0.9

InterQuartileMean, TTempSmooth are now implemented, along with various bug fixes and minor feature additions like "planes" support for FluxSmooth and "-1" scene change threshold to enable reuse of existing scene change properties instead of forcing the call to "misc.SCDetect" under the hood.

[Selur]

Nice! Thanks for the update, away from computer for the rest of the week, but will test next week. :)

Cu Selur

[Adub]

Version 0.10 is up - https://github.com/adworacz/zsmooth/releases/tag/0.10

Most important changes are fully implemented InterQuartileMean and Median filters for 3x3, 5x5, and 7x7 grids.

Also, all float processing is now handled in an "optimized" fashion, with a potential loss in accuracy as a result. This is the exact same effect as `-ffast-math` in GCC/Clang. I made this change configurable however, so it's trivial to produce a build that utilizes more exact (if possibly slower) float processing.

There's also a few optimizations and code cleanups in several other filters, but no one should really see much of a difference.

See the CHANGELOG (https://github.com/adworacz/zsmooth/blob/master/CHANGELOG.md) for details.

[Selur]

Thanks for the update.
InterQuartileMean <- is there any known Vapousynth scripts that use this atm. ?

[Adub]

InterQuartileMean <- is there any known Vapousynth scripts that use this atm. ?

Not that I'm aware of. This is the first time we've had this functionality, unless I missed another implementation somewhere.

This specific implementation came out of Dogway's efforts with his "ex_median" function, which I've not seen replicated anywhere else.

[Adub]

0.11 is up: https://github.com/adworacz/zsmooth/releases/tag/0.11

Simple bug fix release - IQM and Median weren't respecting their default values for "radius".

[Selur]

Thanks

[Adub]

Version 0.12 is up (https://github.com/adworacz/zsmooth/releases/tag/0.12)

Pretty cool feature release if I do say so myself. We now have SmartMedian, TemporalRepair, and CCD (with temporal support).

See the link above and the official changelog (https://github.com/adworacz/zsmooth/blob/master/CHANGELOG.md) for more detail.

Edit: As always, benchmarks can be found in the repo. (https://github.com/adworacz/zsmooth/blob/master/BENCHMARKS.md)

[Selur]

Nice! Thanks for the update, will do some testing tomorrow. :)

Cu Selur