How good/bad is this noise removal of a single image using avisynth?

[cagali]

Pardon me if this is not the appropriate forum for asking things about noise removal of images instead of video clips.

This process uses 1 single avs script only. (1 file, not 1 line :D)

Please comment on how good or bad this removal looks.
It will be greatly appreciate if you could provide some idea or even a complete script for a better result.
:thanks:

Requirement:
1. remove most of the dots.
2. keep most of the sharpness of texts.
3. this script already takes 5 minutes to run for 1 image, so speed is not the first priority, quality is.

Source:
https://i.imgur.com/nea6I15.jpg

Result:
https://i.imgur.com/0AFVVpR.jpg

[lansing]

This is not noise, this is called screening and there is no avisynth filter that can correctly fix this.

You'll need Sattva Descreen (http://www.descreen.net/eng/soft/descreen/descreen.htm).

[cagali]

This is not noise, this is called screening and there is no avisynth filter that can correctly fix this.

You'll need Sattva Descreen (http://www.descreen.net/eng/soft/descreen/descreen.htm).

Thanks. I think descreening is the proper way that I should look into.

I tuned my avs script a little bit.
I think I get half of the area correct.
With a bit more time I think I can get the remaining half correct too.
https://i.imgur.com/rv5xQbI.png

[lansing]

I would say don't even bother, this field is so specialized that you would just want to "leave it to the pro".

[cagali]

I would say don't even bother, this field is so specialized that you would just want to "leave it to the pro".

Thank you for trying to help.
I hope the pros will read this thread by chance.

[cagali]

x1 original
https://i.imgur.com/zdInHcU.png

x4 original
https://i.imgur.com/F6BjBmW.png

The screening pattern on the original scan was probably more regular.
However the version I have seems to be scaled down, so the pattern is not that regular.
Are there any descreen filters that can still process these kind of images probably?
(for video-related topics I think doom9 is the proper place, for these image-related topics is there any similar forum? reddit?)

x1 descreen filter in gimp, default parameters
https://i.imgur.com/FQmJiI8.png

x4 denoising with avisynth filters
https://i.imgur.com/J4ujwip.png

[poisondeathray]

It looks good

Instead of default gimp parameters, you can use a mask (e.g. luma mask) with layers to improve the gimp filter, just like you can use a mask in avisynth for the text. Because it's "black" text it's "easier" to isolate

[lansing]

Thank you for trying to help.
I hope the pros will read this thread by chance.

What I mean is just buy their software if you really need a decreen filter.

[cagali]

What I mean is just buy their software if you really need a decreen filter.

Yeah from the sample of the software, I think it will work well if the scans that I have are original, not scaled down.

"Sattva Descreen plug-in for Adobe Photoshop
...
You need just provide the plug-in with a hi-resolution scan, usually 600...1200 pixels/inch. "

The scans I have are just 300 dpi, 2000px x 3000px.

[lansing]

Yeah from the sample of the software, I think it will work well if the scans that I have are original, not scaled down.

"Sattva Descreen plug-in for Adobe Photoshop
...
You need just provide the plug-in with a hi-resolution scan, usually 600...1200 pixels/inch. "

The scans I have are just 300 dpi, 2000px x 3000px.

I think the minimum resolution requirement for the plugin is 300 dpi, so you should be fine.

[cagali]

I think the minimum resolution requirement for the plugin is 300 dpi, so you should be fine.

Really? That's great.

Then next time I could try to use that PS plugin.

As I did not cook and eat outside for the whole week, this time I think I will stick to the gimp descreen plugin, also FFT based.
:thanks:

[cagali]

It looks good

Instead of default gimp parameters, you can use a mask (e.g. luma mask) with layers to improve the gimp filter, just like you can use a mask in avisynth for the text. Because it's "black" text it's "easier" to isolate

Thank you!
Maybe it is afterall better to use an image software for image processing? What do you think?

I know a little bit avisynth scripting but nothing about gimp scripting. Maybe not that difficult anyway.

What I originally planned to do is, do some basic descreening/noise removal in 1 avs, and then in the same script use waifu2x to do some tiny final clean-up.

[StainlessS]

You can do a D9 search on POSTS(Not Threads) for "Mogrify", for AVS writing/reading images for ImageMagick processing .
(if you can find some descreen stuff for that, there is some IM stuff for creating halftone/screening, also think Martin53 did something to create screening in AVS).

[See specifically entries under CallCmd, RT_Call (RT_Stats) threads, for writing, reading and calling ImageMagick Mogrify.exe]
Perhaps can access GIMP using similar methods(dont know, never tried it).

EDIT: This guy has a lot of ImageMagick scripts, think we used some things from there, dont know if anything of interest.
http://www.fmwconcepts.com/imagemagick/retinex/index.php

EDIT: Heres an interesting one if VideoFred is watching:- http://www.fmwconcepts.com/imagemagick/removecolorcast/index.php

EDIT: Above linked scripts run under bash on linux, but note below
Windows users can run my scripts, if they install the unix Cygwin system for Windows and then install the IM Cygwin binary.

A very detailed description of Cygwin installation, relevant script syntax and examples has been provided by user snibgo on his
web site at http://im.snibgo.com/cygwin.htm. A brief outline that he wrote can be
found at http://www.imagemagick.org/discourse-server/viewtopic.php?f=26&t=25910&start=30#p113836. Many thanks to snibgo for this excellent document.

Another guide to using bash shell scripts under Cygwin on Windows has been provided by Wolfgang Hugemann in his excellent page, Usage Under Windows


Windows 10 (64-bit) users can also run my scripts without the need for Cygwin. Bruce Rusk reports: All that was need was the installation of the Unix bc tool and
Imagemagick through apt/apt-get. Everything else worked right out of the box, no changes to PATH variables required. See How to Install and Use the Linux Bash
Shell on Windows 10.

[cagali]

You can do a D9 search on POSTS(Not Threads) for "Mogrify", for AVS writing/reading images for ImageMagick processing .
(if you can find some descreen stuff for that, there is some IM stuff for creating halftone/screening, also think Martin53 did something to create screening in AVS).

[See specifically entries under CallCmd, RT_Call (RT_Stats) threads, for writing, reading and calling ImageMagick Mogrify.exe]
Perhaps can access GIMP using similar methods(dont know, never tried it).

EDIT: This guy has a lot of ImageMagick scripts, think we used some things from there, dont know if anything of interest.
http://www.fmwconcepts.com/imagemagick/retinex/index.php

EDIT: Heres an interesting one if VideoFred is watching:- http://www.fmwconcepts.com/imagemagick/removecolorcast/index.php

EDIT: Above linked scripts run under bash on linux, but note below

Thank you for your effort for gathering these information!!

Now I remember that a long time ago I read about noise removal in imagemagick.
Building imagemagick from source with fft support...
Creating spectrum mask from the phase and magnitude of the image...

Fred's scripts collection is really a big one, I think it is enough for me to spend a few nights.
:)

[JReiginsei]

You can do this with the GIMP. Look at this video on Youtube https://youtu.be/3137dDa6P4s

[Seedmanc]

I used to deal a lot with scans so I can probably make a few suggestions here.

As mentioned earlier this is a moire/screen effect, unlike the film noise it has a couple of fixed frequencies which makes it possible to remove it using those frequency peaks on FFT, instead of going in blindly. If you really want to do it with Avisynth there are filters like DeFreq (http://avisynth.org.ru/defreq/defreq.html) or Vcmohan's FFTQuiver (http://www.avisynth.nl/users/vcmohan/FFTQuiver/FFTQuiver.htm) (not sure if that one exactly, he has loads of various scripts). I only used DeFreq, it's tricky to use since you have to describe the FFT areas for removal in text form. The only upside of doing it in Avisynth is batch-processing of loads of similar scans taken from the same source, because then the FFT discrepancies would be the same and it'll be enough to configure the filter once, then follow up with a denoiser as well. At some point I even tried to write a JS helper for it that would at least allow the user to select the areas with a mouse and generate AVS based on that as well as generating a way to import multiple images that have a slightly different size into a single video stream, but's still not very convenient.

It would make a lot more sense to use a specialized image editing software here. Considering that you're probably using a denoiser like Waifu2x without GPU acceleration (since it take 5 mins per image and the AVS ver indeed doesn't have it), you're not going to do batch-processing anyway. Here I can suggest a decent free alternative to photoshop plugins - Image Analyzer (http://logicnet.dk/Analyzer/), which, among other things, has a special tool for editing the frequency spectrum - the Frequency Domain Filter. I wrote a manual (http://meesoft.logicnet.dk/Analyzer/help/FrequencyDomainFilter/FrequencyDomainFilter.html)for that tool in application to scan retouching. Keep in mind, though, that it's still a long manual process and even after removing most of the noise noise you'll be left with whatever random noise there was in the image. Still, for best results I would recommend applying FDF before the conventional denoisers, otherwise you'll often be left with lots of moire remaining in the image or you'll have to use extreme noise removal presets and lose the details.
As for a followup general denoiser for images I can recommend NeatImage (https://ni.neatvideo.com/). Though it's not free it has a decent trial functionality and you can always search for the free alternatives of it.

[cagali]

You can do this with the GIMP. Look at this video on Youtube https://youtu.be/3137dDa6P4s

Thank you. The result looks pretty nice.

Actually one can do this in programming ways.
For example in imagemagick, generate a spectrum mask with a threshold (so one does not need to draw black spots on the FFT image manually), then multiply it with the magnitude of the image, and then... one gets a filtered, cleaner image.

[cagali]

I used to deal a lot with scans so I can probably make a few suggestions here.

As mentioned earlier this is a moire/screen effect, unlike the film noise it has a couple of fixed frequencies which makes it possible to remove it using those frequency peaks on FFT, instead of going in blindly. If you really want to do it with Avisynth there are filters like DeFreq (http://avisynth.org.ru/defreq/defreq.html) or Vcmohan's FFTQuiver (http://www.avisynth.nl/users/vcmohan/FFTQuiver/FFTQuiver.htm) (not sure if that one exactly, he has loads of various scripts). I only used DeFreq, it's tricky to use since you have to describe the FFT areas for removal in text form. The only upside of doing it in Avisynth is batch-processing of loads of similar scans taken from the same source, because then the FFT discrepancies would be the same and it'll be enough to configure the filter once, then follow up with a denoiser as well. At some point I even tried to write a JS helper for it that would at least allow the user to select the areas with a mouse and generate AVS based on that as well as generating a way to import multiple images that have a slightly different size into a single video stream, but's still not very convenient.

It would make a lot more sense to use a specialized image editing software here. Considering that you're probably using a denoiser like Waifu2x without GPU acceleration (since it take 5 mins per image and the AVS ver indeed doesn't have it), you're not going to do batch-processing anyway. Here I can suggest a decent free alternative to photoshop plugins - Image Analyzer (http://logicnet.dk/Analyzer/), which, among other things, has a special tool for editing the frequency spectrum - the Frequency Domain Filter. I wrote a manual (http://meesoft.logicnet.dk/Analyzer/help/FrequencyDomainFilter/FrequencyDomainFilter.html)for that tool in application to scan retouching. Keep in mind, though, that it's still a long manual process and even after removing most of the noise noise you'll be left with whatever random noise there was in the image. Still, for best results I would recommend applying FDF before the conventional denoisers, otherwise you'll often be left with lots of moire remaining in the image or you'll have to use extreme noise removal presets and lose the details.
As for a followup general denoiser for images I can recommend NeatImage (https://ni.neatvideo.com/). Though it's not free it has a decent trial functionality and you can always search for the free alternatives of it.

It is really amazing that you find out so many things that I want to do, but not mentioned in the first post. XD

I am going to try the methods you suggested in an image software, see how clean I can get.
If it is much better than what I get using avs, I am going to pick the approach.
Otherwise I will stick with the avs way, coz there is a small batch to go, less than a 100 images.

Thank you for you suggestions!!

[WaxCyl]

I use this FFT action/plugin for pattern removal:
https://www.retouchpro.com/forum/tools/software/39278-ft-pattern-suppressor
It is FREE!

[lansing]

I use this FFT action/plugin for pattern removal:
https://www.retouchpro.com/forum/tools/software/39278-ft-pattern-suppressor
It is FREE!

This is pretty much unusable, I have a 4000 x 5600 color scan and this action script took up 5.3G of my drive as scratch disk and 7G and counting of RAM until my photoshop crash because of out of memory.

[Seedmanc]

a 100 images.

That's quite a batch already. In that case you might still profit from using Avisynth and maybe my helper tools (https://www.dropbox.com/s/ftv36c4s009x7j3/defreqjs-helper.zip?dl=0) as well.
There are two pages, one for generating a script that loads multiple images of different resolutions into a single videostream (by overlaying them over a blank background of maximum resolution, since you can't load variable-sized frames into AVS), optionally individually or globally cropping them (since as you might have learned already from my FDF tutorial it is important to get rid of sharp edges in image before applying it, to avoid distracting features in the spectrum). "Mirror fill" will attempt to avoid edges between the image and background as well.
Then once you got that script you'll need to use DeFreq plugin in "show=2" mode after playing the script once (to average spectrums). Save that image and open in the second page, you'll be presented with a way to select areas of the spectrum which will generate batches of DeFreq calls with appropriate values.

As you can see this is quite cumbersome so it's only worth doing if you have a really large batch which you absolutely need to do the FDF-way. If you're satisfied with the results in your 1st post you might be better off batch-processing images with the conventional denoisers. I can recommend waifu2x-caffe (https://github.com/lltcggie/waifu2x-caffe/releases) with GPU acceleration if you have NVidia.

[WaxCyl]

This is pretty much unusable, I have a 4000 x 5600 color scan and this action script took up 5.3G of my drive as scratch disk and 7G and counting of RAM until my photoshop crash because of out of memory.

Attached photo is from a newspaper, before and after pattern removal and minor other touch-ups. Please note the lack of detail loss.

https://www.retouchpro.com/forum/too...ern-suppressor
I have no affiliation to this website, just a happy user!

I have been using FT Pattern Suppressor for a few years. It is very powerful and effective. I have set Photoshop to use 17G of RAM and I have 100G free space on my SSD scratch disc. The photo attached would take one minute to process. The software is performing Fourier Transformation which is very CPU intensive.
I think Image Analyser can also do this if Photoshop is not available.