Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 12th January 2019, 06:22   #1  |  Link
cagali
Registered User
 
Join Date: May 2014
Posts: 42
How good/bad is this noise removal of a single image using avisynth?

Pardon me if this is not the appropriate forum for asking things about noise removal of images instead of video clips.

This process uses 1 single avs script only. (1 file, not 1 line )

Please comment on how good or bad this removal looks.
It will be greatly appreciate if you could provide some idea or even a complete script for a better result.


Requirement:
1. remove most of the dots.
2. keep most of the sharpness of texts.
3. this script already takes 5 minutes to run for 1 image, so speed is not the first priority, quality is.

Source:


Result:
cagali is offline   Reply With Quote
Old 12th January 2019, 07:40   #2  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,155
This is not noise, this is called screening and there is no avisynth filter that can correctly fix this.

You'll need Sattva Descreen.
lansing is offline   Reply With Quote
Old 12th January 2019, 11:54   #3  |  Link
cagali
Registered User
 
Join Date: May 2014
Posts: 42
Quote:
Originally Posted by lansing View Post
This is not noise, this is called screening and there is no avisynth filter that can correctly fix this.

You'll need Sattva Descreen.
Thanks. I think descreening is the proper way that I should look into.

I tuned my avs script a little bit.
I think I get half of the area correct.
With a bit more time I think I can get the remaining half correct too.

Last edited by cagali; 12th January 2019 at 12:47.
cagali is offline   Reply With Quote
Old 12th January 2019, 17:20   #4  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,155
I would say don't even bother, this field is so specialized that you would just want to "leave it to the pro".
lansing is offline   Reply With Quote
Old 13th January 2019, 02:36   #5  |  Link
cagali
Registered User
 
Join Date: May 2014
Posts: 42
Quote:
Originally Posted by lansing View Post
I would say don't even bother, this field is so specialized that you would just want to "leave it to the pro".
Thank you for trying to help.
I hope the pros will read this thread by chance.
cagali is offline   Reply With Quote
Old 13th January 2019, 03:11   #6  |  Link
cagali
Registered User
 
Join Date: May 2014
Posts: 42
x1 original


x4 original


The screening pattern on the original scan was probably more regular.
However the version I have seems to be scaled down, so the pattern is not that regular.
Are there any descreen filters that can still process these kind of images probably?
(for video-related topics I think doom9 is the proper place, for these image-related topics is there any similar forum? reddit?)

x1 descreen filter in gimp, default parameters


x4 denoising with avisynth filters
cagali is offline   Reply With Quote
Old 13th January 2019, 03:15   #7  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,975
It looks good

Instead of default gimp parameters, you can use a mask (e.g. luma mask) with layers to improve the gimp filter, just like you can use a mask in avisynth for the text. Because it's "black" text it's "easier" to isolate
poisondeathray is offline   Reply With Quote
Old 13th January 2019, 03:44   #8  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,155
Quote:
Originally Posted by cagali View Post
Thank you for trying to help.
I hope the pros will read this thread by chance.
What I mean is just buy their software if you really need a decreen filter.
lansing is offline   Reply With Quote
Old 13th January 2019, 04:52   #9  |  Link
cagali
Registered User
 
Join Date: May 2014
Posts: 42
Quote:
Originally Posted by lansing View Post
What I mean is just buy their software if you really need a decreen filter.
Yeah from the sample of the software, I think it will work well if the scans that I have are original, not scaled down.

"Sattva Descreen plug-in for Adobe Photoshop
...
You need just provide the plug-in with a hi-resolution scan, usually 600...1200 pixels/inch. "

The scans I have are just 300 dpi, 2000px x 3000px.
cagali is offline   Reply With Quote
Old 13th January 2019, 08:38   #10  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,155
Quote:
Originally Posted by cagali View Post
Yeah from the sample of the software, I think it will work well if the scans that I have are original, not scaled down.

"Sattva Descreen plug-in for Adobe Photoshop
...
You need just provide the plug-in with a hi-resolution scan, usually 600...1200 pixels/inch. "

The scans I have are just 300 dpi, 2000px x 3000px.
I think the minimum resolution requirement for the plugin is 300 dpi, so you should be fine.
lansing is offline   Reply With Quote
Old 13th January 2019, 09:35   #11  |  Link
cagali
Registered User
 
Join Date: May 2014
Posts: 42
Quote:
Originally Posted by lansing View Post
I think the minimum resolution requirement for the plugin is 300 dpi, so you should be fine.
Really? That's great.

Then next time I could try to use that PS plugin.

As I did not cook and eat outside for the whole week, this time I think I will stick to the gimp descreen plugin, also FFT based.
cagali is offline   Reply With Quote
Old 13th January 2019, 09:41   #12  |  Link
cagali
Registered User
 
Join Date: May 2014
Posts: 42
Quote:
Originally Posted by poisondeathray View Post
It looks good

Instead of default gimp parameters, you can use a mask (e.g. luma mask) with layers to improve the gimp filter, just like you can use a mask in avisynth for the text. Because it's "black" text it's "easier" to isolate
Thank you!
Maybe it is afterall better to use an image software for image processing? What do you think?

I know a little bit avisynth scripting but nothing about gimp scripting. Maybe not that difficult anyway.

What I originally planned to do is, do some basic descreening/noise removal in 1 avs, and then in the same script use waifu2x to do some tiny final clean-up.
cagali is offline   Reply With Quote
Old 13th January 2019, 14:34   #13  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 7,189
You can do a D9 search on POSTS(Not Threads) for "Mogrify", for AVS writing/reading images for ImageMagick processing .
(if you can find some descreen stuff for that, there is some IM stuff for creating halftone/screening, also think Martin53 did something to create screening in AVS).

[See specifically entries under CallCmd, RT_Call (RT_Stats) threads, for writing, reading and calling ImageMagick Mogrify.exe]
Perhaps can access GIMP using similar methods(dont know, never tried it).

EDIT: This guy has a lot of ImageMagick scripts, think we used some things from there, dont know if anything of interest.
http://www.fmwconcepts.com/imagemagi...inex/index.php

EDIT: Heres an interesting one if VideoFred is watching:- http://www.fmwconcepts.com/imagemagi...cast/index.php

EDIT: Above linked scripts run under bash on linux, but note below
Quote:
Windows users can run my scripts, if they install the unix Cygwin system for Windows and then install the IM Cygwin binary.

A very detailed description of Cygwin installation, relevant script syntax and examples has been provided by user snibgo on his
web site at http://im.snibgo.com/cygwin.htm. A brief outline that he wrote can be
found at http://www.imagemagick.org/discourse...art=30#p113836. Many thanks to snibgo for this excellent document.

Another guide to using bash shell scripts under Cygwin on Windows has been provided by Wolfgang Hugemann in his excellent page, Usage Under Windows


Windows 10 (64-bit) users can also run my scripts without the need for Cygwin. Bruce Rusk reports: All that was need was the installation of the Unix bc tool and
Imagemagick through apt/apt-get. Everything else worked right out of the box, no changes to PATH variables required. See How to Install and Use the Linux Bash
Shell on Windows 10.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 13th January 2019 at 15:26.
StainlessS is offline   Reply With Quote
Old 13th January 2019, 17:25   #14  |  Link
cagali
Registered User
 
Join Date: May 2014
Posts: 42
Quote:
Originally Posted by StainlessS View Post
You can do a D9 search on POSTS(Not Threads) for "Mogrify", for AVS writing/reading images for ImageMagick processing .
(if you can find some descreen stuff for that, there is some IM stuff for creating halftone/screening, also think Martin53 did something to create screening in AVS).

[See specifically entries under CallCmd, RT_Call (RT_Stats) threads, for writing, reading and calling ImageMagick Mogrify.exe]
Perhaps can access GIMP using similar methods(dont know, never tried it).

EDIT: This guy has a lot of ImageMagick scripts, think we used some things from there, dont know if anything of interest.
http://www.fmwconcepts.com/imagemagi...inex/index.php

EDIT: Heres an interesting one if VideoFred is watching:- http://www.fmwconcepts.com/imagemagi...cast/index.php

EDIT: Above linked scripts run under bash on linux, but note below
Thank you for your effort for gathering these information!!

Now I remember that a long time ago I read about noise removal in imagemagick.
Building imagemagick from source with fft support...
Creating spectrum mask from the phase and magnitude of the image...

Fred's scripts collection is really a big one, I think it is enough for me to spend a few nights.
cagali is offline   Reply With Quote
Old 13th January 2019, 18:08   #15  |  Link
JReiginsei
Registered User
 
Join Date: Apr 2002
Posts: 75
Gimp

You can do this with the GIMP. Look at this video on Youtube https://youtu.be/3137dDa6P4s
__________________
Intel Core i5-4250U, 8 GB Ram, Intel HD 5000
JReiginsei is offline   Reply With Quote
Old 14th January 2019, 17:16   #16  |  Link
Seedmanc
Registered User
 
Join Date: Sep 2010
Location: Russia
Posts: 88
I used to deal a lot with scans so I can probably make a few suggestions here.

As mentioned earlier this is a moire/screen effect, unlike the film noise it has a couple of fixed frequencies which makes it possible to remove it using those frequency peaks on FFT, instead of going in blindly. If you really want to do it with Avisynth there are filters like DeFreq or Vcmohan's FFTQuiver (not sure if that one exactly, he has loads of various scripts). I only used DeFreq, it's tricky to use since you have to describe the FFT areas for removal in text form. The only upside of doing it in Avisynth is batch-processing of loads of similar scans taken from the same source, because then the FFT discrepancies would be the same and it'll be enough to configure the filter once, then follow up with a denoiser as well. At some point I even tried to write a JS helper for it that would at least allow the user to select the areas with a mouse and generate AVS based on that as well as generating a way to import multiple images that have a slightly different size into a single video stream, but's still not very convenient.

It would make a lot more sense to use a specialized image editing software here. Considering that you're probably using a denoiser like Waifu2x without GPU acceleration (since it take 5 mins per image and the AVS ver indeed doesn't have it), you're not going to do batch-processing anyway. Here I can suggest a decent free alternative to photoshop plugins - Image Analyzer, which, among other things, has a special tool for editing the frequency spectrum - the Frequency Domain Filter. I wrote a manual for that tool in application to scan retouching. Keep in mind, though, that it's still a long manual process and even after removing most of the noise noise you'll be left with whatever random noise there was in the image. Still, for best results I would recommend applying FDF before the conventional denoisers, otherwise you'll often be left with lots of moire remaining in the image or you'll have to use extreme noise removal presets and lose the details.
As for a followup general denoiser for images I can recommend NeatImage. Though it's not free it has a decent trial functionality and you can always search for the free alternatives of it.
Seedmanc is offline   Reply With Quote
Old 15th January 2019, 08:40   #17  |  Link
cagali
Registered User
 
Join Date: May 2014
Posts: 42
Quote:
Originally Posted by JReiginsei View Post
You can do this with the GIMP. Look at this video on Youtube https://youtu.be/3137dDa6P4s
Thank you. The result looks pretty nice.

Actually one can do this in programming ways.
For example in imagemagick, generate a spectrum mask with a threshold (so one does not need to draw black spots on the FFT image manually), then multiply it with the magnitude of the image, and then... one gets a filtered, cleaner image.
cagali is offline   Reply With Quote
Old 15th January 2019, 08:53   #18  |  Link
cagali
Registered User
 
Join Date: May 2014
Posts: 42
Quote:
Originally Posted by Seedmanc View Post
I used to deal a lot with scans so I can probably make a few suggestions here.

As mentioned earlier this is a moire/screen effect, unlike the film noise it has a couple of fixed frequencies which makes it possible to remove it using those frequency peaks on FFT, instead of going in blindly. If you really want to do it with Avisynth there are filters like DeFreq or Vcmohan's FFTQuiver (not sure if that one exactly, he has loads of various scripts). I only used DeFreq, it's tricky to use since you have to describe the FFT areas for removal in text form. The only upside of doing it in Avisynth is batch-processing of loads of similar scans taken from the same source, because then the FFT discrepancies would be the same and it'll be enough to configure the filter once, then follow up with a denoiser as well. At some point I even tried to write a JS helper for it that would at least allow the user to select the areas with a mouse and generate AVS based on that as well as generating a way to import multiple images that have a slightly different size into a single video stream, but's still not very convenient.

It would make a lot more sense to use a specialized image editing software here. Considering that you're probably using a denoiser like Waifu2x without GPU acceleration (since it take 5 mins per image and the AVS ver indeed doesn't have it), you're not going to do batch-processing anyway. Here I can suggest a decent free alternative to photoshop plugins - Image Analyzer, which, among other things, has a special tool for editing the frequency spectrum - the Frequency Domain Filter. I wrote a manual for that tool in application to scan retouching. Keep in mind, though, that it's still a long manual process and even after removing most of the noise noise you'll be left with whatever random noise there was in the image. Still, for best results I would recommend applying FDF before the conventional denoisers, otherwise you'll often be left with lots of moire remaining in the image or you'll have to use extreme noise removal presets and lose the details.
As for a followup general denoiser for images I can recommend NeatImage. Though it's not free it has a decent trial functionality and you can always search for the free alternatives of it.
It is really amazing that you find out so many things that I want to do, but not mentioned in the first post. XD

I am going to try the methods you suggested in an image software, see how clean I can get.
If it is much better than what I get using avs, I am going to pick the approach.
Otherwise I will stick with the avs way, coz there is a small batch to go, less than a 100 images.

Thank you for you suggestions!!
cagali is offline   Reply With Quote
Old 15th January 2019, 13:51   #19  |  Link
WaxCyl
Registered User
 
Join Date: Mar 2016
Posts: 7
Pattern Suppressor

I use this FFT action/plugin for pattern removal:
https://www.retouchpro.com/forum/too...ern-suppressor
It is FREE!

Last edited by WaxCyl; 15th January 2019 at 13:52. Reason: Added It is FREE!
WaxCyl is offline   Reply With Quote
Old 15th January 2019, 21:35   #20  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,155
Quote:
Originally Posted by WaxCyl View Post
I use this FFT action/plugin for pattern removal:
https://www.retouchpro.com/forum/too...ern-suppressor
It is FREE!
This is pretty much unusable, I have a 4000 x 5600 color scan and this action script took up 5.3G of my drive as scratch disk and 7G and counting of RAM until my photoshop crash because of out of memory.
lansing is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 07:13.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.