Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 24th December 2005, 19:37   #1  |  Link
Isochroma
Registered User
 
Join Date: Mar 2005
Posts: 468
Note about denoising method

I had a private message this morning inquiring about whether I use Deen() for denoising. This inspired me to write this long and windy description of how the current (much improved!) workflow goes. It makes use of two denoisers, one radically different in both method and workflow than the usual avisynth filters.

Here is my reply:

Nope, no Deen()!!!

Here's a short and quick summary of the last 6 months of denoising work...

I found that a program called Neat Image denoises far better on the spacial plane than any other program could hope to:

http://www.neatimage.com/

It uses wavelets rather than FFT and the results on still images are superb. So, I built Neat Batch and Neat Batch AVI:

http://isochroma.com/Testfiles/NB/neatbatch.htm

Here are some comparative screenshots of before and after. Note that the text above them is very out-of-date, and doesn't reflect my current workflow:

http://isochroma.com/Testfiles/DeltaChroma-2.htm#Noir

Due to limitations in Windows (all versions) selector boxes crash when trying to select more than 16,383 files at once. The goal was to separate a video into still images and pass them through Neat Image, then reassemble them.

Worse, Neat Image uses more memory as more images are loaded into the queue, making it impossible to queue more than 4000 with reasonable memory usage.

So I built Neat Batch & Neat Batch AVI to create batch files which call Neat Image's commandline interface. This means virtually no memory usage and processing of vast numbers of images. Please see the site for more details.

I used this program to process Armitage: Poly-Matrix. The strange thing is, each frame looks beautiful on its own, but when animated, temporal noise is very visible and a big problem!

In just the last two days I realized that if the video were preprocessed with a temporal denoiser (FFT3D) it would remove the noise that the purely spacial Neat Image didn't touch. Then NI would do the spacial cleaning.

So I made the usual batch file with Neat Batch AVI, and made sure to use FFT3D in the .avs file used as the source (frameserved into an AVI via ffdshow's AVIS maker):

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\DGDecode.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Decomb521.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\fft3dfilter.dll")
MPEG2Source("G:\Shinjuku\AVISynth Scripts\Escaflowne\VOB Input\Escaflowne.IP.Tex.d2v")
Telecide(order=1)
Decimate(cycle=5, mode=0)
FFT3DFilter(sigma=3,plane=4)
LimitedSharpen(ss_x=2.0, ss_y=2.0, Smode=3, strength=800)

VFAPI cannot be used in this case because it will not process .avs files containing the FFT3DFilter function, due to some strange error - this behaviour has been verified by Fizick.

The processing is very slow with two denoisers, running at 1/10 fps on my Duron 1600 in the basement. Beyond just denoising, the batch script sequentially calls IrfanView to convert Neat Image's raw BMP output to PNG compression level 2. This saves disk space and uses little CPU.

But Wow! The frames are so slick they just slide through my mind like greased bananna peels. No temporal flicker like the Armitage disaster! And don't forget, this isn't the end of processing, only the beginning!

As the processing is occuring on the machine downstairs, I sit upstairs running Vegas 6.0, editing the sharpened raw frame by frame, selecting the best duplicate in every dup-group for replicative replacement of its neighbors. The current project is Escaflowne the movie, which rings in at 140,636 frames. My record is 6,500 frames in a day, and average is 4,000.

The Vegas 6.0 projects are each around 4,000 frames long. More than this and Vegas starts to slow down too much and uses too much memory. Sequences are created by using Trim(start,end) on the original source:

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\DGDecode.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Decomb521.dll")
MPEG2Source("Escaflowne.IP.d2v")
Telecide(order=1)
Decimate(cycle=5, mode=0)
Trim(4194,8220)
ConvertToRGB24()
Crop(13,11,696,454)
Lanczos4Resize(704,464)
ConvertToYUY2()
LimitedSharpen(ss_x=2.0, ss_y=2.0, Smode=3, strength=800)

A VFAPI AVI file is made from this script and loaded into Vegas for editing. Before frames can be replaced, they must be split from the video uniblock on the timeline. For this, I use Hot Keyboard, with a simple script that repeats the s key and the right arrow key thousands of times. It takes about 10 minutes to split the 4,000 frames.

After editing in Vegas is complete for a segment, I replace the VFAPI AVI file with another of the same name, made from a different script:

ImageSource(file = "S:\VideoDone\Escaflowne\EF.%06d.png", start = 0, end = 140636, fps = 23.976)
Trim(4194,8220)
Crop(13,11,696,454)
Lanczos4Resize(704,464)

The first line is autogenerated by Neat Batch AVI, the other three were added by me later.

It is importatant to never open the folder with these files in it, as this locks explorer for hours as it counts the vast number of files. Instead, all work is done via the .avs and .bat files auto-generated by Neat Batch AVI. The .avs contains the user-specified image-sequence reassembly (see above) and other arguments added later. The .bat auto purges all the images when work with them is done - just double-click it to activate:

DEL /Q /F "D:\VideoDone\Escaflowne\EF.*.png" >nul

When I load the Vegas project, it is fooled and takes the substituted file as if it was the original. So the frame-selection and replacement based on quality that is made from the sharpened (to highlight differences) raw is applied to the temporal-spacial denoised material. Relative quality between frames in a dup-group (note that I do not use the dup() filter - just a convenient term) is probably the same in both versions or extremely close.

The file is then rendered to a lossless format and to DIV6 with quantizer 1 with GMC/QPEL unchecked. Any DCT codec ruins the perfectly smooth, blockless, grainless gradients made by this method, though wavelet codecs such as SNOW work fine. So I am upgrading the playback machine so it can run Lagarith & HFYU in VMR7 renderless with subs, maybe even MSU...

The rendered segments are merged using VirtualDub's direct stream copy mode. The DIV6 file is then used for subtitle work. When subtitles are done, the lossless copy is muxed into an mkv with appropriate AR corrrection values, .ass file, fonts, etc.

The results of this process are breathtaking, to say the least.

Last edited by Isochroma; 24th December 2005 at 19:45.
Isochroma is offline   Reply With Quote
Old 24th December 2005, 20:11   #2  |  Link
rfmmars
Registered User
 
Join Date: Feb 2004
Posts: 743
Quote:
Originally Posted by Isochroma
I had a private message this morning inquiring about whether I use Deen() for denoising. This inspired me to write this long and windy description of how the current (much improved!) workflow goes. It makes use of two denoisers, one radically different in both method and workflow than the usual avisynth filters.

Here is my reply:

Nope, no Deen()!!!

Here's a short and quick summary of the last 6 months of denoising work...

I found that a program called Neat Image denoises far better on the spacial plane than any other program could hope to:

http://www.neatimage.com/

It uses wavelets rather than FFT and the results on still images are superb. So, I built Neat Batch and Neat Batch AVI:

http://isochroma.com/Testfiles/NB/neatbatch.htm

Here are some comparative screenshots of before and after. Note that the text above them is very out-of-date, and doesn't reflect my current workflow:

http://isochroma.com/Testfiles/DeltaChroma-2.htm#Noir

Due to limitations in Windows (all versions) selector boxes crash when trying to select more than 16,383 files at once. The goal was to separate a video into still images and pass them through Neat Image, then reassemble them.

Worse, Neat Image uses more memory as more images are loaded into the queue, making it impossible to queue more than 4000 with reasonable memory usage.

So I built Neat Batch & Neat Batch AVI to create batch files which call Neat Image's commandline interface. This means virtually no memory usage and processing of vast numbers of images. Please see the site for more details.

I used this program to process Armitage: Poly-Matrix. The strange thing is, each frame looks beautiful on its own, but when animated, temporal noise is very visible and a big problem!

In just the last two days I realized that if the video were preprocessed with a temporal denoiser (FFT3D) it would remove the noise that the purely spacial Neat Image didn't touch. Then NI would do the spacial cleaning.

So I made the usual batch file with Neat Batch AVI, and made sure to use FFT3D in the .avs file used as the source (frameserved into an AVI via ffdshow's AVIS maker):

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\DGDecode.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Decomb521.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\fft3dfilter.dll")
MPEG2Source("G:\Shinjuku\AVISynth Scripts\Escaflowne\VOB Input\Escaflowne.IP.Tex.d2v")
Telecide(order=1)
Decimate(cycle=5, mode=0)
FFT3DFilter(sigma=3,plane=4)
LimitedSharpen(ss_x=2.0, ss_y=2.0, Smode=3, strength=800)

VFAPI cannot be used in this case because it will not process .avs files containing the FFT3DFilter function, due to some strange error - this behaviour has been verified by Fizick.

The processing is very slow with two denoisers, running at 1/10 fps on my Duron 1600 in the basement. Beyond just denoising, the batch script sequentially calls IrfanView to convert Neat Image's raw BMP output to PNG compression level 2. This saves disk space and uses little CPU.

But Wow! The frames are so slick they just slide through my mind like greased bananna peels. No temporal flicker like the Armitage disaster! And don't forget, this isn't the end of processing, only the beginning!

As the processing is occuring on the machine downstairs, I sit upstairs running Vegas 6.0, editing the sharpened raw frame by frame, selecting the best duplicate in every dup-group for replicative replacement of its neighbors. The current project is Escaflowne the movie, which rings in at 140,636 frames. My record is 6,500 frames in a day, and average is 4,000.

The Vegas 6.0 projects are each around 4,000 frames long. More than this and Vegas starts to slow down too much and uses too much memory. Sequences are created by using Trim(start,end) on the original source:

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\DGDecode.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Decomb521.dll")
MPEG2Source("Escaflowne.IP.d2v")
Telecide(order=1)
Decimate(cycle=5, mode=0)
Trim(4194,8220)
ConvertToRGB24()
Crop(13,11,696,454)
Lanczos4Resize(704,464)
ConvertToYUY2()
LimitedSharpen(ss_x=2.0, ss_y=2.0, Smode=3, strength=800)

A VFAPI AVI file is made from this script and loaded into Vegas for editing. Before frames can be replaced, they must be split from the video uniblock on the timeline. For this, I use Hot Keyboard, with a simple script that repeats the s key and the right arrow key thousands of times. It takes about 10 minutes to split the 4,000 frames.

After editing in Vegas is complete for a segment, I replace the VFAPI AVI file with another of the same name, made from a different script:

ImageSource(file = "S:\VideoDone\Escaflowne\EF.%06d.png", start = 0, end = 140636, fps = 23.976)
Trim(4194,8220)
Crop(13,11,696,454)
Lanczos4Resize(704,464)

The first line is autogenerated by Neat Batch AVI, the other three were added by me later.

It is importatant to never open the folder with these files in it, as this locks explorer for hours as it counts the vast number of files. Instead, all work is done via the .avs and .bat files auto-generated by Neat Batch AVI. The .avs contains the user-specified image-sequence reassembly (see above) and other arguments added later. The .bat auto purges all the images when work with them is done - just double-click it to activate:

DEL /Q /F "D:\VideoDone\Escaflowne\EF.*.png" >nul

When I load the Vegas project, it is fooled and takes the substituted file as if it was the original. So the frame-selection and replacement based on quality that is made from the sharpened (to highlight differences) raw is applied to the temporal-spacial denoised material. Relative quality between frames in a dup-group (note that I do not use the dup() filter - just a convenient term) is probably the same in both versions or extremely close.

The file is then rendered to a lossless format and to DIV6 with quantizer 1 with GMC/QPEL unchecked. Any DCT codec ruins the perfectly smooth, blockless, grainless gradients made by this method, though wavelet codecs such as SNOW work fine. So I am upgrading the playback machine so it can run Lagarith & HFYU in VMR7 renderless with subs, maybe even MSU...

The rendered segments are merged using VirtualDub's direct stream copy mode. The DIV6 file is then used for subtitle work. When subtitles are done, the lossless copy is muxed into an mkv with appropriate AR corrrection values, .ass file, fonts, etc.

The results of this process are breathtaking, to say the least.
Thanks for sharing a detailed acount of your research. I have used Neat Image too and it is great. In my slide work I use Noise Ninja, I wonder if your batch file will work there too?

richard
rfmmars is offline   Reply With Quote
Old 24th December 2005, 21:03   #3  |  Link
Isochroma
Registered User
 
Join Date: Mar 2005
Posts: 468
Noise Ninja commandline?

It will work, but Neat Batch AVI would need to be modified to support different command-line options that Noise Ninja provides... this of course assumes that NN provides a command-line interface?
Isochroma is offline   Reply With Quote
Old 24th December 2005, 22:53   #4  |  Link
Isochroma
Registered User
 
Join Date: Mar 2005
Posts: 468
Live pre- and post-denoised

Fun!

The avs files below allow slow, but live examination of the before and after frames of the currently running process. Save the three files below anywhere and (presuming you have AVISynth and VirtualDub), open the two avs files. EF-DN is the denoised version, EF-RAW is raw. Currently the processing is up to frame 10,813, so you should be fine seeking up to there.

http://isochroma.com/Testfiles/Misc/EF-DN.AVS
http://isochroma.com/Testfiles/Misc/EF-RAW.avs
http://isochroma.com/Testfiles/Misc/EF.d2v

To view the EF-RAW (raw) you'll need:

http://isochroma.com/Testfiles/Apps/decomb521.zip
http://isochroma.com/Testfiles/Apps/dgmpgdec145.zip

The .d2v was made with 1.4.5, so you'll need that version of the plugin in your AVISynth plugins folder.

If you open both in two vdubs, you can alt-tab between them and step through the timeline to view the difference.

Last edited by Isochroma; 24th December 2005 at 23:03.
Isochroma is offline   Reply With Quote
Old 25th December 2005, 01:50   #5  |  Link
Tripx
Registered User
 
Join Date: Jul 2005
Posts: 13
Hey Isochroma,

I've tried Neat Image and it does do great noise reduction. However, using a DVD source at 720 x 480 Neat Image will often say that it can't find a uniform area to build a profile. If I click to continue, it will blur the picture badly and not denoise it very well. The workaround I have found is to super size the picture to 2880 x 1920 in order for Neat Image to find a uniform area to build a profile for noise reduction. Doing so greatly slows the process down though.

Am I doing something wrong? How do you denoise with Neat Image without resizing and still get good results?

Thanks.

Last edited by Tripx; 25th December 2005 at 01:53.
Tripx is offline   Reply With Quote
Old 25th December 2005, 01:57   #6  |  Link
Isochroma
Registered User
 
Join Date: Mar 2005
Posts: 468
Don't resize the image... NI uses the size of the noise pixels to build a noise profile (frequency vs. power spectrum). Resizing the sample will tell NI to remove noise of that size from the image - which will not match.

Instead, step thru the movie using virtualdub until you find a frame with a uniform area which contains just noise. Then copy the frame using CTRL-1 and paste it into paint or irfanview, save as bmp (do not save in a lossy format like jpg!).

Use this to generate a noise profile, then save the profile as .dnp. If you can't find a good frame (you should be able to), here's a sample which might resemble the grain in your film:

http://isochroma.com/Testfiles/Misc/...wne.127712.dnp

It is pretty uniform fine-grain noise in this case.
Isochroma is offline   Reply With Quote
Old 25th December 2005, 02:07   #7  |  Link
Tripx
Registered User
 
Join Date: Jul 2005
Posts: 13
Thanks, I'll try that.

Will the noise profile from one image apply throughout the video or should I find a profile for each scene?
Tripx is offline   Reply With Quote
Old 25th December 2005, 03:01   #8  |  Link
Isochroma
Registered User
 
Join Date: Mar 2005
Posts: 468
Film grain tends to remain the same (at least the frequency spectrum), though the amount may change throughout the film. NI has an option to auto-fine-tune the noise filter.

Now, MPEG noise is notoriously nonlinear and can change its frequency spectrum depending on the image and its motion. Checking through the frames, you should be able to distinguish the amount of noise vs. the MPEG amplification of this noise and other nonlinear distortions.
Isochroma is offline   Reply With Quote
Old 25th December 2005, 04:54   #9  |  Link
falcon2000eg
Registered User
 
falcon2000eg's Avatar
 
Join Date: Apr 2005
Location: Egypt
Posts: 70
Neat Video v1.0 is here
falcon2000eg is offline   Reply With Quote
Old 25th December 2005, 05:27   #10  |  Link
Tripx
Registered User
 
Join Date: Jul 2005
Posts: 13
Neat Video 1.0 even has support for VirtualDub, I was just trying out the demo version.

Edit: It also works with Adobe After Effects 6.0 / 6.5 and Adobe Premiere Pro 1.0 / 1.5

Last edited by Tripx; 25th December 2005 at 05:46.
Tripx is offline   Reply With Quote
Old 25th December 2005, 05:35   #11  |  Link
Isochroma
Registered User
 
Join Date: Mar 2005
Posts: 468
Holy crap! The guy damned stole my idea! I let him know some time ago on his forum about my tool...
Isochroma is offline   Reply With Quote
Old 25th December 2005, 05:38   #12  |  Link
FredThompson
Registered User
 
FredThompson's Avatar
 
Join Date: Feb 2002
Location: Charlotte, NC USA
Posts: 1,981
Your screenshot samples are blurring a lot of detail. Are those post or pre-LimitedSharpen()?

Neat Video is for VirtualDub and not AviSynth? Would be far better to stay with the same color model if at all possible. RGB is fine if you're doing lossless capture, not so great if the source is any flavor of MPEG...
__________________
Reclusive fart.
Collecting Military, Trains, Cooking, Woodworking, Fighting Illini, Auburn Tigers
FredThompson is offline   Reply With Quote
Old 25th December 2005, 05:43   #13  |  Link
Tripx
Registered User
 
Join Date: Jul 2005
Posts: 13
Quote:
Originally Posted by Isochroma
Holy crap! The guy damned stole my idea! I let him know some time ago on his forum about my tool...
Well, I don't think it's stealing an idea if you spam your technique all over their boards.

At the very least they listened to their users for what they wanted.

Quote:
Originally Posted by DaveQ
Neat Batch / Neat Batch AVI: Video Processing for Neat Image

After several weeks of intense coding, I am finally ready to present v.1 to the world!

The reason for all this work, was the need to process very large numbers of video frames with Neat Image (119,000 +). I soon realized that I could add features to the barebones code and make the tools useful to others, who had also expressed an interest in this usage.

http://www.isochroma.com/Testfiles/NB/neatbatch.htm

Partial list of features:

> Open Source
> AVI or Image File input
> Very large batches (up to 2,147,483,646 images)
> Priority setting
> Input/Output sequence name & number remapping
> Input & Output image format conversion
> Automatic AVS generation for video output
> DNP, NFP, all switches supported
> Automated cleanup of input, output files

All done on-the-fly... with only 1 frame overhead.
Tripx is offline   Reply With Quote
Old 25th December 2005, 06:08   #14  |  Link
FredThompson
Registered User
 
FredThompson's Avatar
 
Join Date: Feb 2002
Location: Charlotte, NC USA
Posts: 1,981
Quote:
Originally Posted by Isochroma
Holy crap! The guy damned stole my idea! I let him know some time ago on his forum about my tool...
Yeah, nobody who ever did noise reduction for photos would think of noise reduction on a series of images...
__________________
Reclusive fart.
Collecting Military, Trains, Cooking, Woodworking, Fighting Illini, Auburn Tigers
FredThompson is offline   Reply With Quote
Old 25th December 2005, 08:07   #15  |  Link
Isochroma
Registered User
 
Join Date: Mar 2005
Posts: 468
I was just kidding! But really, I'm very glad they finally made the code... the developer and I had great correspondences and I even purchased Neat Image for just the job...
Isochroma is offline   Reply With Quote
Old 25th December 2005, 13:08   #16  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,565
Quote:
Originally Posted by FredThompson
Your screenshot samples are blurring a lot of detail. Are those post or pre-LimitedSharpen()?

Neat Video is for VirtualDub and not AviSynth? Would be far better to stay with the same color model if at all possible. RGB is fine if you're doing lossless capture, not so great if the source is any flavor of MPEG...
Neat Image is definitely optimized for high-resolution photos that aren't totally sharp, I've found that using it on crisp lower resolutions is almost guaranteed to blur edges up no matter what settings. Even following with Photokit Sharpener (Photoshop's version of Limited Sharpen) doesn't get it all back.

Still, I'm interested in their new video plugin.
foxyshadis is offline   Reply With Quote
Old 26th December 2005, 03:18   #17  |  Link
Mug Funky
interlace this!
 
Mug Funky's Avatar
 
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,555
there's a limited sharpen for potatoshop?

dang, and i've been emulating it using buttloads of layers and masks and putting it into an action... kinda makes you appreciate the blinding speed of avisynth in comparison.
__________________
sucking the life out of your videos since 2004
Mug Funky is offline   Reply With Quote
Old 26th December 2005, 10:11   #18  |  Link
Fizick
AviSynth plugger
 
Fizick's Avatar
 
Join Date: Nov 2003
Location: Russia
Posts: 2,183
Quote:
Originally Posted by FredThompson
Yeah, nobody who ever did noise reduction for photos would think of noise reduction on a series of images...
I suppore, he also read this:
http://forum.doom9.org/showthread.php?t=90042


It seems, "neat video" is similar to fft3dfilter and defreq.
Lets will be use it as a tutorial.
__________________
My Avisynth plugins are now at http://avisynth.org.ru and mirror at http://avisynth.nl/users/fizick
I usually do not provide a technical support in private messages.

Last edited by Fizick; 26th December 2005 at 10:28.
Fizick is offline   Reply With Quote
Old 26th December 2005, 13:29   #19  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,565
Quote:
Originally Posted by Mug Funky
there's a limited sharpen for potatoshop?

dang, and i've been emulating it using buttloads of layers and masks and putting it into an action... kinda makes you appreciate the blinding speed of avisynth in comparison.
Well, it costs $100 and is put together by the finest minds in the industry, so I wouldn't feel too bad. You'd be interested to note that it's mainly put together as a series of sharpen/blur/mask/repeat steps. I'm not sure of the specifics since it usually goes by too fast to tell. (It also has capture and output sharpening in addition to utility sharpening.)

http://www.pixelgenius.com/sharpener/
foxyshadis is offline   Reply With Quote
Old 26th December 2005, 20:01   #20  |  Link
Isochroma
Registered User
 
Join Date: Mar 2005
Posts: 468
Indeed, that was the mystical article that informed me of Neat Image's possibilities, and so I decided that it would be an interesting avenue to explore! I was doing an exhaustive search & test of all the denoisers available, and happened to find it in my travels.
Isochroma is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 02:29.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.