Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 11th October 2013, 17:24   #1  |  Link
pancserzso
Registered User
 
Join Date: Oct 2004
Posts: 131
How to denoise rendered image sequence

I am trying to filter image sequences rendered by a visualization program. The problem is that the program was not created with animation in mind, and it created an image sequence full of artifacts / temporal noise.

You can see an example of such source here.

Some time ago I was trying to solve the same problem, and I got some help from Didée in this thread however this time some of the samples are worse in quality and the original script didn't really work on them.

Here is the code which I tried to use for filtering:
Code:
LoadPlugin( "e:\apps\utils\Video tools\AVSplugins\RemoveGrainSSE3.dll" )
LoadPlugin( "e:\apps\utils\Video tools\AVSplugins\mt_masktools-26.dll" )
LoadPlugin( "e:\apps\utils\Video tools\AVSplugins\EEDI2.dll" )

o=last 
ox=o.width 
oy=o.height

# round/soften hard corner pixels
o.pointresize(ox*3,oy*3).removegrain(4)
mt_lutxy(last,last.removegrain(11,-1),"x x y - 2.51 * +",U=2,V=2).bilinearresize(ox,oy)
o3=last

bubb0 = o3.eedi2(maxd=8).bicubicresize(ox,oy,0,0,0,0.5).turnleft().eedi2(maxd=8).turnright().bicubicresize(ox,oy,0,0,-0.5,0)
return(bubb0)
And here is the filtered result I got: filteredYV12.mp4

I think my problem is that the code which I'm trying to use is not really strong enough for my source.

Is there any way to create a filter which removes this strong temporal flickering / noise? It's not a big problem if the end result is a bit blurry.

Last edited by pancserzso; 11th October 2013 at 19:31.
pancserzso is offline   Reply With Quote
Old 11th October 2013, 20:10   #2  |  Link
raffriff42
Retried Guesser
 
raffriff42's Avatar
 
Join Date: Jun 2012
Posts: 1,373
The images are very aliased - is this how the original images are? (crop + 2x blowup)


Can you turn on antialiasing in your render engine? If not, can you render at 4x final size? If so, you can downsample your images with smoothing in Avisynth, or in an image batch processor.

Last edited by raffriff42; 18th March 2017 at 01:24. Reason: (fixed image link)
raffriff42 is offline   Reply With Quote
Old 11th October 2013, 20:57   #3  |  Link
pancserzso
Registered User
 
Join Date: Oct 2004
Posts: 131
Quote:
Originally Posted by raffriff42 View Post
The images are very aliased - is this how the original images are? (crop + 2x blowup)


Can you turn on antialiasing in your render engine? If not, can you render at 4x final size? If so, you can downsample your images with smoothing in Avisynth, or in an image batch processor.
I know, this is the problem. The object in 3D is initially very far from the origin, and the rendering program was not written with this in mind. So when the object is transparent and is far from the origin, these artifacts appear.

The problem with re-rendering is that even in 1.5x resolution it would take me at least 7 days non-stop rendering, and I don't have enough time for that. Also, since its a GPU based program, I might not be able to render some objects, as it'd need >2 GB of GPU memory, and my card only has 2.

Pretty much my only choice is to do filtering in Avisynth.

I've tried a few spatio-temporal denoiser, and so far FFT3DFilter was able to get it really smooth, although very blurry.

I was thinking about using the FFT3D as an input to MVTools and somehow combine it with the original source, but I don't know how to do it. What do you think?
pancserzso is offline   Reply With Quote
Old 11th October 2013, 21:37   #4  |  Link
raffriff42
Retried Guesser
 
raffriff42's Avatar
 
Join Date: Jun 2012
Posts: 1,373
I'm looking at the script you posted - I like it. Much better than my naive GaussianBlur>Sharpen. The remaining temporal noise is helped by increasing the frame rate from 24 to 60fps with Interframe. This not only adds more temporal smoothing, but the eye is less sensitive to noise at the higher frame rate. Brightening this particular source seems to help as well.

Allowing a small amount of twinkling/shimmering is OK, IMHO- to the uneducated eye it says "sharpness." The added blur of Interframe may allow you to back off some of the smoothing elsewhere.
Code:
LoadPlugin( "e:\apps\utils\Video tools\AVSplugins\RemoveGrainSSE3.dll" )
LoadPlugin( "e:\apps\utils\Video tools\AVSplugins\mt_masktools-26.dll" )
LoadPlugin( "e:\apps\utils\Video tools\AVSplugins\EEDI2.dll" )
LoadPlugin(pathBase + "mvtools-2.5.11.9-svp\mvtools2.dll")
Import(pathBase + "Interframe\InterFrame2.avsi")
LoadPlugin(pathBase + "SmoothAdjust-2,00\avs-2.5.x\x86\SmoothAdjust.dll")

SmoothLevels(0, 1.0, 160, 0, 255)

o=last 
ox=o.width 
oy=o.height

# round/soften hard corner pixels
o.pointresize(ox*3,oy*3).removegrain(4)
mt_lutxy(last,last.removegrain(11,-1),"x x y - 2.51 * +",U=2,V=2).bilinearresize(ox,oy)
o3=last

bubb0=o3.eedi2(maxd=8).bicubicresize(ox,oy,0,0,0,0.5)
\    .turnleft().eedi2(maxd=8)
\    .turnright().bicubicresize(ox,oy,0,0,-0.5,0)

return bubb0.InterFrame(NewNum=60, NewDen=1)
EDIT - one more thing:
Code:
return bubb0.InterFrame(NewNum=60, NewDen=1).Sharpen(0.4)

Last edited by raffriff42; 11th October 2013 at 22:19. Reason: sharpen
raffriff42 is offline   Reply With Quote
Old 11th October 2013, 21:52   #5  |  Link
Mystery Keeper
Beyond Kawaii
 
Mystery Keeper's Avatar
 
Join Date: Feb 2008
Location: Russia
Posts: 724
Do you really need to denoise it? The flickering you see doesn't seem like artifacts to me, but rather like usual 3D->2D projection result. I don't see z-order glitch either, so program has probably sorted the faces too. Though I can not be sure, since the objects look quite small. I too suggest using multisampling or supersampling->downsampling in the rendering program.
__________________
...desu!
Mystery Keeper is offline   Reply With Quote
Old 12th October 2013, 00:03   #6  |  Link
pancserzso
Registered User
 
Join Date: Oct 2004
Posts: 131
Thanks for the suggestions, but there is nothing I can do with the source at the moment. I am not sure if I need to do denoising or something else, but I would like to make it visually pleasing, without bright points / flickering in brightness.

Here are some filtering I have been trying:

Code:
DeGrainMedian(limitY=10, limitUV=13, mode=1, interlaced=false, norow=false)
DeGrainMedian(limitY=10, limitUV=13, mode=1, interlaced=false, norow=false)
=> result not too bad, handles the extreme noisy part quite well (2-7 sec)

Code:
FFT3DFilter(sigma=3, plane=4, bw=48, bh=48, bt=4, ow=16, oh=16, sharpen=0)
=> result

Code:
tr = 24
FFT3DFilter(sigma=12,bw=tr,bh=tr,ow=tr/2,oh=tr/2,bt=5,plane=0)
=> result

Code:
FFT3DFilter( sigma=6, plane=0, bt=5, bw=32, bh=32, ow=16, oh=16 )
=> result my favourite so far, but may be a bit too blurry

Code:
FluxSmoothST(temporal_threshold=7, spatial_threshold=7)
=> result

Code:
o = last
pre = o.fft3dfilter(sigma=8,sigma2=12,sigma3=10,sigma4=5,bw=24,bh=24,ow=12,oh=12,bt=5).FluxSmoothT(4)

sup1 = o.MSuper(hpad=16, vpad=16, pel=2, levels=1,sharp=2)
sup2 = pre.MSuper(hpad=16, vpad=16, pel=2, levels=0,sharp=0)
bv3  = MAnalyse(sup2,delta=3,truemotion=false,global=true,blksize=16,overlap=16/2,search=5,searchparam=4,isb=true, dct=5)
bv2  = MAnalyse(sup2,delta=2,truemotion=false,global=true,blksize=16,overlap=16/2,search=5,searchparam=4,isb=true, dct=5)
bv1  = MAnalyse(sup2,delta=1,truemotion=false,global=true,blksize=16,overlap=16/2,search=5,searchparam=4,isb=true, dct=5)
fv1  = MAnalyse(sup2,delta=1,truemotion=false,global=true,blksize=16,overlap=16/2,search=5,searchparam=4,isb=false,dct=5)
fv2  = MAnalyse(sup2,delta=2,truemotion=false,global=true,blksize=16,overlap=16/2,search=5,searchparam=4,isb=false,dct=5)
fv3  = MAnalyse(sup2,delta=3,truemotion=false,global=true,blksize=16,overlap=16/2,search=5,searchparam=4,isb=false,dct=5)

o.MDegrain3(sup1,bv1,fv1,bv2,fv2,bv3,fv3,thSAD=360)
=> result really nice looking result at the beginning (0-1 sec) and the end (7-16 sec), but really bad in the middle (2-7 sec)

Last edited by pancserzso; 12th October 2013 at 00:58.
pancserzso is offline   Reply With Quote
Old 13th October 2013, 22:23   #7  |  Link
cretindesalpes
͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿
 
cretindesalpes's Avatar
 
Join Date: Feb 2009
Location: No support in PM
Posts: 712
My attempt, similar to the last try of your previous post :
Code:
d     = 2
a1    = RemoveGrain (4).RemoveGrain (12)
a     = a1.TemporalSoften (4, 25, 25, 15, 2)
sup_a = a.MSuper ()
vec   = sup_a.MAnalyse (multi=true, delta=d, blksize=8, overlap=4, search=5, searchparam=4, DCT=5)
sup   = MSuper (levels=1)
MDegrainN (sup, vec, d, thSAD=800)
Quite effective, but a bit blurry at the end.
=> result
__________________
dither 1.28.1 for AviSynth | avstp 1.0.4 for AviSynth development | fmtconv r30 for Vapoursynth & Avs+ | trimx264opt segmented encoding
cretindesalpes is offline   Reply With Quote
Old 14th October 2013, 03:04   #8  |  Link
pancserzso
Registered User
 
Join Date: Oct 2004
Posts: 131
cretindesalpes, thanks a lot! It is almost perfect!! Only a bit too strong / blurry for my taste, because as you can see the most visible fruits in the animation are at the 'end' of their single path.

raffriff42, thanks, but I cannot modify the frame rate nor the brightness, I have to use the clip as it is.

I run the filters on the final clip, which is composed both from flickering fruits and smooth ones.

The challenge here is how to keep the details in the 'big' good examples while filtering the flickering ones out! Cretindesalpes' solution is the best so far, but it's a bit loosing too much details.

Cretindesalpes, would it be possible to either post sharpen or use a bit less filtering?

Here are the results from the final composition:
source-final

cretindesalpes

dgm573
Code:
DeGrainMedian(limitY=5, limitUV=7, mode=3, interlaced=false, norow=false)
DeGrainMedian(limitY=5, limitUV=7, mode=3, interlaced=false, norow=false)
dgm703
Code:
DeGrainMedian(limitY=7, limitUV=0, mode=3, interlaced=false, norow=false)
DeGrainMedian(limitY=7, limitUV=0, mode=3, interlaced=false, norow=false)
Didée old

fft3d-3-48-16
Code:
FFT3DFilter(sigma=3, plane=4, bt=4, bw=48, bh=48, ow=16, oh=16)
fft3d-6-32-16
Code:
FFT3DFilter(sigma=6, plane=0, bt=5, bw=32, bh=32, ow=16, oh=16)
flux
Code:
FluxSmoothST(temporal_threshold=7, spatial_threshold=7)
mdg3
Code:
o = last
pre = o.fft3dfilter(sigma=8,sigma2=12,sigma3=10,sigma4=5,bw=24,bh=24,ow=12,oh=12,bt=5).FluxSmoothT(4)

sup1 = o.MSuper(hpad=16, vpad=16, pel=2, levels=1,sharp=2)
sup2 = pre.MSuper(hpad=16, vpad=16, pel=2, levels=0,sharp=0)
bv3  = MAnalyse(sup2,delta=3,truemotion=false,global=true,blksize=16,overlap=16/2,search=5,searchparam=4,isb=true, dct=5)
bv2  = MAnalyse(sup2,delta=2,truemotion=false,global=true,blksize=16,overlap=16/2,search=5,searchparam=4,isb=true, dct=5)
bv1  = MAnalyse(sup2,delta=1,truemotion=false,global=true,blksize=16,overlap=16/2,search=5,searchparam=4,isb=true, dct=5)
fv1  = MAnalyse(sup2,delta=1,truemotion=false,global=true,blksize=16,overlap=16/2,search=5,searchparam=4,isb=false,dct=5)
fv2  = MAnalyse(sup2,delta=2,truemotion=false,global=true,blksize=16,overlap=16/2,search=5,searchparam=4,isb=false,dct=5)
fv3  = MAnalyse(sup2,delta=3,truemotion=false,global=true,blksize=16,overlap=16/2,search=5,searchparam=4,isb=false,dct=5)

o.MDegrain3(sup1,bv1,fv1,bv2,fv2,bv3,fv3,thSAD=360)

Last edited by pancserzso; 14th October 2013 at 03:13.
pancserzso is offline   Reply With Quote
Old 14th October 2013, 14:34   #9  |  Link
pandy
Registered User
 
Join Date: Mar 2006
Posts: 1,049
MLAA?

http://visual-computing.intel-research.net/publications/papers/2009/mlaa/mlaa.pdf
http://iryoku.com/aacourse/downloads/06-MLAA-on-PS3.pdf

However im not a AVS script guru...
pandy is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 09:03.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.