Moving rainbows in anime title

[snarfies]

A little derainbowing help, please?

Normally I use DFMDerainbow, and that works well for me, and occasionally I pull out Bifrost2 if I really need to. But I'm working on a title with a LOT of chain-link fences in it, and in a lot of scenes there's scrolling along the fences: DFMDerainbow and Bifrost are useless in non-static scenes like that.

I've tried experimenting with every other derainbowing filter I've been able to get my hands on: Chubbyrain2, ASTDR, SmartSSIQ, none of which touch these rainbows. Can anyone provide a bit of advice, please?

Sample video clip: https://drive.google.com/open?id=0BzzCX9wMG-ntRjZQbG45bnpIRmc

[kuchikirukia]

Tcomb?

[LemMotlow]

Tracking issues? Color issues? Looks like analog to DV to me, not a good choice if you expect to do lossless repair work. In any event, I gave it a try based on other posts with similar glitches and moire patterns. Probably a dozen ways to do it, none perfect.

CamcorderColorDenoise (aka "CCD") also helped a little. The original sample is telecined (3:2 pulldown) -- and has twittery borders and dot crawl. You can use SmoothUV() instead of BiFrost(), same result. You need a cleaner capture setup, not DV or composite.

I used this (slow as hell):

LoadVirtualDubPlugin("path\to\VirtualDub\plugins\ccd.vdf", "ccd",0) #<- adjust path statement for your setup

ConverttoYUY2(interlaced=true)
Crop(0,2,0,0).AddBorders(0,2,0,0)
TFM(pp=0).TDecimate()
ConvertToYV12(interlaced=false)
QTGMC(InputType=2,preset="faster",border=true)
MergeChroma(MCTemporalDenoise(settings="very high"))
BiFrost()
ReplaceFramesMC2(49,4)
ReplaceFramesMC2(114,1)
ConvertToRGB32(interlaced=false)
Crop(10,0,-6,-2).AddBorders(8,0,8,2)
ccd(40,1)
ConvertToYV12(interlaced=false)

snarfies_nine_rainbows_rework.mp4 -> https://www.mediafire.com/?m0ndi0ty0mrpfkg
Other ideas?

[Taurus]

@Lem
Nice job with this messy source.
Your script is slow as hell...:devil::)

[LemMotlow]

Yes, it is. It's the way it runs MCTemporalDenoise (aka MCTD). I've used it this way many times, but the big secret if you don't have a mega-PC is to run the script in two steps. First step is running everything up to and including QTGMC, save that as a lossless intermediate, then open it again with a second script that has MCTD and the remaining lines.

[poisondeathray]

Very nasty pattern looking at U,V channels. It looks tough to fix properly without causing too much damage or blurring the chroma too much. Maybe some method of using Y to guide the U,V fix ?

If they are mostly static background with some panning, you can use motion tracking / stabilization / clean plates to "fix" it . Avisynth can be used for some of it, such as part of the clean plate generation (e.g. median or merge on some of the stabilized sequence). Since it is a "static BG" , you can actually replace everything, even the dark smudge that occurs briefly on the lower border . You can expand and fill in the frame edges and pillarbox (edge fill) if you wanted to. Or you can choose to fix only some problems. Or you can merge the fixed U,V back with the original Y if you wanted to retain the original Y noise pattern minus the rainbows (maybe not on stuff like this, but for things like original film grain). It takes some manual work and needs to be done per scene - it takes a few minutes per scene if they are this simple (or can be much longer depending on complexity), and can be done with fairly common tools like Adobe AE. Let me know if you want more info.

https://www.mediafire.com/?xicfn3e0ddetb2a

[LemMotlow]

I get the general idea, and would appreciate seeing a link to further material since I have AE installed but never used it for the procedure mentioned. OTH, I have a feeling the entire video has similar rainbow and other problems, the submitted problem sample is only a small piece of the pie. I wonder, too, if snarfies or other readers have AE.

[real.finder]

http://rgho.st/private/89QQzxHkZ/e281df30bf47b266d7bf2a6e0d93b0cf

A quick test


SMDegrain (http://forum.videohelp.com/threads/369142-Simple-MDegrain-Mod-v3-1-2d-A-Quality-Denoising-Solution?p=2413356&viewfull=1#post2413356)(prefilter=2,tr=10,thSADc=1000,interlaced=true,chroma=false,truemotion=false,plane=3)
tfm(pp=0)
TDecimate(1)
daa3mod

[LemMotlow]

http://rgho.st/private/89QQzxHkZ/e281df30bf47b266d7bf2a6e0d93b0cf

A quick test


SMDegrain (http://forum.videohelp.com/threads/369142-Simple-MDegrain-Mod-v3-1-2d-A-Quality-Denoising-Solution?p=2413356&viewfull=1#post2413356)(prefilter=2,tr=10,thSADc=1000,interlaced=true,chroma=false,truemotion=false,plane=3)
tfm(pp=0)
TDecimate(1)
daa3mod
That script would probably work if I had a version of SMDegrain that worked. I got a little weary from collecting all the versions, mods, re-mods, revisions, updates, update mods, and whatnot, and I don't ever want to see daa3mod again, among others. I downloaded the new 11/2016 mod anyway. Maybe next week I'll have time for all the required support mods, support re-mods, support updates, etc., if their download links don't disappear by then. Seriously. Avisynth has some great stuff, and some nice docs for developers, but I've never been able to make SMDegrain run, anywhere, any time, so I can't trust it. Maybe snarfies can give it a go. What happened to him/her, anyway?

[EDIT] But I tried it anyway. Oh, well, back to the drawing board. Probably something very simple.

[Taurus]

@Lem.
Would you mind sharing your version of ReplaceFramesMC2?
I found an ancient function that works.
But I also found versions that use svflow and more.
I don't want to clutter my avisynth directory with unused stuff :(;)
Thanks in advance

[kuchikirukia]

Smdegerain. I think this is everything.

http://s000.tinyupload.com/?file_id=78846684607212255611

[LemMotlow]

@Lem.
Would you mind sharing your version of ReplaceFramesMC2?
Mine is attached, from an old Videohelp post.
[EDIT]Attachment removed. Scroll down a couple of posts to find the script.

Smdegerain. I think this is everything.

http://s000.tinyupload.com/?file_id=78846684607212255611
Yes, that's last year's which I have, thanks. The version mentioned in the earlier post is SMDegrainMod from 2016.
As I said, probably something very simple. Meanwhile I just use the plain MDegrain's from MVTools.

[Taurus]

@Lem:
Waiting for the attachments pending approval.
Thanks anyway :thanks:

[LemMotlow]

@Lem:
Waiting for the attachments pending approval.
Thanks anyway :thanks:Shucks. Well here it is (should have done this way in the fist place, I just happened tyo have it zip'd):
# ######################################################################################################
#
# http://forum.videohelp.com/threads/353818-luma-error-on-VHS-restoration?p=2224484&viewfull=1#post2224484
#
# Note: This function was originally called "RX".
#
########################################################################################################

function ReplaceFramesMC2(clip Source, int N, int X)
{
# N is number of the 1st frame in Source that needs replacing.
# X is total number of frames to replace
# e.g. "RX(101, 5)" would replace 101,102,103,104,105, by using
# 100 and 106 as reference points for mflowfps interpolation.

start=Source.trim(N-1,-1) #one good frame before, used for interpolation reference point
end=Source.trim(N+X,-1) #one good frame after, used for interpolation reference point

start+end
AssumeFPS(1) #temporarily FPS=1 to use mflowfps

super = MSuper(pel=2, hpad=0, vpad=0, rfilter=4)
backward_1 = MAnalyse(super, chroma=false, isb=true, blksize=16, searchparam=3, plevel=0, search=3, badrange=(-24))
forward_1 = MAnalyse(super, chroma=false, isb=false, blksize=16, searchparam=3, plevel=0, search=3, badrange=(-24))
backward_2 = MRecalculate(super, chroma=false, backward_1, blksize=8, searchparam=1, search=3)
forward_2 = MRecalculate(super, chroma=false, forward_1, blksize=8, searchparam=1, search=3)
backward_3 = MRecalculate(super, chroma=false, backward_2, blksize=4, searchparam=0, search=3)
forward_3 = MRecalculate(super, chroma=false, forward_2, blksize=4, searchparam=0, search=3)
MBlockFps(super, backward_3, forward_3, num=X+1, den=1, mode=0)

AssumeFPS(FrameRate(Source)) #return back to normal source framerate for joining
Trim(1, framecount-1) #trim ends, leaving replacement frames

Source.trim(0,-N) ++ last ++ Source.trim(N+X+1,0)
}

[StainlessS]

There maybe solution, which is adaptive block :)
The things is that Groucho2004 is not using 1 command with a block size, but recalculate option, which I assume may help with quality.

It definitely helps with denoising and stuff like that, not with frame interpolation
http://forum.doom9.org/showthread.php?p=1783609#post1783609

MRecalculate may be just a time waster.

See here for how Overlap can change results, but some versions (all except pinterf, I think) are broken for Overlap>0 without at least 1 MRecalculate(Overlap=0).
http://forum.doom9.org/showthread.php?p=1786042#post1786042

https://s20.postimg.org/4cohur1j1/Olap8_zpsbrsjy3u5.png (https://postimg.org/image/grb9v2t15/)


EDIT: I guess we really need image with OLap=0 and MRecalculate and Flow=True/False.
EDIT: Also, hpad/vpad is beneficial in frame interpolation.

[Taurus]

Shucks. Well here it is (should have done this way in the fist place, I just happened tyo have it zip'd)
@Lem:
Thanks for sharing!

[StainlessS]

Here without Overlap, MRecalculate do make a difference


Function ReplaceFramesMC_Test(clip Source, int N, int X,Bool "Flow",Bool "Recalc",Bool "Pad") {
Flow = Default(Flow,False)
Recalc = Default(Recalc,False)
Pad = Default(Pad,False)
# N is number of the 1st frame in Source that needs replacing.
# X is total number of frames to replace
# e.g. "RX(101, 5)" would replace 101,102,103,104,105, by using
# 100 and 106 as reference points for mflowfps interpolation.

start=Source.trim(N-1,-1) #one good frame before, used for interpolation reference point
end=Source.trim(N+X,-1) #one good frame after, used for interpolation reference point

start+end
AssumeFPS(1) #temporarily FPS=1 to use mflowfps

super = MSuper(pel=2, hpad=(PAD)?16:0, vpad=(PAD)?16:0, rfilter=4)
bv = MAnalyse(super, chroma=false, isb=true, blksize=16, searchparam=3, plevel=0, search=3, badrange=(-24))
fv = MAnalyse(super, chroma=false, isb=false, blksize=16, searchparam=3, plevel=0, search=3, badrange=(-24))
bv = (ReCalc)?MRecalculate(super, chroma=false, bv, blksize=8, searchparam=1, search=3):bv
fv = (ReCalc)?MRecalculate(super, chroma=false, fv, blksize=8, searchparam=1, search=3):fv
bv = (ReCalc)?MRecalculate(super, chroma=false, bv, blksize=4, searchparam=0, search=3):bv
fv = (ReCalc)?MRecalculate(super, chroma=false, fv, blksize=4, searchparam=0, search=3):fv
(Flow)
\ ? MFlowFps( super, bv, fv, num=X+1, den=1)
\ : MBlockFps(super, bv, fv, num=X+1, den=1, mode=0)
AssumeFPS(FrameRate(Source)) #return back to normal source framerate for joining
Trim(1, framecount-1) #trim ends, leaving replacement frames

Source.trim(0,-N) ++ last ++ Source.trim(N+X+1,0)
}

Function Sub(clip c,string tit) {
S = "SubTitle("+Chr(34)+tit+Chr(34)+")"
StackVertical(c.BlankClip(height=20).ScriptClip(S),c)
}

Function Test(Clip c,Int n,Int x,Bool "Flow",Bool "Recalc",Bool "Pad") {
Flow = Default(Flow,False) Recalc = Default(Recalc,False) Pad = Default(Pad,False)
c.ReplaceFramesMC_Test(n,x,Flow,Recalc,Pad).Sub("Flow="+String(Flow)+" Recalc="+String(Recalc)+" Pad="+String(Pad))
Return Trim(n,-1)
}

FN="F:\v\StarWars.avi"
Avisource(FN)
PAD=true
STARTFRAME=24057
A=Test(STARTFRAME,1,Flow=False,Recalc=False, Pad=PAD)
B=Test(STARTFRAME,1,Flow=False,Recalc=True , Pad=PAD)
C=Test(STARTFRAME,1,Flow=True ,Recalc=False, Pad=PAD)
D=Test(STARTFRAME,1,Flow=True ,Recalc=True , Pad=PAD)

H = A.Blankclip(Height=4,color=$FFFFFF)
LFT = StackVertical(A,H,B)
RGT = StackVertical(C,H,D)
V = LFT.Blankclip(Width=4,color=$FFFFFF)
Return StackHorizontal(LFT,V,RGT)


https://s20.postimg.org/qpw8hk2gt/MCTest_zpsttqaqjsa.png (https://postimg.org/image/8n35qc6m1/)

EDIT: Above is not exactly same as prev images, prev images synthesized new frame between n and n+1, whereas this script deletes frame
n and replaces with frame synthesized with n-1 and n+1.

EDIT: Mod of previous script (end lines only), duplicates each frame, then produces similar result to compare with images from other thread.

FN="F:\v\StarWars.avi"
Avisource(FN)
SelectEvery(1,0,0)
PAD=true
#STARTFRAME=24058
STARTFRAME=48115
A=Test(STARTFRAME,1,Flow=False,Recalc=False, Pad=PAD)
B=Test(STARTFRAME,1,Flow=False,Recalc=True , Pad=PAD)
C=Test(STARTFRAME,1,Flow=True ,Recalc=False, Pad=PAD)
D=Test(STARTFRAME,1,Flow=True ,Recalc=True , Pad=PAD)

H = A.Blankclip(Height=4,color=$FFFFFF)
LFT = StackVertical(A,H,B)
RGT = StackVertical(C,H,D)
V = LFT.Blankclip(Width=4,color=$FFFFFF)
Return StackHorizontal(LFT,V,RGT)


https://s20.postimg.org/6wk4oup31/MCTest2_zpsd8v1ldgj.png (https://postimg.org/image/iyfiizybd/)

EDIT: HPAD, seems to make a difference when there are verticals near left and right edges (sometimes produces oscillating edges [disappear, reappear] without HPAD, some clips only, maybe with horizontal jitter), presumably VPAD does same for top/bottom, but cant say I've ever noticed problems there.

EDIT: I've only just noticed, but the MC function with MRecalculate uses different SearchParam at each stage, perhaps need to play with that also.

[LemMotlow]

Thank you, Stainless. I have Have another project where ReplaceFramesMC2 solved one glitch but caused another, lesser one. Will try yours.

[snarfies]

Okay, thanks guys! Looks like this SMDegrain is the way to go, will try experimenting with it soon.

[snarfies]

It takes some manual work and needs to be done per scene - it takes a few minutes per scene if they are this simple (or can be much longer depending on complexity), and can be done with fairly common tools like Adobe AE. Let me know if you want more info.

https://www.mediafire.com/?xicfn3e0ddetb2a

Well, I like the results, and I don't know that I'm looking to do this on over three hours of footage, but sure, I'm interested in knowing a little more.

[poisondeathray]

I get the general idea, and would appreciate seeing a link to further material since I have AE installed but never used it for the procedure mentioned. OTH, I have a feeling the entire video has similar rainbow and other problems, the submitted problem sample is only a small piece of the pie. I wonder, too, if snarfies or other readers have AE.

Well, I like the results, and I don't know that I'm looking to do this on over three hours of footage, but sure, I'm interested in knowing a little more.




<Sorry for the long post>


My impression from the description was some problem areas with "Scrolling chain link fences", and a lot of them. So I would use "normal" methods for everything else, you just divide it up (e.g with trim() or even in a NLE). If you can provide a better description of the other sections maybe there are other ways or combinations to tackle this that are easier or more efficient. Or if there are more complex scenes that are more challenging to track that have the fences (e.g. moving foreground objects with the fences) that would be good to know too...

I think real.finder 's smdegrain suggestion looks ok. Like many temporal solutions, there is a "lag" period before the filter kicks in full strength , maybe a way around that reverse() the sequence then replace those frames for each scene that requires it?

You actually don't need AE (it's just easy to use and quite common - AE comes with the Adobe subscription) , basically any compositing tool, like blender (free) , or commercial freeware (is that an oxymoron?) like hitfilm, or nuke, or fusion (all have free versions now with limitations such as non commercial useage, limited to 1080p for some of them, a few other limits). Anyone familiar with those types programs should know already what I was talking about, but otherwise there is a bit of a learning curve

A point tracker is all you need for tracking and stabilization for 2d content that have simple motion like this. There are a gazillion tutorials on youtube or vimeo on point tracker and stabilization. To summarize - You just put the tracking crosshairs on the fence somewhere high contrast that stays in the scene (like the vertex where fence poles cross) and push a button. On a scale of 1-10 for difficulty in tracking this type of scene is about a 1.5 (I added .5 because of a bit of noise, a bit of compression artifacts) . Even if there were foreground objects, or points move off scene, there are ways to handle it, and there are more complex trackers in AE and the bundled mocha that can handle more difficult tracking scenarios. Motion tracking once, serves 2 purposes : 1) to stabilize for clean plate generation ; 2) to track the composite fix back in so it "sticks" or "covers up" the defects on the original. In the example provided, actually everything was covered up; but in more typical cases, you typically only "cover up" specific defects by compositing only those parts.

When you stabilize it (I'm talking rock stable stabilization, not something like deshaker) , you negate the pan. So every frame overlaps superimposed on the previous frame, as if the camera was locked off on a tripod. Since the frames are now aligned spatially and temporally , you can use median() or mean (merge() in avisynth) or various other averaging techniques to help with clean plate generation on a sequence of frames. It won't be "perfect" by itself, because if the motion was a linear pan, you need to fix at least 1 trailing edge (e.g. cloning, content aware fill or other techniques). If camera pans L to R, this means the BG moves from viewer R to L . When stabilized, the frames will move viewer L to R increasing over time. Thus the L border will be increasing the more frames you take to average (actually alpha channel / transparent) . But the central portion will be good and aligned if the track was stable. If you look closely at the file I uploaded you can probably see defects in the peripheral edges because I did them very quickly. You can actually do it automatically with a batch fill action in PS . Or fill with various inpainting (e.g. exinpaint in avisynth) , mirroring, other techniques etc... Sometimes you need a few clean plates per scene depending on the movement of the shot pan. The more movement, the more clean plates needed for coverage. If you can imagine a lengthy tracking/dolly shot - that might require many clean plates. That example used one at the beginning , one at the end. The averaging with stabilized plate is a time saver in generating clean plate(s) - they might only need minor touchups.

You can choose where you want a clean plate to be taken from , or reframe the shot by using an offset null object . So for example, if I take the very last frame, and have applied stabilization only based on the first frame as reference, it will only show the right edge with a large black (actually transparent border) . That's not ideal for clean plate because you're missing 1/2 the picture. You want to shift all the frames over by the same "x" amount (so they are still aligned). You do this by parenting the the null object on the frame of interest, then moving the null. This moves all the "children" (which is the stabilized sequence of frames) by the same amount. So you can "center your window" and pick which frame range to use for the averaging. Once you have your clean plates, you link the plates back to the motion data by parenting on the same frame from which they were taken from . This probably won't make sense to anyone unless they've used AE or some compositing tool before

Because the chroma was quite messed up, you needed more frames than you usually would when using merge or median or various other averaging methods. But the side effect is the luma is also impacted because AE is working in RGB. Sometimes that's good if you want to clean up everything, including random luma defects like the dark blip in this example. But sometimes it can over soften. But you can control that of course with mergeluma() in avisynth; you might want the original luma , or maybe only mild filtering, or differential filtering of the luma compared to the chroma

So the basic tutorials you would look for involve the keywords "motion tracking", "point tracker" , "parenting", "compositing", "clean plate". There won't be any perfect all in 1 tutorial that covers everything for this scenario. It's just a bunch of basic concepts used together to fix something else. Normally all those tools and workflow are used for different types of work. If you need more info or help just ask.

[snarfies]

SMDegrain (http://forum.videohelp.com/threads/369142-Simple-MDegrain-Mod-v3-1-2d-A-Quality-Denoising-Solution?p=2413356&viewfull=1#post2413356)(prefilter=2,tr=10,thSADc=1000,interlaced=true,chroma=false,truemotion=false,plane=3)


Throws the message: "Denoising chroma with luma only vectors is bugged in mvtools and this not supported (SMDegrain v3.1.2d.avsi, line 113)"

[StainlessS]

Snarfies, I'm using current Pinterf mod of MvTool v2.0:- http://forum.doom9.org/showthread.php?t=173356

I commented out line 118 (not 113) ie

#!chroma ? Assert(plane==0, "Denoising chroma with luma only vectors is bugged in mvtools and thus unsupported") : nop()


and it seems to work ok, Pinterf must have fixed the problem (rainbows all gone, nice one Real.Finder).

EDIT: I'm using 2016 06 23 mod version SMDegrain (which I named "SMDegrain_20160623_mod.avs"):- http://pastebin.com/tTcXNNtw

EDIT:
Quote by LemMotlow
That script would probably work if I had a version of SMDegrain that worked. I got a little weary from collecting all the versions, mods, re-mods, revisions, updates, update mods, and whatnot

Using Pinterf Mvtools 2.0, (link above)
Masktools v2.1,(avs v2.6 version):- http://avisynth.nl/index.php/MaskTools2
Medianblur plugin by tsp:- http://avisynth.nl/index.php/MedianBlur
No problems at all (1st time that I've ever tried SMDegrain).

EDIT: Just updated to new pinterf upload

Change log
2.7.6.22 (20161204) - fixes and speedup

EDIT: from Wiki:- http://avisynth.nl/index.php/SMDegrain
Optional

Dither for 16 bit processing (v1.26.5 or higher)
RgTools for contrasharp=true or prefilter=1 or 2
LSFmod for contrasharp > 0.0
dfttest for prefilter=3
nnedi3 for subpixel=3

[snarfies]

Snarfies, I'm using current Pinterf mod of MvTool v2.0:- http://forum.doom9.org/showthread.php?t=173356

I commented out line 118 (not 113) ie

#!chroma ? Assert(plane==0, "Denoising chroma with luma only vectors is bugged in mvtools and thus unsupported") : nop()


and it seems to work ok, Pinterf must have fixed the problem (rainbows all gone, nice one Real.Finder).

EDIT: I'm using 2016 06 23 mod version SMDegrain (which I named "SMDegrain_20160623_mod.avs":- http://pastebin.com/tTcXNNtw



That combo seems to have done the trick, thanks!

Is there any sort of proper documentation for SMDegrain? The settings I've been handed seem to do the trick nicely, but I'd like to understand what those settings, like, actually are. the VideoHelp thread: (http://forum.videohelp.com/threads/369142) told me next to nothing, and I don't really understand the code of the .avsi file itself.

[snarfies]

So the basic tutorials you would look for involve the keywords "motion tracking", "point tracker" , "parenting", "compositing", "clean plate". There won't be any perfect all in 1 tutorial that covers everything for this scenario. It's just a bunch of basic concepts used together to fix something else. Normally all those tools and workflow are used for different types of work. If you need more info or help just ask.

My, that's a lot to take in for a hack amateur like myself, but thanks for the writeup. I will have to explore that at some point... perhaps not today though.

[poisondeathray]

Is there any sort of proper documentation for SMDegrain?


The wiki has a short explanation of some of the switches
http://avisynth.nl/index.php/SMDegrain

[real.finder]

Snarfies, I'm using current Pinterf mod of MvTool v2.0:- http://forum.doom9.org/showthread.php?t=173356

I commented out line 118 (not 113) ie

#!chroma ? Assert(plane==0, "Denoising chroma with luma only vectors is bugged in mvtools and thus unsupported") : nop()


and it seems to work ok, Pinterf must have fixed the problem (rainbows all gone, nice one Real.Finder).

EDIT: I'm using 2016 06 23 mod version SMDegrain (which I named "SMDegrain_20160623_mod.avs"):- http://pastebin.com/tTcXNNtw



the last one always here http://forum.videohelp.com/threads/369142-Simple-MDegrain-Mod-v3-1-2d-A-Quality-Denoising-Solution?p=2413356&viewfull=1#post2413356 ;)

it wasn't bug, but with only luma you will not get accurate vectors
see here http://forum.doom9.org/showthread.php?p=1783275#post1783275

maybe if you did it in something more than 8 bit you can get better vectors, and if your source is 8 bit then I think if you did prefilter in something more than 8 bit you will get better vectors, but I didn't try that and don't know if it possible to have prefilter with bit depth different from source

edit: Medianblur plugin by tsp:- http://avisynth.nl/index.php/MedianBlur
why not http://avisynth.nl/index.php/MedianBlur2 ?