SpotLess - Temporal denoise

[FranceBB]

That is actually correct.
You're bobbing, filtering, dividing in fields, so that's alright.
By the way, I would encourage users to do it themselves rather than relying on it 'cause it's definitely gonna work for truly interlaced sources which will be bobbed to 59.940p or 50p, however, on telecined sources it will make the temporal prediction of MVTools useless, besides, even with truly interlaced sources, with Bob, fields would jump "up-down-up-down" making it harder to analyze and filter, so it's always better to let users handle it (hoping they'll use something decent as a deinterlacer and that they're aware of their source).
As to the alternative, again, you're right, as you're filtering each field separately and then putting them back together.
The second approach is probably the safest and I prefer it more than bobbing and re-interlacing. :)


Well done, Stainly, you're really following the government advice:

https://i.imgur.com/b3KhBsF.png

[StainlessS]

Cheers FaB, tanks for feedback.
I'll not bother adding to base script then.

EDIT: By the way, Bobber default of "Bob" is only because its a builtin filter, I would not recommend it.

[Emulgator]

FranceBB, now that poster is brilliant !

[piscator]

NOT FOR cartoon/anime, live video only, sorry.

What limitations are there that it should not work so great for cartoons? I have some DVDs with old children cartoons from 1956+ (Mole TV series, children cartoon) which I am trying to clean up. I have been looking for a bit to clean the spots and scratches (black and white, some quite large) from it and SpotLess works great on it :D Thanks, StainLessS!

[StainlessS]

Thanks for the feedback piscator,
MvTools motion compensation type stuff dont usually work to well on anime.
Did you use any non default args ? [maybe large block size, which I thought might help].

[piscator]

I just used default arguments.
Perhaps it works so well because this cartoon is quite 'static'. The background is usually completely fixed with sometimes a bit of panning /zooming on it. So, it is all really quite slow / limited motion.

[StainlessS]

static background might help,
anyways glad you are happy.

[FranceBB]

That's because cartoons / anime are generally not with the real framerate.
I mean, if you go through an anime encoded at 23.976fps, you'll almost definitely find repeated frames, but it's not a mistake, they're there by purpose as whoever animated it (I mean the studio) wasn't really able to deliver so many frames per seconds in terms of each individual character moving a part of the body etc. These repeated frames don't generally work well with MVTools, hence the assumption that it might not work for anime.
In your case it worked absolutely fine, so don't get me wrong, you might be lucky and have it working perfectly fine, but generally speaking, MVTools doesn't like non-live contents. ;)

FranceBB, now that poster is brilliant !

Ehehehe thanks

[piscator]

I mean, if you go through an anime encoded at 23.976fps, you'll almost definitely find repeated frames

Thanks, I can understand why that would make it more difficult. My source was just 25fps interlaced. And the motion is so slow, that when dropping half of the frames, you barely see a difference :D

I encoded this source for fun on crf 30 (x265, Ultralow) and it looks the same as on higher bitrates. But degraining, descratching and removing the spots make it much better.

[Frank62]

An idea:
Maybe possible to
-drop the dups, log it, maybe in StainlessS' database thing
-removedirt/spotless
-take the logged dups in another clip
-removedirt these
-bring everything together again
Could be a cleaning-chance for old anime :)

[Frank62]

Sorry, bad idea. Should rather be the opposite. Doubles are good for removedirting. Motion-compensated frames are of course less good.
So forget my proposal...

[StainlessS]

Maybe just try the temporal median filter that spotless uses.

RadT = ??? [int, try bout 2]
MedianBlurTemporal(radiusY=0,radiusU=0,radiusV=0,temporalradius=RadT) # Temporal median blur only [not spatial]

If it is so static, it might work well.

[Frank62]

What exactly makes it more static than f. e. RemoveDirt?
To me it seems somehow more stepless, but how does it do this? "Blur" implicates some temporal interpolating, but it seems to be more than this.

[StainlessS]

Dont know but,
Here it is on Wiki:- http://avisynth.nl/index.php/MedianBlur2
And link (from wiki page) to a paper on it:- http://nomis80.org/ctmf.html

[Frank62]

Thanks, but it says nothing about the temporal part of filtering.

[Arx1meD]

Another variant of the script with saving fast moving objects.


# MvTools2, RemoveDirt, RgTools, DeSpot
Function SpotLess2(clip c, int "RadT", int "Spot", bool "DeGrain", int "RadT", int "ThSAD", int "ThSAD2", int "pel", bool "chroma", int "BlkSz", Int "Olap", bool "tm", Bool "glob") {
RadT = Default(RadT, 1) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD, 10000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2, ThSAD) # SAD threshold at radius RadT.
Pel = Default(pel, 2) # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma = Default(chroma, true) # MAnalyse chroma arg. If set to true, use chroma in block matching.
BlkSz = Default(BlkSz, 8) # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips. Maybe also better where BIG noise.
OLap = Default(OLap, 2) # Default half of BlkSz.
Tm = Default(tm, true) # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
Glob = Default(glob, true) # Default True, Allow set MAnalyse(global) independently of TrueMotion.
Spot = Default(Spot, 32) # Spot size
DeGrain = Default(DeGrain, false) # MDeGrain in MvTools

pad = max(BlkSz, 8)
sup = c.ColorYUV(levels="PC->TV").MSuper(hpad=pad, vpad=pad, pel=pel, sharp=2)
sup_rend = MSuper(c, hpad=pad, vpad=pad, pel=pel, sharp=2, levels=1) # Only 1 Level required where not MAnalyse-ing.
MultiVec = MAnalyse(sup, multi=true, delta=RadT, blksize=BlkSz*2, blksizeV=BlkSz*2, overlap=OLap*2, chroma=Chroma, truemotion=Tm, global=Glob)#, search=5, dct=7, divide=0, plevel=0)

# c0 = DeGrain ? MDeGrainN(c1, sup_rend, MultiVec, RadT, mt=true, plane=4, limit=255.0, thSad=400) : c # gave a bad result

c1 = MCompensate(c, sup_rend, MultiVec, tr=RadT, thSad=ThSAD, thSad2=130)
c1 = c1.ForwardClense() #faster than MedianBlurTemporal(radiusY=0, radiusU=0, radiusV=0, temporalradius=RadT) # Temporal median blur only [not spatial]

c1 = Repair(SelectEvery(c1, RadT*2+1, RadT), c.RemoveDust().Sharpen(0.1), mode=17) # Return middle frame

bvec1 = MAnalyse(sup, isb=true, blksize=BlkSz, blksizeV=BlkSz, overlap=OLap, divide=0, delta=1, search=5, dct=7, plevel=0, truemotion=true) # backward vectors
fvec1 = MAnalyse(sup, isb=false, blksize=BlkSz, blksizeV=BlkSz, overlap=OLap, divide=0, delta=1, search=5, dct=7, plevel=0, truemotion=true) # forward vectors
backw1 = MFlow(c1, sup_rend, bvec1)
forw1 = MFlow(c1, sup_rend, fvec1)
clipa = interleave(backw1, c1, forw1)

filtered = DeSpot(clipa, p1=6, p2=6, pwidth=Spot, pheight=Spot, mthres=16, mwidth=Spot/2, mheight=Spot/2, blur=1, motpn=true, seg=2, color=false, ranked=true, show=0)
restore = clipa.RemoveDust().Sharpen(0.1)

clipb = RestoreMotionBlocks(filtered, restore, alternative=RemoveGrain(clipa,2), pthreshold=10, cthreshold=10, gmthreshold=160, dist=3, dmode=2, noise=3, noisy=9, grey=false, debug=false, show=false)

SelectEvery(clipb, 3, 1)

DeGrain ? MDeGrain1(last, sup_rend, bvec1, fvec1, plane=4, limit=255.0, thSad=400) : last
}


function RemoveDust(clip input, int "mode") {
mode = Default(mode, 17)

clensed = Clense(input)
rep=Repair(clensed, input, mode=mode)

RemoveGrain(rep)
}

[StainlessS]

Arx1meD,
Perhaps you should create your own thread for your script function, also maybe give it another name other than Spotless2(),
as it does not function much at all like the Spotless filter.
I only tried this to see if it threw an error or not due to you having 2 [EDIT: formal] args named RadT,


AviSourcE("D:\parade.avi")
Spotless2()


However, caused access violation on line 33, ie the DeSpot line.

Suggest you create new thread, and correct/rename your script filter,
and post an example sample clip together with a working client script.
EDIT:

Function SpotLess2(clip c, int "RadT", int "Spot", bool "DeGrain", int "RadT", int "ThSAD", int "ThSAD2",
\ int "pel", bool "chroma", int "BlkSz", Int "Olap", bool "tm", Bool "glob") {


also, it is not good practice to destroy compatibility by inserting new formal arguments before existing arguments ie your

int "Spot", bool "DeGrain"


EDIT: Version Despot I was using was v3.6.3.0, there is a v3.6.3.1 here:- http://www.avisynth.nl/index.php/DeSpot
V3.6.3.1 does not cause access violation.

Strangely, it does not throw an error due to 2 different instances of the formal parameter "RadT", maybe a bit more checking needed in Avs+.

[Arx1meD]

Thank you StainlessS. I will fix the errors and create another thread.
I am using Avs+. When testing the script, I got no errors.

[StainlessS]

New Test [1.07beta] version of SpotLess, comments ?

Changed bBlur default from 0.0 [OFF] to 0.6, slower but should be considerably better on MC analysis.
Would it be preferable to remain at 0.0 faster.

Also, added DC [prefiltered detection clip], where supplied, bBlur prefiltering ignored and uses dc clip instead.


/*
SpotLess v1.07Beta. temporal spot/noise remover, by StainlessS @ Doom9. https://forum.doom9.org/showthread.php?t=181777
Original idea from Didée post :- https://forum.doom9.org/showthread.php?p=1402690#post1402690
Req:- Pinterf MvTools2(), Pinterf Medianblur2() with MedianBlurTemporal rather than the MedianBlurT() from Didée post.
With appropriate plugins, will be AVS+ colorspace and x64 compatible.
Fine using Avs+ Prefetch() so long as current Pinterf plugins, and Frame Accurate source. RequestLinear() following the source filter might suffice as frame accurate source filter.

NOT FOR cartoon/anime, live video only, sorry.

v1.01, Remove RadT Max check. Add MSuper(hpad=16,vpad=16). Add BlkSz arg.
v1.02, Add some stuff.
v1.03, Frame size auto config of args removed (found instances where caused problems). Added Glob and bBlur args.
v1.04, Script breaking changes, I guess is more flexible if can also set ThSAD, inserted ThSAD 3rd arg. RadT now default 1, was 2, dont over denoise unless requested.
v1.05, Additional checks on args.
v1.06, Glob Default true, Almost always better.
v1.07Beta, bBlur Default changed from 0.0 OFF, to 0.6, slower but better for mc analysis. Maybe try 1.0 for HD to overcome HD 'gritiness'.
Added dc arg, detect clip.

SpotLess(clip c,int "RadT"=1,int "ThSAD"=10000,int "ThSAD2"=ThSAD,int "pel"=2,bool "chroma"=true, int "BlkSz"=8,Int "Olap"=BlkSz/2,
\ bool "tm"=true,Bool "glob"=True,Float "bBlur"=0.6, clip "dc"=Undefined)

RadT, 1 or more, Default 1. Removes Spots on up to RadT [Temporal Radius] consecutive frames.
RadT > 2 will usually be overkill. Setting too high could possibly result in blurring.
Each pixel in result frame is median pixel value of (2*RadT+1) motion compensated frames (including source, ie current_frame-RadT to current_frame+RadT).

ThSAD, Default 10000=NEARLY OFF(ie ignore hardly any bad blocks), 0 < ThSAD < 16320(8*8*255). 8x8 block SAD threshold at radius 1 (ie at current_frame +- 1) [SAD, Sum of Absolute (pixelwise) Differences].
ThSAD and ThSAD2 suggested absolute minimum of maybe about 400.
ThSAD and ThSAD2 thresholds are based on 8 bit 8x8 block, irrespective of colorspace depth or BlkSz, max=8x8x255=16320, use same thresholds where High Bit Depth.
In mvTools MCompensate(), when creating a compensated block the SAD between compensated block and the same original block in current_frame, the 8 bit SAD is measured and if
greater than SAD threshold then that block is ignored and uses original block from current frame instead. [The compensated block is judged too different, so ignored & original block used instead
in the result MCompensated frame].
Where eg ThSAD=64, AVERAGE absolute single pixel difference threshold would be 64/(8*8)=1, so AVERAGE absolute pixel difference greater than 1 would ignore that mcompensated block and use the
block from current frame in the resulting mcompensated frame instead. This example allows for all pixels in a 8x8 block to be different by 1, or a single pixel in 8x8 block to be different by 64,
or some other mixture.
A problem with above is, if a low ThSAD and current_frame block is mostly noise, so compensated blocks could be judged bad because they are too different to the bad noisey block, and the result
block may/will be just as bad as the noisy source block. A possible solution to this problem is to have a higher SAD threshold and/or have a bigger BlkSize so that the number of bad source pixels
after converting/scaling to as if an 8x8 block, will contain fewer bad noise pixels. So, SpotLess BlkSz arg would ideally maybe 4 or more times the area of the largest spots that you have, and a SAD
threshold big enough so as to not ignore the block [ wild guess minimum SAD threshold for big spot sizes of (8x8x255)/4 = 4080 ].
Where a complete source frame is bad, then maybe should have very high (eg 10000) SAD threshold, and BlkSz may not really matter too much.
It is not the end of the world if some of the compensated blocks are ignored and swapped for the original current_frame block. Nor is it the end of the world if
no blocks were ignored because of high SAD threshold. The final result pixel is median pixel value of (2*RadT+1) motion compensated blocks, so allowing for some mistakes by choosing the
middle pixel value.
I've just tested real bad double frame, full frame luma and chroma corruption, with below line:
SpotLess(RadT=5,ThSAD=1000000,ThSAD2=1000000,pel=2,chroma=false,BlkSz=8,Olap=4,tm=false,glob=false,bBlur=0.0)
And although both SAD thresholds of 1 million, are totally impossible and so no blocks could possibly be ignored and yet we still got pretty good results, all frames were fixed
as we still had the temporal median filter to fall back on and pick the middle pixel value.

From mvtools2 docs:
ThSAD is SAD threshold for safe (dummy) compensation.
If block SAD is above the thSAD, the block is bad, and we use source block instead of the compensated block. Default is 10000 (practically disabled).

ThSAD2, Default ThSAD, 0 < ThSAD2 < 16320(8*8*255), Lower removes fewer spots, but less chance of blurring.
ThSAD2 sets the SAD [Sum of Absolute Differences] threshold for most distant frame from current_frame at distance RadT, with those frames that are distances in-between 1 and RadT
acquiring a SAD threshold linearly interpolated between the two.
From mvtools2 docs:
ThSAD2:
Defines the SAD soft threshold for the furthest frames at current_frame +- RadT.
The actual SAD threshold for each reference frame is a smooth interpolation between the original thSAD (close to the current frame)
and thSAD2. Setting thSAD2 lower than thSAD allows large temporal radii and good compensation for low SAD blocks while reducing the global error and the
risk of bluring when the result of MCompensate is passed to a temporal denoising filter.
EDIT: Although I have said that SAD threshold being too high could result in blurred frames, that is really taken from above "risk of bluring" line from mvtools docs,
however, that warning says "temporal denoising filter", which might suggest pixel averaging, whereas we are using pixel median. I'm not sure that blurring would be the result
of having too high a SAD threshold.

Pel, Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=half pixel, 4=quarter pixel)

Chroma, Default True. MAnalyse chroma arg. If true, use chroma in block matching when creating vectors. Maybe use False if source B&W or color noise.

BlkSz, Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better for HD clips. [Info: current Pinterf MvTools allows for BlkSize=12, and overlap=6]

OLap, Default half BlkSz, Block overlap.

Tm, TrueMotion Default True. Some folk swear truemotion=false is better.

Glob, Default True (True v1.06, was same as Tm, true almost always better), Allow set MAnalyse(global) independently of TrueMotion.
From MvTools2 docs for MAnalyse,
global
Estimate global motion (at every level) and use it as an additional predictor.
Only pan shift is estimated (no zoom and rotation).
Use false to disable, use true to enable.

bBlur, Default 0.6. If used, Suggest about 0.6, where MAnalyse create vectors is performed on denoised (blurred) super clip
for better motion analysis. Maybe try 1.0 for HD clip to counteract HD 'grittiness'. bBlur ignored if dc clip specified,
If providing bBlur, then ALWAYS specify as named argument ie bBlur=bBlur, we will likely insert any additional MvTools args
before bBlur to keep them together.

dc, Default UnDefined. If dc Specified, then must be same size and colorspace as source clip.
If detection clip specified then bBlur prefilter arg is ignored, and analysis is performed on
dc.MSuper() clip [ instead of c.Blur(bBlur).MSuper() ]. Allows to provide your own prefiltered clip.
If providing dc clip, then ALWAYS specify as named argument ie dc=DC, we will likely insert any additional MvTools args
before bBlur to keep them together.

*/

Function SpotLess(clip c,int "RadT",int "ThSAD",int "ThSAD2",int "pel",bool "chroma", int "BlkSz",Int "Olap",bool "tm",Bool "glob",
\ Float "bBlur", clip "dc" ) {
myName = "SpotLess: "
RadT = Default(RadT,1) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD,10000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2,ThSAD) # SAD threshold at radius RadT.
Pel = Default(pel,2) # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma = Default(chroma,True) # MAnalyse chroma arg. If set to true, use chroma in block matching.
BlkSz = Default(BlkSz,8) # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips. Maybe also better where BIG noise.
OLap = Default(OLap, BlkSz/2) # Default half of BlkSz.
Tm = Default(tm,True) # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
Glob = Default(glob,True) # Default True, Allow set MAnalyse(global) independently of TrueMotion.
bBlur = Default(bblur,0.6) # Default 0.6. Suggest about 0.6 for better motion analysis, but a bit slower.
HasDC = dc.Defined # bblur ignored if HasDC, ie user already provided prefiltered clip.
Assert(1 <= RadT,myName + " 1 <= RadT")
Assert(0.0 <= bblur <= 1.58, myName + "0.0 <= bblur <= 1.58")
Assert(pel==1 || pel==2 || pel==4, myName + "pel==1 || pel==2 || pel==4")
pad = max(BlkSz,8)
sup = (HasDC ? dc : bBlur>0.0 ? c.blur(bBlur) : c ).MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2)
# Only 1 Level required where not MAnalyse-ing.
sup_rend = (HasDC||bBlur>0.0) ? c.MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2,Levels=1) : sup
MultiVec = sup.MAnalyse(multi=true, delta=RadT,blksize=BlkSz,overlap=OLap,chroma=Chroma,truemotion=Tm,global=Glob)
c.MCompensate(sup_rend, MultiVec, tr=RadT, thSad=ThSAD, thSad2=ThSAD2)
MedianBlurTemporal(radiusY=0,radiusU=0,radiusV=0,temporalradius=RadT) # Temporal median blur only [not spatial]
SelectEvery(RadT*2+1,RadT) # Return middle frame
}


I'm quite happy with it as is, but anybody anything to say before I post as new non beta update.

Here some Local de-flicker whotsit, can be used for DC clip where local flicker.

Function DeFlickLocal(clip c) {
/*
Local DeFlicker. Didee, https://forum.doom9.org/showthread.php?p=1599682#post1599682
Calms local flickering, where some small object in frame flickers, rather than whole frame flickering.
[Didee:]
A small clip is created. The small clip is temporal-softened.
The difference achieved by temporalsoften is taken from the small clip, then re-scaled, and applied to the original clip.
Effectively, this is (very similar to) a spatial lowpass on the difference of a normal temporalsoften.
In particular with MVTools-Denoising etc., then the result makes a good source for motion vector search. No need for sluggish DCT=1.
*/

c
w=Width/8*2 h=Height/8*2 # Can be altered, but ~25% of original resolution seems reasonable
sm = c.bicubicresize(w,h, b=1.0/3, c=1.0/3) # b,c as defaults, b=blurring, c=ringing
smm = sm.temporalsoften (1,32,255,24,2).merge(sm,0.25)
smm = smm.temporalsoften(2,12,255,20,2)
c.mt_makediff(mt_makediff(sm,smm,U=3,V=3).bicubicresize(c.width,c.height,b=0,c=0),U=3,V=3) # blurring=0.0, ringing=0.0
}


Also, GMJCZP posted a script function Small_Deflicker():- https://forum.doom9.org/showthread.php?p=1812060#post1812060
which is based on DeFlickLocal(), which is an implementation of a Didee script as a function.
NOTE Didee comments.

[GMJCZP]

Hi TinMan, I wish you good luck with your script. The biggest challenge you are going to have is with anime, specifically with scenes with rain, because if the spot detection method is incorrect you will notice how the raindrops disappear before your eyes, KillerSpots was precisely tested exhaustively under these conditions

[StainlessS]

Hi GMJCZP, [I think I messaged you when I implemented Spotless, to let you know I extended your/didee script.]
I'm personally not bothered about anime, and said 'not for anime' in 1st or 2nd post, as uses RadT greater than 1, there is little
chance that any anime rain would survive RadT>=2.
If you aint tried it [SpotLess], give it a go, is almost magic with RadT>1, removes double spots, and even had it remove 4 consecutive BIG BIG BIG spots.
Glad to see you back, keep safe.

[Sharc]

SpotLess is perfect for my VHS sources. Thank you!

[StainlessS]

Fixed in post #16 [not the Spotless script]
Was

Prog = (UNSPOT_I) ? Prog.SpotLess(Radt=RADT_P,thsad2=ThSAD2_P,blksz=BLKSZ_P,tm=Tm_P,bBlur=bBlur_P) : Prog


Fixed

Prog = (UNSPOT_P) ? Prog.SpotLess(Radt=RADT_P,thsad2=ThSAD2_P,blksz=BLKSZ_P,tm=Tm_P,bBlur=bBlur_P) : Prog


Doing Babylon5 again [spent huge amount of time [hand edits] first time around).
But think I'll use TFM(pp=0) beforehand to get rid of Perverse Telecine [in some episodes], before using post #16 script.

Babylon5 [PAL version] Perverse Telecine mentioned here:- https://forum.doom9.org/showthread.php?p=1785472#post1785472

EDIT:
Episode 1 season 1 has some double frame spots [on progressive parts], so I thought I'de do it all again with spotless,
via the post #16 script, script is a bit slow [with doublerate output, which I'm doing this time].
I'm gettin about 60FPS with i7-8700 [so I guess maybe not that slow, but is only a bit faster than real time 50FPS output,
and 110 episodes, so is slow/tedious].

EDIT: "with doublerate output, which I'm doing this time"
OUTPUT=3 # 3=DblRate 50fps. Prog=Interpolate.
and TFM(pp=0) after AssumeTFF.

[coolgit]

Doing Babylon5 again [spent huge amount of time [hand edits] first time around).

EDIT:
Episode 1 season 1 has some double frame spots [on progressive parts], so I thought I'de do it all again with spotless,
via the post #16 script, script is a bit slow [with doublerate output, which I'm doing this time].
I'm gettin about 60FPS with i7-8700 [so I guess maybe not that slow, but is only a bit faster than real time 50FPS output,
and 110 episodes, so is slow/tedious].

EDIT: "with doublerate output, which I'm doing this time"
OUTPUT=3 # 3=DblRate 50fps. Prog=Interpolate.
and TFM(pp=0) after AssumeTFF.

Are you still doing B5 then? I have done s1 ep1 a while back, removed all the dust, scratch, etc with dirt cleaner which was about 95% good. The rest I'd spotted during playback. Upsampled it to 1920x1048, fixed some colouring, sharpening etc.
Hell a lot better than my dvd for sure.

Onto ep2 - Soul Hunter. Can you read this link for me

https://forum.videohelp.com/threads/402493-How-do-I-remove-these-vertical-lines

And possibly come up with a solution. Tried many software and can't clean it up. If you have the dvd then look at frame 13745 to 18634 (9mins 9 secs to 12:25) and you see those 3 static lines that appear in all scenes involving the non-regular casts.

I have minimised it using photoshop to create 3 vertical fixed mask and denoise to remove the bluish lines but there is a faint white line in parts left. Those scenes have slight band from WB bad attempted removal before dumping it onto the market. Those scenes looks too dark as if WB tried to reduce the visibility of the lines. Did some colour correction and looks good, just those pesky whitish lines.

Let me know if you have any solutions. Cheers.

[StainlessS]

OK, I found the lines, (although I did not originally even notice them in your sample file - I thought it must have been cleaned result).
Just tried my used script, and still there afterwards.

I personally have reached the conclusion that life is too short to be spending eons on such things,
do yourself a favour and live with them (look the other way when they show up on screen).

Wish I could give you a better answer, but I dont got one :(

EDIT:
I think this is about same as template script I used [I stopped mid Season 3, and am busy doing other stuff, continuing later].

###############
# PAL Hybrid Interlaced/Progressive mix, output to singlerate or doublerate.
###############

### DGIndex Template, PAL_Hybrid_DeintDoubleRate.avs
VideoFileName ="__vid__"
AudioFileName ="__aud__"
AudioDelay =Value("__del__")

MPEG2Source(VideoFileName)
AudioExt=RT_GetFileExtension(AudioFileName)

Audio= (AudioExt==".ac3") ? NICAC3Source(AudioFileName,channels=2,DRC=0)
\ : (AudioExt==".mpa"||AudioExt=="mp1"||AudioExt==".mp2"||AudioExt==".mp3") ? NicMPG123Source(AudioFileName,Normalize=False)
\ : (AudioExt==".wav") ? RaWavSource(AudioFileName)
\ : (AudioExt==".dts") ? NicDTSSource(AudioFileName)
\ : 0

Assert(!isInt(Audio),"NO AUDIO")

(!isInt(Audio)) ? AudioDub(Audio).DelayAudio(AudioDelay).Trim(0,0) : NOP # Trim, chop/add audio to length

(!isInt(Audio) && AudioRate() <> 44100) ? ResampleAudio(44100) : NOP
####### CONFIG ##########

AssumeTFF # Whatever (check) : Almost always TFF for PAL DVD. Default field order is BFF.

# Get rid Of Perverse Telecine
TFM(pp=0)
O=Last
AutoAdjust(Auto_Gain=true).Merge(o,0.6)


OUTPUT = 3 # 0=SingleRate 25FPS Progressive. Progressive src frames as input, interlaced src frames are even frames from dbl rate deinterlacing.
# 1=SingleRate 25FPS Hybrid. Progressive src frames as input, interlaced src frames dbl rate deinterlaced, reinterlaced to 25FPS after processing.
# 2=DblRate 50fps. Prog=ChangeFPS. Progressive src frames doubled with ChangeFPS, interlaced src frames are dbl rate deinterlaced.
# 3=DblRate 50fps. Prog=Interpolate. Progressive src frames double rate interpolated with JohnFPS [ie MvTools], interlaced src frames are dbl rate deinterlaced.

UNSPOT_P=true # Use SplotLess() on Progressive parts to remove spots and noise.
UNSPOT_I=False # Use SplotLess() on Interlaced parts to remove spots and noise.

RADT_P = 2 # SpotLess temporal radius (spots on 2 consecutive frames)
ThSAD_P = 5000 # Almost OFF.
ThSAD2_P = 5000 # Almost OFF.
BlkSz_P = 16
Tm_P = false
Glob_P = true
bBlur_P = 0.6

FRAMES_P=1 # Progressive McDegrainSharp(frames=FRAMES_P), (0=Dont McDegrainSharp)
CSHARP_P=0.5

RADT_I = 2 # SpotLess temporal radius (spots on 2 consecutive frames)
ThSAD_I = 5000 # Almost OFF.
ThSAD2_I = 5000 # Almost OFF.
BlkSz_I = 16
Tm_I = false
Glob_I = true
bBlur_I = 0.6

FRAMES_I=1 # Interlaced McDegrainSharp(frames=FRAMES_I), (Maybe double FRAMES_P for detail migration, although max possible for McDegrainSharp=3). (0=Dont McDegrainSharp)
CSHARP_I=CSHARP_P

# IsCombedTIVTC args, Defaults, 1st after comment hash.
CTHRESH = 12 # 10), Area combing threshold.
# Valid settings are from -1 (every pixel will be detected as combed) to 255 (no pixel will be detected as combed).
# This is basically a pixel difference value. A good range is between 8 to 12.
MI = 80 # 80), Number of combed pixels inside any of the blocky by blockx size blocks on the frame for the frame to be detected as combed.
CHROMA = True # True), Whether or not chroma is considered in the combed frame decision. [We want chroma as interlaced parts in color]
BLOCKX = 16 # 16), X-axis size of the window used during combed frame detection.
BLOCKY = 16 # 16), Y-axis size of the window used during combed frame detection.
# So in any single block, if 85[MI] out of 256 pixels pass CThresh as combed, then is Interlaced.
METRIC = 1 # 0) Detecto method: 0=Old, 1=New

# Deinterlace mode
BOBMOD=1 # 0=BOB, 1=YadifMod2, 2=QTGMC : QTGMC here is way memory/CPU intensive, req x64. Suggest YadifMod2

BOB_FUNC_0 = """Bob()""" # Cheap & cheerful Doublerate simple Bob.
BOB_FUNC_1 = """YadifMod2(Order=-1,Mode=1,edeint=NNedi3(Field=-2, nns=2))""" # Doublerate Yadifmod2 filter.
# # Field-2=double rate, internal parity value to start : Order-1=Internal Parity. Mode 1=double rate do spactial check (3=no spatial check).

BOB_FUNC_2 = """QTGMC(Preset="fast",NoisePreset="Fast",SourceMatch=2,Lossless=2,EZKeepGrain=0.4,Sharpness=0.1, tr2=0)""" # Doublerate QTGMC filter.

FUNC = Select(BOBMOD, BOB_FUNC_0, BOB_FUNC_1, BOB_FUNC_2)
FNAME = Select(BOBMOD,"BOB", "YADIF", "QTGMC")


PRE_FETCH=6 # 1 = No Prefetch, Else eg 4 = Prefetch(4) ie MT mode

### # Display stuff
SUBS = True # Show where Deinterlaced/ReInterlaced
STACK = True # Show Src and Result
TITLE = True # Show Title Bar
TITNUM = True # Show Frame Number on Title Bar (Only if TITLE)
FINAL = True # True, switch off SUBS, STACK, and Title

##### End Config ##########

Src=Last

SUBS = (!FINAL&&SUBS)
TITLE = (!FINAL&&TITLE)
STACK = (!FINAL&&STACK)


### IsCombedTIVTC with args, as a string eg "IsCombedTIVTC(cthresh=10,mi=85,chroma=true,blockx=16,blocky=16,metric=1)"
COMBDET = RT_String("IsCombedTIVTC(cthresh=%d,mi=%d,chroma=%s,blockx=%d,blocky=%d,metric=%d)",CTHRESH,MI,CHROMA,BLOCKX,BLOCKY,METRIC)
SUBTXT = (OUTPUT==0) ? FNAME+"_DEINTERLACED-EVEN" : (OUTPUT==1) ? FNAME+"_RE-INTERLACED" : FNAME+"_DblRate-DEINTERLACED"

# Interlaced parts
# DeSpot, Deinterlace to doublerate, McDeGrainSharp to migrate detail between original fields[now frames].
Deint = Src.Eval(FUNC) # Double Rate deinterlace
Deint = (OUTPUT==0) ? Deint.SelectEven : Deint # If (OUTPUT==0) McDegrainSharp & SpotLess done after Deinterlace.SelectEven, Else after doublerate deinterlace.
Deint = (UNSPOT_I) ? Deint.SpotLess(Radt=RADT_I,thsad=ThSAD_I,thsad2=ThSAD2_I,blksz=BLKSZ_I,tm=Tm_I,glob=Glob_I,bBlur=bBlur_I) : Deint
Deint = (FRAMES_I>0) ? Deint.McDeGrainSharp(frames=FRAMES_I,csharp=CSHARP_I) : Deint
Deint = (OUTPUT==1) ? Deint.SeparateFields.SelectEvery(4,0,3).Weave : Deint # If singlerate Re-Interlacing
Deint = (SUBS) ? Deint.Subtitle(SUBTXT,Align=5) : Deint

# Progressive parts, Despot, Degrain sharp done still @ singlerate
Prog = Src
Prog = (UNSPOT_P) ? Prog.SpotLess(Radt=RADT_P,thsad=ThSAD_P,thsad2=ThSAD2_P,blksz=BLKSZ_P,tm=Tm_P,glob=Glob_P,bBlur=bBlur_P) : Prog
Prog = (FRAMES_P>0) ? Prog.McDeGrainSharp(frames=FRAMES_P,csharp=CSHARP_P) : Prog
Prog = (OUTPUT==2) ? Prog.ChangeFPS(Src.FrameRateNumerator*2,Src.FrameRateDenominator) : (OUTPUT==3) ? Prog.JohnFPS() : Prog # Progressive to O/P rate

SrcTest = (OUTPUT<=1) ? Src : Src.ChangeFPS(Src.FrameRateNumerator*2,Src.FrameRateDenominator)
# Select Progressive / Deinterlaced frame, based on IsCombedTIVTC detection on original source clip frame.
Result=ConditionalFilter(SrcTest, Deint, Prog, COMBDET,"=","true")

SrcOut = (!TITLE)?SrcTest:(OUTPUT<=1) ? Src.TSub("Source",TITNUM) : Src.TSub("Source DblRate via ChangeFPS, ie dupes",TITNUM).ChangeFPS(Src.FrameRateNumerator*2,Src.FrameRateDenominator)

Result = (!TITLE)?Result :(OUTPUT<=1) ? Result.TSub(String(OUTPUT,"%.0f) Result SingleRate"),TITNUM) : (OUTPUT==2) ? Result.TSub("2) Result DblRate (Progressive frames via ChangeFPS)",TITNUM)
\ : Result.TSub("3) Result DblRate (Progressive frames via JohnFPS, ie MvTools2)",TITNUM)

(STACK) ? Stackhorizontal(SrcOut, Result) : Result

(PRE_FETCH>1) ? Prefetch(PRE_FETCH) : NOP # I had problems using x64 and Prefetch [Access Violation], culprit was old IsCombedTIVTC, fixed via Pinterf version.
RT_SignalDar2(16,9)
Return Last

#####################
# Stack Overhead Subtitle Text, with optional FrameNumber shown.
Function TSub(clip c,string Tit,Bool "ShowFrameNo",Int "Col"){
c.BlankClip(height=20,Color=Default(Col,0))
(Default(ShowFrameNo,False))?ScriptClip("""Subtitle(String(current_frame,"%.f] """+Tit+""""))"""):Trim(0,-1).Subtitle(Tit)
Return StackVertical(c).AudioDubEx(c)
}


EDIT: I am just doing the lot [110 eps] with exact same script [no messing about].
EDIT: Resulting above scripts were fed direct into MeGUI x64. [all x64 plugs are available]
EDIT: I found that there are RadT=2 spots in seasons 1, 2 & 3, others unknown
(I scan through ep1 of a season, if single double spot found, then I do the lot)
I dont even bother with spots on interlaced sections, they're naff/indistinct anyways and dont need more over processing.

EDIT:
Wish I could give you a better answer, but [B]I dont got one :(
My "English" has been greatly enhanced by this here clip [Andrew Dice Clay, "Even bums don't not got a car", in "One night at McCools"]:-
[!!!WARNING!!!, Colourful language]:- https://www.youtube.com/watch?v=uthKzsSY2AY

[johnmeyer]

I pointed out in your thread over in the Videohelp format, that eliminating vertical lines is a tough challenge. However, unlike film scratches which are virtually impossible to remove, you can remove your scratches. However, the problem will be the inevitable blur that will result from blending pixels from the adjacent area. I suggested using the VirtualDub Delogo plugin, but I'm sure there are better choices.

Spotless is almost certainly the wrong tool because it is looking for differences between frames: when a spot shows up for just one frame, it is removed. Since your lines do not change from frame-to-frame, there are no difference vectors, and Spotless likely won't do anything.

Some of the experts on using AVISynth masking may have some more clever ideas. You might start your own thread here and see what you get.

[StainlessS]

I think he was just asking me because I said I was doin the same series, just in case I did something clever with those near invisible lines.
110 eps is one helluva lot of viewing never mind examining and special casing for those lines.
If you're getting big bucks for a total remaster of original source, then is maybe sane, otherwise not.

[StainlessS]

John or coolgit or anybody really,



Avisource("SH_2_secs.avi.avi") # Did not have codec for FOURCC, redid in ffmpeg to UT_Video so AVI.AVI
ConvertToYV24
O=Last

# Intention: Single pixel width, vertical line/scratch reduction.
GeneralConvolution(0, "
1 0 1
2 4 2
1 0 1
")

StackVertical(O,Last)


I thought above might work-ish, just made up the matrix string on gut feelin' no idea if is any kind of standard matrix
but it does seem to make some difference.
[I cant really see it properley lousy eyes and I'm on a smaller monitor, main machine busy doing something else, and not
got Avs stuff installed on it (actually messin with linux install on it)].

Can you test out above and see if it is of any worth at all [and if it blurs much].

Only my guess work and I got almost zero experience of same, so might be rubbish.

EDIT: Could be applied to frames/ranges using clipclop. Gotta be faster/less-user-interaction than delogo or descratch or whatever.

EDIT: In VDub2 at 200%, I'de say reduces scratch to about 50% visiblility, not perfect but does not seem to create additional artefacts
that I can see.

[coolgit]

Stainless

Is your template above for MeGui only?

The GeneralConvolution didn't do anything. I don't understand it either. Vdub2 has it and I've never used it. All it did what change the output to black with very faint movements.

What i am looking for, dunno if it can work, it a simple replace bad pixel type plugin. The way i see it would work like this.

Instruct start hor pixel(say 300) and end hor pixel (305) and full vert length - like masking and only work in this area and nowhere else.
Pixel 300 to 305 are considered bad.
Lets focus on 5 main colours. BWRGb. | = start and end mask boundary.


Small example

Good bad good
pixels pixels pixels
BBBB|WbbbWW|BBBB
BBBB|WWbbbW|BBBB
GGGG|WWbbbW|RRRR
GGGG|WbbbWW|RRRR

would become by using previous good hor pixel(299) and hor pixel(306) on both side alternately to overwrite the bad pixels

BBBB|BBBBBB|BBBB
BBBB|BBBBBB|BBBB
GGGG|GGGRRR|RRRR
GGGG|GGGRRR|RRRR

The above looks simple enough to me. No blurring, no band lines, no random guessing based on fancy algorithms, etc.

How hard can it be?

[coolgit]

I pointed out in your thread over in the Videohelp format, that eliminating vertical lines is a tough challenge.

I know but i have been making headway using photoshop and topaz denoise.

However, unlike film scratches which are virtually impossible to remove, you can remove your scratches. However, the problem will be the inevitable blur that will result from blending pixels from the adjacent area. I suggested using the VirtualDub Delogo plugin, but I'm sure there are better choices.

Topaz denoise removed source file faint bands and also removed blur which was bloody useful. It actually removed the grain inside the whites of the eyes making the eyes look visibly better. Faint white lines about 3/4 pixel width remains. I had an idea how to remove bad pixels and replace them with good pixels in my last post above. Is it feasible with avisynth?

Some of the experts on using AVISynth masking may have some more clever ideas. You might start your own thread here and see what you get.

This what i need, masking. I want to be able to focus on the 3 lines and nothing else. Hence why i am using photoshop.

[StainlessS]

I think I cocked up the initial post, and removed a ",8" after the matrix string, maybe you took a copy before I fixed it.

Anyways, here a comparison, top is result - Original [should be mid grey-ish],
bottom is same but differences amplified GREATLY, maximum amp'ed.
You can see that the light lines have been modified, and are darker, meaning darker than original, as intended.
It does produce global frame changes, as does most denoise type processing.


Avisource("SH_2_secs.avi.avi") # Did not have codec for FOURCC, redid in ffmpeg to UT_Video so AVI.AVI
ConvertToYV24
O=Last
GeneralConvolution(0, "
1 0 1
2 4 2
1 0 1
")
D1=ClipDelta(Last,O,false)
D2=ClipDelta(Last,O,true)
StackVertical(D1,D2) # Top Subtract, bottom amplified
Return last

# Return Clip Difference of input clips (amp==true = Amplified, show==true = show background)
Function ClipDelta(clip clip1,clip clip2,bool "amp",bool "show") {
amp=Default(amp,false)
show=Default(show,false)
c2=clip1.levels(128-32,1.0,128+32,128-32,128+32).greyscale()
c1=clip1.subtract(clip2)
c1=(amp)?c1.levels(127,1.0,129,0,255):c1
return (show)?c1.Merge(c2):c1
}


Click Me
https://i.postimg.cc/zygf74S6/a-00.jpg (https://postimg.cc/zygf74S6)

Pretty much all avisynth scripts, can of course by loaded into Megui main screen top input Avisynth Script editbox.
[or just view in eg VDub2]

[coolgit]

StainMoreH:D:D

Will try your new script later.

Just played with photoshop (36 action commands) a bit more and boooooom, i manage to remove the white line in 98% of all the frames. Just a small 1 pixel thin, about 10 pixel high, line in some obscure location. At normal zoom i can not see anything. I would have to zoom to 200% or more before noticing faint band, small bits of line and i know where to look. A random person wouldn't. Will convert the frames to mp4 and see how that pans out.

[coolgit]

Click Me
https://i.postimg.cc/zygf74S6/a-00.jpg (https://postimg.cc/zygf74S6)



That is what i got using vdub2. I copied your script but changed Avisource to FFVideoSource.

What am i suppose to do with the grey clip?

[StainlessS]

That is just the difference between src and dest. Use the original (fixed) script.

EDIT: Fixed original

Avisource("SH_2_secs.avi.avi") # Did not have codec for FOURCC, redid in ffmpeg to UT_Video so AVI.AVI
ConvertToYV24
O=Last

# Intention: Single pixel width, vertical line/scratch reduction.
GeneralConvolution(0, "
1 0 1
2 4 2
1 0 1
")

StackVertical(O,Last)

[StainlessS]

Coolgit,
maybe of use [I've used my GeneralConvolution thingy as fixed clip, but could use some other de-streak'ed clip].


/*
Requires Avs+, FrameSel[v2.21 just posted new version], ClipClop, RT_Stats
Frames file specifies frames/ranges to fix.
*/

Avisource("SH_2_secs.avi.avi")
STACK = false # True, result is STACKED, else Not
SHOW = True # True, Show ClipClop metrics, else Not
Frames = ".\Frames.txt" # Frames/Ranges to fix, file, either single frame number or range eg 100,110 or 100 110.
###
Frames = Frames.RT_GetFullPathName
###
ConvertToYV24
O=Last

# We're gonna use GeneralConvolution de-streaker whotsit, is applied to ENTIRE clip.
De_Streak = GeneralConvolution(0, "
1 0 1
2 4 2
1 0 1
")

FixFrames(O,De_Streak,Frames,Show=SHOW) # Replace frames/ranges in frames file with De_Streaked frames

###
STK = StackHorizontal(O,Last) # Lft=Original, Rgt=Fixed
#STK = StackVertical(O,Last) # Top=Original, Bot=Fixed ## If Preferred

(STACK) ? STK : Last
ConvertToYV12
Return last

Function FixFrames(clip c,clip repc,string Frames,Bool "Show") {
Show = Default(Show,False)
Assert(Frames!="","FixFrames: Frames cannot be ''")
Frames = Frames.RT_GetFullPathName
Assert(Frames.Exist,"FixFrames: Frames file Does Not Exist")
# Temp filename for Frames file re-written as ClipClop Command file [with clip index of 1 for replacement clip].
ClipClopCmd=(".\~ClipClipCmd_" + RT_LocalTimeString(file=True) + ".Txt").RT_GetFullPathName
# Convert To ClipClop Cmd file, REQUIRES FrameSel v2.21 for PruneIx arg and ClipClop clip_index=1 conversion.
FrameSel_CmdReWrite(c,ClipClopCmd,cmd=Frames,Ordered=true,range=true,Prune=True,PruneIx=1)
ClipClop(c,repc,Cmd=ClipClopCmd,show=SHOW)
RT_FileDelete(ClipClopCmd) # Done with it.
return Last
}

EDITED: OOps, added in blue. RT_FileDelete(ClipClopCmd)

EDIT:
Contents of Frames file [Frames.txt] to fix frame 5, and range 10 to 20, and range 30 to 40

5
10,20
30,40

EDIT: 5; 10,20; and 30,40 picked out of the hat, use your own numbers.

[coolgit]

Stainless or anybody

Need help getting the output right. Currently both left and right viewers show the spotless output.
I want the left to stick with the original and the right to show the output so i can compare.

LoadPlugin("C:\Program Files (x86)\AviSynth\plugins+\mvtools2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\plugins+\MedianBlur2.dll")
source=FFVideoSource("E:\currser\BABYLON5\new\Soul Hunter.avi").convertTOYV12(interlaced=false).killaudio()

Function SpotLess(clip c,int "RadT",int "ThSAD",int "ThSAD2",int "pel",bool "chroma", int "BlkSz",Int "Olap",bool "tm",Bool "glob",
\ Float "bBlur", clip "dc" ) {
myName = "SpotLess: "
RadT = Default(RadT,1) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD,10000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2,ThSAD) # SAD threshold at radius RadT.
Pel = Default(pel,2) # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma = Default(chroma,True) # MAnalyse chroma arg. If set to true, use chroma in block matching.
BlkSz = Default(BlkSz,8) # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips. Maybe also better where BIG noise.
OLap = Default(OLap, BlkSz/2) # Default half of BlkSz.
Tm = Default(tm,True) # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
Glob = Default(glob,True) # Default True, Allow set MAnalyse(global) independently of TrueMotion.
bBlur = Default(bblur,0.0) # Default 0.6. Suggest about 0.6 for better motion analysis, but a bit slower.
HasDC = dc.Defined # bblur ignored if HasDC, ie user already provided prefiltered clip.
Assert(1 <= RadT,myName + " 1 <= RadT")
Assert(0.0 <= bblur <= 1.58, myName + "0.0 <= bblur <= 1.58")
Assert(pel==1 || pel==2 || pel==4, myName + "pel==1 || pel==2 || pel==4")
pad = max(BlkSz,8)
sup = (HasDC ? dc : bBlur>0.0 ? c.blur(bBlur) : c ).MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2)
# Only 1 Level required where not MAnalyse-ing.
sup_rend = (HasDC||bBlur>0.0) ? c.MSuper(hpad=pad,vpad=pad,pel=pel, sharp=2,Levels=1) : sup
MultiVec = sup.MAnalyse(multi=true, delta=RadT,blksize=BlkSz,overlap=OLap,chroma=Chroma,truemotion=Tm,global=Glob)
c.MCompensate(sup_rend, MultiVec, tr=RadT, thSad=ThSAD, thSad2=ThSAD2)
MedianBlurTemporal(radiusY=0,radiusU=0,radiusV=0,temporalradius=RadT) # Temporal median blur only [not spatial]
SelectEvery(RadT*2+1,RadT) # Return middle frame
}

output=SpotLess(source)

return(output)

Tired interleave but got original left, and right both original and new output, very slow way of checking.

So far impressed with this one.
cheers

[StainlessS]

this what you're after


# ...

output=SpotLess(source)
StackHorizontal(source,output) # stack side by side, source left, output right
#StackVertical(source,output) # stack vertical, source top, output bot
return last

EDIT: Untested

EDIT:
Currently both left and right viewers show the spotless output.
At a guess, maybe you are using Vdub2, and seeing VD2 input and output panes.
Those panes are VD2 input and output, which are probably both the same, [assuming no editing in vd2].
What you want is input and output in avisynth, as stacked above. [I mostly have VD2 output pane @ tiny size].

EDIT: Also, maybe try with

output=SpotLess(source,tm=false,bblur=0.6)

Spotless will likely not be of much use for your streaky lines thingies, unless there is horizontal motion 'though' them.

[coolgit]

this what you're after


# ...

output=SpotLess(source)
StackHorizontal(source,output) # stack side by side, source left, output right
#StackVertical(source,output) # stack vertical, source top, output bot
return last


That worked, not perfect but better than nothing.



At a guess, maybe you are using Vdub2, and seeing VD2 input and output panes.
Those panes are VD2 input and output, which are probably both the same, [assuming no editing in vd2].
What you want is input and output in avisynth, as stacked above. [I mostly have VD2 output pane @ tiny size].

So there is no way one can override right side of vdub using avisynth?

EDIT: Also, maybe try with

output=SpotLess(source,tm=false,bblur=0.6)

Spotless will likely not be of much use for your streaky lines thingies, unless there is horizontal motion 'though' them.

The above threw an error column 29 missing , or ] edit: scrap that, i had [/COLOR] in my script.

Got it to work however output=SpotLess(source) is better. A defect 100% cleared up and the above only 70% cleared up. Culprit tm=false... when true is did better, say 95%.

I was just testing your script. It worked a lot of better when there are motion (fast and/or close to camera) as dirt cleaner took chunks of pixel and left artefacts behind.

However, 1st and last frame before scene changes are not processed. Does your script needs to look behind and ahead 1 frame for analysis to detect spot?

How i denoise a bit more?

[StainlessS]

That worked, not perfect but better than nothing.
Well if its still your streaks you're after fixing, then Spotless is probably not a good solution,
Spotless is intended to fix temporal anomalies, mainly Spots [hence the name].
A spot exists [usually] on a single frame, and it is that that enables detection, your stripey streaky things are
spatial, in the same spacial location, temporally kinda constant, so the spotless doofah dont work too well.

Did you try the script I recently gave, using ClipClop, can just use frames file containing "0,0" meaning all frames if you want to see all frames.
It will not be perfect but gottat be bettr than Spotless. [calms your streaks to maybe 50% as visible].

So there is no way one can override right side of vdub using avisynth?
Vdub has no idea at all what the input to avisynth was, only the output from it.

Does your script needs to look behind and ahead 1 frame for analysis to detect spot?
Yes, its a temporal filter, and only detects spots where the non spot pixels outnumber the spot pixels, with a temporal span of 2*RadT+1 frames. [RadT=temporal radius]

Give me a little while, I want to try something, only I'm a bit busy so might take some time.

[coolgit]

Coolgit,
maybe of use [I've used my GeneralConvolution thingy as fixed clip, but could use some other de-streak'ed clip].


/*
Requires Avs+, FrameSel[v2.21 just posted new version], ClipClop, RT_Stats
Frames file specifies frames/ranges to fix.
*/

Avisource("SH_2_secs.avi.avi")
STACK = false # True, result is STACKED, else Not
SHOW = True # True, Show ClipClop metrics, else Not
Frames = ".\Frames.txt" # Frames/Ranges to fix, file, either single frame number or range eg 100,110 or 100 110.
###
Frames = Frames.RT_GetFullPathName
###
ConvertToYV24
O=Last

# We're gonna use GeneralConvolution de-streaker whotsit, is applied to ENTIRE clip.
De_Streak = GeneralConvolution(0, "
1 0 1
2 4 2
1 0 1
")

FixFrames(O,De_Streak,Frames,Show=SHOW) # Replace frames/ranges in frames file with De_Streaked frames

###
STK = StackHorizontal(O,Last) # Lft=Original, Rgt=Fixed
#STK = StackVertical(O,Last) # Top=Original, Bot=Fixed ## If Preferred

(STACK) ? STK : Last
ConvertToYV12
Return last

Function FixFrames(clip c,clip repc,string Frames,Bool "Show") {
Show = Default(Show,False)
Assert(Frames!="","FixFrames: Frames cannot be ''")
Frames = Frames.RT_GetFullPathName
Assert(Frames.Exist,"FixFrames: Frames file Does Not Exist")
# Temp filename for Frames file re-written as ClipClop Command file [with clip index of 1 for replacement clip].
ClipClopCmd=(".\~ClipClipCmd_" + RT_LocalTimeString(file=True) + ".Txt").RT_GetFullPathName
# Convert To ClipClop Cmd file, REQUIRES FrameSel v2.21 for PruneIx arg and ClipClop clip_index=1 conversion.
FrameSel_CmdReWrite(c,ClipClopCmd,cmd=Frames,Ordered=true,range=true,Prune=True,PruneIx=1)
ClipClop(c,repc,Cmd=ClipClopCmd,show=SHOW)
RT_FileDelete(ClipClopCmd) # Done with it.
return Last
}

EDITED: OOps, added in blue. RT_FileDelete(ClipClopCmd)

EDIT:
Contents of Frames file [Frames.txt] to fix frame 5, and range 10 to 20, and range 30 to 40

5
10,20
30,40

EDIT: 5; 10,20; and 30,40 picked out of the hat, use your own numbers.

No function named RT_GetFullPathName error

[kedautinh12]

Requires Avs+, FrameSel[v2.21 just posted new version], ClipClop, RT_Stats

[coolgit]

Requires Avs+, FrameSel[v2.21 just posted new version], ClipClop, RT_Stats

Got all that.

[StainlessS]

Got all that.
RT_GetFullPathName is from RT_Stats,
Maybe you got x64 plug in x86 plugins, or something.

[coolgit]

dll was missing for some strange reason.

Got it working to and i can't see any changes at all.

[coolgit]

Stainless

Spotless worked great. Tried many ways and watched them on my TV. Small improvements here and there. Lines can't been seen on the TV, monitor yes but that's because it's in my face. On the TV the noticeable part is when the camera is still and there is a close up scene. The 3 lines shows up when there is dark background on the bottom 1/3.

So i decided to have cuppa break. Then a brainwave. Photoshop clone. Previously didn't work because one can not clone precisely frame by frame thus it created more flickering. My brainwave is do the cloning quickly for the noticeable parts, approx 75 frames, then apply spotless. The cloning differences between frames was removed and flickering non-existent. Now it looked great. Miles better than the original.

Pleased with my hard work I proceed to save it. Didn't work. I got a vid file with both org and new from the output in vdub.

How do i save the result of the work after spotless?

[StainlessS]

STACK = false # True, result is STACKED, else Not
SHOW = True # True, Show ClipClop metrics, else Not

Make STACK=False, that was intended for user configuration.
SHOW was for if you were using ClipClop. [where you would switch that off too].


Good Job :)

[coolgit]

STACK = false # True, result is STACKED, else Not
SHOW = True # True, Show ClipClop metrics, else Not

Make STACK=False, that was intended for user configuration.
SHOW was for if you were using ClipClop. [where you would switch that off too].


Good Job :)

The above isn't the spotless script?

[StainlessS]

OK, then just comment out StackHorizontal/StackVertical lines [precede line with '#'],
or delete them.

You really ortta be gettin to learn a bit of avisynth script yourself, people are not gonna be wantin' to spoon feed you forever. :)

[coolgit]

It is your script. It is not my fault if avisynth is complicated with crap 'manual'.

Showing a new output and saving it, is the most basic task that all software does in 5 seconds. 24 hours later and i still can't do the basic. I can only learn as i go along bit by bit.

This is the latest and obviously "the script's return value was not a video clip" pops up.

LoadPlugin("C:\Program Files (x86)\AviSynth\plugins+\mvtools2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\plugins+\MedianBlur2.dll")
source=FFVideoSource("E:\currser\BABYLON5\new\Soul Hunter.avi").convertTOYV12(interlaced=false).killaudio()



Function SpotLess(clip c,int "RadT",int "ThSAD",int "ThSAD2",int "pel",bool "chroma", int "BlkSz",Int "Olap",bool "tm",Bool "glob",
\ Float "bBlur", clip "dc" ) {
myName = "SpotLess: "
RadT = Default(RadT,1) # Temporal radius. (MCompensate arg)
ThSAD = Default(ThSAD,10000) # SAD threshold at radius 1 (Default Nearly OFF).
ThSAD2 = Default(ThSAD2,ThSAD) # SAD threshold at radius RadT.
Pel = Default(pel,2) # Default 2. 1, 2, or 4. Maybe set 1 for HD+. (1=precision to pixel, 2=precision to half pixel, 4=quarter pixel)
Chroma = Default(chroma,True) # MAnalyse chroma arg. If set to true, use chroma in block matching.
BlkSz = Default(BlkSz,8) # Default 8. MAnalyse BlkSize. Bigger blksz quicker and perhaps better, esp for HD clips. Maybe also better where BIG noise.
OLap = Default(OLap, BlkSz/2) # Default half of BlkSz.
Tm = Default(tm,True) # TrueMotion, Some folk swear MAnalyse(truemotion=false) is better.
Glob = Default(glob,True) # Default True, Allow set MAnalyse(global) independently of TrueMotion.
bBlur = Default(bblur,0.3) # Default 0.6. Suggest about 0.6 for better motion analysis, but a bit slower.
HasDC = dc.Defined # bblur ignored if HasDC, ie user already provided prefiltered clip.
Assert(1 <= RadT,myName + " 1 <= RadT")
Assert(0.0 <= bblur <= 1.58, myName + "0.0 <= bblur <= 1.58")
Assert(pel==1 || pel==2 || pel==4, myName + "pel==1 || pel==2 || pel==4")
pad = max(BlkSz,8)
sup = (HasDC ? dc : bBlur>0.0 ? c.blur(bBlur) : c ).MSuper(hpad=pad,vpad=pad,pel=pel,sharp=2)
# Only 1 Level required where not MAnalyse-ing.
sup_rend = (HasDC||bBlur>0.0) ? c.MSuper(hpad=pad,vpad=pad,pel=pel,sharp=2,Levels=1) : sup
MultiVec = sup.MAnalyse(multi=true, delta=RadT,blksize=BlkSz,overlap=OLap,chroma=Chroma,truemotion=Tm,global=Glob)
c.MCompensate(sup_rend, MultiVec, tr=RadT, thSad=ThSAD, thSad2=ThSAD2)
MedianBlurTemporal(radiusY=0,radiusU=0,radiusV=0,temporalradius=RadT) # Temporal median blur only [not spatial]
SelectEvery(RadT*2+1,RadT) # Return middle frame
}


#output=SpotLess(source,tm=true,bblur=0.6)
#output=SpotLess(source)
#StackHorizontal(source,output) # stack side by side, source left, output right
#StackVertical(source,output) # stack vertical, source top, output bot
#return source
return last
#return spotless(source)

[VoodooFX]

This is the latest and obviously "the script's return value was not a video clip" pops up.

You are trying to return "last" when not doing anything in your script.

Maybe this you want:
output=SpotLess(source)
return output