"Feedback" - a filter that lets you use a filter chain's output as its own input

[wonkey_monkey]

Feedback v0.1 (http://horman.net/avisynth/)
Direct link (http://horman.net/avisynth/download/feedback.zip)

https://i.imgur.com/esAjaDs.png

Video ( https://youtu.be/4YngPQZpkaw)


I think this one's a contender for my most esoteric filter yet.

Feedback
========

Feedback is a pair of filters that allow the output of a filter chain to be
passed back as an input, further up the chain, to be used in outputting the next
frame of the chain.

For example, it can be used to fill in the empty areas of a stablised clip by
compositing in the previous output frame (which itself will have been composited
using Feedback).

Usage
=====

FeedbackInput(
(clip),
(int) id
)

FeedbackOutput(
(clip),
(int) id
)

FeedbackInput will return frame 0 of its input clip as frame 0 of its output,
but after that it returns frame n-1 of the associated FeedbackOutput clip.

FeedbackOutput sets a clip as a source for a FeedbackInput clip.

The id parameter is optional, but must be given unique values whenever there is
more than one FeedbackInput/FeedbackOutput pair in a script environment.

Example
=======

Assume that you have a clip, with an alpha channel, that has been stabilised and
so has a moving black border around the stabilised image. Then you can composite
the stabilised clip on top of its previous frames by doing the following:

Layer(stab.FeedbackInput(), stab).FeedbackOutput()

Frame 0 of the resulting clip will just be frame 0 of "stab" composited on itself.

Frame 1 will be frame 1 of "stab" composited on the previously composited frame (0)

Frame n will be frame n of "stab" composited on the previously composited frame (n-1)

Caveats
=======

Linear access is fast - FeedbackOutput keeps a copy of the previously-output
frame - but non-linear access will be slow as it will need to traverse all the
way back to the beginning of the FeedbackOutput clip.

Using Trim, or any other filter which changes the relative frame numbers of a
clip (e.g. ChangeFps or SelectEven) is probably not a good idea and could lead
to hangs or other strange behaviour.

[kedautinh12]

Thanks

[Dogway]

Nice, I might add it to Stab() as a new filling border option if it isn't too slow.

[wonkey_monkey]

It shouldn't cause any additional slowdown as long as frames are requested in order.

[chmars]

Very interesting, I am impressed.

[real.finder]

thanks, and if the previous frame has nothing to composite with?

[johnmeyer]

Very interesting. When used with stabilization, as in your example, do you think it would produce better edge correction than what is contained in Depanstabilize, Deshaker, Mercalli, etc.?

Now that I've asked that, having spent 20+ years looking at the artifacts produced by Deshaker's edge correction, I wonder if Guner's (Deshaker author) code is doing somewhat the same thing. In fact, as I visualize what would happen with your function I'll bet this is almost exactly what he does, but of course his code is inside Deshaker, whereas your function is generalized to work with anything else one can dream up in AVISynth.

Nice job!

[wonkey_monkey]

thanks, and if the previous frame has nothing to composite with?

Can you elaborate?

Very interesting. When used with stabilization, as in your example, do you think it would produce better edge correction than what is contained in Depanstabilize, Deshaker, Mercalli, etc.?

I've no idea how they work, so I don't know, but probably not. The example I've posted works because all the frames are near-perfectly aligned - if instead you had a smoothed, rather than 100% fixed-in-space clip, then you wouldn't get such a good result, unless you have a way of remapping the previous frame to align with the current one.

[real.finder]

Can you elaborate?


like in Scene changed

[StainlessS]

I think this one's a contender for my most esoteric filter yet.
Yep, you watch way too much Dr Who :)

[wonkey_monkey]

like in Scene changed

It just serves the previous output frame as an input frame, regardless of what they contain or what is done with them, so scene changes aren't within its remit.

Yep, you watch way too much Dr Who :)

Actually this filter came about because of a Star Wars project.

[real.finder]

I did update stab3 to use it (try with stab3(use_Feedback=true)), https://github.com/realfinder/AVS-Stuff/blob/Community/avs%202.5%20and%20up/StabilizationToolsPack.avsi

it's kinda a test update, let see if there any problem

edit: redownload it, I did another update that make it work even better

[StainlessS]

Wow that was quick R.F, I'm still tryin' to figure out what it is and why it is :)

[johnmeyer]

Wow that was quick R.F, I'm still tryin' to figure out what it is and why it is :)It seems akin to reentrancy, something we all learned in the classroom, but always seemed to be difficult to get to work in the real world without causing problems (at least that was my experience). Hopefully this won't have memory issues, or lose its place (my problems the few times I tried having a function call itself).

[StainlessS]

Thanx John, tis the Wonderful World of Wonkey Willy's Chocolate Factory,
Only the likes of Vincent Van Gogh would understand.
[That other nutter that chopped his ear off. You still got two ears Wonkey ?]

[wonkey_monkey]

Cellular automaton (Conway's Life) in Avisynth (play linearly from frame 0, don't try and seek because it will crash):

blankclip(pixel_type = "y8", width = 360 + 40, height = 360 + 100, length = 30000, fps = 60)

Expr("sx 176 == sy 204 == & sx 175 == sy 205 == & | sx 177 == sy 205 == & | sx 176 == sy 206 == & | sx 183 == sy 214 == & | sx 181 == sy 215 == & | sx 183 == sy
215 == & | sx 180 == sy 216 == & | sx 181 == sy 216 == & | sx 183 == sy 216 == & | sx 179 == sy 218 == & | sx 180 == sy 218 == & | sx 181 == sy 218 == & | 255 0 ?")

FeedbackInput.Expr("x[-1,-1] x[0,-1] x[1,-1] x[-1,0] x[1,0] x[-1,1] x[0,1] x[1,1] + + + + + + + B^ B 510 == x 255 ? C^ B 765 > 0 C ? D^ B 510 < 0 D ?").FeedbackOutput

crop(20, 20, -20, -80) # required to hide gliders crashing at edges

[wonkey_monkey]

Something interesting from Blur/Shapen:

BlankClip(width = 32, height = 32, length = 30000).invert
AddBorders(944, 524, 944, 524)

FeedbackInput.Sharpen(1).Blur(1).FeedbackOutput

Even trippier when combined with my Warp plugin:

BlankClip(width = 32, height = 32, length = 30000).invert
AddBorders(944, 524, 944, 524)

FeedbackInput.Sharpen(1).Warp("
0,0, 0,-1
1920,0 1921.1,0
1920,1080 1920,1081.2
0,1080 -1.3,1080
").Blur(1).FeedbackOutput

[Dogway]

Hi wonkey_monkey! I'm testing your feedback snippets and it's very cool although the first one with Sharpen().Blur() crashed for me.

I'm also trying to implement it into ex_median() for the temporal modes but had some issues. I tested several combinations but all crash for me. To start I had to disable Prefetch(4) otherwise I would get an ID related error.

# b = a.selectevery(1,-1)
b = fb ? a.FeedbackInput() : a.selectevery(1,-1)
f = a.selectevery(1,+1)

isy ? Expr(a, b, f, str, optSingleMode = false ) : \
UV == 1 ? Expr(a, b, f, str, "", optSingleMode = false ) : \
Expr(a, b, f, str, ex_UVexpr(str, UV, bi, rgb, fs), optSingleMode = false, scale_inputs="none")

fb ? FeedbackOutput() : last

[wonkey_monkey]

Difficult to say. Feedback is pretty unlikely to work well with Avisynth multithreading, and frame access should be linear. If you skip too many frames I think it ends up trying to calculate/load all the skipped frames into memory at once so there's a limit to how many frames you can skip. Same goes for reloading a script in VirtualDub when you're not on frame 0 already.

[Dogway]

Yes, the sharpen.blur script of yours that failed was because I had trim at the end. Anyways I solved it by calling it twice, halves the performance but if you have a better idea...


b = rc ? a.selectevery(1,-1).ex_median(mode,false,UV,fs,thr) : a.selectevery(1,-1)
f = a.selectevery(1,+1)
b2 = temp5 ? rc ? b.selectevery(1,-1) : a.selectevery(1,-2) : nop()
f2 = temp5 ? a.selectevery(1,+2) : nop()