Attempt to stabilize fluctuation(flickering?) in old anime

[lansing]

This question arose occasionally here and I didn't really see any proper solution to it. Stabilizing filters like depan or mvdepan was not enough, in which the inner part of the frame still fluctuate like crazy on playback. Here's my attempt to clean it up

LRemoveDust(2,10) #stabilize inner frame fluctuation
stab() #global stablize
fastdegrain() #reduce flickering along lines

The problem I have with this is that LRemoveDust has to set to (2,>10) in order to be effective, but at the same time it blurs everything that pans

I've uploaded 2 samples to test with, the first one is a stilled scene, where and script works well, and the second sample is the one I have problem with panning.

Is there any way to modify LRemoveDust to the leave panning areas along?


UPDATE:

Link to updated script