Thanks for the input markanini. Yes dehalo_alpha and hqdering are more appropriate for animation, also they don't work with big ringing (halos?) like these. Anyways it served me to have a second look to HQDering and update it with a masktools2 version I found. I hope to solve this issue once for all.
if anyone's curious I ended up using
stabilise for deshake, it requires lots of extra work, but it is the filter to be.