How to use 'DePan' in StaxRip?.

[pitoloko]

Hello. Please note that I don't have much experience in these matters...

The thing is, I have some DVDs of the 'Dragon Ball Z' movies from the 80s and 90s. The format is Interlaced (Top Field First), and the image has everything: rainbows (like red and blue color stripes flickering in the white of the eyes), ringing and haloing, a lot of noise, and a slight constant image shake between frames...

I've been researching the latter, and I think the term is "jitter," or perhaps "telecine wobble" or "film weave."

I don't think there's a filter in StaxRip to solve this problem.

So, using ChatGPT, he recommended the 'Depan' and 'DepanStabilize' filters, which are part of the 'Mvtools2' package: https://github.com/pinterf/mvtools/releases

In theory, these filters are supposed to stabilize the image to correct that "camera shake"-like effect. So I've picked the MSVC x64 builds of Mvtools2.

The problem is, I have no idea how to implement it in my StaxRip script.

I'm using StaxRip's own interface to edit the script, and all I managed to do was add these three lines without the GUI giving me an error:

LoadPlugin("C:\Program Files (x86)\AviSynth+\plugins64+\mvtools2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth+\plugins64+\Depan.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth+\plugins64+\DepanEstimate.dll")

Can anyone help me by explaining the procedure?

ChatGPT is completely useless when it comes to answering questions about using filters in a StaxRip script. No matter how hard I try, he doesn't even understand that filters aren't used by passing a clip through it.

Anyway, I'm completely lost.

I'm using the "DeRainbow" filter, "mClean" for noise, "DehaloAlpha" for ringing, and "NNEDI3" with "Spline64resize" to upscale to 1080p in a 4:3 ratio:

http://i.imgur.com/WbvJdmAl.png (https://i.imgur.com/WbvJdmA.png)
(Click on the image to enlarge)

(yes I'm lacking a filter to deinterlace the DVD image, probably I will use QTGMC with double framerate. I still need to test many filter combinations.)

The result is far from perfect, but at least many artifacts are corrected or reduced a lot, and overall it looks better than the original DVD image.

I just want to be able to stabilize the image.

Thanks in advance!

[hello_hello]

Try the Stab() function. It uses Depan/DePanEstimate to stabilize.
http://www.avisynth.nl/index.php/Stab
It's safe to ignore the warning about not using it with color sources (on the wiki page).

[pitoloko]

Try the Stab() function. It uses Depan/DePanEstimate to stabilize.
http://www.avisynth.nl/index.php/Stab
It's safe to ignore the warning about not using it with color sources (on the wiki page).

Thanks for your help.

EDIT: I finally managed to get the 'Stab' function working by copying the 'Stab.avsi' file into the AviSynth+ plugins directory and adding the following line to my script in StaxRip:

Stab(range=1, dxmax=1, dymax=1, mirror=0)

That said, what this function actually does is smooth out the camera shake rather than truly correcting or eliminating it. In fact, the result feels even more disorienting to me — the image seems to float unnaturally, which can cause dizziness or (much more)discomfort compared to the original shaky footage of the DVD video.

Exists a better solution?.

[Selur]

a. dxmax=1, dymax=1 seems rather small (I never had a source with such a small 'shake')
b. you should make sure to crop after applying Stab
No clue whether there is a better solution, without having an unprocessed sample of the file you filter.
You should also share the script you use.
There are other filters that could be used for stabilization you could try, see: http://avisynth.nl/index.php/External_filters#Stabilization

[pitoloko]

a. dxmax=1, dymax=1 seems rather small (I never had a source with such a small 'shake')
b. you should make sure to crop after applying Stab
No clue whether there is a better solution, without having an unprocessed sample of the file you filter.
You should also share the script you use.
There are other filters that could be used for stabilization you could try, see: http://avisynth.nl/index.php/External_filters#Stabilization

Thanks for the cropping tip, of course to remove the mirrored image or black color I suppose.

This is an unprocessed sample of 8 seconds (well, it is a re-encoded sample but without using any plugin, so it preserves the shake EDIT: only used 'Crop(8, 0, -8, -0)' but the shake is visible):

- https://www.mediafire.com/file/ttnnluizm6e7o3i/normal.mkv/file

This is a 1080p upscale test of that same scene, using Stab with 'range=5, dxmax=1, dymax=1' (among other restoration plugins):

- https://www.mediafire.com/file/ccem3493drsup1m/upscale+test.mkv/file

And finally, this is the full script generated by StaxRip to make that upscaled video:

AddAutoloadDir("C:\Program Files\StaxRip\Apps\FrameServer\AviSynth\plugins\")
LoadPlugin("C:\Program Files\StaxRip\Apps\Plugins\AVS\masktools2\masktools2.dll")
LoadPlugin("C:\Program Files\StaxRip\Apps\Plugins\AVS\MedianBlur2\MedianBlur2.dll")
LoadPlugin("C:\Program Files\StaxRip\Apps\Plugins\AVS\RgTools\RgTools.dll")
Import("C:\Program Files\StaxRip\Apps\Plugins\AVS\Zs_RF_Shared\Zs_RF_Shared.avsi")
LoadPlugin("C:\Program Files\StaxRip\Apps\Plugins\AVS\FFT3DFilter\fft3dfilter.dll")
LoadPlugin("C:\Program Files\StaxRip\Apps\Plugins\Dual\FFT3D Neo\neo-fft3d.dll")
LoadPlugin("C:\Program Files\StaxRip\Apps\Plugins\AVS\TMM2\TMM2.dll")
LoadPlugin("C:\Program Files\StaxRip\Apps\Plugins\AVS\MipSmooth\MipSmooth.dll")
LoadPlugin("C:\Program Files\StaxRip\Apps\Plugins\AVS\MSharpen\msharpen.dll")
LoadPlugin("C:\Program Files\StaxRip\Apps\Plugins\AVS\AutoAdjust\AutoAdjust.dll")
LoadPlugin("C:\Program Files\StaxRip\Apps\Plugins\Dual\f3kdb Neo\neo-f3kdb.dll")
LoadPlugin("C:\Program Files\StaxRip\Apps\Plugins\AVS\modPlus\modPlus.dll")
LoadPlugin("C:\Program Files\StaxRip\Apps\Plugins\AVS\mvtools2\mvtools2.dll")
LoadPlugin("C:\Program Files\StaxRip\Apps\Plugins\AVS\AddGrainC\AddGrainC.dll")
LoadPlugin("C:\Program Files\StaxRip\Apps\Plugins\AVS\Bwdif\BWDIF.dll")
LoadPlugin("C:\Program Files\StaxRip\Apps\Plugins\AVS\DFTTest\dfttest.dll")
Import("C:\Program Files\StaxRip\Apps\Plugins\AVS\Dither\dither.avsi")
LoadPlugin("C:\Program Files\StaxRip\Apps\Plugins\AVS\Dither\dither.dll")
LoadPlugin("C:\Program Files\StaxRip\Apps\Plugins\AVS\EEDI2\EEDI2.dll")
LoadPlugin("C:\Program Files\StaxRip\Apps\Plugins\AVS\EEDI3\eedi3.dll")
LoadPlugin("C:\Program Files\StaxRip\Apps\Plugins\AVS\NNEDI3CL\nnedi3cl.dll")
LoadPlugin("C:\Program Files\StaxRip\Apps\Plugins\AVS\TDeint\TDeint.dll")
LoadPlugin("C:\Program Files\StaxRip\Apps\Plugins\AVS\vinverse\vinverse.dll")
LoadPlugin("C:\Program Files\StaxRip\Apps\Plugins\AVS\yadifmod2\yadifmod2.dll")
Import("C:\Program Files\StaxRip\Apps\Plugins\AVS\DehaloAlpha\Dehalo_alpha.avsi")
Import("C:\Program Files\StaxRip\Apps\Plugins\AVS\Scripts\DeRainbow.avsi")
LoadPlugin("C:\Program Files\StaxRip\Apps\Plugins\Dual\ffms2\ffms2.dll")
LoadPlugin("C:\Program Files\StaxRip\Apps\Plugins\AVS\JPSDR\Plugins_JPSDR.dll")
Import("C:\Program Files\StaxRip\Apps\Plugins\AVS\mClean\mClean.avsi")
Import("C:\Program Files\StaxRip\Apps\Plugins\AVS\QTGMC\QTGMC.avsi")
tcFile = (...too large line to display in this post...)\D2_t01.mkv_temp\D2_t01_timestamps.txt" # timestamps file path
Exist(tcFile) ? FFVideoSource(...too large line to display in this post...)
Crop(8, 0, -8, -0)
QTGMC(preset="Slower")
nnedi3_rpow2(rfactor=2, nns=4, qual=1, pscrn=4, cshift="Spline64Resize", fwidth=1320, fheight=1080)
Stab(range=5, dxmax=1, dymax=1)
DeRainbow(thresh=10, interlaced=false)
mClean(thSAD=400, chroma=true, sharp=20, rn=14, deband=4, depth=0, strength=20)
DeHalo_alpha_mt(rx=2.0, ry=2.0, darkstr=0.85, brightstr=1.0, lowsens=40, highsens=60, ss=2.0)
Trim(51410, 51841) + Trim(87750, 89278) + Trim(176248, 177342)


Note: I've noticed that applying all the restoration plugins AFTER scaling the image with nnedi3_rpow2, it increases a lot the final video file size in comparison if I apply them BEFORE, but I also think the restoration plugins does a better job giving better results this way. Please correct me if I'm wrong and it's just a false feeling of mine.

[hello_hello]

Stab normally works pretty well, but as Selur said, limiting it to correcting any movement by only one pixel is fairly err..... limiting. :)
Very occasionally Stab will attempt to correct "shake" where there is none, due to an object in the picture itself moving, but it's really only a problem with animation as live action has natural gradients and detail rather than large flat areas of single colors (edit: I just downloaded your sample and realised it's animation).

I haven't used Stab often myself, but if it can only move the picture in increments of a whole pixel (I'm not sure) and you're upscaling, you could try placing it in the script before the upscaling, then try it again after the upscaling, to see which way works better.

In a perfect world you'd probably de-interlace, stabalize, then apply any denoising and any other filters, although having said that, if you're de-interlacing with QTGMC you might as well do the denoising with it too. As part of the de-interlacing process QTGMC tends to remove a little noise anyway, but you can enable it's own denoising if the source needs it. QTGMC(EzDenoise=1.0) might be a good start. If there's still noise remaining try incrementing the EzDenoise value in increments of 0.5 until you're happy.

[pitoloko]

Stab normally works pretty well, but as Selur said, limiting it to correcting any movement by only one pixel is fairly err..... limiting. :)

The problem was I misinterpreted those parameters as: "do not move/shift/misalign the image (from its "center" point) more than, in this case, 1 pixel from X,Y coordinates"

Because the examples given in the AviSynth documentation with the default values ​​are supposed to be optimal (or at least very convenient to use for most common scenarios), when I saw dxmax=1 dymax=1 I thought that this configuration would give the best results and that's why I used this logic to reach my erroneous conclusion.

Really thanks for the suggestions you wrote for me.

[hello_hello]

I gave your sample a quick try and it looks better with Stab to me. I assume the sample is a de-interlaced re-encode of the source. I upscaled it to see how stable it'd look. I indulged my obsession with cropping the borders clean and the DAR to at least 4:3 while I was at it, but that's just me. I assume converting the colors from SD to HD colorimetry was the right thing to do, but there's no color info in the sample.

Edit: I noticed your upscaled version has SD colorimetry written to the video stream, so it should display with the same colors as the sample I uploaded, as long as the player pays attention to that sort of thing.


normal2.mkv (https://files.videohelp.com/u/210984/normal2.mkv)

Stab()
TemporalDegrain2_Fast(Strength=10, RadT=2, BlkSz=16)
nnedi3_rpow2(rfactor=2, nns=4, qual=1, pscrn=4, cshift="Spline64Resize")
CropResize(1440,1080, 8,4,-8,-4, InSAR=12.0/11.0, ColorConvert="AVSResize", ColorMode="601p-709")

[Selur]

The format is Interlaced (Top Field First),
...
This is an unprocessed sample of 8 seconds (well, it is a re-encoded sample but without using any plugin, so it preserves the shake EDIT: only used 'Crop(8, 0, -8, -0)' but the shake is visible):
That file isn't interlaced. Did you deinterlace, or isn't the source interlaced?

Using stab, small deflicker (https://forum.doom9.org/showthread.php?p=1812060#post1812060) doesn't seem bad: https://www.mediafire.com/file/dk05qrvyjp3rb6u/stab_small_deflicker.mp4/file
(if the source really is interlaced, working on the fields might be worth a try too)

Cu Selur

[pitoloko]

I gave your sample a quick try and it looks better with Stab to me.

It seems to me that cropping after Stab really makes a difference, the shake is very less noticeable unless you focus your eyes on the left or right side of the frame to see things "weaving".

I assume the sample is a de-interlaced re-encode of the source.

The sample was not deinterlaced, only reencoded. It looks progressive to you?. Because it looks progressive to me, but Mediainfo says the source (DVD) is interlaced, although I don't notice separated fields or "ghosting". The interlaced metadata can give a "false positive"?.

I assume converting the colors from SD to HD colorimetry was the right thing to do, but there's no color info in the sample.


The process of "converting the colors from SD to HD colorimetry" It is something that goes beyond my basic knowledge. And I don't know the benefits or disadvantages. If you want to give any tip, I would love to learn more.

Not sure if the following has a relationship, but I just can tell you that when encoding the upscaled video (with x264 in StaxRip) I used "--colormatrix bt470bg" parameter to try keep the source colors of the DVD, because in the past StaxRip changed the colors of other anime DVD (a red hair changed to a brighter red color, etc) and I discovered using that parameter fixes it. i've asked ChatGPT to try ensure 'bt470bg' was the correct color space to use for this video sample of 'Dragon Ball Z', the AI said yes, so I've used it.

Thanks for everything and for sharing yous script.

----------------------------------------------------------------
----------------------------------------------------------------
----------------------------------------------------------------

That file isn't interlaced. Did you deinterlace, or isn't the source interlaced?

The sample I provided was not deinterlaced, only has been (cropped and)reencoded with x264.

Mediainfo says the source (DVD) is interlaced with top field first. Although when I analyze the frames in Staxrip it looks progressive to me, I don't notice separated fields nor ghosting.

As it seems to be very important than what I thought at first sight (excuse me for thinking that way), I've uploaded a true unprocessed sample (same scene but longer), a segment of the DVD-Remux splitted with MKVToolnix:

- https://www.mediafire.com/file/878a2f96fqyabgt/D2_t01+(1).mkv/file

This is the mediainfo report:

Format : MPEG Video
Format version : Version 2
Format profile : Main@Main
Format settings : CustomMatrix / BVOP
Format settings, BVOP : Yes
Format settings, Matrix : Custom
Format settings, GOP : M=3, N=12
Format settings, picture st : Frame
Codec ID : V_MPEG2
Codec ID/Info : MPEG 1 or 2 Video
Duration : 24 s 0 ms
Bit rate mode : Variable
Bit rate : 6 526 kb/s
Maximum bit rate : 9 800 kb/s
Width : 720 pixels
Height : 576 pixels
Display aspect ratio : 4:3
Frame rate mode : Constant
Frame rate : 25.000 FPS
Standard : PAL
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Top Field First
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.629
Stream size : 18.7 MiB (96%)
Original source medium : DVD-Video

Using stab, small deflicker (https://forum.doom9.org/showthread.php?p=1812060#post1812060) doesn't seem bad: https://www.mediafire.com/file/dk05qrvyjp3rb6u/stab_small_deflicker.mp4/file
(if the source really is interlaced, working on the fields might be worth a try too)

Motion stabilization in your sample looks way better than what I managed to do. Could you please share the parameters used for Stab and small deflicker?. Just to try mimic your good results.

Thank you in any case!

Also, even though the image appears slightly blurrier, it's noticeably cleaner and has fewer artifacts compared to my 1080p sample. Great work. Of course, I'm still far from achieving the kind of results that true specialists on this community can reach.

[VideoMilk78]

I would recommend using the better/newer method recommended by videoFred
introduced here:
https://forum.doom9.org/showthread.php?p=1981048#post1981048

Or the a more advanced version here:
https://forum.doom9.org/showthread.php?p=1849311#post1849311

But if you want to use stab, here's the new method inserted into the old function:

function Stab (clip "clip", int "dxmax", int "dymax", int "mirror") {

dxmax = default(dxmax, 4)
dymax = default(dymax, 4)
mirror = default(mirror, 0)

vectors= clip.MSuper().MAnalyse(isb=false)
mdata= MDepan(clip,vectors,rot=false, zoom=false, error= 65)

stab= clip.DePanStabilize(data=mdata,dxmax= dxmax,dymax= dymax,mirror=mirror)
return stab
}

[hello_hello]

The sample was not deinterlaced, only reencoded. It looks progressive to you?. Because it looks progressive to me, but Mediainfo says the source (DVD) is interlaced, although I don't notice separated fields or "ghosting". The interlaced metadata can give a "false positive"?.

Encoders can encode video as interlaced even when it's progressive. Pretty much every PAL DVD I've met has been encoded as interlaced. For HD that sort of nonsense was officially given a name.
https://en.wikipedia.org/wiki/Progressive_segmented_frame

MediaInfo only reports the encoding method. It can't know whether the content is truly interlaced or progressive.


The process of "converting the colors from SD to HD colorimetry" It is something that goes beyond my basic knowledge. And I don't know the benefits or disadvantages. If you want to give any tip, I would love to learn more.

Not sure if the following has a relationship, but I just can tell you that when encoding the upscaled video (with x264 in StaxRip) I used "--colormatrix bt470bg" parameter to try keep the source colors of the DVD, because in the past StaxRip changed the colors of other anime DVD (a red hair changed to a brighter red color, etc) and I discovered using that parameter fixes it. i've asked ChatGPT to try ensure 'bt470bg' was the correct color space to use for this video sample of 'Dragon Ball Z', the AI said yes, so I've used it.

There's different matrices used to convert YUV to RGB for display. The SD and HD matrices are slightly different, so a conversion with the wrong matrix can change the colors a little.
SD PAL uses bt470bg (often referred to generically as rec.601) and HD uses rec.709.
The difference you would have seen probably wasn't StaxRip changing the colors, it was the matrix used by the player to convert the YUV to RGB. In the absence of any color information in the video stream, a player should used rec.601 for standard definition and rec.709 for HD. You probably upscaled a SD video which should use rec.601, and when the player/renderer converted it to RGB for your display, it used the rec.709 matrix because the video was HD. The colors wouldn't have actually changed as such, but the conversion to RGB on playback changed them.

Writing bt470bg to the video stream should cause a player to use the SD matrix regardless of the resolution, so the upscaled version then looks the same as the SD source. Alternatively you can physically convert the colors to rec.709 and write rec.709 to the video stream, and the colors will still look the same after upscaling because rec.709 will be used to convert it to RGB on playback. Out of habit I tend to use the "convert the colors" method, plus it doesn't rely on the player obeying the colorimetry info written to the video stream.

UHD uses yet another matrix (rec.2020) and quite different primaries compared to SD and HD. For the record, the primary colors are what defines pure red, blue and green. Here's what it looks like. The points of the triangle are the definition of pure red, blue and green, so the color space can produce all the colors inside the triangle, but none of the colors outside it.
PAL and NTSC
https://en.wikipedia.org/wiki/File:CIExy1931_Rec_601.svg
HD
https://en.wikipedia.org/wiki/File:CIExy1931_Rec_709.svg
UHD
https://en.wikipedia.org/wiki/Rec._2020#/media/File:CIExy1931_Rec_2020.svg

The matrix used to convert to RGB and the definition of the primary colors are two different things. For example you could have a video with HD primaries that uses the UHD matrix to convert the colors to RGB. :)
Because the SD and HD primaries are so similar, as far as I'm aware the difference is generally ignored when converting between them. It's the matrix used to convert to RGB on playback that makes a noticeable difference. If a rec.709 video is converted to RGB using rec.601, red will become darker and blue and green slightly brighter, or when a rec.601 video is converted with rec.709 on playback, red will become more orange and blue and green a little darker.
https://en.wikipedia.org/wiki/Rec._709#Standards_conversion

[Selur]

Stab (range=3, dxmax=4, dymax=4)
Small_Deflicker(preset=3, cnr=true)
I cropped 4 pixels on all sides.
After denoising (using makes denoising might help to preserve more details), with cartoon/animes line darkening and some sharpening can help go get the appearance of sharpness back.
(didn't really look into sensitive denoising&co since stabilization was the main goal,...)

Cu Selur