Super 8 scan upscale improvements

[domb84]

Hi all, I purchased one of those cheap frame by frame film scanners to transfer some old family films from Super 8. The results are not terrible given the cost of the machine, but I'd like to do what I can to improve it. I've had some success at removing the horrible dancing grain which I assume added by the poor sensor and relatively low bit rate of the scanner. Is there anything I can now do to deblur or sharpen the scan? Ideally I'd like to upscale to 4k, deblur, then crop and resize back down to 1080.

I've tried some of the AI upscalers (basicsvr++ and various vgsan models) without much success.

This is my script so far.


# Imports
import vapoursynth as vs
from vapoursynth import core
import havsfunc as haf
import lostfunc as lof
import muvsfunc as muf
import CropResize as cr
import adjust
from vsbasicvsrpp import basicvsrpp
import G41Fun

# Load source
src = core.ffms2.Source(source="\\\\nas\\Restoration\\source\\Film Scans\\0130.MP4")
src = core.std.AssumeFPS(src, fpsnum=18, fpsden=1)

# make a new source
clip = src

# convert to YUV
clip = core.resize.Bicubic(clip, format=vs.YUV444P8, matrix_in_s='709')

# levels correction
clip = haf.SmoothLevels(clip, input_low=32, gamma=1, input_high=255, output_low=8, output_high=235)

# degrain / block
clip = G41Fun.TemporalDegrain2(clip, degrainTR=2, postFFT=3, postSigma=1, extraSharp=True)

# stabilise
clip = haf.Stab(clp=clip,mirror=0,dxmax=120,dymax=120)

# despot
clip = lof.DeSpot(clip)

# convert to RGBS
clip = core.resize.Bicubic(clip, format=vs.RGBS)

# auto white balance
clip = core.grwrld.grayworld(clip, cc=1)

# upscale
#clip = basicvsrpp(clip, model=8, tile_w=240, tile_h=240, tile_pad=16, length=7)
"""Improving Video Super-Resolution with Enhanced Propagation and Alignment

:param clip: Clip to process. Only RGBH and RGBS formats are supported.
RGBH performs inference in FP16 mode while RGBS performs inference in FP32 mode.
:param device_index: Device ordinal of the GPU.
:param model: Model to use.
0 = Video Super-Resolution (REDS) 4x enhancement
1 = Video Super-Resolution (Vimeo-90K BI degradation) 4x enhancement, quick, ok
2 = Video Super-Resolution (Vimeo-90K BD degradation) 4x enhancement, quick, good results
3 = NTIRE 2021 Video Super-Resolution # 4x sr, ok
4 = NTIRE 2021 Quality Enhancement of Compressed Video - Track 1 # 1x enhancement, loses detail
5 = NTIRE 2021 Quality Enhancement of Compressed Video - Track 2 # 1x enhancement, loses detail
6 = NTIRE 2021 Quality Enhancement of Compressed Video - Track 3 # 1x enhancement, loses detail
7 = Video Deblurring (DVD) 1x fast deblur
8 = Video Deblurring (GoPro) 1x fast deblur
9 = Video Denoising # 1x fast denoise
:param length: Sequence length that the model processes.
:param cpu_cache: Send the intermediate features to CPU.
This saves GPU memory, but slows down the inference speed.
:param tile_w: Tile width. As too large images result in the out of GPU memory issue, so this tile option
will first crop input images into tiles, and then process each of them. Finally, they will
be merged into one image. 0 denotes for do not use tile.
:param tile_h: Tile height.
:param tile_pad: Pad size for each tile, to remove border artifacts.
"""

#clip = core.nnedi3cl.NNEDI3CL(clip, field=0, dh=True, dw=True)
#clip = core.resize.Spline36(clip, src_left=-0.5, src_top=-0.5, format=vs.YUV444P16, matrix_s='709')

# resize and convert to yuv
#clip = cr.CropResize(clip, 1440, 1080, 108, 108, 72, 72, InDAR=4.0/3.0, OutDAR=4.0/3.0, Info=False, Borders=True)
clip = core.resize.Bicubic(clip, format=vs.YUV444P16, matrix_s='709')
#clip = core.resize.Bicubic(clip, 1440, 1080, format=vs.YUV444P16, matrix_s='709')

# deband
#clip = core.neo_f3kdb.Deband(clip, dither_algo=1, grainy=128, grainc=128)

# add grain
#clip = core.grain.Add(clip)

# resize source
# crop bars
src = core.resize.Bicubic(src, format=vs.RGBS, matrix_in_s='709')
src = core.resize.Bicubic(src, 1440, 1080, format=vs.YUV444P16, matrix_s='709')
# histogram and compar
# add name and histogram
clip = muf.DisplayHistogram(clip)
clip = core.text.Text(clip, 'output')
src = muf.DisplayHistogram(src)
src = core.text.Text(src, 'source')
# compare
clip = core.std.StackHorizontal([src, clip])

# output
clip.set_output(0)


Output examples here:

https://mega.nz/file/5eongAIR#LekEPD9BbZ36DmCXFGm2bDLI8q4BxDaRaXktZMECs0w
https://mega.nz/file/1eYUhJAJ#HlZkh5MBWCFyhmNnLEFosnLsVT41pES4dPoi1c460qI
https://mega.nz/file/kKISFbLC#q-FJreOs-OKf7Unod_x-kL5o052NmZW_iU5F84YXJZQ

Any help appreciated!

[domb84]

I assume that is some ghosting as well, but I'm unsure on what i should be targeting with lghost in order to remove it. Any advice for that would be great.

[VideoMilk78]

I feel bad for you man, those scanners are a mess, I got one to mod and even then it still wasn't great

[domb84]

Personally for the job I need it for and the cost its fine. I just want to get the last 10% out of the scans.

[Selur]

Is there anything I can now do to deblur or sharpen the scan?
Share a sample of the source not some output images,...

[domb84]

Share a sample of the source not some output images,...

Here's a short sample

https://mega.nz/file/JSJSDarb#KmJu_OmhynQDlDZV3FKQ3UBo74yFpMY-CX-KCvVEDrQ

This is the worst scan of the bunch due to the low light, so thought the worst case scenario would be a good place to start.

On another note, all I get using reginex is a blue cast over everything. Any ideas why that might be?

[Selur]

Retinex works fine here, so it's something in your script.
https://i.ibb.co/k8DcHg3/grafik.png (https://ibb.co/nc1kgs3)
You get the blue effect if you disable msrcp.

I player a bit with the sample and used a LUT for the coloring and Despot + BasicVSR++ and SCUNet for the denoising (https://www.mediafire.com/file/6a98ndvqpmrvcp3/BasicVSR%252B%252B_SCUNet.mp4/file) (script (https://pastebin.com/JV59kXWm))

[domb84]

Perhaps because I'm doing it lower down the chain after removing the noise and attempting to rectify the levels a bit.
Ive not disable msrcp.

Any advice on up-scaling the footage or improving it beyond what I have? I appreciate it's never going to be perfect. I seem to have hit a bit of a wall though.

[Selur]

The only thing that might add artificial details is something like CodeFormer.