Video enchancment / If time was not a factor?

[Lynx_TWO]

Theoretical Discussion here:

If time and computing power was not a factor:

Main Tools - StaxRip and VapourSynth

Starting video - Interlaced 720p footage (or even interlaced DVD) with some film judder

Workflow:

Step 1. Deinterlace

QTGMC | Keep FrameRate | QTGMC Placebo =
clip = core.std.SetFieldBased(clip, 2) # 1 = BFF, 2 = TFF
clip = havsfunc.QTGMC(clip, TFF = True, Preset = "Placebo", FPSDivisor=2, ForceTR=3, NNeurons = 4, SourceMatch=3, Lossless=2, Sharpness=0.0, TR2=3, NoiseProcess=1, NoiseRestore=0.0, Denoiser="dfttest", DenoiseMC=True, NoiseTR=2, NoiseDeint = "Generate", Sigma=4.0)

Step 2. Stabilize high frequency film judder

Stab = clip = haf.Stab(clip, range=1, dxmax=4, dymax=4, mirror=0)


Step 3. Upscale for denoiseing and further processing

http://www.infognition.com/videoenhancer/, Super-resolution. Output x254 lossless (q0) or similar, 2 passes to increase resolution to 2880p

Step 4. - Denoising

4pass SMDegrain =
pass1 = havsfunc.SMDegrain(clip, tr = 3, pel = 2, subpixel = 2, Str = 3.0, blksize = 8, overlap = 2, thSAD = 400, contrasharp = True, truemotion = True, RefineMotion = True)
pass2 = havsfunc.SMDegrain(pass1, tr = 3, pel = 2, subpixel = 2, Str = 3.0, blksize = 8, overlap = 2, thSAD = 350, contrasharp = True, truemotion = True, RefineMotion = True)
pass3 = havsfunc.SMDegrain(pass2, tr = 3, pel = 2, subpixel = 2, Str = 3.0, blksize = 8, overlap = 2, thSAD = 300, contrasharp = True, truemotion = True, RefineMotion = True)
clip = havsfunc.SMDegrain(pass3, tr = 3, pel = 2, subpixel = 2, Str = 3.0, blksize = 8, overlap = 2, thSAD = 250, contrasharp = True, truemotion = True, RefineMotion = True)

Step 5. - Increase detail

This, I don't know... Any machine learning based plugins to do this artificially yet?

Step 6. - Automatic color correction/re-balancing

I don't know any plugins that do this reliably based on scene changes...

Step 7. - Add some really fine grain back into the high rez upscale

TrueGrain or similar?

Step 8. - Downsize back to 1080p

Sinc = clip = core.fmtc.resample(clip, w=1920, h=1080, kernel="sinc", taps=128)

Step 9. - Compress

Whatever compression settings you usually work with

---------------------------------------------------------------

Any thoughts or suggestions? Again, we are looking for tools that achieve maximum theoretical quality here. In addition, it would be nice to have a plugin or option to be able to restart a computer as needed without using too much processing time, so processing in chunks might be an option...

Thanks in advance!

[Lynx_TWO]

and apparently I can't spell :(

[poisondeathray]

Starting video - Interlaced 720p footage (or even interlaced DVD) with some film judder


Huh ? Isn't that an oxymoron? "interlaced" and "720p" ? If it was "720p" you wouldn't deinterlace . What is the original, original source? film, video ? film transfer to video ? And what you have now ?



Step 2. Stabilize high frequency film judder




If there is a film judder, this implies "film" at some point in the history. This probably means you have duplicate frames if you were deinterlacing and using fpsdivisor=2. You should probably decimate duplicates in that case before anything else - because all temporal filters will fail (or work less effectively) when you have duplicates. You need 1:1 unique film frames


Step 3. Upscale for denoiseing and further processing


Why upscale ?

And if you have good reasons , why upscale then farther process ? It will just be slower . And then you downscale ? I don't see the benefit

[feisty2]

Upscaling enables subpixel filtering, maybe that's what he wants

[feisty2]

Step 5. - Increase detail

This, I don't know... Any machine learning based plugins to do this artificially yet?

Yup, key words: pixelRNN/pixelCNN, variational autoencoder, generative adversarial network

[hello_hello]

I'd give serious consideration to denoising with QTGMC, or cranking it's denoising up if it's enabled, rather than following it with SMDegrain (if you're running both on the same video). I find QTGMCs noise removal/blurring ratio is pretty good. Probably better than SMDegrain, especially where there's motion. For reasons I can't recall, when I compared QTGMC's default denoising to Denoiser="dfttest", I prefered the default denoising. I can't remember why as it was a long time ago.

FPSDivisor=2 seems like a horrible thing to inflict on interlaced sources, IMHO.

Speaking of QTGMC, I'd recommend comparing it to SMDegrain for denoising progressive sources.
Because QTGMC "stabilises" the picture, I find when using it for denoising it's often better to use a faster speed preset. The logic being, if QTGMC doesn't remove all the noise, it tends to stabilise what's left, resulting in static noise, if there is such a thing. That can produce something of a "looking through a flyscreen" effect for really noisy sources. Faster presets stabilise less, so faster is sometimes better. I rarely get much more excited than this, aside from adjusting the amount of denoising and changing the speed preset.

QTGMC(InputType=1, Preset="medium", EzDenoise=1.5)

I can't say I'd be inclined to add grain. To prevent banding though, some dithering at the end of the script is usually a good idea.

Gradfun3()

Does all that upscaling and filtering and downscaling again result in a tangible quality improvement? I can't say I've ever tried it.

[Ghitulescu]

Any thoughts or suggestions? Again, we are looking for tools that achieve maximum theoretical quality here.

The maximum quality at the maximum expense of time would be to take each frame, one by one, and pixel-paint every section of it, taking into account the surrounding zones and scenes (for instance, if another scene is taking place in the decors and/or props of another one, that info may be used to recreate damaged zones of the first scene) - but this is for life sentences :)

[feisty2]

The maximum quality at the maximum expense of time would be to take each frame, one by one, and pixel-paint every section of it, taking into account the surrounding zones and scenes (for instance, if another scene is taking place in the decors and/or props of another one, that info may be used to recreate damaged zones of the first scene) - but this is for life sentences :)

That's exactly how pixelRNN works, it generates pixels based on a Bayes model

[lansing]

I'm suspecting that you're creating a restoration script for some old film? SMDegrain definitely needs to go because it's by nature altering the image on moving objects (destroying the image). The best commercial denoiser on the market is neatvideo but it needs the old virtualdub as a bridge and we didn't have a loadvirtualdubplugin filter in vs. The best all around denoiser in avisynth is MCTD but we don't have a port for it either.

[cwk]

Yup, key words: pixelRNN/pixelCNN, variational autoencoder, generative adversarial network

Reading about these things has officially made my head explode. A computer-generated image of a cat that appears natural to human observers. It sounds like I'll be watching HD versions of my 80's era hockey games soon.

[feisty2]

Reading about these things has officially made my head explode. A computer-generated image of a cat that appears natural to human observers. It sounds like I'll be watching HD versions of my 80's era hockey games soon.

If u define "soon" as "I'm getting my own supercomputer", we all know Moore's law is coming to an end and computers r no longer getting faster rapidly

[Logan9778]

QTGMC @ Placebo / x264 @ Placebo. :p