Log in

View Full Version : Analog Remastering process, where to start


Pages : [1] 2

FranceBB
17th July 2020, 09:18
Hi there,
as a follow up to This (https://forum.doom9.org/showthread.php?t=181234) and after endless power point presentations to convince my boss that the project was worth it, on the 15 of June we finally began our Remastering process for... uh... about 466'000 tapes that are stored in our library underground with no light whatsoever and at constant temperature and humidity. Some of the tapes there are really old, we're talking about 1987, so oxide tapes, some other were instead recorded on 1 inch Type C and we even have film, like, literally, film that I could hold on my bare hands (I didn't touch it directly of course, I don't wanna destroy it with my greasy hands after all these years). It's gonna take a lot of time.

https://images.squarespace-cdn.com/content/v1/5992fd6737c581f4bd00ec6a/1508208930836-VM7TDX6TRJ9CNK13QOSS/ke17ZwdGBToddI8pDm48kMXRibDYMhUiookWqwUxEZ97gQa3H78H3Y0txjaiv_0fDoOvxcdMmMKkDsyUqMSsMWxHk725yiiHCCLfrh8O1z4YTzHvnKhyp6Da-NYroOW3ZGjoBKy3azqku80C789l0luUmcNM2NMBIHLdYyXL-Jww_XBra4mrrAHD6FMA3bNKOBm5vyMDUBjVQdcIrt03OQ/IMG_2992.JPG
https://upload.wikimedia.org/wikipedia/commons/4/4c/Betacam_betamax_tapes.jpg
https://budgetvideo.com/images/gallery/sony_dvw-a500.jpg


Files are captured in lossless 720x576 25i 8bit at about 166 Mbit/s which roughly accounts for a bit more than 1 GB per minute, so it's actually fine. Then, I encode each and every file manually to get rid of every issues like noise, raimbowning, spots, scratches and so on, then I make sure levels are right, I convert matrix and primaries with 16bit precision, I upscale with Spline64 + NNEDI to FULL HD and then I dither it down to 8bit again with the Floyd-Steinberg error diffusion and encode them to our mezzanine format. The Remaster is then archived and it's readily available to our system thanks to the metadata. Of course, since contents are interlaced, I bob-deinterlace them first to 50p with QTGMC, filter out everything and then re-interlace them to 25i TFF.
Now a problem arises: it's too slow and it would take me forever to encode 400k contents, so they're asking me to automatize the encoding process. (https://i.imgur.com/tr0Q9mI.png)
I tried to setup a script, but I can't really seem to get a grip on it 'cause there are so many contents and they're so different that it's really hard to setup something good. Besides, using ColorYUV with autogain=true and autowhite=true saved some materials and screwed up some others. Although the "one script to rule them all" goes against my policy of encoding everything manually, I recognize that such a thing it's simply not doable. So, in the end, what should I do? What would you do if you were in my shoes?

I was thinking about something like this:


#Bob-deinterlace to 50p
QTGMC(Preset="placebo")

#Bring everything to 16bit planar
ConvertBits(bits=16)

#Convert to 4:2:2 planar 16bit
Converttoyuv422(matrix="Rec601")

#Automatic levels correction with 16bit planar precision
ColorYUV(autowhite=true, autogain=true)

#Adding Borders to get a 1.33 PB with 16bit planar precision
AddBorders(152, 0, 152, 0)

#Upscale to FULL HD with Spline64 + NNEDI and 16bit planar precision
nnedi3_rpow2(cshift="Spline64ResizeMT", rfactor=2, fwidth=1920, fheight=1080, nsize=4, nns=4, qual=1, etype=0, pscrn=2, threads=0, csresize=true, mpeg2=true, threads_rs=0, logicalCores_rs=true, MaxPhysCore_rs=true, SetAffinity_rs=false)

#Sharpening with 16bit planar precision
SeeSaw(resized, Sstr=8.8, Spower=5)

#From 16bit planar to 16bit interleaved
ConvertToDoubleWidth()

#Matrix Conversion from BT601 to BT709 with 16bit interleaved precision
Matrix(from=601, to=709, rg=1.0, gg=1.0, bg=1.0, a=16, b=235, ao=16, bo=235, bitdepth=16)

#From 16bit interleaved to 16bit planar
ConvertFromDoubleWidth()

#Dithering from 16bit planar to 8bit planar with the Floyd-Steinberg error diffusion
ConvertBits(bits=8, dither=1)

#Limiter TV Range 0.0 - 0.7V
Limiter(min_luma=16, max_luma=235, min_chroma=16, max_chroma=240)

#Loudness Correction
Normalize(0.22, show=false)

#Re-Interlace to 25i TFF PAL
assumeTFF()
separatefields()
selectevery(4,0,3)
weave()



however ColorYUV fucked up some contents spectacularly.
Should I be more "conservative" and keep that script but without ColorYUV() and keep everything, including dot crawl, tons of grain, noise and many other things or should I do something more aggressive like:

#Bob-deinterlace to 50p
QTGMC(Preset="placebo")

#De-raimbowning
ChubbyRain2(th=10, radius=10, show=false, sft=10, interlaced=false)

#De-scratch
DeScratch(mindif=5, asym=10, maxgap=3, maxwidth=3, minlen=90, maxlen=2048, maxangle=5, blurlen=15, keep=30, border=2, modeY=3, modeU=3, modeV=3, mindifUV=0, mark=false, minwidth=1)

#De-spot
DeSpot(mthres=16, mwidth=7, mheight=5, merode=33, interlaced=false, show=0, show_chroma=false)

#De-haloing
BlindDeHalo3(lodamp=0.0, hidamp=0.0, sharpness=0.0, tweaker=0.0, PPmode=0, interlaced=false)

#From 4:2:2 planar 8bit to 4:2:2 interleaved 8bit
ConverttoYUY2()

#Ghost Removal
Ghostbuster(offset=3, strength=11)

#From 4:2:2 interleaved 8bit to 4:2:2 planar 8bit
Converttoyv16(matrix="Rec601", interlaced=false)

#De-grain
super = MSuper(pel=2, sharp=1)
bv1 = MAnalyse(super, isb = true, delta = 1, overlap=4)
fv1 = MAnalyse(super, isb = false, delta = 1, overlap=4)
bv2 = MAnalyse(super, isb = true, delta = 2, overlap=4)
fv2 = MAnalyse(super, isb = false, delta = 2, overlap=4)
MDegrain2(super,bv1,fv1,bv2,fv2,thSADC=800, thSAD=800)

#Bring everything to 16bit planar
ConvertBits(bits=16)

#Convert to 4:2:2 planar 16bit
Converttoyuv422(matrix="Rec601")

#Denoise with 16bit planar precision
neo_dfttest(sigma=64, tbsize=1, Y=3, U=3, V=3, dither=0, opt=0)

#Adding Borders to get a 1.33 PB with 16bit planar precision
AddBorders(152, 0, 152, 0)

#Upscale to FULL HD with Spline64 + NNEDI and 16bit planar precision
nnedi3_rpow2(cshift="Spline64ResizeMT", rfactor=2, fwidth=1920, fheight=1080, nsize=4, nns=4, qual=1, etype=0, pscrn=2, threads=0, csresize=true, mpeg2=true, threads_rs=0, logicalCores_rs=true, MaxPhysCore_rs=true, SetAffinity_rs=false)

#From 16bit planar to 16bit interleaved
ConvertToDoubleWidth()

#Matrix Conversion from BT601 to BT709 with 16bit interleaved precision
Matrix(from=601, to=709, rg=1.0, gg=1.0, bg=1.0, a=16, b=235, ao=16, bo=235, bitdepth=16)

#From 16bit interleaved to 16bit planar
ConvertFromDoubleWidth()

#Dithering from 16bit planar to 8bit planar with the Floyd-Steinberg error diffusion
ConvertBits(bits=8, dither=1)

#Limiter TV Range 0.0 - 0.7V
Limiter(min_luma=16, max_luma=235, min_chroma=16, max_chroma=240)

#Loudness Correction
Normalize(0.22, show=false)

#Re-Interlace to 25i TFF PAL
assumeTFF()
separatefields()
selectevery(4,0,3)
weave()




Or perhaps I should use the script made by Fred Van der Putte or are they ok only for Super8? You know, I hate to ask you this and I hate not to be able to provide a sample, but, you know, there are so many different contents that focusing on one single content would be pointless and would do more harm than good. So the question is: what would you do if your boss asks you to setup something for an automatic remastering of several different archive tapes from the 80s and 90s?

scharfis_brain
17th July 2020, 10:14
Can you get yourself the transform-PAL-decoder?
It should completely suppress rainbowing.

To my knowledge the ld-decode project has it implemented in its software.
And the BBC runs it on custom hardware.

FranceBB
17th July 2020, 10:34
Can you get yourself the transform-PAL-decoder?
It should completely suppress rainbowing.

To my knowledge the ld-decode project has it implemented in its software.
And the BBC runs it on custom hardware.

I'm actually using a Blackmagic card as suggested by Derek Prestegard and I'm not planning to change it as it works great. Besides, it's SDI to SDI, so the dot crawl and raimbowning I'm seeing has been recorded several years ago for whatever reason.
That said, the workflow is already up and running, but I just wanna know how I should handle post-processing in Avisynth ''cause I'm not sure whether to go for a very basic detail-driven approach or for some heavy filter-chain. Anyway, it has to be done through Avisynth (I already took care of the Automatization part).

feisty2
17th July 2020, 11:14
your "remaster" script seems a bit too naïve, some quick and simple suggestions:
1) 4:2:2 is useless, just convert everything to 4:4:4, it is possible to obtain native 4:4:4 out of 4:2:2 or 4:2:0 thru (luma) guided resampling

2) upscaling to 1080p is a waste of space and also damages the quality

3) frequency domain denoisers (DFTTest, FFT3D, etc.) are extremely destructive to components without a sparse frequency representation (or in human language, high frequency components) and introduce this gross "greasy" look, these should not be used as general purpose denoising filters, instead, you should only denoise low frequency components with these filters (something like DFTTest(sstring = "0:64 0.5:12 0.75:0.1 1:0"))

4) sharpening should be strictly prohibited unless it increases the resolution (image looks "finer", more "fine grained" after sharpening), it is NOT possible to increase the resolution with most if not all naïve avisynth sharpening scripts, you get the opposite when you apply something like LSFMod, the image becomes even "coarser" and "bloated".

FranceBB
17th July 2020, 12:03
your "remaster" script seems a bit too naïve, some quick and simple suggestions:
1) 4:2:2 is useless, just convert everything to 4:4:4, it is possible to obtain native 4:4:4 out of 4:2:2 or 4:2:0 thru (luma) guided resampling


BetaCAM were native yv16. Besides, our FULL HD mezzanine files are stored as yv16 as well, so I can't really go to 4:4:4 'cause in the end, even if I do, I'm gonna have to go back to 4:2:2.



2) upscaling to 1080p is a waste of space and also damages the quality


I would very much like to have Spline64 + NNEDI to upscale to FULL HD than relying on Omneon Playout ports which are gonna do it badly on playback using a FastBicubic().



3) frequency domain denoisers (DFTTest, FFT3D, etc.) are extremely destructive to components without a sparse frequency representation (or in human language, high frequency components) and introduce this gross "greasy" look, these should not be used as general purpose denoising filters, instead, you should only denoise low frequency components with these filters (something like DFTTest(sstring = "0:64 0.5:12 0.75:0.1 1:0"))


I know, they're prone to make wax-face look and oil-painting look sometimes, but what choice do I have...?



4) sharpening should be strictly prohibited unless it increases the resolution (image looks "finer", more "fine grained" after sharpening), it is NOT possible to increase the resolution with most if not all naïve avisynth sharpening scripts, you get the opposite when you apply something like LSFMod, the image becomes even "coarser" and "bloated".

So you're more for a blurry approach? The resulting file will look extremely blurry if I apply denoise and I upscale...

ChaosKing
17th July 2020, 12:41
I liked the sharpening result by the *new* CAS filter https://github.com/HomeOfVapourSynthEvolution/VapourSynth-CAS
I would also try out denoising with bm3d https://github.com/HomeOfVapourSynthEvolution/VapourSynth-BM3D (maybe combine it with smdegrain, scroll doooown)

I know these are vapoursynth filters. But it is very easy to use both avs+vs together.

Dogway
17th July 2020, 13:39
You can run a pass with RT_Stats and find the min, max, average for levels correction. This can also work very fine to find out if the video is dark or bright so you can tune the x264 settings depending on which. I would also run a RT_Stats pass over a motion mask to check if the clip has a lot of motion or not, similar to what Netflix (https://netflixtechblog.com/per-title-encode-optimization-7e99442b62a2?gi=2e08e0333629)does for encoding their shows.


By the way where I can find the Matrix() function? I downloaded HDRTools but still get an error.
EDIT: nevermind, it's the HDRCore plugin.

feisty2
17th July 2020, 13:58
the point of "remaster" is to enhance the perceptual quality, so things that are no good to the perceptual quality should definitely be avoided.

1) there's nothing stopping you from storing a 4:4:4 digital file, it's just a file, in fact my remasters are simply stored as uncompressed fp32 bitstreams. the bitstream could be handily loaded by any memory map raw source filter then encoded to something like ProRes or H264 as needed, I don't understand your argument here.

2) I don't know what "Omneon Playout" is, but high quality renderers like madVR could do nnedi3 upscaling at realtime when the video is playing, if you have to upscale SD content to 1080p, at least go for something more computationally expensive so it'd be worth it, something like GANs (https://forum.doom9.org/showthread.php?t=176961).

3) NLMeans works reasonably well on high frequency components, you could merge the results of DFTTest and NLMeans with a bandpass filter to get the best of both.

# vaporsynth script

def bandpass(clip):
low_freq = core.dfttest.DFTTest(clip, slocation=[0,0, 0.48,0, 0.56,1024, 1,1024], ...)
high_freq = core.std.MakeDiff(clip, low_freq)
return low_freq, high_freq

dft = core.dfttest.DFTTest(clip, slocation=[0,16, 0.48,16, 0.56,0, 1,0])
nlm = core.knlm.KNLMeansCL(clip, a=8, h=2.4)
dft_low, _ = bandpass(dft)
_, nlm_high = bandpass(nlm)

clip = core.std.MergeDiff(dft_low, nlm_high)

as ChaosKing suggested, V-BM3D is also worth a shot.

4) "fine" (native sharpness) looks better than "blurry" looks better than "deliberately enhanced", when you apply (naïve) sharpening, the result doesn't really resemble the look of native sharpness, you get this ugly "fat" and "coarse" and "over enhanced" look that a pair of trained eyes could instantly tell. the whole thing looks very "cheap" and "dumb", maybe take a look at this thread (https://forum.doom9.org/showthread.php?t=173756) if you want more information about this part.

tormento
17th July 2020, 14:01
I'd do that job for free, just to have access to that terrific movie collection...

Dogway
17th July 2020, 14:15
4) "fine" (native sharpness) looks better than "blurry" looks better than "deliberately enhanced", when you apply (naïve) sharpening, the result doesn't really resemble the look of native sharpness, you get this ugly "fat" and "coarse" and "over enhanced" look that a pair of trained eyes could instantly tell. the whole thing looks very "cheap" and "dumb", maybe take a look at this thread (https://forum.doom9.org/showthread.php?t=173756) if you want more information about this part.

I was reading your finesharp approach and saw the deconvolution line but no matrix whatsoever. What kind of sharpening matrix and size do you use for that example (SD)?

Frank62
17th July 2020, 17:29
Will the lossless files be deleted after encoding?
If so, I'd suggest: Do absolutley nothing, instead of encoding interlaced with highest possible bitrate. Dont't filter for storage, only if you need final results for broadcasting.
If there are no direct plans to delete the lossless captures, in a few years maybe someone will suddenly have the idea to do so - storage of big files is more expensive.
During the last years I had to work more and more often with wrongly and bad deinterlaced, upscaled, too much filtered, a.s.o. materials. And very often they did not keep any not harmed copy of these old treasures, so, do the world a favour.

johnmeyer
17th July 2020, 18:29
My advice: forget about the restoration.

Reason: you are probably not doing it right (everyone's techniques get better over time), but more critical is that it will massively increase the time this takes. Also, a LOT of the material you are going to capture is not important and probably doesn't merit the effort.

Your goal instead should be to focus only on digitizing those assets, and do that while they can still be digitized. Capture them in the best format that is financially viable, i.e., without wasting space by choosing formats that don't provide visually perceptible improvements in quality.

If you really have 466,000 tapes, and each of them contains half an hour of content, then if you run ten VTR machines for eight hours a day it will take 582 weeks to capture them all. If you multiply that number by any amount, where that multiple represents the increase in time required by your restoration, the project will quickly exceed your remaining lifetime. Even if you capture 24/7, you are still looking at multiple years to get the job done, unless you can employ a larger number of tape players.

Oh, as far as film being fragile and worrying about handling it, don't get that worried. I have transferred film from 1928 and it was as supple as the day it came back from developing. Yes, I have also handled vinegar syndrome film, and it is a sorry mess.

If you are worried about hurting the film by touching it, just wear old-fashioned cotton work gloves. Nothing fancy is needed, although I can point you to some sites which specialize in materials used for film transfers.

BTW, 1987 is hardly "really old" in the videotape world. Videotape was invented by Ampex in the late 1950s. If you had early 1960s Quadruplex tape, then that would be "really old." I say this only because tapes from 1987 onward should be absolutely fine. I have transferred audio tape from 1948 (I have it sitting here in my office) and it did not shed, and I have transferred many consumer-grade VHS and Beta tapes from the late 1970s, also with no tape issues. The main shedding problems were with tape binders used on commercial tapes from the 1970s. It sounds like you don't have any of those.

Given that your tapes have been stored in absolutely idea conditions, you are unlikely to have very many tape issues, although there are always a few stinkers in any large lot.

FranceBB
18th July 2020, 01:10
1) there's nothing stopping you from storing a 4:4:4 digital file, it's just a file, in fact my remasters are simply stored as uncompressed fp32 bitstreams. the bitstream could be handily loaded by any memory map raw source filter then encoded to something like ProRes or H264 as needed, I don't understand your argument here.


I know but mezzanine files are meant to be able to be played back on the fly by video servers, playout ports and so on, so you can't encode in whatever format you want, otherwise it's a mess. In broadcasting companies everything is encoded with the very same codec, resolution, sampling, frame rate, bit depth, primaries, color matrix and GOP. In my case, it's yv16 so I'm not gonna encode it in yv24.



2) I don't know what "Omneon Playout" is, but high quality renderers like madVR could do nnedi3 upscaling at realtime when the video is playing, if you have to upscale SD content to 1080p


I don't expect you to know Omneon, EVS etc but just keep in mind that a playout port is essentially a video server which literally plays a masterfile which is encoded on the fly and then aired, so it's better to leave to it nothing to do other than playing the file as it is 'cause even upscaling is done poorly. For instance, have you ever wondered why when you turn on your TV and you see a FULL HD channel that is airing an old SD contents your TV is still saying "1080i"? Well, unless it's something special, nobody upscaled the masterfile, it's probably some playout port playing an MPEG-2 IMX or DV25 file and upscaling it on the fly with something like a fast Bilinear or a fast Bicubic. For instance you can route an SDI signal with an hardware matrix to an Evertz device that will upscale it for you but again it's done poorly. Even modern VTR can be used to upscale but it's done poorly again as it has to be in real time. I would totally go for Spline64 + NNEDI over any crappy hardware upscale. (And... No, you can't air in SD except on SD channels).



3) NLMeans works reasonably well on high frequency components, you could merge the results of DFTTest and NLMeans with a bandpass filter to get the best of both.
as ChaosKing suggested, V-BM3D is also worth a shot.


I'll take a look at them both. :)

I'd do that job for free, just to have access to that terrific movie collection...

Hahahahaha I know you would XD

Will the lossless files be deleted after encoding?
If so, I'd suggest: Do absolutley nothing, instead of encoding interlaced with highest possible bitrate. Dont't filter for storage, only if you need final results for broadcasting.
If there are no direct plans to delete the lossless captures, in a few years maybe someone will suddenly have the idea to do so - storage of big files is more expensive.
During the last years I had to work more and more often with wrongly and bad deinterlaced, upscaled, too much filtered, a.s.o. materials. And very often they did not keep any not harmed copy of these old treasures, so, do the world a favour.

I know what you mean. I wanted to archive the lossless files too but that's sadly not feasible. The original physical supports will be kept, though, so if someone wants to re-encode them in the future, he will be able to digitalize them again.
As to the basics, I'm definitely gonna get the simple things right: interlacing, color matrix, color primaries and loudness. That's not really a problem. Perhaps I should really just do those few things...

a LOT of the material you are going to capture is not important and probably doesn't merit the effort.


That's true, there are some inestimable treasures in there but also some contents that will probably not be aired ever again, but still the plan is to digitalize everything.


Your goal instead should be to focus only on digitizing those assets, and do that while they can still be digitized. Capture them in the best format that is financially viable, i.e., without wasting space by choosing formats that don't provide visually perceptible improvements in quality.


Absolutely, in fact guys from Production Metadata are making me a list of which contents I should digitalize first.



If you really have 466,000 tapes, and each of them contains half an hour of content, then if you run ten VTR machines for eight hours a day it will take 582 weeks to capture them all. If you multiply that number by any amount, where that multiple represents the increase in time required by your restoration, the project will quickly exceed your remaining lifetime. Even if you capture 24/7, you are still looking at multiple years to get the job done, unless you can employ a larger number of tape players.


Give or take, they're more than 450k and less than 500k, but some of them are duplicates 'cause people before me kept the original tape and the duplicated one. Our current infrastructure is made of 16 Sony VTR (8 Sony on one side + 8 Sony on the other side), but my colleagues like to joke about the fact that by the time I'll retire, the project will still be ongoing (and I'm young!) XD


Oh, as far as film being fragile and worrying about handling it, don't get that worried. I have transferred film from 1928 and it was as supple as the day it came back from developing. Yes, I have also handled vinegar syndrome film, and it is a sorry mess.
If you are worried about hurting the film by touching it, just wear old-fashioned cotton work gloves. Nothing fancy is needed, although I can point you to some sites which specialize in materials used for film transfers.


Oh, I see! You know, I never saw something like that before in my life as I'm young (I'm from the 90s generation) so I was really scared to handle those old things 'cause I thought they were going to break like nothing.



BTW, 1987 is hardly "really old" in the videotape world. Videotape was invented by Ampex in the late 1950s. If you had early 1960s Quadruplex tape, then that would be "really old." I say this only because tapes from 1987 onward should be absolutely fine. I have transferred audio tape from 1948 (I have it sitting here in my office) and it did not shed, and I have transferred many consumer-grade VHS and Beta tapes from the late 1970s, also with no tape issues. The main shedding problems were with tape binders used on commercial tapes from the 1970s. It sounds like you don't have any of those.

Given that your tapes have been stored in absolutely idea conditions, you are unlikely to have very many tape issues, although there are always a few stinkers in any large lot.

Yeah... Well... I wasn't even born in the 80s, so handling things from that era for me is handling something older than me, that's why I said "really old"... Like, it is from my perspective cause it's older than me xD
Anyway, it's funny that you named Ampex 'cause that's the brand I've seen on many tapes and that's the company that my colleagues mentioned a lot. They also told me that they were not sure whether tapes were still ok or not 'cause Ampex guaranteed each and every tape to last 15 years but about 36 years passed... So far so good, though.


As a side note, it's funny to read log sheets / recording sheets left 36 years ago by other broadcast operators and their notes on how they did what they did, where they inserted breaks, how they calibrated levels with the bars note reference and so on... I wonder if they ever thought that, 36 years later, someone was going to open the case, read their annotations and play the tape accordingly... :')

feisty2
18th July 2020, 01:35
then you should pick a more sophisticated upscaling algorithm than nnedi3, it's too "cheap" to be hard encoded.
GAN based models (https://forum.doom9.org/showthread.php?t=176961)produce results much closer to native HD than nnedi3

feisty2
18th July 2020, 01:48
I was reading your finesharp approach and saw the deconvolution line but no matrix whatsoever. What kind of sharpening matrix and size do you use for that example (SD)?

iirc, it was deconvolution with a circular PSF of 1 pixel radius. it's not unsharp masking (convolution) so there's no kernel matrix to begin with. the deconvolution filter in that thread was probably FQSharp

FranceBB
18th July 2020, 11:55
then you should pick a more sophisticated upscaling algorithm than nnedi3, it's too "cheap" to be hard encoded.
GAN based models (https://forum.doom9.org/showthread.php?t=176961)produce results much closer to native HD than nnedi3

Is there an Avisynth port? The whole workflow is based on Avisynth, it would be a pain to use vapoursynth. Does it work with vapoursource? Do I have to install vapoursynth to use vapoursource? If so, which version and is it gonna be x64? Am I gonna lose performances?

If you look here https://forum.doom9.org/showthread.php?t=176655 every block is either a BAT, an Avisynth script or an AutoIT code for the GUI. It would be a pain in the butt to integrate vapoursynth, besides I would also have to ask Steinar and other two people as I'm not the only developer and he's the main developer of the project. Lastly, I spent years from 2006 to 2020 working on my own with Avisynth, I'm not really ready to move to a totally unknown scripting language like python and vapoursynth.

ChaosKing
18th July 2020, 12:41
Is there an Avisynth port? The whole workflow is based on Avisynth, it would be a pain to use vapoursynth. Does it work with vapoursource? Do I have to install vapoursynth to use vapoursource? If so, which version and is it gonna be x64? Am I gonna lose performances?


There is no avs port.

Why don't you just try out VS? I knew nothing about VS and switched completely to VS after using it for about 3 days. Python is very easy to understand (and to learn) even if you never used it before. You just need to know the absolute basics to write/untderstand a script.

Vapoursource should be able to load any VS script in avs. If you're using avs-64 you will need also VS-64. As an alternative you could use "avfs.exe" to create a fake avi and open it in any program or with avisource in AVS-32 or 64. Yes you need to install VS. Performance should not be affected, maybe a tiny bit but not by much.

Frank62
18th July 2020, 12:53
I know but mezzanine files are meant to be able to be played back on the fly by video servers, playout ports and so on, so you can't encode in whatever format you want, otherwise it's a mess. In broadcasting companies everything is encoded with the very same codec, resolution, sampling, frame rate, bit depth, primaries, color matrix and GOP.
You mean, you will HAVE TO encode in HD?


Even modern VTR can be used to upscale but it's done poorly again as it has to be in real time. I would totally go for Spline64 + NNEDI over any crappy hardware upscale. (And... No, you can't air in SD except on SD channels).

Of course, but the ugliest thing about these upscales is not that it has to be made in real-time. In 99% it is bad handling of interlaced material. Bicubic, lanczos and even spline resizing would be fast enough to upscale in real-time.


The original physical supports will be kept, though, so if someone wants to re-encode them in the future, he will be able to digitalize them again.

If possible to copy them without errors then. We just made a series of which three different physical stored copies were corrupt, in almost each and every episode. We had to save it by mixing the parts that were ok. Otherwise the whole series (5 seasons) would have been lost. There is no error-free copy left.
The material of DigiBeta-cassettes of two of the copies began to dissolve.
Maybe it's the very(!) last chance to save these programmes. Be a hero! ;)

ChaosKing
18th July 2020, 13:07
And btw I tried mcbob (with parameters from link) and it looked very good https://forum.doom9.org/showthread.php?p=1917370#post1917370
I would compare it with qtgmc and see which one looks better

Boulder
18th July 2020, 13:54
There is no avs port.

Why don't you just try out VS? I knew nothing about VS and switched completely to VS after using it for about 3 days. Python is very easy to understand (and to learn) even if you never used it before. You just need to know the absolute basics to write/untderstand a script.

Vapoursource should be able to load any VS script in avs. If you're using avs-64 you will need also VS-64. As an alternative you could use "avfs.exe" to create a fake avi and open it in any program or with avisource in AVS-32 or 64. Yes you need to install VS. Performance should not be affected, maybe a tiny bit but not by much.

MVTools in Avisynth is a bit ahead of the Vapoursynth port. I use MVTools quite a lot so I switched back to AVS+ from Vapoursynth some time ago.

FranceBB
18th July 2020, 19:10
There is no avs port.

Vapoursource should be able to load any VS script in avs. If you're using avs-64 you will need also VS-64. Yes you need to install VS. Performance should not be affected, maybe a tiny bit but not by much.

Oh, ok. I'll try installing VapourSynth and use VapourSource then. If upscaling with this method is so much better than Spline64 + NNEDI then I'll give it a shot.

You mean, you will HAVE TO encode in HD?


Yep...


Of course, but the ugliest thing about these upscales is not that it has to be made in real-time. In 99% it is bad handling of interlaced material.


Well, I've seen all sort of things, but mostly it's either an extremely blurry image when using Fast Bilinear on hardware or ugly ringing and aliasing spread everywhere when using Fast Bicubic on hardware, which is why I said that I would choose Spline64 + NNEDI over literally any real time hardware upscale.


If possible to copy them without errors then. We just made a series of which three different physical stored copies were corrupt, in almost each and every episode. We had to save it by mixing the parts that were ok. Otherwise the whole series (5 seasons) would have been lost. There is no error-free copy left.
The material of DigiBeta-cassettes of two of the copies began to dissolve.
Maybe it's the very(!) last chance to save these programmes. Be a hero! ;)

Well, yeah... that's what happens as time passes... BetaCAM are supposed to last 15 years. Ours are 36 years old and they are still perfectly fine, so I think it's a mix of luck and preservation (i.e how they've been preserved). Luckily ours have been kept undeground (-2 floor), without lights, at constant temperature and humidity, so I don't expect them to fade, but yeah, in the long run, that's a possibility. Anyhow, I think I'm just gonna go with bobbing, Upscale in the best possible way (I was thinking Spline64 + NNEDI but now I'm gonna play with other Neural Network upscalers as well), matrix/primaries conversion, safe color limiter, loudness correction and proper re-interlace so that in the future they're gonna be ok even if someone wants to work with them again (even if I'm gonna encode them in lossy).

feisty2
18th July 2020, 19:56
Oh, ok. I'll try installing VapourSynth and use VapourSource then. If upscaling with this method is so much better than Spline64 + NNEDI then I'll give it a shot.



NTSC master
https://i.imgur.com/Qb2NTEm.png

nnedi3
https://i.imgur.com/RD3yuqP.png

esrgan + post processing
https://i.imgur.com/IoZDWUk.png






need I say more?

jpsdr
18th July 2020, 20:42
NNEDI3 gives better grain skin, so it provides indeed better result (at least, from my point of view), so, personnaly, i'll choose NNEDI3.

feisty2
18th July 2020, 20:49
and you know it's cheap upscaled image at first sight when you see the result produced by nnedi3, while the esrgan one looks half convincing that it's native HD

I would pick the native HD look over "grain skin" or other insignificant trivial detail any day of the week

edit: also, the loss of skin detail was caused by post processing, not esrgan, the raw output of esrgan is here

esrgan
https://i.imgur.com/NuZ2r56.png





the post processing aims to further reduce the gap between native HD and the result produced by esrgan, it costs some mid-frequency detail, but further enhances the native HD look.

real.finder
18th July 2020, 21:15
both are far from real HD, and each one has different bad points in terms of quality

In my opinion, NNEDI3 is reasonable to use it over esrgan since it faster and available for more people and easer to use

also I considered moving to VS years ago (when ultim disappear before pinterf contributions in avs+) but I didn't see any thing worth to bother with, especially that vs need python and every script has to start with "from blah import core", "import blah as blaaaah" and in the end "blah.output(sys.stdout)" then I said no thanks! avs is better for me even if it need more development, which is currently happening in avs+, not to mention easy loading any avs version (with just Avisynth.dll) with any tool, and easy load encoding script in ffmpeg/x264

I think it's better for both to make the plugins work in both just like LSMASHSource, I don't like how vs plugins developer made the plugin work only for vs while they take/port many plugins and codes from avs, they even made the vs load avs/avs+ plugins and they even make something like mpp that load avs script inside vs, I think it's not fair after all this to make new/(updated ports) plugins work only for vs

Cary Knoop
18th July 2020, 21:17
Couple of comments:

Deinterlacing

QTGMC is great but it does a lot of denoising that could be better done in a separte process.
Rather than using a preset I would make a custom setting (for instance one that that includes things like SourceMatch=3, Lossless=2)
Lossless is important because you can undo the operation (but not the heavy QTGMC default denoising).

Chroma upsampling
Simple chroma upsampling does notthing to improve the quality, it is a rather destructive process.
Only now more intelligent chroma upsampling methods are becoming available, until you use those methods upsampling the chroma only destroys any future potential for improvement.

Auto levels, limites, white balance, etc.
I would completely stay away from this. Also be aware that many sources could have superwhite information, you should not just bring that in using ProAmp.
Superwhites have an overflow function and should be rolled off instead.

Denoise
I would only use V-BM3D or even higher quality neural based stuff.
Ideally you would want to do some denoising before deinterlacing but that may not be possible as it depends on how the video source macroblocks were encoded.

Cropping
This should always come at the end of your workflow and never before denoising.

Resizing
nnedi3_rpow2 is great for animation, less great for real-life footage.
I would wait untill the mehods get better.
Some neural based resizing is promising but there are still issues: you would need to scene detect your source and split them and also most neural based scaling is based on individual images which for video gives continuation problems. You might want to take a look at TecoGAN that tries to avoid this issue.

Dithering
I would not use Floyd-Steinberg, admittedly it is a very good one but it will only create compression problems for H.264/265 encodings. Use an alternative, a simpler one.

Cary Knoop
18th July 2020, 21:26
NTSC master
https://i.imgur.com/Qb2NTEm.png

That's is an NTSC master as in real NTSC SD video?
If so then that is an extremely good master.

feisty2
18th July 2020, 21:41
That's is an NTSC master as in real NTSC SD video?
If so then that is an extremely good master.

it's a master file of NTSC format (SMPTE D10), but the video itself was not shot by an NTSC camera (I mean the broadcast camera that outputs 60i analog or digital videos). the video (and probably most music videos before Arri Alexa replaced film cameras in the movie industry) was shot by cinema level film cameras like Arriflex 435 or 235 then produced in SD format, that's why the quality of the master far exceeds the quality of "normal" NTSC videos.

feisty2
18th July 2020, 22:19
In my opinion, NNEDI3 is reasonable to use it over esrgan since it faster and available for more people and easer to use



nnedi3 is reasonable to use for certain intermediate processing that requires supersampling, this includes:
a) interpolation for subpixel precision filters like MSuper and aWarp4
b) realtime upscaling at playback, this is what madVR does
in such cases it is a very good replacement for traditional resampling methods like bicubic or spline

however, it is unfit to be used as a general purpose upscaling filter in 2020. and by general purpose, I mean to enlarge the image while preserving the perceptual quality as much as possible.

Frank62
18th July 2020, 22:44
Anyhow, I think I'm just gonna go with bobbing, Upscale in the best possible way (I was thinking Spline64 + NNEDI but now I'm gonna play with other Neural Network upscalers as well), matrix/primaries conversion, safe color limiter, loudness correction and proper re-interlace so that in the future they're gonna be ok even if someone wants to work with them again (even if I'm gonna encode them in lossy).


This will be irreversible (if interlaced material). Take the best bobber there is, scale it up with the best AI filter there is and reinterlace - fields won't fit 100%. And if you downsacale later you will get aliasing interlace artefacts, that one will have to correct - while losing resolution and quality.
But I understand that you have to do so. What a pity.

Cary Knoop
18th July 2020, 23:11
This will be irreversible (if interlaced material). Take the best bobber there is, scale it up with the best AI filter there is and reinterlace - fields won't fit 100%. And if you downsacale later you will get aliasing interlace artefacts, that one will have to correct - while losing resolution and quality.
But I understand that you have to do so. What a pity.
I do not see the problem, you can deinterlace losslessly.

FranceBB
19th July 2020, 00:48
Couple of comments:

Deinterlacing

QTGMC is great but it does a lot of denoising that could be better done in a separte process.
Rather than using a preset I would make a custom setting (for instance one that that includes things like SourceMatch=3, Lossless=2)
Lossless is important because you can undo the operation (but not the heavy QTGMC default denoising).


Thanks for this bit. I will definitely use SourceMatch=3, Lossless=2 to avoid the default denoise. :)


Chroma upsampling
Simple chroma upsampling does notthing to improve the quality, it is a rather destructive process.
Only now more intelligent chroma upsampling methods are becoming available, until you use those methods upsampling the chroma only destroys any future potential for improvement.


Well, BetaCAM are supposed to be native in yv16() and I'm encoding in yv16() so I think I'm gonna be fine.


Auto levels, limites, white balance, etc.
I would completely stay away from this. Also be aware that many sources could have superwhite information, you should not just bring that in using ProAmp.
Superwhites have an overflow function and should be rolled off instead.


Ok, perfect, so no auto levels and... are you saying I should just be clipping the so called super-blacks / super-whites (i.e everything below 16 and above 235 / 0.7V)? Just as a "history lesson", I know why we have to air in Limited TV Range, but why are BetaCAM super whites bad and should be clipped out rather than brought down to legal values? I mean, it's just curiosity, clipping them for me is far easier than bringing them down.


Denoise
I would only use V-BM3D or even higher quality neural based stuff.
Ideally you would want to do some denoising before deinterlacing but that may not be possible as it depends on how the video source macroblocks were encoded.


I wouldn't do anything before deinterlacing because I'm not sure fields are 100% ok and the last thing I want would be screwing up fields with some kind of post-processing so that I'm not gonna be able to properly bob-deinterlace. Anyway, I'll definitely take a look at BM3D.


Resizing
nnedi3_rpow2 is great for animation, less great for real-life footage.
I would wait untill the mehods get better.
Some neural based resizing is promising but there are still issues: you would need to scene detect your source and split them and also most neural based scaling is based on individual images which for video gives continuation problems. You might want to take a look at TecoGAN that tries to avoid this issue.


Oh, I see... so it was trained for animation contents...
As to other neural networks, I was looking at VSGAN as suggested by Feisty (https://github.com/rlaPHOENiX/VSGAN) but I just noticed that it says "VapourSynth Single Image Super-Resolution Generative Adversarial Network (GAN)", so it's for still pictures, not frame of a video 'cause it can lead to correlation issues introduced during the upscaling process...
I wouldn't want to upscale, but since it has to be done in a way or another, I would totally go for something safe like NNEDI rather than risk errors or artifacts using a frameserver I don't know (vapoursynth), which is used in a frameserver I know (Avisynth) using vapoursource, and using on frames of a video a neural network I don't know (VSGAN) even if it's meant for still images and all of this in production. Like... I'm young, but I'm not mad...
I guess I'm gonna stick with Spline64 and NNEDI.



Dithering
I would not use Floyd-Steinberg, admittedly it is a very good one but it will only create compression problems for H.264/265 encodings. Use an alternative, a simpler one.

Yeah, well, everything has to stay within 50 Mbit/s which should be enough considering that it's something upscaled from an SD footage and with 1.33 PB black bars which will save bitrate, but still, if you say that it might still be too complex to be encoded properly, I'll probably use something different, but which one? Atkinson? Stucki? Ordered dithering?

both are far from real HD, and each one has different bad points in terms of quality

In my opinion, NNEDI3 is reasonable to use it over esrgan since it faster and available for more people and easer to use


Yes, they're far from real HD footages as they're always gonna be upscales, but I agree with you, NNEDI3 isn't bad, honestly. Even if Feisty will be disappointed, I think I'm gonna stick with a Spline64 + NNEDI approach.

nnedi3 is reasonable to use for certain intermediate processing that requires supersampling, this includes:
a) interpolation for subpixel precision filters like MSuper and aWarp4
b) realtime upscaling at playback, this is what madVR does
in such cases it is a very good replacement for traditional resampling methods like bicubic or spline

however, it is unfit to be used as a general purpose upscaling filter in 2020. and by general purpose, I mean to enlarge the image while preserving the perceptual quality as much as possible.

I disagree...
Look, I know you're very passionate about those things, but until those new methods will be established to be working fine, reliably, without correlation issues between one frame and the other and in Avisynth, I'll stick with Spline64 + NNEDI3 with 16bit planar precision, which have been proven to be working fine for ages and in my opinion, they're not so bad either... I mean, it's still gonna be way better than any other classic upscaling method, so...

This will be irreversible (if interlaced material). Take the best bobber there is, scale it up with the best AI filter there is and reinterlace - fields won't fit 100%. And if you downsacale later you will get aliasing interlace artefacts, that one will have to correct - while losing resolution and quality.
But I understand that you have to do so. What a pity.

What do you mean? If you "must" you could bob-deinterlace it again, use a reverse upscale algorithm like DeBilinear, DeBicubic, DeLanczos, DeSpline and so on to bring it back to the original resolution and re-interlace it. It can be undone with little to no damage as long as I stick with it. :)
(the only "pity" is that I have to encode it in lossy, but I hope it's not gonna be too lossy, let's say...)

I do not see the problem

Precisely. :)

feisty2
19th July 2020, 01:04
use nnedi3, don't use nnedi3, I don't even care, it's not my videos

but, I can't decide if you guys are trolling or viewing pics on a flip phone or there's something wrong with your display device, as I find it simply incomprehensible to say
https://i.imgur.com/fRHSA0s.png
looks not bad compared to
https://i.imgur.com/KBRPIF5.png

I have no words

Cary Knoop
19th July 2020, 02:32
Thanks for this bit. I will definitely use SourceMatch=3, Lossless=2 to avoid the default denoise. :)

You might want to set NoiseProcess as well.

Ok, perfect, so no auto levels and... are you saying I should just be clipping the so called super-blacks / super-whites (i.e everything below 16 and above 235 / 0.7V)? Just as a "history lesson", I know why we have to air in Limited TV Range, but why are BetaCAM super whites bad and should be clipped out rather than brought down to legal values? I mean, it's just curiosity, clipping them for me is far easier than bringing them down.

Clipping is losing information so that's no good. On the other hand super whites are really overflow values, most of the time it will not recover anything dramatic.
Bringing them in using procamp will mess up your gamma, a roll-off (i.e. s-curve them up to 235) should be best.
If you actually have sub-blacks you might want to check if there are no 7.5 IRE setup related issues from your analogue source, sub-blacks should normally not happen.


I wouldn't do anything before deinterlacing because I'm not sure fields are 100% ok and the last thing I want would be screwing up fields with some kind of post-processing so that I'm not gonna be able to properly bob-deinterlace.

Yes, denoising pre-deinterlace it's definitely more work and sometimes it is bad due to the encoding. Denoise after is ok, it's not a day and a night difference anyway.

Ordered dithering should be more palatable for H.264/265.

Boulder
19th July 2020, 09:23
Thanks for this bit. I will definitely use SourceMatch=3, Lossless=2 to avoid the default denoise. :)

I recommend adding also tr2=0. It will skip the final degraining part and it's my go-to setting whenever I use QTGMC.

FranceBB
19th July 2020, 10:06
I recommend adding also tr2=0. It will skip the final degraining part and it's my go-to setting whenever I use QTGMC.

Oh, got it. I'll add it as well. :)

real.finder
19th July 2020, 10:41
I recommend adding also tr2=0. It will skip the final degraining part and it's my go-to setting whenever I use QTGMC.

Oh, got it. I'll add it as well. :)

in this case you may want to try mcbob as ChaosKing said

And btw I tried mcbob (with parameters from link) and it looked very good https://forum.doom9.org/showthread.php?p=1917370#post1917370
I would compare it with qtgmc and see which one looks better

mcbob was made by Didée in same era when he made TempGaussMC which is became QTGMC later by Vit

also, in the code that ChaosKing mention smam use QTGMC, it's use the best of things as I said https://forum.doom9.org/showthread.php?p=1917316#post1917316

note: mcbob is preserves the original fields, but smam make that not work since it use QTGMC and some others, so if you want preserves the original fields then use it without smam

Frank62
19th July 2020, 12:13
...
use a reverse upscale algorithm like DeBilinear, DeBicubic, DeLanczos, DeSpline and so on to bring it back to the original resolution and re-interlace it. It can be undone with little to no damage as long as I stick with it. :)


Try it.

Dogway
19th July 2020, 13:38
but, I can't decide if you guys are trolling or viewing pics on a flip phone or there's something wrong with your display device...

Didn't know esrgan was in VS. I'll give it a try when it's ported to AVS+ although most work I do is on animation.

johnmeyer
19th July 2020, 16:01
I do not see the problem, you can deinterlace losslessly.There is no such thing as "lossless" deinterlacing. The act of deinterlacing ALWAYS degrades the video. The reason is that you have to create fields that were not there in the original video. Whether you estimate, or duplicate, or blend, the resulting new fields are never perfect and you therefore will always end up with something that is worse than the original.

It can't be helped.

I always recommend to NEVER deinterlace, unless you are doing an operation, like resizing, where you have to do it. The final display device can deinterlace and, because the device's deinterlacer is tied to its hardware, it usually does a better job. Also, there is absolutely nothing which says that display devices will forever be incapable of displaying interlaced material without deinterlacing.

Finally, it is one more thing that has to be done to the video, and with a project of this magnitude, every percent added to the time it takes to finish a tape pushes the finish date for the project out not just by days or weeks, but by months or years.

I will repeat my recommendation: just do the digitizing, and forget all restoration until it comes time to broadcast the material.

Cary Knoop
19th July 2020, 16:45
There is no such thing as "lossless"
There is, a lossless deinterlacing operation merely reconstructs the missing fields into full frames without touching the original fields.

If you want to undo the operation you simply discard the reconstructed fields and you have the original back.

This is pretty basic.

The act of deinterlacing ALWAYS degrades the video.
Only if you don't know what you are doing.

By the way, it is not true that lossless deinterlacing neccesarily gives the best results.
This is expecially in cases where the "scanlines" are not stable.

real.finder
19th July 2020, 17:30
There is, a lossless deinterlacing operation merely reconstructs the missing fields into a full frames without touching the original fields.

If you want to undo the operation you simply discard the reconstructed fields and you have the original back.

This is pretty basic.


Only if you don't know what you are doing.

true

colorbars()
converttoyv16
Bob(0.0, 1.0).interlaced60or50 (https://github.com/realfinder/AVS-Stuff/raw/master/avs%202.5%20and%20up/Zs_RF_Shared.avsi)(BFF=!GetParity()).Compare(last)

you can use any other than Bob(0.0, 1.0) for the test

but I agree with johnmeyer in case of not do restoration or do restoration but keep the digitizing copy some where

Frank62
19th July 2020, 23:12
true

True, if you do nothing than deinterlace, and then revert.

False, if you

-deinterlace
-resize (which is unavoidable in this case)
-compress, even with a high bitrate

and THEN try to revert this process. This will do damage, and will leave interlace artefacts in the end. As I said: Just try it and look at the results.

@FranceBB
Question: Can De-Bilinear, De-Bicubic a. s. o. be 100% perfect in reconstructing, if you handle the first resizing in respect of later reverting? I am not sure about this.

FranceBB
15th August 2020, 13:04
In case you were wondering, the whole thing is proceeding just fine.
So far, I've been encoding more than 108 tapes and I've been fixing issues manually with very pleasing results, especially in getting rid of spots, scratches, noise, grain, raimbowning, dot crawl, chroma bleeding and other issues. I mean, the whole thing is working just fine and Avisynth is filtering out pretty much everything. :D

Anyway... the interesting thing is that I never watched the events recorded on those Oxyde BetaCAM or 1 inch Type C tapes or film that I'm capturing and I found some really interesting things and videos of very old athletes and competition which I never saw before!
I watched Coppi and Bartoli on a bicycle, Joe Di Maggio playing baseball, Ingemar Stenmark skiing, John McEnroe, Björn Borg, Jimmy Connors and Chris Evert playing tennis, Magic Johnson playing basketball, O. J. Simpson running for 2000 yards in a single season, Ayrton Senna and Niki Lauda driving in F1, Michel Platini, Pelé and Karl-Heinz Rummenigge playing soccer... I mean c'mon, now that I'm watching 'em all playing, I'm astonished by the quality of those people and what they did in their respective fields... Just wow... :eek:
Oh, for the records, those were mostly on 1 inch Type C or Oxyde BetaCAM or other supports

https://i.imgur.com/1rprjI7.png
https://i.imgur.com/4S2gyNk.png
https://i.imgur.com/YQxh61u.png

while I found movies only on film, like this one:

https://i.imgur.com/GOCwv6M.png


All in all I gotta say that I'm kinda enjoying this whole Remastering process as everything I'm seeing is brand new for me and I'm watching people that made history... for real...


A few side note:

Side note 1: I also found some footages from as early as 1938, like the soccer world cup final, but unfortunately they're analog copies done in the 70s of the real reel/tape/whatever it was recorded on (I don't know what was used in the 30s) and I have no idea about what happened to the original... :(

Side note 2: Women didn't shave their armpits and that's gross...

ajp_anton
23rd August 2020, 21:52
Side note 2: Women didn't shave their armpits and that's gross...
Eh... what? And I didn't realize men used to shave their armpits while women didn't.

Cary Knoop
23rd August 2020, 22:03
Side note 1: I also found some footages from as early as 1938, like the soccer world cup final, but unfortunately they're analog copies done in the 70s of the real reel/tape/whatever it was recorded on (I don't know what was used in the 30s)
That could no bee anything else than film.

You might want to see how it was telecined (increased speed to 25p, blending, or frame doubling).

FranceBB
24th August 2020, 01:39
That could no bee anything else than film.

You might want to see how it was telecined (increased speed to 25p, blending, or frame doubling).

It was speeded up a lot to 25p. Right now I'm saving it as v210 as it is, but I'm planning to slow it down to the original framerate and then blend it to 50p and divide it in fields to 25i to create my mezzanine. What's the FPS for those very old materials? 16fps like Super8? 'cause for being at 25p they're extremely fast so it must have been a big jump... I mean, it definitely wasn't 23.976...
Tomorrow I'll make a few tests to find out.

johnmeyer
24th August 2020, 01:47
It was speeded up a lot to 25p. Right now I'm saving it as v210 as it is, but I'm planning to slow it down to the original framerate and then blend it to 50p and divide it in fields to 25i to create my mezzanine. What's the FPS for those very old materials? 16fps like Super8? 'cause for being at 25p they're extremely fast so it must have been a big jump... I mean, it definitely wasn't 23.976...You are not making much sense. I hope you have access to better information if you really are dealing with first generation material and are doing the restoration yourself.

First of all 16mm and 35 mm sound film is 24 fps.

Second, when sound film is transferred to PAL, for broadcast in Europe, it is often simply sped up to 25 fps, sometimes pitch-corrected for the sound, sometimes not. Most people would not consider this "speeding up a lot."

Third, silent film can be any speed. Really old stuff was hand-cranked, so there was no standard. I have transferred 1928 film which I timed to play at 12 fps, and even that may have been too fast. You have to look at the date on the can, if any, to determine what speed is going to work best. Most silent 16mm film from the 30s and 40s is 16 fps, as is most 8mm film.

However, Super 8 film is always 18 fps, NOT 16 fps. That actually IS a standard (like sound film at 24 fps), and there is no exception. Hopefully you know the difference between 8mm and Super 8. They have the same width, but sprocket and frame size are totally different. The two standards are actually quite different.

As for "blending it to 50p and dividing it into fields to 25i" I have no clue what you mean by that, but it doesn't sound like the right thing to do, at all.

Just transfer the film, save it as progressive, and time it to play at 16, 18, or 24 fps (for most film). Don't do anything else until and unless you have to broadcast it.

Cary Knoop
24th August 2020, 02:05
It was speeded up a lot to 25p. Right now I'm saving it as v210 as it is, but I'm planning to slow it down to the original framerate and then blend it to 50p and divide it in fields to 25i to create my mezzanine. What's the FPS for those very old materials? 16fps like Super8? 'cause for being at 25p they're extremely fast so it must have been a big jump... I mean, it definitely wasn't 23.976...
Tomorrow I'll make a few tests to find out.

Hold on, blending? No, please! There ought to be a law! :p

For 16 and 18 I would keep that at the original (or cadence that at 50/60p as an intermedia (easy to undo). Optical flow is just too hard at those slow framerates and is to be avoided.

By the way, with respect to creating a digital master I would capture with a suitable bitdepth and apart from setting up the proper procamp level to ensure nothing is captured out of range I would do nothing else.
For a playable version I would create a copy that is properly repaired, denoised and deinterlaced to 50p/60p unless it is film-based.

Here is an example of a 16p film played at 60p using a AAA,BBBB,CCCC,DDDD cadence:

https://www.youtube.com/watch?v=YHYaYOMc4eo

StainlessS
24th August 2020, 02:09
Much to learn from there John, many thanks. [even though I am unlikely to remember the framerates of each, I can always ask you :) ].