Deep Space Nine upscale project

[ryrynz]

I recently saw this post (https://www.extremetech.com/extreme/308505-star-trek-deep-space-9-upscale-project-update) over at Extremetech.com and wondered if there was any video editors or upscale/denoiser gurus that might help the author out as he's a bit of an Avisynth noob and is after some pointers to make this upscale even better.

[Katie Boundary]

I sent him an email saying I could help with the deinterlacing part.

[Stereodude]

I sent him an email saying I could help with the deinterlacing part.
Hopefully he ignores it as nothing good will come from it. :p

[Stereodude]

I recently saw this post (https://www.extremetech.com/extreme/308505-star-trek-deep-space-9-upscale-project-update) over at Extremetech.com and wondered if there was any video editors or upscale/denoiser gurus that might help the author out as he's a bit of an Avisynth noob and is after some pointers to make this upscale even better.
The upscale portion of his process isn't his problem. It's what he's feeding into it.

He has no clear stated goal. Is he trying to make 24p? 30p? 60p? VFR?

My understanding is that DS9 has telecined 24p and interlaced video mixed together in frames (where VFX is used). I don't know how you can fix that correctly unless you use masks & treat each portion separately which would be totally impractical because the mask would have to change on a per frame basis. Even then, what do you with the two halves to combine them? Only 120Hz video can contain both without judder or blending on either.

His audio problems make no sense. QTGMC doesn't change the duration of the video. It just doubles the frame rate. And, in my experience QTGMC will shimmer when used on telecined content, which is what he sees. It does do some noise reduction (not as good as MCTD) and it also sharpens the output (unless you explicitly dial it back).

[wonkey_monkey]

The telecined 24p should include all fields at least once so QTGMC should be unnecessary/harmful for those parts (as Stereodude points out it causes shimmer).

The PAL DVDs should be higher resolution (the first two seasons of TNG were converted from NTSC to PAL, after that they were done properly), although the video/effects parts may suffer from the DEFT(?) conversion.

[Stereodude]

The telecined 24p should include all fields at least once so QTGMC should be unnecessary/harmful for those parts.
Yes, but it's a mess. You can be in the middle of a telecined 24p sequence when all of a sudden some VFX work (which is i60) is composited into the scene for a few seconds. How do you handle that? Engage QTGMC for a few seconds, then turn it back off? Then you've got a 24p section, a 60p section, and a 24p section.

[huhn]

if it is 24p and 60p maybe 30p is nothing new. stuff like this is released even to day.

VFR encode is clearly do able.
it did this as a test about 10 years ago with a 24p 30p mix source which is pretty easily in comparison because a deinterlancer wasn't needed.

not sure if it was http://www.avisynth.nl/index.php/ExactDedup or just dedup but the trick was 120 HZ and by simply manually multiplying every scene and running the filter over it with a zero tolerance.

[StainlessS]

For ExactDeDup, dupe frames have to be bit identical, so probably not work as required on previously compressed source.

EDIT: Maybe I have misunderstood, not sure (no sleep again, brain not working at all). (Will work if newly artifically created dupes [EDIT: ExactDedup will work, not the brain, brain is beyond help ])

[SaurusX]

I applaud the person trying to tackle this project, but I fear he doesn't know enough to actually accomplish his goal. He talks about Handbrake and QTGMC at the same time, which makes me think he's just flopping about. If I were to do this I would inverse telecine and have a fallback video stream from QTGMC to use if any residual interlacing was detected. TFM() could easily accomplish this and I use a similar approach all the time. I think trying to do a variable frame rate encode based on visual effects and scene cuts is just setting yourself up for project burnout. After all, Paramount remastered TNG for blu-ray ending up with a solid 23.976fps all the way through. So... Do a lossless run to obtain your 23.976fps "master" and then run that through the Topaz software. That's as close as you can get without access to real studio masters.

[wonkey_monkey]

After all, Paramount remastered TNG for blu-ray ending up with a solid 23.976fps all the way through.

Didn't they redo all the video effects shots?

[Stereodude]

Didn't they redo all the video effects shots?
Yes. https://trekmovie.com/trek-remastered/tng-remastered/

[SaurusX]

Yes. https://trekmovie.com/trek-remastered/tng-remastered/

Unfortunately, we hobbyists don't have that option. So, the TNG 60i effects were redone to 23.976p. Our best option is then to deinterlace the 60i areas and choose specific frames, or more precisely, let TFM do it for us.

[Stereodude]

Our best option is then to deinterlace the 60i areas and choose specific frames, or more precisely, let TFM do it for us.
And what is the output framerate?

What does your proposed solution look like from a script standpoint?

[Katie Boundary]

My understanding is that DS9 has telecined 24p and interlaced video mixed together in frames (where VFX is used). I don't know how you can fix that correctly unless you use masks & treat each portion separately which would be totally impractical because the mask would have to change on a per frame basis.

Yes, but it's a mess. You can be in the middle of a telecined 24p sequence when all of a sudden some VFX work (which is i60) is composited into the scene for a few seconds. How do you handle that? Engage QTGMC for a few seconds, then turn it back off?

I solved that problem years ago:

mpeg2source("212.d2v")

Y=nnedi3(field=-2)

A=yadifmod2(mode=1,edeint=Y).selecteven()
B=yadifmod2(mode=1,edeint=Y).selectodd()
C=Tfm(field=1,mode=0,cthresh=2,mthresh=2,clip2=A,micmatching=0)
D=Tfm(field=0,mode=0,cthresh=2,mthresh=2,clip2=B,micmatching=0)
Interleave(C,D)

Tweak other parameters to taste.

I applaud the person trying to tackle this project, but I fear he doesn't know enough to actually accomplish his goal. He talks about Handbrake and QTGMC at the same time, which makes me think he's just flopping about

He admitted to being an AVIsynth noob.

Unfortunately, we hobbyists don't have that option. So, the TNG 60i effects were redone to 23.976p. Our best option is then to deinterlace the 60i areas and choose specific frames, or more precisely, let TFM do it for us.

Or just bob to p60 and leave it at that.

[wonkey_monkey]

I solved that problem years ago:

You did not "solve the problem." You patched it up to your satisfaction, which is a very different thing.

[Stereodude]

You did not "solve the problem." You patched it up to your satisfaction, which is a very different thing.
I'm not even sure what that script's going to give you as the output. I'm not sure I'm curious enough to try it and see.

I mean I get the output for i60 input will be 60fps and shouldn't have combing, but beyond that I have no idea how the p24 content will be represented in the output.

I'm almost tempted to go get a DS9 DVD. Almost...

[huhn]

not sure what's so hard about this topic VFR encoding are made for like ever.

VFR is one way to get the correct frame rate for all parts the other is 120 HZ which is not an optimal choice for obvious reasons.

time didn't stop:
i never used this way to archive VFR but there are simply better people to answer how to do it but it's even an example for http://avisynth.nl/index.php/TIVTC

it can't be to hard to automatically deint the parts that are detected as hybrid in stead of treating them as PsF.

[Stereodude]

not sure what's so hard about this topic VFR encoding are made for like ever.

VFR is one way to get the correct frame rate for all parts the other is 120 HZ which is not an optimal choice for obvious reasons.

time didn't stop:
i never used this way to archive VFR but there are simply better people to answer how to do it but it's even an example for http://avisynth.nl/index.php/TIVTC

it can't be to hard to automatically deint the parts that are detected as hybrid in stead of treating them as PsF.
It's hard because both types of content are in the exact same frame. It's not some i60 and some telecined p24 content and it switches on a scene by scene or shot by shot basis.

They shot it on film, telecined the film to i60 and then composited into that i60 special effects.

[huhn]

sorry still don't get the issue or what's unusually about that.

i know that the mixed part will have some motion issues that's the creators intent. so motion adaptive deint should be the way to go for that part.

[Katie Boundary]

I'm not even sure what that script's going to give you as the output. I'm not sure I'm curious enough to try it and see.

I mean I get the output for i60 input will be 60fps and shouldn't have combing, but beyond that I have no idea how the p24 content will be represented in the output.

It looks a lot like it would if you ran it through QTGMC, except better because the fields are properly matched and then left the hell alone instead of bobbed and then run through a dozen rounds of blurring and sharpening.

[bladerunner1982]

I also had this upscale project in mind for the past three years. (taking long breaks in between :D)
From time to time I experimented a little bit with different scripts and
parameters using the first episode of season 6 (the first 20 minutes).
Here is the script I came so far using the NTSC DVDs:


myvideo = "ep1ntsc.d2v"
DGDecode_mpeg2source(myvideo, idct=5)

#deint = QTGMC().SelectEven() #using QTGMC as deinterlacer optional
deint = tdeint()
tfm(d2v=myvideo,clip2=deint)
TDecimate(hybrid = 1)

QTGMC(InputType=1) #using QTGMC on progressive frames again to clean up a little bit
#QTGMC( Preset="Slower", InputType=1 ) #optional

tweak(cont=1.3,sat=1.1)

ConvertToRGB24()
SuperResXBR(Passes=5, Str=1, Soft=0.0, XbrStr=2.7, XbrSharp=1.3, MatrixIn="PC601")
SuperResXBR(Passes=3, Str=1, Soft=0.0, XbrStr=2.7, XbrSharp=1.3, MatrixIn="PC601", fWidth=1440, fHeight=1080, fKernel="SSim")
ConvertToYV12()

addgrainc()
trim(0,26000)+trim(49114,50000)
AssumeFPS(25)


If someone is interested I can provide a link to the encoded result via pm.
Maybe it´s not a pro solution but for me it was ok.
I´m always open for suggestions.

[ryrynz]

Well the pro solution would be something similar to this that would be created with the help of people here. The project creator has far less knowledge of Avisynth filters. Wouldn't hurt to flick that across to him for starters. You could also maybe provide a, sample for us with this to evaluate and maybe improve upon?

[Stereodude]

I also had this upscale project in mind for the past three years. (taking long breaks in between :D)
From time to time I experimented a little bit with different scripts and
parameters using the first episode of season 6 (the first 20 minutes).
Here is the script I came so far using the NTSC DVDs:


myvideo = "ep1ntsc.d2v"
DGDecode_mpeg2source(myvideo, idct=5)

#deint = QTGMC().SelectEven() #using QTGMC as deinterlacer optional
deint = tdeint()
tfm(d2v=myvideo,clip2=deint)
TDecimate(hybrid = 1)

QTGMC(InputType=1) #using QTGMC on progressive frames again to clean up a little bit
#QTGMC( Preset="Slower", InputType=1 ) #optional

tweak(cont=1.3,sat=1.1)

ConvertToRGB24()
SuperResXBR(Passes=5, Str=1, Soft=0.0, XbrStr=2.7, XbrSharp=1.3, MatrixIn="PC601")
SuperResXBR(Passes=3, Str=1, Soft=0.0, XbrStr=2.7, XbrSharp=1.3, MatrixIn="PC601", fWidth=1440, fHeight=1080, fKernel="SSim")
ConvertToYV12()

addgrainc()
trim(0,26000)+trim(49114,50000)
AssumeFPS(25)


If someone is interested I can provide a link to the encoded result via pm.
Maybe it´s not a pro solution but for me it was ok.
I´m always open for suggestions.
I'd just assume let MadVR upsample it with NGU during playback vs. SuperXBR in the script. But, getting it competently deinterlaced/IVTC'd as best as it can be would be nice.

What happens after 20 minutes? Why do you need to change parameters?

[Stereodude]

S06E01 is not a good sample IMHO. I just pulled it from the DVD to try bladerunner1982's script with my own twist. There are virtually no combed frames in the first 20 minutes that make it past the IVTC. The only real problem area in those 20 minutes is the wormhole opening at the end of the Intro. And, the script from bladerunner1982 does not handle it well. The default MI of 80 in TFM is way too high. It will pass visibly combed frames in that area. Cranking it down to 5 helps, but still doesn't get 100% of them. Basically you've got to take it to 1 which causes it to see lots small areas as needing deinterlacing in progressive sections, but the impact on the output seems negligible.

After playing around a bit I would suggest something like this:

source=MPEG2Source("VTS_02_1.d2v", idct=5, info=0)

deint=source.QTGMC( Preset="Slower", InputType=0, SourceMatch=3, Lossless=2, Sharpness=0.2, EZKeepGrain=0.0, ShowSettings=false ).SelectEven()

source.tfm(d2v="VTS_02_1.d2v", PP=7, MI=1, clip2=deint, display=false)

TDecimate(hybrid = 1, dupThresh=1.5, vidThresh=2.3, display=false)

MCTemporalDenoise(settings="low", radius=3)

f3kdb(range=16, grainY=15, grainC=10, sample_mode=2, dither_algo=3, dynamic_grain=true, keep_tv_range=false, output_depth=10)
Then encode it with a SAR of 8:9

[SaurusX]

Sounds like the series would require TFM override files for every episode of the series. Any time theres a special effect the regular TFM approach would need to be examined to make sure it’s catching everything. If not, then add the frame sequence to the override file to be deinterlace. Not impossible, but very time consuming.

[bladerunner1982]

Using MadVR requires cpu und gpu power during playback. What about people watching on their SmartTV oder standalone player?
So why not take enough computing power in the upscale process before encoding?
Ending up with a 1440x1080 video file, PAR 1:1

I only encoded 20 minutes because I only have an i5-7500, took me 2 hours to encode.
Tested mainly the parameters of SuperResXBR, trying to find the balance between smoothening and sharpening.

So what episode would be best for testing?

[Stereodude]

Using MadVR requires cpu und gpu power during playback. What about people watching on their SmartTV oder standalone player?
I didn't say my preference was right for everyone.

So what episode would be best for testing?
I'm not sure. I haven't ripped them all. I've only looked at 2: S06E01 and S01E01. It might not be a bad episode for testing, but I didn't see it in the first ~20 minutes. It's only 88.29% film per the D2V. S01E01 is 97% film.

Maybe there aren't that many problem areas in the typical episode when using TFM. Much of the special effects are telecined film content. They're not all interlaced video. I do wish Tdecimate had some different parameters for controlling the blend vs. drop in the hybrid mode.

[scharfis_brain]

btw.: does anyone know wheter there is an NTSC DVD release of DS9/VOY with german audio?

[Chir]

Greetings --

I'm actually tackling this project. I am no expert on this, but I've developed workflow that produces decent results. I came here looking for solutions to my problem where it looks like the frame rate drops -- it gets all jerky in scenes. It's usually heavily CG scenes. You guys helped explain a lot, and I appreciate that. No I understand its the mix of video and film.

I just started to learn about working with video at this level, so please be patient. My current workflow is this:

1. Rip with dvddecryptor to get each episode into a vob file.
2. Open the VOB in DGIndex; enable forced film and save the D2V file.
3. Create an AVISynth script that consumes the D2V file with the following as my template:


LoadPlugin("D:\Video\Tools\DGMPGDec\DGDecode.dll")
SetFilterMTMode("QTGMC",2)
MPeg2Source("D:\DS9S1D1\VIDEO_TS\VTS_02_1 - 0xE0 - Video - MPEG-2 - 720x480 (NTSC) - 4~3.d2v")
ConvertToYV12()
AssumeTFF()
QTGMC(Preset="slower", FPSDivisor=2, Edithreads=4)
Prefetch(threads=6)


4. Pull the video out with ffmpeg:
ffmpeg.exe -i <full path to .avs> -map 0:v -c:v copy <destination.mkv>

6. At this point, I upscale the video with VideoEnhance AI. It spits out PNG files for all the frames in the video file (destination.mkv)

7. I then reassemble the PNGs into Video with FFMPG using the original framerate from the DVD
ffmpeg -r 23.976 -f image2 -s 1920x1080 -i <path to png files>\%06d.PNG -vcodec libx265 -preset medium -crf 28 -pix_fmt yuv420p EncodedOutput.mkv

8. And finally, I mux everything back together with the original audio from the disc, chapters, and subtitles.


Its far from perfect.... and I'm hoping to pick up some tips to streamline my project and get a better result. I'm going to look into the sample scripts above to try and get a better understanding of what they're doing. I have a feeling I'm in over my head and doing unnecessary steps -- but I'm willing (and trying!) to learn!

[SaurusX]

Using QTGMC to deinterlace the image essentially means you’ll be upscaling a 240p picture to 1080p or 4K or whatever. You need to inverse telecine as your main process and deinterlace only the scenes you need to.

[manono]

2. Open the VOB in DGIndex; enable forced film and save the D2V file.

Unless the source is 100% FILM (check the bottom of the D2V file), that is a very bad idea.
ConvertToYV12()
It's already YV12.

[bladerunner1982]

@Stereodude

I encoded your script combined with SuperResXbr (without the output_depth=10 parameter in f3kdb).
Looked good but the space scenes (starships) had motion judder.

@scharfis_brain
I don't think there is an ntsc version disc with german audio. I use the ntsc discs, speed them up after processing to 25fps at the end of my script, then I encode. Speed up the english track to 25fps as well, leave the german as it is.

@Chir
A.I. Gigapixel? You can feed the video directly in it using newest version. No need for frame based encoding.

There is now a folder at mega.nz which contains the testvideos generated by scripts in this thread.
Everyone is invited to have a look at it for comparison. PM me :)

If you want to contribute your testvideo send me a link. Please encode only first 20min of S01E01 or S06E01 without audio to 1440x1080 (I use x264,crf16,preset slow)

[Stereodude]

@Stereodude

...
Looked good but the space scenes (starships) had motion judder.
Can you give me an exact timecode for when you see judder? Pretty much all the special effects scenes appear to be telecined film when I skimmed through it.

[bladerunner1982]

Timecode 01:38 ....right after the look back scene...check PM...

[Stereodude]

Timecode 01:38 ....right after the look back scene...check PM...
It's a bit complicated. If you comment out the Tdecimate line in your script and use this TFM line instead:

source.tfm(d2v="VTS_02_1.d2v", PP=1, MI=1, clip2=deint, display=true)

and then step through the section frame by frame in VD2. There's clearly a duplicated frame that should be removed by tdecimate.

In looking at this, I did discover that I misunderstood the meaning of this section of the TFM help. "...if PP >= 5 (i.e. using motion adaptation) then TFM will build the mask as usual and only pixels in moving areas will be taken from the frames in clip2." It doesn't replace only areas where combing was found it like I was thinking it did. It replaces anywhere that motion was found. In this case because of the motion this changes the output frames significantly (basically completely replaces them).

So, in this section the low MI value set in TFM causes TFM to pull some of the frames from the deinterlaced version which then causes areas of irregular motion in the final output.

Changing the TFM line to:
source.tfm(d2v="VTS_02_1.d2v", PP=7, MI=1, clip2=deint, display=true, flags=5)

fixes the problem in this area. Flags=5 will prevent TFM from looking for combing in any part of the source (except scene changes) that's encoded as 24p with a soft pulldown flag on the DVD. This still leaves areas that are encoded as video.

To address those I would suggest increasing the MI some. Some minor combing will get through, but it will cut down on unnecessary frame replacement with softer deinterlaced frames.

Short of making a full manual override file where you manually and visually inspect every frame with a MIC > 0 you have the trade off of letting minor combing through vs. unnecessary frame replacement. The MI value sets the threshold.

I'm prepping a sample to upload and maybe we'll get some better ideas from others.


Edit: At least for S06E01 bumping up the MI to 24, adding flags=5 to TFM, using an override file for TFM, and an override file for tdecimate seems to basically fix the episode. (Note, I didn't do a frame by frame examination of the whole show.)
TFM Override:
13329,13945 -
16383,16504 +
The first keeps the comet at the start of the "intro" from being seen as combed. The second fixes the combing in the wormhole at the end of the "intro".

Tdecimate Override:
79880,81234 v
This forces the end credits to be blended.

Other episodes will need the same areas fixed with overrides, but of course the frame numbers will be different.

[Stereodude]

Well the pro solution would be something similar to this that would be created with the help of people here. The project creator has far less knowledge of Avisynth filters. Wouldn't hurt to flick that across to him for starters. You could also maybe provide a, sample for us with this to evaluate and maybe improve upon?
Here's a sample. It's starts with a section that's video and then it's a section that alternates between film/video as the titles appear on screen.

https://mega.nz/file/1g9DBSpC

and what you'll need to get it:
64cZaOQ8mHnbvDRTthMf6M0RenYwdb3M0A7I_ufaQqc

[mbcd]

My try, looks good, but not perfect ... (hope you can download it):

https://1drv.ms/u/s!AuYZhY0Moi2f5hxSoo6pSTQlKl2u

LWLibavVideoSource("VTS_02_1_sample.m2v")
TFM(mode=1)
TDecimate(mode=0,hybrid=1)
AssumeFps(23.976)

There are still some blended frames left, so movement is not that smooth, but that scenes look almost ""fine"" ...

[mbcd]

My try, looks good, but not perfect, no magic done ... (hope you can download it):

https://1drv.ms/u/s!AuYZhY0Moi2f5hxSoo6pSTQlKl2u



There are still some blended frames left, so movement is not that smooth, but that scenes look almost ""fine"" ...

Look like field oder is changing, I get mostly duplicated bad frames on those ...

I dont think you will get a good result, they layed two different framerate together, title changes, but background is still (doubled frames) ...

[bladerunner1982]

@mbcd
Thanks for participation. That's a rather basic approach. But where's the upscaling part? :)

@Stereodude
I believe to think that I can see your point. I changed MI to 24 but used PP=6. Used override files for comet and wormhole.
After some experimenting I came up to this:


LoadPlugin("d:\gordianknot\DGMPGDec\DGDecode.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\plugins\LSMASHSource.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\Plugins\nnedi3.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\Plugins\TIVTC.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\Plugins\vinverse.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\Plugins\Shader.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth\Plugins\AddGrainC.dll")

SetFilterMTMode("DEFAULT_MT_MODE", 2)

myvideo = "ep1ntsc.d2v"

DGDecode_mpeg2source(myvideo, idct=5)

## choose deinterlacer ##
#deint = QTGMC( Preset="Slower", InputType=0, SourceMatch=3, Lossless=2, Sharpness=0.2, EZKeepGrain=0.0, ShowSettings=false ).SelectEven()
deint = tdeint()

## IVTC hybrid ntsc with override filles ##
tfm(d2v=myvideo, PP=6, MI=24, clip2=deint, flags=5,ovr="s06e01_tfm_ovr.txt")
TDecimate(hybrid = 1,ovr="s06e01_tdecimate_ovr.txt")

## clean up ##
#MCTemporalDenoise(settings="low", radius=3)
#f3kdb(range=16, grainY=15, grainC=10, sample_mode=2, dither_algo=3, dynamic_grain=true, keep_tv_range=false)
#or
#QTGMC(InputType=1) #using QTGMC on progressive frames again to clean up a little bit
#or
QTGMC( Preset="Slower", InputType=1 )

## color tweak ##
tweak(cont=1.3,sat=1.1)

## Upscale ##
ConvertToRGB24()
SuperResXBR(Passes=5, Str=1, Soft=0.0, XbrStr=2.7, XbrSharp=1.3, MatrixIn="PC601")
SuperResXBR(Passes=3, Str=1, Soft=0.0, XbrStr=2.7, XbrSharp=1.3, MatrixIn="PC601", fWidth=1440, fHeight=1080, fKernel="SSim")
ConvertToYV12()

## grain only use if you didn't use f3kdb
addgrainc()


trim(0,15000)

## PAL Speedup for Europe
AssumeFPS(25)



Result is version 3 in this folder https://mega.nz/folder/A10lULqT

[Stereodude]

@Stereodude
I believe to think that I can see your point. I changed MI to 24 but used PP=6. Used override files for comet and wormhole...
FWIW, I don't think there is any difference between 6 or 7 if you're using clip2.

I'm also on the fence about using mild deblocking in the MPEG2 source filter on the luma channel. For example, the comet in the show intro has significant blocking when decoded with no deblocking.

[Stereodude]

Result is version 3 in this folder https://mega.nz/folder/A10lULqT
You made some post processing decisions that I personally wouldn't have made, but that doesn't mean they're wrong. To me it mostly looks like there is a static grain pattern sitting on top of the image much of time that I don't care for.

However, I don't think the MPEG-2 source could be handled much better getting it to 480p24 in a basically automated way. FWIW, you could crop about 4 pixels off the right side at the source filter.

DGDecode_mpeg2source(myvideo, idct=5).crop(0,0,-4,-0)

[bladerunner1982]

TFM options:
"if PP >= 5 (i.e. using motion adaptation) then TFM will build the mask as usual and only pixels in moving areas will be taken from the frames in clip2."

PP6 = motion-adaptive cubic interpolation deinterlacing... gives to me a sharper picture than PP7 (maybe it´s just my imagination)

And therefore I tend to QTGMC for post processing...MCTemporalDenoise and f3kdb seem to clean to much so there is less detail left for upscaling.

I compared both solutions eye by eye...but tastes differ...

Anyone with a suggestion for a better upscaler?

[Stereodude]

TFM options:
"if PP >= 5 (i.e. using motion adaptation) then TFM will build the mask as usual and only pixels in moving areas will be taken from the frames in clip2."

PP6 = motion-adaptive cubic interpolation deinterlacing... gives to me a sharper picture than PP7 (maybe it´s just my imagination)
It's your imagination. The deinterlacing routine isn't used. clip2 is used. The same detection algorithms are used for either.

Anyone with a suggestion for a better upscaler?
You could try waifu2x

https://forum.videohelp.com/threads/394851-Script-to-scale-videos-with-waifu2x-(AviSynth)

(I haven't tried this)

[Chir]

So the problem that we have is that some frames need to be reverse telecined and some need to be deinterlaced, but not both. Would that be a correct statement of the problem?

To deal with this, we'd need to create an OVR file (two actually), which if I understand it right, allows us to choose which process to apply to the frame.

If that is a correct statement -- are there any tools to help create the OVR files? Or is this something that you have to step through frame by frame to determine the type and manually create the OVR file?

[Stereodude]

So the problem that we have is that some frames need to be reverse telecined and some need to be deinterlaced, but not both. Would that be a correct statement of the problem?

To deal with this, we'd need to create an OVR file (two actually), which if I understand it right, allows us to choose which process to apply to the frame.

If that is a correct statement -- are there any tools to help create the OVR files? Or is this something that you have to step through frame by frame to determine the type and manually create the OVR file?
bladerunner1982 and myself have basically hammered out a pretty decent template/theory of handling the episodes.

You want to IVTC all the parts of the original DVD that are encoded as film with a soft pulldown. You still want to IVTC as much of the rest of the video portions as possible. You only want to deinterlace any frames that have residual combing or are interlaced video.

TFM checks for residual combing in a frame and comes up with a number based on how much combing it measures. You set the threshold for how high number that needs to be before it will deinterlace the frame.

You only need an override file when that threshold doesn't work for a section. In S06E01 I found 3 problem areas. 1) At the start of the show "intro", the comet generates a high MIC number despite not being combed. 2) The wormhole opening at the end of the "intro" has combing but doesn't generate a high MIC number. 3) The credits appear to be pure interlaced video.

So, you make an override file with the frame ranges and what it should do for those ranges.13329,13945 -
16383,16504 +
79880,81234 p
I expect these issues will happen in every episode. Of course the frame numbers will vary for each episode.

The second issue is that tdecimate doesn't always make the correct decision on whether to drop a frame restoring 24p or to blend the 30p video section to 24p. The most flagrant area it got wrong was the credits, so I made an override to force them to be treated as video (blended).79880,81234 vI expect this issue will also happen in every episode and the frame numbers will need to be adjusted. I'm also undecided if the wormhole opening at the end of the "intro" should be blended or treated as film, but it's something to consider.

So you want to do something like this:
source=MPEG2Source("VTS_02_1.d2v", idct=5, moderate_h=40, moderate_v=40, cpu2="xxoooo").crop(0,0,-4,-0)

deint=source.QTGMC( Preset="Slower", InputType=0, SourceMatch=3, Lossless=2, Sharpness=0.2, EZKeepGrain=0.0).SelectEven()

#for testing
#source.tfm(d2v="VTS_02_1.d2v", PP=1, MI=24, clip2=deint, display=true, flags=5)
source.tfm(d2v="VTS_02_1.d2v", PP=7, MI=24, clip2=deint, display=false, flags=5, ovr="D:\DS9S6D1\E01\tfm_ovr.txt")

#for testing
#TDecimate(hybrid = 1, dupThresh=1.5, vidThresh=2.0, display=true)
TDecimate(hybrid = 1, dupThresh=1.5, vidThresh=2.0, display=false, ovr="D:\DS9S6D1\E01\td_ovr.txt")


You can visually inspect what TFM is doing and the see the metrics for each frame by uncommenting the first TFM line and commenting out the 2nd. You can to the same for tdecimate by uncommenting the first tdecimate line and commenting out the 2nd. You check TFM and tdecimate separately.

I do not think you need to go frame by frame. I would think a quick skim of the video sections of the episode will be sufficient. The only portions that are video are fades to and from black, the intro, where text is overlaid on the screen, and the credits. The intro and credits should have the same behavior from episode to episode, just update the frame numbers. Fades in and out of black I would just leave be, which really means the only parts you have to check are the text overlays for a few minutes after the intro where they give the guest actors, director, etc.

[SaurusX]

The interlaced fade-ins and fade outs can be handled with the smartfade() routine right after the IVTC. I think I pulled it out of the AnimeIVTC script. Might as well get rid of those if you can.

#based on smartfade v0.2 by martino - Aimed at removing interlaced fades in anime. Uses luma difference between two fields as activation threshold.
#mod by thetoof : removed degrainmedian post-processing,
# changed how the fields are blended together (average of 2 nnedi interpolations + contra-sharpening to retain more detail) and
# added cache parameter to forward the setting of animeivtc to this function
# renamed global parameter to something less general than "threshold"

function smartfademod(clip c, float "sfthr", bool "sfshow", int "cache") {

global sfthr = default ( sfthr, 0.4 ) # Threshold for fade detection.
show = default ( sfshow, false ) # Displays luma difference between fields without processing anything.
cache = default ( cache, 10 ) # Cache the frames to help seeking and other functions requesting many frames

show ? scriptclip(c, "subtitle(string(abs( separatefields().selectodd().averageluma() \
- separatefields().selecteven().averageluma())))") : \
scriptclip(c, "sep = separatefields() \
avg = sep.selectodd().averageluma() - sep.selecteven().averageluma() \
abs(avg) > sfthr ? interleave(sep.selecteven(),sep.selectodd()).weave().mcdaa3() : last")

cache<0 ? last : last.RequestLinear(8, cache, 5, false, false)

return last

}

[Chir]

I'm trying to read up and understand so I don't ask too many questions, but I'm a little lost.

I understand what we're trying to do, but I'm struggling to put it all together. I grabbed Season 6, Episode 1 so I could use your examples to learn, but it's not clicking for me.

I started with DVD Decryptor and pulled the epsiode off into a VOB file. I opened the VOB file with DGINDEX and set it to Honor Pulldown flags, then saved the D2V file. I fired up avspmod and pasted the script in that you provided.

I commented things out to do "testing":

source=MPEG2Source("VTS_02_1.d2v", idct=5, moderate_h=40, moderate_v=40, cpu2="xxoooo").crop(0,0,-4,-0)

deint=source.QTGMC( Preset="Slower", InputType=0, SourceMatch=3, Lossless=2, Sharpness=0.2, EZKeepGrain=0.0).SelectEven()

#for testing
source.tfm(d2v="VTS_02_1.d2v", PP=1, MI=24, clip2=deint, display=true, flags=5)
#source.tfm(d2v="VTS_02_1.d2v", PP=7, MI=24, clip2=deint, display=false, flags=5, ovr="D:\DS9S6D1\E01\tfm_ovr.txt")

#for testing
TDecimate(hybrid = 1, dupThresh=1.5, vidThresh=2.0, display=true)
#TDecimate(hybrid = 1, dupThresh=1.5, vidThresh=2.0, display=false, ovr="D:\DS9S6D1\E01\td_ovr.txt")


When I display the first frame, I the frame, the type of match, and (D2V). I believe this mean that no filters are applied and its just displaying the D2V -- is that correct?

When I get to frame 832 (Kira standing at a control panel, but the start of a Scene change to a CG scene) The (D2V) tag goesaway and it has a MIC=9. At 838, it switches to a Full CG Scnene and goes from Clean Frame to Combed Frame. This continues for frame 1017. I'm assuming this is a frame that TFM detected that it needs to process -- is that correct?


As I progress into the video, I see a few mic values pop up -- but only for a frame or two. I think this is ok, as it appears that a changing scene or lots of movement causes this. However, at 7102 (black screen) the (D2V) indicator goes away again -- is this TFM detecing a frame to process again? It still says CLEAN FRAME with a rather low mic value until frame 7352 (as the comet leaves the screen to the right). You mentioned this part of the video in your comments above, and its where I see problems with my previous attempts -- there is a HUGE jump right here). I'm expecting TFM is picking this up, so I don't need to do anything in the overrride file... is that correct>

You provided a sample ovr for TFM. If I jump to that frame, I'm expecting to see something in the stats change, or something on the screen, that idicates what I'd be looking for to include in the ovr file. At 13329, I see Odo talking to Quark. This appears just after the opening credits are finished (after the cg intro). I don't see anything -- either in the image itself or the TFM debug output that would have clued me in to add this to my ovr file. Any hints at what I should be looking for in these scenes to add them to the override?

I have a hunch that I'm not working with exactly the same media and your examples might be a little off from mine. avspmod claims that I have 81773 frames before decimating.

[Chir]

oh, my. I feel like a complete idiot. I had my D2V file pointing to the wrong VOB, so I was actually looking at a different episode...

Time for me to take a break!

[Stereodude]

When I display the first frame, I the frame, the type of match, and (D2V). I believe this mean that no filters are applied and its just displaying the D2V -- is that correct?
The D2V means TFM is applying an IVTC and assuming the output is progressive (not looking for combing) per the information/flagging in the D2V file. flags=5 forces this behavior (which is desirable here).

When I get to frame 832 (Kira standing at a control panel, but the start of a Scene change to a CG scene) The (D2V) tag goesaway and it has a MIC=9. At 838, it switches to a Full CG Scnene and goes from Clean Frame to Combed Frame. This continues for frame 1017. I'm assuming this is a frame that TFM detected that it needs to process -- is that correct?
When D2V disappears you're in a video section of episode and TFM will manually look for a pattern to IVTC. It will also look for combing (possible interlacing) after applying the IVTC. The MIC number is a measure of how much combing it finds.

[bladerunner1982]

@Chir
Don't worry, I made that error myself...Frame 13329 corresponds to the first VOB file of S06E01 (VTS_02_1.VOB) without any TFM or Tdecimate applied..where the comet is starting. Maybe we should set that file as a reference point